Shader reflection API issues

Hi,
I don't know if it's the right place to ask questions but I will post it here and if it is not appropriate, I can move the discussion elsewhere.
I am using slang reflection API to feed my engine but I have some troubles.
For some UTs, I have this shader in slang:

struct Stuff
{
    int a;
    float b;
};

struct Things
{
    Texture2D t;
    ParameterBlock<Stuff> stuff;
};

ParameterBlock<Stuff> gStuff;
Texture2D gTex;
ConstantBuffer<Things> gThings;
SamplerState s;

RWStructuredBuffer<float> result;

[shader("compute")]
[numthreads(16, 16, 1)]
void main(uint3 threadId: SV_DispatchThreadID)
{
    result[0] = (float)gStuff.a + gStuff.b + gTex.SampleLevel(s, float2(0, 0), 0).x + (float)gThings.stuff.a+ gThings.stuff.b + gThings.t.SampleLevel(s, float2(0, 0), 0).x;
}

This is just for unit testing so don't look at the stuff in main.
That will compil into this : (with latest branch 2024.1.18
#version 450
layout(row_major) uniform;
layout(row_major) buffer;

#line 18 0
layout(std430, binding = 3) buffer StructuredBuffer_float_t_0 {
    float _data[];
} result_0;

#line 4
struct Stuff_std140_0
{
    int a_0;
    float b_0;
};

layout(binding = 0, set = 1)
layout(std140) uniform _S1
{
    int a_0;
    float b_0;
}gStuff_0;

#line 14
layout(binding = 0)
uniform texture2D gTex_0;

#line 16
layout(binding = 2)
uniform sampler s_0;

#line 16
layout(binding = 1)
uniform texture2D gThings_t_0;

#line 16
layout(binding = 0, set = 2)
layout(std140) uniform _S2
{
    int a_0;
    float b_0;
}gThings_stuff_0;
layout(local_size_x = 16, local_size_y = 16, local_size_z = 1) in;
void main()
{

#line 24
    const vec2 _S3 = vec2(0.0, 0.0);

#line 24
    result_0._data[0U] = float(gStuff_0.a_0) + gStuff_0.b_0 + (textureLod(sampler2D(gTex_0,s_0), (_S3), (0.0))).x + float(gThings_stuff_0.a_0) + gThings_stuff_0.b_0 + (textureLod(sampler2D(gThings_t_0,s_0), (_S3), (0.0))).x;
    return;
}

What I am trying to achieve is to get the correct binding and set for Vulkan for each variable. This shader is basded on an old comment I found in your repo when you were still using childSet in SimpleBindingOffset.
I am using this to compute the relative binding and set of variable

        /// Create an offset based on offset information in the given Slang `varLayout`
        SimpleBindingOffset(slang::VariableLayoutReflection* varLayout)
        {
            if (varLayout)
            {
                bindingSet = varLayout->getBindingSpace() + (Uint32)varLayout->getOffset(SLANG_PARAMETER_CATEGORY_SUB_ELEMENT_REGISTER_SPACE);
                binding = (Uint32)(varLayout->getOffset(SLANG_PARAMETER_CATEGORY_DESCRIPTOR_TABLE_SLOT));
                pushConstantRange =
                    (uint32_t)varLayout->getOffset(SLANG_PARAMETER_CATEGORY_PUSH_CONSTANT_BUFFER);
            }
        }

And I am adding recursively the offset with the parent. THe thing is that I get binding = 1 and set = 2 for gThings. But the gThings.t is in set 0 as the GLSL is showing (and binding 1). How am I supposed to get the right result ?
I read the documentation but I still have trouble understanding everything.
You can see part of my code here! I modified it a bit since last time but nothing fancy. https://github.com/shader-slang/slang/issues/3318
Thanks,
Baptiste

I think you should be fine by just summing up all getOffset(SLANG_PARAMETER_CATEGORY_SUB_ELEMENT_REGISTER_SPACE) from the var layouts on the path the to variable. getBindingSpace() shouldn't be needed if you are summing up things recursively in this manner.

Ok I deleted the getBindingSpace() from the code so now,
 if (varLayout)
 {
     bindingSet = (Uint32)varLayout->getOffset(SLANG_PARAMETER_CATEGORY_SUB_ELEMENT_REGISTER_SPACE);
     binding = (Uint32)(varLayout->getOffset(SLANG_PARAMETER_CATEGORY_DESCRIPTOR_TABLE_SLOT));
     pushConstantRange =
         (uint32_t)varLayout->getOffset(SLANG_PARAMETER_CATEGORY_PUSH_CONSTANT_BUFFER);
 }

I've got this code for each layout and summing up for the children recusively. But it doesn't help with the bug.
gThings is still in set 2 and binding 1 but in the real code gThings_t_0 is in set 0 binding 1 and gThings_stuff_0 is in set 2 binding 0.
I saw something with the category, maybe it comesfrom here ? For gThings the slang::ParameterCategory is Mixed and the category count is 2. Should I do moething from here ?
Thanks,
Baptiste

No idea ?

Ok thanks ! I'll try it.

And btw removing the getBindingSpace() makes some other shader reflection issues on my shaders. So for now, I'll let it until I figured out what's wrong. Thank you for your time :)

Sorry, I don't see an exe called slang-reflection-test. I see the slang-test.exe sio I tried something like this ./slang-test slang-reflection-test (I rapidly read the code, it seems this is the way to do ?) and it seems the output should be in the console but there is nothing as output. Am I doing something wrong ?
Thanks,
Baptiste

You are right, I apologize for my mistake. Currently slang-reflection-test is not compiled as a standalone exe but as a dll that is called by slang-test.exe.
You can run reflection test by compiling slang-reflection-test to exe, or just use slang-test.exe using the following steps:

Add the following comment line to your shader:

//TEST:REFLECTION:-target spirv

Note that there should be no space between // and TEST.
2. Put your file under <slang-repo-dir>/tests
3. Run slang-test from <slang-repo-dir> with argument: tests/your-shader-file.

This is the output !
result code = 0
standard error = {
}
standard output = {
{
    "parameters": [
        {
            "name": "gStuff",
            "binding": {"kind": "subElementRegisterSpace", "index": 1},
            "type": {
                "kind": "parameterBlock",
                "elementType": {
                    "kind": "struct",
                    "name": "Stuff",
                    "fields": [
                        {
                            "name": "a",
                            "type": {
                                "kind": "scalar",
                                "scalarType": "int32"
                            },
                            "binding": {"kind": "uniform", "offset": 0, "size": 4}
                        },
                        {
                            "name": "b",
                            "type": {
                                "kind": "scalar",
                                "scalarType": "float32"
                            },
                            "binding": {"kind": "uniform", "offset": 4, "size": 4}
                        }
                    ]
                },
                "containerVarLayout": {
                    "bindings": [
                        {"kind": "descriptorTableSlot", "index": 0},
                        {"kind": "subElementRegisterSpace", "index": 0}
                    ]
                },
                "elementVarLayout": {
                    "type": {
                        "kind": "struct",
                        "name": "Stuff",
                        "fields": [
                            {
                                "name": "a",
                                "type": {
                                    "kind": "scalar",
                                    "scalarType": "int32"
                                },
                                "binding": {"kind": "uniform", "offset": 0, "size": 4}
                            },
                            {
                                "name": "b",
                                "type": {
                                    "kind": "scalar",
                                    "scalarType": "float32"
                                },
                                "binding": {"kind": "uniform", "offset": 4, "size": 4}
                            }
                        ]
                    },
                    "binding": {"kind": "uniform", "offset": 0, "size": 16}
                }
            }
        },
        {
            "name": "gTex",
            "binding": {"kind": "descriptorTableSlot", "index": 0},
            "type": {
                "kind": "resource",
                "baseShape": "texture2D"
            }
        },
        {
            "name": "gThings",
            "bindings": [
                {"kind": "descriptorTableSlot", "index": 1},
                {"kind": "subElementRegisterSpace", "index": 2}
            ],
            "type": {
                "kind": "constantBuffer",
                "elementType": {
                    "kind": "struct",
                    "name": "Things",
                    "fields": [
                        {
                            "name": "t",
                            "type": {
                                "kind": "resource",
                                "baseShape": "texture2D"
                            },
                            "binding": {"kind": "descriptorTableSlot", "index": 0}
                        },
                        {
                            "name": "stuff",
                            "type": {
                                "kind": "parameterBlock",
                                "elementType": {
                                    "kind": "struct",
                                    "name": "Stuff",
                                    "fields": [
                                        {
                                            "name": "a",
                                            "type": {
                                                "kind": "scalar",
                                                "scalarType": "int32"
                                            },
                                            "binding": {"kind": "uniform", "offset": 0, "size": 4}
                                        },
                                        {
                                            "name": "b",
                                            "type": {
                                                "kind": "scalar",
                                                "scalarType": "float32"
                                            },
                                            "binding": {"kind": "uniform", "offset": 4, "size": 4}
                                        }
                                    ]
                                },
                                "containerVarLayout": {
                                    "bindings": [
                                        {"kind": "descriptorTableSlot", "index": 0},
                                        {"kind": "subElementRegisterSpace", "index": 0}
                                    ]
                                },
                                "elementVarLayout": {
                                    "type": {
                                        "kind": "struct",
                                        "name": "Stuff",
                                        "fields": [
                                            {
                                                "name": "a",
                                                "type": {
                                                    "kind": "scalar",
                                                    "scalarType": "int32"
                                                },
                                                "binding": {"kind": "uniform", "offset": 0, "size": 4}
                                            },
                                            {
                                                "name": "b",
                                                "type": {
                                                    "kind": "scalar",
                                                    "scalarType": "float32"
                                                },
                                                "binding": {"kind": "uniform", "offset": 4, "size": 4}
                                            }
                                        ]
                                    },
                                    "binding": {"kind": "uniform", "offset": 0, "size": 16}
                                }
                            },
                            "binding": {"kind": "subElementRegisterSpace", "index": 0}
                        }
                    ]
                },
                "containerVarLayout": {

                },
                "elementVarLayout": {
                    "type": {
                        "kind": "struct",
                        "name": "Things",
                        "fields": [
                            {
                                "name": "t",
                                "type": {
                                    "kind": "resource",
                                    "baseShape": "texture2D"
                                },
                                "binding": {"kind": "descriptorTableSlot", "index": 0}
                            },
                            {
                                "name": "stuff",
                                "type": {
                                    "kind": "parameterBlock",
                                    "elementType": {
                                        "kind": "struct",
                                        "name": "Stuff",
                                        "fields": [
                                            {
                                                "name": "a",
                                                "type": {
                                                    "kind": "scalar",
                                                    "scalarType": "int32"
                                                },
                                                "binding": {"kind": "uniform", "offset": 0, "size": 4}
                                            },
                                            {
                                                "name": "b",
                                                "type": {
                                                    "kind": "scalar",
                                                    "scalarType": "float32"
                                                },
                                                "binding": {"kind": "uniform", "offset": 4, "size": 4}
                                            }
                                        ]
                                    },
                                    "containerVarLayout": {
                                        "bindings": [
                                            {"kind": "descriptorTableSlot", "index": 0},
                                            {"kind": "subElementRegisterSpace", "index": 0}
                                        ]
                                    },
                                    "elementVarLayout": {
                                        "type": {
                                            "kind": "struct",
                                            "name": "Stuff",
                                            "fields": [
                                                {
                                                    "name": "a",
                                                    "type": {
                                                        "kind": "scalar",
                                                        "scalarType": "int32"
                                                    },
                                                    "binding": {"kind": "uniform", "offset": 0, "size": 4}
                                                },
                                                {
                                                    "name": "b",
                                                    "type": {
                                                        "kind": "scalar",
                                                        "scalarType": "float32"
                                                    },
                                                    "binding": {"kind": "uniform", "offset": 4, "size": 4}
                                                }
                                            ]
                                        },
                                        "binding": {"kind": "uniform", "offset": 0, "size": 16}
                                    }
                                },
                                "binding": {"kind": "subElementRegisterSpace", "index": 0}
                            }
                        ]
                    },
                    "bindings": [
                        {"kind": "descriptorTableSlot", "index": 0},
                        {"kind": "subElementRegisterSpace", "index": 0}
                    ]
                }
            }
        },
        {
            "name": "s",
            "binding": {"kind": "descriptorTableSlot", "index": 2},
            "type": {
                "kind": "samplerState"
            }
        },
        {
            "name": "result",
            "binding": {"kind": "descriptorTableSlot", "index": 3},
            "type": {
                "kind": "resource",
                "baseShape": "structuredBuffer",
                "access": "readWrite",
                "resultType": {
                    "kind": "scalar",
                    "scalarType": "float32"
                }
            }
        }
    ],
    "entryPoints": [
        {
            "name": "main",
            "stage:": "compute",
            "parameters": [
                {
                    "name": "threadId",
                    "semanticName": "SV_DISPATCHTHREADID",
                    "type": {
                        "kind": "vector",
                        "elementCount": 3,
                        "elementType": {
                            "kind": "scalar",
                            "scalarType": "uint32"
                        }
                    }
                }
            ],
            "threadGroupSize": [16, 16, 1],
            "bindings": [
                {
                    "name": "gStuff",
                    "binding": {"kind": "subElementRegisterSpace", "index": 1}
                },
                {
                    "name": "gTex",
                    "binding": {"kind": "descriptorTableSlot", "index": 0}
                },
                {
                    "name": "gThings",
                    "bindings": [
                        {"kind": "descriptorTableSlot", "index": 1},
                        {"kind": "subElementRegisterSpace", "index": 2}
                    ]
                },
                {
                    "name": "s",
                    "binding": {"kind": "descriptorTableSlot", "index": 2}
                },
                {
                    "name": "result",
                    "binding": {"kind": "descriptorTableSlot", "index": 3}
                }
            ]
        }
    ]
}
}

So if I understand correctly gThings has a parent binding to index 1 and set 2 but gTHings.t has a descriptorTableSlot to 0 as well as gThings.gStuff.
I know it's a weird case but I'm just creating UT and this one comes from one of your old repo.

Looking at the reflection data, I think it is correct because SUB_ELEMENT_REGISTER_SPACE for a variable stores the starting set index for the fields of the variable whose type is a ParameterBlock, that means gThings's sub-element register space stores the set to use for gThings::stuff.
SLANG_PARAMETER_CATEGORY_SUB_ELEMENT_REGISTER_SPACE should be ignored for anything not inside a parameter block.

I'm going to close this for now, assuming there's no further assistance needed, but please feel free to open a new issue or reopen this one if there's something we can address.


NLTK and pattern.en incorrectly classifying some proper nouns

I am using nltk and pattern.en to analyze some text. In a test run, I used the text from Little Red Riding Hood. I noticed that pattern.en did recognize 'Little', 'Red', and 'Hood' as proper nouns (probably due to capitalization?), but 'Riding' was seen as a verb and not part of the proper noun 'Little Red Riding Hood'.
For example, the sentence in the story:
text = "Little Red Riding Hood, was so long until the large, large trough was quite full."
was tagged as
[('"', '"'), ('Little', 'NNP'), ('Red', 'NNP'), ('Riding', 'VBG'), ('Hood', 'NNP'), (',', ','), ('was', 'VBD'), ('so', 'RB'), ('long', 'RB'), ('until', 'IN'), ('the', 'DT'), ('large', 'JJ'), (',', ','), ('large', 'JJ'), ('trough', 'NN'), ('was', 'VBD'), ('quite', 'RB'), ('full', 'JJ'), ('.', '.')]
My code:
import nltk
from pattern.en import tag
tagged_text = tag(text)

Is this just a problem with the pattern.en, or am I assuming nltk should do something it doesn't? If the latter, how do I improve on the rendering of proper nouns so 'Riding' is tagged in this instance as a NNP and not a VBG so it is not converted to 'rid'?
A final question for the linguists in the crowd. Why is the lemma of 'riding' 'rid', and not 'ride'?
Thanks!
Mark

You can use the following code, it does works.
from nltk.tokenize import word_tokenize
text = "Little Red Riding Hood, was so long until the large, large trough was quite full."
nltk.pos_tag(word_tokenize(text))
[('Little', 'JJ'), ('Red', 'NNP'), ('Riding', 'NNP'), ('Hood', 'NNP'), (',', ','), ('was', 'VBD'), ('so', 'RB'), ('long', 'RB'), ('until', 'IN'), ('the', 'DT'), ('large', 'JJ'), (',', ','), ('large', 'JJ'), ('trough', 'NN'), ('was', 'VBD'), ('quite', 'RB'), ('full', 'JJ'), ('.', '.')]

This code:
import nltk
from pattern.en import tag
tagged_text = tag(text)

only uses pattern's tagging.
Mohaned's answer is right. I'll close this as resolved accordingly.
Also, the lemma of 'riding' ought to be 'ride', but no lemmatization algorithm is flawless, and seemingly it failed here.


allow rendering objects in the settings UI

I'm refactoring the audioCue and alert settings to be in a new object together. A downside is that now users have to edit the settings in the settings.json file, which can be a challenge for blind people. It would be great if these could be rendered in the settings UI.

If those are the only allowed keys, does adding "additionalProperties": false work?

that worked, thanks!


FindMembers not working so NavigateDown and NavigateUp don't work

I'm sorry to bother you Jason but I've troubleshooted most of the issues that I've had but I've run into this one and I can't figure it out;
Error detected while processing function OmniSharp#NavigateDown:
line    1:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/josephferano/.vim/bundle/Omnisharp/python/OmniSharp.py", line 57, in findMembers
    return quickfixes_from_response(json.loads(js));
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 365, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 383, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
E858: Eval did not return a valid python object
line    5:
E121: Undefined variable: lnum
E15: Invalid expression: qf_taglist[l].lnum
line    6:
E121: Undefined variable: col
E15: Invalid expression: qf_taglist[l].col
line    7:
E121: Undefined variable: qf_line
E15: Invalid expression: qf_line > line || (qf_line == line && qf_col > column)

I'm trying to use the NavigateUp and NavigateDown functions but they don't work and I get that error. However, it's only in one of the projects, the same one I was having issues with the Highlight Types. I changed the timeout amount to 100 and that fixed the one issue, but I can't figure this one out. FindMember doesn't work either. I'll take a better look at it tomorrow but I just wanted to let you know in case it was something stupid.
OmniSharpFindMembers shows this;
Error detected while processing function OmniSharp#FindMembers:
line    1:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/josephferano/.vim/bundle/Omnisharp/python/OmniSharp.py", line 57, in findMembers
    return quickfixes_from_response(json.loads(js));
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 365, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 383, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
E858: Eval did not return a valid python object

So pretty much the same thing so it's probably because of this function that its failing.
Thanks!

Closing old issue. The hastebin link (unsurprisingly) no longer works so there's not much to debug here. Please re-open if this is still a problem.


Filter accel/gyro for q estimator

This is an attempt to fix #5060.
It pre-filters accel and gyro data before it's used in the attitude_estimator_q.
@bkueng please have a look through, thanks.

yep, looks good

Rebased and applied, thanks!


fix: Fixed Docus runnable locally, outside of docs/ repo

Closes # .
Checklist:

[ ] issue number linked above after pound (#)

replace "Closes " with "Contributes to" or other if this PR does not close the issue

[ ] manually checked my feature / checking not applicable
[ ] wrote tests / testing not applicable
[ ] attached screenshots / screenshot not applicable

works for me:


Merge to master

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.2 out of 4 committers have signed the CLA.:white_check_mark: stuartcarnie:white_check_mark: r1sc:x: rogeralsing:x: gitter-badger


Fixing the copying method for the init_weights

model.load_state_dict(init_weights) on its second iteration will make it constant leading to no change in its weights due to training. On changing it to model.model.load_state_dict(init_weights), we are directly assigning the model with its initial weights. More on https://huggingface.co/transformers/main_classes/trainer.html
Summary:
Simple fix
Features:
Checklist:

[ ] Your code is documented (To validate this, add your module to tests/documentation_test.py).
[ ] Your code is tested with unit tests.
[ ] You moved your Issue to the PR state.

@catchmoosa I was wondering if you have tested the changes afterwards and saw a difference, since we provide this functionality (https://github.com/ElementAI/baal/blob/master/src/baal/transformers_trainer_wrapper.py#L112) and the change you made is equivalent to this function. Let me know :)

Hi! Makes sense. It looks the same. But for some reason on training, the model weights on my end wouldn't update unless I made the change. This was observed the second time the "load_state_dict" function is called in the iterator. I'll check it out and get back if I find anything. Thanks :)

@catchmoosa we added a test in the new release that shows that model weights should be updated when you use the "load_state_dict" here. So I am going to close this PR. Thanks a lot for opening this :)


[SMALLFIX] Reduce RPC retry max duration for shell CLIs

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/20113/
Test FAILed.

@calvinjia user can do this:
$ bin/alluxio fs ls / -Dalluxio.user.rpc.retry.max.duration=1s

which will overwrite the '5s' value in script

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/20133/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/20144/
Test PASSed.

@aaudiber  PTAL


Styled Components: Global Styles (injectGlobal) In Wrong Order

When I use injectGlobal to inject global styles using styled-components in e.g. the _documents.js the resulting <style> tag for global styles will be generated below the component's <style> tag. Therefore global styles will override component styles. This should be turned around since global styles like e.g. normalize.css shouldn't override component styles.

[x] I have searched the issues of this repository and believe that this is not a duplicate.

Expected Behavior

The <style> tag for global styles should be generated before the <style> tag of component styles
Current Behavior

It's generated in the wrong order.
Steps to Reproduce (for bugs)
Download this zip: example.zip
This example is based on the example with styled-components.
Run $ npm install and open localhost:3000, then view the HTML source code.
This proofes that non-local CSS isn't generated at the top:

Context

I'm trying to add normalize.css at the very top of the HTML file.
Your Environment

Tech
Version

next
4.2.3

node
6.11.3

OS
Win 10

browser
FF 58

etc

This issue is related to this : https://github.com/styled-components/styled-components/issues/1109

Thanks @lucleray sounds like an upstream issue.

Hey, I'm facing with this issue. Tried to add global.js to a top of _app.js or _document.js and still getting wrong order.

I have the same problem in my project, @bartdominiak  Did you find any solution ?

@alanhr Yes. Just add global.js on top of your index.js

@bartdominiak  add  the global.js in all of pages ?

nope, I added it only to index.js, and its visible in other pages.

Thanks @bartdominiak , I add in my Layout component  and now everything is ok

@bartdominiak Just stumbled on this and I was wondering if you could clarify what exactly your global.js looks like/what it is? I'm also trying to have styled-components bring in normalize.css. Thanks a ton.

@clayhan did you manage to figure it out? I'm having the same issue...

@bartdominiak - could you share your global styles component?


Spacing does not work

Regardless of the SPACING and MERGELINES settings, vector edges are aligned/adjacent to one another without the SPACING, which logically would allow for a cnc router bit's cutting diameter.

Have to admit I'm a laser cutter... played around with the spacing to add to the tooltip explanations but haven't paid it much attention.
That said I do know the MergeLine logic I put in would have 0 chance of working with a CNC kerf but spacing... could you attach your Project, preferably with loadable Parts; loads of them in the DxfFiles folde in the repo just to demonstrate what your setup is, what gets generated and how the output doesn't meet your expectations?

Sure thing.
The ZIP file contains the dxfs in case you wanted them as well.
I really like what you’ve done so far but I use CNC for my work.
Thank you
Mike Schoonmaker
<EMAIL_ADDRESS>cell:<PHONE_NUMBER>
From: 9swampy @.>
Sent: Monday, October 24, 2022 2:15 PM
To: 9swampy/DeepNestSharp @.>
Cc: mnschoonmaker @.>; Author @.>
Subject: Re: [9swampy/DeepNestSharp] Spacing does not work (Issue #12)
Have to admit I'm a laser cutter... played around with the spacing to add to the tooltip explanations but haven't paid it much attention.
That said I do know the MergeLine logic I put in would have 0 chance of working with a CNC kerf but spacing... could you attach your Project, preferably with loadable Parts; loads of them in the DxfFiles folde in the repo just to demonstrate what your setup is, what gets generated and how the output doesn't meet your expectations?
—
Reply to this email directly, view it on GitHubhttps://github.com/9swampy/DeepNestSharp/issues/12#issuecomment-1289481611, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALPFSW2HDB7SUGGZYMAXFQLWE3N4NANCNFSM6AAAAAARNC7ELM.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>

Not sure where the zip went; any chance you could commit to a fork of the repo and I'll pick it up and take a look?

I don't know how to GitHub but I'll try
Mike Schoonmaker
Sent from my iPhone
Please excuse any spelling errors
On Oct 26, 2022, at 4:45 PM, 9swampy @.***> wrote:

Not sure where the zip went; any chance you could commit to a fork of the repo and I'll pick it up and take a look?
—
Reply to this email directly, view it on GitHubhttps://github.com/9swampy/DeepNestSharp/issues/12#issuecomment-1292688276, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALPFSW7BUZD2AVN6KKU32MDWFGRADANCNFSM6AAAAAARNC7ELM.
You are receiving this because you authored the thread.Message ID: @.***>

How does one "commit to a fork"?

400 - Copy.zip
attaching the zip file here

That zip only has your dxfs; I was hoping to get your dnproj too; that would have (most) of the settings you're trying to use which would help.
Regardless I set up a new project with a single 100x100 sheet and loaded 3x1.dxf (from the Dxfs folder in the application folder).
Each 1.dxf is 50x50; with SheetSpacing=0 and Spacing=0 then all 3 fit onto the 100x100 sheet.

With SheetSpacing=10 the sheet is reduced accordingly and only 1 part fits on each sheet:

Bumping the sheet up to 150x100 two now fit horizontally (and there's no space between the two parts), but still only 1 vertically:

Dropping SheetSpacing=0 and bumping Spacing=10 still get 2 parts horizontally, only 1 vertically:

Note the holes are smaller too; that's because the code is working by adding to the size of the parts; but that's only internal to the application.

If you

pop out the NestResult and then
pop out the SheetPlacement and then
LoadAllExact you'll see:

The two parts are horizontally spaced and that's what you'd get upon export.
I do admit this isn't a great UX... but it's all inherited from the original coding and I've just not paid it much attention. See if that helps you out; I was getting similar results using your Dxfs...

Didn't change any settings; loaded your project and relinked the parts files and ran it:

Which exports/LoadAllExact gives

As a side note, there may be something going wrong with your scaling because the parts were tiny compared to the size of your sheet (which I reduced to 300x50 for the purposes of taking the screenshots above); but the spacing works correctly as shown above. Aren't you getting the same?

Mike, how did this work out for you?

THANK YOU for following up! This forced me to take another look. I found 2 things:

(based upon your note re: sizing), I changed my dxf parts to use their metric counterparts. This explains your sizing concerns.
The lack of spacing that I brought up was based upon viewing the layouts in the app itself - NOT the dxf exports! When I (re)ran this with metric parts, I observed NO spacing in the app sheet layouts, but when I actually exported the dxf layout, the spacing was present! Not sure if this is the expected behavior or "by design", but it was kind of non-intuitive to not see the spacing while in the app preview.

Thanks!

Glad it worked out for you.

Nice, makes sense.
It's a consequence of what I started with and tbh I hadn't noticed because I always work with a zero kerf. The LoadAllExact I only put in to fix the previews which show the approximations actually nested by default so I could make sense of why the nest resulted as it did. It's making sense now but some crazy stuff was happening with early incarnations... so it was just an unintended side effect of that which clearly confused you; sorry for that but glad the explanation has helped.


Added "next.local.wholetale.org" development patterns

Please review and merge https://github.com/whole-tale/deploy-dev/pull/28 before looking at this PR.
Problem
We can now deploy the new Angular rewrite of the WT dashboard alongside Girder, but we do not have an easy way to develop against or make edits to the deployed code.
Approach
Similar to our processes for building Ember, I added rebuild_dashboard_next and watch_dashboard_next directives to the Makefile.
I also included watch_dashboard_dev, as this is helpful for quickly developing the legacy EmberJS dashboard and builds much faster compared to the production build.
NOTE: --deleteOutputPath=false is needed on ng build because by default the dist/ folder is deleted when rebuilding - this messes up the NGINX Docker container and causes mount failures. We avoid this by telling Angular to preserve this directory, thus preserving the Docker mount.
NOTE 2: for watch_dashboard_next, using --poll 15000 is not ideal, but --watch to build when files change was not behaving as expected when running within Docker. ng serve provides auto-refresh and many other nice features, but serves entirely from memory and does not produce a dist/ folder - this does not fit our desired pattern for mounting/serving the built code from NGINX.
How to Test
Prerequisites:

Checkout and run this branch locally
Deploy WT stack using make clean && make dev

You should have a full WT stack running, including a running bodom0015/ng-dashboard:wt container

Verify that you can access https://next.local.wholetale.org

You should be directed to the new Angular dashboard

One-Time Rebuild (Prod)

Make an edit to the UI (for example, modify https://github.com/bodom0015/wt-ng-dash/blob/master/src/app/%2Btale-catalog/tale-catalog/tale-catalog.component.html#L9 from Public Tales to Public Tales, see?
Run make rebuild_dashboard_next

yarn should attempt to update dependencies
An ng build process should start

Wait for the build to complete
Manually refresh https://next.local.wholetale.org

You should NOT receive a 500 error (this can happen when Docker loses track of the mounted path)
You should NOT receive a 403 error (this can happen when NGINX is serving a path with improper permissions)
You should see that your changes are reflected in the refreshed view

Polling Files for Changes (Dev)

Run make watch_dashboard_next

yarn should attempt to update dependencies
An ng build process should start

Wait for the build to complete
Make an edit to the UI source code and save the file (for example, modify https://github.com/bodom0015/wt-ng-dash/blob/master/src/app/%2Btale-catalog/tale-catalog/tale-catalog.component.html#L9 from Public Tales, see? to Public Tales - now I see)
Wait for a new build to start (this can take ~15 seconds, per the --poll argument passed to the command)

An ng build process should start

Wait for the build to complete
Refresh https://next.local.wholetale.org

You should NOT receive a 500 error (this can happen when Docker loses track of the mounted path)
You should NOT receive a 403 error (this can happen when NGINX is serving a path with improper permissions)
You should see that your changes are reflected in the refreshed view

@bodom0015 I'm sorry I merged this by accident. However, since it doesn't affect the other pieces of the deployment, let's just keep it that way and add "upgrades" in a subsequent PR. Is that OK with you?

That works for me 😆
I'll try to update all paths and references and whatnot to match the ngx-dashboard repo name


URL links are broken

URL links are broken
Most of the links does not work in this page:
Manual training
Feature Tables
Feature Store
Model registry
MLFlow tracking
(not checked all of them)
Please check and correct. Thx.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: b8f41568-e2b1-2f7f-62f8-d63d230ba5d9
Version Independent ID: b697ab11-568c-6f00-1abb-80da3023cf62
Content: What is Databricks Machine Learning?
Content Source: databricks/scenarios/what-is-azure-databricks-ml.md
Service: azure-databricks
GitHub Login: @leifbro
Microsoft Alias: saperla

@csokaakos
Thanks for your feedback! We will investigate and update as appropriate.

@csokaakos
Thanks for your feedback! I've assigned this issue to the author who will investigate and update as appropriate.
@leifbro
Please review this issue and update as appropriate.

Thanks for providing feedback that helps improve our documentation. This issue has been resolved. All of the links on this page have been fixed and verified that they're working. We are closing the issue.
#please-close


fix(progress-spinner): Rename ProgressCircle to ProgressSpinner

Closes #1495
R: @jelbourn

Make selectors md-progress-circle and mat-progress-circle work. Thanks for review! Please take another look.

Added  progress-circle.ts. PTAL. Thanks!

LGTM


cquery blogpost

Attempt #4

Looks fine to me.
A couple of things I couldn't add anywhere else:

Tried to comment on the last line of the file but GitHub didn't let me; grr... Anyway. I'd suggest to not tell people to contact you directly: today you are the expert on this topic, but who knows what will happen in the future. People reaching this blog post months/years down the road should be able to contact a generic resource for advise. And note that we have a commenting system on the blog, so they can ask there. (Note that you are already the post author, so you are getting credit anyway in a much nicer way!)

Would be nice to drop the uppercase letters from the filename. If you git clone your branch with the post, and then just git mv the post to the new name/date followed by a git commit and a git push, things should be fine. I have never used the GitHub UI so can't tell how it'll behave...

It's live! https://blog.bazel.build/
Note that the excerpt is not great though. If you want, you can add an "excerpt" tag to the post's metadata and it should be picked up as the summary to display in the front page: https://jekyllrb.com/docs/posts/#post-excerpts
(I was also about to click approve and suggest to use "Squash and merge" with a custom commit message, because the automatically-generated message that was persisted is quite ugly... but you beat me to it.)

Nice!
On Wed, Jun 6, 2018 at 2:29 PM Julio Merino<EMAIL_ADDRESS>wrote:

It's live! https://blog.bazel.build/
Note that the excerpt is not great though. If you want, you can add an
"excerpt" tag to the post's metadata and it should be picked up as the
summary to display in the front page:
https://jekyllrb.com/docs/posts/#post-excerpts
(I was also about to click approve and suggest to use "Squash and merge"
with a custom commit message, because the automatically-generated message
that was persisted is quite ugly... but you beat me to it.)
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel-blog/pull/100#issuecomment-395168528,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAVPDlcXyvbP-KBlkq8iJBLqBzRryom2ks5t6B9wgaJpZM4UXNOy
.


Docs: Change the border thickness guide

This is a beginner guide focused on changing the openpilot outer border width. I tried to focus on writing a guide that would've helped me a lot when I first started trying to figure out how to modify the openpilot UI.
After finishing the guide, the border should be thicker as shown below.

Testing:
I tested these steps on my MacBook 16in and my Ubuntu 20.04 machine. They worked as expected IF on a stable faster internet connection.
Screenshot preview of the guide

Closing as this isn't substantially different from the turning the speed blue guide. Feel free to make a PR to update that guide a bit.

If this is your first time using terminal
Our docs aren't the place for this. There's tons of other content online.


gpg does not recognized the nitrokey

Hello, I’ve upgraded the nitrokey into the latest firmware, and the operation ran fine:
(nitrokey) sebastien@trunk:~$ nitropy nk3 status
Command line tool to interact with Nitrokey devices 0.4.36
UUID:               B9EF61DC738B0E569C389B06AA249067
Firmware version:   v1.4.0-rc.2
Init status:        ok
Free blocks (int):  60
Free blocks (ext):  478

However, the card is not recognized by gpg:
(nitrokey) sebastien@trunk:~$ LANG=C gpg --card-status
gpg: selecting card failed: No such device
gpg: OpenPGP card not available: No such device

If I plug a nitrokey start instead the key is identified so this does not seems to be a configuration error from gpg:
(nitrokey) sebastien@trunk:~$ LANG=C gpg --card-status
Reader ...........: Nitrokey Nitrokey Start (FSIJ-1.2.19-43131736) 00 00
Application ID ...: D276000124010200FFFE431317360000
Application type .: OpenPGP
Version ..........: 2.0

Am I missing something ?

Is pcscd running on your system (i. e. does pgrep pcscd show some output)?  If yes, what is the output of pcsc_scan -r when the Nitrokey 3 is connected?

Hello, I’ve tried with both the nk3 and the nitrokey start.
With the nk3, the application does not return, just waiting:
sebastien@trunk:~$ pcsc_scan -r
Using reader plug'n play mechanism
Scanning present readers...
Waiting for the first reader... /

At the opposite, with the nitrokey start, I have an answer directly:
sebastien@trunk:~$ pcsc_scan -r
Using reader plug'n play mechanism
Scanning present readers...
0: Nitrokey Nitrokey Start (FSIJ-1.2.19-43131736) 00 00

For your information, the nk3 comes from a pre-order batch from 2021/04

What distribution do you use?  Can you find out the version of libccid on your system?

I’m using debian stable. The version of the library is 1.4.34-1

Okay, 1.5.0 is the first release of pcscd that supports the Nitrokey 3.  Please try the following steps to fix this problem:

Backup the current /etc/libccid_Info.plist file.
Obtain a current Info.plist file:

You can download it from this repository.
You can extract it from the Debian package for 1.5.0 or later.
You can build it from source:  Download the latest release from https://ccid.apdu.fr/files/ and run make Info.plist inside src.

Make sure that the file contains the string Nitrokey 3.
Copy the new Info.plist file to /etc/libccid_Info.plist.
Restart pcscd and re-run pcsc_scan -r.

I’ve backported the version from testing on my PC and this works now.
For the other people using debian stable, here is the documentation I’ve followed and the steps I’ve used for that:
mkdir backports && cd backports/
apt source ccid
cd ccid-1.5.2/
sudo mk-build-deps --install --remove
dch --bpo
fakeroot debian/rules binary
dpkg-buildpackage -b -us -uc
cd ..
sudo apt install ./libccid_1.5.2-1~bpo11+1_amd64.deb

And now the key is recognized by gpg:
sebastien@trunk:~$ LANG=C gpg --card-status
Reader ...........: Nitrokey Nitrokey 3 [CCID/ICCD Interface] 00 00
Application ID ...: D276000124010304000FB9EF61DC0000
Application type .: OpenPGP
Version ..........: 3.4

Thanks for your help, the issue is solved!

I have the same problem. With latest ccid pcsc_scan detects the key, but not  gpg --card-status


Move CertificationData to its own module

Could you rename this from CertificationData to something like PemCertificates or PemChain? That's what's actually being stored in here...

Could you rename this from CertificationData to something like PemCertificates or PemChain? That's what's actually being stored in here...

From https://download.01.org/intel-sgx/latest/dcap-latest/linux/docs/Intel_SGX_ECDSA_QuoteLibReference_DCAP_API.pdf table 9

Data required to verify the QE Report
Signature depending on the value of the Certification Data Type:

Byte array that contains concatenation of PPID, CPUSVN, PCESVN (LE), PCEID (LE).
Byte array that contains concatenation of PPID encrypted using RSA-2048-OAEP, CPUSVN, PCESVN (LE), PCEID (LE).
Byte array that contains concatenation of PPID encrypted using RSA-3072-OAEP, CPUSVN, PCESVN (LE), PCEID (LE).
PCK Leaf Certificate
Concatenated PCK Cert Chain (PEM formatted). PCK Leaf Cert||Intermediate CA Cert|| Root CA Cert
Intel® SGX Quote (currently not supported)
PLATFORM_MANIFEST (currently not supported)

Only type 5 is a pem chain, and maybe that's a flaw/issue with this PR.  Perhaps CertificationData should be an enum.  We get type 3 when one does a local attestation.  I haven't seen any of the others.


Height is incorrect

https://codesandbox.io/embed/v888k169wl
The skeleton ends up being 1 pixel taller than given in props. A simple workaround is to override the skeleton element's line-height
span {
    line-height: normal !important;
}

@jaripekkala
@dvtng
Hello!
height is 3 pixels greater than original -
if i have time i'll suggest a PR to solve it!

This is an unexpectedly complex issue that deals with the inner workings of line-height. I did some experimentation and here is what I came up. Ultimately, I do not think this is a bug, so there won't be any code changes to Skeleton.

Another way to do it is by directly altering the style:
.react-loading-skeleton {
    line-height: unset !important;
}

I use unset instead of normal because even with normal, it is still offset a couple of levels down.


Adds Poe and further configure some tools

Metadata

Fixed Issue: #42
Contributors: @iosifache

Proposed Changes

Poe integration by pyproject.yaml

New Functioning
Only the development environment was changes. No functionality was added to MutableSecurity.

As Coveralls have a scheduled maintenance in this moment, I will merge the pull request despite the fail of the check (the unit tests passed).


Fix DOMException on play()

See:
https://developer.chrome.com/blog/play-request-was-interrupted/
Chrome:
2audio.ts:30 Uncaught (in promise) DOMException: The play() request was interrupted by a new load request. https://goo.gl/LdLk22
(anonymous) @ audio.ts:30
localhost/:1 Uncaught (in promise) Error: A listener indicated an asynchronous response by returning true, but the message channel closed before a response was received
audio.ts:30 Uncaught (in promise) DOMException: The play() request was interrupted by a new load request. https://goo.gl/LdLk22

Firefox:
Uncaught (in promise) DOMException: The fetching process for the media resource was aborted by the user agent at the user's request.

fixed in 89a2813b07bf60f3cffa7b54a93a19ebef4752d6


Unable to use DefaultAzureCredential on Azurite

Which service(blob, file, queue, table) does this issue concern?
Blob
Which version of the Azurite was used?
3.27.0
Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)
npm
What's the Node.js version?
14.21.3
What problem was encountered?
I have been trying to setup Azurite in some python unit tests with DefaultAzureCredential.  I have referred to a previous ticket https://github.com/Azure/Azurite/issues/1750 but it does not specify how to setup. And on https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azurite?tabs=visual-studio#azure-blob-storage,
var client = new BlobContainerClient(
    new Uri("https://<IP_ADDRESS>:10000/devstoreaccount1/container-name"), new DefaultAzureCredential()
  );

, it shows my idea is doable. My test code
from azure.storage.blob import BlobServiceClient
from azure.identity import DefaultAzureCredential

token_credential = DefaultAzureCredential()
blob_service_client = BlobServiceClient(
    account_url="https://<IP_ADDRESS>:10000/devstoreaccount1", credential=token_credential, connection_verify=False
)
azurite_container = "abc"
container_client = blob_service_client.get_container_client(container=azurite_container)
if not container_client.exists():
    blob_service_client.create_container(name=azurite_container)

My above test code never works with below errors
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
To check existing issues, please visit: https://github.com/Azure/azure-cli/issues

        AzurePowerShellCredential: PowerShell is not installed
        AzureDeveloperCliCredential: Azure Developer CLI could not be found. Please visit https://aka.ms/azure-dev for installation instructions and then,once installed, authenticate to your Azure account using 'azd auth login'.

as the manual has never suggested what is the default AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET for Azurite.
Steps to reproduce the issue?
As above. No relevant log available on Azurite.
Have you found a mitigation/solution?
Connections using using shared key and sas work fine.

@phoebusm
Do you mean:

BlobContainerClient created from following code works well:

var client = new BlobContainerClient(
    new Uri("https://<IP_ADDRESS>:10000/devstoreaccount1/container-name"), new DefaultAzureCredential()
  );

BlobContainerClient created from following code not work :

token_credential = DefaultAzureCredential()
blob_service_client = BlobServiceClient(
    account_url="https://<IP_ADDRESS>:10000/devstoreaccount1", credential=token_credential, connection_verify=False
)
azurite_container = "abc"
container_client = blob_service_client.get_container_client(container=azurite_container)

If so, this looks the request send from SDK to Azurite should be different in the 2 scenarios (but they should be similar).
Would you please share Azurite debug log for request send from BlobContainerClient in above 2 scenarios, and we can have more idea on this.

@blueww Thanks for your prompt reply
1.
var client = new BlobContainerClient(
    new Uri("https://<IP_ADDRESS>:10000/devstoreaccount1/container-name"), new DefaultAzureCredential()
  );

is copied from https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azurite?tabs=visual-studio#azure-blob-storage. I use it as an example to show DefaultAzureCredential authentication should be supported by Azurite but the manual lacks guidelines for setup, e.g. what is the default AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET for Azurite?
And  no log was captured on Azurite while trying DefaultAzureCredential.

With shared key credential, my python example will work:

blob_service_client = BlobServiceClient(
    account_url="https://<IP_ADDRESS>:10000/devstoreaccount1",
    credential=AzureNamedKeyCredential(
        "devstoreaccount1", "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=="
    ),
    connection_verify=False,
)
azurite_container = "abc"
container_client = blob_service_client.get_container_client(container=azurite_container)
if not container_client.exists():
    blob_service_client.create_container(name=azurite_container)

I use below command for starting Azurite:

azurite --skipApiVersionCheck --blobPort 10000 --blobHost <IP_ADDRESS> -d debug.log --key server.key --cert server.pem

@phoebusm
It looks you don't start Azurite with parameter --oauth basic.
By default, Azurite doesn't support OAuth and bearer token. Please see how to enable OAuth authentication in :  https://github.com/Azure/Azurite#oauth-configuration

@blueww
I have tried adding parameter --oauth basic for starting Azurite. It doesn't change the test result.
Understand that the .net example works. Can you provide a working python example as well?

@phoebusm
I will close this issue as not hear from you for a long time.
From the linked https://github.com/Azure/azure-sdk-for-python/issues/33297, it looks this is not Azurite issue, but some client code issue.
Feel free to contact us again if you need any further assistance on Azurite.

Hi. I have the same error with nodejs
Doesn't work using DefaultAzureCredential
If I use the DefaultCredential example from official docs,it doesn't work
https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azurite?tabs=visual-studio%2Cblob-storage#azure-sdks
const { DefaultAzureCredential } = require("@azure/identity");
const { BlobServiceClient } = require("@azure/storage-blob");
const defaultAzureCredential = new DefaultAzureCredential();

async function uploadBlob(url, containerName, blobName, blobContents) {
    const blobServiceClient = new BlobServiceClient(url, defaultAzureCredential);

    const containerClient = blobServiceClient.getContainerClient(containerName);

    try {
        await containerClient.createIfNotExists();
        const blockBlobClient = containerClient.getBlockBlobClient(blobName);
        const uploadBlobResponse = await blockBlobClient.upload(blobContents, blobContents.length);
        console.log(`Upload block blob ${blobName} successfully`, uploadBlobResponse.requestId);
    } catch (error) {
        console.log(error);
    }
}

uploadBlob("https://localhost:10000/devstoreaccount1", "container-name", "foo.txt", "bar");

Log
RestError: Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.
RequestId:689a0715-0dbc-482a-bb74-99a4373018e1
Time:2024-10-03T21:41:31.245Z
 {
  "name": "RestError",
  "code": "AuthorizationFailure",
  "statusCode": 403,
  "request": {
    "url": "https://localhost:10000/devstoreaccount1/container-name?restype=REDACTED",
    "headers": {
      "x-ms-version": "REDACTED",
      "accept": "application/xml",
      "user-agent": "azsdk-js-azure-storage-blob/12.24.0 core-rest-pipeline/1.16.0 Node/22.9.0 OS/(x64-Windows_NT-10.0.19045)",
      "x-ms-client-request-id": "65e11060-f4f4-4c77-b237-01bc6f02c68c",
      "x-ms-date": "REDACTED",
      "authorization": "REDACTED"
    },
    "method": "PUT",
    "timeout": 0,
    "disableKeepAlive": false,
    "streamResponseStatusCodes": {},
    "withCredentials": false,
    "tracingOptions": {
      "tracingContext": {
        "_contextMap": {}
      }
    },
    "requestId": "65e11060-f4f4-4c77-b237-01bc6f02c68c",
    "allowInsecureConnection": true,
    "enableBrowserStreams": false
  },
  "details": {
    "errorCode": "AuthorizationFailure",
    "server": "Azurite-Blob/3.30.0",
    "x-ms-request-id": "689a0715-0dbc-482a-bb74-99a4373018e1",
    "content-type": "application/xml",
    "date": "Thu, 03 Oct 2024 21:41:31 GMT",
    "connection": "keep-alive",
    "keep-alive": "timeout=5",
    "transfer-encoding": "chunked",
    "message": "Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.\nRequestId:689a0715-0dbc-482a-bb74-99a4373018e1\nTime:2024-10-03T21:41:31.245Z",
    "code": "AuthorizationFailure"
  },
  "message": "Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.\nRequestId:689a0715-0dbc-482a-bb74-99a4373018e1\nTime:2024-10-03T21:41:31.245Z"
}

It works using StorageSharedKeyCredential  instead DefaultCredential
const { DefaultAzureCredential } = require("@azure/identity");
const { BlobServiceClient } = require("@azure/storage-blob");
const { StorageSharedKeyCredential } = require("@azure/storage-blob");

async function uploadBlob(url, containerName, blobName, blobContents) {

    const AZURE_STORAGE_ACCOUNT = 'devstoreaccount1';
    const AZURE_STORAGE_ACCESS_KEY = 'Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==';
    const sharedKeyCredential = new StorageSharedKeyCredential(AZURE_STORAGE_ACCOUNT, AZURE_STORAGE_ACCESS_KEY);

    const blobServiceClient = new BlobServiceClient(url, sharedKeyCredential);

    const containerClient = blobServiceClient.getContainerClient(containerName);

    try {
        await containerClient.createIfNotExists();
        const blockBlobClient = containerClient.getBlockBlobClient(blobName);
        const uploadBlobResponse = await blockBlobClient.upload(blobContents, blobContents.length);
        console.log(`Upload block blob ${blobName} successfully`, uploadBlobResponse.requestId);
    } catch (error) {
        console.log(error);
    }
}

uploadBlob("https://localhost:10000/devstoreaccount1", "container-name", "foo.txt", "bar");

Log
Upload block blob foo.txt successfully aec8994d-217e-40b6-834b-d324bc76833a

Contenxt

Nodejs : v22.9.0
Azurite: 3.30.0

Additional
I detected this using c# trying to setup a EventProcessorClient with a checkpoint based on blob storage


Feature/decimal answer format

Hi. Thank you for your contribution in our code. Please, apply ktlint to your code and make a new commit. Other than that, it looks good :+1:
Please send us a request in the top right corner of this page when you are finished so we can get notified to review as quick as possible.

Looks good to me.

PR is updated, however I can seem to be able to add any reviewer

I applied Ktlint on the whole code, it should be good. However I don't see why the empty lines are not showing on github...


[RFC] Plans for torchao

Summary
Last year, we released pytorch-labs/torchao to provide acceleration of Generative AI models using native PyTorch techniques. Torchao added support for running quantization on GPUs, including int8 dynamic quantization (W8A8) and weight-only quantization (int8 and int4) that were composable with torch.compile. Combined, the APIs launched in torchao were able to power SOTA generative AI models across multiple modalities: Segment Anything, Stable Diffusion, and LLaMa.
The results were showcased in these blog posts -
https://pytorch.org/blog/accelerating-generative-ai/,
https://pytorch.org/blog/accelerating-generative-ai-2/,
https://pytorch.org/blog/accelerating-generative-ai-3/
Our investment in torchao is to accelerate Generative AI, using native PyTorch features, ensuring composability with torch.compile.
In 2024, we plan to adopt the following strategy for development of torchao

We will launch torchao with the most important quantization techniques for LLMs and other GenAI models via a simple UX. Examples - GPTQ, AWQ, int8 dynamic quant.
We will stay on top of SOTA kernels within these spaces through to PTC and commit cpu/gpu kernels ourselves as necessary. Torchao will host a limited set of performant kernels for server (cpu/gpu) and executorch, with a clear recommendation on how to integrate and run inference on these backends.
Torchao will host non-standard dtypes, implemented via tensor subclasses. Examples - nf4, any4, mx4
Following the PyTorch design principle, the offerings of torchao will be usable and simple, including setup, dependencies, API surfaces.
We actively engage with the community/researchers to contribute new quantization techniques in native PyTorch code and developers to author performant kernels for these into torchao for different backends. An example would be to upstream the kernels built by the CUDA_MODE community into torchao.
As the code gets more mature/based on community demand - we will upstream techniques/kernels into PyTorch Core.

Let’s dive deeper into some of the coverage areas mentioned above.
Emerging dtypes
Dtypes like NF4, MX4, groupwise quantized int4 are used for implementing various optimization techniques in the models. Last year, we posted a plan on how we wish to support these dtypes in PyTorch. In torchao, we will host tensor subclass based implementation of dtypes, existing examples include uint4 and NF4 that users can use for their own quantization techniques or override the implementation to support other dtypes that might be useful.
Moreover, users don’t need to write triton or cuda kernels for their custom dtypes. The implementation can be in python and torch.compile will take care of generating performant kernels under the hood.
Quantization techniques
Quantization can be done on only weights or weights+activations. Typically LLM quantization techniques for BS 1 (memory BW bound) use weight-only quantization techniques. But for larger batch sizes, or longer context length cases or for general throughput bound models quantizing the activations is also beneficial. Quantization, however, impacts the model accuracy and researchers have published techniques to mitigate this accuracy impact which currently exist externally as one repository per technique.
In torchao, we will plan to support the following class of techniques using PyTorch, made available via a simple UX and following the one-file-per-technique principle.
LLM weight only quantization techniques
Post training quantization
The two most popular techniques externally are GTPQ and AWQ, available via AutoGPTQ and AutoAWQ which include the technique as well as the performant kernels for faster quantization ops.
To that end, we will start by re-implementing the GPTQ and AWQ techniques into torchao using PyTorch via a simple/intuitive UX that supports saving/loading of quantized models, while realizing the memory savings on disk. Some open questions we need to address here include -
How much VRAM will be required for different quantization techniques
How do we convert to-from weights quantized for different backends (cpu and gpu today use different weight packing format)
In the future, as more interesting and cutting edge techniques are introduced, researchers can directly implement them in torchao or our team can re-implement them in PyTorch.
Weight and activation quantization techniques
Post training quantization
We’ve already implemented W8A8 quantization via the int_mm kernel in core. This has shown speedup on models like SAM, SDXL without any impact to model accuracy and can be turned on via a simple one-line UX implemented via module swap or tensor subclass.
However the challenge here is that some smaller layer shapes might not benefit from quantization due to the overhead in quantizing and dequantizing the activation tensors. Users can either statically ignore quantizing these layers or have a higher level API that figures out which layers are sensitive to quantization. We plan to provide a higher level API via the auto quantizer that applies this technique to the layers that stand to benefit the most to provide the benefits of quantization without having to worry too much about the configs to use.
Quantization aware training
Techniques here require access to fine-tuning, to tune the model to reduce accuracy impact of quantization. Recently, research like LLM-QAT is promising, showing that we can go down to W4A8 and 4-bit KV cache for LLMs. Moreover, newer lower bit techniques like AQLM, Quip# also include a component of fine-tuning to improve the model accuracy.
We will include the APIs and workflow to enable users to do QAT on LLMs, starting with implementing the LLM-QAT paper in torchao and further extending it to support other dtypes like MX4.
Optimized kernels
Kernels
Optimized kernels are key to making models run faster during inference. Today, in core we already have performant kernels like int_mm and 4-bit weight quantization kernels for cpu (via intel) and gpu (via tinygemm). torchao will host performant kernels that will work with different backends with a guide on how to plug in these kernels into PyTorch models via the custom ops API. These kernels will compose with torch.compile, with the expectation that the user is expected to write a meta kernel implementation for this. For executorch, the expectation is that if the user provides a kernel that works with executorch then it should also work in eager mode.
We will also directly engage with the community, to upstream their performant kernels into torchao.
Autotuner
In order to use any CUDA kernel efficiently, we'll need to pick the right kernel hyperparameters. For an eager mode kernel, the same is true as well. A kernel autotuner will help here. We expect that the auto quantizer along with the kernel autotuner will make int8 dynamic quantization and int8/int4 weight-only quantization more usable and performant. A WIP example of what this might look like can be found here.
Release engineering
Shipping optimized, custom kernels requires extensibility mechanisms and release channels. We have custom operator support that integrates broadly, but our release mechanism might need to be optimized. It can be quite difficult to ship custom binaries across a broad range of operating systems and accelerators.
Conversion to-from popular model formats
We can add a conversion util from popular model storage formats like gguf into PyTorch’s state_dict format. This will enable users to take a pre-existing quantized model from llama.cpp and have it run via PyTorch eager mode for desktop cpu/gpu and executorch for on-device cases. We’ll share more details here soon.
Pruning
In addition to quantization, we’ve seen promising results with sparsity as well on GPUs. We will share more updates on what torchao will host for the space of sparsity/pruning in the near future.
We'd love to hear any feedback or questions from the OSS community on this RFC. Thank you!
cc @msaroufim @cpuhrsch @jerryzh168 @HDCharles @andrewor14 @jcaip @jisaacso

How about HQQ ?

No calibration needed
Supp
Supports 8,4,3,2,1 bits, quality

How about HQQ ?

No calibration needed
Supports 8,4,3,2,1 bits
Very fast quantization, instead of waiting for hours with GPTQ/AWQ
The quality is on-par or better than GPTQ/AWQ, especially at low bits
bit-unpacking and dequantization CUDA kernels available for all bits
Supports backprop for QLoRA training
Works with FSDP for distributed QLoRA training

@mobicham HQQ looks great and it would be great to add a PyTorch implementation of this to torchao. Supporting 3, 2, 1 bits is pretty neat and the QLoRA support is useful for us too (we have support for NF4 tensor).
We're currently in the process of adding GPTQ to enable running quantized models on GPU, CPU and executorch (on-device).
Would you be interested in contributing? We have a lightweight API recommendation for new techniques like - https://github.com/pytorch-labs/ao/blob/main/torchao/quantization/quant_api.py#L50. We can also add the kernels into torchao so users can take full advantage of the e2e inference speedups.

It would be nice to ensure that FSDP works out-of-the-box with quantized models -- we've explained the steps needed to make this work in these articles (and have provided a demonstration script, along with the needed to modifications to HQQ and bitsandbytes):

https://www.answer.ai/posts/2024-03-06-fsdp-qlora.html
https://www.answer.ai/posts/2024-03-14-fsdp-qlora-deep-dive.html

I'd be interested in hearing more details on how these changes will be implemented, and how extensible things will be. Will quantization support be added to triton? And torch.compile will be able to convert the python API to triton?
Will there be some fairly low level python API that we could pass all the needed quant state too, so that new algorithms could be largely implemented in pure python?

Would you be interested in contributing? We have a lightweight API recommendation for new techniques like - https://github.com/pytorch-labs/ao/blob/main/torchao/quantization/quant_api.py#L50. We can also add the kernels into torchao so users can take full advantage of the e2e inference speedups.

Great thanks, I will take a look at it @supriyar !

I think one important thing to look at is having a standardized Pytorch way of bitpacking/unpacking that all quantization methods can use. Currently, each quantization technique would implement its own bitpacking logic, which makes the CUDA kernels incompatible between different (linear quant) methods.

triton already supports 8-bits, and can also be used to do 4-bits too. We gave this a shot last year but saw significantly worse perf. CUTLASS might be a better option here (torch.compile should be able to pick the right kernels for us).

@supriyar
Initializing an empty tensor first like this is  faster that using torch.cat to bit-unpack tensors. torch.compile still struggles with int32 bitpacking, mainly used for 3-bit. We have a barchart that compares inference speed of a quantized Llama2-7B model usingtorch.compile for the dequantization step vs. using CUDA kernels: https://github.com/mobiusml/hqq/tree/master?tab=readme-ov-file#backend

What is the relationship between this project and torch.ao ? Is torchao a seperate project / development repo?
Happy to have found torchao in any case, lots of goodies...

Hi @mklasby (we met last year at NeurIPS) so eventually once things get proven out in torchao they would get upstreamed to torch.ao so the goal is to have this be a standalone repo to have a higher development velocity

Thanks @msaroufim! Looking forward to contributing to the effort!

one more thing to add to the list might be data layouts. For example, for unstructured weight sparsity, storing activations as [batch,  ..., features] is much less efficient than  [..., batch] on GPUs, because the latter allows for coalesced access patterns. E.g, the Sputnik paper just spares this a single comment

To enable coalesced memory accesses into all input and
output matrices, we store dense matrices in row-major layout
and sparse matrices in CSR format

But this actually means that you might need to add many transpose operations if the sparse matmuls are interspersed with standard, non-pointwise operations.
Similarly, I think, e.g., mixing convolution and attention layers currently would be quite problematic, because one uses [batch, feature, seq] and the other [batch, seq, feature] storage order. And of course, there is the old NCHW vs NHWC channels first/last issue.

Is there any interest in achieving better alignment of ao.pruning with torch.nn.utils.prune functionalities?
For example, torch.nn.utils.prune reparameterizes modules with <param_name>_orig and sets the original param name to the pruned tensor. In contrast, ao.pruning reparameterizes modules with parametrizations.<param_name>.original.
My assumption is that the sparsifier is intended for dynamic mask updates, which is somewhat hacky to perform on top of torch.nn.utils.prune functions currently. However, I think aligning these modules, where possible, will lead to a more compelling and intuitive user experience.

@mklasby - I think set of functional pruners (similar to torch.nn.functional) would be fairly universal. Then we can build modules that track state needed for incremental pruning or such. Do you think that'd fit the requirements?

@cpuhrsch Yes, that is essentially what I am envisioning. The functional pruners would be essentially a sophisticated topk function to score parameters based on the specific pruning algorithm and return the updated mask. Any state / buffers required to score the params can be passed from caller to the pruners.
I note that jaxpruner and the cerebras' pruning library wrap or subclass the optimizer, respectively, for dynamic sparse training. This is a potential route to consider as well for the modules that track state if we feel that having an additional sparsifier object is less than ideal.

Hi, your work is fantastic. Do you have plan to support static quantization? That is to say, not computing amax of activation when running inference, but stead using calibration to pre-compute the quantization scale to reduce dynamic scale overhead?
And do you have plan to support more op like conv2d?
Thanks!

Hi @zhexinli - Yes, we want to create a design that can separate calibration from quantization and that should include this as well. We can also add support for conv2d. We have limited support for 1-by-1 convolutions by swapping them for linears.

@mobicham are you on CUDA MODE? If not is there an email you could share? Mine is<EMAIL_ADDRESS>we're quite excited to see an HQQ contribution so wanted to see where your head is at and how we could collaborate together

Hi, your work is fantastic. Do you have plan to support static quantization? That is to say, not computing amax of activation when running inference, but stead using calibration to pre-compute the quantization scale to reduce dynamic scale overhead? And do you have plan to support more op like conv2d? Thanks!

Hi @zhexinli are you looking for static quant of specific ops like conv/linear or general graph based quantization support? And on what backends?
In addition to what @cpuhrsch said, we have a PT2 export based quantization flow in PyTorch that's based on full-graph capture that you can use to run models on x86 CPU and edge runtimes (https://pytorch.org/tutorials/prototype/pt2e_quant_ptq_x86_inductor.html)

hi, thanks for your introduction, currently I'm looking for a cuda backend for static quantization.

@msaroufim sure would to do that! I will send you an email for a follow-up!


[Detox 20] funpermaproxy.callable | undefined is not a function

What happened?
❯ yarn detox build -c ios.staging
yarn run v1.22.17
$ /Users/frexuz/www/currenxie-mobile-react-native/node_modules/.bin/detox build -c ios.staging
/Users/frexuz/www/currenxie-mobile-react-native/node_modules/detox/src/realms/DetoxContext.js:73
  element = funpermaproxy.callable(() => this[symbols.worker].element);
                          ^
TypeError: undefined is not a function
    at DetoxPrimaryContext.<instance_members_initializer> (/Users/frexuz/www/currenxie-mobile-react-native/node_modules/detox/src/realms/DetoxContext.js:73:27)
    at new DetoxContext (/Users/frexuz/www/currenxie-mobile-react-native/node_modules/detox/src/realms/DetoxContext.js:20:14)
    at new DetoxPrimaryContext (/Users/frexuz/www/currenxie-mobile-react-native/node_modules/detox/src/realms/DetoxPrimaryContext.js:31:5)
    at Object.<anonymous> (/Users/frexuz/www/currenxie-mobile-react-native/node_modules/detox/src/realms/primary.js:3:18)
    at Module._compile (internal/modules/cjs/loader.js:1085:14)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
    at Module.load (internal/modules/cjs/loader.js:950:32)
    at Function.Module._load (internal/modules/cjs/loader.js:790:12)
    at Module.require (internal/modules/cjs/loader.js:974:19)
    at require (internal/modules/cjs/helpers.js:93:18)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

What was the expected behaviour?
Tried follow the Migration instructions. Expecting build/test to work :)
Was it tested on latest Detox?

[X] I have tested this issue on the latest Detox release and it still reproduces.

Help us reproduce this issue!
No response
In what environment did this happen?
Detox version: 20.0.0
React Native version: 0.70.4
Node version: 14.18.1
Test-runner (select one): jest
Also

"jest": "29.3.1",
"ts-jest": "29.0.3",

Detox logs
No response
Device logs
No response
More data, please!
{
  "globalSetup": "detox/runners/jest/globalSetup",
  "globalTeardown": "detox/runners/jest/globalTeardown",
  "maxWorkers": 1,
  "preset": "ts-jest",
  "reporters": ["detox/runners/jest/reporter"],
  "setupFilesAfterEnv": ["./init.ts"],
  "testEnvironment": "detox/runners/jest/testEnvironment",
  "testTimeout": 300000,
  "verbose": true
}

init.ts
import { device } from 'detox'

beforeAll(async () => {
  await device.launchApp()
})

beforeEach(async () => {
  await device.reloadReactNative()
})

detox.config.js
module.exports = {
  testRunner: {
    $0: 'jest',
    args: {
      config: 'e2e/config.json',
      _: ['e2e'],
    },
  },
  behavior: {
    init: {
      exposeGlobals: process.env.DETOX_EXPOSE_GLOBALS === '0' ? false : true,
    },
  },
  apps: {
    'ios.staging.app': {
      type: 'ios.app',
      binaryPath:
        'ios/build/Build/Products/Release.Staging-iphonesimulator/CX Staging.app',
      build:
        "RN_SRC_EXT=e2e.ts,e2e.tsx,e2e.js xcodebuild -workspace ios/CurrenxieMobileReactNative.xcworkspace -scheme 'CurrenxieMobileReactNative - Release.Staging' -configuration Release.Staging -sdk iphonesimulator -derivedDataPath ios/build -parallelizeTargets -UseModernBuildSystem=YES | xcpretty",
    },
  },
  devices: {
    simulator: {
      type: 'ios.simulator',
      headless: process.env.CI ? true : undefined,
      device: {
        type: 'iPhone 13 Pro',
        os: 'iOS 15.2',
      },
    },
  },
  configurations: {
    'ios.staging': {
      device: 'simulator',
      app: 'ios.staging.app',
    },
  },
}

❯ cat node_modules/funpermaproxy/package.json
{
  "name": "funpermaproxy",
  "version": "1.0.1",

Thank you for the report!
The problem can be resolved either by re-creating yarn.lock file.
The generic fix is coming:
Thttps://github.com/wix/Detox/pull/3691

Upgrade your Detox dependency to ^20.0.1.
If the fix does not help, please reopen the issue.
Thanks for the report! 🙏

recreating yarn.lock worked 👌 20.0.1 should too :)

@kg-currenxie thanks again for the prompt report!
Would you consider adding currenxie to our showcase page? If so, please tap the join showcase button 🙏🏻

Hell yeah we would! :)


Gate blocked on master testmtlswithauthpolicy

(NOTE: This is used to report product bugs:
To report a security vulnerability, please visit https://istio.io/about/security-vulnerabilities/
To ask questions about how to use Istio, please visit https://discuss.istio.io
)
Bug description
gate is failing with
Flag --export has been deprecated, This flag is deprecated and will be removed in future.
apiVersion: authentication.istio.io/v1alpha1
kind: MeshPolicy
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"authentication.istio.io/v1alpha1","kind":"MeshPolicy","metadata":{"annotations":{},"labels":{"app":"security","chart":"security","heritage":"Tiller","release":"istio"},"name":"default","namespace":""},"spec":{"peers":[{"mtls":{"mode":"PERMISSIVE"}}]}}
  generation: 1
  labels:
    app: security
    chart: security
    heritage: Tiller
    release: istio
  name: default
  selfLink: /apis/authentication.istio.io/v1alpha1/meshpolicies/default
spec:
  peers:
  - mtls:
      mode: PERMISSIVE
error: error parsing /tmp/kubedelete569964428.yaml: error converting YAML to JSON: yaml: line 2: mapping values are not allowed in this context
authn_policy_test.go:49: command failed: "error: error parsing /tmp/kubedelete569964428.yaml: error converting YAML to JSON: yaml: line 2: mapping values are not allowed in this context\n" exit status 1

Affected product area (please put an X in all that apply)
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[X] Security
[ ] Test and Release
[ ] User Experience
[X] Developer Infrastructure
Expected behavior
no gate fialures
Steps to reproduce the bug
Just submit a dummy PR and the gate will fail with this signature.  This is a recent regression in master
Version (include the output of istioctl version --remote and kubectl version)
master: d78245f533d6697c27fd1b5510df2fd2f739b507
How was Istio installed?
it wasn't
Environment where bug was observed (cloud vendor, OS, etc)
linux prow CI
Additionally, please consider attaching a cluster state archive by attaching
the dump file to this issue.

Example Failure: https://gubernator.k8s.io/build/istio-prow/pr-logs/pull/istio_istio/15259/istio-pilot-e2e-envoyv2-v1alpha3-master/3119#istioioistiotestse2etestspilot-testmtlswithauthnpolicy

This may have been fixed yesterday with commit: 8d263a7e07347c389b834bee82ad4bf8d31864d8
PR: https://github.com/istio/istio/pull/15287 will validate the current gate.

The validation PR passed the gate.


[SpankBang] unable to extract uploader

[x] I've verified and I assure that I'm running youtube-dl 2016.08.12

Before submitting an issue make sure you have:

[x] At least skimmed through README and most notably FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

$ y -v 'http://spankbang.com/3vvn/video/fantasy+solo'
[debug] System config: []
[debug] User config: ['-4', '-f', 'best', '--prefer-free-formats', '--no-cache-dir', '--no-mtime', '--youtube-skip-dash-manifest']
[debug] Command-line args: ['-v', 'http://spankbang.com/3vvn/video/fantasy+solo']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.08.12
[debug] Python version 3.5.2+ - Linux-4.7.0-rc7-686-i686-with-debian-stretch-sid
[debug] exe versions: ffmpeg 3.1.2, ffprobe 3.1.2, rtmpdump 2.4
[debug] Proxy map: {}
[SpankBang] 3vvn: Downloading webpage
[SpankBang] 3vvn: Checking 1080p video format URL
[SpankBang] 3vvn: 1080p video format URL is invalid, skipping
[SpankBang] 3vvn: Checking 720p video format URL
[SpankBang] 3vvn: Checking 480p video format URL
[SpankBang] 3vvn: Checking 240p video format URL
WARNING: unable to extract uploader; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
[debug] Invoking downloader on 'http://spankbang.com/_3vvn/IjE0NzExMDA4NTAuNzki.brg5c9egXY0VEKsEQ56fYH0SXpQ/title/720p__mp4'

http://spankbang.com/3vvn/video/fantasy+solo

Thanks for the report. Uploader and description for Spankbang videos are fixed.


Publish the working demo on github.io

Is your feature request related to a problem? Please describe.
I'm missing a working demo somewhere online
Describe the solution you'd like
As a developer, I'd like to see a working demo in the browser before cloning the repo
Describe alternatives you've considered
What about an online working demo at stone_css.github.io?
It's all on GitHub then and easy to maintain.
Additional context
n/a

I have updated the readme and I published a demo here: https://kasuken.github.io/stone.css/


amy: ADSR envelope for SINE, SAW... doesn't do Attack properly

It looks the Attack segment goes to max amp over the first 5.8ms window, then sits there until the end of the attack segment.  It's supposed to ramp up gradually across the entire attack time.
Thus,
alles.send(osc=0, bp0='100,1.0,1000,0.0,0,0.0', bp0_target=alles.TARGET_AMP)
alles.send(osc=0, freq=2000, vel=1)

generates a waveform like this:

Notice how the envelope is basically rectangular for the first 100ms, before beginning its decay.

What happens if you do
alles.send(osc=0, bp0='0,0,100,1.0,1000,0.0,0,0.0', bp0_target=alles.TARGET_AMP) ?


Issue #176 cal license notification

Add check to Analysis Summary for CAL licenses usage and when the number of schedulers are less than the number of CPUs.  This is an info warning and would still require engineer to validate but brings light to the low scheduler use along with the CAL license which restricts to 20 physical or 40 logical cores.

@jagriffin5 I see you have create a PR. Are you ready for us to review or are you still working on it? I also just made a suggestion for a change

Yes, it is good to go.  I noticed the comments and approved/committed those suggestions.  Thank you for that.

Yes, it is good to go. I noticed the comments and approved/committed those suggestions. Thank you for that.

Great! I will test and review within the next 2-3 days and merge if all looks good.


Validator coloring thresholds

We need to decide which thresholds we want to use for the validator warnings:

Pulling in @gamarin2 @cwgoes @rigelrozanski
Scope:

[ ] decide on thresholds
[ ] implement thresholds

Slashes: First we need to define how many slashes one gets for each fault. Double signing should probably count as 20 or 30 slashes if 1 slash is given for 1 downtime penalty.

I like the idea. How about just adding up slashed percent: 0.01%  + 0.1% + 5% = 5.11%

Commission: I'm still skeptical that we need colours for this one. It really depends on so many factors like Atom price, etc.

But those factors go for everybody. I don't really get your hesitation. Could you maybe try to convince me a little more? :)

But those factors go for everybody. I don't really get your hesitation. Could you maybe try to convince me a little more? :)

How do you objectively determine if a commission is too high? You can't put absolute numbers, cause imagine Atom price tanks so hard that all validators have to substantially increase their commissions. Everybody would be in red.
So then you have the option of looking at the median commission or something like that, and then put in red validators that deviate from the median from +20%? +30%? It's too arbitrary for me.  But if we do decide to go with colors for commission, then I think it should be based on a median mechanism

I get it now. Thank you. I like the median idea. Let's go with your proposal and not color it in the first iteration.

I like the median idea too, and it might apply to more than just commission - delegators are always choosing between a set of real validators, why not base our UI indicators on how those validators are actually performing instead of hardcoded thresholds? In any case, for all stats, could we show on hover the percent deviation (up or down) from the median commission?
I think uptime duration and percentage are both important. Duration maybe doesn't require a color - and I don't think we want to treat "low" durations as bad - but it's definitely an important characteristic in a delegator choosing whether or not to put their funds at risk.
I'm not too opinionated on how exactly we count slashes; I think it's more important to link to the specific events in question (and maybe give the validator a chance to explain the situation - what if Tendermint had a bug?). Maybe a simple "has ever been slashed" green/red box would do.

maybe we want to reconsider changing the priority for this?
Tweet Forbole
Tweet Melea

What needs to be done to implement here?

it's quite easy on our end. nothing needed from the SDK, aside from some insight and guidance on how determine reasonable thresholds.
when does a validator turn from green to red?
not high priority but would be cool to hear thoughts from others if they have feelings on this subject.

@jbibla can you produce some numbers you think are good and then we can go with/discuss those?


Digital Ocean Terraform code to create multiple instances in multiple…

Add version support

This PR is the same as #237
Please update that one instead


Support environment variables for server hostname, port options

Feature:

Supports the following environment variables:

REDIS_HOST: Server hostname
REDIS_PORT: Server port

CLI options override the above environment variables

LGTM. Thank you!

Thank you for this tool. Really helpful when I just need the client but not the server. :+1:

Thank you for your contribution @GochoMugo  🌹


Enable contenthash in filename for dynamic assets

This should solve the main problem of dynamic assets caches getting stale after a version upgrade. Everything not affected will use querystring-based cache-busting, which includes files loaded via HTML or the worker scripts.
I tested with monaco and the PNG of jquery-minicolors and it's working fine for those.
This is basically a stripped-down version of https://github.com/go-gitea/gitea/pull/18632 which does not go all the way with contenthash in all filenames (as that requires non-trivial backend changes as well), but should be sufficient to solve the immediate issue of stale browser cache after upgrade.

Did one improvement to service worker path comparison, as I have not much faith on whether scriptUrl is consistent across browsers in regards to query string or absolute/relative URL.

https://github.com/go-gitea/gitea/pull/20813/commits/8247395de4a53912dc026097d0758bc8f3870a50 introduces assetVersion which holds the escaped query that is appended to the html-based and worker assets.

I ended up changing it to a hash again, URLs were to ugly with the plain version string, and I think it reflects badly on us if we don't encode it to a hash.

The reason why I prefer to use index.js?v=1.17.1 is: some users just report frontend errors like "Error occurs on index.js?v=.... line x" from console. Using app version directly will make maintainers to know the version in the first time, without asking more questions, and I do not think it's ugly.
Without introducing more unnecessary MD5, either approach LGTM.

I think it looks ugly and I don't really see a point in this helping bug reports, users are forced to enter a version into a bug report anyways, and it's visible on every page as well.

I think it looks ugly and I don't really see a point in this helping bug reports, users are forced to enter a version into a bug report anyways, and it's visible on every page as well.

Because the "forced version" only exists in GitHub issue template, while I collect many user feedbacks from non-github channels, like chats, forums, etc. 😄 Many users just post a screenshot and say "hey, it doesn't work"

I think I have to agree with @wxiaoguang here, what's the benefit of SHAing the version here? You may as well just put the version in.

Consistency I guess, 8 character opaque value used both by backend and webpack. We can change the mechanism at any time later and it will still look consistent. I don't care which hashing mechanism, but I generally avoid deprecated hashes like MD5 and SHA1, that's why I went with SHA256.

The reason why I prefer to use index.js?v=1.17.1 is: some users just report frontend errors like "Error occurs on index.js?v=.... line x" from console. Using app version directly will make maintainers to know the version in the first time, without asking more questions, and I do not think it's ugly.
Without introducing more unnecessary MD5, either approach LGTM.

But it will make user confusing that which the version mean? It's the index.js' special version or the whole site version?

But it will make user confusing that which the version mean? It's the index.js' special version or the whole site version?

Does it matter? Is there really any user would read or parse that query parameter? That's only valuable for maintainers and developers.
ps: I do not insist for it as my comment above, either approach LGTM.

It's back at AppVer again

But it will make user confusing that which the version mean? It's the index.js' special version or the whole site version?

Does it matter? Is there really any user would read or parse that query parameter? That's only valuable for maintainers and developers.
ps: I do not insist for it as my comment above, either approach LGTM.

Yes, I mean it may make contributors confusing. We have already over 1 thousand contributors from beginning.

Yes, I mean it may make contributors confusing. We have already over 1 thousand contributors from beginning.

Can you provide some details about how could they be confused and what's the negative effect for it?
ps: it could make the URL not ugly without double escaping.

I made a commit.  Major changes:

Remove the url.QueryEscape, the Go HTML template will escape it correctly. Otherwise there will be double-escaping.
Rename the JS assetVersion to assetVersionEncoded to declare that it is an already-encoded value and can be used in URL directly.
Replace the plus + to a URL-safe char ~ (others could are . and _), then the URL becomes more readable.

Now, that's what you will see:
/assets/css/index.css?v=1.18.0~dev-324-gfa8fc5d3a
/assets/js/index.js?v=1.18.0~dev-324-gfa8fc5d3a
/assets/js/eventsource.sharedworker.js?v=1.18.0~dev-324-gfa8fc5d3

Seems like useless extra work to me and I think we're nitpicking too much.

Don't really agree as it mangles the data and other places in JS code might need the value in original form later, but whatever, it works for the current purpose.

Seems like useless extra work to me and I think we're nitpicking too much.

Don't really agree as it mangles the data and other places in JS code might need the value in original form later, but whatever, it works for the current purpose.

Hmm, I didn't mean to make the problem more complex. My initial thought was that no need to do more MD5, and shared some thoughts about the version format when I was trying catch frontend bugs for users. For different version formats, I have written "either LGTM" in all my comments. 😁


[ERROR] negative phonon frequency in TDEP

Hello, Dear TDEP developer,
I used TDEP to extract the phonon band, but I meet a negative phonon frequency. I'm sure my material is stable and the phonon from DFPT (VASP) is quite good. Do you have any suggestions to help me solve this question? Thank you very much.
TDEP:

DFPT:

Hello,
you are running a 2D material, and it seems you have issues with the
interpolation. For TDEP there is a flag you can play with ( no rotational),
for 2D materials but also hermiticity and Huang invariances. Try turning
them all off to begin with.
https://tdep-developers.github.io/tdep/program/extract_forceconstants/
Matthieu
On Mon, Jul 29, 2024 at 2:24 PM WenjiangZhou @.***>
wrote:

Hello, Dear TDEP developer,
I used TDEP to extract the phonon band, but I meet a negative phonon
frequency. I'm sure my material is stable and the phonon from DFPT (VASP)
is quite good. Do you have any suggestions to help me solve this question?
Thank you very much.
TDEP:
image.png (view on web)
https://github.com/user-attachments/assets/5a127015-fb5e-46f0-864c-7937130e1ea3
DFPT:
image.png (view on web)
https://github.com/user-attachments/assets/e4d548b9-7715-45ee-82ad-4fce9978c5af
—
Reply to this email directly, view it on GitHub
https://github.com/tdep-developers/tdep/issues/81, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABWOT3YSKJXFDOZ3N5F6HYTZOYYBFAVCNFSM6AAAAABLUFLRL6VHI2DSMVQWIX3LMV43ASLTON2WKOZSGQZTKMRXHAYTQOA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
Professor Matthieu J Verstraete
Fellow, American Physical Society
Chair, Steering Committee, European Theoretical Spectroscopy Facility
www.etsf.eu
Alumnus Fellow, Young Academy of Europe  yacadeuro.org

Institute for Theoretical Physics (ITP) Department of Physics
Buys Ballot Gebouw/Building, Princetonplein 5, office
University of Utrecht,  3584 CC Utrecht
ITP Secretariat: +31 30 253 5928 E-mail<EMAIL_ADDRESS>
Group web page (Liège): http://www.nanomat.ulg.ac.be/

Nanomat lab, Q-Mat center, Université de Liège
Département de Physique, Bat. B5a, 4/50
Allée du 6 août, 19 B-4000 Sart Tilman, Liège Belgium
Phone : +32 4 366 90 17

European Theoretical Spectroscopy Facility (ETSF)

Mail<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

Thank you very much. I've solved this issue when I turn off the Huang invariances. I also find that you think deriving the proper invariances for a two-dimensional material is necessary. I'd like to contribute more if have chances.


Remove max rows per segment from compaction dialog

Description
Configuring auto compaction with single_dim partitioning in the web console, requires specifying maxRowsPerSegment in the "Tuning config" field, rather than the "Max rows per segment" field. Specifying maxRowsPerSegment in the "Tuning config" also works for dynamic and hash partitions. To make the workflow consistent for all partitioning schemes, remove the "Max rows per segment" field so that the value is always specified in the "Tuning config" field.
Before

After

This PR has:

[x] been self-reviewed.
[x] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
[x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
[x] added integration tests.

Abandoning in favor of the improved compaction dialog UI added by https://github.com/apache/druid/pull/10417. I'll add the E2E tests in a different PR.