Kaleido hangs on repeated write_image calls.

I am trying to use kaleido to make a gif, by writing out a series of png's then concatenating them together. The first few calls to write_image succeed, but at some point it just hangs. I have pickled my plotly figure and attach it to this post, as well as code to reproduce the phenomenon. I am using kaleido==0.0.3.post1 but I have tried it on many other pip install-able versions (0.0.1, 0.0.2, 0.0.3) to get the same effect. BTW, I really like this software! It makes it so much easier to write out plotly figures.
import pickle
import tempfile
from time import time
import numpy as np

with open('example_fig.obj', 'rb') as ff:
    figure = pickle.load(ff)

tdir = tempfile.gettempdir()
n_frames=60
zoom=2.5
z_offset=0.5

for i in range(n_frames):
    start = time()
    theta = (i * 6.28) / n_frames
    camera = dict(
        eye=dict(x=np.cos(theta) * zoom,
                 y=np.sin(theta) * zoom, z=z_offset)
    )
    figure.update_layout(scene_camera=camera)
    figure.write_image(tdir + f"/tgif{i}.png")
    print(time()-start)

example_fig.zip
This is the output before it stalls:
10.502809762954712
10.133813619613647
10.165985584259033
10.156563758850098
10.303324222564697
10.326725006103516
10.667879581451416
10.48402190208435
10.299865007400513
10.463427543640137
10.515920162200928
10.529618263244629
10.70783519744873
10.670512676239014
10.657939195632935
10.771514415740967
10.917421340942383
10.875455617904663
11.05155086517334
11.001903772354126
11.043968915939331
11.229110717773438
11.24083161354065
11.347798109054565
11.507253408432007
11.527276039123535
11.55422854423523
13.812325954437256
13.134565353393555
26.34061861038208
30.875459909439087

Thanks for the report and example @36000, I was able to reproduce the issue. Here is my output (after stopping the kernel after hanging for a minute or two):
12.307554006576538
10.964718103408813
10.78917670249939
10.571112632751465
10.875031471252441
10.775719165802002
10.85896372795105
11.100398302078247
11.008890390396118
11.293341398239136
11.152736186981201
11.235199689865112
11.43349838256836
11.315977096557617
11.50989055633545
11.462449789047241
11.620534181594849
11.627352714538574
11.686328887939453
11.837352275848389
12.046703577041626
12.051520586013794
12.280725002288818
12.206238508224487
12.196724653244019
12.240059614181519
12.327786684036255
12.407614469528198
12.416523456573486
17.26522159576416
22.575580835342407
22.482662439346313
---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
<ipython-input-13-12fa48e63b5c> in <module>
     11     )
     12     figure.update_layout(scene_camera=camera)
---> 13     figure.write_image(tdir + f"/tgif{i}.png")
     14     print(time()-start)

~/anaconda3/envs/kaleido_install/lib/python3.7/site-packages/plotly/basedatatypes.py in write_image(self, *args, **kwargs)
   3249         import plotly.io as pio
   3250
-> 3251         return pio.write_image(self, *args, **kwargs)
   3252
   3253     # Static helpers

~/anaconda3/envs/kaleido_install/lib/python3.7/site-packages/plotly/io/_kaleido.py in write_image(fig, file, format, scale, width, height, validate, engine)
    249         height=height,
    250         validate=validate,
--> 251         engine=engine,
    252     )
    253

~/anaconda3/envs/kaleido_install/lib/python3.7/site-packages/plotly/io/_kaleido.py in to_image(fig, format, width, height, scale, validate, engine)
    129     fig_dict = validate_coerce_fig_to_dict(fig, validate)
    130     img_bytes = scope.transform(
--> 131         fig_dict, format=format, width=width, height=height, scale=scale
    132     )
    133

~/anaconda3/envs/kaleido_install/lib/python3.7/site-packages/kaleido/scopes/plotly.py in transform(self, figure, format, width, height, scale)
    101         # response dict, including error codes.
    102         response = self._perform_transform(
--> 103             figure, format=format, width=width, height=height, scale=scale
    104         )
    105

~/anaconda3/envs/kaleido_install/lib/python3.7/site-packages/kaleido/scopes/base.py in _perform_transform(self, data, **kwargs)
    216             self._proc.stdin.write("\n".encode('utf-8'))
    217             self._proc.stdin.flush()
--> 218             response = self._proc.stdout.readline()
    219
    220         response_string = response.decode('utf-8')

KeyboardInterrupt:

There was also some internally buffered error log
import plotly.io as pio
scope = pio.kaleido.scope
print(scope._std_error.getvalue().decode())

<--- Last few GCs --->

[17066:0x359b00000000]   429018 ms: Scavenge 3589.3 (3616.1) -> 3589.4 (3616.3) MB, 5.9 / 0.0 ms  (average mu = 0.143, current mu = 0.000) allocation failure
[17066:0x359b00000000]   429024 ms: Scavenge 3590.5 (3616.9) -> 3590.5 (3616.9) MB, 4.3 / 0.0 ms  (average mu = 0.143, current mu = 0.000) allocation failure
[17066:0x359b00000000]   429029 ms: Scavenge 3590.5 (3616.9) -> 3590.4 (3617.7) MB, 5.2 / 0.0 ms  (average mu = 0.143, current mu = 0.000) allocation failure

<--- JS stacktrace --->

[0919/053546.664013:FATAL:memory.cc(38)] Out of memory. size=0
#0 0x55ebe72ef8a9 base::debug::CollectStackTrace()
#1 0x55ebe725e203 base::debug::StackTrace::StackTrace()
#2 0x55ebe726e925 logging::LogMessage::~LogMessage()
#3 0x55ebe7288ab9 base::internal::OnNoMemoryInternal()
#4 0x55ebe6eb95d0 (anonymous namespace)::OnNoMemory()
#5 0x55ebe6eb9359 blink::BlinkGCOutOfMemory()
#6 0x55ebe63e4b00 v8::Utils::ReportOOMFailure()
#7 0x55ebe63e4ac5 v8::internal::V8::FatalProcessOutOfMemory()
#8 0x55ebe654ab85 v8::internal::Heap::FatalProcessOutOfMemory()
#9 0x55ebe654c1dc v8::internal::Heap::RecomputeLimits()
#10 0x55ebe6548867 v8::internal::Heap::PerformGarbageCollection()
#11 0x55ebe6546128 v8::internal::Heap::CollectGarbage()
#12 0x55ebe6546641 v8::internal::Heap::CollectAllAvailableGarbage()
#13 0x55ebe655189e v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath()
#14 0x55ebe6522f60 v8::internal::FactoryBase<>::NewFixedDoubleArray()
#15 0x55ebe6688146 v8::internal::(anonymous namespace)::ElementsAccessorBase<>::GrowCapacity()
#16 0x55ebe67fa80f v8::internal::Runtime_GrowArrayElements()
#17 0x55ebe6d20638 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit
Task trace:
#0 0x55ebe78e8e6d IPC::(anonymous namespace)::ChannelAssociatedGroupController::Accept()
IPC message handler context: 0xC5F65154

Received signal 6
#0 0x55ebe72ef8a9 base::debug::CollectStackTrace()
#1 0x55ebe725e203 base::debug::StackTrace::StackTrace()
#2 0x55ebe72ef445 base::debug::(anonymous namespace)::StackDumpSignalHandler()
#3 0x7ff56341f540 (/usr/lib/x86_64-linux-gnu/libpthread-2.30.so+0x1553f)
#4 0x7ff56270d3eb gsignal
#5 0x7ff5626ec899 abort
#6 0x55ebe72ee3a5 base::debug::BreakDebugger()
#7 0x55ebe726edc2 logging::LogMessage::~LogMessage()
#8 0x55ebe7288ab9 base::internal::OnNoMemoryInternal()
#9 0x55ebe6eb95d0 (anonymous namespace)::OnNoMemory()
#10 0x55ebe6eb9359 blink::BlinkGCOutOfMemory()
#11 0x55ebe63e4b00 v8::Utils::ReportOOMFailure()
#12 0x55ebe63e4ac5 v8::internal::V8::FatalProcessOutOfMemory()
#13 0x55ebe654ab85 v8::internal::Heap::FatalProcessOutOfMemory()
#14 0x55ebe654c1dc v8::internal::Heap::RecomputeLimits()
#15 0x55ebe6548867 v8::internal::Heap::PerformGarbageCollection()
#16 0x55ebe6546128 v8::internal::Heap::CollectGarbage()
#17 0x55ebe6546641 v8::internal::Heap::CollectAllAvailableGarbage()
#18 0x55ebe655189e v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath()
#19 0x55ebe6522f60 v8::internal::FactoryBase<>::NewFixedDoubleArray()
#20 0x55ebe6688146 v8::internal::(anonymous namespace)::ElementsAccessorBase<>::GrowCapacity()
#21 0x55ebe67fa80f v8::internal::Runtime_GrowArrayElements()
#22 0x55ebe6d20638 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit
  r8:<PHONE_NUMBER>000000  r9: 00007fffa1a3fe10 r10:<PHONE_NUMBER>000008 r11:<PHONE_NUMBER>000246
 r12: 00007fffa1a410d8 r13: 00007fffa1a400b0 r14: 00007fffa1a410e0 r15: aaaaaaaaaaaaaaaa
  di:<PHONE_NUMBER>000002  si: 00007fffa1a3fe10  bp: 00007fffa1a40060  bx: 00007ff5622bb100
  dx:<PHONE_NUMBER>000000  ax:<PHONE_NUMBER>000000  cx: 00007ff56270d3eb  sp: 00007fffa1a3fe10
  ip: 00007ff56270d3eb efl:<PHONE_NUMBER>000246 cgf: 002b000000000033 erf:<PHONE_NUMBER>000000
 trp:<PHONE_NUMBER>000000 msk:<PHONE_NUMBER>000000 cr2:<PHONE_NUMBER>000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.

So it looks like we have a object/memory leak in the JavaScript portion of kaleido that shows up here due to the large figure size. This will probably take some time to track down, so here's a workaround you can use in the meantime
import pickle
import tempfile
from time import time
import numpy as np
import os
import plotly.io as pio
scope = pio.kaleido.scope

with open('example_fig.obj', 'rb') as ff:
    figure = pickle.load(ff)

# tdir = tempfile.gettempdir()
tdir = './out4'
os.makedirs(tdir)

n_frames=60
zoom=2.5
z_offset=0.5

for i in range(n_frames):
    start = time()
    theta = (i * 6.28) / n_frames
    camera = dict(
        eye=dict(x=np.cos(theta) * zoom,
                 y=np.sin(theta) * zoom, z=z_offset)
    )
    figure.update_layout(scene_camera=camera)
    figure.write_image(tdir + f"/tgif{i}.png")

    # Shutdown kaleido subprocess to free memory, it will
    # be started again on next image export request
    scope._shutdown_kaleido()

    print(time()-start)

The scope._shutdown_kaleido() call will shutdown the kaleido subprocess so that it will be started fresh on each export.  This will add a couple of seconds to each call to write_image, but there won't be any memory build up.
Hope this holds you over until we can track this down and get a fix out. Thanks again for the report.

Do you have any tips / tricks on how to speed up this gif creation process? I don't know the details, but it seems like each time I call write_image, under the hood, I am opening up the same figure using chromium. Is there an easy way to open the figure once, then rotate and take screenshots within the browser?

I don't know of anything different to do right now.
It would take some benchmarking to hone in on what would actually is taking so long right now.  I expect that a lot of the time is being taken up with JSON serialization (Python to C++, then C++ to JavaScript) given the size of the figures.
Do these figures use large numpy arrays? We've talked about optimizing this serialization path in the past, but haven't gotten it off the ground yet.
All that said, I could see a place for adding support to kaleido for batching image export requests.  One option would be to support inputting figures that define frames (https://plotly.com/python/animations/) and exporting these to a gif.  For your case of rotating a camera, I think the frames could each be pretty small, basically just defining the camera orientation.

My inputs to the scatter3d traces are long numpy arrays, I think those are what make the figures so big. I like that last idea of adding support for inputting figures that define frames.


Whitelist dokku commands

Add the ability to whitelist the dokku commands that users can run, globally or per-app.
@bpostlethwaite Please review.

Looks good to me :dancer:

@josegonzalez If you have any thoughts on this (especially on the secure multi-tenancy section I added to the README) I'd appreciate those too!

Acl usually lets you do something like
acl:add user group
acl:remove user group
acl:list-groups
acl:user-permissions

etc. I'd prefer that sort of interface, and then people can whitelist/blacklist perms for specific users/groups. You can even keep the special admin user around if you'd like.

@josegonzalez Groups are an interesting idea, but they would make this PR a lot more complicated.
First of all, the existing plugin already lets you run acl:add user app so we'd need a second "add" command for groups, which would be a bit confusing.
Secondly, storing and querying which users are in which group would add a fair bit of complexity to the implementation of this PR, and it would make configuration more complex. Instead of two lists of commands, the administrator would have to configure two lists of commands per group, increasing the chances of configuration errors.
I think if someone needs group functionality they can implement it in their own PR. Does that seem reasonable? I don't think anything in this PR prevents that from being done in the future.

Fair enough.


XML support

It's less relevant nowadays but it's used still by SAML etc... We and users will need such shards.
Implement shards to read and write using https://crates.io/crates/quick-xml

Hey team! Please add your planning poker estimate with Zenhub @guusw @sinkingsugar

Not an high priority but why not.


Unexpected conversion from 'true'/'false' to some integer  types

Description
Converting from "value": true or "value": false behaves unexpectedly and differently for certain integer types.
I was expecting all conversions from a boolean true/false to integer value to fail, however it is stored as a 1 or 0.
Reproduction steps
Given:
template<typename T>
class Value {
public:
    T value;
private:
    NLOHMANN_DEFINE_TYPE_INTRUSIVE(Value<T>, value);
};

This will result in a 1 in value.
auto v = R"({
    "value" : true
})"_json.get<Value<int>>();

Using uint64_t will throw a type_error.
auto v = R"({
    "value" : true
})"_json.get<Value<uint64_t>>();

Expected vs. actual results
I expected all boolean to integer conversions to fail.  Instead a true -> 1 and false -> 0 for certain integer values.
Various integer conversions - https://godbolt.org/z/7Wrh6EanW
The uint64_t case is taking the path through get_arithmetic_value
template<typename BasicJsonType>
inline void from_json(const BasicJsonType& j, typename BasicJsonType::number_unsigned_t& val)
{
    get_arithmetic_value(j, val);
}

The int case is taking the path through

struct from_json_fn
{
    template<typename BasicJsonType, typename T>
    auto operator()(const BasicJsonType& j, T&& val) const
    noexcept(noexcept(from_json(j, std::forward<T>(val))))
    -> decltype(from_json(j, std::forward<T>(val)))
    {
        return from_json(j, std::forward<T>(val));
    }
};

// which calls

template < typename BasicJsonType, typename ArithmeticType,
           enable_if_t <
               std::is_arithmetic<ArithmeticType>::value&&
               !std::is_same<ArithmeticType, typename BasicJsonType::number_unsigned_t>::value&&
               !std::is_same<ArithmeticType, typename BasicJsonType::number_integer_t>::value&&
               !std::is_same<ArithmeticType, typename BasicJsonType::number_float_t>::value&&
               !std::is_same<ArithmeticType, typename BasicJsonType::boolean_t>::value,
               int > = 0 >
inline void from_json(const BasicJsonType& j, ArithmeticType& val)
{
//...

        case value_t::boolean:
        {
            val = static_cast<ArithmeticType>(*j.template get_ptr<const typename BasicJsonType::boolean_t*>());
            break;
        }
//...
}

Minimal code example
https://godbolt.org/z/fs4frqz7G
#include <cassert>
#include <iostream>

#include <nlohmann/json.hpp>

template<typename T>
class Value {
public:
    T value;
private:
    NLOHMANN_DEFINE_TYPE_INTRUSIVE(Value<T>, value);
};

int main() {
    try {
        std::cout << "uint64_t" << std::endl;
        auto v = R"({
            "value" : true
        })"_json.get<Value<uint64_t>>();

        assert(false);
    } catch (const nlohmann::detail::type_error& e) {
        std::cout << e.what() << std::endl;
        std::cout << "ok - exception expected" << std::endl;
    }

    try {
        std::cout << "int" << std::endl;
        auto v = R"({
            "value" : true
        })"_json.get<Value<int>>();

        assert(false);
    } catch (const nlohmann::detail::type_error& e) {
        std::cout << e.what() << std::endl;
        std::cout << "ok - exception expected" << std::endl;
    }
}

Error messages
No response
Compiler and operating system
Latest MSVC & Clang
Library version
3.11.3
Validation

[ ] The bug also occurs if the latest version from the develop branch is used.
[ ] I can successfully compile and run the unit tests.

I am having the same issue and apologies if I am not understanding this correctly as I am new to C++ and have only just started using this library, but it looks like  the integer value is not being considered a basic_json and so is calling the explicit cast from_json. At least the comment above the function leads me to believe this.
// overload for arithmetic types, not chosen for basic_json template arguments
// (BooleanType, etc..); note: Is it really necessary to provide explicit
// overloads for boolean_t etc. in case of a custom BooleanType which is not
// an arithmetic type?

hi i am working on this.Will get something by this weekend.

Are you still working on this? I'm planning to look into it.

If I'm planning on contributing, is there any guidelines or styles that I need to follow while writing the code? If yes, where can I read about it? Thanks.

See https://github.com/nlohmann/json/blob/develop/.github/CONTRIBUTING.md

@amirghaz Do you need further assistance?

@nlohmann I'm a beginner with this library, so any insight that you have is highly appreciated.

I see, but I lack the experience dealing with preprocessor macro. @jboelterintc do you have any suggestion?

I managed to write a (partial) fix that passed all 98 unit tests. However, it is only a partial fix because, although all other integer types return a type error, uint8_t still perform the conversion.
I first write a fix that force all integer types to throw a type error, but doing so will causes some unit tests (22, 23, 24) that involve uint8_t to bool conversion to fail. So, I end up implementing this partial fix.

It seems my newest fix still fail some checks. Before moving forward with another fix, maybe we should discuss if the partial fix I mentioned above is the way to go. Currently, only int64_t and uint64_t will return a type error, and based on the unit tests, it seems that conversion from uin8_t to boolean is required (except if there is a way to circumvent this that I am unaware of).
So, any fix potentially will still have some inconsistencies because of uint8_t. Is this really what we want? Any opinion @nlohmann  @jboelterintc on this?


Add auth and database features

What was changed

This PR includes two primary functionality enhancements to the helm chart:

Addition of Auth settings to both server and web
Allowing database username to optionally be specified in a secret alongside the password

Why?

I'll address each area separately:
Auth - production environments need some form of authentication.  To use authentication, it was required to be enabled in both the server and web components.  Additionally, an internal-frontend service was added (per the v1.20.0 release) to allow the workflow service to function.  This also addresses an open issue (#304).
Database Creds - many dynamic database creds generation systems (such as hashicorp vault) generate both the username and password dynamically, which is more secure than just rotating the password.  The functionality has been updated to allow either or both of these to be optionally set via a secret with any or both of username and password keys.
Checklist

Closes  #304

How was this tested:

Tested a number of times using various alternatives of the configurations.

Any docs updates needed?

Included documentation via commented out items in values.yaml

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thank you for your contribution :)
Please don't mix concerns in the same PR, split out: 1. database credentials 2. internal-frontend support
For the auth config side, I'm not sure about proxying all of the values this way. Web can be configured via additionalEnv and for server auth I think maybe just have a auth: { enabled: true, config: { ... } } that gets copied as well.


Add the case in fem_pos_deviation_osqp_interface when the number of optimized path points is 3

The minimum number of optimized path points is 3 in fem_pos_deviation_osqp_interface. Currently, the QP modeling in the CalculateKernel function only works when the number of path points is greater than or equal to 4. If the number of path points is 3, the code will crash. So I add a separate case for the QP modeling when the number of optimized path points is 3.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@qwetqwe pls help to review this!


Fixing log probs for l_t

See: https://github.com/kevinzakka/recurrent-visual-attention/issues/26#issuecomment-541442792

Thanks @clvcooke !!


v2 - loading the document requires knowing the format in advance

new OpenApiStreamReader().Read(stream, out var diagnostic);

(or its async variant)
Did not require knowing the format of the document (JSON/YAML) in advance. This was handy as when building general purpose tools, it's not something that I know in advance. Parsing the file extension/url extension/response content type header might help discovering it, but it's not reliable.
The equivalent method
OpenApiDocument.LoadAsync(input, "unknown", settings)

Requires passing the format, which I believe is a regression in functionality. We should work to restore the same level of functionality.
Also, there are a number of places in the documentation and doc comments that still refer to the old initialization and require to be updated before GA.
https://github.com/search?q=repo%3Amicrosoft%2FOpenAPI.NET OpenApiStreamReader&type=code

We could create an overload that looks like this:

OpenApiDocument.LoadAsync(input, settings);

We first try and parse as JSON.  If the JSON fails, we can say "either you have bad JSON, or you gave me YAML, and you have not registered the YAML reader, in which case you need to take a dependency on the OpenAPI.Readers package, and register the YAML reader."

Try and parse as JSON
If it fails, check to see if YAML parser is registered
If it is try parsing with the YAML parser

Thank you for the additional information.
I don't understand why we to direct the parsing between two different parsing logics:

The YAML 1.23 specification was published in 2009. Its primary focus was making YAML a strict superset of JSON

source
We could simply feed everything to the YAML parser, couldn't we? It it happens to "only be JSON" there's extra parsing logic that would be used. It might have a performance impact since yaml parsing ultimately converts back to JSON nodes.
It's also worth noting this is what the library used to do before the recent changes.
Then for people who really care about the performance impact, we could still leave in place the overloads that accept a format argument, bypassing this overhead. Also for any method that accepts a URI, we could look at the response content type and call the right format parser.
Thoughts?

@baywet Parsing Graph beta as JSON takes ~450ms. Using YamlSharp it take around 5secs. I think that is a difference we should try harder to take advantage of. Especially because many consumers of the library will not tell us what format they are sending.
@maggie. Maybe we should revisit sniffing the content. We might be able to "peek" at the first few bytes without consuming the content. Also, after reviewing the full set of APIs yesterday I have a few suggested changes. One of them includes making the TextReader go away. I don't think it is useful for JSON. Scenarios. We may be able to support it just in the OpenAPIYamlReader.

@baywet Parsing Graph beta as JSON takes ~450ms. Using YamlSharp it take around 5secs. I think that is a difference we should try harder to take advantage of. Especially because many consumers of the library will not tell us what format they are sending.
@Maggie. Maybe we should revisit sniffing the content. We might be able to "peek" at the first few bytes without consuming the content. Also, after reviewing the full set of APIs yesterday I have a few suggested changes. One of them includes making the TextReader go away. I don't think it is useful for JSON. Scenarios. We may be able to support it just in the OpenAPIYamlReader.

Then I think we can continue reviewing the work I did in this PR which includes sniffing the content to detect the format https://github.com/microsoft/OpenAPI.NET/pull/1929


Migrated to Swift 5

@K-Be thanks - I'll try to take a look at this soon! I'll have to get the build passing, too, which I think should be a relatively simple fix to the travis CI config.

@K-Be FYI I just added a small commit on your branch to try to get the travis build working - if we can get a passing build, then I can get this merged.

Codecov Report

Merging #210 into master will decrease coverage by 0.9%.
The diff coverage is 50%.

@@            Coverage Diff             @@
##           master     #210      +/-   ##
==========================================
- Coverage   26.99%   26.08%   -0.91%
==========================================
  Files          15       13       -2
  Lines        2297     2308      +11
==========================================
- Hits          620      602      -18
- Misses       1677     1706      +29

Impacted Files
Coverage Δ

Source/XMLIndexer+XMLIndexerDeserializable.swift
71.67% <50%> (+11.08%)
:arrow_up:

Tests/SWXMLHashTests/XMLParsingTests.swift
0% <0%> (ø)
:arrow_up:

...LHashTests/TypeConversionPrimitypeTypesTests.swift
0% <0%> (ø)
:arrow_up:

Tests/SWXMLHashTests/shim.swift

Source/shim.swift

...SWXMLHashTests/TypeConversionBasicTypesTests.swift
0.51% <0%> (+0.51%)
:arrow_up:

Source/SWXMLHash.swift
82.06% <0%> (+3.98%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d6bb31...94d2e22. Read the comment docs.

4.9.0 is now released - thanks again @K-Be!

Thanks for reporting and having fixed the issue :)
Working great!


Removes dead style guide link; corrects link to contribution guide

Noticed the style guide was a hangover from the glossary cloning, and our contribution page is called how-to. This should correct the latter but note I'm not building locally I'm inferring this change from the existing structure.

@garry-cairns - can you ensure that commit passes the DCO check?

changes will conflict with upcoming changes in https://github.com/cncf/architecture/pull/19 - IMO we can close this PR here and continue with #19


Keybinding for switching tabs should work from anywhere.

Is this change possible?
A clear example of something I can't do right now:

cmd+f to search in the current file
type something & search
cmd+shift+[ or cmd+shift+] to change tabs and search the same string in a different file.

Pressing esc gets me out of the "find" panel but that's not good enough.  Unless the "find" panel is using the same keybinding for something, maybe it could just fall through and let me change tabs?

Don't want to pollute the issue, but please-please fix it.


⚠ Samsung has degraded performance

In e6087da, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 2608 ms

Resolved: Samsung performance has improved in 01c5322.


Support for deploying JupyterLab

The cpanel-api has been updated to allow deploying JupyterLab, this updates the
frontend to use the new functionality and refactors the tools template to
support multiple tools instead of hardcoding rstudio
https://trello.com/c/2i9vRoTQ/1338-stretch-jupyter-deployable-from-the-control-panel

 Stretch:  Jupyter deployable from the control panel


[ENH] Simplify auth and correctly overwrite singleton tenant+db

Reviewer notes

I would recommend avoiding the diff and reviewing the end state as if it were fresh code. Trying to understand the diff will lead to madness.
Crossed-out notes left for posterity but outdated.
ServerAuthenticationProvider returns None if it fails but ServerAuthorizationProvider raises an error -- should we standardize this behavior?
The FastAPI boilerplate in chromadb/server/fastapi/__init__.py is a little unwieldy. I'm fine with it since this file is mostly boilerplate and we very infrequently update it, but I'm open to ideas for how to make it cleaner.

However, everything should probably be async yeah?

ServerAuthenticationProvider now has the job of overwriting singleton tenant and database if the setting is specified and it's possible. This leads to a bit of weirdness where we have to call into this twice -- once in the authn/authz flow and then again in the core request flow once authn/authz is complete. Two questions about this:

Should the overwrite functionality live somewhere else? ServerAuthenticationProvider is the arbiter of user identity so it seems like a natural place in theory..
Should authenticate_and_authorize_or_raise() just return the new tenant and db if they exist? That feels like a weird mix of responsibilities but would mean we only need to call the method once. The method is very very cheap though.

Description of changes
Summary and end state

This PR takes away two minor functionalities from users. Other than these, it does not reduce user abilities.

If anyone overrode ClientAuthProtocolAdapter to periodically re-authenticate their clients (unlikely imo) they are no longer able to do so. If someone tells us they have actually been doing this it will be easy to add re-auth functionality back.

We don’t offer a baked-in auth scheme which requires re-authentication.

Users now must store auth info (e.g. usernames and passwords) in files instead of passing them directly as config to their Client or Server. I’m open to adding this back.

Client auth is simpler. It now all takes place in a single configurable ClientAuthProvider class.
Server authn is simpler. It now all takes place in a single configurable ServerAuthnProvider.
Server authz is simpler. It now all takes place in a single configurable ServerAuthzProvider.
Server authn and authz are now explicitly executed as part of each API handler. This is also where we overwrite request tenant and db if we configured to. See authenticate_and_authorize_or_raise in chromadb/server/fastapi/__init__.py.
Overwriting singleton tenant and db from auth is now much clearer and well-tested.

Overall changes

Organized Settings.
Wrote or cleaned up docstrings for all auth-related classes.
Best-effort clean up of every formatting and linting error in every file I touched.
Combined chroma_server_auth_token_transport_header and chroma_client_server_auth_transport_header into chroma_auth_token_transport_header.
Deleted AuthenticationError as it was unused.
Deleted the registry (registry.py). It’s a pattern we should either have across our entire codebase or nowhere — as it stands, it only served to make the code harder to read while barely simplifying users’ configs.
Renamed everything server-side to “authn” or “authz”. Nothing server-side has only “auth” in it now unless it applies to both (e.g. chroma_auth_token_transport_header, chroma_server_auth_ignore_paths).
I eliminated all server middleware. I would like to use it, but that would require doing deep request inspection (especially once we enable e.g. collection-level auth). This would mean we essentially maintain a completely separate schema of paths and request fields — it seems best in the short, medium, and long runs to explicitly do authentication and authorization as part of handler bodies.

Client-side

Deleted ClientAuthConfigurationProvider as it was unused.
Deleted ClientAuthCredentialsProvider as it was extremely small and only ever used by ClientAuthProvider. Folded its functionality into ClientAuthProvider.
Deleted AuthInfoType as it was doing nothing (we only supported AuthInfoType.HEADER in practice).
Deleted ClientAuthResponse as it was just a dict for headers. Replaced with ClientAuthHeaders type.
Deleted ClientAuthProtocolAdapter as it was just a complicated way to call ClientAuthProvider.authenticate() and inject headers. Folded this functionality into ClientAuthProvider and the FastAPI client directly.
Deleted TokenAuthHeader as it was only used in one place (TokenAuthClientProvider.authenticate()). Folded it into said place.

Server-side authn

Configuration

Deleted chroma_server_auth_configuration_provider as it was unused.
Deleted chroma_server_auth_configuration_file as it was unused.
Deleted ServerAuthConfigurationProvider as it was unused.
This leaves no configuration besides the creds file. This is fine. There’s no configuration for our built-in authn providers. If users create their own, they can configure them as they’d like with files or env vars.

Credentials

Deleted chroma_server_auth_credentials — we now require users to set up auth through a credentials file.
Deleted HtpasswdConfigurationServerAuthCredentialsProvider and TokenConfigServerAuthCredentialsProvider as they are unused now that chroma_server_auth_credentials is gone.
With HtpasswdConfigurationServerAuthCredentialsProvider gone, the only subclass of HtpasswdServerAuthCredentialsProvider was HtpasswdFileServerAuthCredentialsProvider so I combined them.

User identity + request representation

Deleted UserIdentity as there was only one instance of it (SimpleUserIdentity).
Renamed SimpleUserIdenity to UserIdentity and made it a dataclass.

If anyone needs to, they can subclass this and add whatever they want.

Deleted AuthorizationRequestContext. It was just a wrapper around the request and nothing else.

Request and Response representation

Deleted ServerAuthenticationRequest as it was only ever used to wrap a dict lookup in several layers of indirection. Replaced with raw starlette Headers.
Deleted the ServerAuthenticationResponse abstraction as it was only ever used as a SimpleServerAuthenticationResponse and FastAPIServerAuthenticationResponse, two subclasses which contained the same data.
Replaced all instances of ServerAuthenticationResponses with an Optional[UserIdentity] which is all they really were.

Middleware

Deleted ChromaAuthMiddleware, FastAPIChromaAuthMiddleware and FastAPIChromaAuthMiddlewareWrapper in favor of explicit authn within request handlers.

ServerAuthProviders, AbstractCredentials, and ServerAuthCredentialsProviders

We previously had two each of SecretStrAbstractCreds, ServerAuthProvider, and ServerAuthCredentialsProvider: one Basic and one Token. (So we had BasicAuthCredentials + TokenAuthCredentials, BasicAuthServerProvider + TokenAuthServerProvider, etc.) You could only ever use all Basic or all Token; they didn’t mix and match as they had a lot of shared assumptions about e.g. which keys existed in which dicts.

To make things extra confusing, the BasicAuthServerCredentialsProvider was actually called HtpasswdServerAuthCredentialsProvider and lived in providers.py.

I deleted all the Credentials classes and AuthCredentialsProvider classes. Folded all this functionality into two auth providers: BasicAuthServerProvider and TokenAuthServerProvider.
Deleted chroma_server_auth_credentials_provider since we no longer have separate ServerAuthCredentialsProviders.

Added functionality to ServerAuthenticationProvider to get a user’s singleton tenant and database if they exist and if the relevant setting is True.

Server-side authz

Deleted chroma_server_authz_config as it seems like nobody was using it.
Deleted chroma_server_authz_ignore_paths since it is pretty much the same as chroma_server_auth_ignore_paths. We now use chroma_server_auth_ignore_paths for both.
Deleted AuthzUser as it was identical to UserIdentity except for one unused field.
Deleted ChromaAuthzMiddleware, FastAPIChromaAuthzMiddleware and FastAPIChromaAuthzMiddlewareWrapper in favor of explicit authz within request handlers.
Deleted AuthzResourceTypes as it only ever contained data which was trivially derivable from the action being taken and was only used to print.
Modified AuthzResource to explicitly have all the fields it may need. We can extend it in the future, or users can override it.
Deleted AuthzAction as it was only ever used to print an error string and only ever contained information included in AuthzResourceActions.
Renamed AuthzResourceActions to AuthzAction since it’s just an enum representing the set of actions we might want to run authz checks on.
Deleted AuthorizationContext as it was just a wrapper around some args. We pass args directly now.
Deleted LocalUserConfigAuthorizationConfigurationProvider as all it did was load a config file and it was only ever used in SimpleRBACAuthorizationProvider. Folded its functionality (opening a file) into SimpleRBACAuthorizationProvider.
Deleted all the dynamic auth stuff. We now explicitly do authn and authz in request handlers.

Testing

Added tests for the new ServerAuthenticationProvider behavior (deciding which paths to ignore for auth based on config, and finding a user’s singleton tenant and database if chroma_overwrite_singleton_tenant_database_access_from_auth = True.
Added tests for the newly supported multiple-usernames-and-passwords-in-htpasswd-file flow.
Split out everything to manage token + RBAC config for tests into chromadb/test/auth/strategies.py. Open to a different home for these utilities.
Simplified the token authn and rbac authz tests.
Wrote a new property test for ServerAuthenticationProvider's new functionality to specify a tenant and database to overwrite those passed by the user. We want to make sure this is done correctly on all current and future API endpoints.

Test plan
How are these changes tested?

[x] Tests pass locally with pytest for python, yarn test for js, cargo test for rust

Documentation Changes
Are all docstrings for user-facing APIs updated if required? Do we need to make documentation changes in the docs repository?

Docs PR: https://github.com/chroma-core/docs/pull/237


Logger

Add a logger with multiple severities (Debug, info, warning, error).
It can add the time, date, PID and thread ID when writing a message.
In the project, the logger is set with a loglevel of info, displays only the time of the message and is linked to the file "log.txt" at the root (see main.cpp).
Close #16

Need #21 validation before.


Better explanation for the "Too many chat channels" message

Checklist

[X] I'm reporting a problem with Chatterino
[X] I've verified that I'm running the most recent nightly build or stable release
[X] I've looked for my problem on the wiki
[X] I've searched the issues and pull requests for similar looking reports

Describe your issue
Whenever I open chatterino, I get the following message on some of the opened channel tabs:
You are connected to too many chat channels. Please close some existing chat connections to join a new chat

This only started happening after the last chatterino update. Before this, there was no issues connecting to as many channels as I want. I have over 100 tabs open in my current setup to keep track of all the channels I frequently visit.
What changed? Did Twitch impose a new API limit affecting this, or is this just client side? How can this be worked around? What even is the limit anyway?
Screenshots
No response
OS and Chatterino Version
Chatterino 2.5.1 (commit 3aead09) built with Qt 6.7.0 Running on Fedora Linux 40 (Forty), kernel: 6.8.9-300.fc40.x86_64

This only started happening after the last chatterino update.

This is a Twitch restriction of 100 channels.

Well that is not optimal. I suppose I'll need to use multiple accounts then. I see chatterino can login with multiple accounts but how is this used? I don't see any option for choosing an account on the tabs I have open, nor when creating new tabs/splits.
Right now most of the channels appear to be using my alt and I still see the channel limit message on other tabs. Is chatterino only able to use one of the accounts at a time? And how is the switching accomplished?

I just thought of something else. Is there a way to make chatterino not join a channel until it's actually live? Even though I have a bunch open, it's usually only a couple dozen concurrent live channels in total. That plus maybe joining offline channels when the tab is selected should be sufficient, I'd think. Of course, that does mean offline chat messages for channels that aren't selected would be missed.

Well that is not optimal. I suppose I'll need to use multiple accounts then.
Right now most of the channels appear to be using my alt and I still see the channel limit message on other tabs. Is chatterino only able to use one of the accounts at a time? And how is the switching accomplished?

We cannot suggest you bypass the rate limits. You can only be connected with one account at a time in a single instance of the program. You can change which account that is with the User button by the tabs.

Is there a way to make chatterino not join a channel until it's actually live?

No we currently do not support anything like that. Channels can have live notifications even if they are not open though.

No we currently do not support anything like that. Channels can have live notifications even if they are not open though.

So if I don't have a tab for a particular channel I can still get a notification in chatterino when they're live? If so I'd like to find out how to get that configured.

So if I don't have a tab for a particular channel I can still get a notification in chatterino when they're live? If so I'd like to find out how to get that configured.

You can just add the channel to your live notifications. Either through the split header menu (... -> More -> Notify when live) or the settings:

Ah ok, I thought so.

Either through the split header menu (... -> More -> Notify when live)

I take it that option adds to the above list? Well that'll do fine then. I'm assuming these will show up in the /live tab then, which works for me. Now to import all these channels.

Thanks for the help, by the way.


reserved identifier violation

I would like to point out that identifiers like “_DLL_HEADER” and “_MOLOCH_LUA_” do eventually not fit to the expected naming convention of the C language standard.
Would you like to adjust your selection for unique names?

Feel free to submit to PRs.


Deploy SWE-agent to cloud

Describe the feature
Since getting SWE-agent up and running locally can be difficult for newcomers and currently does not seem to work on Apple Silicon Macs, it would be nice to have a documented way to run it in the cloud instead, with a (secured) API endpoint exposed for direct (curl style) or programmatic use.
Since the use case for SWE-agent inherently involves connecting to cloud APIs to run model inference anyway, this seems like a natural mode of operation -- if I am running AWS Bedrock models anyway, why not run the agent orchestrator on ECS/Fargate?
Potential Solutions
Since SWE-agent is already containerized, this is probably more a matter of adding a How-To to the documentation than actually adding any new features, but it also helps make the deployment more repeatable for users.

this is smart, on the readme, there could easily be a deploy to github codespaces button , or something similar for "our favorite cloud provider" , or even huggingface.

We would enthusiastically support anything that would make SWE-agent easier for more people to use. Especially if it's just a matter of setting up a guide or an image somewhere.

Since getting SWE-agent up and running locally can be difficult for newcomers and currently does not seem to work on Apple Silicon Macs

@JensRoland Please let us know about the difficulties you run into. It should absolutely works on Apple Silicon Macs. I think our whole team is running on Apple Silicon.

i dont know what @JensRoland has in mind but "codespaces" could be an easy(ier) place to start , happy to pitch in , but there's folks that set up the development environments that will also know how to do it ;-)

I agree that bundling the dependencies and everything in a codespace would be a nice way to help new developers get started :)

@klieret I switched to an M1 Macbook not too long ago and migrated all of my old data and settings from my old Intel Mac. This seems to have left my Homebrew installation in a less-than-ideal state, including things like Python, Pyenv, Docker, and Colima. So when I first tried running SWE-agent, I got errors related to Colima/Docker and tried reinstalling them. That led me to the fun error:
brew cleanup Error: Permission denied @ apply2files

I was stuck on that for a while, until I found someone online pointing out that Homebrew apparently doesn't migrate cleanly from Intel to Apple Silicon, and recommended deleting Homebrew and reinstalling everything with the appropriate architecture flags, so I spent an evening doing that. And it did get Colima running again, but as it turns out.... Homebrew now installs to /usr/local/opt/ rather than /usr/lib and a lot of things were still pointing to the old locations, so things were breaking in interesting new ways. Trying to run SWE-agent gave this one:
dyld[99385]: Library not loaded: /usr/local/opt/gettext/lib/libintl.8.dylib
  Referenced from: <E5AE053B-2D1A-3827-96AC-422BC13EF22B> /Users/jensr/.pyenv/versions/3.9.10/bin/python3.9
  Reason: tried: '/usr/local/opt/gettext/lib/libintl.8.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/opt/gettext/lib/libintl.8.dylib' (no such file), '/usr/local/opt/gettext/lib/libintl.8.dylib' (no such file), '/usr/local/lib/libintl.8.dylib' (no such file), '/usr/lib/libintl.8.dylib' (no such file, not in dyld cache)

Again, some Google-fu took me to a recommendation to simply create a symlink from the non-existing location to the location that holds that file, so I did: bash-3.2# ln -s /opt//homebrew/lib/libintl.8.dylib /usr/local/opt/gettext/lib/libintl.8.dylib
But now trying to run SWE-agent gives me this error:
dyld[5598]: Library not loaded: /usr/local/opt/gettext/lib/libintl.8.dylib
  Referenced from: <E5AE053B-2D1A-3827-96AC-422BC13EF22B> /Users/jensr/.pyenv/versions/3.9.10/bin/python3.9
  Reason: tried: '/usr/local/opt/gettext/lib/libintl.8.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/opt/gettext/lib/libintl.8.dylib' (no such file), '/usr/local/opt/gettext/lib/libintl.8.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/usr/local/lib/libintl.8.dylib' (no such file), '/usr/lib/libintl.8.dylib' (no such file, not in dyld cache), '/opt/homebrew/Cellar/gettext/0.22.5/lib/libintl.8.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/Cellar/gettext/0.22.5/lib/libintl.8.dylib' (no such file), '/opt/homebrew/Cellar/gettext/0.22.5/lib/libintl.8.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/usr/local/lib/libintl.8.dylib' (no such file), '/usr/lib/libintl.8.dylib' (no such file, not in dyld cache)

So I'm stuck with incompatible architecture (have 'arm64', need 'x86_64') and that's where I gave up since it seemed to tell me the container wants x86_64 and I'm on arm64. And with the note in the README about compatibility issues with Arm architecture, I thought that was the end of it. So I'm very surprised to hear that you guys are running on M series Macs yourselves? How? Did you sacrifice a goat to Bjarne Stroustrup?

But back to the original topic: Codespaces seems like a great option, which I would actually prefer to running it locally (even if I could get it to work).

What would it take to get it to work on codespaces?

What would it take to get it to work on codespaces?

just an environment.yaml and a smart little deploy pipeline. you can use the same principle for development environments btw.

@JensRoland Could you open a new issue for that? (just copy it over)

And with the note in the README about compatibility issues with Arm architecture, I thought that was the end of it. So I'm very surprised to hear that you guys are running on M series Macs yourselves?

I think the note in the README is about evaluating on instances/issues from certain repositories (which require the installation of certain dependencies), not for running the software in general.

We welcome contributions that would enable SWE-agent to run on codespaces. We have a release of things we are working on that will probably come out next week, and then this will be one of our primary focus points.

We welcome contributions that would enable SWE-agent to run on codespaces. We have a release of things we are working on that will probably come out next week, and then this will be one of our primary focus points.

Speaking of, I was just lurking through commits on the Web UI branch and it disappeared? What gives? :-)

I've set up a basic version. After you create it you still have to do the following:

Open terminal
Change to root sudo su (else you can't use docker)
conda activate swe-agent

And you should be good to go.
Improvements:

Step 2 should be improved by adding the user to the correct linux group to talk to docker
Step 3 should be already done as a post-init stop in .devcontainer.json  (not sure why it's not already working)

All done. Will add a video/some documentation that automatically opens in the codespace later today :)

That's so awesome, thank you so much!!! Do you think it would make sense to put the video in the README? that could be cool

That's so awesome, thank you so much!!! Do you think it would make sense to put the video in the README? that could be cool

we'll need to make a gif, then host it accordingly, push and update with the static blob address :-)

Completed with #294


[Feature]: Provide the official Milvus Backup image.

Is your feature request related to a problem? Please describe.
We need to use Milvus-Backup in Kubernetes (k8s) cluster.
Describe the solution you'd like.
No response
Describe an alternate solution.
No response
Anything else? (Additional Context)
No response

/assign

https://hub.docker.com/r/milvusdb/milvus-backup/tags

Add installation in README.md?  :-)

@zhuwenxing


Hide Monokle when workspace is not Kubernetes related

Currently, when I open a directory that does not relate to Kubernetes and is not a git repo then I get a subtle error "Monokle: Needs attention" in the status bar at the bottom of VSC.
In my opinion we should completely hide Monokle when the plugin is irrelevant to the end-user. Great plugins are transparent and you don't notice they are there. The only reason for me to get rid of this indicator is to uninstall the plugin - you don't want to give people those kind of idea!

I've been using the extension for a while and believe we should even go a step further as the red cross with "needs attention" and thinking icon when loading are still to present.

Icon colors should be the same as the rest of status bas (white by default; might be defined by theme color?)
Icons on their own are sufficient with good hover text. I would omit text like "needs attention" because text changes cause layout shifts in the status bar which draw unwanted attention.
The default logo should maybe be the Monokle logo?
Thinking icon when loading/working should just be the default logo; it's expected that extensions work in the background and as a user you should not need to be aware of it.

All of this will lead to a much more sober experience which I think will be beneficial. Monokle should be there when you need it and otherwise get out of your way; this is how all good extensions behave.

Looks like custom icons are not supported in status bar (see https://github.com/microsoft/vscode/issues/72244 and https://stackoverflow.com/a/70544972, well, as not part of custom theme).
So either we go with one from default set or just a text. Some proposals:

Large circle

Small circle

Monokle text

M letter

Well, apart from dedicated icon, it's just UTF text so we can use any UTF character there (that's how emojis works now), but I haven't found anything suitable TBH. IMHO Monokle seems to do the job for me, it's simple, doesn't take too much space and you don't have to wonder where is the icon from.

edit: installing Monokle will also cause both Monokle & Sarif to appear in the bar which both do the same when you click it (i.e. open the validation pane). An alternative is to not show Monokle by default and Monokle only appears when something is off?

On this one, the difference is that both icons when clicked have the same action but Monokle one bring additional information when hovered.

Currently, when I open a directory that does not relate to Kubernetes and is not a git repo then I get a subtle error "Monokle: Needs attention" in the status bar at the bottom of VSC.
In my opinion we should completely hide Monokle when the plugin is irrelevant to the end-user. Great plugins are transparent and you don't notice they are there. The only reason for me to get rid of this indicator is to uninstall the plugin - you don't want to give people those kind of idea!

For this I think more static status bar icon does the job, but we also have separate related issue which can improve this further - #38. So even if active folder will not be a git repo it can still be validated with local/default policy.
And going even further, if we want to completely hide Monokle icon, we could detect if project contains any K8s resources. If not just hide all extension related UI. But then it will require a bit smarter approach to restore it when such resources are added. so it's not a quick fix.

We still need an build-in icon for when Monokle needs attention right? What about the following, it's clear this is Monokle and the small-circle is simply there to prevent layout shifts while it has some recognition to our logo.
const icon = needsAttention ? "warning-icon" : "small-circle"
const label = `${icon} Monokle`

I also had a look at Prettier which might be a decent approach? I like the double checkmark wrt validation
https://github.com/kubeshop/vscode-monokle/assets/7761005/e3a2185a-aea6-41f6-8106-d0df9a14d192

Wrt hiding, if its not a quick-fix then let's split that for a separate task and tackle it later. This more sober experience will make that less urgent.

Ok, yes, let's have the icon so we can show invalid status. I went with first proposal = circle + warning icons.
As for hiding, I extracted it to separate issue as it is more broader than just hiding an icon, see #44.


fix unclosed HTML tags

fix unclosed HTML tags

This change is

Certain tags do not need to be closed, and IIRC. @mitsuhiko prefers the "standalone" version of e.g. <p> tags.


Total size of resourceFiles cannot be more than 32768 characters

Hi,
I get this error when I try to run the application in blob store from ADF
Is it possible to keep the zip file in blob and call the application

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a73a1ff2-94e8-a570-ff7b-7a57acca06e6
Version Independent ID: 9ec244a7-d939-3918-fc87-34ab1eec5293
Content: Use custom activities in an Azure Data Factory pipeline
Content Source: articles/data-factory/transform-data-using-dotnet-custom-activity.md
Service: data-factory
GitHub Login: @douglaslMS
Microsoft Alias: douglasl

@nageshsubramanian You may refer MSDN thread which addressin similar issue.
Hope this helps.

@nageshsubramanian We will now proceed to close this thread. If there are further questions regarding this matter, please comment and we will gladly continue the discussion.


Update to GA version of MSAL v4

Hey Brian, do we want to use the graph specific SDK that wraps this?

Are you referring to the Microsoft.Graph.Auth NuGet package?  I think that is what Darrel was demo'ing on previous Microsoft Graph community call correct?  If so it is currently in preview.  I'd prefer to stick with GA of MSAL v3 (3.0.8 at the moment) first.  I already have a first start at updating in my own fork of this repo, feature-msalv3 branch.  Not yet finished but take a look.

Updating code and issue to reflect MSAL .Net v4 as that was released this month.

Scheduling blog post for publish 6/11/2019.  Merged #50 to implement these features.


Lookup pom.xml in current directory by default

First of all, great project!
Exactly the help I need to put my library to Maven Central ❤ !
Now, when I was executing
pomchecker

in the root directory of my project, I'd expect pomchecker to search for the pom.xml in the current directory by default. Wdyt?
Wouldn't mind making the change myself btw.

That would make the --file parameter be optional and look for a pom.xml in the current directory, right?

That would make the --file parameter be optional and look for a pom.xml in the current directory, right?

Exactly!

🎉 This issue has been resolved in v1.10.0 (Release Notes)


No failure message for cop violation

# @example EnforcedStyle: symbol (default)
      #   # bad
      #   create('user')
      #   build "user", username: "NAME"

I believe I am breaking this rule with before { create('project') }
I can see the cop being loaded, and the file where I break this rule is also checked.
bin/rubocop  --debug
configuration from /Users/brent/projects/assessments/confab/.rubocop.yml
configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>Default configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>Inheriting configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>Use parallel by default.
Running parallel inspection

Can you recommend how I can debug this further?
I am using Rails 8.0.0.beta1.

Sorry, what cop?

configuration from<EMAIL_ADDRESS>this one.
before { create('project') }
is this correct according to this gem?

What of the cops you expected to fail? https://docs.rubocop.org/rubocop-factory_bot/cops.html
You’ve already provided an excerpt from the cop’s documentation. May I kindly ask you to also put the cop’s name in the issue title or description?

Can you recommend how I can debug this further?

The above will be a solid first step.
After that, I suggest checking the cop’s spec. Does it look into hooks? Might be not. If not - send a PR with a failing spec. Kindly appreciated if you add specs for both styles.
Next, we can discuss in the PR how to actually fix the issue.
At a glance, I can’t tell why it doesn’t work.
Also, the “Explicit” option is confusing.

@brentgreeff  Thank you for your report. Please re-open this PR and let us know when you have more information. Until then, we will close the PR once.


Support numpy ufuncs with array_ufunc

NumPy 1.13.0 supports dispatching on numpy ufuncs with the __array_ufunc__ protocol.  This should enable us to make it so that np.sin(my_dask_array) is the same as da.sin(my_dask_array).  A release candidate for 1.13.0 is out now.  I suspect that the numpy community would appreciate it if we tested this prior to their full release.
cc @shoyer @jcrist

As a template, I would suggest the implementation of __array_ufunc__ from np.lib.mixins.NDArrayOperatorsMixin:
https://github.com/numpy/numpy/blob/8ea619a8717d2c82d42e4662b2266121faba58e4/numpy/lib/mixins.py#L84
This should work sensibly even without inheriting from the base class.
In dask's case, it is probably best to unilaterally return NotImplemented (for now) if method != '__call__' or ufunc.signature is not None (indicates a generalized ufunc), and otherwise implement the operation with da.elemwise.

This is implemented.


Not always changing hash in files

Hi,
Sometimes the paths inside the files I've configured are not updating the hash.
Here is an example of one line inside an PHP file:
<script src="<?php echo __URL__; ?>/assets/js/scripts.min.1ba47e1224542500.js"></script>
This problem occurs after I've deleted the assets/js/scripts.min.1ba47e1224542500.js file and executed the grunt task again. I've deleted it because sometimes (still haven't figured it out why) it creates more than one version of the file. To make it work again, I have to rename the filepath to <?php echo __URL__; ?>/assets/js/scripts.min.js inside the PHP file and then run the grunt task again.
I'm using version 1.1.0  and this is my Gruntfile.js config:
        cacheBust: {
            taskName: {
                options: {
                    deleteOriginals: true,
                    assets: [
                        'assets/stylesheets/*.min.css',
                        'assets/js/*.min.js',
                    ]
                },
                src: [
                    'header.php',
                    'footer.php',
                    'index.php',
                    'example.php',
                ]
            }
        },

And my Grunt task:
grunt.registerTask('js', ['uglify', 'cacheBust']);
Any idea why this is happening?

When I run cacheBust with grunt watch and then again with grunt js, or vice versa,  he is creating multiple files.

Hi @kossmann - sounds like you've run into a similar use cases as in #180. Have a read through the comments on there, especially the one about creating a clean copy of the code before running the task

Hi @hollandben
I have similar issue. The problem is hash remain same on every time. If I run grunt task again and again but file name remain same. I use grunt-contrib-clean for cleaning publish directory first on each grunt build.
cacheBust: {
      options: {
        baseDir: 'public',
        deleteOriginals: true,
        jsonOutput: true
      },
      allAssets: {
        options: {
          assets: [
            'publisher/*.js',
            'publisher/*.css',
	    'admin/*.js',
            'admin/*.css',
	  ],
          jsonOutputFilename: 'asset-fingerprints.json'
        },
        files: [{
          expand: true,
          cwd: 'public',
          src: [
            'publisher/*.html',
            'admin/*.html'
          ]
        }]
      }
    }

Is there any option for new hash on every bust?

Is there any option for new hash on every bust?

That's not the purpose of content hashing. The idea is that the files are hashed, giving you a unique string for the contents of that file. If you change 1 character in the file, then the hash will change. If you run the tasks 10 times without changing anything, then you'll get the same result everytime.

Thanks. Really appreciate your quick and detailed answer.
I was trying to run task without changing content that’s why it was not
updating.
On Tue, 12 Sep 2017 at 1:42 PM, Ben Holland<EMAIL_ADDRESS>wrote:

Is there any option for new hash on every bust?
That's not the purpose of content hashing. The idea is that the files are
hashed, giving you a unique string for the contents of that file. If you
change 1 character in the file, then the hash will change. If you run the
tasks 10 times without changing anything, then you'll get the same result
everytime.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/hollandben/grunt-cache-bust/issues/183#issuecomment-328776635,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIHzCykhFDJMVxUL7T2t-A7dh3TGRnktks5shjz6gaJpZM4Hf5nz
.

--
Thank you,
Vijay P
I help you to build reliable and unique applications.CONFIDENTIALITY
CAUTION - This page and any accompanying files are intended solely for the
named addressee, are confidential and may contain legally privileged
information. If you have received this email in error, please desist from
reading the contents and let us know by telephone or return the email to
the sender and destroy all copies. Thank you.

@hollandben after update files references with the hashed url, this plugin recalculates the hash of the modified file again?
The process is this:
read files → generate hash → replace filenames → find and replace references
or this:
read files → generate hash → replace filenames → find and replace references → repeat all over until nothing changes
In case of the first (which I believe it is), I can work in the solution.
Thank you!


Allow the override of the preconfigure command

What this PR does / why we need it:
When deploying gocd in a kubernetes cluster, we would like to preconfigure the server
with custom settings. This can be done by a script shell or by a python script.
Before we could only specify if we want the server to be preconfigured but
there was no way to override the configuration script.
With this change, one can write a script, put it in a configmap, mount this configmap
on the pod and override the preconfigure command using 'preconfigureCommand'.
For example :
preconfigureCommand:

"/usr/bin/python"
"/preconfigure.py"

Which issue this PR fixes
None
Special notes for your reviewer:
@arvindsv @bdpiparva @GaneshSPatil @varshavaradarajan @maheshp : let me know if this change suites you
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md

Signed-off-by: Fatih KARAKAS<EMAIL_ADDRESS>

/ok-to-test

This looks alright to me. @ankitsri11 or @sheroy - Can you take a look and try it out?

Hi @fakarakas,
With this change, one can write a script, put it in a configmap
How do you plan to create this new configmap?

Ok, so the expectation is that the configmap exists before the chart is installed, right?

One part I'd be worried about is, how much of the existing preconfigure command is necessary.

I'm going to approve this for a merge.
@sheroy @ankitsri11 and I talked. The existing preconfigure script will always be mounted, as soon as shouldPreconfigure is set. If it is not set, then no preconfigure scripts will be run anyway.
It is recommended that users overriding server.preconfigureCommand do call /preconfigure_server.sh before or after their own scripts, since /preconfigure_server.sh sets up the plugin and an elastic profile, etc.
So, consider something like: ["/bin/bash", "-c", "/preconfigure_server.sh && /my-own-script.sh"] for the value of server.preconfigureCommand. Unless you really know what you're doing.

/lgtm

Thank you, @fakarakas!

Your are welcome and thank all.
@arvindsv, as you said :

Unless you really know what you're doing.

For the preconfigure script, we wrote it in python and reused the configuration idea of the official preconfigure.sh script provided in the chart.
If you are interested this is the script : https://gist.github.com/fakarakas/1392ebe01b3b5c2ff4f75b28b81cb3ef
Also notice that this script is an upsert style, so it tries to get the resource and if it exists make an update else a create api request. It is done in python because in bash it is more complicated. This is helpfull because sometimes the gocd-server restarts (error or need to change the configuration) and the script is executed.

@fkarakas - Makes sense. You're definitely in the "really know what you're doing" camp. :)


GEODE-8694: use redis tests with patch

use the redis/redis repo's test with a patch containing changes to tests
that fail on unimplemented commands when testing geode-redis.
Thank you for submitting a contribution to Apache Geode.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

[x] Has your PR been rebased against the latest commit within the target branch (typically develop)?

[x] Is your initial contribution a single, squashed commit?

[x] Does gradlew build run cleanly?

[x] Have you written or updated unit tests to verify your changes?

[x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

Note:
Please ensure that once the PR is submitted, check Concourse for build issues and
submit an update to your PR as soon as possible. If you need help, please send an
email to<EMAIL_ADDRESS>

the failure in the Redis PR check is probably because the PR pipeline picked up only part of your change (the port number in the rule) but not the other half (this is intentional for security reasons).  Once merged all should be well.

the failure in the Redis PR check is probably because the PR pipeline picked up only part of your change (the port number in the rule) but not the other half (this is intentional for security reasons). Once merged all should be well.

Thanks for the context, Owen!

thanks @sabbey37 for all your suggested changes! I forgot to check what other changes I was committing. oops


_pickle.PicklingError: Can't pickle <function mytest at 0x000001FACD58FC80>: it's not the same object as main.mytest

Using the example given:
import time
import timeout_decorator

@timeout_decorator.timeout(5, use_signals=False)
def mytest():
    print("Start")
    for i in range(1, 10):
        time.sleep(1)
        print("%d seconds have passed" % i)

if __name__ == '__main__':
    mytest()

I receive:
Traceback (most recent call last):
  File "C:/Users/Will/PycharmProjects/tests/timeout.py", line 15, in <module>
    mytest()
  File "C:\Users\Will\AppData\Local\Programs\Python\Python36\lib\site-packages\timeout_decorator\timeout_decorator.py", line 91, in new_function
    return timeout_wrapper(*args, **kwargs)
  File "C:\Users\Will\AppData\Local\Programs\Python\Python36\lib\site-packages\timeout_decorator\timeout_decorator.py", line 146, in __call__
    self.__process.start()
  File "C:\Users\Will\AppData\Local\Programs\Python\Python36\lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "C:\Users\Will\AppData\Local\Programs\Python\Python36\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\Will\AppData\Local\Programs\Python\Python36\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "C:\Users\Will\AppData\Local\Programs\Python\Python36\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\Will\AppData\Local\Programs\Python\Python36\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function mytest at 0x000001FACD58FC80>: it's not the same object as __main__.mytest

System info:
Windows 10
Python 3.6.3
timeout-decorator 0.4.0

There's a working solution in #37 as a temporary work around for anyone else who finds this.

You can use my fork - https://github.com/bitranox/wrapt-timeout-decorator
there it works fine, I just tested ...
Yours sincerely
Robert


Add examples

Idea: Translate "example" folder of raylib from C to Lua

[x] core (done partially)
[x] shapes (done partially)
[x] textures (done partially)
[x] text (done partially)
[ ] models
[ ] audio
[ ] shaders
[ ] physics

References:

https://github.com/raysan5/raylib-lua/tree/master/examples
https://github.com/RobLoach/raylib-lua-sol/tree/master/examples

Status:
A few example files were translated to Lua, which (imo) would be enough for demonstration.
(Reopen this issue if we need more examples?)


Update prefetch docs with browser support information

Description (required)
Update the prefetch docs based on the core PR linked below, which includes some internal behaviour changes.
I also added a new browser support section to explain the behaviour difference between browsers, and how to ensure prefetching works reliably.
Related issues & labels (optional)
For next Astro patch. See astro PR https://github.com/withastro/astro/pull/10999 (not a minor)

Lunaria Status Overview
🌕 This pull request will trigger status changes.

Learn more
By default, every PR changing files present in the Lunaria configuration's files property will be considered and trigger status changes accordingly.
You can change this by adding one of the keywords present in the ignoreKeywords property in your Lunaria configuration file in the PR's title (ignoring all files) or by including a tracker directive in the merged commit's description.

Tracked Files

Locale
File
Note

en
guides/prefetch.mdx
Source changed, localizations will be marked as outdated.

Warnings reference

Icon
Description

🔄
The source for this localization has been updated since the creation of this pull request, make sure all changes in the source have been applied.

Thanks! I like your suggested formatting too. I committed them and view locally and it looks like this:

Screenshot (Vercel link should also be possible)

Seeing this, maybe we could remove the Support: prefix on each paragraphs? Since the heading "Browser support" already implies so.

Yup, agreed! I had it in my head that maybe the different original list items were going to be different, repeatable "things"... but nope, I think without will be just fine!

Thanks! I've updated and removed them now.

@sarah11918 the core PR has been released. Waiting on your approval just in case and I think we can then merge this

@bluwy Thanks, taking one more final pass now, then I'll merge!


Adding regionality to main page

Adds regionality as a new column on the main page:

For https://github.com/usgs-makerspace/makerspace-sandbox/issues/20

This should go year-month-week for sessions

Sorry that I was unable to review! Looks like you and Laura got it 👍


Reset / Revert Image

After panning / zooming, changing WW/WC and placing annotations on the image, what should be called to revert the image to its initial state?

call displayImage() to reset it with an undefined viewport

Is this a new feature (added in recent commits or should this be working in old code)? It does not appear to work using the older version of Cornerstone.
This works slightly better although there are still some quirks:
var defViewport = cornerstone.getDefaultViewport(element, image);
cornerstone.displayImage(element, image, defViewport);

I know this is closed but I was looking for this and ended up using
cornerstone.reset(element);


[WIP] feat: add output directory option

Hi,
Thanks for putting this module together. It's been very useful.
I wanted to make a few additions to cover my specific use-cases. I had posted two other requests earlier, but I later realized that I was working off the npm code, which is not up-to-date, so I started again using the current version 1.3.0.
Options functionality has been added to the .write method. Two options are currently available. The output_dir option covers the requirement in Issue #44.
You can see the readme for more information.
Thanks.

Hey @obihill would be grateful for contributions 🙌, mind potentially splitting this into a few pull requests with descriptions of one distinct thing that is being added or fixed in each one 👌

Hi @bcoe. 'Not sure how I would simplify it further. If you look at the diff breakdown, the updates seem pretty straightforward.
Alternatively, you could actually just take the code [and rework it], then we could close the pull request.
Basically, all I've done is add optionality to pti.write, then added two options[backup and output_dir] to enable timestamped backups and output to a custom directory respectively.

@obihill I think if we could keep this PR centered to just the output_dir feature:

describing how you set the option in the README.
adding a couple tests for the feature.

I'd be really happy to land this. If you don't have the time, understandably, I would be happy to rework -- but if you do want to pick up this PR, I'm more than happy to provide feedback.

Hi @bcoe, any chance in having this cherry-picked instead of waiting for the author?

@mvegter yeah, I would be happy to take a patch that uses @obihill's work as a starting point.
@obihill I'm sorry that I didn't see this over the finish line, I appreciate the work you put in (it's a great start).

@bcoe My apologies for the delayed response to this.
Regarding the options info on the readme, I have something penned on the forked repo here where I've implemented this feature already, also to an npm package.
As I said earlier, I'm ok with you using this as your guide to implement directly i.e. outside this pull request. If you look at the diff, you'll see the changes are not major.

@obihill I just want to make sure you get credit for your contribution 👍
Perhaps you could squash your work on this branch into one commit, and then I can just submit a patch against it?

@bcoe No, it's fine, no credit sought. Besides, I just made a small tweak.
The substantive commit is this one:
9ee8a49. You can disregard the others. I'll see if I can delete them.

Hi @obihill , I cherry-picked your changes as we also had them in our own fork. Perhaps your backup changes can be proposed to and it all will be in a single package.

Hi @mvegter. Sure, I don't mind. Can you guide me on how to make that happen? Alternatively, the substantive code for the backup feature is line 98 to 112 in lib/output-files.js, with requirement of lib/utils.js for generating the datetime code i.e. if you wanted to cherry-pick.


Call out libtinfo-dev dependency

Just noticed this depends on libtinfo-dev, which you may want to call out in the README or provide an install script to encompass

Interesting. I'm not sure where this dependency comes from, did it happen to say?

Sorry, I went to fetch you the error that led me to that lib and my laptop had crashed since the output. It was on a stack install, and I figured since this is a project you're building on your native OS, you might already have had it installed and not been aware of it. It occurs to me that maybe it installed a GHC version though, perhaps that is the source.


version up to 8.0.4

@MiroshnikovAndrey thanks for the patch and sorry I missed it! I updated the code to 8.0.5. Cheers!


Barba Views and Promise - delay Transition till page content loaded?

I have a custom Barba View for certain pages, which loads a google map in the onEnter function. This works great, but Im also using a Fade Transition and the Transition is starting before the google map has loaded, though I would prefer it to wait till the map is loaded.
I assume this would be achieved in the FadeTransitions start function using the promises, but Im not sure how to go about inserting a new promise. Im guessing I would add it here
start: function() {
        Promise
        .all([this.newContainerLoading, this.fadeOut()])
        .then(this.fadeIn.bind(this));
    },

Most likely as an entry in the .all([...]) promise array.
Another thing Im unsure of is, for most pages that dont use the custom Barba View, I dont need to wait for a google map to load, the default behaviour is fine, so if I add a new promise to the array, how will I ensure that normal pages dont get stuck waiting forever since that new promise wont get fulfilled?

Hi there @matthewjumpsoffbuildings ,
there are different solution here.
The most simple and logic to me is to leave the Transition as it is now, without taking care of the google map.
Then in the View with the google maps you can fadein when that is ready.
Would this suits to you?
I'm also planning in the v2 to be able to get the instance context of the new and old View during the Transition, so it will be easier to make into account the loading of gmap during the transition

That is one solution, but its not ideal, since the page transition is not going to be a simple fade when its completed - page transitions will be slides... so Id prefer the transition to wait till the google map is loaded before sliding in the new page.
Having access to the instance of the new and old View would be helpful for sure. Would you expose some kind of Promise from the View that could be accessed in the Transition?

Hi matthewjumpsoffbuildings,
Is there any chance you could post your google maps code as an example so I could see how you've got it working so far? I'm a bit of a novice with the js side of things...
Before trying to port my code over to barba, I was loading my maps code using window.onload(), however it's now not working obviously as it's being ajaxed in. If you could point out anything I could try instead of window.onload() it would be appreciated. My code so far is below:

`var Services = Barba.BaseView.extend({
  namespace: 'services',
  onEnter: function() {
      // The new Container is ready and attached to the DOM.

  console.log('services loaded');

        window.onload = function () {

            var latlng = new google.maps.LatLng(-37.81764265020812,<PHONE_NUMBER>);

            var styles = [
                {
                    featureType: "landscape",
                    stylers: [
                        { color: '#eeddee' },
                        { saturation: -100 }
                    ]
                },{
                    featureType: "natural",
                    stylers: [
                        { hue: '#ff0000' },
                        { saturation: -100 }
                    ]
                },{
                    featureType: "road",
                    stylers: [
                        { hue: '#5500aa' },
                        { saturation: -100 }
                    ]
                },{
                    featureType: "building",
                    elementType: "labels",
                    stylers: [
                        { hue: '#000066' },
                        { saturation: -100 }
                    ]
                },{
                    featureType: "poi", //points of interest
                    stylers: [
                        { hue: '#0044ff' },
                        { saturation: -100 }
                    ]
                }
            ];

            var isTouchDevice = !('ontouchstart' in document.documentElement);

            var myOptions = {
                zoom: 14,
                center: latlng,
                mapTypeId: google.maps.MapTypeId.ROADMAP,
                disableDefaultUI: true,
                zoomControl: true,
                styles: styles,
                scrollwheel:  false,
                draggable: isTouchDevice
            };

            map = new google.maps.Map(document.getElementById('map'), myOptions);

            var marker = new google.maps.Marker({
                position: latlng,
                title: "Brow 101"
            });

            marker.setMap(map);

        }

  },
  onEnterCompleted: function() {
      // The Transition has just finished.
  },
  onLeave: function() {
      // A new Transition toward a new page has just started.
  },
  onLeaveCompleted: function() {
      // The Container has just been removed from the DOM.
  }
});

// Don't forget to init the view!
Services.init();`

Can you edit that comment and put all the code in a github gist or pastebin or something? I dont mind taking a look, but its not really relevant to the topic of this issue, this is about the Barba View/Transition/Promise api

I am looking for the same thing here. The page slows down when init heavy js code together with the transition animation. And sometime the animation finish before the js code finish init, which has element displacement for example carousel after loading especially for low computing power device and slow network. It would be ideal if the Transition can listen to Promise inside View in someway. Thanks.


Added pil() method

Provides PIL output for patterns:

Thanks to Philipp and Jean-Luc for their help!

Looks good and the pr build is passing. Merging.


Check that attributes exist before adding them to the count

In this PR we add a check on the attributes that are taken into account for the purposes of counting tests, assertions, failures, errors and time.
Some XML reports in jUnit format may not always have these attributes, for example the report file generated by PHP_CodeSniffer.
This can cause warning messages like this:
PHP Warning: A non-numeric value encountered in /vendor/andreas-weber/php-junit-merge/src/Console/Command.php on line 221

This is actually very useful.
When is this planned to be merged?

Hello @andreas-weber.
Can you please take a look at this?
It's very useful as it's breaking our build system for a very trivial error.

@andreas-weber Thank you!
Please also bump the version, otherwise nobody that is already using the library can update 😉

@albertodimaio3 I've merged your changes.
@enricobono I've created a new release.
Thank you for your work! 🥇

Thank you so much, much appreciated 😊


Vulkan: loader, headers, and glslang updated to sdk-<IP_ADDRESS>

Updated glslang and Vulkan headers/loader following the instructions found in thirdparty/README.md.
glslang was updated to the 'known good' matching Vulkan SDK version <IP_ADDRESS>.
Vulkan headers and loader were updated to the commit tagged for sdk-<IP_ADDRESS>.
'vk_mem_alloc.h' and 'vk_mem_alloc.c' are unchanged since there hasn't been a new tagged release since 2.3.0.
The fix-mingw-snprintf patch in glslang was not applied in the previous update (#42817).  Assuming this was a mistake, I re-applied the patch.
Here are the relevant Vulkan SDK release notes:
https://vulkan.lunarg.com/doc/sdk/<IP_ADDRESS>/windows/release_notes.html

I just noticed the typo in my branch name (126 should be 162).  I don't want to make it look like I'm spamming with an identical PR so let me know and I'll open a new one with the correctly named branch: jacobcoughenour:vulkan-sdk-<IP_ADDRESS>

I just noticed the typo in my branch name (126 should be 162). I don't want to make it look like I'm spamming with an identical PR so let me know and I'll open a new one with the correctly named branch: jacobcoughenour:vulkan-sdk-<IP_ADDRESS>

It's fine, the branch name doesn't matter much as long as the commit itself is correct.

The fix-mingw-snprintf patch in glslang was not applied in the previous update (#42817). Assuming this was a mistake, I re-applied the patch.

@bruvzg Do you know if this patch is still relevant? We do enable MINGW_HAS_SECURE_API, but do we sometimes compile without actually having support for those? Or should we try to get a fix upstream?

@fire Maybe you can give this a try before we merge as your MinGW pipeline often breaks on Vulkan SDK updates.

The fix-mingw-snprintf patch in glslang was not applied in the previous update (#42817).

but do we sometimes compile without actually having support for those?

No, it's supported on all Windows newer than XP. Original issue is probably fixed in upstream by https://github.com/KhronosGroup/glslang/pull/2333, at least I did not encounter any errors with the current master.

@bruvzg Thanks! @jacobcoughenour You can undo the fix-mingw-snprintf patch changes then and remove the patch file (and its mention in thirdparty/README.md.

It's my vacation. So things are slow. I was pinged. I'll see what I can do.

Thanks! And congrats for your first merged Godot contribution :tada:


KLEE crash with LLVM 8 in klee::Executor::getKValueInfo(klee::ExecutionState&, klee::KValue const&) const

System: Ubuntu 18.04 (ubuntu:bionic container)
LLVM & Clang: 8
Build type: RelWithDebInfo (Debug is does not crash though)
Compiler: clang-8
This crash can be seen in many test cases in the new CI draft, for instance in simple_true-valid-memsafety_false-valid-memcleanup.c.
Output with ASAN & UBSAN enabled:
$ symbiotic --prp=memcleanup tests/simple_true-valid-memsafety_false-valid-memcleanup.c
8.0.0-pre-llvm-8.0.0-symbiotic:94e53f0e-dg:06c8ca8c-sbt-slicer:b0864a5b-sbt-instrumentation:c34d55f2-klee:10e5fae9
INFO: Looking for unfreed memory
INFO: Starting instrumentation
INFO: Instrumentation time: 0.08374357223510742
INFO: Optimizations time: 0.02287125587463379
Linked our definitions to these undefined functions:
  __VERIFIER_exit
INFO: Starting slicing
INFO: Total slicing time: 0.021975278854370117
INFO: Optimizations time: 0.012139558792114258
INFO: After-slicing optimizations and transformations time: 1.0967254638671875e-05
INFO: Starting verification
/symbiotic/klee/lib/Core/Executor.cpp:2977:28: runtime error: downcast of address 0x604000018bd0 which does not point to an object of type 'klee::KGEPInstruction'
KLEE: WARNING ONCE: Alignment of memory from call "malloc" is not modelled. Using alignment of 8.
/symbiotic/klee/lib/Core/Memory.h:167:25: runtime error: index 18 out of bounds for type 'const char [10]'
==27187==ERROR: AddressSanitizer: global-buffer-overflow on address 0x000000d15192 at pc 0x0000005ea60f bp 0x7ffccc75f230 sp 0x7ffccc75e9d8
KLEE: output directory is "/tmp/symbiotic-odl2y93e/klee-out-0"
KLEE: Using Z3 solver backend
/symbiotic/klee/lib/Core/Executor.cpp:2977:28: runtime error: downcast of address 0x604000018bd0 which does not point to an object of type 'klee::KGEPInstruction'
0x604000018bd0: note: object is of type 'klee::KInstruction'
 0b 00 00 53  d0 cb e6 00 00 00 00 00  38 30 00 00 80 60 00 00  d0 71 01 00 30 60 00 00  b0 6e 02 00
              ^~~~~~~~~~~~~~~~~~~~~~~
              vptr for 'klee::KInstruction'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /symbiotic/klee/lib/Core/Executor.cpp:2977:28 in
KLEE: WARNING ONCE: Alignment of memory from call "malloc" is not modelled. Using alignment of 8.
/symbiotic/klee/lib/Core/Memory.h:167:25: runtime error: index 18 out of bounds for type 'const char [10]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /symbiotic/klee/lib/Core/Memory.h:167:25 in
=================================================================
==27187==ERROR: AddressSanitizer: global-buffer-overflow on address 0x000000d15192 at pc 0x0000005ea60f bp 0x7ffccc75f230 sp 0x7ffccc75e9d8
READ of size 1 at 0x000000d15192 thread T0
    #0 0x5ea60e in strlen (/symbiotic/install/llvm-8.0.0/bin/klee+0x5ea60e)
    #1 0x7ff1f5202406 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&) (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0x127406)
    #2 0x7b54f1 in klee::MemoryObject::getSegmentString[abi:cxx11]() const /symbiotic/klee/lib/Core/Memory.h:167:12
    #3 0x768d75 in klee::Executor::getKValueInfo[abi:cxx11](klee::ExecutionState&, klee::KValue const&) const /symbiotic/klee/lib/Core/Executor.cpp:3187:33
    #4 0x75cb86 in klee::Executor::terminateStateOnExit(klee::ExecutionState&) /symbiotic/klee/lib/Core/Executor.cpp:3430:15
    #5 0x8c841e in klee::SpecialFunctionHandler::handle(klee::ExecutionState&, llvm::Function*, klee::KInstruction*, std::vector<klee::Cell, std::allocator<klee::Cell> > const&) /symbiotic/klee/lib/Core/SpecialFunctionHandler.cpp:257:7
    #6 0x739583 in klee::Executor::callExternalFunction(klee::ExecutionState&, klee::KInstruction*, llvm::Function*, std::vector<klee::Cell, std::allocator<klee::Cell> > const&) /symbiotic/klee/lib/Core/Executor.cpp:3691:31
    #7 0x736652 in klee::Executor::executeCall(klee::ExecutionState&, klee::KInstruction*, llvm::Function*, std::vector<klee::Cell, std::allocator<klee::Cell> > const&) /symbiotic/klee/lib/Core/Executor.cpp:1507:7
    #8 0x754bb8 in klee::Executor::executeInstruction(klee::ExecutionState&, klee::KInstruction*) /symbiotic/klee/lib/Core/Executor.cpp:2145:7
    #9 0x763c15 in klee::Executor::run(klee::ExecutionState&) /symbiotic/klee/lib/Core/Executor.cpp:3139:5
    #10 0x77bb34 in klee::Executor::runFunctionAsMain(llvm::Function*, int, char**, char**) /symbiotic/klee/lib/Core/Executor.cpp:4498:3
    #11 0x6cd1df in main /symbiotic/klee/tools/klee/main.cpp:1808:18
    #12 0x7ff1f412abf6 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21bf6)
    #13 0x5d6fc9 in _start (/symbiotic/install/llvm-8.0.0/bin/klee+0x5d6fc9)

0x000000d15192 is located 46 bytes to the left of global variable '<string literal>' defined in '/symbiotic/klee/lib/Core/Memory.h:173:14' (0xd151c0) of size 9
  '<string literal>' is ascii string 'symbolic'
0x000000d15192 is located 8 bytes to the right of global variable '<string literal>' defined in '/symbiotic/klee/lib/Core/Memory.h:167:25' (0xd15180) of size 10
  '<string literal>' is ascii string 'Segment: '
SUMMARY: AddressSanitizer: global-buffer-overflow (/symbiotic/install/llvm-8.0.0/bin/klee+0x5ea60e) in strlen
Shadow bytes around the buggy address:
  0x00008019a9e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x00008019a9f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x00008019aa00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x00008019aa10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x00008019aa20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x00008019aa30: 00 02[f9]f9 f9 f9 f9 f9 00 01 f9 f9 f9 f9 f9 f9
  0x00008019aa40: 04 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9
  0x00008019aa50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x00008019aa60: 00 02 f9 f9 f9 f9 f9 f9 00 00 00 00 00 00 00 00
  0x00008019aa70: 00 00 00 00 00 00 00 00 00 00 00 07 f9 f9 f9 f9
  0x00008019aa80: 00 00 00 00 00 02 f9 f9 f9 f9 f9 f9 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==27187==ABORTING
klee answered ERROR (KLEE exited with 1)
INFO: Verification time: 0.15619134902954102
Failure!
NOTE: In the future, the result is going to be reported in SV-COMP format only with --report=sv-comp switch
RESULT: ERROR (KLEE exited with 1)
INFO: Total time elapsed: 0.6952133178710938

Blocks: #196
edit: add more precise system information

Note, that this issue might be present in upstream KLEE as well.

Nope, upstream KLEE does not use segments in pointer representation. This was solely our bug.


add Monte Carlo method

Hello @mikkokotila! Thanks for opening this PR.


docs: improving headings

Leaving out title case and some formatting from headings
Signed-off-by: Pekka Niskanen<EMAIL_ADDRESS>

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Update UseAuthentication middleware to support sessions

Description
The UseAuthentication middleware currently verifies that the JWT token is valid. To support session management, this needs to be updated to instead verify that the session ID present in the cookie matches a session ID present in the database. If said session has expired, the middleware should abort the request with a 401 error.
For details on how this should be modified, view the example sequence diagram.
Approach

Extract the session ID from the cookie
Validate the session exists in the database
Validate the session has not expired

Blocks
Blocked By

#272
#273

Related To

This won't be backwards compatible with the proposed changes.


run bam QC metrics in parallel with detect variants

once the bam is created, we don't have to wait for the QC metrics to be generated before starting detect-variants. Splitting this apart will reduce wallclock time

this is already being done - my mistake -  closing


MY simulation

First of all, thank you so much for this great application!
We have a total of 22 RTS drives, two of which are shutters that cannot set a MY position, and this is very annoying.
I was thinking that it would be relatively easy to implement a "pseudo-MY" function in ESPSomfyRTS for such drives by assigning a specific target position to the shutter as the MY position and simulating the MY behavior based on this value.
This could even be used to retrofit awnings or blinds with a MY position.
(Of course, something similar could also be realized individually in a higher level system like HA (or in my case Loxone) using the Target command, but this would require repeated additional efforts...)
By using Somfy, you're just used to their genious "MY comfort"... ;)

I have added this function in the v2.4.3 pre-release.  There is now a checkbox on the shade definition for a simulated favorite.

@rgr101 I have added this functionality to v2.4.3 and will be releasing it this weeken.

Just installed - amazing !!!! This works really great !!! 👍🙏🙏🙏

EDIT: Because you didn’t ask it will allow you to set the my position using the remote as well by holding down the my button.

EDIT: Because you didn’t ask it will allow you to set the my position using the remote as well by holding down the my button.

Indeed!!

EDIT: Because you didn’t ask it will allow you to set the my position using the remote as well by holding down the my button.

...could the position be remote-dependant?

...could the position be remote-dependant?

Because you asked. Sure!


Fix IRS spectra links

Fixes the broken URLs for the IRS spectra.
Link to relevant issue: Closes #347

I'd like to merge #404 first.


Shell script for connecting to database.

Created shell script for connecting to the database.
This automates the connection process, saving time and cutting out human error.

Where is the tools/ folder?