Episode about coroutines: Application in Real world code

Channel
C++Weekly
Topics
Coroutines: Application in real world code and what you can gain from them.
Length
I guess as this topic is complicated, its a longer episode. 20+ mins.
CppCon 2016: Gor Nishanov “C++ Coroutines: Under the covers" [3] shows some of the issues but none of the talks i searched and looked on YT seem to cover the statemachine like shown in the Kolb Report [1] Page 10. Bjarne Stroustroup wrote or said somewhere that he used them back in the days for simulations. (I cant find the reference.) Others wrote that you can use them for e.g. AI in games. Gor Nishanov had a talk [2][4] on CppCon and said: They are negative overhead abstractions as he used the prefetch assembly instruction to fetch data into the cache and then resume the function. So these people all suggest that they have great potential.
They are a C++20 feature and soon is C++23 out or already feature frozen.
When i think about them and how to apply them to my code (Statemachines in Realtime applications) i tend to always think about actors with an queue/ring-buffer with a variant and then an attached statemachine. When the statemachines get more complicated it leads to something that you just feed a Boost.MSM or Boot-ext.SML statemachine. So: What can i gain from them in such environments? How to write complex statemachines without the need for these libraries? How can you write complex statemachines with them? WIth complex statemachines i think about ~30-50 states.
What are the situations when they really shine and what situations do you better avoid them.
I also got them working on an embedded target (STM32F103RBT6 / Olimexino-STM32 Board). There you need a custom operator new/delete in the promise type and let it allocate the memory from a static pool. Somehow related to the memory pool that you "uncovered" in the analysis of the C&C Code with the units, [5]
So maybe this could be an interesting Episode :)
[1] https://wiki.tum.de/download/attachments/93291100/Kolb report - Coroutines in C%2B%2B20.pdf?version=1&modificationDate=1532345204090&api=v2
[2] CppCon 2015: Gor Nishanov “C++ Coroutines - a negative overhead abstraction"
[3] https://www.youtube.com/watch?v=8C8NnE1Dg4A&t=278s
[4] https://www.youtube.com/watch?v=j9tlJAqMV7U&t=2431s
[5] https://youtu.be/Oee7gje-XRc?t=3083

I would like to hear that as well!

double dispatched std::visit with overloads is amazing, but if you get beyond 5 different types in each variant you get some serious compile time explosion to worry about

@lefticus I'm lucky I'm just at about 5 types now haha. But I'll keep that in mind. It is most certainly not the most optimal state machine, because the reentry of std::visit costs one unnecessery runtime branch. I was reflecting on using label pointers for this which I come to like more and more. Maybe you could also make a video on them sometimes :>

I feel like this would be best as a [fill in the blank] episode, so we can talk about the generic application of coroutines. Hopefully with a language that has better support for them

What makes C++ Coroutines so challenging at the moment are the 20+ customization points. They make it very expert friendly.

As i cant stop thinking about the coroutines i went to the history of them. M.E. Conway described them here
https://www.melconway.com/Home/pdf/compiler.pdf
He wrote in this paper:

A program
organization is separable if it is broken up into processing
modules which communicate with each other according to
the following restrictions: (I) the only communication
between modules is in the form of discrete items of information;
(2) the flow of each of these items is along
fixed, one-way paths; (3) the entire program can be laid
out so that the input is at the left extreme, the output is at
the right extreme, and e'erywhere in between all information
items flowing between modules have a component of
motion to the right.
Under these conditions each module may be made into
a coroutine; that is, it may be coded as an autonomous program
which communicates with adjacent modules as if
they were input or output subroutines.

My interpretation is, that they are generators which feed other generators until the output is produced.
On Page 2 he shows how he connected 2 subroutines: Write data to tape, rewind it and feed the input to B. This is the essence of a ringbuffer/queue.
                          |
                   +---+  |  +---+
                   | A +--|->| B |
                   +---+  |  +---+
                          |

                       Tape

+---+   +----------+   +---+     +----------+    +---+
|   |   |Tape      |   |+-+|     |Tape      |    |   |
| A |+->|Writing   +-->|| ||+--->|Reading   |+-->| B |
|   |   |Subroutine|   |+-+++    |Subroutine|    |   |
+---+   +----------+   +---++    +----------+    +---+

When i apply this thinking to event driven applications, the data are the events. The statemachines are the coroutines that transform the data into another representation The actions that a statemachine does are outputs.

Here i got a sample which is like the description of Conway (in my point of view)
#include <iostream>
#include <variant>
#include <array>
#include <cassert>
#include <coroutine>

template <std::size_t MAX_BUFFER_SIZE, typename ...Events>
class EventQueue
{
public:
    static_assert(MAX_BUFFER_SIZE >= 1);
    static_assert((not std::is_void_v<Events> && ...), "No void arguments allowed");
    static_assert((std::is_copy_assignable_v<Events> && ...), "All Event types must be copy assignable");

    using Variant = std::variant<Events...>;

    [[nodiscard]] bool full() const
    {
        return m_full;
    }
    [[nodiscard]] bool empty() const
    {
        return m_empty;
    }

    [[nodiscard]] std::size_t capacity() const
    {
        return MAX_BUFFER_SIZE;
    }

    [[nodiscard]] std::size_t size() const
    {
        if (m_full)
        {
            return m_buffer.size();
        }
        if (m_empty)
        {
            return 0;
        }

        if (m_tail >= m_head)
        {
            return m_tail - m_head;
        }
        else
        {
            return m_buffer.size() - m_head + m_tail;
        }
    }

    template <typename T>
    [[nodiscard]] auto push(const T value)
    {
        static_assert((std::is_same_v<T, Events> || ...), "Only exact types can be pushed into the EventQueue");

        struct Awaiter
        {
            explicit Awaiter(EventQueue& queue, const T value) : m_queue{ queue }, m_value{ value } {}

            bool await_ready() const
            {
                return not m_queue.full();
            }

            void await_suspend(std::coroutine_handle<> h)
            {
                assert(m_queue.m_push == nullptr);
                m_queue.m_push = h;
            }

            void await_resume()
            {
                m_queue.m_push = nullptr;
                m_queue.InternalPush(m_value);
            }

            EventQueue& m_queue;
            T m_value;
        };

        return Awaiter{ *this, value };
    }

    [[nodiscard]] auto pull()
    {
        struct Awaiter
        {
            explicit Awaiter(EventQueue& queue) : m_queue{ queue } {}

            bool await_ready() const
            {
                return not m_queue.empty();
            }

            void await_suspend(std::coroutine_handle<> h)
            {
                assert(m_queue.m_pull == nullptr);
                m_queue.m_pull = h;
            }

            Variant await_resume()
            {
                m_queue.m_pull = nullptr;
                return m_queue.InternalPull();
            }

            EventQueue& m_queue;
        };

        return Awaiter{ *this };
    }

    void Execute()
    {
        if (not full() && m_push)
        {
            // resume the writer
            m_push.resume();
        }
        if (not empty())
        {
            // resume the reader
            m_pull.resume();
        }
    }
private:
    Variant InternalPull()
    {
        assert(not empty());

        Variant result{ m_buffer[m_head] };
        m_buffer[m_head] = Variant{};
        m_head = (m_head + 1) % m_buffer.size();

        m_full = false;
        m_empty = (m_head == m_tail);

        return result;
    }
    void InternalPush(Variant value)
    {
        assert(not full());

        m_buffer[m_tail] = value;
        m_tail = (m_tail + 1) % m_buffer.size();

        m_empty = false;
        m_full = (m_head == m_tail);
    }

    std::array<Variant, MAX_BUFFER_SIZE> m_buffer{};
    std::size_t m_head = 0;
    std::size_t m_tail = 0;
    bool m_empty{ true };
    bool m_full{ false };

    std::coroutine_handle<> m_pull = nullptr;
    std::coroutine_handle<> m_push = nullptr;
};

using EQueue = EventQueue<2, int>;

struct Task
{
    struct promise_type;
    using Handle = std::coroutine_handle<promise_type>;

    explicit Task(Handle handle) : m_handle{ handle }
    {
    }
    ~Task()
    {
    }
    Task(Task&& rhs) noexcept
    {
        m_handle = rhs.m_handle;
    }
    Task& operator=(Task&& rhs) noexcept
    {
        m_handle = rhs.m_handle;
        return *this;
    }

    Task(const Task&) = delete;
    Task& operator=(const Task&) = delete;

    struct promise_type {
        Task get_return_object() {
            return Task{ Handle::from_promise(*this) };
        }

        std::suspend_never initial_suspend() { return {}; }
        std::suspend_always final_suspend() noexcept { return {}; }
        void return_void() {}
        void unhandled_exception() { std::terminate(); }
    };
private:
    Handle m_handle;
};

Task A(EQueue& queue)
{
    for (int i = 0; i < 100; ++i)
    {
        co_await queue.push(i);
    }
}

Task B(EQueue& queue)
{
    while (true)
    {
        EQueue::Variant v = co_await queue.pull();
        std::visit([](auto&& arg)
            {
                std::cout << arg << '\n';
            }, v);
    }
}

int main() {
    EQueue q;
    Task a = A(q);
    Task b = B(q);

    while (1)
    {
        q.Execute();
    }

    return 0;
}


[W6.1e][W13-B1] Aadyaa Maddi

LO-JUnit:
Added a JUnit test to UtilsTest.java to test the isAnyNull(Object...) method in Utils.java.

@amad-person
Some comments added. Please ack & close the PR after reading comments.

Acknowledged, thanks!


handle RequestTimedOut

This pull request adds a no-op handler for RequestTimedOut errors to allow the normal retry logic to handle those cases in SimpleConsumer. It fixes #401.

This looks all fine to me.
@emmett9001 would you be able to sort the merge conflict? I fried my system this morning, am still working to set up a trusted box to give push access to...


ensure_active_livy_session creates new Livy session when fired from bg task

As opposed to stopping / restarting the current one.  It's likely related to the stale db connection that bg tasks has?
Same Livy session is used, but creates a new Combine LivySession.   Difficult to untangle, but definitely related to https://github.com/WSULib/combine/issues/311.

Hoping this will also be addressed by https://github.com/WSULib/combine/issues/184


Bluetooth instead of WiFi on ESP32

Hi,
first thank you for this great piece of Software. I wonder if it would be possible to pair ESP32 based Devices via Bluetooth instead of WiFi?
Kind regards
Stefan

I had a plan to make a bluetooth version, hoping that low-energy bluetooth would allow HomeKit devices to run on batteries. But since Espressif gave up on low-energy part, I have not incentive to do that: it's a lot of work with no apparent benefit: ESP32 devices are much less ubiquitous as ESP8266.


Fix to make it work with the current unerleased servant 0.5 version

As it doesn't work with the actually released servant 0.4 version you might prefer to merge it in another branch until servant 0.5 is released.
Best!

@yogsototh this should work with the current servant-0.4. @fizruk can you confirm?

@yogsototh it should work with servant-<IP_ADDRESS> (at least it does for me).
Can you tell us more about what is not working for you?

I only changed the code of example to make it work with swagger 0.5 (which is still on github only).
For example I changed EitherT to ExceptT.
This pull request is here to help when swagger will upgrade to 0.5.
I don't believe you should merge it in master for now.
I tried to compile it with swagger-<IP_ADDRESS> and it didn't compile. The compiler complain about the use of ExceptT instead of EitherT.

@yogsototh I see your changes bring example project up to date with servant's HEAD, yes.
We are indeed not going to merge these changes until servant-0.5 is out.

I tried to compile it with swagger-<IP_ADDRESS> and it didn't compile.

How did you try to compile it? stack build example? cd example/ && cabal install?

First I removed the dependencies to my local servant in my stack.yml:
git diff                                                                                ⏎ master ✱
diff --git a/stack.yaml b/stack.yaml
index 0abc7ca..e1c029e 100644
--- a/stack.yaml
+++ b/stack.yaml
@@ -7,17 +7,17 @@ resolver: lts-3.11
 packages:
 - '.'
 - example/
-- '../servant/servant/'
-- '../servant/servant-blaze/'
-- '../servant/servant-cassava/'
-- '../servant/servant-client/'
-- '../servant/servant-docs/'
-- '../servant/servant-examples/'
-- '../servant/servant-foreign/'
-- '../servant/servant-js/'
-- '../servant/servant-lucid/'
-- '../servant/servant-mock/'
-- '../servant/servant-server/'
+# - '../servant/servant/'
+# - '../servant/servant-blaze/'
+# - '../servant/servant-cassava/'
+# - '../servant/servant-client/'
+# - '../servant/servant-docs/'
+# - '../servant/servant-examples/'
+# - '../servant/servant-foreign/'
+# - '../servant/servant-js/'
+# - '../servant/servant-lucid/'
+# - '../servant/servant-mock/'
+# - '../servant/servant-server/'

 # Packages to be pulled from upstream that are not in the resolver (e.g., acme-missiles-0.3)
 extra-deps:

And then in the root I simply did a stack build:
stack build                                                                                master ✱
example-<IP_ADDRESS>: configure
example-<IP_ADDRESS>: build

--  While building package example-<IP_ADDRESS> using:
      /Users/esposito/.stack/setup-exe-cache/setup-Simple-Cabal-<IP_ADDRESS>-x86_64-osx-ghc-7.10.2 --builddir=.stack-work/dist/x86_64-osx/Cabal-<IP_ADDRESS> build exe:swagger-file exe:swagger-server --ghc-options " -ddump-hi -ddump-to-file"
    Process exited with code: ExitFailure 1
    Logs have been written to: /Users/esposito/dev/y/servant-swagger/.stack-work/logs/example-<IP_ADDRESS>.log

    Configuring example-<IP_ADDRESS>...
    Preprocessing executable 'swagger-file' for example-<IP_ADDRESS>...
    Preprocessing executable 'swagger-server' for example-<IP_ADDRESS>...
    [1 of 1] Compiling Main             ( Server.hs, .stack-work/dist/x86_64-osx/Cabal-<IP_ADDRESS>/build/swagger-server/swagger-server-tmp/Main.o ) [Servant.Swagger changed]

    Server.hs:58:41:
        Couldn't match type ‘Control.Monad.Trans.Either.EitherT
                               ServantErr IO Swagger’
                       with ‘ExceptT ServantErr IO Swagger’
        Expected type: Server API
          Actual type: ExceptT ServantErr IO Swagger
                       :<|> ((TodoId
                              -> Control.Monad.Trans.Either.EitherT ServantErr IO Todo)
                             :<|> ((TodoId
                                    -> Todo
                                    -> Control.Monad.Trans.Either.EitherT ServantErr IO (Maybe Todo))
                                   :<|> (Control.Monad.Trans.Either.EitherT ServantErr IO Todo
                                         :<|> (Todo
                                               -> Control.Monad.Trans.Either.EitherT
                                                    ServantErr IO Todo))))
        In the second argument of ‘serve’, namely ‘endpoints’
        In the second argument of ‘($)’, namely
          ‘serve (Proxy :: Proxy API) endpoints’

@yogsototh, It seems like stack is still looking for servant-0.5 (Since it's expecting ExceptT over EitherT). We should probably put an upper bound on servant (and just version the deps. in general) inside servant-swagger cc @fizruk.

@yogsototh the error indicates that your local servant is using ExceptT, so I guess you have HEAD rather than servant-0.4.*.
You can simply remove local servant packages from stack.yaml to build against servant from Stackage snapshot.
@dmjio The upper bound might make sense for the example/ project. For servant-swagger itself I am not so sure, since it should work for HEAD too (as you can see from this PR, only example's code is changed).

Might be good to keep this open for when servant-0.5 is released.


please do not remove my comments in toml file automaticlly.

I need setting options.
do not remove my comments in toml file automaticlly.
https://github.com/segeljakt/toml-fmt/issues/2

Thanks for the report! The serialization in this crate doesn't support comments, however, and that's where crates like toml-edit work instead.


improve +handler option

Hi there,
I just wanted to use this template to start a new project, but I had to update it a little for my usage. And I thought those updates could be useful for other people.
Basically this improves a little the existing handler to serve all the static assets in resources as well as the index.html.
Development
figwheel will take care of running the server handler itself (as it was doing before). So the development flow doesn't change.
But now, if you update the server code, it should now be reloaded automatically. (thanks to ring.middleware.reload/wrap-reload).
Production
You can now create a standalone jar that will work perfectly with both client side and server side code,  using:
lein uberjar

By default the server will use port 3000, except if the PORT environment variable is set.
I have added the cljs compilation to :prep-tasks, so that the clojurescript code is compiled before to create the jar.
I might need to add some task to prep-tasks to compile less/sass code?
I believe this should also resolve #24.
Thanks.

@matthieu-beteille Wow, thank you so much for the PR - this looks fantastic! Just a heads up, may be a day or two until I can test locally.

@matthieu-beteille Thanks again for the PR!! I'll include a prep-task for less/garden if they are present. Will hopefully cut a release with this update in the next few days. :)

Hmm, I tried adding ["less" "once"] and ["garden" "once"] to the :prep-tasks if their respective profile was used ... but i am getting in a weird loop with garden ... it just keeps printing out that it is compiling (less seems to work fine tho).
In short, when I said I will add the prep-tasks for less/garden ... I lied, I am going to leave them out.  A PR is welcomed though. :)


feat: enable asserts when running dart_frog dev

Status
READY
Description
Enable asserts usage when running the server in dev mode. This is useful when dealing with packages that provides certain features at development time, like Isar Database Inspector (Isar) or Development Filter (Logger), which executes functions like the example below to guarantee that these features will be executed only in development builds:
// Tree shake the inspector for profile and release builds.
assert(() {
   // Do your things....

    return true;
}());

Type of Change

[x] ✨ New feature (non-breaking change which adds functionality)
[ ] 🛠 Bug fix (non-breaking change which fixes an issue)
[ ] ❌ Breaking change (fix or feature that would cause existing functionality to change)
[ ] 🧹 Code refactor
[ ] ✅ Build configuration change
[ ] 📝 Documentation
[ ] 🗑 Chore

Hello there, thanks for the contribution!
This looks good to me.
We will need some tests for this tough.

@hericles-koelher , would you be able to implement the tests? Otherwise, I can do it and merge it for you.

closing in favor of https://github.com/VeryGoodOpenSource/dart_frog/pull/766


Update Module Resolution.md

"as if were merged together" => "as if they were merged together"

Thanks!


Use vendir to generate chart

Use vendir to generate chart.

I've created a new WC using this version of cilium-app and it worked fine.

Updated README.md with instructions on how to upgrade the app.

@mcharriere I've removed the rm -rf from the Makefile here https://github.com/giantswarm/github/pull/1203


[Service Bus] Update to use latest preview version of core-amqp

This issue is to track changes required to bring Service Bus upto speed with changes on master.
This builds over changes on master and feature/service-bus-track-2 branches

feature/service-bus-track-2 branch now has version 1.0.0 of core-amqp. Please use this issue to track the work needed to update Service Bus to use version 1.0.0 of core-amqp


rocksdb: add Env related API

PTAL @zhangjinpeng1987 @siddontang @andelf
Signed-off-by: Cholerae Hu<EMAIL_ADDRESS>

After discuss with jinpeng, we found out that thread pool is a singleton, we don't need to expose Env API. So close this PR.

so, how do we share the same Env with multi rocksdbs?

We do not need to do this. All rocksdb will share one thread pool implicitly.


Buttons feature request

I have some feature requests.

Can you add an option to let the buttons render on top of the textarea?
Can the buttons including custom ones get arranged for display? For example, bring the Heading buttons left, next to italic?

@vimxts about 1: I think this is an edge-case that shouldn't affect the size or performance of woofmark. Maybe this is possible with clever external CSS?

@derhuerst I can do it with CSS and of course JavaScript too, but wouldn't it be better if woofmark includes the option to prepend the buttons before the textarea?
Also, I haven't been able to do it with CSS, maybe because I'm not clever enough:

I want .wk-commands and .wk-switchboard to float left so that it become one bar.
They have to be positioned on top of the textarea.
I should be able to add new command buttons to the sets and still be able to see everything.

I would use JS for this actions, I suppose it is a get around to the issue, I can even do it for the 2.
But I wish woofmark can include these helpful options while generating the editor, so a bigger liberty to styling can be achieved.

I'm sorry but I'm not interested in adding this to core


add MIT license text

The license texts needs to be included in every copy of the code, see this part of the MIT license:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
We need this to be able to ship with Fedora

Nice pickup, thanks :+1:


Problems installing Qt 6.5 on GH runners

I've been using the action successfully on Qt 5.x versions on GH action runners successfully.
We're trying to add some Qt6 compiles, and for example on Mac I'm getting this:
bash-3.2$ aqt list-qt mac desktop
6.5.0

Okay, it knows about 6.5.0, although strangely on the runner it doesn't know about other versions?
aqt install-qt mac desktop 6.5.*
aqtinstall(aqt) v2.1.0 on Python 3.11.3 [CPython Clang 13.0.0 (clang-13<IP_ADDRESS>)]
Resolved spec '6.5.*' to 6.5.0
Specified Qt version is unknown: 6.5.0.
Failed to locate XML data for Qt version '6.5.0'.

This obviously isn't a bug with your action, but maybe you have some suggestions?

That looks like it could be a bug in aqtinstall. You are using a version of aqtinstall that's really too old to install 6.5.0 properly, in many situations. I would suggest that you specify aqtversion: '==3.1.*' in your .yml file, and v3 of this action in case you are affected by #176. If you're still having trouble, it may be worthwhile to open an issue at https://github.com/miurahr/aqtinstall/issues .

Great, thanks. I'll give the aqtversion: '==3.1.*' a try now. (I'm already using v3 of the action)
It may also be a transient issue - sometimes the GH runners seem to have trouble hitting the network.

That was exactly the problem - using a newer aqtversion worked.


Cleanup and fix documentation

Codecov Report

Merging #5 into master will increase coverage by 0.03%.
The diff coverage is 85.71%.

@@            Coverage Diff             @@
##           master       #5      +/-   ##
==========================================
+ Coverage   96.73%   96.77%   +0.03%
==========================================
  Files           6        6
  Lines          92       93       +1
  Branches        3        3
==========================================
+ Hits           89       90       +1
  Misses          3        3

Impacted Files
Coverage Δ

src/index.js
100% <100%> (ø)
:arrow_up:

src/createVerifyData/index.js
91.17% <84.61%> (+0.26%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bf6319b...e952b26. Read the comment docs.


InlineAutoData Isn't Following XUnit Serialization of Params for Visual Studio test runner

Currently, if I use InlineData from XUnit the Visual Studio test runner will put all the ones of theory nest under the test and you can expand and see each one's results. If I use InlineAutoData and that is the only change then that very nice feature goes away. Doing some research this is explained by Brad Wilson why and what is required by XUnit to support this feature and when it doesn't work why it isn't working.
So I think you would have to do something similar in order to make it work.  It is all explained here in this XUnit issue:
https://github.com/xunit/xunit/issues/1473
You can take any Theory that has InlineData that takes any of the supported types that make the feature work (ie: char, string, byte, short, ushort, int, uint, long, ulong, double, decimal, bool, DateTime, DateTimeOffset.  There is an example of code in that issue above as well.
This is a very nice feature in Visual Studio's Test Runner I hope you can fix this in your side as I am removing AutoFixture from places where it was only doing a simple string or and nothing complex as I rather have the expanding node on the test.

Thanks for raising the issue 👍
I do understand the reasons behind the behavior you describe and it's very unlikely we can do something about that. The reason is more global and is not directly related to the mentioned issue.
AutoFixture data attributes (AutoData, InlineAutoData) are a tool to generate dynamic parameter values for tests. Each invocation generates a unique set of values, so it's currently impossible to run test twice with the same dynamic parameter values. Some tests runners (including VS) use two phases to work with tests - discovery and actual run. If you look at InlineData, its values are static and cannot change between the discovery and run. On other hand, if you try to discover tests for e.g. AutoData, values will constantly change between discovery and run and runner will be unable to match the test to run. To support such scenarios xUnit offers a way to disable the discovery enumeration for theory, so test parameters are not read. Rather the whole theory (with all cases) is treated as a single entity and all cases are run together. Search for the DisableDiscoveryEnumeration parameter to get more info.
AutoFixture uses the mentioned way as well and disables discovery for the AutoData and InlineAutoData attributes. See e.g. here:
https://github.com/AutoFixture/AutoFixture/blob/d3e14a818b933d948ea82464b197a5aeb152f070/Src/AutoFixture.xUnit2/AutoDataAttribute.cs#L17-L24
In theory we could improve our logic to allow enumeration for InlineAutoData attribute when number of specified arguments is equal to the number of parameters, but I'm very reluctant to do that:

it's very rare case when you use InlineAutoData instead of InlineData - what's the value? 😉
usually you don't use InlineAutoData in own projects, but create a custom attribute with customized fixture. Therefore, the attribute type will be different and our fix will still do nothing.
it makes things harder to maintain.
it's VS runner specific, so it might be improved there in further versions. For instance R# renders the stuff nicely after the run:

Given that I don't see what can be done at our side. If you see - please suggest 😅

Closing this one due to inactivity. Feel free to re-open the issue if you believe that change from our side is required.

Well, XUnit was able to solve the problem, and it is difficult to monitor unit tests and to be able to introduce new people to testing and showing them Theory based testing when it's really difficult to know which "data"  based test in a theory has failed.  I have to halt new uses of AutoFixture.
I feel did suggest a way, the way XUnit solved the same problem and which works perfectly and if I remove autofixture the theory unit test expands to show each "data" element as a test with a result broken out. If I place autofixture  back it goes away. So clearly it is autofixtures problem to fix and one that existed in XUnit until it was addressed by the issue I provided. Seems like either you want to fix it and can try what XUnit did and if it doesn't find a way to do it as it clearly is based on how and what types can be represented to the underlying MS testing hooks.
I don't have times to do more research for you that I already did, as it is far easier to stop adding autofixture and remove over time than it is to find the time to figure out an open source projects code and debug and figure out the solution and make a PR. I wanted to make you aware of the issue and point you to a possible solution since XUnit had the exact same issue and they solved it. It is now in the project maintainers and other contributors hands to decide if they want to solve the issue or not, not in the one reporting it.
You have a good side community of contributors and posting an issue to the group to see if anyone is willing to take it on seems simple and worth doing than just closing down the problem and ignoring it. The worse that happens is no one takes it on with the possibility that someone will and solve it.

I do fully understand your concerns and the difficulties you have! But let's have the full overview of the issue (in addition to what I already posted here).

I feel did suggest a way, the way XUnit solved the same problem ...
Seems like either you want to fix it and can try what XUnit did ...
... since XUnit had the exact same issue and they solved it.

In fact, xUnit hasn't solved this problem and suffers from it as well. The reason why you don't meet it is that you mostly work with data which is constant and supports enumeration. Consider the following sample with pure xUnit (notice the DisableDiscoveryEnumeration = true part):
    public static TheoryData<int> TestValues => new TheoryData<int> {1, 2, 3};

    [Theory]
    [MemberData(nameof(TestValues), DisableDiscoveryEnumeration = true)]
    public void PureXUnit(int primitiveValue)
    {
      Assert.NotEqual(0, primitiveValue);
    }

Execution result:

As you can see, run values are not displayed even with pure xUnit. AutoFixture uses the same approach under the hood (disabled enumeration), so that's why you don't see any values as well.

Theory based testing when it's really difficult to know which "data" based test in a theory has failed. I have to halt new uses of AutoFixture.

Before making the final decision ensure that you haven't missed the following window:

If you select the failed test you can see the values used for this run (in the window below). It's probably not that convenient as inline values, however it's fully usable. If that's a deal breaker for you, you might change your opinion 😉
FWIW if you use an alternative runner (like R#), you will get this feature out of the box.

worth doing than just closing down the problem and ignoring it.

Usually I try to not close the issue if I see any actions we could make to improve the situation In this particular I see nothing reasonable we can do about it:

pure xUnit suffers from it as well;
there is already a way to see run values, so issues is not critical.

I am confused xUnit did solve the ISSUE I am having as using the only xUnit attributes it all works fine if I stick with the supported types, and when I use AutoFixture it stops working. SO the issue I HAVE REPORTED has been fixed by xUnit and I only wish AutoFixture to support the same types in the same way.
I know for a FREAKING  (choose your own expletive)  FACT THIS ISSUE IS UNDER AUTOFIXTURES CONTROL and you are choosing to deflect it and say essentially "Not my problem" because either you do not understand the issue I am have presented or you don't see it as an issue or you don't care.
 So AutoFixture can fix it or don't I am done and decided to drop AutoFixture all together as its not worth the trouble when reporting bugs to have to work this hard to get people to just accept the facts. I can risk the next bug I may need to report having the same red tape.  So the issue is closed it can stay closed and I have unsubscribed.

@CreepyGnome Despite the fact you decided to opt-out from the discussion, I still want to bring the clarity into this thread. I'm under the strong impression that we misunderstood each other and are speaking about different scenarios.
I see 3 potential scenarios to consider:
// Scenario 1. We are using xUnit's InlineData to specify _all_ the theory parameters.
// Notice, all the parameters are static.
[Theory]
[InlineData(42, "42")]
public void Scenario1(int number, string str)
{
    Assert.Equal(number.ToString(), str);
}

// Scenario 2. We are using AutoFixture's InlineAutoData to specify _all_ the theory parameters.
// Notice, all the parameters are static.
[Theory]
[InlineAutoData(42, "42")]
public void Scenario2(int number, string str)
{
    Assert.Equal(number.ToString(), str);
}

// Scenario 3. We are using AutoFixture's InlineAutoData to specify _only part_ of the theory parameters.
// Notice, 'str' parameter is generated dynamically during each invocation.
[Theory]
[InlineAutoData(42)]
public void Scenario3(int number, string str)
{
    Assert.Equal(number.ToString(), str);
}

The issue you describe in this thread sounds: "If I follow the scenario 1 I can see all the theory cases in the VS Tests tree. If I switch to the scenario 2, theory cases are no longer rendered in the tree. Please fix."
First of all please pay attention that InlineAutoData is not a replacement for the InlineData and should not be used instead of it whenever possible. If all the test parameters are static, you should use the InlineData. It indeed offers slightly better runner experience and likely slightly better performance.
The InlineAutoData should only be used in Scenario 3 - when part of your theory parameters are fixed, but other part is dynamically generated by AutoFixture. The InlineAutoData attribute is designed and optimized for such scenarios only.
The InlineAutoData attribute is not optimized for the Scenario 2 as it brings no benefit there. Technically it could be used and will work fine, but you'll observe the inconveniences mentioned in this thread.

Given that InlineAutoData is designed for Scenario 3, we explicitly disabled the theory discovery. Otherwise, you will be unable to run the tests after the discovery. As result you observe that theory cases are not rendered in the tree. To be clear: there is NO WAY to fix the tree issue and support the Scenario 3 in the same time.
As for the Scenario 2. Technically we could improve the InlineAutoData to support the Scenario 2 and show data in the tree if you explicitly specified all the theory parameters. However, that will be super complicated and brings no benefit:

if all the parameters are constant you are free to use InlineData attribute;
very often you don't use InlineAutoData directly in your projects, because you want to globally customize the fixture (e.g. to enable Moq/NSubstitute integration). To achieve that you create your own type of this attribute, like InlineAutoNSubstituteData inherited from the InlineAutoData (or not inherited, it's not required). In this case there is no way to reliably detect whether you specified all the parameters or not - custom attribute might have different signature. Therefore, this feature will work only in limited number of cases - it doesn't worth it.

Therefore, you were right - we can partially solve the issue. But there is no practical sense to do that. If you wish, I could provide you with instructions and you'll do that in your project locally 😉

Given all the information above in head I closed the issue, as there is no way to solve it and still support Scenario 3 - the primary scenario.
Sorry if you were confused by the poor/absent documentation describing this. I'm already working on the doc site where all such things are going to be described.
Have a good day!

P.S. Please do not ignore the fact that you can see the actual invocation parameters as mentioned here, so issue isn't that critical. However, of course, it's a matter of your priorities and if AutoFixture doesn't bring the benefits to your project - feel free to get rid of it 😉

How many years later and this hasn't been addressed?
I do understand @CreepyGnome  frustration as I just brought - for some ledgit reasons - our tests running with NUnit to the XUnit.
Now we figured, that all of our parameterized tests are showing as a single one under the Visual Studio test explorer. The test summary is showing the data used, plus which data failed/succeeded.
@zvirja Unfortunately I must also disagree with you, as you didn't consider this:
[Theory]
[CustomInlineAutoData(42)]
public void Scenario3(int number, string str, IFixture f)
{
    Assert.Equal(number.ToString(), str);
   f.Create<MyService>(); // setup in the CustomInlineAutoData
}

CustomInlineAutoData customizes the Autofixture internally and also fixes 42 for the parameter number. I would want to have both worlds combined, like I did with NUnit package until now.
This is a big back-lash for me, and makes me wanna think if using xunit was the right choice.
Are there any known workarounds? Do I need to make my own InlineAutoDataAttribute?

@R0boC0p at the time when this thread was created the test arguments were excluded from the output to prevent displaying data that was inconsistent with he actual test run.
I believe that since then the way pre-discovery works in Visual Studio has changed and the workaround is no longer necessary as the data in the test explorer is consistent with the data in the actual test.
Additionally the xUnit 2 AutoFixture attributes are currently being re-designed and the custom "data discoverer" will be disabled in the next v5 preview version.


heapster can not connect to the influxdb

can not see the cpu usage and mem usage.

Hi @auqf, how did you install MicroK8s? Did you select a channel? What addons do you have enables? Did you try the Prometheus and the metrics server addons?


Update labels on the Docker image

Currently the Docker image merges labels from the base centos:7 image as well as our own maintainer LABEL:
$ docker inspect docker.elastic.co/elasticsearch/elasticsearch:6.1.1 | jq .[0].ContainerConfig.Labels
{
  "build-date": "20171128",
  "license": "GPLv2",
  "maintainer": "Elastic Docker Team<EMAIL_ADDRESS>  "name": "CentOS Base Image",
  "vendor": "CentOS"
}

We need to add more labels to clarify the license for Elastic products, as well as name/vendor/build-date.

cc @mgreau @jarpy @beniwohli

As I'm currently working on it for other Elastic projects right now, I think we should use an SPDX identifier in the license label content.
Also, FYI a group is working on a convention for a standardized way to express container labels:

http://label-schema.org/

It's still a WIP but it's maintained by well-known people in the container world:  https://github.com/label-schema/label-schema.org/blob/gh-pages/rc1.md

It looks as though the Centos:7 image has recently moved to Label Schema but it isn't quite right yet.
Anyway, we can be more confident that we can overwrite their labels, since they are clearly moving to Label Schema.

Interestingly, Label Schema doesn't actually have a licence field.

Fixed in #166


Can I make an Android App that uses this ?

I'm a student, and it's just a programming project.
Thanks! I love it ! So fun ! 💯

Yes! Feel free to use it in any of your projects.

Thank you ! 🤟🏼


add OpenCensus tracing

Feature Request
Service Bus, Event Hubs and the Azure SDK have added OpenCensus tracing spans to every service call so users can more easily troubleshoot when things go wrong; users can view these traces in App Insights, which supports OpenCensus for Go apps.
Could you add OpenCensus spans to Storage package calls as well? Thank you.

Hi, @joshgav. Thank you for posting this request. I have logged this for our team's discussion, and we will get back to you when we have looked more into OpenCensus and the feasibility of supporting it in the Go Sdk.


tile software

Hey man great project! Was wondering what package /usr/local/bin/tile comes from? What would I need to install in order to try it out?

Thanks :)
That binary is our tiler: https://github.com/freesewing/tile
You'll need to compile it, as we don't provide binaries. However, that should be trivial.
If you run into trouble, feel free to create an issue. I'll close this one as a assume this answers your question.


Add PooledByteBufAllocator.dumpStatus() which allows to obtain a huma…

…n-readable status of the allocator.
Motiviation:
Sometimes it is useful to dump the status of the PooledByteBufAllocator and log it. Doing this is currently a bit cumbersome as the user needs to basically iterate through all the metrics and compose the String. we would better provide an easy way to do this.
Modification:
Add dumpStatus() method.
Result:
Easier to get a view into the status of the allocator.

@trustin PTAL

LGTM except the method name. Could you also fix the build failure?

Cherry-picked into 4.0 (856d6b94023e55f08b5f008bfedbfd7eabdd1ab4) and 4.1 (200ca39b5c947cfeed153d7cb683ead86fe3e879)


Implement declarative context API

CodeSandbox of an initial implementation: https://codesandbox.io/s/r0vjwmm2mp.
This replaces #1485.

The React Hooks API moves away from declarative contexts. 👌

The React Hooks API moves away from declarative contexts.

it doesn't, though. You can still use providers in the tree, and this is the way you should be doing it in class components as you can't use hooks in them.
This has been implemented in @skatejs/sk-context.


Transparent screen is dispayed for a very short while before the values are populated in any dropdown.

Describe the Bug
Try to open up any dropdown we could see a transparent screen for a very short time before the values are populated.
Steps to Reproduce

click on a any dropdown in the modeler.
We could see a transparent screen for a very short time before the values are populated.

I could not make a screen cast  as the transparent screen appears only for a second but it is visible in every dropdown of the application.
Expected Behavior
Clear the transparent screen before values are populated in the dropdown.
Environment

OS: [Windows 10]
Camunda Modeler Version: [5.0.0]
Execution Platform: [e.g. Camunda Platform, Camunda Cloud]

I can't reproduce it on macOS, so it's maybe related to OS-related native form element styles. To prevent this, we could consider having unified form elements, that look and behave the same on any OS (and prevent unexpected, OS-specific behaviors).
Can we somehow reproduce this behavior on other OS (Linux and Windows)? /cc @MaxTru @marstamm @Skaiir @philippfromme

Can't reproduce on Linux either. But if it is just very brief, I might just be to blind to see it

Closing as we can't reproduce it. Please reopen the ticket if you still feel it is valid


"VNodeChild is not a valid JSX element"

vue-tsc >= v0.2.3 throws an error when using a render function as a component. I am not too familiar with Vue 3 but this seems incorrect as the app renders fine when not using vue-tsc.
Example
<script setup lang="ts">
import { createVNode, RenderFunction } from 'vue'

const HelloWorld: RenderFunction = () => {
  return createVNode('h1', {}, 'Hello world')
}
</script>

<template>
  <HelloWorld />
</template>

src/App.vue:10:4 - error TS2786: '__VLS_2' cannot be used as a JSX component.
  Its return type 'VNodeChild' is not a valid JSX element.
    Type 'undefined' is not assignable to type 'Element | null'.

10   <HelloWorld />
      ~~~~~~~~~~

Vue SFC Playground example that renders successfully: https://sfc.vuejs.org/#eyJBcHAudnVlIjoiPHNjcmlwdCBzZXR1cD5cbmltcG9ydCB7IGNyZWF0ZVZOb2RlIH0gZnJvbSAndnVlJ1xuXG5mdW5jdGlvbiBIZWxsb1dvcmxkKCkge1xuICByZXR1cm4gY3JlYXRlVk5vZGUoJ2gxJywge30sICdIZWxsbyB3b3JsZCcpXG59XG48L3NjcmlwdD5cblxuPHRlbXBsYXRlPlxuICA8SGVsbG9Xb3JsZCAvPlxuPC90ZW1wbGF0ZT4iLCJpbXBvcnQtbWFwLmpzb24iOiJ7XG4gIFwiaW1wb3J0c1wiOiB7XG4gICAgXCJ2dWVcIjogXCJodHRwczovL3VucGtnLmNvbS9AdnVlL3J1bnRpbWUtZG9tQDMuMi4xOS9kaXN0L3J1bnRpbWUtZG9tLmVzbS1icm93c2VyLmpzXCJcbiAgfVxufSJ9
Steps to reproduce

Clone https://github.com/bradlc/vue-ts-render-function
Run npm install
Run npm run build

In next version, invalid functional component type is any in template, so it don't report error and not support props type-checking.

Does this mean that volar will not support type checking for any .vue component that uses the <slots> feature?

This problem is RenderFunction return type VNodeChild is not compatible to vue's JSX element type VNode.

Is this a flaw in vue's typesystem  (hence the upstream label)?

Does this mean that volar will not support type checking for any .vue component that uses the <slots> feature?

If your component typecheck working before, it still working. If the component report '__VLS_x' cannot be used as a JSX component beforer, component type changed to any in new version.

Is this a flaw in vue's typesystem (hence the upstream label)?

Yes this is upstream types problem.


Make indexbuild macos QoS 'utility' by default

What changed?

Adds startup flag to set index builds MacOS QoS to utility by default

Why did it change?

Developers have reported excessive battery usage related to background indexing

How is it tested?

Added debug statements to the bazel_build.sh to see the startup flag being added for index builds
I was unable to directly confirm (via my macbook) that the java process was actually set to utility - open to suggestions on how to do this.

Tried confirming that this change actually changes the qos of the bazel process by using this command sudo powermetrics --show-process-qos --show-process-qos-tiers --show-cpu-qos --samplers tasks -n 1 but I still don't see concrete proof that the flag actually changes the qos

With the merged output bases I don't believe we can, or would want to, take this change?


REGR:Kerberos authentication is not working

Using 9.0-EA4 build Dec 7, the authentication is failed with 401 Authorization error
Steps:
Use the steps from https://github.com/wooldridge/ml-kerberos to setup and to run the test

@wooldridge this is the error message:
Error: read documents: kerberos credentials failed (on /file.html)
Debug: deferred promise rejecting with 1 error messages
Debug: kerberos credentials retrieved
Debug: authenticated request for /v1/documents?format=json&uri=%2Ffile.html&category=content
Debug: kerberos authentication
error
{
  "message": "read documents: kerberos credentials failed (on /file.html)"
}
Error: read documents: kerberos credentials failed (on /file.html)
Error: read documents: response with invalid 401 status (on /file.html) { errorResponse:
   { statusCode: 401,
     status: 'Unauthorized',
     message: '401 Unauthorized' } }
Debug: deferred promise rejecting with 2 error messages

The doc is inserted using admin user as a part of setup and we try to read it as user1 (kerberos principal<EMAIL_ADDRESS>config.js:
var config = {};

config.path = "/space/ml-kerberos/"; // include trailing "/"

config.name = "ml-kerberos";
config.host = "ayuwono-z620.marklogic.com";
config.port = 8019;
//config.port2 = 8060;

config.auth = {
  user: 'admin',
  pass: 'admin',
  sendImmediately: false
};

config.appServer = {
  "server-name": config.name + "-http",
  "server-type": "http",
  "group-name": "Default",
  "root": "/",
  "port": config.port,
  "content-database": "Documents",
  "modules-database": "Modules",
  "authentication": "kerberos-ticket",
  "internal-security": false,
  "external-security": "mykerberos"
}

config.databaseSetup = {
  "database-name": config.name
};

config.forestSetup = {
  "forest-name": config.name + '-1',
  "database": config.name
}

config.restSetup = {
  "rest-api": {
    "name": config.name + "-rest",
    "database": config.name,
    "modules-database": config.name + "-modules",
    "port": config.port,
    "error-format": "json"
  }
}

if (typeof module !== 'undefined' && typeof module.exports !== 'undefined') {
  module.exports = config;
}

I was able to confirm the bug, I see the same 401 response on my system. I can confirm that the Kerberos handshake is happening and I get the string to put in the negotiate header. Trying to debug further with Arthur.

If you set ssl-require-client-certificate to false on the external security and the app-server it will work. But I don't recall that we need to set it to false

I've updated the ml-kerberos project with changes in the following branch:
https://github.com/wooldridge/ml-kerberos/tree/ea4-updates
This includes the fix for setting ssl-require-client-certificate to false.
The one ML configuration step I wasn't able to script was creating external security. I've filed a bug here:
https://bugtrack.marklogic.com/43696
@ayuwono can you test this updated branch? If it works for you I will merge that branch into master. We should also be able to close this bug.

The principal property is not required for the database client setup for Kerberos. That property was extraneous and I removed it in the updated branch. The result you got above shows that the test succeeded.

The updated branch is working fine and @wooldridge has merged it to master. I'm using Dec 12 and tested more with documents write and query with kerberos client


25 write a python script to track and report running processes

adding python script to track and report running processes

I have made a pull request and merged already. But my code has encountered conflicts due to the fact that i first of all created a branch through my terminal and was working on it then pushed, which was a mistake from my ignorance but later on discovered that i was to create a branch but through the git. So I later on went and created a new branch through the git and continued as required, but conflicts have been encountered.

I will like to inform my superior not only to do a review but also a quick check on the conflicts base and get back to me on how it could be solved.


(PUP-5513) Allow blank lines in ssh known hosts file

Previously, blank lines in the default ssh_known_hosts file would
cause errors in the sshkey type due to a faulty regex. This commit
updates said regex to properly match blank lines with no whitespace
characters.

:+1: from me, but I'll be out tomorrow and don't want to merge right before I go.


[FLINK-8771] [Build System] [Checkstyle/Scalastyle] Upgrade scalastyle to 1.0.0

What is the purpose of the change
Upgrade scalastyle from 0.8.0 to 1.0.0
Brief change log

Upgrade scalastyle from 0.8.0 to 1.0.0
Fixed some license style issues along the way, because scalalstyle 1.0.0 detected those as errors

Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:
none
Documentation
none

cc @zentol

Hi @zentol , can you take a look at this PR?


⚠ Bofrost.es has degraded performance

In fb42035, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1761 ms

Resolved: Bofrost.es performance has improved in 5eed0e1 after 7 minutes.


watchtower.py kills my lightningd

Since recently my lightningd crashes more or less constantly (today 22 times), before the crash there is always the same logentry, which I include here. I also replaced the "\n" in the log by newlines for readability.
My node had no clearnet connectivity, so connection problems are not that unusual.
2022-02-08T19:15:34.142Z **BROKEN** lightningd: Plugin for commitment_revocation returned non-result response {"jsonrpc": "2.0", "id": 155, "error": {"code": -32600, "message": "Error while processing commitment_revocation: HTTPConnectionPool(host='[redacted]', port=[redacted]): Read timed out. (read timeout=5)", "traceback": "Traceback (most recent call last):
  File \"/usr/lib/python3/dist-packages/urllib3/connectionpool.py\", line 445, in _make_request
    six.raise_from(e, None)
  File \"<string>\", line 3, in raise_from
  File \"/usr/lib/python3/dist-packages/urllib3/connectionpool.py\", line 440, in _make_request
    httplib_response = conn.getresponse()
  File \"/usr/lib/python3.9/http/client.py\", line 1347, in getresponse
    response.begin()
  File \"/usr/lib/python3.9/http/client.py\", line 307, in begin
    version, status, reason = self._read_status()
  File \"/usr/lib/python3.9/http/client.py\", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), \"iso-8859-1\")
  File \"/usr/lib/python3.9/socket.py\", line 704, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File \"/usr/lib/python3/dist-packages/requests/adapters.py\", line 439, in send
    resp = conn.urlopen(
  File \"/usr/lib/python3/dist-packages/urllib3/connectionpool.py\", line 755, in urlopen
    retries = retries.increment(
  File \"/usr/lib/python3/dist-packages/urllib3/util/retry.py\", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File \"/usr/lib/python3/dist-packages/six.py\", line 719, in reraise
    raise value
  File \"/usr/lib/python3/dist-packages/urllib3/connectionpool.py\", line 699, in urlopen
    httplib_response = self._make_request(
  File \"/usr/lib/python3/dist-packages/urllib3/connectionpool.py\", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File \"/usr/lib/python3/dist-packages/urllib3/connectionpool.py\", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='[redacted]', port=[redacted]): Read timed out. (read timeout=5)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File \"/home/cl/.local/lib/python3.9/site-packages/pyln/client/plugin.py\", line 621, in _dispatch_request
    result = self._exec_func(method.func, request)
  File \"/home/cl/.local/lib/python3.9/site-packages/pyln/client/plugin.py\", line 606, in _exec_func
    return func(*ba.args, **ba.kwargs)
  File \"/home/cl/src/python-teos/watchtower-plugin/watchtower.py\", line 371, in on_commitment_revocation
    tower_signature, available_slots = add_appointment(
  File \"/home/cl/src/python-teos/watchtower-plugin/net/http.py\", line 16, in add_appointment
    response = send_appointment(tower_id, tower, appointment_dict, signature)
  File \"/home/cl/src/python-teos/watchtower-plugin/net/http.py\", line 62, in send_appointment
    response = process_post_response(post_request(data, add_appointment_endpoint, tower_id))
  File \"/home/cl/src/python-teos/watchtower-plugin/net/http.py\", line 113, in post_request
    return requests.post(url=endpoint, json=data, timeout=5)
  File \"/usr/lib/python3/dist-packages/requests/api.py\", line 119, in post
    return request('post', url, data=data, json=json, **kwargs)
  File \"/usr/lib/python3/dist-packages/requests/api.py\", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File \"/usr/lib/python3/dist-packages/requests/sessions.py\", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File \"/usr/lib/python3/dist-packages/requests/sessions.py\", line 655, in send
    r = adapter.send(request, **kwargs)
  File \"/usr/lib/python3/dist-packages/requests/adapters.py\", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='[redacted]', port=[redacted]): Read timed out. (read timeout=5)
"}}
2022-02-08T19:15:34.150Z **BROKEN** lightningd: FATAL SIGNAL 6 (version v0.10.2)
2022-02-08T19:15:34.150Z **BROKEN** lightningd: backtrace: common/daemon.c:38 (send_backtrace) 0x557cb2b75da1
2022-02-08T19:15:34.150Z **BROKEN** lightningd: backtrace: common/daemon.c:46 (crashdump) 0x557cb2b75df0
2022-02-08T19:15:34.150Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f293fb70d5f
2022-02-08T19:15:34.150Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f293fb70ce1
2022-02-08T19:15:34.150Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f293fb5a536
2022-02-08T19:15:34.150Z **BROKEN** lightningd: backtrace: lightningd/log.c:804 (fatal) 0x557cb2b19547
2022-02-08T19:15:34.150Z **BROKEN** lightningd: backtrace: lightningd/plugin_hook.c:179 (plugin_hook_callback) 0x557cb2b447fe
2022-02-08T19:15:34.150Z **BROKEN** lightningd: backtrace: lightningd/plugin.c:521 (plugin_response_handle) 0x557cb2b3ea9b
2022-02-08T19:15:34.151Z **BROKEN** lightningd: backtrace: lightningd/plugin.c:627 (plugin_read_json_one) 0x557cb2b3ecc0
2022-02-08T19:15:34.151Z **BROKEN** lightningd: backtrace: lightningd/plugin.c:672 (plugin_read_json) 0x557cb2b3eea0
2022-02-08T19:15:34.151Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x557cb2bda90d
2022-02-08T19:15:34.151Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x557cb2bdb4b7
2022-02-08T19:15:34.151Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x557cb2bdb4f5
2022-02-08T19:15:34.151Z **BROKEN** lightningd: backtrace: ccan/ccan/io/poll.c:453 (io_loop) 0x557cb2bdd6cb
2022-02-08T19:15:34.151Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:21 (io_loop_with_timers) 0x557cb2b109df
2022-02-08T19:15:34.151Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:1163 (main) 0x557cb2b168d7
2022-02-08T19:15:34.151Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f293fb5bd09
2022-02-08T19:15:34.151Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x557cb2aec399
2022-02-08T19:15:34.151Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffffffffffffff

Sorry to hear that. It is pretty weird since timeout and connectivity issues when posting data to the tower should be accounted for.
https://github.com/talaia-labs/python-teos/blob/master/watchtower-plugin/net/http.py#L115-L119
I have to say that extensive testing against a remote node over tor has not been performed to be fair.
I'll give it a closer look and see what may be missing.  Something you may want to try until I find if that's the only exception that's unhandled would be to increase the timeout for requests. I may add that to the config file with the patch once ready if it happens to be useful.

I'm no developer, and this might be totally wrong, but for now I have patched it like this:
diff --git a/watchtower-plugin/net/http.py b/watchtower-plugin/net/http.py
index 0a6bfe8..670f583 100644
--- a/watchtower-plugin/net/http.py
+++ b/watchtower-plugin/net/http.py
@@ -110,11 +110,15 @@ def post_request(data, endpoint, tower_id):
     """

     try:
-        return requests.post(url=endpoint, json=data, timeout=5)
+        return requests.post(url=endpoint, json=data, timeout=1)

     except ConnectTimeout:
         message = f"Cannot connect to {tower_id}. Connection timeout"

+    except ReadTimeout:
+        message = f"Time out reading from {tower_id}"
+
     except ConnectionError:
         message = f"Cannot connect to {tower_id}. Tower cannot be reached"

As the stack trace says it's a ReadTimeout and not a ConnectionError.
I also set the timeout to 1 to test.

If you have imported ReadTimeout from requests that may prevent the code from crashing, but it may not solve the issue.
The timeout param as it is set affects both connection and read timeouts, so by reducing it to 1 it may increase the frequency of you hitting that exception.
Looks like there are no more timeouts apart from those two, so I'll roll out a patch after some testing (hopefully by tomorrow).

awesome, thanks. waiting for your patch then!

@svewa would you mind giving https://github.com/talaia-labs/python-teos/pull/305 a go?
I've also realized there may be something going on with #303 while testing this, so may roll out another patch for that (that one may take longer though).

I tried it, but now my watchtower.py crashes immediately.
$ python3 src/python-teos/watchtower-plugin/watchtower.py
Traceback (most recent call last):
  File "/home/cl/src/python-teos/watchtower-plugin/watchtower.py", line 19, in <module>
    from common.config_loader import ConfigLoader, UnknownConfigParam
ImportError: cannot import name 'UnknownConfigParam' from 'common.config_loader' (/home/cl/.local/lib/python3.9/site-packages/common/config_loader.py)
cl@cl:~$

Starting as plugin fails, too.
I did a fresh git clone https://github.com/talaia-labs/python-teos.git - still same error, so first need to figure out what happened there :/

That's because the plugin is using an updated teos-common library which is not yet in PyPi, but the version you got installed is the old one. I'll add that to PyPi when releasing a patched version (once this get's patched).
Are you using virtualenvs to install your Python packages or installing them system-wise?

@svewa I've updated the PR so teos-common can be installed standalone, given in your case it won't make much sense to have to install the whole tower in you node-side just to update teos-common.
In order to make this work you should:

Checkout #305
Link the plugin to you cln plugins (fell free to pip install -r requirements.txt just in case)
Update teos-common by running COMMON_ONLY=1 pip install . from the python-teos folder.

Notice this won't be needed once the new version of teos-common is published in PyPi since the updated version of the plugin will pull from that.

okay, stopping watchtower.py (plugin stop) killed my lightningd:
022-02-09T19:15:37.998Z INFO    plugin-watchtower.py: Killing plugin: stopped by lightningd via RPC
2022-02-09T19:15:37.999Z **BROKEN** lightningd: Already in transaction from lightningd/jsonrpc.c:948
2022-02-09T19:15:38.049Z **BROKEN** lightningd: FATAL SIGNAL 6 (version v0.10.2)

But now lightningd runs with the updated watchtower.py - let's see how it performs! thanks!

No worries. Thank you for taking the time, the feedback is being really useful.
Feel free to open an issue regarding the plugin stop, I'm unsure if that was an option at the time the plugin was created, but certainly no management is being done by the plugin on that, so I'll be happy to look into it and add that bit for a next version.

@svewa have you face any other issue with this?
I'll close otherwise.

No, did not happen again. Thanks!


added command to add contract to the flow.json

#88
Description
Command to add an contract to the project struct

For contributor use:

[ ] Targeted PR against master branch
[ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work
[ ] Code follows the standards mentioned here
[ ] Updated relevant documentation
[ ] Re-reviewed Files changed in the Github PR explorer
[ ] Added appropriate labels

I see here that i used args and not flags, that change is the last commit so we might want to revert that. IMHO this reads better with args, since it is just key=value, but if you want to add an alias to testnet, mainnet I can see the argument for flags.

@bjartek great work :)
I think this args/flags dilemma will be solved when we finish the CLI guideline as at that time we will go through all commands and make them consistent. I think args is fine for now.

I tried to rebase this off newest master, but not sure if it even makes sense anymore. Should we just delete this and start anew?

I was thinking this to become part of #124 (init section in the comment) where we create a config top command that has subcommands for adding/removing/viewing config values.


Call zebra_test::init where needed.

Closes https://github.com/ZcashFoundation/zebra/issues/1224 if merged.

Blocked by #1196 due to merge conflicts:

both branches format whitespace in code inside proptest! blocks
#1196 adds new tests, and some of them might not call init

Since whitespace changes cause a lot of merge conflicts, it might be better to remove them from this PR.
If we could automate these changes, we could use the same script to add init calls after #1196 merges.

@oxarbitrage there's still a conflict with #1196, would you like to remove the whitespace changes in this PR, or rebase after #1196 merges?

@oxarbitrage I merged #1196 today, can you rebase this PR on the latest main branch?
#1196 adds some new tests, and moves some tests around.
You can squash and force-push if you'd like. There's no need to keep the commit history for this PR.


useAllianceColor in any BaseAutoBuilder is mirroring the red alliance robot over the x axis instead of the y.

When using RamseteAutoBuilder and setting useAllianceColor to true, the red alliance paths are being mirrored over the x axis of the field instead of the y. We first thought this was a problem with our odometry but looking at the PathPlannerTrajectory class it seems that the field width and height are flipped.
Here's a chiefdelphi post with more info: https://www.chiefdelphi.com/t/pathplanner-trajectory-mirrors-on-x-instead-of-y/423757

This is working as expected. Mirroring over Y would not transform the path for the red alliance. Due to the non-standard field mirroring, it needs to be mirrored over the X axis. Also, since (0, 0) should always be on your side of the field regardless of alliance color, it should not be mirrored over Y to preserve the correct X coordinates.


Check for null default on array

@vnen not sure if this is the right way to deal with this but it wouldn't allow me to set the default to null so just setting it to an empty array..
Fixes:
Exception: Array default value not implemented: type: Array, value: null:
  File "D:\Development\xr_plugins\godot_openvr.4.0\godot-cpp\SConstruct", line 415:
    binding_generator.generate_bindings(json_api_file, env['generate_template_get_node'])
File "D:\Development\xr_plugins\godot_openvr.4.0\godot-cpp\binding_generator.py", line 39:
    header = header_func(used_classes, c, use_template_get_node)
  File "D:\Development\xr_plugins\godot_openvr.4.0\godot-cpp\binding_generator.py", line 310:
    method_signature += " = " + parse_default_value(argument["type"], argument["default_value"])
  File "D:\Development\xr_plugins\godot_openvr.4.0\godot-cpp\binding_generator.py", line 1960:
    raise Exception(f"Array default value not implemented: type: {type_name}, value: {value}")

The thing is that is an issue in some binding. If the type is Array the value should never be null. I fixed one case that I found in https://github.com/godotengine/godot/pull/49600, so if there's another it's better to fix the binding.

Yeah indeed it didn't make sense to me at all that it was null. That said I think it shouldn't fail so readily. We should probably change the code in the bindings export in Godot to give a warning an not store a default.
Though I guess it's all academic now that Reduz is rewriting it all :)

I guess this is moot now since we have a different generator


Multiple issues related to multi-level inheritance

I've discovered a few issues that occur under very specific inheritance scenarios. I haven't been able to debug them yet, but I think I've found a way to accurately reproduce them (my tests were performed using msvc140, built in the default Debug x64 configuration, using a dynamically linked lua 5.3).
Both crashes are related to constructing inheriting bound usertypes from lua land (happens on both usertypes and simple_usertypes). I'll describe the two separate issues I discovered in comments down below to keep everything clean.
Here are my class definitions:
class TestClass00 {
public:
	int Thing() const { return 123; }
};

class TestClass01 : public TestClass00 {
public:
	TestClass01() : a(1) {}
	TestClass01(const TestClass00& other) : a(other.Thing()) {}

	int a;
};

class TestClass02 : public TestClass01 {
public:
	TestClass02() : b(2) {}
	TestClass02(const TestClass01& other) : b(other.a) {}
	TestClass02(const TestClass00& other) : b(other.Thing()) {}

	int b;
};

class TestClass03 : public TestClass02 {
public:
	TestClass03() : c(2) {}
	TestClass03(const TestClass02& other) : c(other.b) {}
	TestClass03(const TestClass01& other) : c(other.a) {}
	TestClass03(const TestClass00& other) : c(other.Thing()) {}

	int c;
};

And the usertype definitions:
sol::usertype<TestClass00> s_TestUsertype00(
	sol::call_constructor, sol::constructors<sol::types<>>(),
	"Thing", &TestClass00::Thing
);

lua->set_usertype("TestClass00", s_TestUsertype00);

sol::usertype<TestClass01> s_TestUsertype01(
	sol::call_constructor, sol::constructors<sol::types<>, sol::types<const TestClass00&>>(),
	sol::base_classes, sol::bases<TestClass00>(),
	"a", &TestClass01::a
);

lua->set_usertype("TestClass01", s_TestUsertype01);

sol::usertype<TestClass02> s_TestUsertype02(
	sol::call_constructor, sol::constructors<sol::types<>, sol::types<const TestClass01&>, sol::types<const TestClass00&>>(),
	sol::base_classes, sol::bases<TestClass01, TestClass00>(),
	"b", &TestClass02::b
);

lua->set_usertype("TestClass02", s_TestUsertype02);

sol::usertype<TestClass03> s_TestUsertype03(
	sol::call_constructor, sol::constructors<sol::types<>, sol::types<const TestClass02&>, sol::types<const TestClass01&>, sol::types<const TestClass00&>>(),
	sol::base_classes, sol::bases<TestClass02, TestClass01, TestClass00>(),
	"c", &TestClass03::c
);

lua->set_usertype("TestClass03", s_TestUsertype03);

Here's the code to reproduce the first crash:
local x = TestClass00()
local y = TestClass02(x)

And it produces the following exception (similar to #166) on line 2 of the lua script: sol: attempt to index (get) nil value "├óΓäó┬ío├»┬╜┬í.(├ó┼ô┬┐├á┬╕┬║├»┬╜┬í ├ó┼ô┬┐├á┬╕┬║)" on userdata (bad (misspelled?) key name or does not exist).
I have also tried other variations of defining the usertypes, where I wouldn't define the constructors for inherited classes assuming they would be inherited from base classes, but that produced the same issue.

The second issue can be reproduced using the following code:
local x = TestClass01()
local y = TestClass03(x)

The issue occurs again on line 2, however it causes a crash, not an exception, as it seems to call the wrong constructor (TestClass03::TestClass03(const TestClass02& other) instead of TestClass03::TestClass03(const TestClass01& other)), and it also passes the other value as a null struct. It therefore crashes with the error other was nullptr..

I understand that 'upcasting' classes like that can be considered bad practice, however in my use case it is necessary due to lack of a better alternative.

Well, I think I fixed it all. I think.

Awesome! If you commit it I can try it out on some more advanced and real-world use cases. :)

The first issue seems to be fixed, however I'm still experiencing the second one. Trying to figure out if there's something wrong with my environment setup and I'll report back.
With that being said, I seem to be having a new issue now which appears to occur when a usertype with a destructor is being destroyed from sol::detail::user_alloc_destroy. I've experienced this issue twice so far, one where the error is __imp_lua_touserdata(...) returned nullptr. and the other where I'm getting a reading violation in my destructor (looks like the this pointer is set to nullptr).

That usually indicates that I've done something wrong and there's a double delete... if you can figure it out, file a new issue. I'm gonna mark this one fixed.
https://github.com/ThePhD/sol2/releases/tag/v2.14.2

Okay, I'll try again with the latest version as I was using the one from this commit. If I'm still experiencing issues I'll let you know. Again, thanks for your amazing work!


The tests are very slow / take a long time to run locally

Describe the bug
When the tests are run as part of the PR checks by CircleCI, the (265 + 1233 =) 1488 tests appear to take less than 8 minutes using pytest -n 2, see, for example, the output from our last PR, which was produced using ESMValCore v2.4.0 on November 23, 2021.
When I ran the tests locally a few months ago using ESMValTool v2.3.0, the tests took ~8 minutes.
However, now, when I run the tests locally using ESMValTool v2.4.0, the (267 + 1228 =) 1495 tests take ~34 minutes!
Has anyone else noticed this?

There were changes to test_cmorize_obs.py between v2.3.0 and v2.4.0 related to the creation of the dummy data. Would it be possible to reconsider these changes?
Also, I wonder whether it would be worth marking, e.g. the integration tests so they can be run separately / excluded from unit tests, using, e.g. -m 'not integration'?

hi @ehogan if you look at CircleCI and Github Actions tests take about 4-5min which is about right for an average CPU and running with 2 cores. You can port the test monitor Action I implemented for ESMValCore and see which tests are hogging the time (I was meaning to get that Action in here too, at some point). I would not run only the unit tests during any given test session, they don't catch everything that needs to be checked :+1: Out of curiosity, why did you assign Autoassess label to this issue?

The tests are also taking a long time for me to run.

@ehogan @tepmo42 please have a look at this test monitor: https://github.com/ESMValGroup/ESMValTool/runs/4716222019?check_suite_focus=true Github Action (see the parse_pymon.py run towards the bottom): for Python 3.9 the slowest test is indeed the WOA cmorizer test at about 33s - this is slow (and I'll have a look at it see if I can make it faster) but it's nowhere near as slow as you are reporting it to be - it's about 40% slower on OSX, but again, 44 odd seconds, not 1700s! So there is either something wrong with your threads or you are using a shared machine - if you are using a shared machine then reporting runtimes is irrelevant unless done through many measurements and getting an average + STD :+1:

Happy new year @valeriupredoi! :) Thank you for your comments and the evidence that the test should take ~30 seconds to run! I'm not using a shared machine, so I'm not sure what is going on :/ Let me investigate a bit more and I'll get back to you :)
(I assigned the AutoAssess label to this as I mention running the tests during the tutorial we've developed for converting AutoAssess metrics and I just wanted to keep track of this issue incase I have to make changes to the tutorial based on the outcome of this issue) :)

Happy New Year, Emma! sure, please have a looksee and ping me if you think something's fishy :fish:

pytest header from test monitor:
platform linux -- Python 3.9.9, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/runner/work/ESMValTool/ESMValTool, configfile: setup.cfg
plugins: forked-1.4.0, metadata-1.11.0, xdist-2.5.0, flake8-1.0.7, html-3.1.1, env-0.6.2, cov-3.0.0, monitor-1.6.3

pytest header from my Linux machine:
platform linux -- Python 3.9.7, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/ehogan/software/ESMValTool/ESMValTool, configfile: setup.cfg
plugins: cov-3.0.0, env-0.6.2, flake8-1.0.7, forked-1.3.0, metadata-1.11.0, html-3.1.1, xdist-2.4.0

So it looks like the versions of Python, forked and xdist are different (and monitor is missing).
I created an environment to match the environment used by the tests monitor but it made no difference to the timings :(

@valeriupredoi I discovered what is going on :) The problem is related to the fact that the test is attempting to download missing data:
2022-01-06 10:30:29,799 UTC [115989] INFO    Input file woa18_decav81B0_t00_01.nc is missing. Start download ...
2022-01-06 10:30:29,806 UTC [115989] DEBUG   Starting new HTTPS connection (1): www.ncei.noaa.gov:443
2022-01-06 10:30:33,098 UTC [115989] DEBUG   https://www.ncei.noaa.gov:443 "GET /data/oceans/woa/WOA18/DATA/temperature/netcdf/decav81B0/1.00/woa18_decav81B0_t00_01.nc HTTP/1.1"<PHONE_NUMBER>67
2022-01-06 10:35:12,991 UTC [115989] INFO    CMORizing var thetao from input set temperature
[...]
2022-01-06 10:35:13,466 UTC [115989] INFO    Input file woa18_decav81B0_s00_01.nc is missing. Start download ...
2022-01-06 10:35:13,474 UTC [115989] DEBUG   Starting new HTTPS connection (1): www.ncei.noaa.gov:443
2022-01-06 10:35:17,253 UTC [115989] DEBUG   https://www.ncei.noaa.gov:443 "GET /data/oceans/woa/WOA18/DATA/salinity/netcdf/decav81B0/1.00/woa18_decav81B0_s00_01.nc HTTP/1.1"<PHONE_NUMBER>64
2022-01-06 10:40:00,700 UTC [115989] INFO    CMORizing var so from input set salinity
[...]

It takes ~5 minutes to download a file, and it downloads 6 files during the test (~30 minutes). How are these files available to the test monitor? Would it be better to store all required test data with the tests?

There was an extra directory that wasn't being considered when writing the dummy data. I have fixed this in #2451 :)

@ehogan you are absolutely correct! I went inside that test and discovered it was never changed to account for the newer WOA cmorizer, I have opened https://github.com/ESMValGroup/ESMValTool/issues/2452 - I first need to make sure the input data structure is correct as requested by the cmorizer script, then I will change the test to account for those new changed paths. Many thanks for insisting on this, bug there was after all :lady_beetle: BTW about the test monitor - that records the system time, that is pretty decent for a Github machine, a bit worse for my UniReading connection, and much worse for my home (Virgin Media) home connection - but even that was something like 4min, no way 35min as you have seen - it means you have a really really slow internets :rofl: - in any case, this download shouldn't happen at all, I'll take it from here :+1:


Mcs 770 stop restart eval

Added the ability to resume a harshly stopped execution.
To test:

Run an eval.
Wait until at least 1 scene has completed successfully.
Stop the eval (ctrl + C twice works)
Rerun the eval with the --resume flag
Verify the already run scenes are not rerun.

You can also manually edit the .last_run_finished.txt file on the Ray head node.

Tested this and was able to rerun the missed scenes. Agreed about possibly coming back to this/potentially adding listeners.
One question: do we add a scene to the resume list if there's an internal error/should we? I had a one off error that I couldn't replicate (internal to mcs python package) where some video didn't get deleted properly, and it didn't look like the scene was retried, but maybe that made sense for that case.

Tested this and was able to rerun the missed scenes. Agreed about possibly coming back to this/potentially adding listeners.
One question: do we add a scene to the resume list if there's an internal MCS error/should we? I had a one off error that I couldn't replicate (internal to mcs python package) where some video didn't get deleted properly, and it didn't look like the scene was retried, but maybe that made sense for that case.

If a scene finishes with error, but isn't supposed to retry, it will be added to the "finished" list and thus won't be run in the resume.


[KBM] Specify multiple targets applications

📝 Provide a description of the new feature
It would be great to have the possibility to specify multiple targets in "Remap shortcuts" (comma separated, semicolon separated or space separated)

If you'd like to see this feature implemented, add a 👍 reaction to this post.

also, i could really use an All Apps EXCEPT ____ feature


Add support for AWS profile and unsigned requests as command line arguments

Add option --profile for a specific profile in the AWS credentials file #151
Add support for unsigned requests #181

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the Developer Certificate of Origin (DCO).

We need to write tests for this, as they would have caught the missing bootstrap in the profile credentials provider.
Test mounting an anonymous-readable bucket
Test specifying profile provider with STS credentials

#191 for the followup on testing.

Thank you, all, for your work on this. I rebuilt the latest version and --profile works great so far.


Displaying timestamp columns in cassandra is broken

Cassandra 3.0 driver ships with support for converting timestamp columns to/from java.util.Date but unfortunately they don't support their concrete implementation of Date, com.datastax.driver.core.LocalDate.
I imagine they'll fix this eventually but for now, DBeaver is broken when viewing any tables that use the timestamp column type.

Repro:

create a new table
create TABLE footest ( timestam timestamp PRIMARY KEY , value text );
insert some data
INSERT INTO cass_tests.footest (timestam, value) VALUES (1, 'foo');
view data with DBeaverEE
failz :(

Thanks for report.
This issue will be fixed in the next version.

some issue here - any workaround available?
adding the error message as text to make it searchable:
com.datastax.driver.core.exceptions.CodecNotFoundException: Codec not found for requested operation: [timestamp <-> com.datastax.driver.core.LocalDate]


Fix missing namespace in server/client macros

Maybe there are more left.
But I'm not using yojimbo namespace in any of my headers and it works now.

Thanks!


Added removing specific SRP service before adding new one.

Problem
SRP doesn't allow to update  services, so in case of registering multiple times the same service (e.g. because of updating its
record content) only first attempt will succeed and the following will end with OT_DUPLICATED result.
Summary of Changes

In MdnsImpl added removing specific service before adding it,to make sure that it won't exist in the moment of adding its the
most present version.

Fixes https://github.com/project-chip/connectedhomeip/issues/6335

Problem
SRP doesn't allow to update services, so in case of registering multiple times the same service (e.g. because of updating its
record content) only first attempt will succeed and the following will end with OT_DUPLICATED result.

This sounds like a limitation in openthread, not in SRP itself.  If so, shouldn't we be filing a bug there?  It seems to me that if you have the private key used to create the initial record, you should be able to update it (and we must have the key if we can delete it anyway).
Deleting-then-publishing creates a window where a bad actor can steal the name reservation and populate with its own key that the original name holder now cannot remove.  This is also extra traffic, isn't it?

@msandstedt

This sounds like a limitation in openthread, not in SRP itself. If so, shouldn't we be filing a bug there?

I already talked with the SRP implementers about missing service updating feature, but I heard that this is actually how SRP protocol works like and remove-readd service pattern is suggested way of making updates. That is also what IETF SRP spec says:
`  <IP_ADDRESS>.2.  Removing some published services
The second alternative is used when some service is being replaced by
a different service with a different service instance name.  In this
case, the old service is deleted as in the first alternative.  The
new service is added, just as it would be in an update that wasn't
deleting the old service.  Because both the removal of the old
service and the add of the new service consist of a valid Service
Discovery instruction and a valid Service Description instruction,
the update as a whole is a valid SRP update, and will result in the
old service being removed and the new one added, or, to put it
differently, in the old service being replaced by the new service.
It is perhaps worth noting that if a service is being updated without
the service instance name changing, that will look very much like the
second alternative above.  The difference is that because the target
for the PTR record in the Service Discovery instruction is the same
for both the Delete An RR From An RRset update and the Add To An
RRSet update, these will be seen as a single Service Description
instruction, not as two instructions.  The same would be true of the
Service Description instruction.`

Deleting-then-publishing creates a window where a bad actor can steal the name reservation and populate with its own private key

I can't really comment if it is a real threat, but seems to me to be worth considering. I can imagine that thanks to the OpenThread feature: https://github.com/openthread/openthread/blob/main/src/core/config/srp_client.h#L162 we could try to send remove and add commands in one message to provide better security, but it doesn't sound like 100% safe solution, as some activity in application may impact on separating this transactions. There is also another threat that I already reported, which is resetting SRP server, what results in removing all saved services and then re-adding it fast by bad actor, before clients will manage to do that again.
I think it will be the best to discuss it in wider group of people, so I created an issue to OpenThread: https://github.com/openthread/openthread/issues/6586. Thanks for reporting the problem @msandstedt and feel free to comment on that in the issue.

@wgtdkp , I think the test you did is behaving as expected, i.e., server is accepting and only remembering the second entry.
The adding of two separate entries with same instance name on the client side is actually incorrect use of the client APIs. We can add checks in OT implementation for this (i.e., reject entries with same name being added again) if we think this may help avoid bad use by the next later. Note that in general, it is expected that the next layer code would manage the SRP service entries. The general idea we adopted on extra validations on OT SRP clinet implementation (as we disucssedin some earlier reviews)  has been to limit and be selective on the extra check/validation to keep the implementation lean and have smaller code size (this is important since the SRP client module needs to run on all types of devices including resource-limited sleepy end devcies). But in this case I think it may be worth it to add a check to protected against incorrect use.
On managing the SRP KEY, note that the user of OT SRP clinet APIs is not involved at all. It is fully manged by ot::Srp::Clinet itself.

Per spec, the KEY on client is actually associated with a device (not specific entries).
The key is persisted (saved in non-volatile) so even after device reset, it would use the same KEY.
Only a factory-reset will wipe a previous key and then cause a new one to be generated on next use (basically a factory-reset of a device will correctly make it look as if it is a new/different device).


Update LocalizedError to use more of the protocol and extend custom error type.

This is a good starting point http://stackoverflow.com/a/39176551/2280737

Here's another example:
enum TaskListError : LocalizedError {

    case noJSONinBody
    case noDescriptionInBody
    case descriptionTooShort(String)
    case descriptionTooLong(String)

    var errorDescription: String? {
        switch self {
        case .noJSONinBody: return "Must include JSON in the body"
        case .noDescriptionInBody: return "Must have the task description"
        case .descriptionTooShort(let string): return "\(string) too short"
        case .descriptionTooLong(let string): return "\(string) too long"
        }
    }
}

extension TaskListError: StringValuePairConvertible {
    var stringValuePairs: StringValuePair {
        return ["error": self.errorDescription ?? ""]
    }
}

Using the example above for TaskListError, do we need a case for every instance that the LocalizedError is used?  Will it be possible for someone to define a unique error message without adding a new case to the enum?
`enum BluePicLocalizedError : LocalizedError {
case noImagesByTag

var errorDescription: String? {
    switch self {
    case .noImagesByTag: return "Failed to find images by tag." ...`

Both of the lines of code below compile-- is this legit?
response.error = BluePicLocalizedError.noImagesByTag
response.error = BluePicLocalizedError(errorDescription: "Failed to retrieve all images.")

I would do this instead:
//...
case noImagesByTag(String)

var errorDescription: String? {
    switch self {
    case .noImagesByTag(let tag): return "Failed to find images by tag \(tag)..."

You could add another case called other that could allow BluePic to throw some new error. Although, I'm kind of against this. I think a finite number of enumerable errors make for a more robust design.

If we do unique error cases, how many do you think there will be?

There will be 13 cases with the current code base.

Hmm, if there is no way to consolidate some of them, then we should make 13 cases.

Fixed by PR: #326


Update notification banner for Sept v10 launch

Summary
This PR updates the notification banner with a callout to the v10 Release Blog.
Ticket Link
n/a

@emdecr - Generated previews of this docs PR are showing the fallback announcement text.

@cwarnermm Hah! I could have sworn it updated 😂 How about now? I didn't change anything related to the banner in my recent, but the update to the main script seems to have done it? Thanks!


npm update doesn't update

$ npm list
├──<EMAIL_ADDRESS>extraneous
$ npm update
$ npm update director
$ npm list
├──<EMAIL_ADDRESS>extraneous
$ npm install director
npm http GET https://registry.npmjs.org/director
npm http 304 https://registry.npmjs.org/director
npm http GET https://registry.npmjs.org/director/-/director-1.0.9-1.tgz
npm http 200 https://registry.npmjs.org/director/-/director-1.0.9-1.tgz
<EMAIL_ADDRESS>./node_modules/director
$ npm list
├──<EMAIL_ADDRESS>extraneous
// Mac OS X 10.7.3
$ npm -v
1.1.1
$ node -v
v0.6.10

This is because 'npm update' works on the basis of your declared dependencies. Since 'director' is extraneous (i.e. not referenced from your package.json), npm does not try to update it. Add it to your package.json, and 'npm update' will update it as you would expect.

I see. Thanx.


help : turn off xlsx worksheet protection function

Hi
I would like to open an excel file and disable the worksheet protection as in the picture.
I have downloaded the craXcel.py file but have no clue how to start to unlock the excel via python.
Since all of the worksheet is protected therefore pd.read_excel and openpyxl.load_workbook('NXTS122ND8S10.xlsx')  cannot read the excel.
Any help would be appreciated.
Thank you.
NXTS122ND8S10.xlsx

Hi there.
Is the problem that you're not sure how to use python/command-line applications?
If so, what you need to do is:

Download Python https://www.python.org/downloads/
Clone this repository (downloading zip might be easiest - https://github.com/petemc89/craXcel-cli/archive/refs/heads/master.zip)
Put your file in the folder with the code
Open a command line (cmd, powershell etc)
Type: python craxcel.py name-of-file.xlsx

Ooooor, you can use www.craxcel.com and not have to worry about any of that :)

Hi,
I will follow the instruction.
Thank you for your help.


SliderEx's step doesn't work correctly

https://github.com/zeozeozeo/microui-go/blob/f386f91fa9b00aaccf6c292c891c30b1d3e5d905/controls.go#L268
Now any step values doesn't have any effects. This should be
v = math.Round(v/step) * step

This was reported by @galsjel at the Ebitengine Discord server.

https://github.com/rxi/microui/commit/8fe5267fc5d5b4fd925fb8822c2f08e362192be9
Actually the latest microui has a similar fix. This casts a float value to an integer value, which should be the same as math.Floor or math.Ceil. (This was found by @Zyko0 )


[locale] fo: relativeTime is incorrect

Description of bug
The "one minute" and "one month" values in the Faroese locale are wrong.
In both cases, the word should be in singular indefinite nominative form.
At the moment, the value for minute is in accusative form (minutt) and month is in dative form (mánaði).
A free Faroese dictionary can be found here: Sprotin.fo
In /locale/fo.js, the relativeTime object should be changed as follows:
relativeTime: {
  m : 'ein minuttur',
  M : 'ein mánaður',
}

Steps to reproduce bug:
Create moment instance with the locale 'fo' and call fromNow() when the time difference is either one minute or one month.

@ragnar123 what do you think?
@Sakarisson can you make a PR in the mean time?

@ichernev I can create a PR. I'll tag you in it if it's okay. Probably won't be until next week


[Bug]: Importing specific internal app after forking it fails to launch or edit in release

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
Internal app GitHub Manager fails to launch or edit after forking it from app.appsmith.com, export and then import on release.
The app opens up edit and launch modes on prod but with a lot of delay.
https://app.appsmith.com/applications/60d72903ee9aa1792c5bf168/pages/614d9b9e18a27e5288afa99d

Steps To Reproduce

Fork GitHub Manager app on app.appsmith.com and observe the 500: internal server error as seen in the video and refresh page to see the app displayed in selected personal folder as expected despite error displayed
Export this app
Import this app on release.appsmith.com
Click on Edit or Launch on app card observe the app behaviour (on release)
Repeat step 4 on prod and observe that the app loads

Environment
Release
Version
Cloud

Abhijeet's changes have fixed this.


Remake Reference in data model into Prop (and its Part and Type in 2nd-level Props) in S-R and T-R relations

Under each Resource for a Statement, we now have Part as a free text field, and Type primary/secondary as dictionary-based value.
But we need to remake the data model here because completely similar structures appear in Detail, where metaprops are the only reasonable place to store it.
There, the structure must be:
MP1: T this text - A has - C edition - R Duvernoy, 2001 ||| MP1.1: MP1 - A has - C2320 page or folio range - V 12-21
For symmetry with those metaprops we need, and for fuller semantics, we need that:

The relation between S and R becomes a prop (GUI implicit, data model explicit; Statement prop, not action prop), i.e.: Statement - has - C resource - R XYZ, of which the user should only see the "R XYZ" part in the GUI, and in the same section as now.
2nd-level prop is allowed underneath this prop-type relation, which will allow to hold part - with prop type C2320 page or folio range prefilled, and V as pre-selected entity type.
Therefore, it also makes sense to make the Type primary/secondary a 2nd-level prop, but keeping the current assistance, i.e. solving it in the GUI by the simple switch. But in data model terms, it will be a 2nd-level prop.

Unfortunately needed to start working and for parsing, i.e. putting it into 1.3.0.

@davidzbiral also here - please give me a call ideally to explain what is needed to be achieved.

Why not to go the References way in Detail of Ts
During a call, @adammertel suggested making it coherent the other way around, i.e. under Detail of a T, have a new References section similar to what we have in Editor (see below).
This will not work completely well because there is (1) several references we need to make under a T: to (a) manuscript witness 1-n, (b) printed edition 1-N, c) digital edition 1-n, so there would be several forms of representation of a T, and we will need to extend and change the dictionaries as we go (e.g. we would like to create "digital edition in XML/TEI" as a subclass of "digital edition" as we find one). I.e., we editors would appreciate the flexibility of props here, i.e. (i) creating them as we go without asking somebody for dictionary updates, and (ii) relating the prop types semantically (e.g. "digital edition in XML" - has - superclass - digital edition; which in turn has the superclass of "edition".

But I understand the intention to keep the GUI and the data model close (even if we will need more assistance since 1.4 anyway, which will make the database (JSON) and the GUI diverge anyway). Therefore, OK, let us go with Adam's solution - handled in issue https://github.com/DISSINET/InkVisitor/issues/797. Closing.

@adammertel and @davidzbiral debated this on 3 March 2022.
Results:

In T-R relations, we will do it with metaprops, i.e. as I write (incl. 2nd-level props for page ranges etc.).
In S-R relations, however, we will keep the current system. Adam acknowledges that it is structurally quite similar to the T-R relation set out above, but sees DDB1 as collected data database, not primary communication format, and wants it to be as easy as possible and as close as possible to an efficient GUI; he sees bringing this S-R relation in line with T-R relations as a candidate for DDB2 transformation.
We agreed on this. Closing.