Clean up repository's files

Clean-up tasks:

[X] Move demo file to a demos directory (done in 65cdef7)

[X] Remember to mark path as documentation for Linguist (done in 4cc7de0)

[X] Extend .gitignore to cover other sorts of common junk (done in 05a0a1c)
[X] Add .editorconfig (done in cefe2bc)

dusts hands


Update avira-antivirus to latest

After making all changes to the cask:

[x] brew cask audit --download {{cask_file}} is error-free.
[x] brew cask style --fix {{cask_file}} left no offenses.
[x] The commit message includes the cask’s name and version.

The script needs to be run by the superuser, otherwise it barely uninstalls anything.


Multiple Connections

I am writing a benchmark / stress tester. I'm currently running into an issue that I can not connect more than two clients at a time. My test is based off of the Echo test.

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using LiteNetLib;
using LiteNetLib.Utils;

namespace LibSample
{
    internal class BenchmarkTest
    {
        static int _messageSize = 13218;

        internal class ClientListener : INetEventListener
        {
            internal int MessagesReceivedCount ;
            internal DateTime StartTime;
            internal DateTime StopTime;
            internal bool Connected;
            internal int Errors;

            internal Stopwatch _watch = new Stopwatch();

            public NetClient Client;

            byte[] testData = new byte[13218];

            public void OnPeerConnected(NetPeer peer)
            {
                Connected = true;
            }

            public void OnPeerDisconnected(NetPeer peer, DisconnectReason disconnectReason, int socketErrorCode)
            {
                Connected = false;
            }

            public void OnNetworkError(NetEndPoint endPoint, int socketErrorCode)
            {
                Console.WriteLine("[Client] error! " + socketErrorCode);
                Errors++;
            }

            public void OnNetworkReceive(NetPeer peer, NetDataReader reader)
            {
                if (reader.AvailableBytes == _messageSize)
                {
                    MessagesReceivedCount++;

                    _watch.Stop();

                    Send();
                }
            }

            public void OnNetworkReceiveUnconnected(NetEndPoint remoteEndPoint, NetDataReader reader, UnconnectedMessageType messageType)
            {
                Console.WriteLine("[Client] ReceiveUnconnected: {0}", reader.GetString(100));
            }

            public void OnNetworkLatencyUpdate(NetPeer peer, int latency)
            {

            }

            void Send()
            {
                var peer = Client.Peer;

                _watch.Start();

                peer.Send(testData, SendOptions.ReliableOrdered);
            }

            private int _threadId;

            public void Start()
            {
                StartTime = DateTime.UtcNow;
                _threadId++;

                ThreadPool.QueueUserWorkItem(PollInternal, _threadId);

                Send();
            }

            void PollInternal(object state)
            {
                var id = (int)state;

                while (id == _threadId)
                {
                    Client.PollEvents();
                    Task.Yield();
                }
            }

            public void Stop()
            {
                StopTime = DateTime.UtcNow;
                _watch.Stop();
                _threadId++;
            }
        }

        internal class ServerListener : INetEventListener
        {
            internal int MessagesReceivedCount;
            internal DateTime StartTime;
            internal DateTime StopTime;
            internal int Errors;
            internal bool Continue;

            public NetServer Server;

            public void OnPeerConnected(NetPeer peer)
            {
                Console.WriteLine("[Server] Peer connected: " + peer.EndPoint);
            }

            public void OnPeerDisconnected(NetPeer peer, DisconnectReason disconnectReason, int socketErrorCode)
            {
                Console.WriteLine("[Server] Peer disconnected: " + peer.EndPoint + ", reason: " + disconnectReason);
            }

            public void OnNetworkError(NetEndPoint endPoint, int socketErrorCode)
            {
                Console.WriteLine("[Server] error: " + socketErrorCode);
                Errors++;
            }

            public void OnNetworkReceive(NetPeer peer, NetDataReader reader)
            {
                MessagesReceivedCount++;

                //echo
                peer.Send(reader.Data, SendOptions.ReliableUnordered);
            }

            public void OnNetworkReceiveUnconnected(NetEndPoint remoteEndPoint, NetDataReader reader, UnconnectedMessageType messageType)
            {
                Console.WriteLine("[Server] ReceiveUnconnected: {0}", reader.GetString(100));
            }

            public void OnNetworkLatencyUpdate(NetPeer peer, int latency)
            {

            }

            private Thread _thread;
            private int _threadId;

            public void Start()
            {
                StartTime = DateTime.UtcNow;
                _threadId++;

                _thread = new Thread(PollInternal);

                _thread.Start();
            }

            void PollInternal()
            {
                var id = _threadId;

                while (id == _threadId)
                {
                    Server.PollEvents();
                    Task.Yield();
                }
            }

            public void Stop()
            {
                StopTime = DateTime.UtcNow;
                _threadId++;
                Server.Stop();
            }
        }

        private int _clientCount = 3;
        internal List<ClientListener> _clients;
        internal ServerListener _serverListener;

        public void Run()
        {
            //Server
            _serverListener = new ServerListener();

            NetServer server = new NetServer(_serverListener, 2, "myapp1");
            if (!server.Start(9050))
            {
                Console.WriteLine("Server start failed");
                Console.ReadKey();
                return;
            }
            _serverListener.Server = server;

            //Clients
            _clients = new List<ClientListener>(_clientCount);

            for (int i = 0; i < _clientCount; i++)
            {
                var _clientListener = new ClientListener();
                var client1 = new NetClient(_clientListener, "myapp1");
                client1.SimulationMaxLatency = 1500;
                client1.MergeEnabled = true;
                _clientListener.Client = client1;
                if (!client1.Start())
                {
                    Console.WriteLine("Client1 start failed");
                    return;
                }
                _clients.Add(_clientListener);
                client1.Connect("<IP_ADDRESS>", 9050);
            }

            // Wait
            bool waitForConnection = true;
            while (waitForConnection)
            {
                Thread.Sleep(15);

                Console.WriteLine("Waiting:: Errors " + _serverListener.Errors + " Peers " + _serverListener.Server.PeersCount);

                if (_serverListener.Errors > 0)
                    return;

                waitForConnection = _serverListener.Server.PeersCount < _clientCount;
            }

            Console.WriteLine("Ready. Press any key to start");
            Console.ReadKey();
            Console.WriteLine("Starting");

            var checker = new HealthChecker();
            checker.Start(this);

            //Send

            _serverListener.Start();

            foreach (var client in _clients)
            {
                client.Start();
            }

            while (!Console.KeyAvailable)
            {
                Thread.Sleep(15);

            }

            Console.WriteLine("DONE");

            _serverListener.Stop();

            foreach (var client in _clients)
            {
                client.Stop();
            }

            server.Stop();

            Console.WriteLine("Press any key to exit");
            Console.ReadKey();
        }
    }

    internal class HealthChecker
    {
        private int _threadId;
        private BenchmarkTest _context;

        public void Start(BenchmarkTest context)
        {
            _context = context;
            _threadId++;
            PollInternal();
        }

        public void Stop()
        {
            _threadId++;

            PrintServer();
            PrintClient();
        }

        async void PollInternal()
        {
            var id = _threadId;

            while (id == _threadId)
            {
                await Task.Delay(TimeSpan.FromSeconds(15));
                PrintServer();
                PrintClient();
            }
        }

        public void PrintServer()
        {
            var fmt = "[{5}] CON {0} MSG {1} TIME {2} OPS {3} LAG {4} ERR {5}";

            var span = DateTime.UtcNow.Subtract(_context._serverListener.StartTime);

            Console.WriteLine(
                fmt,
                _context._serverListener.Server.PeersCount,
                _context._serverListener.MessagesReceivedCount,
                span.TotalSeconds,
                _context._serverListener.MessagesReceivedCount / span.TotalSeconds,
                "NA",
                _context._serverListener.Errors,
                "SERVER");
        }

        public void PrintClient()
        {
            var fmt = "[{5}] CON {0} MSG {1} TIME {2} OPS {3} LAG {4} ERR {5}";

            var span = DateTime.UtcNow.Subtract(_context._serverListener.StartTime);

            var cons = _context._clients.Count;
            var msg = _context._clients.Sum(o => o.MessagesReceivedCount);
            var ops = msg / span.TotalSeconds;
            var lag = _context._clients.Sum(o => o._watch.ElapsedMilliseconds) / msg;
            var err = _context._clients.Sum(o => o.Errors);

            Console.WriteLine(
                fmt,
                cons,
                msg,
                span.TotalSeconds,
                ops,
                lag,
                err,
                "CLIENT");
        }
    }
}

That said, here are my initial results with two clients

[0] CON 2 MSG 1896 TIME 15.0026217 OPS<PHONE_NUMBER>68599 LAG NA ERR 0
[0] CON 2 MSG 1894 TIME 15.0036247 OPS<PHONE_NUMBER>18878 LAG 15 ERR 0
[0] CON 2 MSG 3805 TIME 30.0080228 OPS<PHONE_NUMBER>86095 LAG NA ERR 0
[0] CON 2 MSG 3805 TIME 30.0080228 OPS<PHONE_NUMBER>86095 LAG 15 ERR 0

search for "private int _clientCount = 3;"

Look at second argument of your NetServer - this is max clients.
https://github.com/RevenantX/LiteNetLib/blob/master/LiteNetLib/NetServer.cs#L20

And if you want very high performance - use "server.UnsyncedEvents = true" and "client.UnsyncedEvents = true". But this requires make your code thread safe, because it will don't sync incoming queues.

That was it, thanks

Will update my tests and repost with some results.

Or without UnsyncedEvents - send messages not in Receive method (because it called when you use Poll). Send them from another thread for correct test.

Reoppening, as I am still having issues related to many connections.
5 Connections

[0] CON 5 MSG 4597 TIME 15.0013546 OPS<PHONE_NUMBER>82656 LAG NA ERR 0
[0] CON 5 MSG 4597 TIME 15.0023566 OPS<PHONE_NUMBER>73399 LAG 16 ERR 0
[0] CON 5 MSG 9271 TIME 30.004077 OPS<PHONE_NUMBER>10036 LAG NA ERR 0
[0] CON 5 MSG 9269 TIME 30.004077 OPS<PHONE_NUMBER>02138 LAG 16 ERR 0
[0] CON 5 MSG 13799 TIME 45.0070481 OPS<PHONE_NUMBER>49875 LAG NA ERR 0
[0] CON 5 MSG 13795 TIME 45.0070481 OPS<PHONE_NUMBER>80978 LAG 16 ERR 0

10 Connections

Synced time test: 1/10/2017 2:46:27 PM
[Server] Peer connected: <IP_ADDRESS>:50567
[Server] Peer connected: <IP_ADDRESS>:50568
[Server] Peer connected: <IP_ADDRESS>:50569
[Server] Peer connected: <IP_ADDRESS>:50570
[Server] Peer connected: <IP_ADDRESS>:50571
[Server] Peer connected: <IP_ADDRESS>:50572
[Server] Peer connected: <IP_ADDRESS>:50573
[Server] Peer connected: <IP_ADDRESS>:50574
[Server] Peer connected: <IP_ADDRESS>:50575
[Server] Peer connected: <IP_ADDRESS>:50576
Waiting:: Errors 0 Peers 10
Ready. Press any key to start
Starting
[Server] Peer disconnected: <IP_ADDRESS>:50574, reason: Timeout
[0] CON 9 MSG 2656 TIME 25.5172401 OPS<PHONE_NUMBER>04877 LAG NA ERR 0
[0] CON 10 MSG 2646 TIME 25.5187418 OPS<PHONE_NUMBER>82907 LAG 96 ERR 0
[Server] Peer disconnected: <IP_ADDRESS>:50567, reason: Timeout
[Server] Peer disconnected: <IP_ADDRESS>:50568, reason: Timeout
[Server] Peer disconnected: <IP_ADDRESS>:50569, reason: Timeout
[Server] Peer disconnected: <IP_ADDRESS>:50570, reason: Timeout
[Server] Peer disconnected: <IP_ADDRESS>:50571, reason: Timeout
[Server] Peer disconnected: <IP_ADDRESS>:50572, reason: Timeout
[Server] Peer disconnected: <IP_ADDRESS>:50573, reason: Timeout
[Server] Peer disconnected: <IP_ADDRESS>:50575, reason: Timeout
[Server] Peer disconnected: <IP_ADDRESS>:50576, reason: Timeout

I did update the code as you suggested
https://gist.github.com/NVentimiglia/0cec23e985c12d0166043c44585bacdd

When using UnsyncedEvents - you don't need to call PollEvents. You can remove threads code that just using 100% cpu.
But i don't recommend use UnsyncedEvents. Better send messagess from another thread as i said before.

Corrected code for UnsyncedEvents https://gist.github.com/RevenantX/f212abd189a6f281a330c68aa6f828a6

If you reduce UpdateTime of Client and Server to 1 msec - you will get better results. Echo stress test depends on UpdateTime, because Library doing internal reliability logic with this ticks. But if you will send messagess from another thread both from client and server - you will get real power :)

Without sending for another thread.

[0] CON 10 MSG 50145 TIME 15.004567 OPS 3341.98247773495 LAG NA ERR 0
[0] CON 10 MSG 50145 TIME 15.0055579 OPS 3341.76178814385 LAG 2 ERR 0
[0] CON 10 MSG 101071 TIME 30.0090667 OPS 3368.01544048019 LAG NA ERR 0
[0] CON 10 MSG 101062 TIME 30.0090667 OPS 3367.7155311198 LAG 2 ERR 0

This is about equal to my TCP DotNetCore solution, though that does include a more robust routing (RPC) and serialization layer as well. We might be able to do better.
The updateTime of 1ms smells to me. What is this doing? Is this some sort of 'return to main thread' logic ?
I think we might need to think about how we want to handle threading in the future as it looks like a pretty easy gotcha to bottleneck on. Would you recommend manually queing the message on the main thread with the actual sending being invoked on a background thread (I ask because I did see a que internally, not sure what it is doing)

[0] CON 100 MSG 64044 TIME 15.0125366 OPS 4266.03456207394 LAG NA ERR 0
[0] CON 100 MSG 63962 TIME 15.0145386 OPS 4260.00436670095 LAG 23 ERR 0
[0] CON 100 MSG 129043 TIME 30.0218531 OPS 4298.30229233918 LAG NA ERR 0
[0] CON 100 MSG 128962 TIME 30.0218531 OPS 4295.60425768655 LAG 23 ERR 0

[0] CON 200 MSG 33453 TIME 15.0166789 OPS 2227.72293546211 LAG NA ERR 0
[0] CON 200 MSG 33261 TIME 15.0176778 OPS 2214.78982589439 LAG 90 ERR 0
[0] CON 200 MSG 69212 TIME 30.0229568 OPS 2305.30258765186 LAG NA ERR 0
[0] CON 200 MSG 69025 TIME 30.0299357 OPS 2298.53972014998 LAG 87 ERR 0

Anymore and I run out of memory on my local box.

Sending packets needs some logic, that ticks every UpdateTime (msec) (because of flow control). In version 0.6.1 if you send Unreliable packets they will be send immediately without waiting. If you send every echo 1000 packets instead of 1 - send logic will send all 1000 packets (if flow control not setuped).
Main sense of library - is that you can send Unreliable packets (for Player positions as example)

Gotcha, Good stuff.


ethtool: 4.18 -> 4.19

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/ethtool/versions.
meta.description for ethtool is: '"Utility for controlling network drivers and hardware"'.

Checks done (click to expand)

built on NixOS
/nix/store/lny7lwhflcn4kfvh4wvxym51rsh5dfxv-ethtool-4.19/bin/ethtool passed the binary check.
1 of 1 passed binary check by having a zero exit code.
1 of 1 passed binary check by having the new version present in output.
found 4.19 with grep in /nix/store/lny7lwhflcn4kfvh4wvxym51rsh5dfxv-ethtool-4.19
directory tree listing: https://gist.github.com/2395692cf794e281ef81cb60766b9410
du listing: https://gist.github.com/5d30dc4e062cc426ad2d95b1d8bf3269

Rebuild report (click to expand)

1536 total rebuild path(s)
547 package rebuild(s)
547 x86_64-linux rebuild(s)
499 i686-linux rebuild(s)
0 x86_64-darwin rebuild(s)
490 aarch64-linux rebuild(s)
First fifty rebuilds by attrpath
adapta-gtk-theme
aircrack-ng
akregator
arc-theme
areca
astroid
azureus
balsa
beegfs
bomi
bookworm
calligra
chrome-gnome-shell
chronos
claws-mail
cntk
corebird
deepin.dde-daemon
deja-dup
digikam
dino
dl-poly-classic-mpi
dolphin
dragon
eclipses.eclipse-cpp
eclipses.eclipse-cpp-37
eclipses.eclipse-cpp-47
eclipses.eclipse-java
eclipses.eclipse-java-49
eclipses.eclipse-modeling
eclipses.eclipse-modeling-36
eclipses.eclipse-modeling-47
eclipses.eclipse-platform
eclipses.eclipse-platform-47
eclipses.eclipse-platform-48
eclipses.eclipse-platform-49
eclipses.eclipse-scala-sdk
eclipses.eclipse-scala-sdk-441
eclipses.eclipse-sdk
eclipses.eclipse-sdk-37
eclipses.eclipse-sdk-47
eclipses.eclipse-sdk-48
eclipses.eclipse-sdk-49
eclipses.eclipse_cpp_37
eclipses.eclipse_modeling_36
eclipses.eclipse_sdk_37
eid-mw
ekiga
empathy
eolie

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/lny7lwhflcn4kfvh4wvxym51rsh5dfxv-ethtool-4.19 \
  --option binary-caches 'https://cache.nixos.org/ https://r-ryantm.cachix.org/' \
  --option trusted-public-keys '
  r-ryantm.cachix.org-1:gkUbLkouDAyvBdpBX0JOdIiD2/DP1ldF3Z3Y6Gqcc4c=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(r-ryantm's Cachix cache is only trusted for this store-path realization.)
Or, build yourself:
nix-build -A ethtool https://github.com/r-ryantm/nixpkgs/archive/94ea78840f2153a21136fb214acc38892fed9545.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/lny7lwhflcn4kfvh4wvxym51rsh5dfxv-ethtool-4.19
ls -la /nix/store/lny7lwhflcn4kfvh4wvxym51rsh5dfxv-ethtool-4.19/bin

cc @bjornfor for testing.

Success on x86_64-linux (full log)
Attempted: ethtool
Partial log (click to expand)
these paths will be fetched (0.10 MiB download, 0.32 MiB unpacked):
  /nix/store/0h3i75nqjag57bpv6fy8zjm8dnn4km6k-ethtool-4.19
copying path '/nix/store/0h3i75nqjag57bpv6fy8zjm8dnn4km6k-ethtool-4.19' from 'https://nix-cache.s3.amazonaws.com'...
/nix/store/0h3i75nqjag57bpv6fy8zjm8dnn4km6k-ethtool-4.19

Success on aarch64-linux (full log)
Attempted: ethtool
Partial log (click to expand)
/nix/store/ib45pik04vmn7xvkl4mqpb6yjc98wmgh-ethtool-4.19

No attempt on x86_64-darwin (full log)
The following builds were skipped because they don't evaluate on x86_64-darwin: ethtool
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

Done in #50117


[FixME] features on cl-waffe2 is perfectly working on REPL, but the same operation in the test isn't working...

This problem has become less noticeable as the number of commits has increased, so closing


25: Ongoing, forward, and hold - [merged]

In GitLab by @Asacolips on Oct 30, 2020, 20:54
Merges 25-ongoing-forward-hold -> beta

Add fields for ongoing, forward, and hold. Add custom resource field.
Add additional compendiums (table, macro, journals)

Closes #25

In GitLab by @Asacolips on Oct 30, 2020, 20:56
mentioned in commit<PHONE_NUMBER>a37f02d2ee429e49a5650cd1a544ad


feat: [PIPE-16006]: Add information regarding execution context

Thanks for contributing to the Harness Developer Hub! Our code owners will review your submission.
Description

Please describe your changes: __________________________________
Jira/GitHub Issue numbers (if any): ______________________________
Preview links/images (Internal contributors only): __________________

PR lifecycle
We aim to merge PRs within one week or less, but delays happen sometimes.
If your PR is open longer than two weeks without any human activity, please tag a code owner in a comment.
PRs must meet these requirements to be merged:

[ ] Successful preview build.
[ ] Code owner review.
[ ] No merge conflicts.
[ ] Release notes/new features docs: Feature/version released to at least one prod environment.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://661e75b4fb585909354247a2--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://661f4c5b036744cdae207921--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://661fd1aaa0c502cd0b795e40--harness-developer.netlify.app

Overall looks good, just wanted to suggest two more points:

The execution context is not editable. Users cannot add or remove items from it.
The purpose of the execution context is to help debug previous executions of the pipeline. This is an additional tool on top of the Input/Output variables that are listed for each of the steps.

sure @SudarshanPurohit added these points as well.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://66201894a157032b5da20a12--harness-developer.netlify.app


Is this still being mantained?

@drinkcat is this still being mantained?

There hasn't been a commit in 6 years so I think it's fair to say no... Sorry ,-(


Audio is gone after PC shuts down

This is an issue I found with the audio. Sometimes I chart to the extent of forgetting to shut down my PC properly.
Therefore after rebooting the PC, the audio is gone. I can only solve the problem by closing and reopening the MMW charter.
Is it possible so that this doesn’t happen? Thank you for reading.

I haven't run into this and was not able to replicate it but I was thinking it might be due to miniaudio not shutting down and releasing the audio device properly so I made sure the application does so now. I don't know if this will fix it but you can try updating to the latest version from here:
https://github.com/crash5band/MikuMikuWorld/releases/tag/v1.2.0
I'll keep this open for now so let me know if the issue is resolved.

It is fixed now, thank you.


2 problems that I am facing.

Type: Feature Request

I have

The first problem: This problem is related to trigger suggest, when I press CTRL + Space key combination does not appear IntelliSense even though I check in keyboard shortcut, trigger suggest is still CTRL + Space.
Second problem: This problem is related to relative path, for example when I want to insert the file path I want as assets/css/style.css then I just need to type few words like asse and then press Enter then will display the full name of the assets file, press Enter again to select the css/style.css file. But suddenly one day, when I want to insert the path of the file assets/js/script.js, when I type asse and press Enter, I don't see the assets file selected, it just goes down the line. It makes me very uncomfortable, please solve my problem.

VS Code version: Code 1.78.2 (b3e4e68a0bc097f0ae7907b217c1119af9e03435, 2023-05-10T14:39:26.248Z)
OS version: Windows_NT x64 10.0.22621
Modes:
Sandboxed: Yes

/needsMoreInfo
This issue lacks

mentioning of the language and extension suggestions
copy-paste friendly source code samples

Also, please file a separate issue for the second problem.

First is probably https://github.com/microsoft/vscode/issues/183278#issuecomment-1560771117


feat(typescript): support tsconfig paths

closes https://github.com/esbuild-kit/tsx/issues/3

:tada: This PR is included in version 2.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


refactor dashboard chart html, make several css improvements.

Fixes:

Refactor dashboard chart html (some invalid elements used previously, e.g., <nobr>
Improve dashboard chart header (emphasize with color, wrap long titles, move controls to new line)
Add css rule for nvd3 tooltips to emphasize (nick h. request)
Ensure nvd3 tooltip is visible when chart overflows (lindsay request)

:+1: !!!


ValueError: Function has keyword-only parameters or annotations, use inspect.signature() API which can support them

Thanks for the awesome research and results.
I got the following traceback when the train results were getting printed. Can you please help me in fixing this?
[INFO|trainer.py:1344] 2022-11-13 12:06:39,632 >> Saving model checkpoint to result/my-unsup-simcse-bert-base-uncased
[INFO|configuration_utils.py:300] 2022-11-13 12:06:39,633 >> Configuration saved in result/my-unsup-simcse-bert-base-uncased/config.json
[INFO|modeling_utils.py:817] 2022-11-13 12:06:40,981 >> Model weights saved in result/my-unsup-simcse-bert-base-uncased/pytorch_model.bin
11/13/2022 12:06:41 - INFO - main -   ***** Train results *****
11/13/2022 12:06:41 - INFO - main -     epoch = 1.0
11/13/2022 12:06:41 - INFO - main -     train_runtime = 15286.503
11/13/2022 12:06:41 - INFO - main -     train_samples_per_second = 1.022
11/13/2022 12:06:41 - INFO - main -   *** Evaluate ***
11/13/2022 12:07:37 - INFO - root -   Generating sentence embeddings
11/13/2022 12:08:06 - INFO - root -   Generated sentence embeddings
11/13/2022 12:08:06 - INFO - root -   Training pytorch-MLP-nhid0-rmsprop-bs128 with (inner) 5-fold cross-validation
Traceback (most recent call last):
File "/home/nizam/SimCSE/train.py", line 585, in
main()
File "/home/nizam/SimCSE/train.py", line 567, in main
results = trainer.evaluate(eval_senteval_transfer=True)
File "/home/nizam/SimCSE/simcse/trainers.py", line 129, in evaluate
results = se.eval(tasks)
File "/home/nizam/SimCSE/./SentEval/senteval/engine.py", line 59, in eval
self.results = {x: self.eval(x) for x in name}
File "/home/nizam/SimCSE/./SentEval/senteval/engine.py", line 59, in
self.results = {x: self.eval(x) for x in name}
File "/home/nizam/SimCSE/./SentEval/senteval/engine.py", line 127, in eval
self.results = self.evaluation.run(self.params, self.batcher)
File "/home/nizam/SimCSE/./SentEval/senteval/binary.py", line 57, in run
devacc, testacc = clf.run()
File "/home/nizam/SimCSE/./SentEval/senteval/tools/validation.py", line 78, in run
clf = MLP(self.classifier_config, inputdim=self.featdim,
File "/home/nizam/SimCSE/./SentEval/senteval/tools/classifier.py", line 200, in init
optim_fn, optim_params = utils.get_optimizer(self.optim)
File "/home/nizam/SimCSE/./SentEval/senteval/utils.py", line 89, in get_optimizer
expected_args = inspect.getargspec(optim_fn.init)[0]
File "/home/nizam/anaconda3/envs/simcse-new/lib/python3.9/inspect.py", line 1122, in getargspec
raise ValueError("Function has keyword-only parameters or annotations"
ValueError: Function has keyword-only parameters or annotations, use inspect.signature() API which can support them

Changing inspect.getargspec function with inspect.getfullargspec fixed this issue. Thankd!


fix: improve lnd log line matching

When matching log lines, switch on the regex match rather than an
excerpt, so that there are never any false positives.
Note this allows conditional assignment within parens as well.

Fixes #544


Makes options to RegisterContext's register "instance" properties.

For example:
userAgent = new SIP.UA({
  ...
  traceSip: true,
  register: false
});

userAgent.register({traceSip: false, extraHeaders: [ 'X-Foo: foo', 'X-Bar: bar' ] });

RegisterContext receives the options via
register({traceSip: false, extraHeaders: [ 'X-Foo: foo', 'X-Bar: bar' ] })
before the connection to the socket is established.
Once the transport is connected
RegisterContext's.onTransportConnected is called and registers with empty
options.
This commit changes the options so that the RegisterContext saves them
as 'instance properties', which allows for calling onTransportConnected
with the register options.

But as always please feel free to critique and make suggestions!
Hope I didn't miss anything. Noticed this while working on another PR.

Hi Philipp,
I like this pull request.  It is definitely odd that onTransportConnected doesn't allow for any options to be specified.  I just have a few concerns before pulling it in.

Should the RegisterContext's close method also call unregister with this.options?
I would prefer that extraHeaders not be cached on this.  It doesn't seem necessary to accomplish what this pull request sets out to do, and extraHeaders tend to be modified in places further down the line that could pollute the cached array.  Cloning it from this.options each time feels safer to me.

With those changes addressed (or with comments telling me why I'm utterly wrong, which is not unheard of), I'd be happy to pull this in.
Also, you get bonus points for fixing the whitespace errors. :-)
Cheers,
-Will

Hi Will,
Thanks for the feedback.

I can't come up with a scenario where that might be an issue:
userAgent.stop() is the only place where RegisterContext.close()
(and therefore .unregister()) could be called.
So for the options to unregister {all: true, extraHeaders: [ 'X-Foo: foo', 'X-Bar: bar' ] to get lost,
we would have to call .unregister(options) and only afterwards have userAgent.stop().

userAgent.unregister({all: true, extraHeaders: [ 'X-Foo: foo', 'X-Bar: bar' ]});
userAgent.stop();

And at that point the options would be ignored anyway. (RegisterContext.unregister())
if(!this.registered) {
  this.logger.warn('already unregistered');
  return;
}

But in case I might miss something and that could happen: The options to unregister differ to those provided to register. So calling unregister with the this.options would not be desired anyway. A solution would be to save this.registerOptions and this.unregisterOptions. Which I would gladly provide in case that is needed.
2. Yep, I agree just pushed another commit.

Also, you get bonus points for fixing the whitespace errors. :-)

I'll forever cherish these bonus points.


Investigate compile sizes

Just tried to run cargo-bloat on Wrench (the binary we use for WebRender debugging) and saw this:
kvark@ant /mnt/code/firefox/gfx/wr/wrench $ cargo bloat --release --crates
Compiling ...
Analyzing target/release/wrench

File  .text     Size Crate
1.0%  23.7%   2.0MiB webrender
0.7%  16.7%   1.4MiB ron
0.5%  13.5%   1.1MiB std
0.2%   4.6% 388.4KiB webrender_api
0.2%   4.4% 373.3KiB image
0.2%   4.0% 335.5KiB wrench
0.2%   3.8% 319.6KiB clap
0.1%   2.9% 243.2KiB winit
0.1%   2.8% 233.4KiB gleam
0.1%   2.7% 231.3KiB yaml_rust
0.1%   2.5% 211.6KiB regex_syntax
0.1%   1.7% 140.0KiB regex
0.1%   1.6% 138.3KiB serde

Just tried for the decode example:
[thomas@thomas-pc ron]$ cargo bloat --release --example decode --crates
Compiling ...
Analyzing target/release/examples/decode

File  .text     Size Crate
5.7%  62.3% 147.8KiB std
2.0%  21.7%  51.4KiB ron
1.4%  15.1%  35.7KiB [Unknown]
0.0%   0.5%   1.2KiB serde
0.0%   0.4%     925B decode
0.0%   0.0%      54B base64
9.1% 100.0% 237.1KiB .text section size, the file size is 2.5MiB

Maybe the last line is relevant here?

Note: numbers above are a result of guesswork. They are not 100% correct and never will be.

I don't really see how ron could take up so much space. Our own code is pretty simple, and we don't have any special dependencies:
[thomas@thomas-pc ron]$ cargo tree
ron v0.5.1 (/home/thomas/Workspace/ron)
├── base64 v0.10.1
│   └── byteorder v1.3.1
├── bitflags v1.0.4
└── serde v1.0.90
    └── serde_derive v1.0.90
        ├── proc-macro2 v0.4.27
        │   └── unicode-xid v0.1.0
        ├── quote v0.6.11
        │   └── proc-macro2 v0.4.27 (*)
        └── syn v0.15.30
            ├── proc-macro2 v0.4.27 (*)
            ├── quote v0.6.11 (*)
            └── unicode-xid v0.1.0 (*)
    [dev-dependencies]
    └── serde_derive v1.0.90 (*)
[dev-dependencies]
├── serde_bytes v0.11.1
│   └── serde v1.0.90 (*)
└── serde_json v1.0.39
    ├── itoa v0.4.3
    ├── ryu v0.2.7
    └── serde v1.0.90 (*)

Interestingly, it doesn't show up when building WR examples, only when building Wrench. The difference is that Wrench enables the serde deserialization attributes on a whole ton of structs, plus the code to actually use them (that calls into Ron).

The size blamed on ron is stuff like:
0.0%   0.1% 10.5KiB                   ron? <&mut ron::de::Deserializer as serde::de::Deserializer>::deserialize_struct
0.0%   0.1% 10.5KiB                   ron? <&mut ron::de::Deserializer as serde::de::Deserializer>::deserialize_struct
0.0%   0.1%  9.4KiB                   ron? <&mut ron::de::Deserializer as serde::de::Deserializer>::deserialize_struct
0.0%   0.1%  8.3KiB                   ron? <&mut ron::de::Deserializer as serde::de::Deserializer>::deserialize_struct
0.0%   0.1%  7.0KiB                   ron? <&mut ron::de::Deserializer as serde::de::Deserializer>::deserialize_struct
0.0%   0.1%  7.0KiB                   ron? <&mut ron::de::Deserializer as serde::de::Deserializer>::deserialize_struct
0.0%   0.1%  6.8KiB                   ron? <&mut ron::de::Deserializer as serde::de::Deserializer>::deserialize_struct
0.0%   0.1%  6.5KiB                   ron? <ron::de::Enum as serde::de::VariantAccess>::struct_variant
0.0%   0.1%  6.3KiB                   ron? <&mut ron::de::Deserializer as serde::de::Deserializer>::deserialize_struct

This is just the actual specialized serialize/deserialize methods.
Also cargo bloat reports compiled size and not compile times.

If you compile with the new mangling scheme you can even get more info about what's going on:
RUSTFLAGS="-Z symbol-mangling-version=v0" rustup run nightly cargo bloat --release -n 100 | grep ron
0.0%   0.2% 18.6KiB   [Unknown]? <<webrender[58737326ba1c09f4]::scene_builder::Interners as serde[6276e54a47026ae7]::de::Deserialize>::deserialize::__Visitor as serde[6276e54a47026ae7]::de::Visitor>::visit_map::<ron[d5...
0.0%   0.2% 12.9KiB   [Unknown]? <webrender_api[b9415cac22c9e10f]::display_list::BuiltDisplayList as serde[6276e54a47026ae7]::de::Deserialize>::deserialize::<&mut ron[d53180edc19c75a1]::de
0.0%   0.2% 12.9KiB   [Unknown]? <&mut ron[d53180edc19c75a1]::de::Deserializer as serde[6276e54a47026ae7]::de::Deserializer>::deserialize_struct::<<webrender[58737326ba1c09f4]::render_backend::DataStores as serde[6276e...
0.0%   0.1% 11.7KiB   [Unknown]? <&mut ron[d53180edc19c75a1]::de::Deserializer as serde[6276e54a47026ae7]::de::Deserializer>::deserialize_struct::<<webrender[58737326ba1c09f4]::texture_cache::TextureCache as serde[6276...
0.0%   0.1%  9.5KiB   [Unknown]? <&mut ron[d53180edc19c75a1]::de::Deserializer as serde[6276e54a47026ae7]::de::Deserializer>::deserialize_struct::<<webrender[58737326ba1c09f4]::resource_cache::PlainCacheOwn as serde[62...
0.0%   0.1%  9.0KiB   [Unknown]? <&mut ron[d53180edc19c75a1]::de::Deserializer as serde[6276e54a47026ae7]::de::Deserializer>::deserialize_struct::<<webrender[58737326ba1c09f4]::tiling::TextureCacheRenderTarget as serde...
0.0%   0.1%  8.3KiB   [Unknown]? <&mut ron[d53180edc19c75a1]::de::Deserializer as serde[6276e54a47026ae7]::de::Deserializer>::deserialize_struct::<<webrender[58737326ba1c09f4]::tiling::Frame as serde[6276e54a47026ae7]:...
0.0%   0.1%  7.4KiB   [Unknown]? <&mut ron[d53180edc19c75a1]::de::Deserializer as serde[6276e54a47026ae7]::de::Deserializer>::deserialize_struct::<<webrender[58737326ba1c09f4]::gpu_cache::Texture as serde[6276e54a47026...
0.0%   0.1%  7.4KiB   [Unknown]? <&mut ron[d53180edc19c75a1]::de::Deserializer as serde[6276e54a47026ae7]::de::Deserializer>::deserialize_struct::<<webrender[58737326ba1c09f4]::tiling::ColorRenderTarget as serde[6276e5...
0.0%   0.1%  7.2KiB   [Unknown]? <&mut ron[d53180edc19c75a1]::de::Deserializer as serde[6276e54a47026ae7]::de::Deserializer>::deserialize_struct::<<webrender[58737326ba1c09f4]::tiling::AlphaRenderTarget as serde[6276e5...
0.0%   0.1%  7.0KiB   [Unknown]? <ron[d53180edc19c75a1]::de::Enum as serde[6276e54a47026ae7]::de::VariantAccess>::struct_variant::<<<webrender[58737326ba1c09f4]::tiling::RenderPassKind as serde[6276e54a47026ae7]::de::D...

The idea we discussed with @jrmuizel would be to move as much deserialization code as possible from being monomorphised into run-time. For example, we may have a small piece of code parsing RON and generating run-time reflection of a struct (non-generically), so that RON deserialization then works off it.

Yeah, so you could have #[derive(Reflect)] which would try to produce a compact representation of the struct and add a method that returns a reference to this compact representation which implements Deserialize.

I was just looking at crate sizes for our wasm binary and ron is one of the top ones at 85k (serde-json is ~3X bigger though)

I think there are some wasm specific tools to analyse what code takes up the most space. Perhaps you could find out what part of RON takes up excessive space?


Duplicate entry or exception during serialization if JSON.NET has no way to set a property

Summary
I wanted to serialize classes that follow the Domain-Driven Design approach, i.e. properties have private setters. This didn't work and after a lot of digging I think there is a problem in JSON.NET (or maybe deeper - see orleans issue 5217.
Here is the summary of what I have found:
I have the following arrangement of classes (code shown later)
|--------| 0..1     1 |--------------| 1     0..1
| Book 1 | -----------| ManyToMany 1 |------------|
|--------|            |--------------|      |-----|-----|
                                            |  Author 1 |
|--------| 0..1    1  |--------------|      |-----|-----|
| Book 2 | -----------| ManyToMany 2 |------------|
|--------|            |------------- | 1      0..1

Summary of what I found:

If the ManyToMany relationship properties have private setters then I get a "Self referencing loop detected" exception (details below).
If the ManyToMany relationship properties have public setters then this work.
If I set ReferenceLoopHandling = ReferenceLoopHandling.Ignore then it works with private setters in ManyToMany class, but duplicates Book 2.

UPDATE
I now know why this happens. Json.Net checks that a property can be set before it will serialize the data. This causes the two problems that are described below.
The way around this problem is to provide at serialization time a setting like ContractResolver = new ResolvePrivateSetters() or Json.NET attributes on constructors or properties to allow the property to be set.
Example 1 - fails with exception
The problem with this is the exception message gives no hint that the problem is because the properties cannot be set. Some more clearer message would help a lot.
// Put the types you are serializing or deserializing here
public class TestBook
{
    private TestBook() {}

    private readonly HashSet<ManyToMany> _many;

    public TestBook(ManyToMany many)
    {
        _many = new HashSet<ManyToMany>{many};
    }

    public int TestBookId { get; set; }
    public IEnumerable<ManyToMany> Many => _many.ToList();
}

public class TestAuthor
{
    public int TestAuthorId { get; set; }
    public ICollection<ManyToMany> Many { get; set; }
}

public class ManyToMany
{
    public int TestBookId { get; private set; }
    public int TestAuthorId { get; private set; }
    public TestBook BookLink { get; private set; }
    public TestAuthor AuthorLink { get; private set; }

    public void SetBookAuthor(TestBook book, TestAuthor author)
    {
        BookLink = book;
        AuthorLink = author;
    }
}

Now the unit test
[Fact]
public void TestJsonSerializeNotInDatabase()
{
    //SETUP
    var entities = GetLinkEntities();

    //ATTEMPT
    var json = JsonConvert.SerializeObject(entities, new JsonSerializerSettings()
    {
        PreserveReferencesHandling = PreserveReferencesHandling.Objects,
        Formatting = Formatting.Indented
    });

    //FAILS WITH EXCEPTION
}

private List<TestBook> GetLinkEntities()
{
    var many1 = new ManyToMany();
    var many2 = new ManyToMany();
    var book1 = new TestBook(many1);
    var book2 = new TestBook(many2);
    var author = new TestAuthor { TestAuthorId = 3 };
    many1.SetBookAuthor(book1, author);
    many2.SetBookAuthor(book2, author);
    author.Many = new List<ManyToMany> { many1, many2 };

    return new List<TestBook> { book1, book2 };
}

Exception
Newtonsoft.Json.JsonSerializationException : Self referencing loop detected for property 'AuthorLink' with type 'Test.UnitTests.TestDataResetter.TestDuplicateObjectInJsonSerialize+TestAuthor'. Path '[0].Many[0].AuthorLink.Many[1]'.
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.CheckForCircularReference(JsonWriter writer, Object value, JsonProperty property, JsonContract contract, JsonContainerContract containerContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.CalculatePropertyValues(JsonWriter writer, Object value, JsonContainerContract contract, JsonProperty member, JsonProperty property, JsonContract& memberContract, Object& memberValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList(JsonWriter writer, IEnumerable values, JsonArrayContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList(JsonWriter writer, IEnumerable values, JsonArrayContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList(JsonWriter writer, IEnumerable values, JsonArrayContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType)
   at Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, Object value, Type objectType)
   at Newtonsoft.Json.JsonConvert.SerializeObjectInternal(Object value, Type type, JsonSerializer jsonSerializer)
   at Test.UnitTests.TestDataResetter.TestDuplicateObjectInJsonSerialize.TestJsonSerializeNotInDatabase() in C:\Users\Jon\Source\Repos\EfCore.TestSupport\Test\UnitTests\TestDataResetter\TestDuplicateObjectInJsonSerialize.cs:line 32

Example 2 - runs correctly
All I have to do is change the two linking properties in ManyToMany class to have public setters, e.g.
public class ManyToMany
{
    public int TestBookId { get; private set; }
    public int TestAuthorId { get; private set; }
    public TestBook BookLink { get;  set; }
    public TestAuthor AuthorLink { get;  set; }

    public void SetBookAuthor(TestBook book, TestAuthor author)
    {
        BookLink = book;
        AuthorLink = author;
    }
}

All the rest of the code is the same as in example 1.
Here is the correct serialization (note that there are only two Book objects).
[
  {
    "$id": "1",
    "TestBookId": 0,
    "Many": [
      {
        "$id": "2",
        "TestBookId": 0,
        "TestAuthorId": 0,
        "BookLink": {
          "$ref": "1"
        },
        "AuthorLink": {
          "$id": "3",
          "TestAuthorId": 3,
          "Many": [
            {
              "$ref": "2"
            },
            {
              "$id": "4",
              "TestBookId": 0,
              "TestAuthorId": 0,
              "BookLink": {
                "$id": "5",
                "TestBookId": 0,
                "Many": [
                  {
                    "$ref": "4"
                  }
                ]
              },
              "AuthorLink": {
                "$ref": "3"
              }
            }
          ]
        }
      }
    ]
  },
  {
    "$ref": "5"
  }
]

Example 3 - ReferenceLoopHandling.Ignore causes duplicate
If I set ReferenceLoopHandling = ReferenceLoopHandling.Ignore then it doesn't throw an exception, but it duplicates Book 2.  This happens because JSON.NET doesn't have a way to set the linking properties
Here is the unit test using the same data and classes shown in example 1, i.e. the ManyToMany's BookLink and AuthorLink have private setters.
[Fact]
public void TestJsonSerializeNotInDatabaseReferenceLoopHandlingIgnore()
{
    //SETUP
    var entities = GetLinkEntities();

    //ATTEMPT
    var json = JsonConvert.SerializeObject(entities, new JsonSerializerSettings()
    {
        PreserveReferencesHandling = PreserveReferencesHandling.Objects,
        ReferenceLoopHandling = ReferenceLoopHandling.Ignore,
        Formatting = Formatting.Indented
    });

    _output.WriteLine(json);
}

The JSON output looks like this - notes

There are three books in the JSON (the second one is duplicated)
If I change the ManyToMany's BookLink and AuthorLink public setters it works correctly, i.e. no duplicate Book 2

[
  {
    "$id": "1",
    "TestBookId": 0,
    "Many": [
      {
        "$id": "2",
        "TestBookId": 0,
        "TestAuthorId": 0,
        "BookLink": {
          "$ref": "1"
        },
        "AuthorLink": {
          "TestAuthorId": 1,
          "Many": [
            {
              "$ref": "2"
            },
            {
              "$id": "3",
              "TestBookId": 0,
              "TestAuthorId": 0,
              "BookLink": {
                "TestBookId": 0,
                "Many": [
                  {
                    "$ref": "3"
                  }
                ]
              }
            }
          ]
        }
      }
    ]
  },
  {
    "$id": "4",
    "TestBookId": 0,
    "Many": [
      {
        "$ref": "3"
      }
    ]
  }
]

Using JSON.NET 12.0.1 in NET Core 2.2.1

Hi,
I have tried to diagnose the problem in your package so that I could do a pull request. Unfortunately I couldn't follow the serialization handling of the circular references. However I did write a unit test in your format that I have provided below. You may find this useful in determining what the problem is.
#region License
// Copyright (c) 2007 James Newton-King
//
// Permission is hereby granted, free of charge, to any person
// obtaining a copy of this software and associated documentation
// files (the "Software"), to deal in the Software without
// restriction, including without limitation the rights to use,
// copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following
// conditions:
//
// The above copyright notice and this permission notice shall be
// included in all copies or substantial portions of the Software.
 IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
// HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
// WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
// FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
#endregion

using Newtonsoft.Json.Linq;
using Newtonsoft.Json.Linq.JsonPath;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using Newtonsoft.Json.Serialization;
#if DNXCORE50
using Xunit;
using Test = Xunit.FactAttribute;
using Assert = Newtonsoft.Json.Tests.XUnitAssert;
#else
using NUnit.Framework;
#endif

namespace Newtonsoft.Json.Tests.Issues
{
    [TestFixture]
    public class Issue2049
    {
        [Test]
        public void TestJsonSerialize()
        {
            //SETUP
            var entities = GetLinkEntities();

            //ATTEMPT
            var json = JsonConvert.SerializeObject(entities, new JsonSerializerSettings()
            {
                PreserveReferencesHandling = PreserveReferencesHandling.Objects,
                ReferenceLoopHandling = ReferenceLoopHandling.Ignore,
                Formatting = Formatting.Indented
            });

            //VERIFY
            Assert.AreEqual(1, json.Split('\n').Count(x => x.Trim() == "\"Title\": \"Book2\","));
        }

        //-----------------------------------------------------------------
        //classes and setups for test

        private List<TestBook> GetLinkEntities()
        {
            var many1 = new ManyToMany();
            var many2 = new ManyToMany();
            var book1 = new TestBook("Book1", many1);
            var book2 = new TestBook("Book2", many2);
            var author1 = new TestAuthor("Author");
            many1.SetBookAuthor(book1, author1);
            many2.SetBookAuthor(book2, author1);

            author1.Many = new List<ManyToMany> { many1, many2 };

            return new List<TestBook> { book1, book2 };
        }

        public class TestBook
        {
            private TestBook() { }

            private readonly HashSet<ManyToMany> _many;

            public TestBook(string title, ManyToMany many)
            {
                Title = title;
                _many = new HashSet<ManyToMany> { many };
            }

            public int TestBookId { get; set; }
            public string Title { get; set; }
            public IEnumerable<ManyToMany> Many => _many.ToList();
        }

        public class TestAuthor
        {
            public TestAuthor(string name)
            {
                Name = name;
            }

            public int TestAuthorId { get; set; }

            public string Name { get; set; }
            public ICollection<ManyToMany> Many { get; set; }
        }

        public class ManyToMany
        {
            public int TestBookId { get; private set; }
            public int TestAuthorId { get; private set; }
            public TestBook BookLink { get; private set; }
            public TestAuthor AuthorLink { get; private set; }

            public void SetBookAuthor(TestBook book, TestAuthor author)
            {
                BookLink = book;
                AuthorLink = author;
            }
        }
    }
}

I have now diagnosed the problem and changed the issue's title to match. I have updated the initial statement of the problem to detail what fixes this problem, but I still believe producing a duplicate at serialization is a bug (I also think the exception message could be better too).

Hi Jon,
Yes, this seems to be a bug where properties that lack a visible setter are not correctly preserved/referenced.
However the error message is fine and you are getting it because, while preserving/referencing fails, ReferenceLoopHandling is still in the play and, since you already detected AuthorLink in your branch, the serializer throws when tries to write it again.
If you look at the Exception message: Self referencing loop detected for property 'AuthorLink' '.
And then look at the exception's JSONPath: '[0].Many[0].AuthorLink.Many[1]'
You will notice that it leads you to the second occurence of AuthorLink, look at below payload I added Ids to the rest of objects and used ReferenceLoopHandling.Serialize instead of Error or Ignore in order to force serialization of reference loops.
[
  {
    "$id": "1",
    "TestBookId": 1,
    "Many": [
      {
        "$id": "2",
        "TestBookId": 1,
        "TestAuthorId": 3,
        "BookLink": {
          "$ref": "1"
        },
        "AuthorLink"*: {
          "TestAuthorId": 3,
          "Many": [
            {
              "$ref": "2"
            },
            {
              "$id": "3",
              "TestBookId": 2,
              "TestAuthorId": 3,
              "BookLink": {
                "TestBookId": 2,
                "Many": [
                  {
                    "$ref": "3"
                  }
                ]
              },
              "AuthorLink"*: {
                "TestAuthorId": 3,
                "Many": [
                  {
                    "$ref": "2"
                  },
                  {
                    "$ref": "3"
                  }
                ]
              }
            }
          ]
        }
      }
    ]
  },
  {
    "$id": "4",
    "TestBookId": 2,
    "Many": [
      {
        "$ref": "3"
      }
    ]
  }
]

Hi @Jozkee,
Thank you for taking the time to reply to this issue. My initial issue was asking for a way around this problem, but I found that myself. Therefore I don't need a fix, but I left this issue as I thought you would be interested in knowing about this issue.
You are saying there is a bug if ReferenceLoopHandling is set to ReferenceLoopHandling.Ignore. I accept that is going to be difficult to fix and it may be so niche that its not worth it.
I'm not totally sure what you are saying about using ReferenceLoopHandling.Serialize. The json that creates is worse than the output with ReferenceLoopHandling.Ignore as it still duplicates "Book2" and in addition it duplicates the "Author". I don't see that adds anything.


Implemented ValidateStore function for Alibaba Provider

Fixes #861

@gusfcarvalho I am in doubt if we have to validate the Name and Key for AccessKeySecret and AccessKeyID in this case.

Hi @hydeenoble ! The idea behind the ValidateStore is to catch possible configuration errors before the manifest is applied. So, since in SetAuth method we have some validation to see if the Name is available, we should also validate it at ValidateStore method. The idea is that if there is no AccessKeySecret.Name, this is already going to be a fail. So we want it to get as early as possible, at the Validating Webhook method (which calls ValidateStore) :)

@gusfcarvalho please do you have time to review this?

@hydeenoble ! Sorry for the delay!
Looks good to me, thanks for your contribution!

/approve

/merge


Create Conda Environment

Right now there is no way to install all the necessary dependencies for this project. Adding a file to automatically install them would be beneficial.

Resolves #17


Typos in transformer.Transformer and entities.NewLine

elif tag in ("span",):
    entity = self._get_span(attrs)
elif tag in ("tg-spiler",):
    entity = Spoiler()
elif tag in ("p",):
    entity = Paragraph()

I guess it should be called tg-spoiler?

def add(self, entity: "Entity"):
    raise ValueError("Usupported contents for NewLine widget")


Service Definition item (IncidentHotSpots) exists without an associated hosted service

Hosted service Map/MapServer does not exist, but associated service definition item "IncidentHotSpots" 019ef1d8880749269c26e44e490f4f5a exists; please delete the following item; otherwise the hosted service will be published to ops servers.

Item item "IncidentHotSpots" 019ef1d8880749269c26e44e490f4f5a has been deleted


Allow OS.find_scancode_from_string to process keys with modifiers

Fix #17430

As discussed with @reduz, this is sensitive code and a more thorough solution is needed to properly support key combinations in actions (and handle this use case too). This will be worked on for 3.2, so closing this PR for now.


conflict with more recent asciidoctor version

Hi,
Thanks for your pasckage. Unfortunately, I was unable to test it due to conflict ascidoctor version as of today. I have this error after cloning your dir locally and bundle all gems :
Unable to activate asciidoctor-question-0.5.2, because asciidoctor-2.0.10 conflicts with asciidoctor (~> 1.5) (Gem::ConflictError)
I've tried to modify versions number in Gemfile and asciidoctor-questions.gemspec but with no luck, still the same error (it's my maximum understanding level ...haha!)
Please indicate the way for allowing asciidoctor-question to work in 2020.
Thanks in advance

@oturlier  try this:
# Uninstall previous version
gem uninstall asciidoctor-question

# Clone repo
git clone https://github.com/vifito/asciidoctor-question.git

cd asciidoctor-question

# Build and install
rake build
rake install

# Usage
asciidoctor -r asciidoctor-question -r asciidoctor-pdf -b pdf sample.adoc
asciidoctor -r asciidoctor-question sample.adoc

I just get this:  asciidoctor-question-0.6.0/lib/asciidoctor-question/question/post_processor.rb:12:in process': undefined method add_child' for nil:NilClass (NoMethodError)
Also when entering the asciidoctor-question directory as above I get an error:  Required ruby-2.5.7 is not installed.
of course I am using ruby 2.7.
Would be nice if this plugin was updated to be compatible with asciidoctor 2 and more recent versions of ruby.

I just get this:  asciidoctor-question-0.6.0/lib/asciidoctor-question/question/post_processor.rb:12:in process': undefined method add_child' for nil:NilClass (NoMethodError)
Also when entering the asciidoctor-question directory as above I get an error:  Required ruby-2.5.7 is not installed.
of course I am using ruby 2.7.
Would be nice if this plugin was updated to be compatible with asciidoctor 2 and more recent versions of ruby.


celsius-kafka should be celsius_kafka in examples/python and examples/pony

As per the wallaroo naming scheme, these should be celsius_kafka, rather than celsius-kafka

@nisanharamati yes, good catch. But I agree with @SeanTAllen's comment in #1510, we should wait on this until after hard launch.


InvalidStateException when creating a schema

Describe the bug
Receiving the following exception when attempting to create a schema using a new wallet.
Hyperledger.Indy.InvalidStateException: The SDK library experienced an unexpected internal error.at AgentFramework.Core.Runtime.DefaultPoolService.GetPoolAsync(String poolName) in C:\Projects\passport\agent-framework\src\AgentFramework.Core\Runtime\DefaultPoolService.cs:line 32
To Reproduce
Steps to reproduce the behavior:

Deployed app to Azure
Configured WalletOptions and PoolOptions:

"WalletOptions": {
    "WalletConfiguration": {
      "Id": "issuer_wallet_1",
      "StorageConfiguration": { "Path": "D:\\home\\site\\indy\\" }
    },
    "WalletCredentials": { "Key": "issuer_wallet_key_1" }
  },
  "PoolOptions": {
    "GenesisFilename": "genesis.txn",
    "PoolName": "default_pool1",
    "ProtocolVersion": 2
  }

(The genesis.txn file is in place)
3. Configured using endpoint of https:///agent
4. Configured
app.UseAgentFramework<PassportAgentMiddleware>(agentEndpoint,
  obj =>
  {
     obj.AddIssuer("000000000000000000000000Trustee1");
     obj.AddOwnershipInfo(agentName, logoUrl);
   });

Opened app with a browser and use the method ISchemaService.CreateSchemaAsync.
It fails with the error above.

Expected behavior
The method should return the schema id.
Azure:

OS: App Service, Windows, Central US

Using Hyperledger.Indy.Sdk 1.8.0 from NuGet.
Using AgentFramework built from https://github.com/idramp/agent-framework/tree/idr/develop

@idr-spilling do you have the indy-cli installed, can you please try create a pool configuration using the cli with the txn file you have mentioned above and confirm connection to the node pool?

I ran the indy-cli using the following commands on both my local machine and in Azure.
For both I receive the following output:
pool delete default_pool1
Pool "default_pool1" has been deleted.

pool create default_pool1 gen_txn_file=genesis.txn
Pool config "default_pool1" has been created

pool connect default_pool1
Error: Invalid library state
  Caused by: Ledger merkle tree is not acceptable for current tree.

I am now investigating our private nodes and the txn file.

We are running a set of private nodes for testing. The nodes themselves report that they are running. The ports on the VM are open and responding.
We are using the pool_transactions_genesis file from the server to establish a connection. The default 127.0.01 IP address in the file has been replaced by the VM's public IP address.
The result of the pool connect command remains the same as above. This is the result using indy-cli versions 1.8.0 and above.

@idr-spilling Do you run the nodes by passing the public IP as well? If you run them using <IP_ADDRESS> that IP will be registered on the nodes. If you're using the docker file from indy-sdk/ci you can pass an argument to docker with the public IP they should bound to pool_ip.

No, we are only running it with the local host IP address. Then we take the genesis file, and find-replace <IP_ADDRESS> with the public IP address

docker build --build-arg pool_ip=PUBLIC_IP -f ci/indy-pool.dockerfile -t indy_pool .
docker run -itd -p 9701-9709:9701-9709 indy_pool

Or just replace the pool_ip inside the docker file.

It appears this was the issue. When the nodes were originally configured, the public address was not used.
Thank you for your assistance.


Fixed Spelling Mistake.

Fixed Spelling Mistake in Channel Permissions Error.

@mrkirby153 Any chance for a merge?


Converstion of string format timetsamp to datetime

First of all thanks for the nice library.
I am using ingest_data(msg, schema) function to convert my python list to parquet files, but I have a timestamp field which is in string format (%Y-%m-%dT%H: %M: %S. %fZ) which I would like to convert into datetime before I write into parquet file. What is the best way to achieve this conversion?

Good question.  I don't have a ton of support at the moment for this.  At the moment all datetime parsing is done by the pandas library's to_datetime function here:
https://github.com/andrewgross/json2parquet/blob/master/json2parquet/client.py#L66
This function does take a format argument which would work with your string format.  I will work on adding this as a keyword arg.

This has been added in 0.0.15


Implement git repository controller

This implements the git repository types and controller.
Syncs the contents of a local directory to an in-cluster gitea repository for ArgoCD to pick up.
We need https://github.com/cnoe-io/idpbuilder/pull/39 merged first.
closes #59, closes #60

also looks like you've got some conflicts to resolve

This PR's become a bit too large for my liking but it implements the entire workflow with Gitea.
All emebdded manifests are pushed to gitea server under the admin user organization. We may want to make this configurable in the future.

Once they are pushed, argo applications are created. They reference the repos created above. From here, ArgoCD takes care of the rest. The Git repository contents are reconciled once for embedded applications. If users wants to reconfigure things, they can do so in the Gitea UI.

Once this PR merges, I just need to rebase and build should pass.


Better tunnel

I'm working from home today, and working with the tunnel is rough. Especially when I'm trying to jump between the apangea db and the warehouse db.
This PR does two things:

[ ] Lets you start a tunnel on a specific port (so you can have more than one running.)
[ ] Tells you the psql command to run after establishing the tunnel.

@BobTheBotMaker @TheJefe Please review!

Not requested reviewer, but I happened to be curious and looking, so I wanted to say LGTM, too. :rocket:

LGTM

So last time i worked from home, I added to PsqlHelper a tunnel option that.... hardcoded the expected port to 5433: https://github.com/thinkthroughmath/apangea/blob/rc/lib/psql_helper.rb#L95-L97
Should we say that rake ttm:rebuild_with_production_content[tunnel] only supports running the ssh tunnel with the default port, or should we change the rake task/PsqlHelper to handle a variable port as well?

This LGTM. I am going to merge tomorrow unless there are objections.

I would like an answer to my question, once seej is available

Just to add my two cents until seejee can answer, I think it's desirable that we update the lib to handle the variable port as well. However, if it is too much work/out of scope for this, I can absolutely see it in a separate PR.

@srobbibaro That work would be done in apangea, correct? So I think this can be merged regardless.
Personally, I'm fine with assuming the default port in the rebuild scripts because we really only need to specify the port option if we're running two tunnels simultaneously. If yinz disagree, though, let's make the change in apangea in the same PR that we bump the locked scalr version.

@seejee - Yes, you are correct and I agree with you. I guess the question I was choosing between was whether to do that work in the Apangea PR AND gate this PR or not gate the PR.
I do think it might be useful to note it somewhere though -- basically, that it's a known limitation. Wdyt?

crickets


Profiles page

Add a new page to show the profiles and the attributes that they defined.

Done in build 2.47.0


How to change LSAN variables

Hello, I'm trying to do some stuff with this template as base, and I have some leaks, but I don't know how to set the variables on this project.
==86737==LeakSanitizer has encountered a fatal error.
==86737==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1
==86737==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)

My knowledge on CMake is not so big, and this template is not so easy to use.

on bash:
export LSAN_OPTIONS='verbosity=1:log_threads=1'


installing MG-RAST-Tools [Ubuntu 20.04 LTS]

Hello
I am trying to install MG-RAST-Tools on my Ubuntu 20.04 LTS.
When add the follow command:   sudo python setup.py install, I get the following error message:
Traceback (most recent call last):
File "setup.py", line 5, in
from setuptools import setup
ImportError: No module named setuptools
I've uninstalled and installed setuptools with no success.
I am using Python 3.7.6
Any clue on how to solve this issue?
Thank you

You need python3-setuptools to build MG-RAST-Tools, and you can get it on ubuntu 20.04 with
apt-get update && apt-get install -y python3-setuptools


echo Output special characters

Hi phalcon-uploader author :
i download this tool to use, is can't working. and then, i find in front of a blank line has one special character, i delete this blank line and it is working.... but , in browser is show some special characters, what is happen in phalcon-uploader? how to solve it ?

follow  is some special characters info in bowser:

Hello, i was fixed it! Thank you!


boost::get_error_info() fails to lookup info for boost::stacktrace tag

Summary
On macOS Catalina when throwing a boost::enable_error_info annotated exception from a shared library, then catching it from the main executable, and built with default symbol visibility to “hidden”, get_error_info() from main exe fails to locate exception details.
Steps to reproduce
See reproducer at https://github.com/daverigby/boost_error_info_visibility
Analysis
Thanks to @pdimov on #boost cpplang slack, we identified that a number of symbols in exception / stacktrace need BOOST_SYMBOL_VISIBLE applying. In my local Boost 1.75 world adding to the forward-decl and defns of all of the following was required to make the test pass:

boost::error_info
boost::error_info_base
Tag template param for boost::error_info
"T" tempalate param for boost_error_info (which is boost_stacktrace and template params in my case):

boost::stacktrace::frame
boost::basic_stacktrace

Note: Adding the following forward-decls to the above testcase is sufficient to workaround the issue:
namespace boost {
template <class Tag, class T>
class PUBLIC_API error_info;
};
namespace boost { namespace exception_detail {
class PUBLIC_API error_info_base;
};
};
namespace boost { namespace stacktrace {
class PUBLIC_API frame;
template <class Allocator> class PUBLIC_API basic_stacktrace;
};
};

Testcase updated to include this (commented out).

Applied changes from feature/symbol_visibility_test to my local Boost 1.75 via homebewl (I don't have build of boost from source):
git diff origin/develop..origin/feature/symbol_visibility_test include/boost/exception > ~/scratch/boost_exception_strcmp.patch
cd /usr/local/include/boost/exception && patch -p4 < ~/scratch/boost_exception_strcmp.patch ```

This now passes the above testcase 👍

Merged into develop.


Added docstrings

Hello!
I am going to start improving this package significantly. It has helped me greatly so far and now that I am a package "pro" it is time to return the favor.
Over time I will be adding docstring improvements and export improvements because this can be a fully fledge usable package.
I've noticed that the package has almost none exported names. This is unreasonable since it is a registered package. Registered packages export their names, instead of adding the package name before of every possible function call.
Also, in parallel with the docstrings I am also checking for dep warnigs but I think we've all solved these.
After I am done with the docstrings I am suggesting a new release tag.

Sounds good to me. I'd be curious to know what you've been using it for. I'm not a Julia expert and haven't really touched the language in ages, so anything that makes it more idiomatic/usable for a typical Julia user is fine by me. Feel free to open issues for any tasks that you're working on/want added - I've been toying with the idea of working more on this for a little while, so it would help to know what you're planning.

And thank you for taking time to write proper docstrings for this. That's much better than just having some minimal comments and a readme file.

I am using the package for heavy statistical analysis and manipulation of musical rhythms. I am taking part in a side project about "what makes swing swing" and I am the person doing all the work with statistics and coding and manipulation. If you have ever heard Theo Geisel (my supervisor) he has done a lot of work on the nature of musical rhythms.
 Also, I have 0 knowledge about binary data in general, there is no way I could possibly extract this information from a MIDI file without your package.
On the same topic of improving the package, maybe it is worth it to consider making it a part of JuliaIO organization? https://github.com/JuliaIO
This would make it even more "official", which I think it is a good thing. In the end of the day, it is truly an IO package so I find it highly likely that they will accept it. We just have to set up a few tests (I can do set up the code for testing but you have to regirster on TravisCI and AppVeyor).

Oh, that's very cool. I'm glad this package has been helpful for that!
I haven't heard of JuliaIO, but I like the idea of this having a similar interface to other IO packages. In the long run, I'd like it to let you send/receive midi messages in real time to connected devices. I don't know if that's within the mandate of JuliaIO or not, but I'll look into it.


v0.10.0 Changelog and Release Notes

Adds CHANGELOG.md update and docs updates for release notes for v0.10.0

Realized I'm missing the configurable filtering by package ownership, will add that.


Working on non-apple hardware?

Are there any reports of this box working on non-apple hardware, specifically in a Linux host?
In headless mode the box boots fine, but in GUI mode the box won't boot getting stuck at "missing bluetooth controller transport". I'm guessing that's a red herring though, it is probably related to my intel chipset/graphics.

Ah, some more googling solved it for me. You need to set the proper DMI info with setextradata.

How did you fix it? I'm getting the same issue on Ubunutu 14.04

@jadjoubran: see my OSX Vagrantfile


[Bug] pd make the RaftEngine constructor private

Bug Type (问题类型)
logic (逻辑设计问题)
Before submit

[X] 我已经确认现有的 Issues 与 FAQ 中没有相同 / 重复问题 (I have confirmed and searched that there are no similar problems in the historical issue and documents)

Environment (环境信息)
Server Version: 1.5
Expected & Actual behavior (期望与实际表现)
The RaftEngine is intended to be a global singleton, but the constructor is not private.
public RaftEngine() {
        this.stateMachine = new RaftStateMachine();
    }

Vertex/Edge example (问题点 / 边数据举例)
No response
Schema [VertexLabel, EdgeLabel, IndexLabel] (元数据结构)
No response

@dosu
The RaftEngine constructor is actually public, so I think this breaks the design of the singleton pattern


API Doc not refreshed on website

I compared the API documentation in the source code and on the website. They are different. Is the website refreshed at all? Specifically, the sparse embedding op on website doesn't have the "note" section in the source code:
http://mxnet.incubator.apache.org/versions/master/api/python/ndarray/contrib.html#mxnet.ndarray.contrib.SparseEmbedding
https://github.com/apache/incubator-mxnet/blob/master/src/operator/tensor/indexing_op.cc#L269-L277
@aaronmarkham @thinksanky

The newly added LayerNorm also does not appear in the website.

The site will need to be refreshed for these to appear.


ENH: LHS based OptimalDesign (scipy.stats.qmc)

This adds a new QMC engine base on optimized LHS.
This was removed from scipy.stats.qmc during its review  (#10844).
The optimization loop consists in randomly permuting some points from the design. LHS properties are preserved doing so. All design candidates are compared in terms of centered-C2 discrepancy. This is commonly done in the literature. Also, recomputing the C2 discrepancy is helped with a special method which just update the discrepancy (_perturb_disrepancy from Jin et al. corrected as there where mistakes in the article).
The optimization consists in two loops. An outer loop and an inner loop. For more details, see the reference article: Jin et al., "An efficient algorithm for constructing optimal design of computer experiments." Journal of Statistical Planning and Inference, 2005.
Here is the flowchart of the optimization

Does it need to be updated after the LHS bug fix?

Does it need to be updated after the LHS bug fix?

I will double check (and fix the conflict), but I updated it after.

Thanks for the review Matt. To summarise:

[ ] I will change the optimizer to dual_annealing.
[ ] Add validation and tests + your extended version for the update.
[ ] Fix small PEP8 and things.
[ ] Do I add a metric for the objective function? Ability to do Linf instead of C2 for instance.
[ ] Making _perturb_best_doe a parameter? Not so sure as this would require extra logic. Or this could be done using a main optimization class. We would have OptimalDesign(ABC) and OptimalLatinHypercube(OptimalDesign).

The metric is easy and valuable. The other one is more tricky. Optimizing a sample is already something advanced and optimized LHS is commonly used. For the rest it might be too custom so that we have a generic and useful interface.

@mbaudin47 this might interest you 😉

@tupui is this ready for another look? I was waiting for the last box above to be checked and a test of the effectiveness relative to random swaps.

@mdhaber I refactored to only keep your swapping method. Let me know what you think.
The unknown I had was about making a generic function which would allow another type of QMCEngine, but then there might be little value in such function as it would just be: generate n design and take the one with the best metric.
As for OptimalLatinHypercube, if there is a future interest, we could easily at a method to change the metric. The value here comes from the fancy swapping and the faster update of the metric (there are expressions for some other discrepancies). For now, I think this would be enough.

Thanks @tupui. I will try to take a look before Wednesday, but I'm working on the requested bootstrap enhancements.

Thanks @tupui. I will try to take a look before Wednesday. Since you said this does not need to go in this release, though, I'm working on the requested bootstrap enhancements first.

Sure it's not a priority and I don't mind if we merge this later. Thanks Matt.

Looks good.
Need to fix the rendering of the new example.

Ooops, should be fixed now.

It would be nice @tupui if you could produce a few plots showing the discrepancy as a function of iteration so we can gauge whether the convergence criteria are about right. But that is make-better, so we could wait to consider it once we get requests to do so.

Ok sure, I will try to make something useful.

But the documentation is OK and the tests seem to check what they need to: the discrepancy update calculation works, the method argument produces a reasonable error message when it's not valid, the optimized design is better than the original, and it's still a Latin Hypercube. So not too much holding this up.

Glad to read, thanks Matt 🥳

@mdhaber I have some results. Seems like I was lucky and 100, 10_000 seems fine.
Note that the values are sometimes not round (ex. 9830) because I am doing the mean over some realizations (here 5 as it does not change much).
d=10, Ns=200

Default values: 10000, 100 | 0.4153428604265345 [  55 9830]
Case 1
1000, 100 | 0.5698480676322689 [   2 1000]
5000, 100 | 0.4499603262234236 [  37 5000]
10000, 100 | 0.4153428604265345 [  55 9830]
20000, 100 | 0.41132065360601955 [  100 10716]
50000, 100 | 0.41132065360601955 [  100 10716]
Case 2
10000, 10 | 0.7494520426748597 [ 10 297]
10000, 100 | 0.4153428604265345 [  55 9830]
10000, 1000 | 0.4144494432252772 [   17 10000]
10000, 2000 | 0.4144494432252772 [   17 10000]
Case 3
50000, 10 | 0.7494520426748597 [ 10 297]
50000, 100 | 0.41132065360601955 [  100 10716]
50000, 1000 | 0.35679136503433057 [   71 50000]
50000, 2000 | 0.35679136503433057 [   71 50000]

-----------------------------------------------------------------

d=5, Ns=100

Default values: 10000, 100 | 0.3119049788602787 [ 100 3947]
Case 1
1000, 100 | 0.3815846896444995 [  10 1000]
5000, 100 | 0.3119049788602787 [ 100 3947]
10000, 100 | 0.3119049788602787 [ 100 3947]
20000, 100 | 0.3119049788602787 [ 100 3947]
50000, 100 | 0.3119049788602787 [ 100 3947]
Case 2
10000, 10 | 0.60966387214425 [ 10 114]
10000, 100 | 0.3119049788602787 [ 100 3947]
10000, 1000 | 0.2778197429019117 [   23 10000]
10000, 2000 | 0.2778197429019117 [   23 10000]
Case 3
50000, 10 | 0.60966387214425 [ 10 114]
50000, 100 | 0.3119049788602787 [ 100 3947]
50000, 1000 | 0.24349852818152812 [  259 48654]
50000, 2000 | 0.24292300734518152 [   72 50000]

Code to reproduce
To reproduce you also need to add self.debug = [n_nochange, n_iters] before _random_cd returns.
import numpy as np
from scipy.stats import qmc

d, n = 10, 200
n_conv = 5

rng = np.random.default_rng(9842398122)
lhs_seeds = rng.integers(1000, size=n_conv)

def opt_disc(seed, n_nochange, n_iters):
    qmc_gen = qmc.LatinHypercube(d, seed=seed)
    doe = qmc_gen.random(n)
    disc = qmc.discrepancy(doe)

    qmc_gen = qmc.LatinHypercube(d, seed=seed, optimization='random-cd')
    qmc_gen._n_nochange = n_nochange
    qmc_gen._n_iters = n_iters
    doe = qmc_gen.random(n)
    optdisc = qmc.discrepancy(doe)

    return np.array([optdisc/disc, *qmc_gen.debug])

n_nochange = 100
n_iters = 10_000

data = [opt_disc(seed, n_nochange=n_nochange, n_iters=n_iters)
         for seed in lhs_seeds]
data = np.mean(data, axis=0)
ratio = data[0]
debug = data[1:].astype(int)
print(f"Default values: {n_iters}, {n_nochange} | {ratio} {debug}")

print("Case 1")

n_nochange = 100
for n_iters in [1000, 5_000, 10_000, 20_000, 50_000]:
    data = [opt_disc(seed, n_nochange=n_nochange, n_iters=n_iters)
            for seed in lhs_seeds]
    data = np.mean(data, axis=0)
    ratio = data[0]
    debug = data[1:].astype(int)
    print(f"{n_iters}, {n_nochange} | {ratio} {debug}")

print("Case 2")
n_iters = 10_000
for n_nochange in [10, 100, 1_000, 2_000]:
    data = [opt_disc(seed, n_nochange=n_nochange, n_iters=n_iters)
            for seed in lhs_seeds]
    data = np.mean(data, axis=0)
    ratio = data[0]
    debug = data[1:].astype(int)
    print(f"{n_iters}, {n_nochange} | {ratio} {debug}")

print("Case 3")
n_iters = 50_000
for n_nochange in [10, 100, 1_000, 2_000]:
    data = [opt_disc(seed, n_nochange=n_nochange, n_iters=n_iters)
            for seed in lhs_seeds]
    data = np.mean(data, axis=0)
    ratio = data[0]
    debug = data[1:].astype(int)
    print(f"{n_iters}, {n_nochange} | {ratio} {debug}")

@mdhaber I think I covered all the points. There is just this 0d array question (either the current version is fine or I revert it back to a float) but otherwise all is green now.

In case you did not see @mdhaber, Azure CI is not working right now due to #14529

Yes, I noticed. Would you prefer to wait until those checks can run to merge, or do you think that changes since the last successful fun on Azure are unlikely to cause the Azure tests to fail?

Yes, I noticed. Would you prefer to wait until those checks can run to merge, or do you think that changes since the last successful fun on Azure are unlikely to cause the Azure tests to fail?

IMO you can merge. I did not change anything since the last full run on Azure that was failing the type.

Ok! I think this is a valuable change to LatinHypercube. We added the new functionality (optimization of a Latin Hypercube sample to lower the centered discrepancy) with a minimal change to the interface (just the addition of an optional optimization keyword-only argument) to leave open the possibility of expanding in the future. The documentation changes seem complete, and the essential tests have been added:

accuracy of the method used to update the centered discrepancy w.r.t. elementary permutations of a Latin hypercube
the optimized Latin hypercube is still a Latin hypercube
the optimized Latin hypercube has lower centered discrepancy than the original
basic input validation on the optimization keyword argument

Thanks @tupui!

Thanks Matt for your extensive review which really helped to uncover a lot of things!


Baremetrics Set Attribute

Are the Attribute endpoints included in this gem.
See API reference here:
https://developers.baremetrics.com/reference#set-attributes
I used the client.methods to see what was available, but nothing related to attributes showed up.

Hi @nicgranleese; they aren't but we would welcome a PR to have them added.


Error No module named 'cublas_ops'

ModuleNotFoundError: No module named 'cublas_ops'

Okay, I pushed a change which should fix the issue for you. https://github.com/aredden/flux-fp8-api/commit/00f5d2ca1b0526bfa3dc19d4ce40f1840f8a0e86


Dataset Availability

Hi!
Thanks for the work, I love it. I was wandering if the final clean corpus is available somewhere.

Dear Guillem,
Thanks for your interest in the corpus. We are working to improve the current models and create new ones. Once we finish with these tasks we are planning to upload the corpus to Zenodo.
I will let this issue open so that I can reply it again once the corpus is available. This way you will be notified.
Kind regards,
Asier

Awesome work! I am also interested in the dataset and/or the original collected sentences.
I think this would be a huge help to Mozilla Voice project (https://commonvoice.mozilla.org/es), but they need CC0 licensed sentences for their voice collection tool.

I wanted also to try the scripts (at least the examples provided in the README.md). Is this possible?

Dear @alfem, thanks for your interest.

I think this would be a huge help to Mozilla Voice project (https://commonvoice.mozilla.org/es), but they need CC0 licensed sentences for their voice collection tool.

We will try to do our best with the license. However, I think that 570GB of clean text is too much for Common Voice. Maybe you could try with Wikipedia 🙂
Regards,
Asier

Dear @boriel

I wanted to try the scripts too (at least the examples provided in the README.md). Is this possible?

The scripts from README.md can be run in your PC:

Install Anaconda.
Create an environment: conda create -n <your_environment_name>.
Activate your environment: conda activate <your_environment_name>.
Install transformers library in your environment: conda install -c conda-forge transformers.
Start an interactive python console to check if it works: (type) python.
Copy the following code line by line:

from transformers import AutoModelForMaskedLM
from transformers import AutoTokenizer, FillMaskPipeline
from pprint import pprint
tokenizer_hf = AutoTokenizer.from_pretrained('BSC-TeMU/roberta-base-bne')
model = AutoModelForMaskedLM.from_pretrained('BSC-TeMU/roberta-base-bne')
model.eval()
pipeline = FillMaskPipeline(model, tokenizer_hf)
text = f"¡Hola <mask>!"
res_hf = pipeline(text)
pprint([r['token_str'] for r in res_hf])

Just to clarify: fine-tunning scripts will be released very soon.
Regards,
Asier

We will try to do our best with the license. However, I think that 570GB of clean text is too much for Common Voice. Maybe you could try with Wikipedia slightly_smiling_face

Common Voice is already using Wikipedia, but they are allowed to grab only a couple of sentences from every page. And they need many more!

Dear @alfem

Common Voice is already using Wikipedia, but they are allowed to grab only a couple of sentences from every page. And they need many more!

We will take this into consideration when licensing the corpus but we cannot assure anything. Thanks for the information.

Just to clarify: fine-tunning scripts will be released very soon.

Fine-tunning scripts have been released. However, we are aware about the limited availability of some of the datasets.

Thanks for sharing the models! Could you share a bit more information about the corpus? In particular, I'm looking for models trained in Spanish used in America (e.g. Mexico, ...). Thanks!


organize libc tests into a proper folder, and run some of them on Windows

@bors r+

:pushpin: Commit 325af11a21e5c771bf32feb653be6399c734286a has been approved by RalfJung
It is now in the queue for this repository.

:hourglass: Testing commit 325af11a21e5c771bf32feb653be6399c734286a with merge cf2df2d7d84c8ac06d3ab8200f290e9ec8128951...

:sunny: Test successful - checks-actions
Approved by: RalfJung
Pushing cf2df2d7d84c8ac06d3ab8200f290e9ec8128951 to master...


Send message to a client using PeriodicFunctionService

Hi, how are you. I wanted to ask how I can send messages from the server to a specific client using PeriodicFunctionsService. I need to send to the client monitoring in real time. The documentation explains that the injection of a TopicService be performed.But I do not understand how I can do it. Please could you show me some example of this? Thank you very much.

Hey,
Currently periodic timer is only for internal work. But it's an interesting feature to plug a periodic timer inside topic. I'll look into this feature this week-end.
At the moment you can achieve that by publishing each x second from the client side.

Thanks for your prompt response. I would be very grateful that you could add that functionality. I'll be waiting for it! Regards.

It's now available on dev-master !
Check the docs here : https://github.com/GeniusesOfSymfony/WebSocketBundle/blob/master/Resources/docs/TopicSetup.md#periodic-timer-topic--connection
Feel free to contribute on docs if something is not clear !

Genius! Thank you very much for adding this functionality. I'll try everything to see how I can use it. The documentation seems very good, questions or improvement'll make you a consultation. Greetings and thanks again.


(workflow) ghcr.yml with paths/paths-ignore conditions for pull_request

What is the problem that this fixes or functionality that this introduces? Does it fix any open issues?
Attempt to reduce CI workflow execution time, for PR's only, dependent on affected paths.
E.g. we don't need to build images if changes were only in docs folder or to .md files.
@xingyaoww I have left out the frontend folder as that is covered by other tests, and this PR is not touching push, so this should be no problem?

Give a summary of what the PR does, explaining any non-trivial design decisions
Applied to the pull_request section paths and paths-ignore criteria.

Removed paths, only paths-ignore entries now.

...and run deploy-docs only for changes to docs folder


Fix:  Localize page overflow place

Source issue: https://github.com/metanorma/xmlgraphics-fop/issues/4
Original issue: https://github.com/metanorma/mn2pdf/issues/163

Done in the branch fix/messages.


file backend: using external file - doesn't work

The docs indicate that the file backend can "include" an exernal fil with this syntax:
[file]
filename = "rules.toml"

This doesn't work.
I presume the rules.toml file has to be located in the same directory as the main traefik.toml config file. There is nothing in the docs that says otherwise. Is this the case?
Is there any support for using a file in another directory using either an absolute or relative path?

@opichon
It works as expected.
Have you ever tried /etc/traefik/rules.toml ?


Error running Docker version on macOS

When using the Docker convenience script on MacOS, Docker will complain about the mount path:
> ./p4runtime-sh-docker --grpc-addr <IP_ADDRESS>:28000 --device-id 1 --election-id 0,1 --config /Users/max/projects/stratum/p4info.pb.txt,/Users/max/projects/stratum/pipeline_config.bin
docker: Error response from daemon: Mounts denied:
The path /var/folders/k9/j5gmsz8j0bj3hd7vxy00dm5m0000gn/T/tmpvas2pu4t
is not shared from OS X and is not known to Docker.
You can configure shared paths from Docker -> Preferences... -> File Sharing.
See https://docs.docker.com/docker-for-mac/osxfs/#namespaces for more info.
.

The problem is here: https://github.com/p4lang/p4runtime-shell/blob/master/p4runtime-sh-docker#L71
Which generates a path outside of the default allowed ones:

By default, you can share files in /Users/, /Volumes/, /private/, and /tmp directly.

Just for testing I replace it with this and it worked just fine:
diff --git a/p4runtime-sh-docker b/p4runtime-sh-docker
index 017030c..f92bb9c 100755
--- a/p4runtime-sh-docker
+++ b/p4runtime-sh-docker
@@ -68,7 +68,7 @@ def main():
         fname_p4info = "p4info.pb.txt"
         fname_bin = "config.bin"

-        tmpdir = tempfile.mkdtemp()
+        tmpdir = "/tmp/p4rt-shell" #tempfile.mkdtemp()
         logging.debug(
             "Created temporary directory '{}', it will be mounted in the docker as '{}'".format(
                 tmpdir, mount_path))

Is using Docker on Mac simple not supported?

It's just never been tested on macOS.
I'm fine with using tempfile.mkdtemp(dir="/tmp") since /tmp seems to be shared by default with the hyperkit VM used by docker. That will work on both Linux and macOS. If you want to open a PR, that would be great.


[Model Update]: Digital Product Passport

Update Reason
General

Payload names are added where missing and needed
CompanyIdentification linked to shared.BusinessPartnerNumber (BPNL and BPNS) OR address and contact
Characteristics
physical dimensions linked to shared.quantity
Sustainability
Carbon footprint is seperate and anymore a list
Carbon and environmental footprint have new properties "performance class", "manufacturing plant" and "declaration"
renewable and recycled content are now identified trough a share
Typology
class is now mandatory
Identification
A list of identifiers can be provided
Operation
manufacturing date is mandatory
Handling
spare parts are now mandatory and the list of suppliers are connected to the spare parts

MS1 Criteria

[ ] The model that should be updated exists
[ ] The referenced use case exists
[ ] The potential updates are discussed with all stakeholders
[ ] A decision on versioning was made in consensus with the affected parties according to the guidelines.
[ ] The potential update will be either

[ ] backward compatible or
[ ] a (chain of) migration strategy(-ies) from all non-deprecated previous versions to the new model will be developed

[ ] A modeling steward has been assigned

Modeling_Team

@MasonTsystems


Add next lint script

This PR adds the next lint script and fixes all the eslint warnings/errors.

📝 Changed routes:

/

(view full size)

Commit deb065d7d58cc657bee1195f4630c9920193bf8e (https://hackathons-28a63zfp4.hackclub.dev).

Yep ignoreDuringBuilds: true should ignore it during builds


Increase KIND log verbosity.

What this PR does / why we need it:
Experimental change to retain and show all possible KIND logs.
We probably need to introduce some notion of verbosity in kuttl to make this opt-in.

Does this give you more logs than already existing in the collected artifacts/kind-* bundle?

On the other hand, I just had a situation where this additional logging helped a lot so 👍

@zen-dog ,

Does this give you more logs than already existing in the collected artifacts/kind-* bundle?

Yes, this is complementary:

the retain means there will actually be something to collect in case cluster create fails mid-way,
the logging means we will get output from kind itself


fix: set "host is down" as corrupted mount

What type of PR is this?
/kind bug
What this PR does / why we need it:
fix: set "host is down" as corrupted mount
When SMB server is down, there is no way to terminate pod which is using SMB mount, would get following error. This PR regard host is down as corrupted mount dir, and then I would work out another PR in k8s to skip UnmountVolume.TearDown process if host is down
Apr 20 11:11:52 aks-nonzone-17963928-vmss000000 kubelet[8516]: E0420 11:11:52.618206    8516 nestedpendingoperations.go:301] Operation for "{volumeName:kubernetes.io/csi/smb.csi.k8s.io^pvc-da830ab0-e8d2-4e0e-89f4-906a4e4398a4 podName:90ee5993-7f80-4fe3-9d88-7bc89b19cbfe nodeName:}" failed. No retries permitted until 2021-04-20 11:11:56.61815874 +0000 UTC m=+337.879942743 (durationBeforeRetry 4s). Error: "UnmountVolume.TearDown failed for volume \"persistent-storage\" (UniqueName: \"kubernetes.io/csi/smb.csi.k8s.io^pvc-da830ab0-e8d2-4e0e-89f4-906a4e4398a4\") pod \"90ee5993-7f80-4fe3-9d88-7bc89b19cbfe\" (UID: \"90ee5993-7f80-4fe3-9d88-7bc89b19cbfe\") : kubernetes.io/csi: mounter.TearDownAt failed to clean mount dir [/var/lib/kubelet/pods/90ee5993-7f80-4fe3-9d88-7bc89b19cbfe/volumes/kubernetes.io~csi/pvc-da830ab0-e8d2-4e0e-89f4-906a4e4398a4/mount]: stat /var/lib/kubelet/pods/90ee5993-7f80-4fe3-9d88-7bc89b19cbfe/volumes/kubernetes.io~csi/pvc-da830ab0-e8d2-4e0e-89f4-906a4e4398a4/mount: host is down"

Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Release note:
fix: set "host is down" as corrupted mount

/assign @msau42
I see PR template of this repo is
Sorry, we do not accept changes directly against this repository. Please see
CONTRIBUTING.md for information on where and how to contribute instead.

Shall we use another contribution welcome PR template?

Sorry, this is really confusing. But basically this repo is a mirror of kubernetes/kubernetes/staging/mount-utils. So the PR you have for kubernetes/kubernetes will get published here whenever a kubernetes tag is made.

Sorry, this is really confusing. But basically this repo is a mirror of kubernetes/kubernetes/staging/mount-utils. So the PR you have for kubernetes/kubernetes will get published here whenever a kubernetes tag is made.

@msau42 in that case, there would be a dependency on k8s release, that's time consuming.
I would still think SMB CSI driver could depend on https://github.com/kubernetes/utils/, it's more efficient to make some changes. Can we merge this PR into k8s/utils first? https://github.com/kubernetes/utils/pull/203
In the meantime, I will make same code change into k/k

Intermediate kubernetes tags are also pushed too. So I don't think it should be a long wait.

https://github.com/kubernetes/kubernetes/pull/101398 would fix this issue.


Cannot add linear consumption sensor from climate sensor / lack of error message

I am trying to add a 'climate' sensor and calculate consumption from its attribute 'heat requests'. Using linear. During the second form, it reloads just after clicking on the submit button with an exclamation mark but without any comment and the form is emptied. Seems like a bug.

Could you check the error logs for any errors?

Sorry, can you tell me how? I am trying any other possibilities with linear config, no luck.

This error originated from a custom integration.
Logger: custom_components.powercalc.config_flow
Source: custom_components/powercalc/config_flow.py:359
Integration: Powercalc (documentation, issues)
First occurred: 12:22:37 (5 occurrences)
Last logged: 12:25:22
Entity domain not supported for linear mode. Must be one of: fan,light,media_player, or use the calibrate option

ok, so I am trying 'climate' and use 'heat request' which is how much is the valve open that is 0-100%. I do not think that this can be done by fixed strategy.
You should add the error message from the log to the dialog. Thanks for pointing me here.

I see, need to figure out why this error is not shown in the GUI.
Anyway you need to use the calibrate mode as stated, because otherwise powercalc does not know what the linear scale needs to be.
For fans (percentage) and media players (volume) it will set from 1-100, and for lights (brightness) from 1-255, but for other domains we cannot know this, so you'll need to specify that yourself.

I was thinking about it based on the error message. Thanks!

Translation will be fixed in next version. See #1257


 We require pure SVGs. Could you please attach a pure SVG or the AI file it's based on. You need to ZIP the file to upload it to this issue.
Thanks.

OK I'm looking into it. Thanks for the heads-up.

@dankohn I've attached a pure SVG as requested, please let me know if it's correct. Thanks.
on-prem.svg.zip

You're set. Thank you!
https://landscape.cncf.io/selected=on-prem