Optimize Tokenizers in Python

This PR completes the first "TODO" in the README by optimizing the tokenizers in Python while respecting the clean code style as much as possible. Of the two, the regex tokenizer is optimized way more. Using the train.py script on the Taylor Swift text with verbose=False and doing 2k merges (i.e. vocab_size = 2256), this regex tokenizer is 45X faster than the stock version of the repo. That multiplier increases as you train larger vocabs on more data.
Main Optimizations

Safely batching merges to reduce the number of calls to get_stats and merge (actually now multi_merge). "Safely" means respecting which combinations of bytes are merged, and keeping them in a non-problematic order. They aren't merged in the same exact order, but the slight shifts to the order of merges are only performed when we can be sure that all texts would get tokenized the same way. I'm happy to explain that in more detail, but moving on...
Use bytes objects and lists interchangeably. For the purposes of this repo, bytes objects already have all the functionality we need from lists, so there's never a reason to unpack a bytes object into a new list just so you can work with it. In other words:

li = [97, 98]
by = bytes(b'cd')
li.extend(by) # -> li will be [97, 98, 99, 100]

Use comprehensions where possible. This is most notable in the merge method which now uses a list expression instead of a regular for loop. More on that below. One surprising find was that replacing a generator expression with a list expression saves a considerable amount of time even though it's just adding the square brackets.
Use collections.Counter and heapq.nlargest. Related to the Counter, the get_stats function was being called unnecessarily in _encode_chunk. This was replaced with zip(ids, ids[1:]) since only the keys from the get_stats Counter were being used anyway.

There were smaller things too. The commits in this fork are pretty pointillistic often with more details about each optimization.
Options for this PR

This PR passes all the tests and tries to respect the style of the repo while adding considerable performance improvements. So it is mergeable (ha!) as is. Note that I use the walrus operator, :=, so accepting the PR will make this repo require Python 3.8+.
Revert changes to basic.py. The greatest pedagogic value is in the basic.py tokenizer since that's the simplest to understand. In particular, the merge function (in base.py) that the basic tokenizer uses could be reverted to its simpler form since the list comprehension I replaced it with is probably the most unreadable thing in this PR. If you go that route, it would be good to keep both versions of merge, since the regex tokenizer still uses one-at-a-time merges for individual words in _encode_chunk. That way basic.py can optimize for readability, and regex.py can optimize for speed.
Refactor these changes into an additional tokenizer option and keep basic.py and regex.py the same.

This deep dive has been a lot of fun! Thanks for making this topic so much more accessible.

With the latest commits and in python 3.12 this is much faster:

training runtime reduced ~98%
encoding runtime reduced ~54%
decoding runtime reduced ~15%

It would be great to try this using the jit-compiled version of python 3.13 and also the no-gil 3.13 version. I tried the May beta version of 3.13 and it was a little faster, but importing tiktoken no longer worked. I couldn't figure out how to build the jit-compiled and no-gil versions of python 3.13, but these seem like the most promising ways to get further speed improvements here.
The current goal is make this performant enough to be able to comfortably train a full ~100k vocab on gigabytes or even terabytes of data on a laptop. I want it to be easy for people to experiment with new tokenization approaches at a scale where they can train them and be confident they're actually better/worse without needing special hardware. I think the only other thing necessary for that goal is to manage the processing of big files that don't fit in memory.

Not breaking after hitting a collision and just skipping it, can lead to a suboptimil vocab compared to BPE. Here is a minimalistic example:
let 99.99% of the text chunks in ids be  aa....a (32 times a)  and there is a single chunk yz, and suppose we can do 5 merges. Clearly BPE adds just these tokens to the vocab: β=aa, β2, ..., β5. So the merger of yz is not in the final vocab.
Now, in your code, in the first iteration of while, all the merges are from original characters (because there is no mergers yet). Assuming max_clutch_size =2, the first pair to merge is aa; and the second pair is yz. Thus a different vocab, with 50% efficiency compared to BPA.
        while merges_remaining > 0:
            seen_first = set() # set of ints that were seen in the first position in pairs
            seen_last = set() # set of ints that were seen in the last position in pairs
            pairs_to_merge = {}
            # count the number of times every consecutive pair appears
            stats = get_stats(ids)
            # find the pairs with the highest counts
            # use min to make sure you don't consider pairs beyond the vocab size
            top_pairs = nlargest(min(max_batch_size, merges_remaining), stats, key=stats.get)
            for first, last in top_pairs:  # pairs are (first, last) tuples
                if first in seen_last or last in seen_first:
                    # skip this pair because it overlaps with a previous pair
                    # still add it to the seen sets to avoid future overlaps
                    seen_first.add(first)
                    seen_last.add(last)
                    continue # skip this pair but keep looking for mergeable top_pairs

Interesting example, but that is not how aaaaa...aaaayz would get merged. When you get to 'ay', the 'a' would trigger the if first in seen_last or last in seen_first: condition, because 'a' has shown up previously as the last element in a previous pair, so it meets the last in seen_first part of the condition. Critically though, 'ay' also get added to the seen_first and seen_last sets before the continue. This means that when you get to the 'yz' pair, that pair will also get skipped since 'y' will be in the seen_last set, and now appears first. That triggers the first in seen_last part of the condition.
In addition, while I know this is just an example, you would never train a text of 34 characters with a vocab of 33 new tokens, which is the length of top_pairs it would take to get all the way to the last pair on the first try.
If the continue is still bothersome, you can replace continue with a break and then you have a further layer of reassurance that merges won't get made that don't belong in the vocab. With break instead it's obviously not as fast but it's still a pretty solid speed up.
If you want to find a problematic case, it's pretty easy to train the regex tokenizer on a test text. If it makes merges that the basic tokenizer wouldn't, I would really appreciate knowing about it! The thing is that the respective merges dictionaries will not be in the same order, but their dict.keys() objects (~basically sets) should be equal.

Thank you for your answer. I edited my previous comment to make things clear.
As you mentioned, this exaggerated example is to demonstrate that rare pairs can get into the vocab. In practice it may lead to a little loss in compression, but even 1% is a lot, because it directly affects the throughput of the LLM.
As said, issue #5 discusses the way to cut the number of merges from V, size of vocab, to log(V), guaranteed with same vocab. But that is beyond the purpose of this PR as a python optimization. So as you said, just replacing continue with break seems the proper thing here.
And I went trough your PR closely, very solid indeed. Educational for me.

Thinking about it more, you have a point. I misspoke earlier when I said the there were 33 merge candidates in a sequence of 34 characters. Since this is only unique pairs at this stage, 'aaaa...aaayz' would be only three candidates, 'aa', 'ay', and 'yz' and they would merge as I mentioned before. But if the sequence were 'aaaa...aaayzyz', then the top two pairs would be 'aa' followed by 'yz', so even if you had a batch of just two merges, you would merge 'yz' before merging token 256 ('aa') with another 256. This approach does allow for things to happen out of order, but the idea is that there are still multiple batches so this would merge 256 and 256 on the next batch. So, if you had a situation like this where you made a merge where the new token would repeat many times in a row once merged into the sequence, and that happened on your last batch, then that would impact the vocabulary.
I think it's probably fine if the 100,001st most-common merge happens instead of the 100,000th merge if we're talking about a 100k vocab, but you definitely have a point. Maybe a good simple solution would be to have the last ~10 merges be simple ones. I'll have to keep thinking about it. Thanks for pointing this out!

Please note that the example is about regex.py, not simple.py.

Closing because my fork has developed enough to have a different objective from minbpe's educational one. My fork's purpose is now to facilitate experiments with the tokenization process by providing a fast, lightweight, and pure-python implementation that can build a vocab from gigabytes of text on a basic laptop in minutes. This involves things like safely batching merges, saving/loading dictionary representations of datasets, support for the datasets library, special handling of stop words, etc. which are important things to have for a research-grade tokenizer but would overcomplicate the educational purpose of this repo. I'm happy to make a different pr adding a link to a "notable forks" section if that's wanted.


Debian stable corretto images

I see that Debian based Dockerfiles exist for Corretto (e.g. https://github.com/corretto/corretto-docker/blob/main/17/slim/debian/Dockerfile) but:

they don't seem to be published on https://hub.docker.com/_/amazoncorretto
the image is based on old debian stable, as current stable is bullseye -- https://www.debian.org/releases/stable/.

Are there plans to publish these Debian based Corretto images? If so, would you accept a PR to update the Debian dockerfiles to inherit from debian:stable-slim rather than debian:buster-slim here so that they always follow upstream Debian stable?

Thank you for contacting us. We would welcome a PR, but we have no plans at this time to publish a Debian based image.


Add Eq to the "helper wrappers" (<?>), (<!>), (<#>)

Thank you for contributing that!

Now I'm wondering post-factum whether we should just add all the easy to derive instances for them, or at least Ord so someone could potentially put those in a Map, but that sounds like it's getting extremely niche.

Thanks for the awesome library btw! :heart:

You're welcome!


pvsystem.sapm_celltemp

The pvsystem.sapm_celltemp function currently returns a dict {'tcell':tcell, 'tmodule':tmodule}. I think it should return a DataFrame with the keys temp_cell and temp_module (as described in the wiki).
I actually like the simplicity of the function parameter names here, but maybe they too should be changed.

I think they should be changed.


✨ Make next explicit flow control

tldr; You must call next in middleware unless you want to stop the
exectution.
Previously, next was called implicitly by middleware composition, if
it was not called from the middleware itself.
While this behavior is convinient for some cases, it takes away the
possibility to skip the subsequent middleware execution for the purpose
of caching or any other early exit. This behavior is consistent with
other frameworks, like Express or Koa.

Codecov Report

Merging #8 into master will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master        #8   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            5         5
  Lines           91        85    -6
  Branches        15        14    -1
=========================================
- Hits            91        85    -6

Impacted Files
Coverage Δ

src/lib/server/middleware/compose-middleware.ts
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 42988ed...a120f5c. Read the comment docs.


Result list in select tab: Check whether page scroll can follow highlighted row

Unimportant feature request: When I am looking at a list of surfaces
or topographraphies, I can use the up/down arrows to navigate (which
I like). However, if I go too far up or down, then the selected
topography goes right off the screen. Is there a way to have the
page scroll to follow the highlighted surface?

Not sure, but Michael should be able to answer this.
Good idea and I don't know yet. I'll check ..

Yes, it is possible. @tdbjacobs , this will be available in the next release. Maybe I have to tweak another parameter then,
let's see.


Example data NMR vs MS data

First dataset for example data is NMR data, not Mass spec data. The label for downloading example data is incorrect.

Fixed


TypeError: undefined is not iterable (cannot read property Symbol(Symbol.iterator))

Describe the bug
Uploaded a large blob (GZ compressed SQL) and was inspecting the resulting tree. I started walking down the Link/0 entries, upon opening the 3rd Link/0 entry I hit this error.
Backtrace:
Failed to resolve path /QmPoUfwsfbYgJLwamL3mc8C2jbUhi6M1EcT3Fchp3X64NY/Links/0/Links/0 TypeError: undefined is not iterable (cannot read property Symbol(Symbol.iterator))
    at e.exports (5.311785e4.chunk.js:2)
    at Object.t.resolve (6.16d89f90.chunk.js:2)
    at e.<anonymous> (6.16d89f90.chunk.js:2)
    at c (5.311785e4.chunk.js:2)
    at Generator._invoke (5.311785e4.chunk.js:2)
    at Generator.next (5.311785e4.chunk.js:2)
    at o (5.311785e4.chunk.js:2)
    at 5.311785e4.chunk.js:2

To Reproduce
Steps to reproduce the behavior:

Inspect my local blob (Produced via https://ipfs.io/ipfs/QmPoUfwsfbYgJLwamL3mc8C2jbUhi6M1EcT3Fchp3X64NY?filename=window_post_w_receipts.sql.gz)
Click "Link/0" twice.
Observe error.

Expected behavior
I expected the whole tree to be traversable without error.
Screenshots

Desktop (please complete the following information):

OS: MacOS 10.14.6
Browser: IPFS Desktop UI v2.11.4 Revision 69cd414

Additional context
Add any other context about the problem here.

Thanks @placer14 -- we will look into this deeper in our next weekly triage! (Please note that we're pretty low-bandwidth this week, so please accept our apologies for any delays.)
@lidel - the error here is a duplicate of https://github.com/ipfs-shipyard/ipld-explorer-components/issues/282, FWIW. May also be related to https://github.com/ipfs-shipyard/ipld-explorer-components/issues/279 in some way?
I'm going to leave this in the ipfs-webui repo until we triage, but will want to move to ipld-explorer-components after we discuss.

Note: resolving this may remedy https://github.com/ipfs-shipyard/ipld-explorer-components/issues/279.


add use-ironic design document

Start filling in some of the details about using Ironic.
Signed-off-by: Doug Hellmann<EMAIL_ADDRESS>

LGTM, but would you like to squash this first?


Support different python versions

Linux

[ ] Python 2.7
[ ] Python 3.3
[ ] Python 3.4
[ ] Python 3.5

Windows

[ ] Python 3.3
[ ] Python 3.4
[x] Python 3.5
[x] Python 3.6

Currently having issue compiling GameboyCore using Visual Studio 9/10

Added Python 3.6 version that needs to be complete.
#9

Need to add 64 bit builds for Windows and Linux

Python 3.3 and 3.4 could work if gameboycore-python is compiled manually, with cmake, into a .pyd file since that is just a .dll.
CMake will have to generate the correct site-packages info and correct library names

Completing Python 3.3 and 3.4 on Windows in a separate thread


Only run CI against MRI

IIRC, our minimum Ruby support requirement is:
the latest version of MRI.
It mostly comes down to "what does thoughtbot use?",
and we use whatever Heroku uses.

mike-burns [18:14]

Oh hey, I said that! and I stand by it.
I don't know enough about Travis configuration to know whether this is correct, but it matches what I had said.

I support this change.


HDDS-4392. [DOC] Add Recon architecture to docs

What changes were proposed in this pull request?
Add Recon Architecture and API reference documentation
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-4392
How was this patch tested?
Tested with hugo in local.
cd hadoop-hdds/docs
hugo serve

@swagle @avijayanhwx Thanks for the review comments! I have addressed them all. Please review again.


Enhancement: Install dependencies in install section

This PR

[x] installs dependencies with composer on Travis in install section

💁♂ For reference, see https://docs.travis-ci.com/user/job-lifecycle/.

Thank you, @svpernova09!


Globally declared variables

Can I ask why in the link function there are three items that are not defined at the $scope level? The last three in the snippet below.
This is causing clashes.
        link: function ( $scope, element, attrs ) {

            $scope.backUp           = [];
            $scope.varButtonLabel   = '';
            $scope.scrolled         = false;
            $scope.spacingProperty  = '';
            $scope.indexProperty    = '';
            $scope.checkBoxLayer    = '';
            $scope.orientationH     = false;
            $scope.orientationV     = true;
            $scope.filteredModel    = [];
            $scope.inputLabel       = { labelFilter: '' };
            $scope.selectedItems    = [];
            $scope.formElements     = [];
            $scope.tabIndex         = 0;
            $scope.clickedItem      = null;
            prevTabIndex            = 0;
            helperItems             = [];
            helperItemsLength       = 0;

Hi @RichLinnell ,
Not all variables/functions have to be declared in the $scope. In fact, usually developers will try to put as little stuff as possible into the $scope.
This is just a guess, but the problem happens because I didn't use strict mode to code this (version 3.0.0 is strict). Usually it will also trigger some errors if you try to minify the code.
The workaround is to declare 'use strict' at the beginning of the code, and use explicit variable declaration, such as:
var prevTabIndex = 0;
var helperItems = [];
var helperItemsLength = 0;
There might be other functions need declaration as well. Just add them and you're good to go.
Hope this helps.

yeah, it was WebEssentials bundle/minification that was causing the blowout.  I just excluded the file for now.  I'll try out the strict mode when I have some spare time.


Bloc listeners don't unsubscribe from bloc stream when widget is disposed

Hi, there is a bug, which prevents hooked bloc listeners from unsubscribing from bloc stream. The reason of this is that inside of bloc listener hooks, useMemoized is used, while it looks like useEffect was intented to be used there and because of this mistake, bloc stream subscription is not cancelled when widget is disposed.
So, it means that when we attach listener multiple times to the same long living bloc, listener will be executed multiple times, even though we already disposed widgets which attached those listeners. For example, when listener would be attached inside of dialog which we would open/close multiple times, we would receive this issue.
PR with fix: #69

Simple app which has this issue reproduced, listener continues to be called even after dialog is closed and then when you reopen dialog, one more listener is attached, so you have 2 calls every second
import 'dart:async';

import 'package:flutter/material.dart';
import 'package:flutter_bloc/flutter_bloc.dart';
import 'package:flutter_hooks/flutter_hooks.dart';
import 'package:hooked_bloc/hooked_bloc.dart';

void main() {
  runApp(const MyApp());
}

class MyCubit extends Cubit {
  MyCubit() : super(0);

  start() {
    Timer.periodic(const Duration(seconds: 1), (timer) {
      emit(state + 1);
    });
  }
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      home: BlocProvider(
        create: (_) => MyCubit()..start(),
        child: Builder(builder: (context) {
          return ElevatedButton(
            onPressed: () => showDialog(
              context: context,
              builder: (_) => BlocProvider.value(
                value: BlocProvider.of<MyCubit>(context),
                child: const Dialog(),
              ),
            ),
            child: const Text('Dialog'),
          );
        }),
      ),
    );
  }
}

class Dialog extends HookWidget {
  const Dialog({super.key});

  @override
  Widget build(BuildContext context) {
    final myCubit = useBloc<MyCubit>(closeOnDispose: false);

    useBlocComparativeListener(
      myCubit,
      (bloc, current, context) {
        print('LISTENER CALLED');
      },
      listenWhen: (p, c) => true,
    );

    return ElevatedButton(
      onPressed: () => Navigator.pop(context),
      child: const Text('Close Dialog'),
    );
  }
}

@ruslic19 thx for PR and support. Version with your fixes: 1.4.2
https://pub.dev/packages/hooked_bloc/versions/1.4.2


Define and setup how we measure "success" of our website

Putting a bunch of pages online is one thing, but how do we know they are meeting their goals? What, exactly, are those goals anyway?
This task is for us to answer those questions and setup whatever we need in order to get the required feedback and/or data.
Additional considerations
While the V1 website will be modest in terms of scope, it will nonetheless be the place that all Odyssey users go for docs, help, inspiration, etc. We should therefore think about what kind of insights we could and would like to get, those can inform future iterations of the website (and maybe even Odyssey as a whole).
Note: When thinking about what data to collect and how to do it, we also need to be mindful of potential privacy concerns (there's a reason browsers increasingly block indiscriminate tracking scripts!) and negative impacts on page load performance (3rd party JS scripts are notorious for slowing down page load times).
Relates to #483

edited: typo

2 Nov - James, Svetlana and Zoe to regroup to discuss initial recommendations

24 Nov: Waiting on James & co to pull his finger out

@james-nash @JohannaJoergensen is there a reason for moving this to backlog? It is in progress, but makes sense to not include in stand-ups around Dec 14th release as metrics will add to dev work and are not yet defined.

@zdxd @james-nash We can label this issue as future site work if not part of the soft launch

Updated to website- future milestone

18 Jan - work in progress; work completed - feedback to follow

same as above plus discussed with CX and recommend site team meet with them to discuss above


SB5 React Native multiple installation issues

Describe the bug
Installation with npx -p @storybook/cli sb init does three weird things:

"WARNING: We noticed you're using the useBuiltIns option without declaring a core-js version".
Not a big deal I guess, but concerning.

"Please choose a version of "@storybook/react-native" from this list:"
I've chosen the latest 5.1.0-alpha.13. After installation, this end up installed and in my package.json:

    "@storybook/react-native": "^5.0.5",
    "@storybook/addon-actions": "^5.0.5",
    "@storybook/addon-links": "^5.0.5",
    "@storybook/addons": "^5.0.5",

Not what I selected.

Last and most troubling, when I run yarn run storybook or npm run storybook, all I get is:

**/bin/sh: storybook: command not found**

Indeed there is no storybook binary in /node_modules/.bin/
To Reproduce
Run react-native init test
cd test
npx -p @storybook/cli sb init
npm run storybook
Expected behavior
To start storybook without any issues
System:

OS: MacOS
Device: Macbook Air 2018
Framework: React Native 0.59.1
Env: node v10.15.3, npm 6.4.1, yarn 1.15.2

Thanks for looking into this

Just to clarify, I'm getting the same error (command not found) when I install specifically "@storybook/react-native": "^5.1.0-alpha.13"

@EskelCz This is related to https://github.com/storybooks/storybook/issues/5893
What happens when you run npx -p @storybook/cli@next sb init so that it gets the alpha versions?

@shilman Thanks, that installs the right version.
Then it crashes on missing peer dependencies (babel and webpack), which are hard to notice (since they became such a commonplace these days) but after installing those it finally works. :)
Thank you

@shilman I think I've found how to replicate it:
start-storybook works
storybook start gives the error

@EskelCz thanks for following up on this. This changed in 5.1, but the docs page is for the latest stable version. Here's the migration for 5.1:
https://github.com/storybooks/storybook/blob/next/MIGRATION.md#react-native-server
Any suggestions on what we should do about this?

Oh, I missed the last instruction of the migration info. So I guess it's fine, everyone with the same problem will find this thread.
Just update the docs with the new command when it's out of alpha.
Thanks for your patience! :)

i have installed storybook with npx -p @storybook/cli sb init today...
as suggested from the the cli,  i installed core-js@3, then the start script was not found.
but after installing @storybook/react-native-server and updating storybook script to `start-storybook -p 7111" i solved the problem


Unhandled exception in native epoll: io.netty.channel.unix.Errors$NativeIoException: syscall:read

a service running netty 4.1.12 and using native epoll transport sees the following in its logs:
WAR [20170629-22:20:03.859]: Unhandled exception in connection with /<IP_ADDRESS>:34060, shutting down connection
WAR [20170629-22:20:03.859]: io.netty.channel.unix.Errors$NativeIoException: syscall:read(..) failed: Connection reset by peer
WAR [20170629-22:20:03.859]:     at io.netty.channel.unix.FileDescriptor.readAddress(..)(Unknown Source)

that's on CentOs 7.3.1611 + java 1.8.0_131
We don't have a repro and it shows up intermittently.

@dschobel which class is logging this ? Connect-reset is quite usual, from what I can tell its not something within netty itself that logs this:
─[11:54:32] norman@Normans-MacBook-Pro-8/~/Documents/workspace/netty
( 4.1: ✔
╰─○ grep -lr "Unhandled exception in connection with" *

╭─[11:54:42] norman@Normans-MacBook-Pro-8/~/Documents/workspace/netty
( 4.1: ✔

@normanmaurer yeah the first log line is from our last line exception handler that finagle installs. It seems strange that given the same pipeline we'd see socket read errors caught at different handlers just by varying the transport implementation.
I'm looking into a way to forge RSTs to see if I can get  a simple repro.

To clarify this issue revolves around seeing different behavior from the NIO transport (not seeing connection reset) and Epoll (seeing connection reset)? Connection reset typically is the result of your peer sending a RST (set SO_LINGER to 0 and close to send RST) which depends upon your peer and may occur in the wild.

@Scottmitch the RST happens in both cases but with the Epoll transport it's happening at a different point in our pipeline. I'll do some more testing using the SO_LINGER trick

turns out it was our exception mapping code not being aware of the new exception types from Epoll. sorry for the false alarm

No worries thanks for the update 👍

Am 10.07.2017 um 18:56 schrieb Daniel Schobel<EMAIL_ADDRESS>Closed #6950.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

+1 ... thanks @dschobel !


Add a CI job for python package

We can install the dkv-server and dkv-cli with python:
pip install -e deploy/python -v

then use:
dkv-server

and
dkv-cli

Lgtm

The service of codecov crashed.  Let's merge this PR first.


implement use_blob for execute_mdx_csv, execute_mdx_dataframe

Similar to #848, but reversed, we could channel data between Python and TM1 through blobs to improve the performance of the execute_mdx_csv and execute_mdx_dataframe functions.

resolved with #885


Cannot unmute in amethyst from damus created mute list (kind 30000 and kind 10000)

client initiating action
Amethyst
client receiving or observing action (or lack thereof)
Amethyst (lack thereof)
timestamp
Reported by dont believe the hype
npub1nxa4tywfz9nqp7z9zp7nr7d4nchhclsf58lcqt5y782rmf2hefjquaa6q8
what happens
Amethysts uses kind 10000 for mute. Damus 30000.
If a user adds to a mute list from damus, and tries to remove from amethyst nothing happens.
suggestion

Thanks @believethehype for the report ser 🤝.
@vitorpamplona is it possible to update to non-legacy mute list kind on amethyst?
cc @jb55

@vitorpamplona is it possible to update to non-legacy mute list kind on amethyst?

hum.. unless I am misunderstanding this, we are using the newer version. Damus is using the deprecated mute list according to NIP-51

You are correct - I got it wrong, as advised by @believethehype

Related Damus issue https://github.com/damus-io/damus/issues/1880

Related Damus issue damus-io/damus#1880

This is pending addition of keyword mute in damus (see https://github.com/damus-io/damus/issues/1718)


Internalisation checks for symbol application

fixes #443 by checking each symbol application has the right number of arguments of the correct sort.

Checked and this does not break any existing kevm/kontrol tests


Sorting by recent not working.

ttu-ttu default behaviour is to put first the most recently open book in front.
This is not happening either in the home screen of the app, neither inside the ttu-ttu reather.
Current sorting seems to be by order of being added, not even alphabetical. So when you mass add books you get the random sorting sometimes.

Seems to be a recent addition to ttu.
https://github.com/ttu-ttu/ebook-reader/commit/8fc55ff534832602a74cadd45eb829d6f2edfeec
It might not be straightforward to migrate to the latest version since there could be breaking changes.
To update the local build manually, it would require modifying the indexedb and adding the relevant timestamps. It would incompatible with the data of current users of Immersion Reader.

How about alphabetical order as a stop gap until (if ever) gets updated to the last version of ttu? Is that hard to modify?
I know it's stupid, but the random order is really bothering me.

Well, we first have to define what alphabetical order means for Japanese book titles. Is it the localeCompare() in Javascript which separates the three writing systems when sorting, or do we sort based on the pronunciation of each mora based on a morphological analyzer like mecab,
Not to mention the sorting might be different for users with different language settings on their phones.

I think it doesn't need to be that deep.
Right now I have 3 books with the same title only differentiated by the number and they are sorted like this Vol 3, Vol 1, Vol 2.
I would be fine if the series were together in proper order, even if they were not strictly in "alphabetical" order among them.

Since I plan to remove ttu menu scree and only show the custom screen for listing books in the upcoming version, I can save the reading sessions and sort them accordingly.
To give more context, the main menu screen fetches from the IndexedDB that ttu uses.
In the event that the main menu detects that a new book is added or removed, the new book will given the timestamp of latest sync time and the old books will be removed.

Latest ttu version is ported. No visible issues so far.
https://github.com/mathewthe2/immersion_reader/commit/d35ea1e034a13b76c2bcb1c4175a8d9e4110b152

Update: there is a critical image display bug with the latest ttu version on the mobile browser, and I can't locate the core issue. I've rolled back to an older version without sorting. I will update this when I re-implement this feature


rasa train does not pick GPU

Rasa Open Source version
3.2.10
Rasa SDK version
3.2.2
Python version
3.8
What operating system are you using?
Linux
What happened?
We have tried rasa train with 3.2.10 version and few other 2+ versions also, but rasa train does not pick GPU. In logs it does not complain that there is no GPU. There are other applications which are able to access GPU, as can be shown in nvidia-smi output below.  It is a problem with rasa only, that it is not GPU, on this specific system.
Please suggest what else to do to make it work.
nvidia-smi
Wed Dec  7 14:41:15 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.73.08    Driver Version: 510.73.08    CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla T4            On   | 00000000:00:1E.0 Off |                    0 |
| N/A   29C    P8    13W /  70W |   2880MiB / 15360MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A    443955      C   ...ments/layoutlm/bin/python     2877MiB |
+-----------------------------------------------------------------------------+

rasa --version
Rasa Version      :         3.2.10
Minimum Compatible Version: 3.0.0
Rasa SDK Version  :         3.2.2
Python Version    :         3.8.10
Operating System  :         Linux-5.15.0-1021-aws-x86_64-with-glibc2.29
Python Path       :         /home/ubuntu/rasa_3/bin/python3

rasa train log, which does not complain anything about GPU:
/home/ubuntu/rasa_3/lib/python3.8/site-packages/rasa/shared/utils/io.py:98: UserWarning: You have an overlap between the 'RegexEntityExtractor' and the statistical entity extractors DIETClassifier in your configuration. Specifically both types of extractors will attempt to extract entities of the types inv_date, po_num, inv_num. This can lead to multiple extraction of entities. Please read 'RegexEntityExtractor''s documentation section to make sure you understand the implications.
  More info at https://rasa.com/docs/rasa/components#regexentityextractor
2022-11-30 08:08:01 INFO     rasa.engine.training.hooks  - Restored component 'RegexFeaturizer' from cache.
2022-11-30 08:08:47 INFO     rasa.engine.training.hooks  - Restored component 'LexicalSyntacticFeaturizer' from cache.
2022-11-30 08:09:57 INFO     rasa.engine.training.hooks  - Restored component 'CountVectorsFeaturizer' from cache.
2022-11-30 08:11:26 INFO     rasa.engine.training.hooks  - Starting to train component 'DIETClassifier'.

/home/ubuntu/rasa_3/lib/python3.8/site-packages/rasa/shared/utils/io.py:98: UserWarning: Misaligned entity annotation in

Command / Request
No response
Relevant log output
/home/ubuntu/rasa_3/lib/python3.8/site-packages/rasa/shared/utils/io.py:98: UserWarning: You have an overlap between the 'RegexEntityExtractor' and the statistical entity extractors DIETClassifier in your configuration. Specifically both types of extractors will attempt to extract entities of the types inv_date, po_num, inv_num. This can lead to multiple extraction of entities. Please read 'RegexEntityExtractor''s documentation section to make sure you understand the implications.
  More info at https://rasa.com/docs/rasa/components#regexentityextractor
2022-11-30 08:08:01 INFO     rasa.engine.training.hooks  - Restored component 'RegexFeaturizer' from cache.
2022-11-30 08:08:47 INFO     rasa.engine.training.hooks  - Restored component 'LexicalSyntacticFeaturizer' from cache.
2022-11-30 08:09:57 INFO     rasa.engine.training.hooks  - Restored component 'CountVectorsFeaturizer' from cache.
2022-11-30 08:11:26 INFO     rasa.engine.training.hooks  - Starting to train component 'DIETClassifier'.

/home/ubuntu/rasa_3/lib/python3.8/site-packages/rasa/shared/utils/io.py:98: UserWarning: Misaligned entity annotation in

@sandeepbhutaniexl
Hi, thank you for reporting this issue.
In order to be able to investigate this issue we will like to get the following information:

What version of poetry did you use to install dependencies?
Are you using virtual environment? If yes, which tool have you used to create virtual environment?
Are you running this in cloud or on your local machine?
If you are running on cloud, which one are you using and what type of the machine?


Automatic gzip decompression seems not to work, unless pigz is in the path

hck compiled with
just install-native

This results in garbled characters:
# gzip in PATH: This results in garbled characters.
hck -z -f 1 test.gz

# pigz in PATH: Works.
hck -z -f 1 test.gz

BTW for the fastest gzip decompression igzip of https://github.com/intel/isa-l can be used.
See: https://github.com/zlib-ng/zlib-ng/issues/986 for an example decompression (gzip, pigz with zlib-ng as zlib library and igzip).

Thanks for making an issue! I'll definitely be checking out isa-l!
I've opened a PR here: https://github.com/BurntSushi/ripgrep/pull/1946 with a fix. If that is rejected I'll just vendor it into hck and make it do the expected thing. I'll also add isa-l as a binary to try to find first in line, then pigz, then gzip 👍
I'm not sure how I messed up trying this out before, I definitely thought I tried it with and without pigz in the path.

I don't think pigz is (much) faster than gzip when decompressing (or at least not much) gzipped files as the decompression is always single-threaded.(when zlib-ng is used as zlib library, it will beat gzip (didn't try to compile gzip with zlib-ng yet)).
This entry in the table in the README looks wrong to me:
*.tbz2 | bzip -d -c | gzip
-- | -- | --

Good catch on the readme, fix incoming. I was manually transcribing from here: https://github.com/BurntSushi/ripgrep/blob/9b01a8f9ae53ebcd05c27ec21843758c2c1e823f/crates/cli/src/decompress.rs#L497
Running on ubuntu 20.04, using whatever the apt-get installed versions of gzip and pigz are, pigz seems faster:
❯ pigz -V
pigz 2.4
❯ /usr/bin/time pigz -d -c hyper_data.txt.gz > /dev/null
17.87user 1.16system 0:13.79elapsed 137%CPU (0avgtext+0avgdata 2452maxresident)k
0inputs+0outputs (0major+135minor)pagefaults 0swaps

❯ gzip -V
gzip 1.10
Copyright (C) 2018 Free Software Foundation, Inc.
Copyright (C) 1993 Jean-loup Gailly.
This is free software.  You may redistribute copies of it under the terms of
the GNU General Public License <https://www.gnu.org/licenses/gpl.html>.
There is NO WARRANTY, to the extent permitted by law.

Written by Jean-loup Gailly.
❯ /usr/bin/time gzip -d -c hyper_data.txt.gz > /dev/null
22.28user 0.20system 0:22.49elapsed 99%CPU (0avgtext+0avgdata 1768maxresident)k
0inputs+0outputs (0major+150minor)pagefaults 0swaps

hyper_data.txt.gz is just the gzipped benchmark data linked in the readme.

I don't disagree that bespoke compilations of either gzip / pigz will be better than the system installed ones. All I'm aiming for here is "sensible defaults" for decompression. Performance of the decompression itself is secondary since we are already running the decompression in a separate process. This is purely a convenience over pigz -c -d ./data.tsv.gz | hck, and in no way more performant than doing exactly that :(

However, for gzip in particular, something I have not yet delved into but want to explore would be doing that decompression in process using a rust lib of some sort to be more performant than reading from stdin. I just haven't had time to explore that option yet.

You have https://github.com/rust-lang/flate2-rs/ which allows you to e.g. choose zlib-ng as zlib library.

Not sure if it is related to this issue, but I'm seeing this:
$ cat chk.csv
Name,Maths,Physics,Chemistry
Ith,100,100,100
Cy,97,98,95
Lin,78,83,80

# gz input gives garbled result (as mentioned in the issue)
$ gzip chk.csv
$ hck -d, -f2 -z chk.csv.gz
�(�	Ȩ

# renaming it to tgz works!
$ mv chk.csv.gz chk.csv.tgz
$ hck -d, -f2 -z chk.csv.tgz
Maths
100
97
78

@learnbyexample, that is indeed related! I'm going to give my PR to grep_cli the weekend to see if it gets a response and then go from there. If there isn't a grep_cli fix by Monday I'll make a release that drops pigz for now and go from there.
If you install pigz auto-decompression should work (not ideal, i know).

WIP PR for moving to flate2 for some compression types (gzip, mainly). https://github.com/sstadick/hck/pull/34

See release v0.6.1


RViz may crash when duplicating the Image Display

When I try to duplicate several Image Displays in the Displays Panel, I suddenly press the "Remove" button in the Displays Panel for some reasons. Then I found that RViz crashed after my operations.
I try to find the reason with GDB and finally I got the call stack of RViz. The call stack is shown as below.
 Thread 1 "rviz" received signal SIGSEGV, Segmentation fault.
0x0000000800000008 in ?? ()
(gdb) bt
#0  0x0000000800000008 in  ()
#1  0x00007f14a197e142 in rviz::DisplaysPanel::onDuplicateDisplay() ()
    at /opt/ros/melodic/lib/librviz.so
#2  0x00007f14a0bb4555 in QMetaObject::activate(QObject*, int, int, void**) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#3  0x00007f14a128bba2 in QAbstractButton::clicked(bool) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#4  0x00007f14a128bdba in  () at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#5  0x00007f14a128d19a in  () at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#6  0x00007f14a128d38d in QAbstractButton::mouseReleaseEvent(QMouseEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#7  0x00007f14a11d9048 in QWidget::event(QEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#8  0x00007f14a119a83c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#9  0x00007f14a11a265f in QApplication::notify(QObject*, QEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#10 0x00007f14a0b858d8 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () at /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#11 0x00007f14a11a1632 in QApplicationPrivate::sendMouseEvent(QWidget*, QMouseEvent*, QWidget*, QWidget*, QWidget**, QPointer<QWidget>&, bool) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#12 0x00007f14a11f416b in  () at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
---Type <return> to continue, or q <return> to quit---
#13 0x00007f14a11f67da in  () at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#14 0x00007f14a119a83c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#15 0x00007f14a11a2104 in QApplication::notify(QObject*, QEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#16 0x00007f14a0b858d8 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () at /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#17 0x00007f149b516583 in QGuiApplicationPrivate::processMouseEvent(QWindowSystemInterfacePrivate::MouseEvent*) () at /usr/lib/x86_64-linux-gnu/libQt5Gui.so.5
#18 0x00007f149b518055 in QGuiApplicationPrivate::processWindowSystemEvent(QWindowSystemInterfacePrivate::WindowSystemEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Gui.so.5
#19 0x00007f149b4ef2eb in QWindowSystemInterface::sendWindowSystemEvents(QFlags<QEventLoop::ProcessEventsFlag>) () at /usr/lib/x86_64-linux-gnu/libQt5Gui.so.5
#20 0x00007f1489f51260 in  () at /usr/lib/x86_64-linux-gnu/libQt5XcbQpa.so.5
#21 0x00007f1499f21537 in g_main_context_dispatch ()
    at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#22 0x00007f1499f21770 in  () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#23 0x00007f1499f217fc in g_main_context_iteration ()
    at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#24 0x00007f14a0bde88f in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () at /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#25 0x00007f14a0b8390a in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>---Type <return> to continue, or q <return> to quit---
) () at /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#26 0x00007f14a0b8c9b4 in QCoreApplication::exec() ()
    at /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#27 0x00005628fbd90dcf in main ()

I was confused about this because I press "Duplicate" button firstly, but it seemed that RViz removed the Image Displays which was being copied and then continued to duplicate the Image Displays which had been deleted. Then I do another experiment and got the same result that RViz crashed. In this experiment, I selected more Image Displays to duplicate (more than 50). I pressed the "Duplicate" button on the Displays Panel twice quickly and then I pressed the "Remove" button. RViz crashed and I got the call stack as below.
Thread 1 "rviz" received signal SIGSEGV, Segmentation fault.
0x000000000000001e in ?? ()
(gdb) bt
#0  0x000000000000001e in  ()
#1  0x00007f1305b2c142 in rviz::DisplaysPanel::onDuplicateDisplay() ()
    at /opt/ros/melodic/lib/librviz.so
#2  0x00007f1304d62555 in QMetaObject::activate(QObject*, int, int, void**) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#3  0x00007f1305439ba2 in QAbstractButton::clicked(bool) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#4  0x00007f1305439dba in  () at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#5  0x00007f130543b19a in  () at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#6  0x00007f130543b38d in QAbstractButton::mouseReleaseEvent(QMouseEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#7  0x00007f1305387048 in QWidget::event(QEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#8  0x00007f130534883c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#9  0x00007f130535065f in QApplication::notify(QObject*, QEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#10 0x00007f1304d338d8 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () at /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#11 0x00007f130534f632 in QApplicationPrivate::sendMouseEvent(QWidget*, QMouseEvent*, QWidget*, QWidget*, QWidget**, QPointer<QWidget>&, bool) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#12 0x00007f13053a216b in  () at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
---Type <return> to continue, or q <return> to quit---
#13 0x00007f13053a47da in  () at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#14 0x00007f130534883c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#15 0x00007f1305350104 in QApplication::notify(QObject*, QEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#16 0x00007f1304d338d8 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () at /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#17 0x00007f12ff6c4583 in QGuiApplicationPrivate::processMouseEvent(QWindowSystemInterfacePrivate::MouseEvent*) () at /usr/lib/x86_64-linux-gnu/libQt5Gui.so.5
#18 0x00007f12ff6c6055 in QGuiApplicationPrivate::processWindowSystemEvent(QWindowSystemInterfacePrivate::WindowSystemEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Gui.so.5
#19 0x00007f12ff69d2eb in QWindowSystemInterface::sendWindowSystemEvents(QFlags<QEventLoop::ProcessEventsFlag>) () at /usr/lib/x86_64-linux-gnu/libQt5Gui.so.5
#20 0x00007f12ee0ff260 in  () at /usr/lib/x86_64-linux-gnu/libQt5XcbQpa.so.5
#21 0x00007f12fe0cf537 in g_main_context_dispatch ()
    at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#22 0x00007f12fe0cf770 in  () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#23 0x00007f12fe0cf7fc in g_main_context_iteration ()
    at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#24 0x00007f1304d8c88f in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () at /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#25 0x00007f12ff6bbff3 in QGuiApplication::sync() ()
---Type <return> to continue, or q <return> to quit---
    at /usr/lib/x86_64-linux-gnu/libQt5Gui.so.5
#26 0x00007f1305b82c31 in rviz::RenderWidget::RenderWidget(rviz::RenderSystem*, QWidget*) () at /opt/ros/melodic/lib/librviz.so
#27 0x00007f1305b7d2af in rviz::QtOgreRenderWindow::QtOgreRenderWindow(QWidget*) () at /opt/ros/melodic/lib/librviz.so
#28 0x00007f1305ba4b79 in rviz::RenderPanel::RenderPanel(QWidget*) ()
    at /opt/ros/melodic/lib/librviz.so
#29 0x00007f1244bb6931 in rviz::ImageDisplay::onInitialize() ()
    at /opt/ros/melodic/lib/librviz_default_plugin.so
#30 0x00007f1305b1262d in rviz::Display::initialize(rviz::DisplayContext*) ()
    at /opt/ros/melodic/lib/librviz.so
#31 0x00007f1305bf1e54 in rviz::VisualizationManager::addDisplay(rviz::Display*, bool) () at /opt/ros/melodic/lib/librviz.so
#32 0x00007f1305bf25b6 in rviz::VisualizationManager::createDisplay(QString const&, QString const&, bool) () at /opt/ros/melodic/lib/librviz.so
#33 0x00007f1305b2c0e8 in rviz::DisplaysPanel::onDuplicateDisplay() ()
    at /opt/ros/melodic/lib/librviz.so
#34 0x00007f1304d62555 in QMetaObject::activate(QObject*, int, int, void**) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#35 0x00007f1305439ba2 in QAbstractButton::clicked(bool) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#36 0x00007f1305439dba in  () at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#37 0x00007f130543b19a in  () at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
---Type <return> to continue, or q <return> to quit---
#38 0x00007f130543b38d in QAbstractButton::mouseReleaseEvent(QMouseEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#39 0x00007f1305387048 in QWidget::event(QEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#40 0x00007f130534883c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#41 0x00007f130535065f in QApplication::notify(QObject*, QEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#42 0x00007f1304d338d8 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () at /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#43 0x00007f130534f632 in QApplicationPrivate::sendMouseEvent(QWidget*, QMouseEvent*, QWidget*, QWidget*, QWidget**, QPointer<QWidget>&, bool) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#44 0x00007f13053a216b in  () at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#45 0x00007f13053a47da in  () at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#46 0x00007f130534883c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#47 0x00007f1305350104 in QApplication::notify(QObject*, QEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#48 0x00007f1304d338d8 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () at /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#49 0x00007f12ff6c4583 in QGuiApplicationPrivate::processMouseEvent(QWindowSystemInterfacePrivate::MouseEvent*) () at /usr/lib/x86_64-linux-gnu/libQt5Gui.so.5
---Type <return> to continue, or q <return> to quit---
#50 0x00007f12ff6c6055 in QGuiApplicationPrivate::processWindowSystemEvent(QWindowSystemInterfacePrivate::WindowSystemEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt5Gui.so.5
#51 0x00007f12ff69d2eb in QWindowSystemInterface::sendWindowSystemEvents(QFlags<QEventLoop::ProcessEventsFlag>) () at /usr/lib/x86_64-linux-gnu/libQt5Gui.so.5
#52 0x00007f12ee0ff260 in  () at /usr/lib/x86_64-linux-gnu/libQt5XcbQpa.so.5
#53 0x00007f12fe0cf537 in g_main_context_dispatch ()
    at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#54 0x00007f12fe0cf770 in  () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#55 0x00007f12fe0cf7fc in g_main_context_iteration ()
    at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#56 0x00007f1304d8c88f in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () at /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#57 0x00007f1304d3190a in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () at /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#58 0x00007f1304d3a9b4 in QCoreApplication::exec() ()
    at /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#59 0x000056263ccfbdcf in main ()

According to the call stack, I found that when RViz created an Image Display, it would construct a RenderWidget, which would call QApplication::sync() (the 25th line in the call stack). After QApplication::sync() was called, RViz would process the Qt events, as well as removing the display plugins. After RViz finished to remove the Image Displays, it would go back to rviz::DisplaysPanel::onDuplicateDisplay() and continued to duplicate the Image Displays. However, the Image Displays had beed removed and RViz would visited the null pointer, which would result in crash.
I confuse about why QApplication::sync() is called in the constructor of rviz::RenderWidget. Can QApplication::sync() be removed from the constructor?
If the QApplication::sync() is not able to be removed, I think we should take some other measures to solve this problem. To avoid this problem, one of the solutions is adding a QProcessDiag to the rviz::DisplaysPanel::onDuplicateDisplay() as below. While duplicating the displays, a QProcessDiag will be shown on the RViz, which is modal. Therefore, user will be not able to press the "Remove" button and the crash will be avoided. The codes about the diag are as below.

And the effect picture is also as below.

before add this dialog

This is just a suggestion. If there are more reasonable solution, we can make a further discussion here.
Your environment

OS Version: Ubuntu 18.04
ROS Distro: Melodic
RViz, Qt, OGRE, OpenGl version as printed by rviz:

[ INFO] [1617852872.803608900]: rviz version 1.13.15
[ INFO] [1617852872.803677500]: compiled against Qt version 5.9.5
[ INFO] [1617852872.803696100]: compiled against OGRE version 1.9.0 (Ghadamon)
[ INFO] [1617852872.808645600]: Forcing OpenGl version 0.
[ INFO] [1617852873.113495600]: Stereo is NOT SUPPORTED
[ INFO] [1617852873.113581400]: OpenGl version: 3.1 (GLSL 1.4).

Thanks for this detailed analysis. QApplication::sync() was introduced by @rlober in #1263. I'm not sure why it was needed. @rlober, do you remember?
https://github.com/ros-visualization/rviz/blob/3cdc03a5186a53f644c5abd55b9971457bdffa1c/src/rviz/ogre_helpers/render_widget.cpp#L78-L82
In any case, I like your approach to display a progress dialog. Do you mind filing a corresponding PR?

What a coincidence, I encounter an issue caused by the QApplication::sync() too, the sync here give a huge hidden danger in initialize. sync() force Qt process events, and that will makes qt to call some method, such as getViewData(), even the onIntialize() is not finished. If I access some member initialize after render_panel_ in getViewData(), the program is core dumped. Maybe call sync() in constructor is not a wise approach.

console output:
[ INFO] [1618104707.573993981]: rviz version 1.13.17
[ INFO] [1618104707.574048878]: compiled against Qt version 5.9.5
[ INFO] [1618104707.574082675]: compiled against OGRE version 1.9.0 (Ghadamon)
[ INFO] [1618104707.577835020]: Forcing OpenGl version 0.
[ INFO] [1618104707.674521931]: Stereo is NOT SUPPORTED
[ INFO] [1618104707.674754515]: OpenGL device: llvmpipe (LLVM 10.0.0, 256 bits)
[ INFO] [1618104707.674825411]: OpenGl version: 3.1 (GLSL 1.4).
ImageDisplay::getViewData is accessed when ImageDisplay is not initialized.

hi @rhaschke, It seems @rlober haven't been active for a long time.
The official documents of Qt discourage use QApplication::sync(), and all of us can't see what kind of problem could the sync() solve. Only those problem it cause could be seen. So I propose to delete they.
https://github.com/ros-visualization/rviz/blob/3917c01555058c6c117bd32213fb46919a9d1bb6/src/rviz/ogre_helpers/render_widget.cpp#L77-L82

@tsukione, please test and review #1624.

Thank you accept my proposal. I have test the PR in my local environment, It works good.

Fixed via #1624.


x/gamm: Consider renaming CalcExitPoolShares to CalcExitCoinsFromShares

Background
CalcExitPoolShares calculates the expected tokens output by a pool given some number of shares. With the current name, it's not clear whether or not the function calculates the number of shares from output tokens or output tokens from the number of shares, so I think it might be worth considering renaming it to something like CalcExitCoinsFromShares (or something else that better specifies what the function does).
https://github.com/osmosis-labs/osmosis/blob/85529c76683f2d2f2ffb2121125baf0e39463f84/x/gamm/pool-models/balancer/amm.go#L501
Suggested Design

CalcExitPoolShares -> CalcExitCoinsFromShares

Acceptance Criteria

All existing and new tests pass

I like the idea. Suggest even being more verbose for clarity: CalcExitPoolCoinsFromShares

cc: @ValarDragon @alexanderbez thoughts?


create list of weapons and shields

structure:
Weapon
  name: String
  price: number
  damage: number

Shield
  name: String
  price: number
  defence: number

Firebase DB was released with some new equipment.
Please validate.

@hkirschke create equipments in .json format in /data folder


autokx on /q and /gc invite

as discussed. this closes #67.

Hi @dajohi, I will need to rework most of this PR due to a change of requirements.  I will make sure to gofmt it then.

@marcopeereboom Yes, but directory mode is disabled by default. You need to set "directory = yes" in zkserver.conf.


e2e-simpleTestsMinProfile-master frequent failures in postsubmit

See history https://prow.istio.io/?type=postsubmit&job=e2e-simpleTestsMinProfile-master, looks like ~80% fail rate

this was fixed, passing 100% now


Add Export to Excel feature to listView

Category
[X ] Enhancement
[ ] Bug
[ ] Question
Version
Please specify what version of the library you are using: [     current    ]
Expected / Desired Behavior / Question
Would be nice to have an export to Excel feature on the listview!

Closing as the linked issue has more detail


Move Module Creation command from Kyma CLI to the modulectl

Description
The functionality of creating a new module should be imported from the CLI
Acceptance Criteria

Implement Module Create functionality
Don't copy the Kyma CLI code

Stick to the Cobra framework
Adhere to principles presented here: https://github.com/kyma-project/cli/issues/1908

Covered with unit-tests
Ensure validations for module create command also cover module scaffolding command

when refactoring moduleConfig and making it a public type, update tests/e2e/create/scaffold/scaffold_test.go to use the public type instead the copied one.

Blocked by: https://github.com/kyma-project/modulectl/issues/9

Issue split:

#26
#27
#28


📖 Investigate Bastion ec2

User Story
As a member of the CloudOps team
I want to make sure unnecessary ec2 instances aren't live in AWS.
So that we don't waste money and increase our security posture
Value / Purpose
Ensure bastion ec2 isn't spun up when it shouldn't be
Useful Contacts
Additional Information

Definition of Done (DoD)
Any test bastion ec2 instances are feature flagged and not created unnecessarily

Hey team! Please add your planning poker estimate with ZenHub @ASTRobinson @bagg3rs @kyphutruong @mitchdawson1982 @moontune @mTouhid @wanieldilson


Allow loading TCX from a string, for use in the browser

Hi,
Thanks for the great work on this library! I wanted to use it for a simple browser-based visualisation with a file input, but the fs library is not available cient-side. I changed the Parser constructor to accept an optional argument: the contents of the TCX file as a string. With this small modification I can now load a TCX file with FileReader or fetch before processing the contents with your Parser.
I've included a simple example in parser_test.js - the test still reads the file with fs, but the point is that any file reading method could be used.

Was just looking for this! Please merge this PR. Thanks for the great library and this modification!

Agreed this PR would be a great addition, I too had to modify the package to include this feature when trying to work with TCX as strings returned from APIs

@douglas-watson maybe publish to separate npm package since seems @cjoakim is not around?

Folks - sorry that I haven't seen this PR and comments until today; not sure why.  My schedule is very busy now, but I'll try to take a look at this soon.  I understand that the main ask is to load from a string rather than a file.  Thanks, Chris


Develop

Merge changes from master to develop

Current coverage is 36.04% (diff: 62.02%)

Merging #87 into develop will decrease coverage by 0.62%

@@            develop        #87   diff @@
==========================================
  Files           220        225     +5
  Lines         12335      12571   +236
  Methods           0          0
  Messages          0          0
  Branches       2101       2134    +33
==========================================
+ Hits           4523       4531     +8
- Misses         7201       7468   +267
+ Partials        611        572    -39

Powered by Codecov. Last update 22561bc...f8a1034


Port coin selection and fee calculation from legacy

Context

Coin selection happens in order to choose UTxO from the available ones in a wallet to cover (when possible) for the cost of a transaction. More details in About Coin Selection
In the context of this new wallet backend, we will need a subset of the coin selection previously developed in the legacy code base: see Kernel/CoinSelection.
The first module:

Kernel/CoinSelection/Generic provides a generic DSL to perform CoinSelection on a ledger relying on UTxO accounting.

Kernel/CoinSelection/FromGeneric provides an implementation of the aforementioned DSL for Cardano.

Kernel/CoinSelection/Generic/Fee contains logic to perform fee adjustment on a selection yield by the above modules.

Kernel/CoinSelection/Generic/Grouped is now irrelevant as we won't be considering grouping

Kernel/CoinSelection/Generic/LargestFirst provides a simple heuristic from selecting coins from the largest first. This is the fallback heuristic in case where the random selection fails to yield a satisfactory solution. If the largest first doesn't yield any result, we know that there are not enough funds to cover the transaction cost and, a failure should be reported to users.

Kernel/CoinSelection/Generic/Random contains logic for performing a random selection while improving the selection to minimize the discrepancy between change outputs and normal outputs.

Our final implementation only truly use a subset of the available features from the coin selection "engine", so, it is a legitimate question to ask whether we should port everything in there and preserve the existing format. The generic abstraction makes it possible to have a code that is fairly agnostic to any specificity of a particular coin or wallet. This code has also gone through several round of testing and QA, and despite having some known limitations, is expected to work fairly okay now. On the other hand, it adds several layers of indirection that are probably not needed since we do not intend to support different coins and policy now (and probably not in a near future). Not mentioning that, most of the property and unit testing happens on a concrete implementation using legacy types. So, there's not much from the testing we can re-use.
Decision

We'll port relevant part of the coin selection algorithm, dropping the generic approach and fallback to a specialized implementation using our primitive types. Doing so, we'll also NOT port:

Generic DSL
PrivacyModeOff approach from the random selection
Grouping policy
Receiver pay fee

Instead, we'll focus on providing a concrete implementation that cover our current and upcoming needs, testing as we go and preserving the algorithms as they currently exists in the generic coin selection.
The legacy coin selection implementation is also known to be quite inaccurate in the way it reports error. So, we do want to be extra careful when dealing with errors and make sure we make the implementation very honest and helpful about errors.
In order to simplify the implementation work for this particular bit, we will also push back on the exact fee calculation for transaction and leave the implementation flexible regarding fees, using hard-coded constant for now (and for this, we may consider an interface closely resembling to FeeOptions).
Acceptance Criteria

[ ] We must be able to perform coin selection from available UTxOs, and yield a result in the form of:

a list of inputs
a list of outputs
a list of change coins (later mapped to change addresses)

[ ] Coin selection must preserve privacy and generate, as much as possible, outputs and change addresses that are roughly of the same size (this can be achieved by porting the algorithms used in the legacy coin selection).

[ ] Coin selection should make the sender pay fees, removing fee from its original change

[ ] We should preserve the separation of concerns between coin selection and fee adjustments.

[ ] We may port the various integration test scenarios about coin selections that exist on the legacy code base in the form of unit tests, and control that selection are yielded correctly

[ ] Coin selection shouldn't include any form of grouping at this stage

[ ] Explicit fee calculation and estimation from a transaction may be mocked with dummy values at this stage

Development Plan

PR

Number
Base

#121
master

#140
master

#165
master

#174
master

QA
Please refer to all tests added, unit tests and also property tests (see test/unit/Cardano/Wallet/CoinSelection/Policy)

@paweljakubas, @KtorZ - the ticket is in QA however there is still open pr #176 related to it ( at least it has #92 in Issue number)...

Yes, @piotr.stachyra this PR should be attached to another ticket we opened today because the fee calculation was a bit out of scope for #92

@KtorZ (addressing you since @paweljakubas is off) - seems there are few areas here that are not covered by tests:

https://coveralls.io/builds/23064602/source?filename=src/Cardano/Wallet/CoinSelection/Policy/Random.hs#L135
https://coveralls.io/builds/23064602/source?filename=src/Cardano/Wallet/CoinSelection/Policy/Random.hs#L183
https://coveralls.io/builds/23064602/source?filename=src/Cardano/Wallet/CoinSelection/Policy/Random.hs#L209

So. @piotr-iohk

L135 & L209: This is interesting. It looks like we are not evaluating the resulting change in our tests :thinking: ... I'll have a look into this.

L183: Looking a bit more into condC, it seems that this condition is redundant with condB. Indeed, the only way to exceed the maximum amount would be by selecting an input that doesn't get us closer to our ideal (but get us really far from it!). So in the end, either condB is true and condC is necessarily true. Or condB is false and condC is necessarily false (condB => condC) and, because of lazyness here, condC is never evaluated. We can change the order of evaluation, or, totally remove condC (I can't see any case where condB doesn't imply condC :thinking: )

@piotr-iohk The first previous point has been fixed with #196.
Second point won't be fixed here, the condition is never evaluated in our testing and I do believe we can't encounter it, but I don't want to remove it without proper thinking / analysis and keep it as a guard.

:+1:


Not getting request body data from postman

Not getting request body data from postman its comming null in name

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 583ae3e7-5915-f22b-e54c-6c100cf2a7da
Version Independent ID: 95f2549e-29f9-e31a-b4d0-432b971643ff
Content: Tutorial: Create a web API with ASP.NET Core
Content Source: aspnetcore/tutorials/first-web-api.md
Product: aspnet-core
Technology: aspnetcore-tutorials
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

Hello! Thanks for contacting us. This sounds like a general question about using ASP.NET Core. While we try to look at and respond to all issues, for questions like this we recommend posting to a community support group like Stack Overflow with the asp.net-core tag.


Add --detach option (issue #11)

Woah! Nice one! Will test and review mid next week. This looks exciting.

Can you update the README and give an example?

Ok, I'll update the README.
I'll be glad for a review.
I'am also preparing the test suite to smoothen next development by finding possible regressions during implementation


Add telemetry for usage of activateEnvInCurrentTerminal setting

[ ] Add telemetry for https://github.com/microsoft/vscode-python/pull/7665
[ ] Address Kartik's comment here

when reviewing/validating this, let's make sure it works on a multi workspace scenario

✅  Validated:

load workspace folder with no terminal open: telemetry sent with { isTerminalVisible: false }
load workspace folder with terminal open: terminal activated and telemetry sent with { isTerminalVisible: true }
load multiroot workspace with no terminal open: telemetry sent with { isTerminalVisible: false }
load multiroot workspace with a terminal open: terminal activated and telemetry sent with { isTerminalVisible: true }, however the virtual environment being activated is the one from the first folder in the .code-workspace file, see https://github.com/microsoft/vscode-python/issues/3325#issuecomment-504000731


Fix station records error

Fixes #10589

It does fix it.
For reference, this is what I think is going on
Synchronize while looping GetRecentlyAccessed raises RecordModifiedEvent which sets off BuildCrewManifest which calls GetRecordsOfType, which inside does _recentlyAccessed.Add, which causes the loop to blow up on the next iteration.


Can't obtain on serverside that page has been requested via instantclick

Pjax sends request headers, why Instanclick doesn't?

Would be great if InstantClick set the conventional X-Requested-With: XMLHttpRequest header along with any other custom headers such as X-INSTANTCLICK. A lot of server side code relies on that convention for detecting Ajax requests.

Any progress on this??


BlazorWebView needs to support Razor Class Libraries with Static Web Assets

Ideas:
Support for Razor Class Libraries (RCLs) means that you can build your UI once and then consume it from a Blazor Server app, Blazor WASM app, Mobile Blazor Bindings Hybrid App, etc. There might be some restrictions, but most things should just work.

Also need to demonstrate how to use DI so that when the RCL is hosted on different platforms (e.g. web, iOS, Android), each can provide its own platform-optimized implementation of that service. For example, geolocation APIs can be done differently on browser vs. using Xamarin.Essentials on devices.

How this gonna be manage? Since we have all blazor RCLs on a base project blazor wasm and server needs the wwwroot/index.html as their entry point. I tried working around with a default blazor project and adding reference from the base class I got it up and running but for some reason it doesn't see the pages folder RCLs, I mean index, counter and weather pagse shows 404 page.

Ok I got it working.
I just needed to add and change some reference on program.cs on blazor wasm project
builder.RootComponents.Add<WebUI.App>("app");
and
builder.Services.AddSingleton<CounterState>();
I deleted everything else except the wwwroot folder since wwwoot on the base library is working as embedded resource I don't know how we will load blazor site from a embedded resource.
Also I changed CounterState access modifier to public instead of internal to initialize it over blazor wasm project.
I also see we need to share or have in common some services with blazor wasm/server like for example CounterState.
For everything going on top of xamarin the default project is Ok.
But for blazor wasm and blazor server we may want to initialize others services that are web specific.
On xamarin we may want to use Xamarin.Essentials instead.

I propose making 3 folders for services on base project
Services/Common
Services/Web
Services/Native

This is in!!!!!! Fixed in #178
A huge huge huge HUGE thank you to @jspuij for all of his work, patience, and late night (for him) phone calls with me!

@Eilon when you gonna do a new template and nuget package version? I saw you are still having one more task for take out a new preview based on the current milestone. I am using nighly build at this moment but I saw @jspuij did some changes to the template structure, when we can have this template changes?

Hi @arivera12 the nightly feed has both the updated NuGet packages for the code as well as the NuGet packages that contain the templates!
Learn more here: https://docs.microsoft.com/mobile-blazor-bindings/contribute/nightly-builds

I've written up an initial walkthrough to guide people to build a Razor Class Library (RCL) and use it in a Mobile Blazor Bindings app and in a Blazor Web app.
You can check it out in the pre-publish format here: https://github.com/MicrosoftDocs/mobile-blazor-bindings/blob/master/mobile-blazor-bindings/walkthroughs/shared-web-ui.md
This will eventually go live on the Microsoft Docs site. But in the meantime please have a look if you'd like to learn more and let me know if you have any feedback!


Use default parametric constraint across all fields

Hello,
Sometimes I have a model with many parameters which are a subtype of Real. For example,
struct Model{T1<:Real,T2<:Real, ... Tn<:Real}
     p1::T1
     p2::T2
     ...
     pn::Tn
end

The @concrete macro is very convenient in this case because one does not need to type out the parametric constraints. However, this leaves open the possibility that someone might pass a complex number or a string. Would a default parametric constraint be possible and useful for others?

Does this fit your needs?
@concrete struct Model
     p1<:Real
     p2<:Real
     ...
     pn<:Real
end


🏴 Activity on this project will slow down

I have become lot more busier recently and – TBH – I have lost lot of motivation to keep up with this project (but not entirely, yet).
I will not abandon the project, but the request filling time from my part may take significantly longer then before and pack releases will become possibly less consistent (last two packs were both already late by few weeks).
Even the (modest) popularity of the project is pleasing and curse at the same time: I do enjoy that there's lot of users for this pack (currently over 43 000 downloads!), but the request rate sometimes takes a toll (and sometimes the moderation in general).
If you can fill some of the requests, please do (check the #560). I will check your creations and if they pass the CONTRIBUTING guidelines, I will add them to the pack.
Also, check out the alternative icon pack, aegis-simple-icons.
Thank you for your support :)

Just a warning for those who's reading this or subscribed to this: today I started a vocational college studies for the media and graphic design degree.
Therefore, I possibly have even less time and energy for aegis-icons project in the future.
I try my best to maintain this project, but it most likely only happens on weekends.


fix setting of rules engine environment variable

to: @chunyong-lin
cc: @airbnb/streamalert-maintainers
Background
The STREAMALERT_TRACK_RULE_STATS is currently getting set even if the rules engine config contains something like:
"log_rule_statistics": false
which is bad.
Changes

Updating the environment variable inclusion to look for a truthy setting.

Testing
N/A

Coverage remained the same at 96.173% when pulling 3e739f9a64c72bbeed6db02b51336fdd19c940c4 on ryandeivert-fix-env-var into 0712d093e8fb0e3bfe28f91d70615639c55163d4 on release-3-0-0.


etcd-tester: refactor cluster and failure code

Started doing this for #5754, but seems self-contained enough to split out. Planning to squash before merge.
/cc @gyuho @xiang90

lgtm. now much cleaner.
I tried this code with 3 node cluster and it works.
Defer to @xiang90
Thanks.

Yeah stresser is returning these errors
2016-06-29 17:46:56.672811 I | etcd-tester: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp: too many colons in address http://<IP_ADDRESS>:2379"; Reconnecting to {"http://<IP_ADDRESS>:2379" <nil>}
2016-06-29 17:46:57.853917 I | etcd-tester: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp: too many colons in address http://<IP_ADDRESS>:2379"; Reconnecting to {"http://<IP_ADDRESS>:2379" <nil>}
2016-06-29 17:46:58.586338 I | etcd-tester: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp: too many colons in address http://<IP_ADDRESS>:2379"; Reconnecting to {"http://<IP_ADDRESS>:2379" <nil>}

Other than that, works fine

Just tried with 3-node cluster, and everything seems working.
LGTM.
Thanks!

lgtm


Bug 30: spelling

Description
fix: spelling of filename for CONTRIBUTING.md fixed to amend breaking… link in README.md
CONTRIIBUTING.md => CONTRIBUTING.md
Fixes # (issue)
#30
Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)
[x] This change requires a documentation update

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[ ] Test A
[ ] Test B

Checklist:

[ ] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules

I don't have write access so I think you're gonna have to merge this


Feature request: documentation for retrying failed work

I can't seem to figure out the best way to handle a few scenarios.  Maybe some beefed up docs would help?
Some scenarios:
a)  I want to restart a failed/terminated workflow from the last activity that failed after perhaps deploying new code.
b)  I want to backoff retrying a specific activity that fails.  E.g.  I schedule an activity from the decider.  The activity fails with a specific error.  I want to retry again later.
I think that perhaps the ContinueWorkflowDecision and ManagedContinuations interceptor are related to this problem... but I can't figure out how to apply them.

Hi @ryanwalls
For failed/terminated workflows it depends on what caused the termination.  SWF terminated or user terminated?
SWF will terminate workflows that have gone longer than the StartToClose timeout, and it will also terminate workflows that have more than 25k events.
We typically have a long/always running workflow/activity that watches the ListClosedWorkflowExecutions endpoint and restarts SWF terminated workflows. This doesnt fix everything in all cases though.
can you give a little more detail here?
For backoff, one way you can achieve this is to simply use a timer.  you can set an arbitrary payload in the Control field of the timer, which could even be a *swf.ScheduleActivityTaskDecision.
When the timer fires, handle it and read out and submit the ScheduleActivityTaskDecision.

Thanks @sclasen.  For the first case, these are workflows that have failed for various unforeseen reasons.  Currently we have the workflow fail when an activity fails.  Perhaps the solution to this problem is to make the workflow itself more robust and just keep the workflow execution open instead of failing it anytime we see an error?
Even with it more robust, let's imagine we want to give the user a way to manually cancel/terminate a workflow execution, and then be able to restart it later.  Seems like with all the SerializedState we could restart a workflow from where it last was running, instead of restarting from the beginning?
Thanks for the idea of the timer.  Was already leaning that way, just didn't know if there was some other mechanism that you guys recommend.  Timer will probably work great for our case.

In our current approach, we never terminate or fail workflows from the FSM itself.   We have logic to retry everything inside the FSM when activity or other failures happen.
Rarely, we need to externally terminate a workflow, and restart it.  You can definitely do that by reading the full SerializedState out of the terminated workflow and setting it as the Input to the new execution.  The workflow will be restarted in the proper state.
You can even edit the state data inside the SerializedState if necessary before restarting.
However, your FSM might not start doing anything, as you'd need to do 'something' in an OnStarted handler to handle the restart. So it does take some up front work to be able to smoothly handle restarts.
The ManagedContinuationsInterceptor does basically this, when a workflow gets to a certain age or has a certain number of events, the interceptor attempts to continue the workflow on each decision task, it does this by checking to see if anything is in flight, and if not, it continues the workflow, using the SerializedState as the input to the continued workflow.
Doing this, either manuallly or via the interceptor results in SerializedState.StateVersion being incremented properly across continuations/restarts as well.

Interesting.  Yeah, we're using SWF as a processing pipeline.  We drop in work, it does a lot of calculation steps, and outputs results and finishes.  So our workflows are less like services than yours it sounds.
Will play around with the ManagedContinuationsInterceptor and querying the closed executions.  Will let you know if I develop anything I can contribute back.

@sclasen So almost done implementing this... one question.  In OnTimerFired decider, the last history event has an event type of EventTypeTimerFired which has the StartedEventId.  How do I get the actual timer scheduled event (and therefore access to it's control field)?
This must be an obvious thing that I'm just missing....
EDIT before I even posted....
Figured it out while I was writing this, but will leave here for the next person.
The correlator has the timer info.  So just use https://godoc.org/github.com/sclasen/swfsm/fsm#EventCorrelator.TimerInfo.


How to solve using webAssemblywith webview embedded h5 in flutter

In business requirements, a video player needs to be embedded. The video player uses webAssembly, but the webview in flutter does not seem to support webAssembly. How to solve this problem.

Hi @liqupan,
This issue doesn't seem to describe a bug or a feature request. Please see https://flutter.dev/community for resources and asking questions like this. You may also get some help if you post it on Stack Overflow with a minimal code sample that clearly describes your problem.
Closing, as this isn't an issue with Flutter itself. If you disagree, please write in the comments and I will reopen it.
Thank you


SEPG Automated Release - v2.15.0

SEPG Details
Release Build Link: https://sou-jenkins2.swinfra.net/job/Auto-Release/646/
Build Token: github.com/CAFapi/opensuse-tomcat-jre11-image/master/47638a997834ea32cce7d818d9f4d92e89667402
Versioning Behaviour: Roll on to next minor
Artifacts For Release
Docker

cafapi/opensuse-tomcat-jre11:2.15.0 (Static Tag)
cafapi/opensuse-tomcat-jre11:2.15 (Dynamic Tag)
cafapi/opensuse-tomcat-jre11:2 (Dynamic Tag)
cafapi/opensuse-tomcat-jre11:latest (Dynamic Tag)
cafapi/opensuse-tomcat-jre11-juli:2.15.0 (Static Tag)
cafapi/opensuse-tomcat-jre11-juli:2.15 (Dynamic Tag)
cafapi/opensuse-tomcat-jre11-juli:2 (Dynamic Tag)
cafapi/opensuse-tomcat-jre11-juli:latest (Dynamic Tag)

 Build Stage
Action
Status

Release Checks
completed
:heavy_check_mark:

Gather Release Data

:white_small_square:

QA Documentation Site Deployment

:white_small_square:

QA Deployment Branch Promotion

:white_small_square:

Docker Image Promotion

:white_small_square:

Maven Artifact Promotion (Closed Source)

:white_small_square:

NuGet Artifact Promotion

:white_small_square:

Artifactory Release Artifact Tagging

:white_small_square:

Generic Artifact Promotion

:white_small_square:

Maven Artifact Promotion (Open Source)

:white_small_square:

Maven Staging Repository Removal (Open Source)

:white_small_square:

Git Tagging

:white_small_square:

Git Release

:white_small_square:

Git Release Asset Deployment

:white_small_square:

Pull Request Merge

:white_small_square:

Release Branch Deletion

:white_small_square:

view the build log here


[Issue 269] Index prop necessary for dynamic input textfield.

Adding a static fix for the issue.  Could update with a dynamic solution if preferred.

Just read the issue #269. I think a dynamic solution can be better. At least everyone will be able to add props they want. Something like ...additionalInputProps is great :)

Sounds great.  I wasn't sure how the more advanced change would go over since I didn't get much feedback with the issue.  I'll take care of this later today.

Alright, let me know what you think.   I can update the readme if you feel its appropriate.

Yep, can you update the Readme also please? Thank you

Sorry for the delays but the Holidays had me a little distracted.   I didn't see much relevance on the readme page aside for the link to xinthink.  I believe my changes will reflect the updated information appropriately.

thank you for the PR


[WIP] Adds cooldown timer.  Closes #126.

Assumes 3 hours between tweets.

While I wouldn't characterize the coding in this PR as elegant, it is not the basest method I considered.
I believe the static time difference calculation in this PR to be the simplest method for forcing a delay between Tweets.
Pros:
Fewest new lines of code
Minimal storage usage for Artifact
Cons
No handling of the Tweet queue
sleep command  potentially  high resource usage
Possible unforeseen conflicts from multiple  Tweets in queue

Ugh! Workflow halts with the following message:
Invalid workflow file: .github/workflows/twitter-together.yml#L27
You have an error in your yaml syntax on line 27
Line 27 is literally copied directly from the README.ml of the Action I am calling.

Closing this PR because I have rewritten the cooldown timer in my test organization repo.
I will open a new PR from that repo/branch.


IOPAY-18 Add new api to return browser info

Description
This PR introduces an API to return some browser information when it is invoked by frontend app running on a web browser.
List of Changes

Added new http Azure function that implements the following API:

  "/browsers/current/info":
    get:
      operationId: GetBrowsersInfo
      description: Get info of the current browser used by the user
      responses:
        "200":
          description: Browser info retrieved
          schema:
            $ref: "#/definitions/BrowserInfoResponse"
        "400":
          description: Bad request
        "500":
          description: generic error

        BrowserInfoResponse:
      type: object
      required:
          - ip
          - useragent
          - accept
      properties:
        ip:
          type: string
        useragent:
          type: string
        accept:
          type: string

Motivation and Context
The goal of this new API is to allow a fronted javascript application (like io-pay) to retrieve some information relating to the browser, in particular:

ip:  the remote IP address of the request according to x-forwarded-for header;
useragent: content of User-Agent request header;
accept: content of Accept request header;

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[x] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.

@infantesimone Coverge is a bit low, so we should open another PR and try to increase it


Feature/assumptions

I'm not sure if this is within scope of this page, but it feels strange that it's missing cognitive assumptions which we should be checking when we check everything else.

Hmm, this was more aimed at services that check things for us, rather than a list of things we must check - they're good pages on things we as developers must check, but don't provide services for us to check them, therefore they're not within the scope of this project.
Thanks for the links though! :)


Can't install on Laravel 5.8

We don't use/maintain this code any longer.
If you need it, please fork it and maintain your own copy.


link_to_add_association compatibility with cells 4.0.2

Hi,
link_to_add_association is so good that I cannot do without it in my recent refactoring to the use of apotonick/cells
However it raised an error from within the Cocoon::ViewHelpers render_association method because it the render call was not compatible with rendering in cells.
I have written a monkey patch that works for me and I am wondering if you would like to support cells in your project. I will add my monkey patch and if you think it is worthwhile, I could make a pull request.
from within the cell, when you call link_to_add_association, pass in a render option cells_compatible:true like this:
link_to_add_association "add", @form, :association_name,
                          :partial => "_partial_name", render_options:{cells_compatible:true}

(note the leading underscore required in the partial_name)
in some model, or however you like to do monkey patches, (I use a class_extensions.rb file in my models directory) add this code:
require 'cocoon/view_helpers'
# overwrite the cocoon method which renders the partial for adding associations in the link_to_add_association helper method
module Cocoon
  module ViewHelpers
    def render_association(association, f, new_object, form_name, render_options={}, custom_partial=nil)
      partial = get_partial_path(custom_partial, association)
      locals =  render_options.delete(:locals) || {}
      cells_compatible = render_options.delete(:cells_compatible)
      method_name = f.respond_to?(:semantic_fields_for) ? :semantic_fields_for : (f.respond_to?(:simple_fields_for) ? :simple_fields_for : :fields_for)
      f.send(method_name, association, new_object, {:child_index => "new_#{association}"}.merge(render_options)) do |builder|
        partial_options = {form_name.to_sym => builder, :dynamic => true}.merge(locals)
        if cells_compatible
          render view: partial, locals: partial_options
        else
          render(partial, partial_options)
        end
      end
    end
  end
end

This simply overwrites the same method in Coccon which looks like this
def render_association(association, f, new_object, form_name, render_options={}, custom_partial=nil)
  partial = get_partial_path(custom_partial, association)
  locals =  render_options.delete(:locals) || {}
  method_name = f.respond_to?(:semantic_fields_for) ? :semantic_fields_for : (f.respond_to?(:simple_fields_for) ? :simple_fields_for : :fields_for)
  f.send(method_name, association, new_object, {:child_index => "new_#{association}"}.merge(render_options)) do |builder|
    partial_options = {form_name.to_sym => builder, :dynamic => true}.merge(locals)
    render(partial, partial_options)
  end
end

So if I understand correctly: you are using cells, you are using the cells folder-structure, so you need a different render so it will look for the correct partial. And this is not necessarily global, so on some pages you could use a "normal" link_to_add_association and in some pages you would be rendering a cell, containing some cocoon-form, correctly?
Just checking if it makes sense to be giving an extra parameter, or when using cells, we can always do render view: ... ?

Yes that is correct, I use link_to_add_association sometimes in cells and sometimes as it was designed to be used, though it seems to me a little more than just looking in the right place for the partial.
The number of parameters is different, i.e from cells it is one ( a Hash) normally it is 2 ( partial name, Hash). I think, though I do not understand properly yet, that cells overwrites the global render with this other implementation.

Closing this as it was suggested here https://github.com/apotonick/cells/issues/317#issue that it would be cleaner from me to overwrite the render method in Cells.

@ryan2johnson9 I tried to solve that problem without monkey-patching any methods, so I thought if we have to include Cocoon helper into Cells manually, why don't we include dedicated helper? On my fork I have added Cocoon::CellsHelper, that includes mostly standard Cocoon::ViewHelper,  only difference is render_association calls render_partial method now, which is different for both helpers.
This way seems a little cleaner than both of your proposed solutions, and works well for me both in Cells and standard Rails views. Please correct me if I'm wrong :smiley:
https://github.com/asaletnik/cocoon/commit/7c864f85c892fe21feeead09933f751e6773bdf6

@asaletnik Hey I tried your fix by monkeypatching the relevant methods in my cell. It works for the main part so +1 for you. The only problem remaining is the directory the partial is looked up in. Currently, if the code comes from a FooCell, it will only look up into the inherited cell folders app/cells/foo_and_inherited/ files ending.
But we might want to use a partial in app/cells/bar/ (eg. post_form.fields_for(:comments)) or even (?) app/views/.
If we want to allow looking up files in app/views/, we also need to to look at *html.erb extensions (currently only names in *.erb are resolved)

@Startouf I reached the same point as you where the template was being unnecessarily HTML escaped.
Although it's not the most elegant solution in the world, I was able to resolve that by adding an html_safe flag to the output of the partial rendering like so:
if cells_compatible
  render(partial: partial, locals: partial_options).html_safe
else
  render(partial, partial_options)
end

As with any use of the html_safe flag, you need to be careful that you're not opening yourself up to  XSS stuff. But with this, you're most likely rendering a partial from the server (and no user input).


Unable to get DatastoreCluster

[x] I am using the latest SDK version
[x] This API is compatible with my vCenter version (You can get the info from 'vcenter_version' in each sample)
[x] I have searched existing issues and Troubleshooting wiki

Environment

sdk package version:

python version:

Python 3.8

vSphere version:
v6.7

Operating System/Shell (used to run SDK-based apps):
macOS

Steps or code snippet to reproduce
(Pdb) datastore_summaries = client.vcenter.Datastore.list(filter_spec)
/Volumes/data/venv3/lib/python3.7/site-packages/urllib3/connectionpool.py:847: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
(Pdb) datastore_summaries
[Summary(datastore='datastore-17', name='ds_171_1', type=Type(string='VMFS'), free_space=12486443008, capacity=13421772800), Summary(datastore='datastore-18', name='ds_171_2', type=Type(string='VMFS'), free_space=158336024576, capacity=214479929344), Summary(datastore='datastore-19', name='ds_200', type=Type(string='VMFS'), free_space=194861072384, capacity=214479929344)]

Actual behavior
No API for getting information about datastore cluster
Expected behavior
some API like get, list for getting information about datastore cluster.

This would be very useful

Hey
Is this something that could help?
https://vdc-repo.vmware.com/vmwb-repository/dcr-public/1cd28284-3b72-4885-9e31-d1c6d9e26686/71ef7304-a6c9-43b3-a3cd-868b2c236c81/doc/structures/com/vmware/vcenter/inventory/datastore/svc.info-structure.html
I think you should look for datastore clusters (StoragePod) in inventory/datastore

@Akasurde
Does the above link help?

Can someone confirm there is no capability to get DatastoreCluster?  Or at the very least, determine if a datastore is part of a cluster?
Another nice feature would be the ability to pull the "Related Objects" for a VM via the REST API. 🙏

I believe you are looking for this for Storage DRS Cluster (SDRS POD). If yes, this capability is not with this SDK. You may need to use pyVmomi libs for this capability.