Change remote for 18F/oauth2_proxy to something that's actually maintained

In order to ensure our OAuth2 proxy is secure and supported, we should leverage a supported and maintained project.
In 18F/oauh2-proxy-boshrelease we're using 18F/oauth2_proxy, which is a fork of bitly/oauth2_proxy. Bitly has since archived this repo, so no further development is happening on it.
We should start using something officially supported - either by just switching remotes on 18F/oauth2_proxy, or by getting the work from 18F/oauth2_proxy integrated into a maintained repo and abandoning our fork.

The custodian of oauth2_proxy project has been shifted from bitly to pusher. Pusher and other open source contributors are actively maintaining this fork. It even now supports login.gov as a provider, which I'm guessing might be of interest here.

Recommend time-boxing 2 hours (max) of research to determine level of effort.

Current solution works but if it stops working it'll become very urgent. Getting the research done soon is necessary.

@hillaryj will update issue to reflect that it's a time-boxed research issue to start.

We're currently ahead of the bitly repo by one commit, and behind by several.
The commits we're behind don't matter - they're all README updates about archiving the repo.
The commit we're ahead is mostly concerned with adding several security-related headers.
Looking through the forks of the bitly repo, the most-maintained seems to be OpenShift, but it's maintenance is all about making it only work with openshift. AFAICT, the only other maintained fork is the Pusher fork mentioned above.
Looking at the commits we are ahead of the Bitly fork, I think our best option is one of:

fork pusher, patch to add in the functionality we'd added to bitly, and open a PR with pusher
use pusher unpatched, and add the missing functionality via nginx or some other reverse-proxy layer

Nginx won't work very well for logsearch-platform, so I think forking pusher and adding functionality there is better


Visual Editor flickering issue in nested flow when deleting / inserting

Describe the bug
After inserting / deleting an action under a nested action type (IfCondition, SwitchCondition, Foreach), Visual Editor will be rendered twice (flickering).

Version

Browser

[ ] Chrome
[ ] Safari
[ ] Firefox
[ ] Edge

OS

[ ] macOS
[ ] Windows
[ ] Ubuntu

To Reproduce
Steps to reproduce the behavior:

Create a IfCondition node
Create a nested IfCondition node under the true branch
Insert some nodes / delete some nodes under the nested true branch

Expected behavior

Screenshots

Additional context

This issue is introduced by a latest feature PR #2005 .
In that PR, visual editor storaged all nodes' sizes into a global cache to avoid frequently re-computation of element sizes. It did help us have a stable layout.
However, with that cache mechanism, when user deleting / inserting a leaf node, parent nodes' sizes will still be loaded from cache instead of recalculating the size, it makes the layouter consumes an outdated size, then deletion/insertion of any nested node will cause flickering issue.

The solution is making some improvements on layout cache management:

The solution is making some improvements on layout cache management:

Addressed in #2176.
Fix it just as I described

addressed via https://github.com/microsoft/BotFramework-Composer/pull/2176


merge nested prefab process and prefab variant process to fix implementation difference.

Fixes #104

@MerlinVR I'm so sorry for pinging you. This is significant issue because this issue almost breaks prefab completely as we cannot use for any use so could you review this (and 107) pull request?


HeadsUp Dispatch

Sends parsed CAD data in JSON format to headsup server for display.

Please reopen now that dev has been updated from main


go.mod local dependency

hi, in go.mod there is a local dependency, should it be replaced?
replace github.com/tellor-io/telliot => ../../tellor-io/telliot

and in flashbot.go line257, the type types.DynamicFeeTx is undefined

I update the go.mod github.com/ethereum/go-ethereum v1.10.3 to v1.10.8 and the definition problem is gone.


add 'force' parameter to docker plugin rm to remove an active plugin

ping @anusha-ragunathan @tiborvass

LGTM

@vieux @anusha-ragunathan is there a related PR on docker/docker for that ?

@vdemeester : I think @vieux is doing this in reverse order. i.e engine-api PR first. vendor it in. then make docker/docker PR. He can confirm that.

@anusha-ragunathan is right, sorry I wasn't aware I should do it the other way. I'm going to open the docker/docker PR and link it here

@vieux cool, thanks 😝
LGTM 🐺

LGTM


prepend rez path

fix for https://github.com/nerdvegas/rez/issues/196

Hey @elrond79,
Unfortunately prepending rez bin dir to PATH causes a different error.
It goes like this: It is common practice to use rez-gui via the "rezgui" rez package. It's done this way so that PyQt/PySide doesn't have to be installed into the rez installation's virtualenv (that's a pain, and most studios will already have pyqt/pyside Rez packages).
So, you can run rez-gui like so:
]$ rez-env rezgui pyside -- rez-gui
We happen to be using suites to manage our packages, and we have it set up so that 'rez-gui' is a suite tool, which launches into the same env you would get via the rez-env call above. The problem is, prepending rez bin path to PATH is causing the native rez 'rez-gui' to be visible, rather than the suite's 'rez-gui', causing rez-gui to crash (pyqt/pyside is not available).
In many setups this is going to be undesirable - people don't want to have to install pyqt/pyside a second time into rez's virtualenv, so they will want their own 'rez-gui' tool visible, rather than rez's.
Since the PATH prepend is done in Rex, there's not an easy way to perform a more complex action, such as "replace rez if already in path else append."
Not sure what to do about this just yet, but it's a showstopper right now and I've had to revert our installation.

Ok, so I've fixed this by leaving things as they are, but updating rez-gui so that if it fails due to missing PyQt/PySide, it searches PATH for a different rez-gui executable, and runs that instead.
It feels a little bit dirty, but it does work, so I'll leave it at that unless we hit any more problems..

Cools, sounds good...
On Fri, May 15, 2015 at 4:06 PM, allan johns<EMAIL_ADDRESS>wrote:

Ok, so I've fixed this by leaving things as they are, but updating rez-gui
so that if it fails due to missing PyQt/PySide, it searches PATH for a
different rez-gui executable, and runs that instead.
It feels a little bit dirty, but it does work, so I'll leave it at that
unless we hit any more problems..
—
Reply to this email directly or view it on GitHub
https://github.com/nerdvegas/rez/pull/197#issuecomment-102544617.

Hmm. This isn't going to work in some cases. We're hitting issues where rez-gui does eventually run, but after bouncing around 4 different rez-gui invocations, which causes an unacceptably large delay.
I'm going to change this so that whether rez is prepended or appended becomes configurable. the default will be to prepend as that matches existing behaviour, so @elrond79 if you need the prepend then be sure to set the option as such after I do the push. Should be pretty soon.

It's been a while since I've used rez-gui, but would this be simpler if rez-gui wasn't installed as part of rez itself?
We have a tool (called dora) that serves a very similar purpose to rez-gui.  However it is a separate package that depends on rez (which we also have a package for) and PyQt etc.  To use it, someone must 'rez env' it as they would any other package.  As rez-gui is a layer on top of rez - it's a separate product almost - and not part of the core functionality, this feels acceptable.

Politics. To get that to happen I'd need to get the OK for open source development on another project at my place of work. Generally I agree with you though.

You wouldn't have to take it out of the rez repository necessarily (although I think we should).  Initially you could just change the way it's built/installed within rez.  So instead of being picked up by install.py it's has it's own (somewhere, somehow).

Perhaps we could leave it appending, and then in situations where rez is
calling out to it's own binaries (ie, bez), it calls it explicitly, using a
full path? Of course, this would mean finding such use cases... but
hopefully a search for subprocess would catch most.
Or, thinking along mark's lines, there could be an option on install to NOT
install rez-gui.
Just throwing these out there as ideas.. I'd actually be ok with the
configuration for prepend instead of append... though it feels odd to be
using that just to make a unittest work.
On Mon, May 18, 2015 at 4:21 PM, Mark Streatfield<EMAIL_ADDRESS>wrote:

You wouldn't have to take it out of the rez repository necessarily
(although I think we should). Initially you could just change the way it's
built/installed within rez. So instead of being picked up by install.py
it's has it's own (somewhere, somehow).
—
Reply to this email directly or view it on GitHub
https://github.com/nerdvegas/rez/pull/197#issuecomment-103255694.

@elrond79, there are cases where an absolute path to rez binaries isn't possible. For example, tool wrappers creates by suites are shebanged with _rez_fwd, hard pathing this case would break tools with new rez releases, or leave them on an old release, and neither are an option.
I agree with Mark that rez-gui should be a separate project, for now the configurable option I've added gives us enough to get around any issue and I'm inclined to leave it that way for the moment.
A

Like I said, I'm fine with a configurable option.
However - think you may have misinterpreted my suggestion. I meant that,
instead of doing:
subprocess.call(['bez'])

we instead do:
subprocess.call([os.path.join(system.rez_bin_path, 'bez')])

So we could apply it to some calls and not others (ie, rez-gui, or
wrappers).  Sort of moot, though, since I also agree that making rez-gui
it's own project is a better long-term solution...
On Tue, May 19, 2015 at 11:22 AM, allan johns<EMAIL_ADDRESS>wrote:

@elrond79 https://github.com/elrond79, there are cases where an
absolute path to rez binaries isn't possible. For example, tool wrappers
creates by suites are shebanged with _rez_fwd, hard pathing this case would
break tools with new rez releases, or leave them on an old release, and
neither are an option.
I agree with Mark that rez-gui should be a separate project, for now the
configurable option I've added gives us enough to get around any issue and
I'm inclined to leave it that way for the moment.
A
—
Reply to this email directly or view it on GitHub
https://github.com/nerdvegas/rez/pull/197#issuecomment-103622675.


Upgrade prometheus/prometheus to v2.5.0

Signed-off-by: Sylvain Rabot<EMAIL_ADDRESS>

Looks like e2e tests might found the problem after upgrade.
Have you tried to run this e2e tests manually from local machine?
export THANOS_SKIP_GCS_TESTS='true'
export THANOS_SKIP_S3_AWS_TESTS='true'
export THANOS_SKIP_AZURE_TESTS='true'
export THANOS_SKIP_SWIFT_TESTS='true'
make test

I retried 3x times, so I would say it is actual issue. Rerunning test locally and checking why result is 0 is what I would do (((:

All right I'll do it.

Be my guest, I was pretty much stuck.

No worries, it is dangerous change (:

ANy luck with this so far @bwplotka ?

Aaaand we have 2.6.0 now, seriously it's hard to keep up with the releases :(

That's fine, we don't need to be up to date with Prometheus code as we use only few Prometheus packages.
The point of upgrading Prometheus is to get:

newest promQL improvements and potentially tracing
ability to use Prometheus notifier (first step done: https://github.com/prometheus/prometheus/pull/4955)
ability to use Prometheus rule manager important metric for query evaluation fails: (in progress: https://github.com/prometheus/prometheus/pull/4961)

This means that I need to finish 2 PRs against upstream Prom and get that merged. Once landed there will be clear advantage of getting newest Prometheus dep into Thanos (: Will notify you once we landed all PRs against Prometheus.


Try using Murmurhash 3 for bloom filters

Description
We are currently using Murmurhash 2(MurmurHash64.java) in the bloom filters implementation in lucene where we also have Murmurhash 3 (the latest one available in the MurmurHash family of hash functions) and provides better performance, avalanche effect etc. It provides 128-bit variant which Murmurhash 2 doesn't. This PR aims to use Murmurhash 3 with bloom filters and see if that helps.

Note : We are already using murmur hash 3 in BytesRefHash implementation (#6666)

Next steps :

[DONE] Run luceneutil benchmarks and share the results

@mikemccand rightly pointed out that luceneutil doesn't use bloom filter postings format by default and we should enable it for id field and rerun the benchmarks to see the impact. Will rerun and share the results here.

Maybe we should take advantage of this change to simplify this postings format, by no longer making the hash function configurable, removing the abstraction for hash functions, and cutting over to StringHelper to compute hashes instead of introducing a new implementation?

Maybe we should take advantage of this change to simplify this postings format, by no longer making the hash function configurable, removing the abstraction for hash functions, and cutting over to StringHelper to compute hashes instead of introducing a new implementation?

@jpountz Makes sense, +1 .to remove the abstraction Though I have a couple of points/questions here :

Do you mean to change StringHelper class to add support for 128 bit hash because currently it creates 32-bit hash with Murmur 3? or maybe moving BytesRefHash to also use 128-bit hash?
StringHelper doesn't seem like the most intuitive place for a hash function implementation like this. Do you think we should instead have or copy to something like HashHelper or HashUtil?

So I ran the luceneutil benchmarks with -idFieldPostingsFormat BloomFilter but it was failing as there was no delegate posting format and it wasn't able to find the right postings format class using SPI. I tweaked this code (pasted below) to use the BloomFilteringPostingsFormat for the id field and also use the codecs jar (similar to how its done for core) and then all worked.
public PostingsFormat getPostingsFormatForField(String field) {
      PostingsFormat pf = PostingsFormat.forName(defaultPostingsFormat);
      if (field.equals("id")) {
            return new BloomFilteringPostingsFormat(pf);
      }
      return pf;
 }

Below are the wikimediumall benchmark results(ran twice to get more confidence) which shows ~7-9% performance improvement for PKLookup with p-value of 0.000
Run # 1
                     TaskQPS baseline      StdDevQPS my_modified_version      StdDev                Pct diff p-value
 BrowseRandomLabelSSDVFacets        2.77      (7.0%)        2.72      (5.1%)   -1.7% ( -12% -   11%) 0.374
 BrowseRandomLabelTaxoFacets        3.35     (21.8%)        3.30     (18.1%)   -1.6% ( -34% -   48%) 0.803
             LowSloppyPhrase        6.11      (2.9%)        6.05      (2.5%)   -1.0% (  -6% -    4%) 0.248
           HighTermTitleSort      128.04      (3.7%)      126.78      (3.1%)   -1.0% (  -7% -    6%) 0.365
            HighSloppyPhrase       13.43      (2.7%)       13.31      (2.6%)   -0.9% (  -6% -    4%) 0.254
             MedSloppyPhrase        4.72      (3.4%)        4.68      (2.6%)   -0.8% (  -6% -    5%) 0.393
      OrHighMedDayTaxoFacets        2.99      (5.2%)        2.97      (3.5%)   -0.8% (  -9% -    8%) 0.582
        BrowseDateTaxoFacets        3.85     (19.0%)        3.82     (17.7%)   -0.8% ( -31% -   44%) 0.896
   BrowseDayOfYearTaxoFacets        3.86     (19.0%)        3.83     (17.9%)   -0.7% ( -31% -   44%) 0.900
                   OrHighMed       72.86      (2.0%)       72.58      (2.4%)   -0.4% (  -4% -    4%) 0.586
                  OrHighHigh       20.04      (3.1%)       19.98      (4.3%)   -0.3% (  -7% -    7%) 0.801
                  HighPhrase       24.66      (6.2%)       24.59      (6.6%)   -0.3% ( -12% -   13%) 0.882
                     Prefix3       72.76      (5.0%)       72.59      (4.3%)   -0.2% (  -9% -    9%) 0.873
                     MedTerm      379.26      (3.1%)      378.51      (5.1%)   -0.2% (  -8% -    8%) 0.882
                   MedPhrase       18.99      (5.5%)       18.95      (5.9%)   -0.2% ( -10% -   11%) 0.915
        BrowseDateSSDVFacets        0.90      (7.7%)        0.89      (7.4%)   -0.2% ( -14% -   16%) 0.944
                   LowPhrase       63.34      (2.4%)       63.27      (2.8%)   -0.1% (  -5% -    5%) 0.887
                HighSpanNear        6.18      (3.1%)        6.18      (3.2%)    0.0% (  -6% -    6%) 0.993
                      Fuzzy1       64.49      (1.1%)       64.52      (1.4%)    0.0% (  -2% -    2%) 0.919
                 LowSpanNear       15.60      (2.3%)       15.61      (2.5%)    0.1% (  -4% -    5%) 0.916
        HighTermTitleBDVSort        4.90      (4.2%)        4.90      (3.8%)    0.1% (  -7% -    8%) 0.938
        MedTermDayTaxoFacets        9.26      (5.6%)        9.27      (4.1%)    0.1% (  -9% -   10%) 0.948
     AndHighMedDayTaxoFacets       13.99      (1.7%)       14.02      (1.5%)    0.2% (  -3% -    3%) 0.764
    AndHighHighDayTaxoFacets        5.26      (2.7%)        5.27      (2.4%)    0.2% (  -4% -    5%) 0.819
                     Respell       29.96      (1.1%)       30.03      (1.8%)    0.2% (  -2% -    3%) 0.656
                     LowTerm      398.65      (2.5%)      399.67      (2.9%)    0.3% (  -5% -    5%) 0.765
                 MedSpanNear       38.84      (2.6%)       38.96      (3.0%)    0.3% (  -5% -    6%) 0.729
                    Wildcard       60.70      (1.6%)       60.89      (1.1%)    0.3% (  -2% -    3%) 0.463
       HighTermDayOfYearSort      200.04      (2.5%)      200.70      (3.7%)    0.3% (  -5% -    6%) 0.740
                   OrHighLow      252.74      (2.0%)      253.62      (2.4%)    0.3% (  -3% -    4%) 0.613
               OrNotHighHigh      157.04      (4.6%)      157.63      (4.2%)    0.4% (  -8% -    9%) 0.789
                 AndHighHigh       29.31      (2.4%)       29.42      (3.5%)    0.4% (  -5% -    6%) 0.678
                OrNotHighLow      290.56      (2.1%)      291.81      (1.7%)    0.4% (  -3% -    4%) 0.475
                OrNotHighMed      221.77      (3.5%)      222.84      (2.9%)    0.5% (  -5% -    7%) 0.633
               OrHighNotHigh      167.01      (4.8%)      167.85      (4.5%)    0.5% (  -8% -   10%) 0.731
                    HighTerm      279.21      (4.3%)      280.66      (6.5%)    0.5% (  -9% -   11%) 0.767
                  AndHighLow      374.84      (1.9%)      377.10      (1.8%)    0.6% (  -3% -    4%) 0.308
           HighTermMonthSort     2378.06      (3.6%)     2392.49      (4.1%)    0.6% (  -6% -    8%) 0.618
         LowIntervalsOrdered       12.78      (2.4%)       12.86      (2.8%)    0.6% (  -4% -    6%) 0.443
                OrHighNotLow      269.59      (4.8%)      271.34      (4.9%)    0.6% (  -8% -   10%) 0.672
                      IntNRQ       18.20      (5.9%)       18.32      (5.4%)    0.7% ( -10% -   12%) 0.709
                  AndHighMed       37.67      (2.5%)       37.93      (3.5%)    0.7% (  -5% -    6%) 0.459
         MedIntervalsOrdered        1.80      (3.5%)        1.82      (3.7%)    0.8% (  -6% -    8%) 0.456
                OrHighNotMed      248.23      (4.7%)      250.34      (4.5%)    0.9% (  -7% -   10%) 0.556
                      Fuzzy2       35.10      (1.2%)       35.42      (1.2%)    0.9% (  -1% -    3%) 0.016
       BrowseMonthTaxoFacets        4.13     (30.7%)        4.17     (34.8%)    1.0% ( -49% -   96%) 0.924
       BrowseMonthSSDVFacets        4.37      (9.6%)        4.45      (8.9%)    1.9% ( -15% -   22%) 0.506
        HighIntervalsOrdered        1.58      (4.6%)        1.61      (5.6%)    2.0% (  -7% -   12%) 0.228
                  TermDTSort       96.97      (3.2%)       98.92      (5.0%)    2.0% (  -6% -   10%) 0.132
   BrowseDayOfYearSSDVFacets        3.76      (9.6%)        3.85      (6.5%)    2.3% ( -12% -   20%) 0.366
                    PKLookup      106.69      (1.5%)      114.71      (1.5%)    7.5% (   4% -   10%) 0.000

Run # 2
                    TaskQPS baseline      StdDevQPS my_modified_version      StdDev                Pct diff p-value
      BrowseMonthTaxoFacets        4.08     (31.7%)        3.80      (1.8%)   -6.7% ( -30% -   39%) 0.342
BrowseRandomLabelTaxoFacets        3.30     (21.9%)        3.16      (2.3%)   -4.2% ( -23% -   25%) 0.393
       BrowseDateTaxoFacets        3.85     (20.0%)        3.72      (6.1%)   -3.5% ( -24% -   28%) 0.456
  BrowseDayOfYearTaxoFacets        3.86     (20.0%)        3.73      (6.2%)   -3.3% ( -24% -   28%) 0.476
                 OrHighHigh       26.35      (6.9%)       25.88      (6.8%)   -1.8% ( -14% -   12%) 0.404
      HighTermDayOfYearSort      209.99      (3.7%)      206.48      (4.1%)   -1.7% (  -9% -    6%) 0.177
                  OrHighLow      273.41      (2.3%)      270.80      (3.0%)   -1.0% (  -6% -    4%) 0.252
          HighTermMonthSort     2346.36      (3.2%)     2326.36      (3.6%)   -0.9% (  -7% -    6%) 0.427
       HighTermTitleBDVSort        4.83      (3.7%)        4.80      (3.4%)   -0.7% (  -7% -    6%) 0.522
                    Prefix3      584.55      (3.3%)      580.95      (3.1%)   -0.6% (  -6% -    5%) 0.541
                  OrHighMed       78.07      (2.7%)       77.60      (2.9%)   -0.6% (  -6% -    5%) 0.489
                 TermDTSort       92.32      (4.1%)       91.89      (4.4%)   -0.5% (  -8% -    8%) 0.732
          HighTermTitleSort      138.64      (2.4%)      138.05      (2.5%)   -0.4% (  -5% -    4%) 0.580
                AndHighHigh       20.00      (5.1%)       19.93      (4.3%)   -0.4% (  -9% -    9%) 0.791
       BrowseDateSSDVFacets        0.90      (9.5%)        0.90      (8.0%)   -0.4% ( -16% -   18%) 0.891
                     Fuzzy2       39.56      (1.2%)       39.41      (1.4%)   -0.4% (  -2% -    2%) 0.362
   AndHighHighDayTaxoFacets        2.08      (4.1%)        2.07      (4.0%)   -0.3% (  -8% -    8%) 0.798
                     Fuzzy1       66.33      (1.0%)       66.17      (1.1%)   -0.2% (  -2% -    1%) 0.471
                   Wildcard       40.31      (3.9%)       40.22      (3.7%)   -0.2% (  -7% -    7%) 0.856
           HighSloppyPhrase       11.01      (1.8%)       11.00      (2.2%)   -0.1% (  -4% -    3%) 0.852
               OrHighNotLow      219.55      (7.5%)      219.38      (7.3%)   -0.1% ( -13% -   15%) 0.974
                    Respell       50.51      (1.6%)       50.48      (1.5%)   -0.1% (  -3% -    3%) 0.915
                     IntNRQ       18.46      (8.9%)       18.46      (9.1%)   -0.0% ( -16% -   19%) 0.988
                 AndHighMed       82.85      (3.1%)       82.84      (2.5%)   -0.0% (  -5% -    5%) 0.982
               OrNotHighLow      512.93      (2.0%)      512.86      (1.9%)   -0.0% (  -3% -    3%) 0.982
                LowSpanNear       64.37      (2.4%)       64.44      (2.7%)    0.1% (  -4% -    5%) 0.886
              OrNotHighHigh      278.80      (6.4%)      279.28      (6.0%)    0.2% ( -11% -   13%) 0.931
                    LowTerm      351.93      (4.1%)      352.53      (4.4%)    0.2% (  -7% -    9%) 0.898
               OrNotHighMed      201.78      (5.3%)      202.14      (5.1%)    0.2% (  -9% -   11%) 0.913
              OrHighNotHigh      196.39      (6.5%)      196.74      (6.5%)    0.2% ( -11% -   14%) 0.930
            LowSloppyPhrase        4.06      (4.1%)        4.07      (4.6%)    0.2% (  -8% -    9%) 0.865
    AndHighMedDayTaxoFacets       29.95      (1.5%)       30.04      (1.7%)    0.3% (  -2% -    3%) 0.577
     OrHighMedDayTaxoFacets        3.47      (5.7%)        3.48      (4.3%)    0.3% (  -9% -   10%) 0.857
        MedIntervalsOrdered        7.68      (6.1%)        7.71      (6.6%)    0.4% ( -11% -   13%) 0.858
                    MedTerm      462.78      (5.3%)      464.47      (6.6%)    0.4% ( -10% -   12%) 0.847
                 AndHighLow      274.17      (2.2%)      275.22      (2.5%)    0.4% (  -4% -    5%) 0.606
               HighSpanNear        3.88      (3.9%)        3.90      (4.8%)    0.5% (  -7% -    9%) 0.738
  BrowseDayOfYearSSDVFacets        3.60      (8.3%)        3.62      (9.9%)    0.5% ( -16% -   20%) 0.863
               OrHighNotMed      286.50      (6.4%)      287.93      (6.3%)    0.5% ( -11% -   14%) 0.803
        LowIntervalsOrdered        4.82      (3.9%)        4.85      (3.8%)    0.5% (  -6% -    8%) 0.678
                MedSpanNear        4.81      (3.1%)        4.84      (4.0%)    0.6% (  -6% -    7%) 0.616
       HighIntervalsOrdered        4.37      (4.8%)        4.40      (5.0%)    0.6% (  -8% -   10%) 0.700
       MedTermDayTaxoFacets       11.79      (2.7%)       11.86      (3.1%)    0.6% (  -5% -    6%) 0.507
            MedSloppyPhrase       36.65      (5.2%)       36.95      (4.6%)    0.8% (  -8% -   11%) 0.592
                   HighTerm      318.60      (6.5%)      322.10      (7.5%)    1.1% ( -12% -   16%) 0.621
                  LowPhrase       31.99      (4.0%)       32.35      (2.0%)    1.1% (  -4% -    7%) 0.255
      BrowseMonthSSDVFacets        4.35     (13.5%)        4.41     (14.8%)    1.3% ( -23% -   34%) 0.771
                  MedPhrase       54.81      (3.5%)       55.56      (2.4%)    1.4% (  -4% -    7%) 0.147
BrowseRandomLabelSSDVFacets        2.68      (8.5%)        2.73      (8.2%)    1.6% ( -13% -   19%) 0.556
                 HighPhrase        3.07      (6.6%)        3.15      (4.3%)    2.5% (  -7% -   14%) 0.150
                   PKLookup      105.96      (1.1%)      115.03      (1.4%)    8.6% (   5% -   11%) 0.000

Do you mean to change StringHelper class to add support for 128 bit hash because currently it creates 32-bit hash with Murmur 3?

Sorry, I had overlooked that StringHelper only used 32 bits for its hash. If it's not a good fit, I'm good with hardcoding murmur3 in this postings format.

@shubhamvishu -- I think the PKLookup gains here are compelling, and there is consensus to make this improvement only to the BloomFilterPostingsFormat?  But understanding why supposedly equivalent expressions yield such a different benchmark result remains ...

I think the PKLookup gains here are compelling, and there is consensus to make this improvement only to the BloomFilterPostingsFormat?

Sure @mikemccand , I'll update this PR with the comments Adrien had earlier. I'll also post the new results shortly as its been sometime we last ran the benchmarks.

But understanding why supposedly equivalent expressions yield such a different benchmark result remains ...

For this I think we can open another issue to understand why we see different results.

But understanding why supposedly equivalent expressions yield such a different benchmark result remains ...

The expression ((int) A) >>> 1 + ((int) B) >>> 1 is equivalent to (((int) A) >>> (1 + ((int) B))) >>> 1 when we see the operator precedence and this seems to be causing the hash values to be bigger and sometimes smaller(I manually tried a couple of them) as opposed to using simple ((A/2) + (B/2)) which does not achieving that and produces large hashes everytime. I think this small tweak is generating hashes spread more uniformly in the range and leading to less collisions probably?.Another thing that helps is that the ideally we should have seen a regression if this was generating some garbage or similar hashes but we consistently seen improvement in both wikimediumall and wikibigall earlier. I'll share the latest benchmarks as well when they are completed. Any thoughts? @mikemccand @jpountz

But understanding why supposedly equivalent expressions yield such a different benchmark result remains ...

The expression ((int) A) >>> 1 + ((int) B) >>> 1 is equivalent to (((int) A) >>> (1 + ((int) B))) >>> 1 when we see the operator precedence

OK I see.  Yes indeed the expression is buggy (does not match murmur3 hash function) without the added ( ... ) thanks to java's operator precedence.
It would indeed be surprising if you accidentally discovered a better hashing function than murmur3 with this bug.  It is curious that you see consistently better PKLookup performance ... do you see any gains vs main with the bug fixed (added parens)?
Could we maybe add a random test to assert that your impl in BloomPostingsFormat matches StringUtil's murmur3?

do you see any gains vs main with the bug fixed (added parens)?

I ran the luceneutil benchmarks with this PR again and also when fixing the expression using "()". It required few changes in luceneutil to enable BloomFilteringPostingsFormat for id field (which are in the below commit).

Required luceneutil changes - https://github.com/shubhamvishu/luceneutil/commit/f675c6611a678f932360a89883b22f9ba9b75a8b

Here are the the latest results that I'm seeing with this PR :

1. Run # 1 : Result when using new expression i.e. this PR
                        TaskQPS   baseline  StdDevQPS  my_modified_version   StdDev  Pct diff      p-value
                   OrHighNotMed      151.52      (3.2%)      105.32      (2.6%)  -30.5% ( -35% -  -25%) 0.000
            MedIntervalsOrdered       35.15      (8.4%)       30.00      (7.7%)  -14.7% ( -28% -    1%) 0.000
                     HighPhrase       43.82      (4.9%)       37.83      (3.4%)  -13.7% ( -20% -   -5%) 0.000
                      MedPhrase       36.18     (11.7%)       31.64      (4.6%)  -12.5% ( -25% -    4%) 0.000
           HighIntervalsOrdered       12.19      (8.6%)       10.79      (6.4%)  -11.5% ( -24% -    3%) 0.000
      BrowseDayOfYearSSDVFacets        3.49     (13.7%)        3.11     (12.2%)  -10.8% ( -32% -   17%) 0.008
              HighTermMonthSort     1160.29      (4.2%)     1064.20      (2.8%)   -8.3% ( -14% -   -1%) 0.000
          BrowseMonthSSDVFacets        3.49     (15.0%)        3.27     (18.0%)   -6.5% ( -34% -   31%) 0.216
                    AndHighHigh       90.31      (5.8%)       85.38      (5.4%)   -5.5% ( -15% -    6%) 0.002
    BrowseRandomLabelSSDVFacets        2.17      (5.1%)        2.06      (7.9%)   -5.4% ( -17% -    7%) 0.010
           MedTermDayTaxoFacets        8.82      (6.5%)        8.35      (5.6%)   -5.4% ( -16% -    7%) 0.005
                         IntNRQ       60.02      (9.2%)       57.07     (13.7%)   -4.9% ( -25% -   19%) 0.184
          HighTermDayOfYearSort       75.06      (2.8%)       71.92      (4.4%)   -4.2% ( -11% -    3%) 0.000
                    MedSpanNear       22.04      (5.1%)       21.25      (4.2%)   -3.6% ( -12% -    6%) 0.015
                        Prefix3      417.51      (3.3%)      402.97      (3.5%)   -3.5% (  -9% -    3%) 0.001
                         Fuzzy1       66.49      (3.1%)       64.58      (2.8%)   -2.9% (  -8% -    3%) 0.002
                         Fuzzy2       60.77      (3.9%)       59.11      (5.2%)   -2.7% ( -11% -    6%) 0.061
                     AndHighLow      750.92      (3.5%)      738.33      (3.8%)   -1.7% (  -8% -    5%) 0.150
         OrHighMedDayTaxoFacets        3.19      (6.3%)        3.14      (6.8%)   -1.5% ( -13% -   12%) 0.481
        AndHighMedDayTaxoFacets        7.77      (3.7%)        7.68      (4.6%)   -1.2% (  -9% -    7%) 0.368
                        Respell       42.71      (4.1%)       42.45      (4.5%)   -0.6% (  -8% -    8%) 0.659
       AndHighHighDayTaxoFacets        2.68      (4.3%)        2.66      (4.1%)   -0.6% (  -8% -    8%) 0.673
               HighSloppyPhrase       18.32      (4.5%)       18.26      (6.0%)   -0.3% ( -10% -   10%) 0.851
                   OrNotHighLow      947.96      (4.4%)      946.42      (2.7%)   -0.2% (  -7% -    7%) 0.889
                LowSloppyPhrase       24.72      (4.3%)       24.80      (5.8%)    0.3% (  -9% -   10%) 0.834
                     OrHighHigh       63.44      (8.1%)       64.04      (6.8%)    0.9% ( -12% -   17%) 0.694
            LowIntervalsOrdered       47.35      (6.2%)       47.85      (7.0%)    1.1% ( -11% -   15%) 0.608
                     AndHighMed      175.58      (3.1%)      177.59      (3.6%)    1.1% (  -5% -    8%) 0.286
                  OrHighNotHigh       57.34      (2.5%)       58.02      (2.3%)    1.2% (  -3% -    6%) 0.125
                    LowSpanNear       54.84      (3.3%)       55.69      (4.3%)    1.6% (  -5% -    9%) 0.201
                      OrHighLow      366.06      (3.8%)      372.02      (2.7%)    1.6% (  -4% -    8%) 0.119
                MedSloppyPhrase       24.57      (4.0%)       24.99      (5.0%)    1.7% (  -6% -   11%) 0.224
                       Wildcard      161.09      (2.6%)      165.71      (3.1%)    2.9% (  -2% -    8%) 0.002
                      OrHighMed      116.69      (6.4%)      120.28      (4.6%)    3.1% (  -7% -   15%) 0.082
           HighTermTitleBDVSort        7.68      (2.7%)        7.92      (4.0%)    3.1% (  -3% -   10%) 0.004
                      LowPhrase       21.75      (6.3%)       22.62      (7.7%)    4.0% (  -9% -   19%) 0.072
                        LowTerm      197.26      (2.9%)      210.16      (2.8%)    6.5% (   0% -   12%) 0.000
              HighTermTitleSort       14.21      (4.2%)       15.16      (3.7%)    6.7% (  -1% -   15%) 0.000
                   HighSpanNear        4.45      (3.5%)        4.80      (2.6%)    8.0% (   1% -   14%) 0.000
                       HighTerm      111.71      (3.9%)      121.04      (2.8%)    8.3% (   1% -   15%) 0.000
                   OrNotHighMed      134.75      (3.2%)      146.33      (2.7%)    8.6% (   2% -   14%) 0.000
                       PKLookup      110.99      (4.4%)      124.24      (6.0%)   11.9% (   1% -   23%) 0.000
                  OrNotHighHigh       76.76      (3.0%)       91.73      (4.0%)   19.5% (  12% -   27%) 0.000
           BrowseDateSSDVFacets        0.57     (13.9%)        0.72     (19.1%)   25.6% (  -6% -   68%) 0.000
    BrowseRandomLabelTaxoFacets        1.98      (6.4%)        2.77     (36.1%)   39.9% (  -2% -   88%) 0.000
                        MedTerm      101.03      (4.3%)      145.90      (5.6%)   44.4% (  33% -   56%) 0.000
           BrowseDateTaxoFacets        2.50      (9.8%)        3.63     (41.4%)   45.2% (  -5% -  106%) 0.000
      BrowseDayOfYearTaxoFacets        2.51      (9.3%)        3.70     (42.2%)   47.8% (  -3% -  109%) 0.000
                   OrHighNotLow      100.87      (3.5%)      189.07      (7.6%)   87.4% (  73% -  102%) 0.000
                     TermDTSort       22.89      (2.9%)       54.10     (15.4%)  136.4% ( 114% -  159%) 0.000
          BrowseMonthTaxoFacets        2.41     (10.9%)        6.66     (93.8%)  176.2% (  64% -  315%) 0.000

2. Run # 2 : Result when using new expression i.e. this PR
                        TaskQPS   baseline  StdDevQPS  my_modified_version   StdDev  Pct diff      p-value
                    LowSpanNear       76.20      (2.5%)       67.29      (5.3%)  -11.7% ( -19% -   -3%) 0.000
            LowIntervalsOrdered       51.58      (5.5%)       47.22      (6.0%)   -8.5% ( -18% -    3%) 0.000
          BrowseMonthSSDVFacets        3.49     (20.6%)        3.20     (22.0%)   -8.1% ( -42% -   43%) 0.232
      BrowseDayOfYearSSDVFacets        3.17      (8.9%)        2.94     (13.4%)   -7.2% ( -27% -   16%) 0.045
    BrowseRandomLabelSSDVFacets        2.18     (12.8%)        2.03     (11.1%)   -7.0% ( -27% -   19%) 0.065
                     AndHighLow      738.26      (4.8%)      687.21      (5.3%)   -6.9% ( -16% -    3%) 0.000
                LowSloppyPhrase       91.80      (4.0%)       85.81      (4.9%)   -6.5% ( -14% -    2%) 0.000
                     OrHighHigh       72.55      (9.6%)       68.80      (8.5%)   -5.2% ( -21% -   14%) 0.073
           MedTermDayTaxoFacets        6.88      (6.5%)        6.57      (8.1%)   -4.6% ( -17% -   10%) 0.050
                      MedPhrase       73.44      (2.7%)       70.35      (5.9%)   -4.2% ( -12% -    4%) 0.004
                        Respell       26.61      (4.0%)       25.53      (5.6%)   -4.1% ( -13% -    5%) 0.008
                        LowTerm      202.01      (3.5%)      193.99      (2.6%)   -4.0% (  -9% -    2%) 0.000
                MedSloppyPhrase       35.32      (5.4%)       33.94      (7.1%)   -3.9% ( -15% -    9%) 0.050
                        Prefix3      406.66      (6.5%)      392.60      (5.2%)   -3.5% ( -14% -    8%) 0.063
        AndHighMedDayTaxoFacets       19.45      (4.9%)       18.92      (5.3%)   -2.8% ( -12% -    7%) 0.087
       AndHighHighDayTaxoFacets        2.77      (5.5%)        2.69      (6.7%)   -2.8% ( -14% -   10%) 0.156
         OrHighMedDayTaxoFacets        3.34      (5.0%)        3.25      (7.6%)   -2.7% ( -14% -   10%) 0.184
                       Wildcard      768.29      (3.6%)      751.35      (3.9%)   -2.2% (  -9% -    5%) 0.060
                         Fuzzy2       44.04      (5.4%)       43.09      (5.2%)   -2.1% ( -12% -    8%) 0.201
                   OrNotHighLow      853.76      (3.0%)      835.98      (2.6%)   -2.1% (  -7% -    3%) 0.019
                    AndHighHigh       85.25      (7.5%)       83.76      (6.2%)   -1.7% ( -14% -   12%) 0.422
              HighTermMonthSort     1068.18      (3.6%)     1050.07      (3.9%)   -1.7% (  -8% -    6%) 0.153
               HighSloppyPhrase        5.96      (3.8%)        5.94      (3.6%)   -0.4% (  -7% -    7%) 0.737
           HighTermTitleBDVSort        7.39      (3.9%)        7.36      (2.9%)   -0.3% (  -6% -    6%) 0.785
                       HighTerm       83.27      (3.3%)       83.02      (3.5%)   -0.3% (  -6% -    6%) 0.781
                      LowPhrase       92.82      (3.7%)       92.58      (2.1%)   -0.3% (  -5% -    5%) 0.791
                      OrHighLow      266.30      (2.0%)      268.48      (2.3%)    0.8% (  -3% -    5%) 0.229
           HighIntervalsOrdered        5.48      (5.7%)        5.53      (5.5%)    0.9% (  -9% -   12%) 0.621
            MedIntervalsOrdered       26.98      (5.8%)       27.26      (6.6%)    1.0% ( -10% -   14%) 0.597
                    MedSpanNear        4.02      (2.6%)        4.09      (3.5%)    1.6% (  -4% -    7%) 0.104
                      OrHighMed       65.75      (7.5%)       66.80      (6.5%)    1.6% ( -11% -   16%) 0.472
                     HighPhrase      120.64      (4.8%)      122.75      (4.2%)    1.8% (  -6% -   11%) 0.222
                   HighSpanNear       11.14      (3.6%)       11.34      (2.7%)    1.8% (  -4% -    8%) 0.073
                  OrHighNotHigh       83.12      (2.8%)       84.65      (2.3%)    1.8% (  -3% -    7%) 0.022
                   OrNotHighMed      147.67      (2.2%)      150.48      (2.6%)    1.9% (  -2% -    6%) 0.011
                     AndHighMed      168.61      (3.1%)      173.38      (3.4%)    2.8% (  -3% -    9%) 0.006
                         Fuzzy1       75.52      (7.4%)       77.77      (6.1%)    3.0% (  -9% -   17%) 0.165
                   OrHighNotMed       55.39      (3.3%)       60.07      (3.7%)    8.4% (   1% -   16%) 0.000
                       PKLookup      108.78      (5.8%)      119.09      (6.0%)    9.5% (  -2% -   22%) 0.000
                  OrNotHighHigh      208.07      (3.5%)      235.47      (3.4%)   13.2% (   6% -   20%) 0.000
              HighTermTitleSort        7.70      (3.9%)        9.25      (3.4%)   20.1% (  12% -   28%) 0.000
          HighTermDayOfYearSort       61.64      (2.7%)       75.16      (4.6%)   21.9% (  14% -   30%) 0.000
           BrowseDateSSDVFacets        0.56     (15.7%)        0.69     (18.9%)   22.9% ( -10% -   68%) 0.000
           BrowseDateTaxoFacets        2.30      (4.9%)        3.14      (8.5%)   37.0% (  22% -   52%) 0.000
    BrowseRandomLabelTaxoFacets        1.85      (3.5%)        2.55      (5.6%)   38.0% (  27% -   48%) 0.000
      BrowseDayOfYearTaxoFacets        2.28      (4.7%)        3.25      (6.7%)   42.5% (  29% -   56%) 0.000
                   OrHighNotLow      165.24      (2.9%)      248.52      (4.7%)   50.4% (  41% -   59%) 0.000
                        MedTerm       92.99      (2.9%)      144.38      (5.5%)   55.3% (  45% -   65%) 0.000
                         IntNRQ       31.39     (29.9%)       50.82      (9.1%)   61.9% (  17% -  144%) 0.000
          BrowseMonthTaxoFacets        2.36     (11.1%)        6.79     (93.6%)  188.0% (  75% -  329%) 0.000
                     TermDTSort       12.60      (4.2%)       68.61     (72.4%)  444.5% ( 352% -  544%) 0.000

3. Run # 3 : Result when NOT using new expression
                        TaskQPS   baseline  StdDevQPS  my_modified_version   StdDev  Pct diff      p-value
                       HighTerm      171.08      (3.0%)      101.57      (2.5%)  -40.6% ( -44% -  -36%) 0.000
              HighTermTitleSort       15.14      (3.9%)       12.25      (3.1%)  -19.1% ( -25% -  -12%) 0.000
                    LowSpanNear      170.82      (9.6%)      144.83     (13.7%)  -15.2% ( -35% -    8%) 0.000
                      MedPhrase      148.79      (4.2%)      130.79      (3.5%)  -12.1% ( -19% -   -4%) 0.000
               HighSloppyPhrase       31.34      (7.7%)       27.66      (7.1%)  -11.7% ( -24% -    3%) 0.000
                LowSloppyPhrase       57.48      (8.4%)       51.22      (7.6%)  -10.9% ( -24% -    5%) 0.000
            MedIntervalsOrdered       48.65      (7.5%)       43.49      (5.1%)  -10.6% ( -21% -    2%) 0.000
          BrowseMonthSSDVFacets        3.61     (16.0%)        3.33     (14.5%)   -7.7% ( -32% -   27%) 0.108
                      LowPhrase       21.09      (6.6%)       19.57      (4.9%)   -7.2% ( -17% -    4%) 0.000
    BrowseRandomLabelSSDVFacets        2.21      (4.3%)        2.08      (5.1%)   -5.8% ( -14% -    3%) 0.000
      BrowseDayOfYearSSDVFacets        3.38     (11.7%)        3.19     (14.9%)   -5.4% ( -28% -   24%) 0.203
           HighIntervalsOrdered        6.48      (5.8%)        6.16      (5.9%)   -5.0% ( -15% -    7%) 0.007
                   OrNotHighLow     1349.40      (3.9%)     1284.48      (4.1%)   -4.8% ( -12% -    3%) 0.000
                   HighSpanNear        7.79      (2.6%)        7.42      (3.5%)   -4.8% ( -10% -    1%) 0.000
                       PKLookup      113.95      (3.7%)      108.74      (4.6%)   -4.6% ( -12% -    3%) 0.001
        AndHighMedDayTaxoFacets       44.93      (3.5%)       43.23      (6.7%)   -3.8% ( -13% -    6%) 0.025
                        Respell       35.71      (3.2%)       34.37      (5.6%)   -3.7% ( -12% -    5%) 0.009
                     AndHighMed       96.91      (6.8%)       93.83      (3.9%)   -3.2% ( -13% -    8%) 0.070
                     OrHighHigh       65.98      (8.2%)       64.12     (12.0%)   -2.8% ( -21% -   18%) 0.387
                     AndHighLow     1356.33      (3.6%)     1319.07      (5.2%)   -2.7% ( -11% -    6%) 0.052
                    AndHighHigh       66.89      (7.6%)       65.17      (6.9%)   -2.6% ( -15% -   12%) 0.263
         OrHighMedDayTaxoFacets        2.70      (7.1%)        2.63      (5.6%)   -2.5% ( -14% -   11%) 0.225
                     HighPhrase       80.60      (4.9%)       79.11      (4.6%)   -1.8% ( -10% -    8%) 0.221
       AndHighHighDayTaxoFacets       10.10      (4.0%)        9.91      (4.4%)   -1.8% (  -9% -    6%) 0.175
                       Wildcard      139.44      (2.7%)      137.27      (3.0%)   -1.6% (  -7% -    4%) 0.083
           HighTermTitleBDVSort        8.90      (4.0%)        8.76      (3.2%)   -1.6% (  -8% -    5%) 0.174
           MedTermDayTaxoFacets       16.02      (4.7%)       15.78      (3.2%)   -1.5% (  -9% -    6%) 0.225
                         Fuzzy1       58.33      (3.4%)       57.52      (4.5%)   -1.4% (  -8% -    6%) 0.274
                        Prefix3      495.32      (2.3%)      489.23      (1.9%)   -1.2% (  -5% -    3%) 0.063
                MedSloppyPhrase       36.19      (4.0%)       35.89      (2.5%)   -0.8% (  -7% -    5%) 0.428
          HighTermDayOfYearSort       60.62      (4.4%)       61.15      (4.2%)    0.9% (  -7% -    9%) 0.520
                    MedSpanNear       16.26      (4.8%)       16.55      (6.5%)    1.7% (  -9% -   13%) 0.341
            LowIntervalsOrdered       57.41      (5.4%)       58.49      (4.2%)    1.9% (  -7% -   12%) 0.224
                      OrHighMed      130.42      (5.2%)      133.04      (5.5%)    2.0% (  -8% -   13%) 0.238
                         IntNRQ       70.41      (3.8%)       71.85      (5.2%)    2.0% (  -6% -   11%) 0.153
              HighTermMonthSort     1101.95      (3.8%)     1129.04      (3.2%)    2.5% (  -4% -    9%) 0.025
                   OrHighNotMed       56.97      (3.3%)       58.71      (3.1%)    3.1% (  -3% -    9%) 0.003
                         Fuzzy2       49.17      (3.5%)       51.36      (4.3%)    4.5% (  -3% -   12%) 0.000
                      OrHighLow      290.33      (3.0%)      303.84      (2.3%)    4.7% (   0% -   10%) 0.000
                  OrHighNotHigh       88.66      (3.7%)       93.58      (2.9%)    5.6% (   0% -   12%) 0.000
                        LowTerm      209.03      (2.4%)      228.72      (2.9%)    9.4% (   4% -   15%) 0.000
                        MedTerm       73.56      (3.1%)       86.92      (3.7%)   18.2% (  10% -   25%) 0.000
                   OrNotHighMed      218.55      (2.1%)      267.79      (3.2%)   22.5% (  16% -   28%) 0.000
                  OrNotHighHigh       45.70      (2.9%)       57.84      (4.7%)   26.6% (  18% -   35%) 0.000
           BrowseDateSSDVFacets        0.60     (13.1%)        0.77     (17.4%)   29.2% (  -1% -   68%) 0.000
                   OrHighNotLow      163.54      (2.5%)      214.57      (2.9%)   31.2% (  25% -   37%) 0.000
                     TermDTSort       29.64      (4.0%)       39.09      (6.2%)   31.9% (  20% -   43%) 0.000
    BrowseRandomLabelTaxoFacets        1.92      (2.5%)        2.88     (46.6%)   49.7% (   0% -  101%) 0.000
           BrowseDateTaxoFacets        2.39      (6.1%)        3.79     (63.8%)   59.0% ( -10% -  137%) 0.000
      BrowseDayOfYearTaxoFacets        2.37      (5.4%)        3.85     (67.2%)   62.4% (  -9% -  142%) 0.000
          BrowseMonthTaxoFacets        2.49      (3.7%)        7.08     (93.6%)  183.8% (  83% -  291%) 0.000

I'm seeing some crazy speedups for some tasks in the benchmarks (including PKLookup; a few got little slower) when using the new expression. Looking for you thoughts on this, Is my luceneutil changes right and could we expect this PR to affect other tasks than PKLookup as well like above?. These reported gains are so high that it got me a bit suspicious about the reported gains. I'd need an extra pair of eyes here.
@mikemccand @jpountz Any thoughts ?

do you see any gains vs main with the bug fixed (added parens)?

When not using the new expression i..e using "()" as we normally expect (below code changes) I still see high speedups for some tasks in the benchmarks(earlier comment) but this time PKLookup regresses ~4-5%with this change. ``
@@ -151,8 +151,8 @@ public class FuzzySet implements Accountable {
   public ContainsResult contains(BytesRef value) {
     long[] hash = StringHelper.murmurhash3_x64_128(value);

-    int msb = ((int) hash[0] >>> Integer.SIZE) >>> 1 + ((int) hash[1] >>> Integer.SIZE) >>> 1;
-    int lsb = ((int) hash[0]) >>> 1 + ((int) hash[1]) >>> 1;
+    int msb = (((int) hash[0] >>> Integer.SIZE) >>> 1) + (((int) hash[1] >>> Integer.SIZE) >>> 1);
+    int lsb = (((int) hash[0]) >>> 1) + (((int) hash[1]) >>> 1);
     for (int i = 0; i < hashCount; i++) {
       int bloomPos = (lsb + i * msb);
       if (!mayContainValue(bloomPos)) {
@@ -219,8 +219,8 @@ public class FuzzySet implements Accountable {
    */
   public void addValue(BytesRef value) {
     long[] hash = StringHelper.murmurhash3_x64_128(value);
-    int msb = ((int) hash[0] >>> Integer.SIZE) >>> 1 + ((int) hash[1] >>> Integer.SIZE) >>> 1;
-    int lsb = ((int) hash[0]) >>> 1 + ((int) hash[1]) >>> 1;
+    int msb = (((int) hash[0] >>> Integer.SIZE) >>> 1) + (((int) hash[1] >>> Integer.SIZE) >>> 1);
+    int lsb = (((int) hash[0]) >>> 1) + (((int) hash[1]) >>> 1);
     for (int i = 0; i < hashCount; i++) {
       // Bitmasking using bloomSize is effectively a modulo operation.
       int bloomPos = (lsb + i * msb) & bloomSize;

@mikemccand  @jpountz  Bumping this up. I don't know if there are any major concerns here? If not, I'd we awesome to include this also for 10.0. Looking for your thoughts? Thanks!

Thanks for the reminder @shubhamvishu!

I'm seeing some crazy speedups for some tasks in the benchmarks (including PKLookup; a few got little slower) when using the new expression.

Hmm did you post the full results somewhere?

I'm seeing some crazy speedups for some tasks in the benchmarks (including PKLookup; a few got little slower) when using the new expression.

Hmm did you post the full results somewhere?

OK sorry I see them now.  I have to click on those sophisticated arrows to expand the results, heh.

@jpountz I have addressed your comments now and kept the bit mixing logic simple as proposed initially. Let me know if the change looks good now. Thanks!
On the side note, I see 9.12 RC1 is out so we would be shipping it in Lucene 10.0?

@jpountz I simplified the expression now. Let me know if the change looks good? Thanks!

BTW, why StringHelper is a abstract class? Can we make it final?

I think yes we can make it final as it has no abstract methods so need to have it abstract. Git blame says it was made abstract >20 years ago by Doug so maybe it just stayed like this since.

Thanks @shubhamvishu , I opend https://github.com/apache/lucene/pull/13928.


Fixed bng | NGIC DP support

--Added copyrights
--Formatted code using gofmt

@vinodborole we are starting to review this PR.

Please create one commit, not 7, with the informative description.
Move the NGIC to the examples directory
Reuse the existing functions by importing packages, instead of copying them - ARP, parsing, logging, etc.
This code is C, and not idiomatic Go - can you please follow recomendation from here https://pocketgophers.com/idiomatic-go/
ARP package should be part of packet and reuse and change the existing functions instead of adding new ones.
constants should become constants not vars
Comments should be corrected everywhere and be descriptive and comprehensive.
Utils should be incorporated with the existing common package
Copyright should be changed - NFF-GO authors. We cannot accept GSLab copyrighted code.
Please follow the contributer's guide https://github.com/intel-go/nff-go/blob/master/CONTRIBUTING.md


[feature request] Client certificate support for TLS authentication

Hi there - Thanks for all of the excellent work on this project.  Really great stuff - I've only been using it for about a week and already it's been helpful for me.  I especially appreciate the simplicity of posting messages - it makes it so easy to integrate into other services.
My suggestion:
I was hoping to use client certificates on my front end proxy for TLS authentication to ntfy.  I configured my proxy for this and I'm able to authenticate and then access ntfy via browser - I get prompted for the client cert and all works.  (hooray!)  However, doing the same on the android app, I get the error below.
I'm thinking that the client Android app does not support client certificates based on this.  I am submitting this suggestion in hopes that the feature might be considered as a future enhancement.

I would really like this as well! I use user cert auth for everything (related to my automation systems) and it's definitely be preferable to be able to do that for ntfy as well instead of having to have a separate scheme.


[FIXED] CLI generator bug

fix(cli): always output the original file while no loader can process the assets
re #5

Codecov Report

Merging #10 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #10   +/-   ##
=======================================
  Coverage   95.25%   95.25%
=======================================
  Files          62       62
  Lines         969      969
  Branches      258      258
=======================================
  Hits          923      923
  Misses         40       40
  Partials        6        6

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2383c15...fb0d8fe. Read the comment docs.


Add Simplified Chinese translation

Some of them I don't understand or couldn't find the Chinese translation in wiki, so I Translated them literally:
action bar -> action栏
Help query string. -> 帮助查询字符串。
selector -> 选择器

Hi, thanks for wanting to contribute!
There was already a PR open adding Simplified Chinese (https://github.com/jpenilla/TabTPS/pull/21), which I have just merged. If you notice any issues in the merged translations, please open a new PR to fix them.


WorkflowStep can not save

I follow this document and tried adding WorkflowStep to my Slack bot, WorkflowStep.edit works fine, but WorkflowStep.save fails anyway, I can't find the reason, can you help me?
Reproducible in:
val step = WorkflowStep.builder()
      .callbackId("copy_review")
      .edit { req, context ->
        context.configure(withBlocks {
          section {
            blockId("intro-section")
            plainText("text")
          }
          input {
            blockId("task_name_input")
            label("Task Name")
            element {
              plainTextInput {
                actionId("task_name")
              }
            }
          }
        })
        context.ack()
      }
      .save { req, context ->
        val inputs = buildMap {
          put("taskName", WorkflowSteps.stepInput {
            it.value(req.payload.view.state.findElement("task_name").value)
          })
        }
        val outputs = buildList {
          add(WorkflowSteps.stepOutput {
            it.name("taskName").type("text").label("Task Name")
          })
        }
        context.update(inputs, outputs)
        context.ack()
      }
      .execute { req, context ->
        context.ack()
      }
      .build()
    app.step(step)

The Slack SDK version
+--- com.slack.api:bolt:1.22.0
|    +--- com.slack.api:slack-api-model:1.22.0
|    +--- com.slack.api:slack-api-client:1.22.0
|    |    +--- com.slack.api:slack-api-model:1.22.0 (*)
|    \--- com.slack.api:slack-app-backend:1.22.0
|         +--- com.slack.api:slack-api-model:1.22.0 (*)
|         +--- com.slack.api:slack-api-client:1.22.0 (*)
+--- com.slack.api:bolt-ktor:1.22.0
|    +--- com.slack.api:slack-api-client:1.22.0 (*)
|    +--- com.slack.api:bolt:1.22.0 (*)
+--- com.slack.api:slack-api-model-kotlin-extension:1.22.0
|    +--- com.slack.api:slack-api-model:1.22.0 (*)
+--- com.slack.api:slack-api-client-kotlin-extension:1.22.0
|    +--- com.slack.api:slack-api-model-kotlin-extension:1.22.0 (*)
|    +--- com.slack.api:slack-api-client:1.22.0 (*)
+--- com.slack.api:bolt:1.22.0 (n)
+--- com.slack.api:bolt-ktor:1.22.0 (n)
+--- com.slack.api:slack-api-model-kotlin-extension:1.22.0 (n)
+--- com.slack.api:slack-api-client-kotlin-extension:1.22.0 (n)
+--- com.slack.api:bolt:1.22.0
|    +--- com.slack.api:slack-api-model:1.22.0
|    +--- com.slack.api:slack-api-client:1.22.0
|    |    +--- com.slack.api:slack-api-model:1.22.0 (*)
|    \--- com.slack.api:slack-app-backend:1.22.0
|         +--- com.slack.api:slack-api-model:1.22.0 (*)
|         +--- com.slack.api:slack-api-client:1.22.0 (*)
+--- com.slack.api:bolt-ktor:1.22.0
|    +--- com.slack.api:slack-api-client:1.22.0 (*)
|    +--- com.slack.api:bolt:1.22.0 (*)
+--- com.slack.api:slack-api-model-kotlin-extension:1.22.0
|    +--- com.slack.api:slack-api-model:1.22.0 (*)
+--- com.slack.api:slack-api-client-kotlin-extension:1.22.0
|    +--- com.slack.api:slack-api-model-kotlin-extension:1.22.0 (*)
|    +--- com.slack.api:slack-api-client:1.22.0 (*)
+--- com.slack.api:bolt:1.22.0
|    +--- com.slack.api:slack-api-model:1.22.0
|    +--- com.slack.api:slack-api-client:1.22.0
|    |    +--- com.slack.api:slack-api-model:1.22.0 (*)
|    \--- com.slack.api:slack-app-backend:1.22.0
|         +--- com.slack.api:slack-api-model:1.22.0 (*)
|         +--- com.slack.api:slack-api-client:1.22.0 (*)
+--- com.slack.api:bolt-ktor:1.22.0
|    +--- com.slack.api:slack-api-client:1.22.0 (*)
|    +--- com.slack.api:bolt:1.22.0 (*)
+--- com.slack.api:slack-api-model-kotlin-extension:1.22.0
|    +--- com.slack.api:slack-api-model:1.22.0 (*)
+--- com.slack.api:slack-api-client-kotlin-extension:1.22.0
|    +--- com.slack.api:slack-api-model-kotlin-extension:1.22.0 (*)
|    +--- com.slack.api:slack-api-client:1.22.0 (*)
+--- com.slack.api:bolt:1.22.0
|    +--- com.slack.api:slack-api-model:1.22.0
|    +--- com.slack.api:slack-api-client:1.22.0
|    |    +--- com.slack.api:slack-api-model:1.22.0 (*)
|    \--- com.slack.api:slack-app-backend:1.22.0
|         +--- com.slack.api:slack-api-model:1.22.0 (*)
|         +--- com.slack.api:slack-api-client:1.22.0 (*)
+--- com.slack.api:bolt-ktor:1.22.0
|    +--- com.slack.api:slack-api-client:1.22.0 (*)
|    +--- com.slack.api:bolt:1.22.0 (*)
+--- com.slack.api:slack-api-model-kotlin-extension:1.22.0
|    +--- com.slack.api:slack-api-model:1.22.0 (*)
+--- com.slack.api:slack-api-client-kotlin-extension:1.22.0
|    +--- com.slack.api:slack-api-model-kotlin-extension:1.22.0 (*)
|    +--- com.slack.api:slack-api-client:1.22.0 (*)
+--- com.slack.api:bolt:1.22.0
|    +--- com.slack.api:slack-api-model:1.22.0
|    +--- com.slack.api:slack-api-client:1.22.0
|    |    +--- com.slack.api:slack-api-model:1.22.0 (*)
|    \--- com.slack.api:slack-app-backend:1.22.0
|         +--- com.slack.api:slack-api-model:1.22.0 (*)
|         +--- com.slack.api:slack-api-client:1.22.0 (*)
+--- com.slack.api:bolt-ktor:1.22.0
|    +--- com.slack.api:slack-api-client:1.22.0 (*)
|    +--- com.slack.api:bolt:1.22.0 (*)
+--- com.slack.api:slack-api-model-kotlin-extension:1.22.0
|    +--- com.slack.api:slack-api-model:1.22.0 (*)
+--- com.slack.api:slack-api-client-kotlin-extension:1.22.0
|    +--- com.slack.api:slack-api-model-kotlin-extension:1.22.0 (*)
|    +--- com.slack.api:slack-api-client:1.22.0 (*)
+--- com.slack.api:bolt:1.22.0
|    +--- com.slack.api:slack-api-model:1.22.0
|    +--- com.slack.api:slack-api-client:1.22.0
|    |    +--- com.slack.api:slack-api-model:1.22.0 (*)
|    \--- com.slack.api:slack-app-backend:1.22.0
|         +--- com.slack.api:slack-api-model:1.22.0 (*)
|         +--- com.slack.api:slack-api-client:1.22.0 (*)
+--- com.slack.api:bolt-ktor:1.22.0
|    +--- com.slack.api:slack-api-client:1.22.0 (*)
|    +--- com.slack.api:bolt:1.22.0 (*)
+--- com.slack.api:slack-api-model-kotlin-extension:1.22.0
|    +--- com.slack.api:slack-api-model:1.22.0 (*)
+--- com.slack.api:slack-api-client-kotlin-extension:1.22.0
|    +--- com.slack.api:slack-api-model-kotlin-extension:1.22.0 (*)

Java Runtime version
java version "11.0.12" 2021-07-20 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.12+8-LTS-237)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.12+8-LTS-237, mixed mode)

OS info
ProductName:    macOS
ProductVersion: 12.2.1
BuildVersion:   21D62
Darwin Kernel Version 21.3.0: Wed Jan  5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_X86_64

Hi @Arxing - Are there any logs you can provide for what you see when the WorkflowStep.edit vs. .save is occurring?

Hi @Arxing, I've checked your code and found that the following line of code can throw exceptions (actually the code as-is does not compile for me).

it.value(req.payload.view.state.findElement("task_name").value)

I've revised the part this way:
it.value(stateValues["task_name_input"]?.get("task_name")?.value)

Here is the complete code and its build settings:
import com.slack.api.bolt.App
import com.slack.api.bolt.middleware.builtin.WorkflowStep
import com.slack.api.bolt.socket_mode.SocketModeApp
import com.slack.api.model.kotlin_extension.block.withBlocks
import com.slack.api.model.workflow.WorkflowSteps

fun main() {
  System.setProperty("org.slf4j.simpleLogger.log.com.slack.api", "debug")
  val app = App()

  val step = WorkflowStep.builder()
    .callbackId("socket-mode-step")
    .edit { _, context ->
      context.configure(withBlocks {
        section {
          blockId("intro-section")
          plainText("text")
        }
        input {
          blockId("task_name_input")
          label("Task Name")
          element {
            plainTextInput {
              actionId("task_name")
            }
          }
        }
      })
      context.ack()
    }
    .save { req, context ->
      val stateValues = req.payload.view.state.values
      val inputs = buildMap {
        // This part used to be failing
        put("taskName", WorkflowSteps.stepInput {
          it.value(stateValues["task_name_input"]?.get("task_name")?.value)
        })
      }
      val outputs = buildList {
        add(WorkflowSteps.stepOutput {
          it.name("taskName").type("text").label("Task Name")
        })
      }
      context.update(inputs, outputs)
      context.ack()
    }
    .execute { _, context ->
      context.ack()
    }
    .build()
  app.step(step)

  SocketModeApp(app).start()
}

plugins {
  id("org.jetbrains.kotlin.jvm") version "1.6.21"
  id("application")
}
repositories {
  mavenCentral()
}
dependencies {
  implementation(platform("org.jetbrains.kotlin:kotlin-bom"))
  implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
  implementation("com.slack.api:bolt-socket-mode:1.22.0")
  implementation("com.slack.api:slack-api-model-kotlin-extension:1.22.0")
  implementation("com.slack.api:slack-api-client-kotlin-extension:1.22.0")
  implementation("javax.websocket:javax.websocket-api:1.1")
  implementation("org.glassfish.tyrus.bundles:tyrus-standalone-client:1.17")
  implementation('org.slf4j:slf4j-simple:1.7.36') // or logback-classic
}
application {
  mainClassName = "MyAppKt" // add "Kt" suffix for main function source file
}

I hope this helps. Can you close this issue if my response was helpful?

sorry, findElement() is my extension function
fun ViewState.findElementOrNull(id: String): ViewState.Value? {
  return values.flatMap { (_, stateTree) ->
    stateTree.entries
  }.find { (elementId, _) ->
    elementId == id
  }?.value
}

fun ViewState.findElement(id: String): ViewState.Value = findElementOrNull(id).notNull()

It is expected to return the ViewState.Value of the specified element

Here is the log that triggers Workflow.save after I add step in WorkflowBuilder and click "save"
2022-05-04 11:09:22.970 [eventLoopGroupProxy-4-1] DEBUG App.runMiddleware():1017 - Applying a middleware (name: com.slack.api.bolt.middleware.builtin.SSLCheck)
2022-05-04 11:09:22.970 [eventLoopGroupProxy-4-1] DEBUG App.runMiddleware():1017 - Applying a middleware (name: com.slack.api.bolt.middleware.builtin.RequestVerification)
2022-05-04 11:09:22.970 [eventLoopGroupProxy-4-1] DEBUG SlackSignature$Verifier.isValid():150 - Request verification (timestamp:<PHONE_NUMBER>, body: payload=*****)
2022-05-04 11:09:22.971 [eventLoopGroupProxy-4-1] DEBUG App.runMiddleware():1017 - Applying a middleware (name: com.slack.api.bolt.middleware.builtin.SingleTeamAuthorization)
2022-05-04 11:09:22.971 [eventLoopGroupProxy-4-1] DEBUG App.runMiddleware():1017 - Applying a middleware (name: com.slack.api.bolt.middleware.builtin.IgnoringSelfEvents)
2022-05-04 11:09:22.971 [eventLoopGroupProxy-4-1] DEBUG App.runMiddleware():1017 - Applying a middleware (name: com.slack.api.bolt.middleware.builtin.WorkflowStep)
2022-05-04 11:09:22.971 [eventLoopGroupProxy-4-1] DEBUG WorkflowStep.apply():51 - The WorkflowStep middleware started (request type: WorkflowStepSave)
2022-05-04 11:09:23.569 [eventLoopGroupProxy-4-1] DEBUG DetailedLoggingListener.accept():80 -
[Request URL]
POST https://slack.com/api/workflows.updateStep
[Specified Request Headers]
Authorization: (redacted)
User-Agent: Java-Slack-SDK; slack-api-client/1.22.0; Java HotSpot(TM) 64-Bit Server VM/11.0.12; Mac OS X/11.2; bolt/1.22.0;
[Request Body]
workflow_step_edit_id=3469107548566.2193215711.e43211b2e5dc4224331364fcc259b868&inputs=%7B%22taskName%22%3A%7B%22value%22%3A%22123%22%7D%7D&outputs=%5B%7B%22name%22%3A%22taskName%22%2C%22type%22%3A%22text%22%2C%22label%22%3A%22Task%20Name%22%7D%5D

Content-Type: application/x-www-form-urlencoded
Content Length: 247

[Response Status]
200
[Response Headers]
date: Wed, 04 May 2022 03:09:23 GMT
server: Apache
x-powered-by: HHVM/4.153.1
access-control-allow-origin: *
referrer-policy: no-referrer
x-slack-backend: r
strict-transport-security: max-age=31536000; includeSubDomains; preload
access-control-allow-headers: slack-route, x-slack-version-ts, x-b3-traceid, x-b3-spanid, x-b3-parentspanid, x-b3-sampled, x-b3-flags
access-control-expose-headers: x-slack-req-id, retry-after
x-oauth-scopes: users:read,usergroups:read,users:read.email,users.profile:read,groups:read,channels:read,channels:history,chat:write,app_mentions:read,chat:write.customize,chat:write.public,incoming-webhook,commands,emoji:read,im:write,groups:write,mpim:write,workflow.steps:execute
x-accepted-oauth-scopes: workflow.steps:execute
expires: Mon, 26 Jul 1997 05:00:00 GMT
cache-control: private, no-cache, no-store, must-revalidate
pragma: no-cache
x-xss-protection: 0
x-content-type-options: nosniff
x-slack-req-id: e2eebbf0b8c28db0f2a49382a2f4a77c
vary: Accept-Encoding
content-type: application/json; charset=utf-8
x-envoy-upstream-service-time: 232
x-backend: main_normal main_bedrock_normal_with_overflow main_canary_with_overflow main_bedrock_canary_with_overflow main_control_with_overflow main_bedrock_control_with_overflow
x-server: slack-www-hhvm-main-iad-1hr0
x-slack-shared-secret-outcome: no-match
via: envoy-www-iad-r9ck, envoy-edge-nrt-ut7a
x-edge-backend: envoy-www
x-slack-edge-shared-secret-outcome: no-match
[Response Body]
{"ok":true}

2022-05-04 11:09:23.569 [eventLoopGroupProxy-4-1] DEBUG WorkflowStep.apply():119 - The WorkflowStep middleware completed (request type: WorkflowStepSave)
2022-05-04 11:09:23.569 [eventLoopGroupProxy-4-1] INFO  application.log():214 - 200 OK: POST - /slack/events

the screenshot

it just shows me the error "Sorry, there was a problem trying to save your step." and the dialog doesn't close, but can see in the background that the step have been added, is this a slack WorkflowBuilder bug? Or am I missing some details?

@Arxing I am still unsure about the cause of the issue that you are facing. Can you share your app's App Manifest (the YAML settings that you can find at https://api.slack.com/apps)? If your app does not enable Interactivity, the situation may arise.

Here is my app's App Manifest, I replaced some sensitive information with "*".
display_information:
  name: *
  description: *
  background_color: "#0f3663"
features:
  bot_user:
    display_name: *
    always_online: true
  shortcuts:
    - name: 刪除這則訊息
      type: message
      callback_id: deleteMessage
      description: 刪除這則訊息
    - name: Create a Schedule
      type: global
      callback_id: createSchedule
      description: 快速建立一個 Automation Schedule
  slash_commands:
    - command: /query-top-fatal-issues
      url: *
      description: 查詢 Fatal Crashlytics
      usage_hint: "平台 \b查詢天數 查詢筆數"
      should_escape: false
    - command: /query-top-non-fatal-issues
      url: *
      description: 查詢 Non-Fatal Crashlytics
      usage_hint: "平台 \b查詢天數 查詢筆數"
      should_escape: false
    - command: /gen-appbot-token
      url: *
      description: 取得 App Bot Token
      should_escape: false
    - command: /appbot-set-role
      url: *
      description: 設定使用者的角色
      should_escape: false
    - command: /appbot-admin
      url: *
      description: 管理員功能
      should_escape: false
    - command: /appbot-automation
      url: *
      description: APP 自動化指令
      should_escape: false
    - command: /appbot-test
      url: *
      description: 測試用指令
      should_escape: false
  workflow_steps:
    - name: 測試用 Step
      callback_id: copy_review
oauth_config:
  scopes:
    user:
      - chat:write
      - channels:history
      - reactions:read
      - users:read
      - users:read.email
      - channels:write
      - groups:write
      - mpim:write
      - im:write
    bot:
      - chat:write
      - app_mentions:read
      - chat:write.customize
      - chat:write.public
      - channels:read
      - commands
      - emoji:read
      - groups:read
      - incoming-webhook
      - users.profile:read
      - users:read
      - users:read.email
      - im:write
      - groups:write
      - mpim:write
      - workflow.steps:execute
settings:
  event_subscriptions:
    request_url: *
    bot_events:
      - workflow_step_execute
  interactivity:
    is_enabled: true
    request_url: *
  org_deploy_enabled: false
  socket_mode_enabled: false
  token_rotation_enabled: false

@Arxing Thanks for sharing the app's Manifest. The settings look great. Another possible cause would be a 3-second timeout. Your whole save function execution might take 3+ seconds for some reason. Can you modify your save method as below?
.save { req, context ->
  app.executorService().submit {
    // Run the following code asynchronously
    val stateValues = req.payload.view.state.values
    val inputs = buildMap {
      put("taskName", WorkflowSteps.stepInput {
        it.value(stateValues["task_name_input"]?.get("task_name")?.value)
      })
    }
    val outputs = buildList {
      add(WorkflowSteps.stepOutput {
        it.name("taskName").type("text").label("Task Name")
      })
    }
    context.update(inputs, outputs)
  }
  // Immediately acknowledge the request
  context.ack()
}

The context.update utility internally performs workflows.* API calls. Those APIs tend to be a bit slow. If the context.update takes a bit longer time in your use case, the approach should help. Also, if the code works for you, you can go with this workaround for production apps too.

I've tried calling Context.update() asynchronously and immediately returning Context.ack(), but still the same result. 🥲

@Arxing Thanks for your response. hmm, I don't have anything further to suggest.
I would suggest trying my example and seeing if there is any difference. Here is my App Manifest:
display_information:
  name: workflow-step-socket-mode-app
features:
  bot_user:
    display_name: workflow-step-socket-mode-app
    always_online: false
  workflow_steps:
    - name: socket-mode-step
      callback_id: socket-mode-step
oauth_config:
  scopes:
    bot:
      - workflow.steps:execute
settings:
  event_subscriptions:
    bot_events:
      - workflow_step_execute
  interactivity:
    is_enabled: true
  org_deploy_enabled: false
  socket_mode_enabled: true
  token_rotation_enabled: false

You can create a new app and then generate a new App-Level token with connections:write scope. You can install the app into your Slack workspace. Once you set SLACK_BOT_TOKEN and SLACK_APP_TOKEN env variables, the app should work for you.
If your issue can arise only with a specific app and/or a workspace, please contact our customer support agents for further help by checking the Slack server-side activity data. You can submit inquiries either by /feedback command in your Slack workspace or at https://my.slack.com/help/requests/new .

Hi @seratch I'm happy to tell you that I solved my problem! the problem is because I used ngrok to develop locally, when I deploy to the remote machine everything works fine!
But I immediately encountered another problem, I successfully added the step I wrote in my workflow, but when I am in workflowStepSave or workflowStepExecute I will receive /slack/challenge request, but there is no challenge in the request body, the request body I received looks like this:
{
  "token": "*",
  "team_id": "*",
  "api_app_id": "*",
  "event": {
    "type": "workflow_step_execute",
    "callback_id": "triggerAction",
    "workflow_step": {
      "workflow_step_execute_id": "3569568604502.2193215711.7d10728bace18d43446a83d38ff5e584",
      "workflow_id": "409291001966899534",
      "workflow_instance_id": "409292316780540782",
      "step_id": "fcef04d0-62e0-4c0d-b4c8-8bf064b42d96",
      "inputs": {
        "action": {
          "value": "v-a"
        }
      },
      "outputs": [
        {
          "name": "name-label",
          "type": "text",
          "label": "label"
        }
      ]
    },
    "event_ts": "1653486831.627975"
  },
  "type": "event_callback",
  "event_id": "Ev03HMU2N2QG",
  "event_time":<PHONE_NUMBER>,
  "authorizations": [
    {
      "enterprise_id": null,
      "team_id": "*",
      "user_id": "*",
      "is_bot": true,
      "is_enterprise_install": false
    }
  ],
  "is_ext_shared_channel": false
}

and that workflowStepExecute is not executed, I don't know where the problem is

@Arxing You might miss the logs but I am sure that url_verification request was sent separately. Let me know if you have anything else that you need our help. Otherwise, we will close this issue in a few business days.

Hi, I am bumping into a similar issue @Arxing pointed out in regards to ngrok and although I could work on a similar manner to deploy to a stage environment, curious to understand better as developing locally would boost productivity drastically.

What could be the reason behind that failure ?
Is there any known workarounds to make it work ?

Thanks

Hi @lhrolim @Arxing were you able to figure out the root cause? And If there's a workaround.


 See https://github.com/microsoft/vscode/issues/178733#issuecomment-1500812938

McAfee Antivirus might be the cause. See #178733 (comment)

Thanks For Your Reply, I tried that but no use.


OpenSSL interop not valid

Pasting this into my linux Mint box command line:
echo "U2FsdGVkX1/D7z2azGmmQELbMNJV/n9T/9j2iBPy2AM=\n" | openssl enc -d -aes-256-cbc -a -k p4ssw0rd
gives an error:
error reading input file

OpenSSL broke the compatibility with older version of OpenSSL. Gibberish 2.0 and beyond will stick with just using SJCL compatible encryption.


Link is [[null]] for all imported Kindle highlights

After importing my highlights from Readwise the Link of all highlights is [[null]]

.

Found the Problem. I manually imported the Kindle Highlights with the "My Clippings.txt". Because the source in Readwise was clipping and not kindle the plugin tried to insert the url instead of the kindle location. I did a local workaround for my  use case that all Highlights with category book are treated as kindle Highlights.


Failed to do graphite POST request error

First of all, thank you for publishing Graph Explorer, it looks pretty neat.
I have a working Graphite system (I can generate & view graphs via the default interface) but when I attempt to view graphs in Graph Explorer I get...
Error: Failed to do graphite POST request to http://nlpanms-1:8085/render: error:.
I read the GE documentation and added the three Header set Access-Control lines to httpd.conf and restarted without errors. I have also gone through the config.py and local settings files and don't see any issues there either. When using the Firefox browser call debugger, I do see the following error in the Network Response tab:
SyntaxError: JSON.parse: unexpected end of data at line 1 column 1 of the JSON data.
Any suggestions for additional troubleshooting? I have included the render request & response below.
/* Client Request*/
POST /render HTTP/1.1
Host: nlpanms-1:8085
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:40.0) Gecko/20100101 Firefox/40.0.3 Waterfox/40.0.3
Accept: application/json, text/javascript, /; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Referer: http://nlpanms-1:8080/index/UpdateTime
Content-Length: 131
Origin: http://nlpanms-1:8080
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
from=-24hours&until=now&target=alias(carbon.agents.nlpanms-1-a.avgUpdateTime,'carbon.agents.nlpanms-1-a.avgUpdateTime')&format=json
/Server response/
HTTP/1.0 200 OK
Date: Fri, 18 Mar 2016 14:31:56 GMT
Server: WSGIServer/0.1 Python/2.6.6
Content-Length: 18093
Content-Encoding: gzip
Expires: Fri, 18 Mar 2016 14:32:56 GMT
Vary: Accept-Encoding
Last-Modified: Fri, 18 Mar 2016 14:31:56 GMT
Cache-Control: max-age=60
Content-Type: application/json
<18093 bytes of compressed response data>
A partial snippet of the content after decompression is...
[{"target": "carbon.agents.nlpanms-1-a.avgUpdateTime", "datapoints": [[6.7542357878251513e-05,<PHONE_NUMBER>], [6.5166841853748665e-05,<PHONE_NUMBER>], [6.6682967272671786e-05,<PHONE_NUMBER>], [7.2549581527709965e-05,<PHONE_NUMBER>], [6.5570419484918763e-05,<PHONE_NUMBER>], [7.0108500393954192e-05,<PHONE_NUMBER>], [6.9727789271961554e-05,<PHONE_NUMBER>], [6.9652145559137514e-05,<PHONE_NUMBER>], [7.8617009249600499e-05,<PHONE_NUMBER>], [7.6642253182151096e-05,<PHONE_NUMBER>], [6.7208030007102269e-05,<PHONE_NUMBER>], [6.6188573837280276e-05,<PHONE_NUMBER>], [6.5523277629505502e-05,<PHONE_NUMBER>], [7.4737722223455262e-05,<PHONE_NUMBER>],
I have looked through lots of log files and I don't see anything obviously bad in them. The graphite httpd log file in particular shows the render requests as...
[18/Mar/2016 09:17:22] "POST /render HTTP/1.1" 200 3566
[18/Mar/2016 09:17:22] "POST /render HTTP/1.1" 200 5432
[18/Mar/2016 09:17:22] "POST /render HTTP/1.1" 200 3562
[18/Mar/2016 09:17:22] "POST /render HTTP/1.1" 200 5434
[18/Mar/2016 09:17:22] "POST /render HTTP/1.1" 200 17587
[18/Mar/2016 09:17:22] "POST /render HTTP/1.1" 200 3555
[18/Mar/2016 09:17:22] "POST /render HTTP/1.1" 200 9332
Thank you for your assistance.

I figured this out. I was still referencing the development (builtin) graphite server by port and not the apache server on a different port. Duh!


will execute the same yaml benchmark on different GPUS ON THE same machine overwrite existing CSVS?

If we run ./run_benchmark.sh ./examples/yaml/demo_benchmark.yaml 0  in the DomainLab repo, then we open another command line session, in the same folder, and run ./run_benchmark.sh ./examples/yaml/demo_benchmark.yaml 1, can we ensure the results for both run will be saved in csv files?
For nonbenchmark run, results are just appended into the csv file, not a problem how about for benchmark?
The reason is we have limited computational resources, by running ./run_benchmark.sh ./examples/yaml/demo_benchmark.yaml SEED  , we want to sample different hyperparameters so that we can plot them together with  python main_out.py --agg_partial_bm OUTPUT_DIR

@smilesun,
Both sessions will try to write inside the same .csv files. As these files are appended with every result this should in general not overwrite the results of the other session.
https://github.com/marrlab/DomainLab/blob/860cb4a28e993124dd8bff63a175078e8902abdf/domainlab/compos/exp/exp_utils.py#L182
However, if two processes try to write inside the same file at the same time the line will get messed up. I also assume we will get the header twice in each file.

A solution for this might be to include the random seed in the name of the .csv files, so each session will get their own .csv files inside the same rule_results folder. From the implementation of the agg_partial_bm I know that it does not matter how many files are created inside the rule_results folder, so I think this would be the most convenient solution to this problem.

@smilesun Shall I try to implement this solution?

sounds good, although the scenario you mentioned has low probability to occur (maybe hight probability) i think it is good to avoid it if the simply solution by adding seed to csv file name could help.

for me an extra scenario in mind is if we exeucte ./run_benchmark.sh ./examples/yaml/demo_benchmark.yaml 0  after it finishes, someone execute ./run_benchmark.sh ./examples/yaml/demo_benchmark.yaml 1, what will happen in this case?   I assume the first finished benchmark will delete already the  individual csv files?

@smilesun

Okay, then I will work on this

I looked back at the benchmarks we did for the project. It seems like the partial results are not deleted after they are aggregated into one file. I also looked inside the code and couln't find any code which seems to delete any of these files.

From this I came to the conclusion that if one executes benchmarks like you descrieb in 2.:

./run_benchmark.sh ./examples/yaml/demo_benchmark.yaml 0 will generate partial results in the rule_results folder and aggregate them into a results.csv file after the benchmark is finished. The partial results are not deleted after the aggregation
the second execution ./run_benchmark.sh ./examples/yaml/demo_benchmark.yaml 1 will then append the existing partial results .csv files in the rule_results folder. Hence they will contain results from both benchmarks. The aggregation in the end is implemented such that it overwrites existing .csv files. Hence the resulting aggregated result will contain all results from both benchmarks.

Regarding the issue with the two headers. It seems like this will not happen, as the header is only created if the file does not exist yet. A header is only created if the following line gives True:
https://github.com/marrlab/DomainLab/blob/860cb4a28e993124dd8bff63a175078e8902abdf/domainlab/compos/exp/exp_utils.py#L123

Seems like the issue with running two sessions in snakemake is, that skemake does lock all files and directories used by the session. Hence they cannot be accessed by another session at the same time
Error: Directory cannot be locked. Please make sure that no other Snakemake process is trying to create the same files in the following directory:
/home/carla/Dokumente/Uni/Master/Computational_Methods_for_Single-Cell_Biology/DomainLab
If you are sure that no other instances of snakemake are running on this directory, the remaining lock was likely caused by a kill signal or a power loss. It can be removed with the --unlock argument.

I still get this issues after renaming the files for the partial results, the hyperparameters, the full results and the graphics (by using the random seed).

@smilesun
Before spending more time in solving this issue I was wondering about the advantage of starting two sessions with one hyperparameter sample per algorithm, over starting just one session with two hyperparameter samples per algorithm.
You wrote:
"The reason is we have limited computational resources, by running ./run_benchmark.sh ./examples/yaml/demo_benchmark.yaml SEED"
do you know where this does come from? Maybe it would be easier to solve this issue than getting snakemake to allow two processes working on the same files.

If this will take a lot of time to resolve, then we should patiently wait until one machine to finish all jobs/experiments.
The issue is if we want to run a large benchmark without a slurm cluster but only one machine with several GPUs, to have result within 1 week instead of 1 month, we might want to unitize all 8 GPUs to let them run different jobs/experiments of the benchmark instead of only letting one GPU to do all the work.
That being said, it sounds like we have chosen the wrong issue? A solution that makes more sense is to investigate how to let snakemake to find all available GPUs and dispatch different jobs of the benchmark into different GPUs so all 8 GPUs on the same machine will be busy?

@smilesun
From what I know, it is possible to specify the number of cores in the run_benchmark.sh file
https://github.com/marrlab/DomainLab/blob/860cb4a28e993124dd8bff63a175078e8902abdf/run_benchmark.sh#L28
right now this is hard coded to be one (--cores 1), but one can adapt this depending on the machine which is used.
Maybe it does make sense to add this as a command line argument.
The only thing to be careful with this is, that even if a machine has a specific number of cores, the process may crash when using all cores if the GPU memory isn't big enough. But this might be the same problem when starting multiple sessions at the same time.

I think the "cores" here refer to the CPU cores, not how many GPUs to use

@smilesun
A second thing I realized when running ./run_benchmark.sh ./examples/yaml/demo_benchmark.yaml 0 and ./run_benchmark.sh ./examples/yaml/demo_benchmark.yaml 1:
It seems like both sample the same hyper parameters, because they use the same random seed, which does again explain, why the snakemake sessions do access the same files (after adding the random seed in the file name).
I did find that that the number in the end of the command line arguments do set the DOMAINLAB_CUDA_START_SEED
https://github.com/marrlab/DomainLab/blob/860cb4a28e993124dd8bff63a175078e8902abdf/run_benchmark.sh#L16
This variable is, however, not passed to the snakemake session. In the snakemake session the random seed specified in the benchmark yaml is used.
What is the goal of using the export DOMAINLAB_CUDA_START_SEED=$2?

because we have

https://github.com/marrlab/DomainLab/blob/860cb4a28e993124dd8bff63a175078e8902abdf/domainlab/exp_protocol/benchmark.smk#L19
https://github.com/marrlab/DomainLab/blob/860cb4a28e993124dd8bff63a175078e8902abdf/domainlab/exp_protocol/benchmark.smk#L65
then we could use different random seed on different machines

because we have
1. https://github.com/marrlab/DomainLab/blob/860cb4a28e993124dd8bff63a175078e8902abdf/domainlab/exp_protocol/benchmark.smk#L19

2. https://github.com/marrlab/DomainLab/blob/860cb4a28e993124dd8bff63a175078e8902abdf/domainlab/exp_protocol/benchmark.smk#L65

   then we could use different random seed on different machines

I see, I think I misunderstood the usage of DOMAINLAB_CUDA_START_SEED, it does control the stochastic variation and not the systematic variation.

Ah, OK, I guess then we need to set a different random seed with corresponding environment variable to sample different hyper-parameters?  If we sample different hyper-parameters, then it will be possible to run two different sessions in side the same folder?

Ah, OK, I guess then we need to set a different random seed with corresponding environment variable to sample different hyper-parameters? If we sample different hyper-parameters, then it will be possible to run two different sessions in side the same folder?

I think we are still phasing the same issue then. Both session will need to work on completly different files. Hence we cannot have one results.csv file after the sessions finish.
I would rather investigate wheather it is possible to use multiple GPUs with one session.
Looking into the snakemake documentation it seam prommising to specify the number of GPUs in the rule. e.g.:
input:
        # path to config file as input, thus a full
        # rerun is considered whenever the config yaml changed.
        expand("{path}", path=config_path)
    output:
        dest=expand("{output_dir}/hyperparameters_" + str(os.environ["DOMAINLAB_CUDA_START_SEED"]) + ".csv",
            output_dir=config["output_dir"])
    resources:
        gpu=1
    run:
        from domainlab.utils.hyperparameter_sampling import sample_hyperparameters
        sample_hyperparameters(config, str(output.dest))

https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#gpu-resources

then i guess we can create a branch with
nvidia_gpu=3  added in the smk file and let our collaborator who has a 8 GPU machine to test if this works?

Sounds good.
I was thinking about adding the number of GPUs as a comand line argument or is it better to put it directly inside the smk file?

if we can do it more general , sure, it will be better and we can even merge it to master, and then let our collaborator test it.

I added the resources to all jobs. I tested the code (with the limited resources of my Laptop), so it does not crash.
The code can be found on the branch cf_multiple_GPU. One can execute a benchmark yaml with
./run_benchmark.sh <CONFIGFILE> <DOMAINLAB_CUDA_START_SEED> <NUMBER_GPUS>

I did use CONFIGFILE = ./examples/yaml/demo_benchmark.yaml as this is a very small benchmark. But maybe, when using multiple GPUs it is better to modify the yaml to use more epos and more hyper parameter samples.

By the way, do we still whant to use the DOMAINLAB_CUDA_START_SEED? It can be confusing because it is also set in the benchmark yaml file

Do you mean we have start_seed and end_seed already in the YAML file ?
If DOMAINLAB_CUDA_SEED is not empty, the start_seed will be overwritten and
end seed will be start seed plus the difference in the YAML file.
It is still designed in this way because we want to let different machines
to run the same benchmark with different random seed.
Does that make sense?
On Fri, 7 Apr 2023 at 16:37, Car-la-F @.***> wrote:

By the way, do we still whant to use the DOMAINLAB_CUDA_START_SEED? It can
be confusing because it is also set in the benchmark yaml file
—
Reply to this email directly, view it on GitHub
https://github.com/marrlab/DomainLab/issues/186#issuecomment-1500347581,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAELKRWIOLM7IA32V5TAKBLXAARDZANCNFSM6AAAAAAWUHJFCI
.
You are receiving this because you were mentioned.Message ID:
@.***>

Okay, that makes sense to me

in this commit, i have tested different seed lead to different hyper-parameters, but if i run the same benchmark with two different hyper-parameters, the old results get over-written by the new benchmark
90e44b2
./run_benchmark.sh examples/yaml/demo_benchmark.yaml 10 101
./run_benchmark.sh examples/yaml/demo_benchmark.yaml 10 102

the seed with 102 overwrite the one with 101

Yes, because they are saved in the same output directory. I think the whole pipeline is not meant to run the same .yaml file multiple times on the same machine.
The idea of allowing multiple GPU work on the same benchmark is also to not be forced run a benchmark multiple times with different seeds.
Maybe (to prevent future confusion) we could add a time stamp in the end of the name of the output file. So none of the already existing results can be overwritten?

in this commit, i have tested different seed lead to different hyper-parameters, but if i run the same benchmark with two different hyper-parameters, the old results get over-written by the new benchmark 90e44b2
./run_benchmark.sh examples/yaml/demo_benchmark.yaml 10 101
./run_benchmark.sh examples/yaml/demo_benchmark.yaml 10 102

the seed with 102 overwrite the one with 101


ENH: Improve ignoring Makefile build directories.

When running the Makefile build, files are placed in the source
directory that should be ignored.

@LogicalKnight Great work.  Thanks!

@wjakob This should be ready to merge.

These .gitignore files look highly specific to your setup (emacs? build/*_debug/?) Why does this need to go into master?

@wjakob
The build/_debug  &  build/_release/  are files that are created when testing the default Makefile in the source tree.  It is to avoid accidently including unnecessary binary files into a git commit.
The exclusions are to remove unnecessary files when using a shared mount location or disk that does not support mac file meta-data.
If you feel that having these makes managing the build more difficult, then I will discard this patch set.

I hope that nobody uses the Makefile build with this project (that's the sort of the point ;)). Altogether this doesn't strike me as important enough (If we start adding editor specific files, then that's a rabbit hole that I don't want to open)

@wjakob OK.


Allow chating with MemGPT agents from Slack like chat apps

Allow the possibility to chat with MemGPT agents from an open source chat app (Slack like chat apps).
This will open a great deal of possibilities including the option to chat with MemGPT agents on the
go from android and iphone mobile phones (these chat apps have clients for android/iphone)
or even the possibility to "speak" with the MemGPT agents using the mobile phone.
The are many great open source chat apps like Element (matrix client), Mattermost or RocketChat
https://github.com/mattermost/mattermost
https://github.com/RocketChat/Rocket.Chat
https://www.matrix.org
https://github.com/matrix-org
Matrix wants to be like a common/reference API/implementation in the world of so many different chat apps.
Here are some projects which can be used as example
https://github.com/matrixgpt/matrix-chatgpt-bot
Talk to ChatGPT via any Matrix client!
https://github.com/alphapapa/ement.el
A Matrix client for GNU Emacs
https://github.com/yGuy/chatgpt-mattermost-bot
A very simple implementation of a service for a mattermost bot that uses ChatGPT
in the backend
https://github.com/attzonko/mmpy_bot
A python-based chatbot for Mattermost (http://www.mattermost.org)

For instance
https://etke.cc/help/bots/chatgpt/

This should be possible with MemGPT now -- you just have to have the application send messages to a MemGPT agent running on a server, and either replace or modify the send_message tool in MemGPT to instead send a message to the application you want the message to appear on.
@cpacker recently made a demo for how to connect a MemGPT agent to chat via a tamagotchi front-end, so that might be a helpful reference once its released (hopefully soon).

I think the question here is , could the messages endpoint implement the OpenAI chat parameters? That would allow all the available chat clients/packages out there to work out of the box.

Here
https://docs.letta.com/introduction#letta-api
is says
Letta API -> Stateful ChatCompletions API: an API designed to be a drop-in replacement for the standard ChatCompletions API
From this statement I understand that currently any Letta agent can be used as a "a drop-in replacement" in any of the existing chat clients/frontend which support the ChatCompletions API (including web apps which support ChatCompletions, Slack, Matrix or other chat bots which support the same, VS Code extensions/assistants which can connect to any OpenAI endpoint or Emacs clients which support the standard ChatCompletions API format)
Is my understanding correct?

@distributev exactly - that's the high-level idea. Basically, many frontends / client applications have been written exclusively to support the /v1/chat/completions API (e.g. voice services like VAPI or LiveKit).
That API spec is stateless (meaning there's no memory management, so the client has to pass in the full agent state at each step), unlike Letta's API which is stateful (the server maintains the agent state, and the client only has to pass in a single message).
However, it's possible to connect a Letta server to services that expect a stateless API, and still get the power of a stateful API - you basically "pretend" like you're exposing a /v1/chat/completions API but under the hood make the API stateful. This allows you to connect something like LiveKit to a Letta server to communicate with a stateful Letta agent using voice.
We're currently working on this feature and hope for it to be released in the next few weeks (possibly sooner).

"hope for it to be released in the next few weeks (possibly sooner)"
Are you referring to the LiveKit integration or the ChatCompletions API format standardization itself?


Metadata dynamic IPFS source unit discovery

Until now, if URLs were present in IPFS metadata and no content, we did not look into IPFS to fetch actual source code of the contract. Now we do.
Important note here, it can be slow and result in request timeout as IPFS node does not contain any information under provided CID.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

5 of 20 (25.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.1%) to 75.941%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

metadata/ipfs.go
5
20
25.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.1%

Covered Lines:
5950

Relevant Lines:
7835

💛  - Coveralls


Set the front of the covarience to -1 to express UNUSED

Now the bare topic is like this.
And check the lines with # Need to be -1
header:
  stamp:
    sec:<PHONE_NUMBER>
    nanosec: 746524577
  frame_id: mgs1600gy_link
orientation:
  x: -0.25434098888304396
  y: 0.4801277513756575
  z: 0.036559869881750084
  w: 0.8387200842040088
orientation_covariance:
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
angular_velocity:
  x: 0.0
  y: 0.0
  z: 0.0
angular_velocity_covariance:
- 0.0 # Need to be -1
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
linear_acceleration:
  x: 0.0
  y: 0.0
  z: 0.0
linear_acceleration_covariance:
- 0.0 # Need to be -1
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0

Requirements

MGS1600gy

Start program
ros2 launch mgs1600gy_bringup mgs1600gy.launch.py dev:=/dev/ttyACM0

https://github.com/HarvestX/unity_simulation/issues/84

Now this is the example of topic data
  stamp:
    sec:<PHONE_NUMBER>
    nanosec: 630632586
  frame_id: mgs1600gy_link
orientation:
  x: 0.0059635725420759175
  y: -0.0037329223807611635
  z: 0.04015210575624991
  w: 0.9991688093092044
orientation_covariance:
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
angular_velocity:
  x: 0.0
  y: 0.0
  z: 0.0
angular_velocity_covariance:
- -1.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
linear_acceleration:
  x: 0.0
  y: 0.0
  z: 0.0
linear_acceleration_covariance:
- -1.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0

The error as follow happened. Should I solve it with this issue?
@m12watanabe1a

↑ This error was solved by reloading setup.bash. Sorry.