Handle reorgs

We should handle block reorgs when managing our owned outputs. Nakamoto has some built-in stuff for tracking scripts across reorgs (https://docs.rs/nakamoto-client/0.4.0/nakamoto_client/handle/trait.Handle.html#method.watch), maybe we could use that. However, we also need to make sure that we reset the 'scan' progress when blocks get reorged.

Hey, can i work on this .

Hey, can i work on this .

Extra help would be very much appreciated!
The description of this issue is very outdated. There's basically 2 things that need to happen I think:

We need some way to ask blindbit-oracle if we are currently on the correct chain and if not, how many blocks ago a chain split occurred
If a chain split occurred on block height n, we should delete all the data after block height n, reset the scan height to n, and scan to the new chain tip

I think (1) is quite difficult requires some help from the blindbit developer (setavenger), but we could probably already do (2) by adding a trait method async fn rescan_from_block_height(n: Height) or something.
So if you want to work on it, maybe you could start with (2)?

I also added a similar issue to the Dana wallet repository (a wallet we're working on that consumes sp-client): https://github.com/cygnet3/danawallet/issues/80.

Sure,  I would get started on working on (2), would also check out Dana Wallet


Add benchmark extrinsic command

Makes it possible to benchmark the execution speed of an extrinsic. It does so by filling a block with these extrinsics and measures its execution time.
The results is then divided by the number of extrinsics and printed to console.
Example invocation:
$ substrate benchmark extrinsic --pallet system --extrinsic remark --dev

Creating empty BABE epoch changes on what appears to be first startup.
Building block, this takes some time...
Extrinsics per block: 12000
Running 10 warmups...
Executing block 100 times
Executing a SYSTEM::REMARK extrinsic takes[ns]:
Total: 6897382
Min: 68685, Max: 69797
Average: 68973, Median: 68899, Stddev: 188.99
Percentiles 99th, 95th, 75th: 69293, 69230, 69122

Changes:

Add a benchmark extrinsic <pallet> <extrinsic> command
Implement the command for System::Remark and Balances::TransferKeepAlive
Modify benchmark overhead to re-use the logic
Update node+template to integrate the command

WIP:

Document that it is non-endowing, non-reaping and currently always the same sender and receiver account
[x] Print all available pallets and extrinsics in the help menu
Do some more lenient input transformation (eg. lower-case, trim - and _)

Polkadot companion: https://github.com/paritytech/polkadot/pull/5620
Cumulus companion: https://github.com/paritytech/cumulus/pull/1350

bot merge


Started getting crash on ios after updating to latest version.

Describe the bug
Started getting more crashes in ios after updating to latest version v1.2.3.
To Reproduce
I am not able to regenerate and not able to figure out why it is happening. I had figure out a crash from firebase.
Environment (please complete the following information):
react-native-info
  React Native Environment Info:
    System:
      OS: macOS 10.15.4
      CPU: (8) x64 Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz
      Memory: 609.34 MB / 16.00 GB
      Shell: 3.2.57 - /bin/bash
    Binaries:
      Node: 8.11.3 - ~/.nvm/versions/node/v8.11.3/bin/node
      Yarn: 1.12.3 - /usr/local/bin/yarn
      npm: 5.6.0 - ~/.nvm/versions/node/v8.11.3/bin/npm
      Watchman: 4.9.0 - /usr/local/bin/watchman
    SDKs:
      iOS SDK:
        Platforms: iOS 13.4, DriverKit 19.0, macOS 10.15, tvOS 13.4, watchOS 6.2
      Android SDK:
        API Levels: 21, 22, 23, 24, 25, 26, 27, 28, 29
        Build Tools: 23.0.1, 23.0.2, 23.0.3, 24.0.2, 25.0.0, 25.0.1, 25.0.2, 25.0.3, 26.0.1, 26.0.2, 26.0.3, 27.0.3, 28.0.0, 28.0.2, 28.0.3, 29.0.2
        System Images: android-22 | Google APIs Intel x86 Atom, android-27 | Google APIs Intel x86 Atom, android-28 | Google APIs Intel x86 Atom, android-29 | Google APIs Intel x86 Atom, android-R | Google APIs Intel x86 Atom
    IDEs:
      Android Studio: 3.6 AI-192.7<IP_ADDRESS>92135
      Xcode: 11.4/11E146 - /usr/bin/xcodebuild
    npmPackages:
      react: 16.8.3 => 16.8.3
      react-native: 0.59.4 => 0.59.4
    npmGlobalPackages:
      react-native-asset: 1.1.3
      react-native-cli: 2.0.1
      react-native-onesignal: 3.5.0
      react-native-rename: 2.4.1
      react-native-svg: 11.0.1

react-native-track-player version v1.2.3
The issue was occurring on actual devices. Here is the error stack from firebase

Does anyone have an idea why it is happening or how to regenerate it?

I'm not 100% sure this is the same issue, but there has been recent work to address issues with certain URL formats.
https://github.com/react-native-kit/react-native-track-player/commit/5efe26cd5b46c1c91c18ffaa09fe12dc7299e3ea
In certain cases this will fix some crashes that were previously possible.
This has not been included in a release yet.

See https://github.com/react-native-kit/react-native-track-player/pull/950 for more info.

@curiousdustin : Thanks for the update and providing direction. I will do changes in mediaURL file and make fix as mention in 5efe26c. And release new version of my app and see it reduce crashes in next couple of week or not.
I will get back to you with a result.
Thanks,

@jariwalabhavesh - Did you end up solving this? I'm facing the same issue. See logs below
Thread 7 name:
Thread 7 Crashed:
0   A Life Lived                  	0x000000010315492c MediaURL.init(object:) + 1512 (MediaURL.swift:28)
1   A Life Lived                  	0x0000000103154794 MediaURL.init(object:) + 1104 (MediaURL.swift:28)
2   A Life Lived                  	0x000000010316d65c Track.init(dictionary:) + 668 (Track.swift:38)
3   A Life Lived                  	0x0000000103162038 Track.__allocating_init(dictionary:) + 8 (<compiler-generated>:0)
4   A Life Lived                  	0x0000000103162038 RNTrackPlayer.add(trackDicts:before:resolve:reject:) + 688
5   A Life Lived                  	0x0000000103162038 0x102f4c000 + 2187320 (RNTrackPlayer.swift:290)
6   A Life Lived                  	0x0000000103162be0 @objc RNTrackPlayer.add(trackDicts:before:resolve:reject:) + 220 (<compiler-generated>:0)
7   CoreFoundation                	0x00000001ad5235d4 __invoking___ + 148
8   CoreFoundation                	0x00000001ad3fe9e8 -[NSInvocation invoke] + 448 (NSForwarding.m:3389)
9   CoreFoundation                	0x00000001ad3fefa4 -[NSInvocation invokeWithTarget:] + 80 (NSForwarding.m:3495)
10  A Life Lived                  	0x0000000103057670 -[RCTModuleMethod invokeWithBridge:module:arguments:] + 460 (RCTModuleMethod.mm:569)
11  A Life Lived                  	0x000000010305975c facebook::react::invokeInner(RCTBridge*, RCTModuleData*, unsigned int, folly::dynamic const&) + 244 (RCTNativeModule.mm:108)
12  A Life Lived                  	0x00000001030594c0 facebook::react::RCTNativeModule::invoke(unsigned int, folly::dynamic&&, int)::$_0::operator()() const + 44 (RCTNativeModule.mm:73)
13  A Life Lived                  	0x00000001030594c0 invocation function for block in facebook::react::RCTNativeModule::invoke(unsigned int, folly::dynamic&&, int) + 88 (RCTNativeModule.mm:65)
14  libdispatch.dylib             	0x00000001ad11afd0 _dispatch_call_block_and_release + 32 (init.c:1454)
15  libdispatch.dylib             	0x00000001ad11cac8 _dispatch_client_callout + 20 (object.m:559)
16  libdispatch.dylib             	0x00000001ad123c08 _dispatch_lane_serial_drain + 580 (inline_internal.h:2548)
17  libdispatch.dylib             	0x00000001ad124734 _dispatch_lane_invoke + 408 (queue.c:3862)
18  libdispatch.dylib             	0x00000001ad12e528 _dispatch_workloop_worker_thread + 708 (queue.c:6590)
19  libsystem_pthread.dylib       	0x00000001f4a70908 _pthread_wqthread + 276 (pthread.c:2193)
20  libsystem_pthread.dylib       	0x00000001f4a7777c start_wqthread + 8


ssh-banner/v4: add tests

Previous PR: #643
Changes:

Added README for each test

Link to redmine ticket: https://redmine.openinfosecfoundation.org/issues/4903

Followed By: #730


add async Stream getStream() method and switch command enum test to switch/case

This adds a new method, getStream() that returns a Stream<E> that will iterate on values as they are produced by a closure running on the background agent/isolate.  If the stream throws an exception in the background agent, the exception is captured and set in the agent.error's e and stacktrace variables, and the iteration terminates.
Multiple async streams can be iterated on simultaneously, and the streams can produce any type <E>, not only the state's <T> type.
I think I covered the basics in the new tests, including one test that confirms values are being produced "live" using a stopwatch to measure the delay between values from a Stream<_>.periodic(100 milliseconds) to ensure it's actually generating values live from the agent back to the parent, not just collecting everything at once.  :grin:
I'm new to Dart (and flutter) and I'm exploring the different packages for managing long-lived isolates for my first app.  Thanks for this package, it's nicely done. I hope this PR is useful.  Let me know if you'd like any changes
in this unsolicited PR..

What would you think about a signature like: Stream<E> getStream<E>(Stream<E> Function(T) func) async* where the implementation invokes the function by passing in the state, like Stream<dynamic> stream = func(state); ?

Someone could easily do that themselves today with Dart async generators.  There's no need to edit the Agents for that =)
Stream<int> streamInc(Agent<int> agent) async* {
  while (true) {
    agent.update((x) => x + 1);
    yield await agent.read();
  }
}

I think this PR is close to what I was looking for:
I like the agents style of abstracting away the nastines of Isolates. However, I was wondering how to pass progress messages from the "single run isolate func" back to caller, and this stream would probably fit the bill nicely. In my case the single run isolate func is a long running file format conversion code. I'm not sure the code below would fit the bill, since the progress would have to be "sent" from within the long running update()

Stream<int> streamInc(Agent<int> agent) async* {
  while (true) {
    agent.update((x) => x + 1);
    yield await agent.read();
  }
}

Since I'm quite a newbie when it comes to flutter async streams with isolates, I'm not 100% sure this PR fits the bill either, so any advice?

In my case the single run isolate func is a long running file format conversion code. I'm not sure the code below would fit the bill, since the progress would have to be "sent" from within the long running update()

You could have the state for the agent be the number representing the progress.  In the UI you can have an animation that is polling that state.  I think that would be ideal.  Otherwise, I think you might need to send the agent a stream to communicate back to the main isolate.


Fixed typo in the file name

Just a small name change to the file.

Bruh i was 1s too late... i fu-


Remove aggregate advice from archives configuration.

This ensure it doesn't get hooked into the assemble task, which is bad news.
Finally resolves the issue whereby users executing assemble (or some variants thereof) would also have this plugin's artifacts assembled, which was normally not desirable. Per Gradle, this current automagic behavior is deprecated and will eventually be removed (along with the old "archives" configuration).

Build failed


error downloading gnomad files from ensembl

There is an issue with downloading gnomad data:
in ggd-run.sh
url_prefix=http://ftp.ensemblorg.ebi.ac.uk/pub/data_files/homo_sapiens/GRCh37/variation_genotype/gnomad/r2.1/genomes/
the correct location of the file is:
http://ftp.ensembl.org/pub/data_files/homo_sapiens/GRCh37/variation_genotype/gnomad/r2.1/genomes/
Version info

bcbio version (bcbio_nextgen.py --version):
OS name and version: 1.2.0

To Reproduce

Exact bcbio command you have used
Your sample configuration file
bcbio_nextgen.py upgrade --datatarget gnomad

Observed behavior
Error message or bcbio output.
bcbio_nextgen.py upgrade --datatarget gnomad
Upgrading bcbio
Upgrading bcbio-nextgen data files
--2020-03-20 13:25:38--  https://github.com/chapmanb/cloudbiolinux/archive/master.tar.gz
Resolving github.com (github.com)... <IP_ADDRESS>
Connecting to github.com (github.com)|<IP_ADDRESS>|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/chapmanb/cloudbiolinux/tar.gz/master [following]
--2020-03-20 13:25:38--  https://codeload.github.com/chapmanb/cloudbiolinux/tar.gz/master
Resolving codeload.github.com (codeload.github.com)... <IP_ADDRESS>
Connecting to codeload.github.com (codeload.github.com)|<IP_ADDRESS>|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: ‘STDOUT’
 0K ........ ........ ........ ........ ........ ........ 1.14M

3072K ........ ........ ........ .......                     829K=5.0s
2020-03-20 13:25:43 (1005 KB/s) - written to stdout [5189687]
List of genomes to get (from the config file at '{'genomes': [{'dbkey': 'GRCh37', 'name': 'Human (GRCh37)', 'indexes': ['seq', 'twobit'], 'annotations': ['gnomad', 'GA4GH_problem_regions', 'capture_regions', 'MIG', 'prioritize', 'dbsnp', 'hapmap', '1000g_omni_snps', 'ACMG56_genes', '1000g_snps', 'mills_indels', 'clinvar', 'cosmic', 'ancestral', 'qsignature', 'genesplicer', 'effects_transcripts', 'varpon', 'vcfanno', 'viral', 'transcripts', 'RADAR', 'fusion-blacklist', 'mirbase'], 'validation': ['giab-NA12878', 'giab-NA24385', 'giab-NA24631', 'dream-syn3', 'dream-syn4', 'giab-NA12878-NA24385-somatic', 'giab-NA24143', 'giab-NA24149', 'giab-NA24694', 'giab-NA24695']}], 'genome_indexes': ['bwa', 'star', 'rtg'], 'install_liftover': False, 'install_uniref': False}'): Human (GRCh37)
Running GGD recipe: GRCh37 gnomad 2.1
--2020-03-20 13:25:45--  https://gist.githubusercontent.com/naumenko-sa/d20db928b915a87bba4012ba1b89d924/raw/cf343b105cb3347e966cc95d049e364528c86880/gnomad_fields_to_keep.txt
Resolving gist.githubusercontent.com (gist.githubusercontent.com)... <IP_ADDRESS>
Connecting to gist.githubusercontent.com (gist.githubusercontent.com)|<IP_ADDRESS>|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1938 (1.9K) [text/plain]
Saving to: ‘gnomad_fields_to_keep.txt’
gnomad_fields_to_keep.txt            100%[====================================================================>]   1.89K  --.-KB/s    in 0.001s
2020-03-20 13:25:45 (2.16 MB/s) - ‘gnomad_fields_to_keep.txt’ saved [1938/1938]
--2020-03-20 13:25:45--  http://ftp.ensemblorg.ebi.ac.uk/pub/data_files/homo_sapiens/GRCh37/variation_genotype/gnomad/r2.1/genomes/gnomad.genomes.r2.1.sites.chr1_noVEP.vcf.gz
Resolving ftp.ensemblorg.ebi.ac.uk (ftp.ensemblorg.ebi.ac.uk)... failed: Name or service not known.
wget: unable to resolve host address ‘ftp.ensemblorg.ebi.ac.uk’
Traceback (most recent call last):
File "/projects/xxxxxx/bcbio/1.2.0/tools/bin/bcbio_nextgen.py", line 228, in
install.upgrade_bcbio(kwargs["args"])
File "/projects/xxxxxx/bcbio/1.2.0/bcbio/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 107, in upgrade_bcbio
upgrade_bcbio_data(args, REMOTES)
File "/projects/xxxxxx/bcbio/1.2.0/bcbio/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 370, in upgrade_bcbio_data
args.cores, ["ggd", "s3", "raw"])
File "/lustre5/xxxxxx/bcbio/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 354, in install_data_local
_prep_genomes(env, genomes, genome_indexes, ready_approaches, data_filedir)
File "/lustre5/xxxxxx/bcbio/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 480, in _prep_genomes
retrieve_fn(env, manager, gid, idx)
File "/lustre5/xxxxxx/bcbio/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 875, in _install_with_ggd
ggd.install_recipe(os.getcwd(), env.system_install, recipe_file, gid)
File "/lustre5/0/xxxxxx/bcbio/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/ggd.py", line 30, in install_recipe
recipe["recipe"]["full"]["recipe_type"], system_install)
File "/lustre5/xxxxxx/bcbio/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/ggd.py", line 62, in _run_recipe
subprocess.check_output(["bash", run_file])
File "/projects/xxxxxx/bcbio/1.2.0/bcbio/anaconda/lib/python3.6/subprocess.py", line 356, in check_output
**kwargs).stdout
File "/projects/xxxxxx/bcbio/1.2.0/bcbio/anaconda/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['bash', '/lustre5/xxxxxx/bcbio/1.2.0/bcbio/genomes/Hsapiens/GRCh37/txtmp/ggd-run.sh']' returned non-zero exit status 4.
Expected behavior
download and install of gnomad files
Log files
Please attach (10MB max): bcbio-nextgen.log, bcbio-nextgen-commands.log, and bcbio-nextgen-debug.log.
Additional context
Add any other context about the problem here.

Hi @vdejager!
Thanks for reporting!
I've fixed it - please try again.
We probably also need to update this recipe to use a single vcf from Broad instead of chr vcfs from Ensembl:
https://storage.googleapis.com/gnomad-public/release/2.1.1/vcf/exomes/gnomad.exomes.r2.1.1.sites.vcf.bgz
Sergey

Also note Ensembl is currently moving datacenters, which hampers performance for all services.
See blog.ensembl.org.
Cheers
M

Seems to work stable now, thanks

today I got the same error trying to install the development branch on a fresh machine.
the error is still in https://github.com/chapmanb/cloudbiolinux/blob/f0eb952fd9301a73e88e55648357dbf2e0a970ea/ggd-recipes/hg19/gnomad.yaml#L23

Hi @vdejager!
I've fixed for GRCh37 not hg19, that is why.
We have to fix hg19 as well, but for now the fastest option for you is to use GRCh37.
Would grch37  be an option for you - it is basically the same reference?
Sergey

I was using GRCh37 indeed, but for downstream analysis my colleague requested hg19, so I thought to give it a try on the new infrastructure. I've reverted to GRCh37.

Hey @vdejager !
Sorry for the delay - it took so long :)
I've fixed the recipe, please check out:
https://github.com/chapmanb/cloudbiolinux/pull/362
Some tips:
https://github.com/chapmanb/cloudbiolinux/blob/master/doc/hacking.md#testing-a-ggd-recipe
SN

Thanks, I will test them this weekend.

Please let us know if the fix is not working for you!


[bug] file permissions change during transfer

Ive got an anoying bug on linux where nitroshare breaks the file permissions during a transfer.
It should not change them...

Due to the fact that NitroShare runs on so many different platforms, some file attributes (such as file permissions) may not be transferred correctly. The current implementation (in master) attempts to preserve the read-only and executable attributes as well as the creation / modification timestamps. I can't remember off the top of my head whether all of that was included in 0.3.4 (the latest stable release).


Bumped python-multipart to >=0.0.7

Description

Types of Changes

[x] Core
[x] Bugfix
[ ] New feature
[ ] Enhancement/optimization
[ ] Documentation

Issues Fixed or Closed by This PR

Checklist

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] I have tested the changes and verified that they work and don't break anything (as well as I can manage).

Thanks for adding the RELEASE.md file!

Here's a preview of the changelog:

Fix python-multipart version. There is a new 0.0.7 version to fix a Re-DoS bug
Here's the tweet text:
🆕 Release (next) is out! Thanks to @Seetku for the PR 👏

Get it here 👉 https://beta.strawberry.rocks/release/(next)

I've took the opportunity to update ruff and the config, since it was failing 😊


Fix redeclare method

#13

@stepanenko3 thanks for the PR. I merged the previous one (#12) which also fixes this issue, but again, thanks for your work!


multi-valued PACER referral strings not captured

In everyone's favorite case of the hour:

Waymo LLC v. Uber Technologies, Inc. et al

Assigned to: Hon. William Alsup

Referred to: Magistrate Judge Jacqueline Scott Corley

                   Magistrate Judge Elizabeth D. Laporte (Settlement)

this is parsed as
pb3:juriscraper jhawk$  PYTHONPATH=`pwd` juriscraper/pacerdocket.py  juriscraper/pacer/appellate/02.06.03\ CAND-ECF.html 2>/dev/null  |tail -5
  "jury_demand": "Both",
  "nature_of_suit": "830 Patent",
  "parties": [],
  "referred_to_str": "Jacqueline Scott Corley"
}
pb3:juriscraper jhawk$

Omitting Judge Laporte.

Oh no. Cases can have multiple referring judges? That's....cruel. Aside from fixing this in the parser, we'll need to update CourtListener to handle more than one judge.


Non-proper handling of Content-Length and Transfer-Encoding: chunked headers

Expected behavior
1.Only accept one Content-Length.RFC 7230 says duplicate Content-Length header fields have been generated or combined by an upstream message processor, then the recipient MUST either reject the message as invalid or replace the duplicated field-values with a single valid Content-Length.
2.Only accept identity and chunked Transport-Encoding
In this implementation, the order does not matter (it probably should). The Go implementation only uses the first value of the header.Seems to be in sync with the behaviour of AWS ALB. All other valid (gzip, compress, etc.) and invalid TE will return a 501, since we don't have readers for them I figured this was the right move, but feel free to correct me
Actual behavior

But netty accept all.
2.Netty accpet random TE.

Steps to reproduce
Use two CL to reproduce the first.
Use a chunked TE header and a random TE header.
Smiliar with 9571. It also cause http smuggling. Or see the other issue benoitc/gunicorn#2176 and the PR benoitc/gunicorn#2181
Minimal yet complete reproducer code (or URL to code)
Netty version
all
JVM version (e.g. java -version)
OS version (e.g. uname -a)

@ZeddYu I fixed 1) https://github.com/netty/netty/issues/9861 but I am not sure about 2) as netty is just a library so I think the user should verify the transfer-encoding as it also depends on what handlers are in the pipeline.

@normanmaurer But I test it with following code:
            HttpHeaders headers = request.headers();
            if (!headers.isEmpty()) {
                for (Map.Entry<String, String> h: headers) {
                    CharSequence key = h.getKey();
                    CharSequence value = h.getValue();
                    buf.append("HEADER: ").append(key).append(" = ").append(value).append("\r\n");
                }
                buf.append("\r\n");
            }

Use this request headerto test:
Transfer-Encoding: chunked
Transfer-Encoding: some

Only get one TE header is 'some'.
And for the 1), why I use two CL header to test is still accepted?

@ZeddYu sorry I don't understand your question... Can you please try to rephrase ? As shown in the test-case of #9865 multiple Content-Length headers are not allowed anymore .

@normanmaurer All right. I paste an img.

Using two CL header is still accepted. Netty version is 4.1.43 Final.

@ZeddYu sure as this is not merged yet and will be part of netty 4.1.44.Final.

@normanmaurer All right. But it seems it has got another question.

You see. It allows CRLF after the colon. But RFC7230 says not only allow CRLF + 1*(SP/HTAB).
And for the 2) and this , I am sure it will cause problems.

Here it is, the obs-fold.
header-field   = field-name ":" OWS field-value OWS

     field-name     = token
     field-value    = *( field-content / obs-fold )
     field-content  = field-vchar [ 1*( SP / HTAB ) field-vchar ]
     field-vchar    = VCHAR / obs-text

     obs-fold       = CRLF 1*( SP / HTAB )
                    ; obsolete line folding
                    ; see Section 3.2.4

According to https://tools.ietf.org/html/rfc7230#section-3.3.2.

@ZeddYu If you think there is an other issue related to multi-line headers please open another issue. IHMO this is not related to this one.

@normanmaurer What about the 2)?

@ZeddYu again I am not sure why we should do anything about 2) as what is supported depends on what the user is doing with the pipeline etc. Netty is just a library and not a server implementation itself. For example http_parser is not doing anything about 2) as well.

@normanmaurer But just like

@normanmaurer But I test it with following code:
            HttpHeaders headers = request.headers();
            if (!headers.isEmpty()) {
                for (Map.Entry<String, String> h: headers) {
                    CharSequence key = h.getKey();
                    CharSequence value = h.getValue();
                    buf.append("HEADER: ").append(key).append(" = ").append(value).append("\r\n");
                }
                buf.append("\r\n");
            }

Use this request headerto test:
Transfer-Encoding: chunked
Transfer-Encoding: some

Only get one TE header is 'some'.

@normanmaurer But just like what I said just now. I can't seem to get the first TE header. I use the code which is from  https://netty.io/4.1/xref/io/netty/example/http/snoop/HttpSnoopServer.html.

@ZeddYu please can you provide a few reproducer I can run... This back and forth just eats up a lot of time on both of our ends.

Sorry for that.
Application.class:
public class Application {

    public static void main(String[] args) throws Exception{
        HttpServer server = new HttpServer(8888);// 8081为启动端口
        server.start();
    }
}

HttpServer.class:
import io.netty.bootstrap.ServerBootstrap;
import io.netty.channel.ChannelFuture;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.handler.logging.LogLevel;
import io.netty.handler.logging.LoggingHandler;

import java.net.InetSocketAddress;

/**
 * netty server
 * 2018/11/1.
 */
public class HttpServer {

    int port = 8888;

    public HttpServer(int port){
        this.port = port;
    }

    public void start() throws Exception{
        ServerBootstrap bootstrap = new ServerBootstrap();
        EventLoopGroup boss = new NioEventLoopGroup();
        EventLoopGroup work = new NioEventLoopGroup();
        bootstrap.group(boss,work)
                .handler(new LoggingHandler(LogLevel.DEBUG))
                .channel(NioServerSocketChannel.class)
                .childHandler(new HttpServerInitializer());

        ChannelFuture f = bootstrap.bind(new InetSocketAddress(port)).sync();
        System.out.println(" server start up on port : " + port);
        f.channel().closeFuture().sync();

    }

}

HttpServerInitialize.class
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelPipeline;
import io.netty.channel.socket.SocketChannel;
import io.netty.handler.codec.http.HttpObjectAggregator;
import io.netty.handler.codec.http.HttpServerCodec;

public class HttpServerInitializer extends ChannelInitializer<SocketChannel> {

    @Override
    protected void initChannel(SocketChannel channel) throws Exception {
        ChannelPipeline pipeline = channel.pipeline();
        pipeline.addLast(new HttpServerCodec());
        pipeline.addLast("httpAggregator",new HttpObjectAggregator(512*1024));
        pipeline.addLast(new HttpRequestHandler());

    }
}

HttpRequestHandler.class
import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import io.netty.channel.ChannelFutureListener;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.SimpleChannelInboundHandler;
import io.netty.handler.codec.DecoderResult;
import io.netty.handler.codec.http.DefaultFullHttpResponse;
import io.netty.handler.codec.http.FullHttpResponse;
import io.netty.handler.codec.http.HttpContent;
import io.netty.handler.codec.http.HttpHeaderNames;
import io.netty.handler.codec.http.HttpUtil;
import io.netty.handler.codec.http.HttpHeaderValues;
import io.netty.handler.codec.http.HttpHeaders;
import io.netty.handler.codec.http.HttpObject;
import io.netty.handler.codec.http.HttpRequest;
import io.netty.handler.codec.http.LastHttpContent;
import io.netty.handler.codec.http.QueryStringDecoder;
import io.netty.handler.codec.http.cookie.Cookie;
import io.netty.handler.codec.http.cookie.ServerCookieDecoder;
import io.netty.handler.codec.http.cookie.ServerCookieEncoder;
import io.netty.util.CharsetUtil;

import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Set;

import static io.netty.handler.codec.http.HttpResponseStatus.*;
import static io.netty.handler.codec.http.HttpVersion.*;

public class HttpRequestHandler extends SimpleChannelInboundHandler<Object> {

    private HttpRequest request;
    /** Buffer that stores the response content */
    private final StringBuilder buf = new StringBuilder();

    @Override
    public void channelReadComplete(ChannelHandlerContext ctx) {
        ctx.flush();
    }

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) {
        if (msg instanceof HttpRequest) {
            HttpRequest request = this.request = (HttpRequest) msg;

            if (HttpUtil.is100ContinueExpected(request)) {
                sendContinue(ctx);
            }

            buf.setLength(0);
            buf.append("WELCOME TO THE WILD WILD WEB SERVER\r\n");
            buf.append("===================================\r\n");

            buf.append("VERSION: ").append(request.protocolVersion()).append("\r\n");
            buf.append("HOSTNAME: ").append(request.headers().get(HttpHeaderNames.HOST, "unknown")).append("\r\n");
            buf.append("REQUEST_URI: ").append(request.uri()).append("\r\n\r\n");

            HttpHeaders headers = request.headers();
            if (!headers.isEmpty()) {
                for (Map.Entry<String, String> h: headers) {
                    CharSequence key = h.getKey();
                    CharSequence value = h.getValue();
                    buf.append("HEADER: ").append(key).append(" = ").append(value).append("\r\n");
                }
                buf.append("\r\n");
            }

            QueryStringDecoder queryStringDecoder = new QueryStringDecoder(request.uri());
            Map<String, List<String>> params = queryStringDecoder.parameters();
            if (!params.isEmpty()) {
                for (Entry<String, List<String>> p: params.entrySet()) {
                    String key = p.getKey();
                    List<String> vals = p.getValue();
                    for (String val : vals) {
                        buf.append("PARAM: ").append(key).append(" = ").append(val).append("\r\n");
                    }
                }
                buf.append("\r\n");
            }

            appendDecoderResult(buf, request);
        }

        if (msg instanceof HttpContent) {
            HttpContent httpContent = (HttpContent) msg;

            ByteBuf content = httpContent.content();
            if (content.isReadable()) {
                buf.append("CONTENT: ");
                buf.append(content.toString(CharsetUtil.UTF_8));
                buf.append("\r\n");
                appendDecoderResult(buf, request);
            }

            if (msg instanceof LastHttpContent) {
                buf.append("END OF CONTENT\r\n");

                LastHttpContent trailer = (LastHttpContent) msg;
                if (!trailer.trailingHeaders().isEmpty()) {
                    buf.append("\r\n");
                    for (CharSequence name: trailer.trailingHeaders().names()) {
                        for (CharSequence value: trailer.trailingHeaders().getAll(name)) {
                            buf.append("TRAILING HEADER: ");
                            buf.append(name).append(" = ").append(value).append("\r\n");
                        }
                    }
                    buf.append("\r\n");
                }

                if (!writeResponse(trailer, ctx)) {
                    // If keep-alive is off, close the connection once the content is fully written.
                    ctx.writeAndFlush(Unpooled.EMPTY_BUFFER).addListener(ChannelFutureListener.CLOSE);
                }
            }
        }
    }

    @Override
    protected void channelRead0(ChannelHandlerContext channelHandlerContext, Object o) throws Exception {

    }

    private static void appendDecoderResult(StringBuilder buf, HttpObject o) {
        DecoderResult result = o.decoderResult();
        if (result.isSuccess()) {
            return;
        }

        buf.append(".. WITH DECODER FAILURE: ");
        buf.append(result.cause());
        buf.append("\r\n");
    }

    private boolean writeResponse(HttpObject currentObj, ChannelHandlerContext ctx) {
        // Decide whether to close the connection or not.
        boolean keepAlive = HttpUtil.isKeepAlive(request);
        // Build the response object.
        FullHttpResponse response = new DefaultFullHttpResponse(
                HTTP_1_1, currentObj.decoderResult().isSuccess()? OK : BAD_REQUEST,
                Unpooled.copiedBuffer(buf.toString(), CharsetUtil.UTF_8));

        response.headers().set(HttpHeaderNames.CONTENT_TYPE, "text/plain; charset=UTF-");

        if (keepAlive) {
            // Add 'Content-Length' header only for a keep-alive connection.
            response.headers().setInt(HttpHeaderNames.CONTENT_LENGTH, response.content().readableBytes());
            // Add keep alive header as per:
            // - http://www.w.org/Protocols/HTTP/./draft-ietf-http-v-spec-.html#Connection
            response.headers().set(HttpHeaderNames.CONNECTION, HttpHeaderValues.KEEP_ALIVE);
        }

        // Encode the cookie.
        String cookieString = request.headers().get(HttpHeaderNames.COOKIE);
        if (cookieString != null) {
            Set<Cookie> cookies = ServerCookieDecoder.STRICT.decode(cookieString);
            if (!cookies.isEmpty()) {
                // Reset the cookies if necessary.
                for (Cookie cookie: cookies) {
                    response.headers().add(HttpHeaderNames.SET_COOKIE, ServerCookieEncoder.STRICT.encode(cookie));
                }
            }
        } else {
            // Browser sent no cookie.  Add some.
            response.headers().add(HttpHeaderNames.SET_COOKIE, ServerCookieEncoder.STRICT.encode("key1", "value1"));
            response.headers().add(HttpHeaderNames.SET_COOKIE, ServerCookieEncoder.STRICT.encode("key2", "value2"));
        }

        // Write the response.
        ctx.write(response);

        return keepAlive;
    }

    private static void sendContinue(ChannelHandlerContext ctx) {
        FullHttpResponse response = new DefaultFullHttpResponse(HTTP_1_1, CONTINUE, Unpooled.EMPTY_BUFFER);
        ctx.write(response);
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
        cause.printStackTrace();
        ctx.close();
    }
}

@normanmaurer And can I request a CVE for 1)?

Fix my description of 2).
I use this http request.
POST / HTTP/1.1
Host:localhost
Connection: close
Content-Length: 1
Transfer-Encoding: chunked, something

0

And netty use CL header because of 'something' in TE header. And I get the '0' data successfully.

@ZeddYu yes please request a CVE for incorrectly handling Content-Length and Transfer-Encoding: chunked headers.

@joshbressers If I understand correctly, this was assigned CVE-2019-20445 (according to the CVE description)
https://nvd.nist.gov/vuln/detail/CVE-2019-20445
@normanmaurer Do you know if your pull request #9865 addresses both CVE-2019-20445 and CVE-2020-7238?

@artem-smotrakov yes

@normanmaurer Thanks for confirming that!
I just wrote a couple of tests based on https://github.com/jdordonezn/CVE-2020-72381/issues/1 which is mentioned in CVE-2020-7238. The test passed although I am not 100% sure if they are sufficient
diff --git a/codec-http/src/test/java/io/netty/handler/codec/http/HttpRequestDecoderTest.java b/codec-http/src/test/java/io/netty/handler/codec/http/HttpRequestDecoderTest.java
index 1e780b7959..2548af0e2a 100644
--- a/codec-http/src/test/java/io/netty/handler/codec/http/HttpRequestDecoderTest.java
+++ b/codec-http/src/test/java/io/netty/handler/codec/http/HttpRequestDecoderTest.java
@@ -325,7 +325,30 @@ public class HttpRequestDecoderTest {
     public void testWhitespace() {
         String requestStr = "GET /some/path HTTP/1.1\r\n" +
                 "Transfer-Encoding : chunked\r\n" +
-                "Host: netty.io\n\r\n";
+                "Host: netty.io\r\n\r\n";
+        testInvalidHeaders0(requestStr);
+    }
+
+    @Test
+    public void testWhitespaceBeforeTransferEncoding01() {
+        String requestStr = "GET /some/path HTTP/1.1\r\n" +
+                " Transfer-Encoding : chunked\r\n" +
+                "Content-Length: 1\r\n" +
+                "Host: netty.io\r\n\r\n" +
+                "a";
+        testInvalidHeaders0(requestStr);
+    }
+
+    @Test
+    public void testWhitespaceBeforeTransferEncoding02() {
+        String requestStr = "POST / HTTP/1.1" +
+                " Transfer-Encoding : chunked\r\n" +
+                "Host: target.com" +
+                "Content-Length: 65\r\n\r\n" +
+                "0\r\n\r\n" +
+                "GET /maliciousRequest HTTP/1.1\r\n" +
+                "Host: evilServer.com\r\n" +
+                "Foo: x";
         testInvalidHeaders0(requestStr);
     }

@artem-smotrakov thanks... would you be willing to submit these tests as a PR so we can include them and ensure we not regress ?

@normanmaurer No problem, I'll open a pull request.

Is there any plan to backport this to 3.x? There are certain libraries still out there coupled to the 3.x package naming structure so if this fix isn't backported, they will be forever vulnerable.

@JLLeitschuh sorry but no backport will be done ... netty 3.x is EOL for many years and if projects did not update yet they may have a reason to do now.


Tracking issue for Raspberry Pi 3B+ and the latest kernel

Describe the bug
Using linuxPackages_latest with the current (as of 2018/08/19) sd images will not output to HDMI. Using the default kernel, 4.19, work.
See also:

https://logs.nix.samueldr.com/nixos-aarch64/2019-08-19#1566239909-1566240443;
https://nixos.wiki/wiki/NixOS_on_ARM/Raspberry_Pi#Raspberry_Pi_3B.2B_HDMI_output_issues

With the LinuxPackages_latest my doesn't boot and get stack at ``Starting the linux kernel...```. Let me know I can help you with debugging.

With the linuxPackages_latest my doesn't boot and get stack at Starting the linux kernel.... Let me know I can help you with debugging.

"my" what? :) If this is not for the 3B+ (or 3A+ I assume would fail, too), it's likely to be another issue.
Under ideal conditions, you would have a serial log output from u-boot up to when it fails.

I noticed by accident, that on some (3 out of 5) 3B+ it takes a very long time (3+ minutes) until there is HDMI output after uboot.
This from one of the slow ones
# systemd-analyze critical-chain multi-user.target
The time when unit became active or started is printed after the "@" character.
The time the unit took to start is printed after the "+" character.

multi-user.target @3min 9.547s
└─network-online.target @3min 9.546s
  └─dhcpcd.service @3min 2.733s +6.808s
    └─basic.target @3min 2.608s
      └─sockets.target @3min 2.592s
        └─dbus.socket @3min 2.570s
          └─sysinit.target @3min 2.533s
            └─systemd-timesyncd.service @3min 1.095s +1.421s
              └─systemd-tmpfiles-setup.service @3min 969ms +73ms
                └─local-fs.target @3min 949ms
                  └─boot.mount @3min 870ms +65ms
<EMAIL_ADDRESS>@57.699s +2min 3.136s
                      └─dev-disk-by\x2dlabel-NIXOS_BOOT.device @5.635s

Here is one that boots faster:
# systemd-analyze critical-chain multi-user.target
The time when unit became active or started is printed after the "@" character.
The time the unit took to start is printed after the "+" character.

multi-user.target @17.003s
└─network-online.target @17.001s
  └─dhcpcd.service @10.014s +6.976s
    └─basic.target @9.992s
      └─sockets.target @9.976s
        └─dbus.socket @9.960s
          └─sysinit.target @9.883s
            └─systemd-timesyncd.service @8.482s +1.376s
              └─systemd-tmpfiles-setup.service @8.350s +75ms
                └─local-fs.target @8.329s
                  └─boot.mount @8.205s +110ms
<EMAIL_ADDRESS>@7.874s +287ms
                      └─dev-disk-by\x2dlabel-NIXOS_BOOT.device @6.689s

I also see a lot of these messages in the ringbuffer (only on the slower ones) and they stop right before there is hdmi output:
[   38.368317] [drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [CRTC:116:crtc-2] flip_done timed out
[   48.608315] [drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [CONNECTOR:29:HDMI-A-1] flip_done timed out
[   58.848312] [drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [PLANE:113:plane-20] flip_done timed out
[   69.088293] [drm:drm_atomic_helper_wait_for_flip_done [drm_kms_helper]] *ERROR* [CRTC:116:crtc-2] flip_done timed out

All of them use regular linuxPackages.

They are all the same version, have the same sd card model and are deployed with the same configuration.
@mmahut does your pi maybe also have hdmi output after a few minutes? :)

@WilliButz yes, it does! I have a 3B.

This shouldn't be an issue anymore for new deployments. Only old deployments were affected anyway.


Cross-Links Sidebar Panel

Adds a panel to the sidebar to show how many cross/blocking links there
are.  Clicking the panel cycles through the links, moving to the map to
each. Helpful when zoomed out over dense areas where links may be very
small.

This is awesome! But perhaps can this be changed it into a left sidebar button? Right side panel is becoming longer and longer :disappointed:

Perhaps some future work could be to make expandable sections in the sidebar for each plugin, this might alleviate the 'clutter' problem.

This will be very useful.

This will still be very usefull..


Received exit code 1 while establishing SFTP session

I'm following this guide: http://broddin.be/2016/01/16/deploying-meteor-with-jenkins-and-mupx/
And ran into the following error (output already contains debug flag):
+ DEBUG=* mupx deploy

Meteor Up: Production Quality Meteor Deployments
------------------------------------------------
Configuration file : mup.json
Settings file      : settings.json

“ Checkout Kadira!
  It's the best way to monitor performance of your app.
  Visit: https://kadira.io/mup ”

Meteor app path    : /var/lib/jenkins/workspace/CannaBizRankings
Using buildOptions : {"debug":true}

Started TaskList: Deploy app 'CannaBizRankings' (linux)
[<IP_ADDRESS>] - Uploading bundle
Thu, 24 Mar 2016 06:30:38 GMT nodemiral:sess:<IP_ADDRESS> copy file - src: /tmp/4c9ccca2-0916-436d-a99d-59b0b8a67df7/bundle.tar.gz, dest: /opt/CannaBizRankings/tmp/bundle.tar.gz, vars: undefined
[<IP_ADDRESS>] x Uploading bundle: FAILED
	Received exit code 1 while establishing SFTP session
Build step 'Execute shell' marked build as failure
Finished: FAILURE

Here's my mup.json:
{
	"buildOptions": {
		// build with the debug mode on
		"debug": true
	},

	// Server authentication info
	"servers": [
		{
			"host": "(OMITTED)",
			"username": "root",
			"password": "(OMITTED)",
			//"pem": "~/.ssh/id_rsa",
			// or pem file (ssh based authentication)
			// WARNING: Keys protected by a passphrase are not supported
			//"pem": "~/.ssh/id_rsa"
			// Also, for non-standard ssh port use this
			//"sshOptions": { "port" : 49154 },
			// server specific environment variables
			"env": {}
		}
	],

	// Install MongoDB on the server. Does not destroy the local MongoDB on future setups
	"setupMongo": true,

	// Application name (no spaces).
	"appName": "CannaBizRankings",

	// Location of app (local directory). This can reference '~' as the users home directory.
	// i.e., "app": "~/Meteor/my-app",
	// This is the same as the line below.
	"app": "~/workspace/CannaBizRankings",

	// Configure environment
	// ROOT_URL must be set to your correct domain (https or http)
	"env": {
		"PORT": 80,
		"ROOT_URL": "(OMITED)"
	},

	// Meteor Up checks if the app comes online just after the deployment.
	// Before mup checks that, it will wait for the number of seconds configured below.
	"deployCheckWaitTime": 60,

	// show a progress bar while uploading.
	// Make it false when you deploy using a CI box.
	"enableUploadProgressBar": true
}

Any insights?

Are you targeting an instance of Ubuntu 14+ the only officially supported target?
Those extra formatting characters seem strange? ! What kind of shell are you running this in bash?
Otherwise it says the FTP is not working. You should manually check the login settings and ports are open etc.
Note: a newer version is currently being ported to kadirahq that is more like the development branch here called mupx which solves most problems people have had with this version of mup. So good to see your using mupx and not mup unless you use the kadirahq version which is like mupx but called mup again but its not actually working like mup did in this version. They say to use this mupx for now but if the other works for you its the way forward with Meteor 1.3 etc.


Hide flag inversion when help is hidden

Fixes an issue where the inversion of a flag would not be hidden when the ArgumentHelp.shouldDisplay value is false.
Added a unit test to check for this behavior.
Example:
@Flag(inversion: .prefixedNo, help: .hidden)
var prompt: Bool = true

Would produce the following in the help text:
OPTIONS:
  --no-prompt

But the desired outcome is that there is no listing for this flag in the help text.
Checklist

[x] I've added at least one test that validates that my change is working, if appropriate
[x] I've followed the code style of the rest of the project
[x] I've read the Contribution Guidelines
[x] I've updated the documentation if necessary

@swift-ci Please test


Finalise MedGen xref table - Updated MedGen ROBOT template

Partially addresses:

https://github.com/monarch-initiative/medgen/issues/15

Changes

Update: MedGen ROBOT template - with some fixes, updates, and now also includes MeSH mappings

Related

This PR introduced the changes just pulled down: https://github.com/monarch-initiative/medgen/issues/19

@matentzn Just double checking, we don't need to worry about "conflicts" anymore, right? At this point, we're trusting MedGen as the source for these mappings, so I don't need to run any such analysis, yes?

Thats right, thanks for checking! We are just trusting for now!

(you checked the box in the ticket, but rows are still in there)
@matentzn Glad to have your eyes on these, sorry about my error rate!

What happened here is that I did check multiple times on the MedGen side of things, but I marked the release as a "pre release". I thought it still carried the "latest" tag after that, but it removed it. So when I ran the goal here, it did not select the correct release, and I didn't check the results.
I fixed that issue, re-ran it, and confirmed there is no more MEDGENCUI.

@matentzn I made the appropriate changes in https://github.com/monarch-initiative/medgen/pull/19 and updated the file here. I think this should be good now; maybe take a look. You can also examine as a google sheet: https://docs.google.com/spreadsheets/d/1OD7WUR-nJ9lUqRv01TTghmf3aGvQRbWV5ZhVhthzB6U/edit#gid=1627855620

I chatted with @matentzn and the current thought is that the medgen xrefs can be added now, but not the mesh ones.

NOTE: The issue of proxy merges is being discussed with MedGen is this issue https://github.com/monarch-initiative/mondo-ingest/issues/491

[ ] @joeflack4 medgen repo: Move MeSH xrefs into a secondary robot template which will not currently be used.
[ ] @joeflack4 mondo repo: Pull down into this PR the new, updated original robot template, which will no longer have MeSH xrefs

@matentzn @twhetzel I misunderstood what Trish wrote. I thought this meant that whoever was running the robot template to operate on mondo-edit.obo would do something to avoid merging in the MeSH terms, and that this was an FYI. I understand the task now. Easy fix; will get to this next week!

I've completed the tasks for splitting out MeSH xrefs as requested. Now ready for re-review / merge.
Completed by:

https://github.com/monarch-initiative/medgen/pull/26

Looks awesome! But where is the code to support this change? It does not seem to be in this PR?

@matentzn This is ready to be merged, but I think you still need to mark resolved your requested changes.
The requested change (split MeSH out) has been completed:

here in this PR
In the related MedGen PR (merged already)

This PR has a single file in the diff with MESH and Medgen xrefs together - this does not seem to confirm changes are adressed?


Remove the stage checkout from scm

As suggested by @lordofthejars

Well stage should not be removed, just this line git url: 'https://github.com/redhat-helloworld-msa/aloha' should be changed to checkout scm


should webpack be a peerDependency?

I feel like the user should be able to chose his own webpack version. It would also mean you, as a maintainer do not have to deal with webpack upgrades.
What do you think?

cc @nealgranger

@vvo That is a good idea. I'll changes this in2.0 with confirmed support for webpack 2.

Yeah! Thanks :)

@vvo I'm publishing a v2.0.0-alpha.1 now for ya.

Nice thanks

@vvo npm i ms-webpack@next


Add eslint option to pass in more options

This adds an eslint option to customize the eslint command without affecting the executable callback, so that users can pass more options to eslint such as '--rulesdir $ESLINT_RULES_DIR' (this option cannot be set inside .eslintrc).

Looks fine to me. Merged! :+1:


[YUNIKORN-2098] Change go lint SHA detection

What is this PR for?
In YUNIKORN-285, we previously included git symbolic-ref -q HEAD because Travis CI often encountered a detached head situation. However, now that we have switched to GitHub Actions, we can utilize the checkout@v3 action, which efficiently fetches all Git references.
Consequently, we can remove git symbolic-ref -q HEAD Instead, we can initially set REV='origin/HEAD' If git rev-parse fails, we can fall back to REV=HEAD^
Regarding the modification from fetch-depth: 2 to fetch-depth: 0 , I have compared the GitHub Action times before this pull request and with the master. The duration of the git checkout action remained around 2 seconds. Therefore, I believe this change is acceptable, as it does not introduce significant overhead.
I encountered this issue when I manually triggered a GitHub Action in the k8shim fork repository. In contrast, manually triggering GitHub Actions in the core repository does not lead to this issue. This is because, in the core repository, the Golang CI is only executed when submitting a Pull Request.
However, I believe ensuring consistency in the make lint process between k8shim and core is the correct approach.
Perhaps we could consider aligning k8shim's GolangCI behavior with that of the core? (only triggering during PR)
cc @craigcondit @wilfred-s
What type of PR is it?

[x] - Improvement

What is the Jira issue?
YUNIKORN-2098

related k8shim 715

With the changes made to clean up all lint issues we now run without SHA detection.
This jira is no longer relevant


Text based HF

What does this PR do?
This PR implements several text-based pre-trained seq2seq HF models including mBART or NLLB model. In AST, people increasingly use these kinds of models for initializing AST models. Most popular approach at the moment is coupling a pre-trained speech model (wav2vec2, HuBERT, etc.) as a speech encoder, with the decoder of the mentioned text-based models.
This PR introduces two new HF classes: mbart.py and nllb.py, which handle the HF models. These classes are made compatible with the new HF interface #2116.
I implemented a new scheduler which is called Tri-stage scheduler. I've been seeing that this scheduler works better with the IWSLT recipe.
Two recipes on the tamasheq-french language pair are added. One can see this as an extension of the IWSLT22_lowresource recipe. Indeed, my intention is to illustrate that the use of pre-trained text-based decoder improve the performance of AST model in comparison with the original recipe. One shall see that in the reported BLEU scores in the README.
I prefer to use the SB's beam search rather than using directly the generate() method provided by HF transformers. This is why I also implemented S2SHFTextBasedBeamSearcher, which is not much different from S2STransformerBeamSearcher. The main reason is that for most cases, we would like to use directly the lm_head of the pre-trained model rather than making a new projection layer ourselves. This requires minor modifications to S2STransformerBeamSearcher that could be understood easily in my opinion.
Finally, I implemented an inference interface for the new models (I shall put them on HF shortly). The name W2V2mBARTAST is not very good for now I think. Feel free to propose a better name.

Before submitting

[x] Did you read the contributor guideline?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
[x] Did you make sure to update the documentation with your changes? (if necessary)
[x] Did you write any new necessary tests? (not for typos and docs)
[ ] Did you verify new and existing tests pass locally with your changes?
[ ] Did you list all the breaking changes introduced by this pull request?
[x] Does your code adhere to project-specific code style and conventions?

PR review

Reviewer checklist

[ ] Is this pull request ready for review? (if not, please submit in draft mode)
[ ] Check that all items from Before submitting are resolved
[ ] Make sure the title is self-explanatory and the description concisely explains the PR
[ ] Add labels and milestones (and optionally projects) to the PR so it can be classified
[ ] Confirm that the changes adhere to compatibility requirements (e.g., Python version, platform)
[ ] Review the self-review checklist to ensure the code is ready for review

@gruly might be interested in this PR. Note that I might make another PR which makes use of LaBSE for pre-training SAMU model. The new recipe will still concern IWLST_lowresource for better comparison between different approaches.

I need to add protobuf into the requirement file because both mBART and NLLB models require it. Adding protobuf into the extra-requirement file of the recipe would be another way except that it would fail the pytest because of the import error.

Thank you @mhn226 for submitting this PR. I've reviewed the changes and have a few comments to address before merging:

Permission Issue when running the recipe: It seems there's an issue with the HF models stored in a folder where users do not have permissions. This is leading to a PermissionError. We need to resolve this access problem.
 File "/opt/conda/envs/myenv2/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1195, in hf_hub_download
 os.makedirs(storage_folder, exist_ok=True)

File "", line 215, in makedirs
File "", line 215, in makedirs
File "", line 215, in makedirs
[Previous line repeated 6 more times]
File "", line 225, in makedirs
PermissionError: [Errno 13] Permission denied: '/local_disk'

Recipe Tests: The recipe tests are currently failing. Please run the following command to replicate:
python -c 'from tests.utils.recipe_tests import run_recipe_tests; print("TEST FAILED!") if not(run_recipe_tests(filters_fields=["Dataset"], filters=[["IWSLT22_lowresource"]], do_checks=True, run_opts="--device=cuda")) else print("TEST PASSED")'

Dropbox Model: Please upload the model to the SpeechBrain Dropbox and provide a link to it in the README file.

Additionally, please go through and address the comments made by @Adel-Moumen .

Hi @mhn226,
I did a second review of the code and I am now able to run the recipe. A few things before merging:
1- It looks like one test is failing due to some hparam file in your remote repo.
2- Could you please share with me the checkpoints to add in dropbox?

I changed the structure of the recipe as suggested by @Adel-Moumen . For now we have this order: IWSLT2022_lowresource -> AST -> transformer (because all pre-trained models are transformer-based). It looks good now, @mravanelli ?

Thank you @mhn226 for this PR and @Adel-Moumen for the review!


Modify ng update behavior

Expected behaviour
ng update karma updates:

"karma"
"karma-chrome-launcher"
"karma-coverage-istanbul-reporter"
"karma-jasmine": "~2.0.1"
"karma-jasmine-html-reporter"

Actual behaviour
ng update karma updates, only:

"karma"

Environment Details

Karma version (output of karma --version): 4.4.1
Relevant part of your karma.config.js file

Steps to reproduce the behaviour

Update an existing Angular 7 app to Angular 8 which wont update devDependencies by default.
Do, ng update karma
You can notice only karma gets updated, rest all others mentioned in expected behavior wont update

Please report this issue to the project that supplies "ng".


get node in Shadow DOM

I'm having issues integrating a react app in the shadow dom with packages that use the ReactDOM.findDOMNode function to access a specific node in the react app.
That function will definitely not be used since the app is in the shadow dom, however I'm curious if anyone has ran into this and what you use for the workaround to access the node by its ref?

Is this any better with React 16.x @awwester? I've just done a quick test, and it seems to be fine. However there could be more complex cases that I haven't tested.

@Wildhoney I'll be circling back to this asap. Thanks for the heads up.

Please try again with the new version.


Update agent image.

Switch to official askar only image, and update to 0.10.3.
The previous agent version, 1.0.0-rc-0 is incompatible with ACA-Py >=1.10.1.  See https://github.com/hyperledger/aries-cloudagent-python/issues/2528

Prerequisite update for DTS endorser service updates.

@WadeBarnes I do not have permission to merge the change - it looks good though


Documentation error. Incorrect package name

https://docs.anchorprotocol.com/developers-terra/anchor-cli has an incorrect package name:
$ npm install -g @anchor-protocol/cli
The npm package page has the correct package:
$ npm install -g @anchor-protocol/anchorcli
I didn't find in this repo the docs, if you can point me to the repo location I can send a PR

fixed, thanks!


Can't expand resources

when attempting this;
final sessionData = await stripe.sessions.retrieve(session.id, params: {'expand': ['line_items']});

Stripe responds with an error saying ['line_items'] is not a valid array. Could it be it expects a javascript array?
https://stripe.com/docs/expand

Hello,
Soon I will update the library by adding the Session endpoint for list line_items
Thank you very much

Hello,
Soon I will update the library by adding the Session endpoint for list line_items
Thank you very much


[BUGFIX] Fixed Polymod and Mods Support in FlxPartialSound Using OpenFL Assets

we load the aseet's bytes with Assets.loadBytes (which works asynchronously) then create a haxe.io.BytesInput instance with the bytes retrieved instead of directly reading the file with FileSystem.read to get a FileInput for it
This should be merged with the change in this pr on Funkin

this looks yummy! I will mark this for myself for testing since it doesn't seem like it's too verbose of a workaround/fix! thanks!

@KarimAkra I'm not sure if your testing suggests otherwise, but it seems to me like this code is loading the whole file before cutting out a portion?
As opposed to the old code, where I believe the intent was to use the filesystem to read only a portion of the file directly from disk, this saving memory by never having the full file loaded at once.

I've just made a quick comparison between the  build on itch.io and the one i compiled with these changes
Both seemed to be using the same amount of memory that's varying between 220-230 mb
And it adds up around 5-10 mb on both builds every time a portion plays
So i think it's safe to say this dosent have any impact on the memory usage & optimization!

Turns out Bytes don't get cached or allocated to memory unless they're pushed to the openfl cache map or something like that
So yes, i can confirm this dosen't impact memory usage at all

hmmm it doesn't save the full audio file to memory that does seem to be correct, but one thing about this is that it does at least try to load the whole audio file right? like Assets.loadBytes(path) i assume would attempt to load the full file (even if it doesn't affect memory), and wouldn't complete the little event listener until the full file has been loaded? in which case I think something closer to the original implementation is intended, so that we ONLY load exactly the bytes we want out of the audio file, so that it hopefully loads up quicker.
i havent used fileinput and file reading too much to be honest! my implementation could be doing the same thing! but the intention is to get it to read ONLY the slice of bytes it needs/wants for a preview

I made a quick  benchmark to compare the performance of FileInput and BytesInput  (with loading the whole file bytes)

here are the results

Given these results, the impact on speed when using BytesInput  instead of FileInput is minimal. Therefore, it should be safe to use BytesInput without significant performance concerns.

if you're wondering what's the audio file i used in here
it's the 2hot instrument (1.50 mb)

@KarimAkra Can you revert your most recent change? Assets.loadBytes is SUPPOSED to work here, I need to fix things on the Polymod side.

@KarimAkra Can you revert your most recent change? Assets.loadBytes is SUPPOSED to work here, I need to fix things on the Polymod side.

I looked deep into how Assets.loadBytes works before making these changes
It uses lime.utils.Bytes.fromFile (https://github.com/openfl/lime/blob/develop/src%2Flime%2F_internal%2Fbackend%2Fnative%2FNativeHTTPRequest.hx#L421-L427)
Bytes.fromFile dosen't use the lime/openfl library assets tree it's like sys.io.File.getBytes so it can't really work with polymod afaik
https://github.com/openfl/lime/blob/develop/src%2Flime%2Futils%2FBytes.hx#L94-L98

Bytes.fromFile dosen't use the lime/openfl library assets tree it's like sys.io.File.getBytes so it can't really work with polymod afaik

Before it reaches that call, Lime calls AssetLibrary.loadBytes(), which gets overridden by Polymod. It just isn't implemented properly.
https://github.com/larsiusprime/polymod/blob/6d1bdf79b463ca0baa8471dfc6873ab7701c46ee/polymod/backends/LimeBackend.hx#L626-L642

Reverted it
I think Polymod needs to use a complete alternative method for loadBytes instead of calling the one from the lime AssetLibrary
I don't really know much abt how Polymod functions so I guess you'll have figure this out

I got it working on the latest develop commit of Polymod. It currently has a SLIGHT delay on the first play but otherwise seems to work well even on ZIP mods.


feat: implemented personal builder profile page

Description
Implemented personal builder profile page, that shows my details, my avatar, a short bio and my address
Additional Information

[x] I have read the contributing docs (if this is your first contribution)
[x] This is not a duplicate of any existing pull request

Your ENS/address: 0x7429CbD5eD20736645723E972bE60B7F6BF5959c

@phipsae @technophile-04 made this PR as regards this issue https://github.com/BuidlGuidl/batch10.buidlguidl.com/issues/9
please review

Hi @phipsae Thank you so much for your review, I've made the necessary and suggested changes as specified in the comments, and I also learnt new stuff regarding disabling SSR for client pages.
Please review 🙌🏾

@michojekunle thanks for the page! You really like that you integrate the particles effect :)
Below you'll find comments regarding the code. If there are any questions feel free to ask.
Thanks!

Hi @phipsae Thank you so much for your review, I've made the necessary and suggested changes as specified in the comments, and I also learnt new stuff regarding disabling SSR for client pages.
Please review 🙌🏾

Very nice, thank you a lot!!!
Gonna merge it!


Note about extern crate rol in linking sys packages

I just spent a couple of hours debugging why a sys package didn't link. It turns out that declaring it as a crate in your code is necessary for this to work.

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @wycats (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. The way Github handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.
Please see the contribution instructions for more information.

@steveklabnik Nits have been fixed :-)

On second thought: It's probably more clear if libfoo in this example would be called libfoo-sys? Or maybe libfoo-sys and libfoo instead of libfoo and libbar?

I think that this note may make more sense in the *-sys section of the documentation perhaps? It only comes into play when you're relying on an upstream library to provide some symbols I believe.

This segment of the build doc is about *-sys packages right? Which section of the docs are you referring to?

The is currently in the section about the links key, but there's a dedicated section to *-sys packages down below.

Sorry for the slow response, this fell of my radar for a bit. @alexcrichton: I'm still a bit confused on where you'd like to see this note. It's currently already in the section about *-sys package like you suggested. Or do you think it should be in the case study further down the page?

If you take a look around line 449 you'll see # *-sys Packages which is the section I think this should go in, right now it's just under something which links down there I believe.

Closing due to inactivity, but feel free to resubmit with my comment addressed!


CFFL

CFFL

Linter check found the following problems:
The following problems have been found:
ERROR: /tmp/abs_3cwiahs30q/clone/recipe/meta.yaml:16: pip_install_args: pip install should be run with --no-deps.
ERROR: /tmp/abs_3cwiahs30q/clone/recipe/meta.yaml:21: missing_wheel: For pypi packages, wheel should be present in the host section
ERROR: /tmp/abs_3cwiahs30q/clone/recipe/meta.yaml:40: missing_documentation: The recipe is missing a doc_url or doc_source_url
WARNING: /tmp/abs_3cwiahs30q/clone/recipe/meta.yaml:40: missing_description: The recipe is missing a description
ERROR: /tmp/abs_3cwiahs30q/clone/recipe/meta.yaml:40: missing_license_family: The recipe is missing the about/license_family key.
Errors were found


Any way to get list of available pages / routes?

[x] I have searched the issues of this repository and believe that this is not a duplicate.

pages/
  components/
    component.js
  index.js
  catalog.js
  about.js

const listOfPages = await getListOfPages()
console.log(listOfPages)
// [ 'index', 'catalog', 'about' ]

Is it possible to implement such a thing?

afaik there's no out-of-the-box way with next. if your structure is that simple, you could just list the directory with fs
we have a ton of dynamically generated pages, so we generate a sitemap.xml w/ next-routes, express-sitemap, and some code that fetches slugs from our db.
we then use sitemapper  to turn the xml into a list of urls for tests (though i guess we could just do this while we built the sitemap).
if you don't have a sitemap (which i'm guessing you don't), there's a tool popular among SEOs called screaming frog that'll crawl your site & build an XML sitemap. free up to 500 pages & fairly reliable / credible

@negamaxi you can use https://github.com/zeit/next.js/blob/canary/server/build/webpack.js#L53
const pages = await glob('pages/**/*.js', { cwd: __dirname })


Update

I have tried to update the fork with the latest changes from the original repository.
I have used rebase. However, I am unsure about some of the conflict solutions, which I have applied.
I thought about doing that because of https://github.com/harskish/ganspace/issues/4, let me know if it is not needed.

Situation is clearer after https://github.com/harskish/ganspace/issues/8. Basically, this fork is only intended for weight conversion, and it is better to keep the current version so that people do not face compatibility issues if they try to use GANspace with weights converted with different methods.


Put a limit on sound resolution

I accidentally set the sound resolution to 22500048 and it caused Rainmeter to lock up, and lock up again on startup. I had to manually edit variables.ini to fix it.

The settings meter slider for the sensitivity cannot push over 100. The lua scripts shouldn't force a max value by hard coding 100 as max value, in case someone wants to push over it through the variables.inc.


⚠ Bofrost.fr has degraded performance

In 586571b, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 1424 ms

Resolved: Bofrost.fr performance has improved in b198c50 after 9 minutes.


Add RazorSlice.Write() overloads for other primitive value types

To optimize rendering of double, decimal, char, DateTime, etc. using Utf8Formatter, in the same way as is done already for int and int? values.

This is done now.


Add CodeCov to measure unit test coverage

Since we've been porting components from cfgov-refresh to capital-framework, we need to maintain the same code coverage standards here.

[ ] new/updated code should have 100% coverage
[ ] set up Codecov on this repo to measure and enforce failures on PRs

What are the steps to do this? I haven't set up codecov before. Is there a good guide?


Enable support for Openstack cloud.

Signed-off-by: arcprabh<EMAIL_ADDRESS>

Can one of the admins verify this patch?

Can one of the admins verify this patch?

New commit link is https://github.com/arcprabh/kraken/commit/92f81ed616b416c55ac602fedaa7ac98d9d4f1e2
Please review @paigerube14

New commit link is https://github.com/arcprabh/kraken/commit/92f81ed616b416c55ac602fedaa7ac98d9d4f1e2
Please review @paigerube14

In the get_openstack_nodename function I ran into a problem
`server_list = runcommand.invoke("openstack server list | grep %s" % (os_node_ip))
list_of_servers = server_list.split('\n')
    for item in list_of_servers:
        items = item.split("|")
        counter = 0
        for i in items:
            if i.strip() != "" and counter == 2:
                node_name = i.strip()
                counter += 1
                continue
            item_list = i.split('=')
            if len(item_list) == 2 and item_list[-1].strip() == os_node_ip:
                return node_name
            counter += 1`

In the get_openstack_nodename function I ran into a problem
`server_list = runcommand.invoke("openstack server list | grep %s" % (os_node_ip))
list_of_servers = server_list.split('\n')
    for item in list_of_servers:
        items = item.split("|")
        counter = 0
        for i in items:
            if i.strip() != "" and counter == 2:
                node_name = i.strip()
                counter += 1
                continue
            item_list = i.split('=')
            if len(item_list) == 2 and item_list[-1].strip() == os_node_ip:
                return node_name
            counter += 1`

In the get_openstack_nodename function I ran into a problem with getting the node name from the IP address. It seemed that in our OpenStack server list there were multiple instance matches when greping for the IP address (see example response below)
Ex.) runcommand.invoke("openstack server list | grep %s" % (os_node_ip))
server list response: | 48e050e7-225b-4942-805e-f744f2bf3f62 | ug-bootstrap                                         | ACTIVE  | ugd2932022589-rm94d-openshift=<IP_ADDRESS>, <IP_ADDRESS>                     | rhcos-42.80.20191010.0                      | m1.xlarge      |
| a2ed6f85-5a84-4249-9eb8-c029541ec464 | qe-pr-osp2-hzmvf-worker-0-87jks                                       | ACTIVE  | qe-pr-osp2-hzmvf-openshift=<IP_ADDRESS>                                       | qe-pr-osp2-hzmvf-rhcos                      | m1.large       |
So the response from the function was the 2 node names concatenated and then when it tried to stop the node the OpenStack client couldn’t find a node with name “ug-bootstrapqe-pr-osp2-hzmvf-worker-0-87jks”
I coded up something to get past that point. Feel free to rewrite if there’s a better way
server_list = runcommand.invoke("openstack server list | grep %s" % (os_node_ip))
list_of_servers = server_list.split('\n')     for item in list_of_servers:         items = item.split("|")         counter = 0         for i in items:             if i.strip() != "" and counter == 2:                 node_name = i.strip()                 counter += 1                 continue             item_list = i.split('=')             if len(item_list) == 2 and item_list[-1].strip() == os_node_ip:                 return node_name             counter += 1
I’m not sure if this depends on the set up of an Openshift cluster, but for me the node name and the name of the open stack instance are the same so the get ip address and get open stack node name are redundant. If this is not the case for your set up please keep the functions; just wanted to call it out that that’s how our test environments are set up.
Are you also able to add in the terminate node code when you get a chance.
Looking great though! Thanks for the contributions and quick code changes already

In the get_openstack_nodename function I ran into a problem with getting the node name from the IP address. It seemed that in our OpenStack server list there were multiple instance matches when greping for the IP address (see example response below)
Ex.) runcommand.invoke("openstack server list | grep %s" % (os_node_ip))
server list response: | 48e050e7-225b-4942-805e-f744f2bf3f62 | ug-bootstrap                                         | ACTIVE  | ugd2932022589-rm94d-openshift=<IP_ADDRESS>, <IP_ADDRESS>                     | rhcos-42.80.20191010.0                      | m1.xlarge      |
| a2ed6f85-5a84-4249-9eb8-c029541ec464 | qe-pr-osp2-hzmvf-worker-0-87jks                                       | ACTIVE  | qe-pr-osp2-hzmvf-openshift=<IP_ADDRESS>                                       | qe-pr-osp2-hzmvf-rhcos                      | m1.large       |
So the response from the function was the 2 node names concatenated and then when it tried to stop the node the OpenStack client couldn’t find a node with name “ug-bootstrapqe-pr-osp2-hzmvf-worker-0-87jks”
I coded up something to get past that point. Feel free to rewrite if there’s a better way
server_list = runcommand.invoke("openstack server list | grep %s" % (os_node_ip))
list_of_servers = server_list.split('\n')     for item in list_of_servers:         items = item.split("|")         counter = 0         for i in items:             if i.strip() != "" and counter == 2:                 node_name = i.strip()                 counter += 1                 continue             item_list = i.split('=')             if len(item_list) == 2 and item_list[-1].strip() == os_node_ip:                 return node_name             counter += 1
I’m not sure if this depends on the set up of an Openshift cluster, but for me the node name and the name of the open stack instance are the same so the get ip address and get open stack node name are redundant. If this is not the case for your set up please keep the functions; just wanted to call it out that that’s how our test environments are set up.
Are you also able to add in the terminate node code when you get a chance.
Looking great though! Thanks for the contributions and quick code changes already

After changing those couple of things listed above I was able to succesfully start, stop and reboot nodes in Openstack :)
Please be sure to run "tox" as well to correct all formatting issues.

After changing those couple of things listed above I was able to succesfully start, stop and reboot nodes in Openstack :)
Please be sure to run "tox" as well to correct all formatting issues.

In the get_openstack_nodename function I ran into a problem with getting the node name from the IP address. It seemed that in our OpenStack server list there were multiple instance matches when greping for the IP address (see example response below)
Ex.) runcommand.invoke("openstack server list | grep %s" % (os_node_ip))
server list response: | 48e050e7-225b-4942-805e-f744f2bf3f62 | ug-bootstrap | ACTIVE | ugd2932022589-rm94d-openshift=<IP_ADDRESS>, <IP_ADDRESS> | rhcos-42.80.20191010.0 | m1.xlarge |
| a2ed6f85-5a84-4249-9eb8-c029541ec464 | qe-pr-osp2-hzmvf-worker-0-87jks | ACTIVE | qe-pr-osp2-hzmvf-openshift=<IP_ADDRESS> | qe-pr-osp2-hzmvf-rhcos | m1.large |
So the response from the function was the 2 node names concatenated and then when it tried to stop the node the OpenStack client couldn’t find a node with name “ug-bootstrapqe-pr-osp2-hzmvf-worker-0-87jks”
I coded up something to get past that point. Feel free to rewrite if there’s a better way
`
server_list = runcommand.invoke("openstack server list | grep %s" % (os_node_ip))

list_of_servers = server_list.split('\n')

for item in list_of_servers:

    items = item.split("|")

    counter = 0

    for i in items:

        if i.strip() != "" and counter == 2:

           node_name = i.strip()

            counter += 1

            continue

        item_list = i.split('=')

        if len(item_list) == 2 and item_list[-1].strip() == os_node_ip:

            return node_name

        counter += 1

`
I’m not sure if this depends on the set up of an Openshift cluster, but for me the node name and the name of the open stack instance are the same so the get ip address and get open stack node name are redundant. If this is not the case for your set up please keep the functions; just wanted to call it out that that’s how our test environments are set up.
Are you also able to add in the terminate node code when you get a chance.
Looking great though! Thanks for the contributions and quick code changes already

Tested this code and it works on my set up too.

LGTM

@pruan-rht we can do a follow up PR if needed for clean up.  Merging to get the functionality in.


Add Patrick Zheng as notation maintainer

Add Patrick Zheng (@patrickzheng200) as a seed maintainer of notation based on their activity as per - https://github.com/notaryproject/notation/issues/514
Signed-off-by: Yi Zha<EMAIL_ADDRESS>

Codecov Report

Merging #531 (38890ce) into main (375a701) will not change coverage.
The diff coverage is n/a.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@           Coverage Diff           @@
##             main     #531   +/-   ##
=======================================
  Coverage   29.61%   29.61%
=======================================
  Files          26       26
  Lines        1513     1513
=======================================
  Hits          448      448
  Misses       1048     1048
  Partials       17       17

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

+1 for adding @patrickzheng200
However, this PR also removes the @notaryproject/notation-maintainers list
Suggest we treat these as separate PRs, reviewing Patrick as an added maintainer, and a separate PR to convert the teams reference of @notaryproject/notation-maintainers to the individuals in that list.

LGTM but IANAM

Close as aggregated PR https://github.com/notaryproject/notation/pull/542 was merged


what kind of data are stored in file rel2candidates.json, e1rel_e2.json and path graph

May I know what kind of data are stored in file rel2candidates.json, e1rel_e2.json and path graph?
Can you share how you process the original triple data step by step?

Hi thanks for reaching out! I believe both files for NELL are taken from the MetaR repo and the original NELL-One dataset by Xiong et al. (https://arxiv.org/pdf/1808.09040.pdf). I generated them similarly for the other two datasets.
I believe rel2candidates.json is supposed to contain all possible candidates for the tail node, and e1rel_e2.json is just adjacency list cached for easy access. For the two datasets I added, candidate set is simply all nodes.


The demo code do not work

From<EMAIL_ADDRESS>on August 06, 2011 10:00:28
The demo code do not work
Original issue: http://code.google.com/p/memcached-session-manager/issues/detail?id=108

From martin.grotzke on August 06, 2011 02:28:43
More details please! Which branch, expected vs actual etc

From<EMAIL_ADDRESS>on August 06, 2011 02:33:55
The latest code.
I just downloaded from github

From martin.grotzke on August 06, 2011 02:37:26
What exactly does not work?

From martin.grotzke on August 06, 2011 02:39:13
Honestly: if don't want to invest the time to write a somehow useful bug report you can't expect from me to take the time to guess what's going wrong for you...

From<EMAIL_ADDRESS>on August 06, 2011 02:39:33
After I deployed my own application to it. A lot of error messages come out

From martin.grotzke on August 06, 2011 03:24:54
Please post them here

From<EMAIL_ADDRESS>on February 19, 2015 09:04:42
I think this could clarify the problem: https://github.com/magro/msm-sample-webapp/issues/2

From martin.grotzke on February 19, 2015 14:08:10
Thanks! Btw, samples are now in msm directly as well: https://github.com/magro/memcached-session-manager/tree/master/samples

From martin.grotzke on February 19, 2015 14:08:37
(No comment was entered for this change.)
Status: Fixed


E1129

Merge https://github.com/tensorflow/cleverhans/pull/720 first

Let's wait for #654 to be merged so we don't have to refactor it


Code gen: how to return empty json array

Problem statement
This may be a bug.
I have a service which should return a list of Foo. The list may be empty. However, empty list is converted to null json because (?) generated code contains "omitempty" tag. Adding "required: true" to the response schema causes the model file in question to become empty - I guess generation fails entirely.
Swagger specification
Steps to reproduce
responses:
        200:
          description: List of Foo
          schema:
            type: array
            items:
              $ref: "#/definitions/Foo"

produces:
type ListFooOK struct {
	// In: body
	Payload []*models.Foo `json:"body,omitempty"`
}

and:
responses:
        200:
          description: List of Foo
          schema:
            type: array
            required: true
            items:
              $ref: "#/definitions/Foo"

validates OK but generates empty file.

this should be fixed on master, I got a PR this week which fixes exactly this


Resize viewport always focuses on the same location of website

When studying the website of Bootstrap Docs, I was resizing my browser to test the intricacies of columns and rows. However, when I did so, the page keeps jumping up and down, making it very hard for me to focus on the change that happened. I had to scroll to the correct location every time I stopped resizing the browser width.
Suggestion for a feature: to make websites built with Bootstrap always focus on the same exact location no matter what happens with resizing the browser.
This will furthermore improve user experience.
I am using:
Mac OS El Capitan 10.11.5, MacBook Pro Late 2013 Model, Chrome latest version, Bootstrap latest version.

@byteknacker How would you propose to do this. I can't think of any good ways.

Okay, so lets give this a tought...

Suggestion for a feature: to make websites built with Bootstrap always focus on the same exact location no matter what happens with resizing the browser.

This happens because way page is displayed changes due to media queries. Some elements turn from rows into lists, some images disappear, etc ect. In an nutshell, this changes page's height moving elements around.
At this point you first need to decide what constitutes the "exact location". If, say, you were 30 px scrooled into 140px tall card containing user's message, but now you have turned your device from landscape to portrait, whats the X offset to the card that's now 190px? How are you going to compute it accurately? Card's header was one-line tall (22px), but now it turned to two-line layout, making it 44px. Perhaps relation to it should be used, so we should set scrool 52px into card?
So you'll decide that checkpoint will be scroll relative to top offset of deepest element that's currently intersected by viewport's top edge. But now you need strategy for handling scenario when element itself isn't leaf node (ergo, its .container and user is scrolled right between two deeper elements). You could keep scroll at the position relative to bottom offset of last direct descendant of the intersected element.
And now you need to see how your implementation plays with scroll management in popular view libraries that are doing it as well, like React.js that also implements suffisticated scroll preservation strategies around its DOM renderer.
And then, few weeks later, while adding another preservation strategy for yet another weird edge case, you will ask yourself, if what you have fixed was really considered an issue by your users. ;)

Not something we plan to solve in Bootstrap, sorry.

Also, browser vendors are already working on building similar native features into the browsers themselves:
https://github.com/WICG/interventions/issues/2

Many thanks @cvrebert I think it should be something that browser developer will take care of rather than web development frameworks. Good point @rafalp