cmd/go: when building gccgo, GOPKGPATH encoding does not match pkgpath encoding

What version of Go are you using (go version)?

$ go version
go version go1.12.2 gccgo (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008 linux/amd64

Does this issue reproduce with the latest release?
I'm using gccgo-9, which I think is the latest release.
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/apollo/.gocache"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/apollo/Development/weird-gopath"
GOPROXY=""
GORACE=""
GOROOT="/usr"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/gcc/x86_64-linux-gnu/9"
GCCGO="/usr/bin/x86_64-linux-gnu-gccgo-9"
CC="x86_64-linux-gnu-gcc-9"
CXX="x86_64-linux-gnu-g++-9"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build492198966=/tmp/go-build -gno-record-gcc-switches -funwind-tables"

What did you do?

Code: https://play.golang.org/p/CwSEukNuiAg
// main.go

package main

import "fmt"
import "golang.org/x/tools/container/intsets"

func main() {
	var foo intsets.Sparse
	fmt.Println(foo)
}

Building with regular go works:
> go version
go version go1.12.2 linux/amd64
> CGO_ENABLED=1 go build -v -i -o foo
> ./foo
{{0 [0 0 0 0] <nil> <nil>}}

while building with gccgo fails:
> go version
go version go1.12.2 gccgo (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008 linux/amd64
> CGO_ENABLED=1 go build -v -i -o foo
github.com/paleozogt/golangtoolstest
# github.com/paleozogt/golangtoolstest
/usr/bin/ld: /home/apollo/Development/weird-gopath/pkg/gccgo_linux_amd64/golang.org/x/tools/container/libintsets.a(_go_.o): in function `golang.org..z2fx..z2ftools..z2fcontainer..z2fintsets.block.len':
/home/apollo/Development/weird-gopath/src/golang.org/x/tools/container/intsets/sparse.go:137: undefined reference to `golang.org..z2fx..z2ftools..z2fcontainer..z2fintsets.popcount'
/usr/bin/ld: /home/apollo/Development/weird-gopath/pkg/gccgo_linux_amd64/golang.org/x/tools/container/libintsets.a(_go_.o): in function `intsets.nlz':
/home/apollo/Development/weird-gopath/src/golang.org/x/tools/container/intsets/util.go:51: undefined reference to `golang.org..z2fx..z2ftools..z2fcontainer..z2fintsets.popcount'
collect2: error: ld returned 1 exit status

What did you expect to see?
Build success.
What did you see instead?
Build failure.

What's odd is that there's gccgo code in golang.org/x/tools/container/intsets, but it doesn't seem to be getting hooked up.
Running nm shows this
> nm pkg/gccgo_linux_amd64/golang.org/x/tools/container/libintsets.a | grep popcount
                 U golang.org..z2fx..z2ftools..z2fcontainer..z2fintsets.popcount
0000000000000000 T golang_org_x_tools_container_intsets.popcount
                 U __popcountdi2

so the function definition is there.
That naming is coming from an ASM label in popcnt_gccgo_c.:
extern intptr_t popcount(uintptr_t x) __asm__(GOSYM_PREFIX GOPKGPATH ".popcount");

It appears that gccgo isn't following this naming convention?  I'm not sure where this sort of thing is documented.
At any rate, the problem can be fixed by commenting out the ASM labelling and switching to //extern in popcnt_gccgo.go:
//extern popcount
func popcount(x word) int

It's a bug in the go tool.  The GOPKGPATH macro is supposed to be defined to the encoding of the pkgpath.  But when we changed the pkgpath encoding in https://golang.org/cl/135455 and https://golang.org/cl/200838 we didn't update the go tool accordingly.
CC @thanm

Moving to backlog milestone.  This should still be fixed in the gc version of cmd/go.


Add COLLEGE_ATTENDANCE_FIELD to the tiles and downloads

Description
Add one field (COLLEGE_ATTENDANCE_FIELD) to the tiles and data downloads, including short name.

@esfoobar-usds this is done right?

I defer to @emma-nechamkin for a quick answer, or I can check the codebase.

@BethMattern do we have confirmation on whether this is done?


Text selection highlight is not shown on decorations w/background colors

Version: 1.35.1 (user setup)
Commit: c7d83e57cd18f18026a8162d042843bda1bcf21f
Date: 2019-06-12T14:30:02.622Z
Electron: 3.1.8
Chrome: 66.0.3359.181
Node.js: 10.2.0
V8: 6.6.346.32
OS: Windows_NT x64 10.0.18362
If a decoration is set and includes a background color, selection highlight is not rendered.
In the following image, text decorations (with a background value) exist on the keywords and symbols, but not the whitespace.  The entire section is selected.  Notice that the whitespace receives a selection highlight color, but the decorated text does not.

/duplicate https://github.com/microsoft/vscode-cpptools/issues/3780

@Tyrair , This was duped incorrectly.  The cpptools issue should be duped to this, not vice version.  (I work on cpptools).  The issue is, generally, that if there is a background color specified on a TextEditorDecoration, no highlight selection color will be applied to it when that text is selected.  We can work around the issue in cpptools when the desired background color happens to be the same as the editor background color (so, setting a background color is unnecessary, and avoided).  But, the issue remains in the general case.  Please apply selection highlight color on TextEditorDecorations when they have a background color specified, and their text is selected.

@Colengms ah sorry I thought it was just a limitation of how you did semantic coloring but it I've also seen this issue in the git diff decorations:

This is as designed. Using a color with opacity 100% will result in other things not being visible. The solution is to use a color with an opacity < 100%.

cc @joaomoreno, see the image in https://github.com/microsoft/vscode/issues/75642#issuecomment-504207975

I'm almost certain this should not have been closed.  The proposed solution does not work.  Setting the (only available) "opacity" property of DecorationRenderOptions to 0.5, for example, has no effect on the visibility of the text selection highlight.  As you can see, both the selected and unselected text (w/ red background) in the example below appear the same, despite opacity 0.5 being used.

Using an alpha channel value in the background color itself does not appear to be accepted.
The opacity property would seem to only apply to the foreground color.  Setting the editor background to another color, for example, I can see that the decoration's background color is not blended with it at all.

Furthermore, even if this worked, it would not be a good design to require that anyone who sets a background color on a decoration to use (at this point ambiguous) semi-transparency in order to avoid an issue that would cause user text selection not to be apparent when selecting text within the decoration.  The text selection color should either be blended/inversed with the background color, or replace the background color, so that selection is apparent.

I am seeing something strange. I am seeing different results running from source or running from a built version at the same commit. I am using the following to reproduce:

package.json:

{
    "name": "75642",
    "publisher": "alex",
    "version": "1.0.0",
    "engines": {
        "vscode": "^1.31.0"
    },
    "activationEvents": ["*"],
    "main": "index.js"
}

index.js:

const vscode = require('vscode');

exports.activate = function() {
    const transparentRed = vscode.window.createTextEditorDecorationType({
        backgroundColor: '#ff000033'
    });
    const red = vscode.window.createTextEditorDecorationType({
        backgroundColor: '#ff0000ff'
    });

    const decorate = function() {
        const editor = vscode.window.activeTextEditor;
        if (!editor) {
            return;
        }
        editor.setDecorations(transparentRed, [new vscode.Range(215681, 0, 215681, 6)]);
        editor.setDecorations(red, [new vscode.Range(215683, 2, 215683, 7)]);
    }
    vscode.window.onDidChangeActiveTextEditor(decorate);
    decorate();
};

running from source

running from a built version

Well, after debugging I can no longer reproduce this in the built version...

Ok, now I understand this weirdness, I am running into a code cache bug. I've created https://github.com/microsoft/vscode/issues/78542 to track that.
In any case, going back to our issue. I can set an opacity and it is respected. See the above sample code:

As you can see, this does not really address the issue.  Note how your background color, now that it's blending with the editor background, is no longer red.  You should be able to use a color that does not need blend with the editor background color, yet user text selection is still apparent.


AUTH-1273: create separate docker image for fargate

What?
Create separate docker image for fargate.

Rename existing local image and update docker-compose.yml to use it.
Add a new dockerfile to run the application in fargate.  This image needs all files locally as it cannot rely on a mounted volume.
Use 'cp' instead of 'copyfiles' in one place in package.json as copyfiles was hanging when building and/or running the docker image.

Why?
A separate image is needed to run in fargate as we are currently using a CF node buildpack rather than a docker image.  The existing local image could not be run in fargate as-is.

Looks good. Minor suggestion - It would be useful to add some of the info from the PR into the commit


vendor: upgrade github.com/mdlayher/vsock

Hey folks! I just wanted to let you know that I've landed Go 1.12 runtime network poller support in https://github.com/mdlayher/vsock, meaning that the vsock package should now comply completely with net.Listener and net.Conn! I'd recommend upgrading the package when possible.
Go 1.11 also still works, but you have to put a loop around the Accept method in order to wait for incoming connections. This is no longer necessary as of Go 1.12.
Please do file an issue if you run into anything unusual!

Thanks for the heads up @mdlayher!

I'm continuing to work on vsock and am nearly finished making it completely compliant with nettest: even returning net.OpError and the like just like a TCP net.Conn would. This work isn't quite done yet, but I'll be tracking it in https://github.com/mdlayher/vsock/issues/29.
I suspect the code will continue to work fine in most cases, but I'm aiming for parity with stdlib connection types as much as possible.

At this point, I think it's pretty much ready to roll for anything you might need. Let me know if there's anything missing!

@mdlayher sounds good, hopefully we'll have a PR soon to update the vendoring. Unless you want to push it :)

I'm happy to do so, just wasn't sure what your procedure and testing policy was for upgrading dependencies.

Nice! You can refer to the following documentation on how to run dep https://github.com/kata-containers/community/blob/master/VENDORING.md
Our CI should catch any issue related to the update of this vendoring.


Add a sample app for the micrometer/Stackdriver monitoring integration

Include an integration test for the sample app.
Follow up to #2391.

I think this issue can be closed.

Yes! Fixed in #2455.


Add action and setter methods that take single view

This extends the concept of Action and Setter and make them available to single views while maintaining the same interface for actual action

Why waste the list and loop instead of just calling the action or setter
directly with the one view?
On Mon, Sep 21, 2015, 9:41 PM Or Bar<EMAIL_ADDRESS>wrote:

This extends the concept of Action and Setter and make them available to
single views while maintaining the same interface for actual action
You can view, comment on, or merge this pull request online at:
https://github.com/JakeWharton/butterknife/pull/357
Commit Summary

Add action and setter methods that take single view
use Arrays.asList
remove white line

File Changes

M butterknife/src/main/java/butterknife/ButterKnife.java
https://github.com/JakeWharton/butterknife/pull/357/files#diff-0
(19)
M butterknife/src/test/java/butterknife/ButterKnifeTest.java
https://github.com/JakeWharton/butterknife/pull/357/files#diff-1
(26)

Patch Links:

https://github.com/JakeWharton/butterknife/pull/357.patch
https://github.com/JakeWharton/butterknife/pull/357.diff

—
Reply to this email directly or view it on GitHub
https://github.com/JakeWharton/butterknife/pull/357.

The intention was to not waste the index, but with one view it's meaningless anyway

butterknife/src/main/java/butterknife/ButterKnife.java:12:8: Unused import - java.util.ArrayList.
butterknife/src/main/java/butterknife/ButterKnife.java:400: Line is longer than 100 characters.

Squashed and merged. Thanks!


签署CLA

如何签署

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


How To Set Measurement Period?

For my local testing I'm creating a MeasureEvaluation using my own dataProvider and setting the Measurement Period as below. When the Context.isType compares my class java.util.Date with class org.opencds.cqf.cql.runtime.DateTime it fails.
		Interval period = new Interval(Population.getReportStartDate().getTime(), true,
				Population.getReportEndDate().getTime(), true);
		cxt.setParameter(null, "Measurement Period", period);
		MeasureReport report = new MeasureEvaluation(dataProvider, period).evaluatePatientMeasure(measure, cxt,
				patient.getResourceId());

I tried using DateTime like in FHIRMeasureResourceProvider but that blows up somewhere else because DateTimes can't be cast to Dates.
        context.setParameter(
                null, "Measurement Period",
                new Interval(
                        DateTime.fromJavaDate((Date) measurementPeriod.getStart()), true,
                        DateTime.fromJavaDate((Date) measurementPeriod.getEnd()), true
                )
        );

When you set the context, that should be a DateTime, but the Interval passed into the MeasureEvaluation class should be a Java Date Interval. So this is what you want:
// Assuming these are Java Dates
Interval period = new Interval(Population.getReportStartDate().getTime(), true,
				Population.getReportEndDate().getTime(), true);
context.setParameter(
                null, "Measurement Period",
                new Interval(
                        DateTime.fromJavaDate((Date) period.getStart()), true,
                        DateTime.fromJavaDate((Date) period.getEnd()), true
                )
        );
MeasureReport report = new MeasureEvaluation(dataProvider, period).evaluatePatientMeasure(measure, cxt,
				patient.getResourceId());

Confusing but it does work. Thanks!

No problem. Apologies for the confusion. The CQL (context parameter) requires DateTime types where HAPI (what I am using to build the report) requires Java Date types to build the MeasureReport.period.


fix: make prisma engine exit cleanly

Fixes ENG-511
Fixes ENG-740
Fixes ENG-929

@jensneuse Wait a bit. I need to test on Windows.

@jensneuse Changed syscall.Kill() to os.Process.Kill(), which provides a better implementation on Windows. Good to go! Thanks for the review 👍


How to prevent default html reponse on api only project errors?

I'm starting an api only project with adonis, but every error returns as html content instead of json. For example, when a route is protected with the auth middleware and the token isn't passed, it returns the InvalidJwtToken error html page.
How can I prevent this behavior and return just json or avoid the default validation and check on my own to return json?

AdonisJs in built error handling respects the content negotiation headers like Accept.
If you will set the Accept: application/json header, then all error output will be in JSON over HTML.
Another thing you can do is create a global middleware, which set this header for you, this way your clients won't have to set the header explicitly.
Sample code for the middleware
class SpoofAccept {
  async handle ({ request }, next) {
    request.request.headers['accept'] = 'application/json'
    await next()
  }
}

module.exports = SpoofAccept

Closing since no answer from issue reporter.


Distinguish between apps sets and apps in apps picker

Closed by https://github.com/nuvla/ui/pull/1439


Document CRD considerations when using helm

Attempting to clarify the available options for managing cert-manager CRDs and how they differ to standard helm method. I did add a recommendation for the user which I am unsure if that is the "right" thing to do here. My default is to be secure and managing CRDs outside of helm gives the most safety in that you shouldn't ever have a scenario where helm removes those CRDs causing issues.
All that said, the --set installCRDs=true option is very convenient and I use it daily. It seems most of the users I asked use this option and don't have concerns. So the intent here is just to highlight the considerations and give users an idea about which method they should choose. Ultimately this is down to each user of course and people can do what they like.

/assign @maelvls

/cc @maelvls


Release v1.3 Blazemeter Repository

Fix issue that couldn't provide responseFilters, components and dependencies in templates
Update BZ SDK to comply with Blazemeter API

👀


[Accessibility]A11y_VSCode_ViewExtensions_Screenreader:Voiceover does not read the content written under treeview.

Environment Details:
VSCode Version : 1.24.1
OS Version :MAS1.3.1,MACOS High Sierra 10.13.5 Beta (17F70a)
Additional Details:
Screenreader,Voiceover
Repro Steps:

Launch VS Code.
Navigate to View(Alt+V) menu option and select "Extensions" from dropdown.
Navigate to tree view "Installed" and "Recommendation" under the search extension.
Check what Voiceover is reading.

Actual:
Voiceover does not read the  content written under the treeview.
Expected:
Voiceover should read the  content which is present on the screen.
Recommendations:
Refer below link which is repository of bug fixes code snippets:
https://microsoft.sharepoint.com/teams/msenable/mas/pages/browse-fixes.aspx
User Impact:
User will not able to know which content is written on the screen.
MAS Reference:
https://microsoft.sharepoint.com/:w:/r/teams/msenable/_layouts/15/WopiFrame.aspx?sourcedoc={54f28d1f-a2d1-4dcd-84e1-5c9b87e8aba4}
Does this issue occur when all extensions are disabled?: Yes
Attachment for Reference
[Accessibility]A11y_VSCode_ViewExtensions_ScreenReader_Voiceover.pptx

\duplicate #51968

/duplicate #51968


release 6.19.0

so, this is "the real deal", so, if required, @camunda-community-hub/devrel  help me getting this to central. Thanks!

Hi @jangalinski,
the nexus release is on its way.
You can add auto-releasing to your github workflow: https://github.com/camunda-community-hub/community-action-maven-release/blob/main/example-workflows/java11/deploy.yml#L37


Volunteer count is showing negative when oversubscribing task

While technically correct, it is confusing to show negative needed volunteers:

If the task is fully allocated, we should show "0 volunteers needed".

@phillipoertel is cleaning this up in his feature work already...

Works correctly in new sourcing approach now...


adding selenium

Added these following things -
1 - CACert - This file needs to be imported within the chromedriver, prior to using Selenium Wire, so that it shuts down the CACert not found Warning.
2 - Selenium + Selenium Wire - Main backend for opening the pages, and scraping the content through.
Work in Progress

[ ]  To add headless mode
[ ]  To experiment with Wait Times
[ ]  To properly implement NamedTeporaryFile
[ ]  Make the Code PEP-8 / simplify code.

Closing this again.


torch multiprocessing api failed

Hi I encounter the following error when I finetune eva2.0-xLarge with 4 v100 GPU.
building Enc-Dec model ...
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 15 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 16 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 17 closing signal SIGTERM
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -9) local_rank: 0 (pid: 14) of binary: /opt/conda/bin/python
Traceback (most recent call last):
File "/opt/conda/bin/torchrun", line 8, in
sys.exit(main())
File "/home/weiyihao/.local/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/errors/init.py", line 345, in wrapper
return f(*args, **kwargs)
File "/home/weiyihao/.local/lib/python3.8/site-packages/torch/distributed/run.py", line 724, in main
run(args)
File "/home/weiyihao/.local/lib/python3.8/site-packages/torch/distributed/run.py", line 715, in run
elastic_launch(
File "/home/weiyihao/.local/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/home/weiyihao/.local/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 245, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
I was able to run the interactive mode on 1 P100 GPU without error. Any clue what is causing the error?

hi, could you please show the full log output?

@xwwwwww thanks for your reply. Here is the full log:
WARNING:torch.distributed.run:

Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.

using world size: 6 and model-parallel size: 2

using dynamic loss scaling
[2022-06-17 03:25:20,845] [INFO] [distributed.py:37:init_distributed] Initializing torch distributed with backend: nccl
[2022-06-17 03:25:20,887] [INFO] [distributed.py:37:init_distributed] Initializing torch distributed with backend: nccl
[2022-06-17 03:25:20,896] [INFO] [distributed.py:37:init_distributed] Initializing torch distributed with backend: nccl
[2022-06-17 03:25:20,903] [INFO] [distributed.py:37:init_distributed] Initializing torch distributed with backend: nccl
[2022-06-17 03:25:20,911] [INFO] [distributed.py:37:init_distributed] Initializing torch distributed with backend: nccl
[2022-06-17 03:25:20,913] [INFO] [distributed.py:37:init_distributed] Initializing torch distributed with backend: nccl
initializing model parallel with size 2
[2022-06-17 03:25:20,968] [INFO] [checkpointing.py:629:_configure_using_config_file] {'partition_activations': False, 'contiguous_memory_optimization': False, 'cpu_checkpointing': False, 'number_checkpoints': None, 'synchronize_checkpoint_boundary': False, 'profile': False}
[2022-06-17 03:25:20,968] [INFO] [checkpointing.py:629:_configure_using_config_file] {'partition_activations': False, 'contiguous_memory_optimization': False, 'cpu_checkpointing': False, 'number_checkpoints': None, 'synchronize_checkpoint_boundary': False, 'profile': False}
Pretrain Enc-Dec model
arguments:
model_config ................. /mnt/user/weiyihao/EVA-main/src/configs/model/eva2.0_model_config.json
model_parallel_size .......... 2
fp16 ......................... True
do_train ..................... True
do_valid ..................... True
do_eval ...................... True
train_ratio .................. 1.0
valid_ratio .................. 1
test_ratio ................... 1
batch_size ................... 16
gradient_accumulation_steps .. 1
train_iters .................. -1
epochs ....................... 5
weight_decay ................. 0.01
checkpoint_activations ....... True
checkpoint_num_layers ........ 1
deepspeed_activation_checkpointing  True
clip_grad .................... 1.0
seed ......................... 422
lr_decay_style ............... noam
lr ........................... 0.0001
warmup ....................... 0.01
load ......................... /mnt/user/weiyihao/EVA-main/checkpoints/eva2.0_xLarge
load_optimizer_states ........ False
load_lr_scheduler_states ..... False
no_load_strict ............... True
save ......................... /mnt/user/weiyihao/EVA-main/results/eva2.0-xLarge/finetune2
save_interval ................ 1000
log_file ..................... /mnt/user/weiyihao/EVA-main/results/eva2.0-xLarge/finetune2/log.txt
log_interval ................. 100
distributed_backend .......... nccl
local_rank ................... 0
eval_batch_size .............. None
eval_interval ................ 1000
eval_generation .............. False
temperature .................. 0.9
top_p ........................ 0.9
top_k ........................ 0
max_generation_length ........ 128
min_generation_length ........ 2
num_beams .................... 1
no_repeat_ngram_size ......... 3
repetition_penalty ........... 1.2
early_stopping ............... False
length_penalty ............... 1.8
rule_path .................... None
data_path .................... /mnt/user/weiyihao/EVA-main/data2/
cache_path ................... None
tokenizer_path ............... /mnt/user/weiyihao/EVA-main/bpe_dialog_new
data_ext ..................... .txt
num_workers .................. 2
enc_seq_length ............... 256
dec_seq_length ............... 128
deepspeed .................... True
deepspeed_config ............. /mnt/user/weiyihao/EVA-main/src/configs/deepspeed/eva_ds_config.json
deepscale .................... False
deepscale_config ............. None
deepspeed_mpi ................ False
cuda ......................... True
rank ......................... 0
world_size ................... 6
dynamic_loss_scale ........... True
[2022-06-17 03:25:20,970] [INFO] [checkpointing.py:629:_configure_using_config_file] {'partition_activations': False, 'contiguous_memory_optimization': False, 'cpu_checkpointing': False, 'number_checkpoints': None, 'synchronize_checkpoint_boundary': False, 'profile': False}
[2022-06-17 03:25:20,972] [INFO] [checkpointing.py:629:_configure_using_config_file] {'partition_activations': False, 'contiguous_memory_optimization': False, 'cpu_checkpointing': False, 'number_checkpoints': None, 'synchronize_checkpoint_boundary': False, 'profile': False}
[2022-06-17 03:25:20,977] [INFO] [checkpointing.py:629:_configure_using_config_file] {'partition_activations': False, 'contiguous_memory_optimization': False, 'cpu_checkpointing': False, 'number_checkpoints': None, 'synchronize_checkpoint_boundary': False, 'profile': False}
[2022-06-17 03:25:20,982] [INFO] [checkpointing.py:629:_configure_using_config_file] {'partition_activations': False, 'contiguous_memory_optimization': False, 'cpu_checkpointing': False, 'number_checkpoints': None, 'synchronize_checkpoint_boundary': False, 'profile': False}
[2022-06-17 03:25:20,989] [INFO] [checkpointing.py:248:model_parallel_cuda_manual_seed] > initializing model parallel cuda seeds on global rank 0, model parallel rank 0, and data parallel rank 0 with model parallel seed: 3140 and data parallel seed: 422
No cache, processing data
Building prefix dict from the default dictionary ...
Loading model from cache /tmp/jieba.cache
Building prefix dict from the default dictionary ...
Loading model from cache /tmp/jieba.cache
Building prefix dict from the default dictionary ...
Loading model from cache /tmp/jieba.cache
Building prefix dict from the default dictionary ...
Loading model from cache /tmp/jieba.cache

Loading data from /mnt/user/weiyihao/EVA-main/data2/train.txt:   0%|          | 0/37729 [00:00<?, ?it/s]Building prefix dict from the default dictionary ...
Loading model from cache /tmp/jieba.cache
Building prefix dict from the default dictionary ...
Loading model from cache /tmp/jieba.cache
Dumping model to file cache /tmp/jieba.cache
Dumping model to file cache /tmp/jieba.cache
Dumping model to file cache /tmp/jieba.cache
Dumping model to file cache /tmp/jieba.cache
Dump cache file failed.
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/site-packages/jieba/init.py", line 154, in initialize
_replace_file(fpath, cache_file)
PermissionError: [Errno 1] Operation not permitted: '/tmp/tmp5h9gvay8' -> '/tmp/jieba.cache'
Loading model cost 2.320 seconds.
Prefix dict has been built successfully.
Dumping model to file cache /tmp/jieba.cache
Dump cache file failed.
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/site-packages/jieba/init.py", line 154, in initialize
_replace_file(fpath, cache_file)
PermissionError: [Errno 1] Operation not permitted: '/tmp/tmpy5ida7h1' -> '/tmp/jieba.cache'
Loading model cost 2.406 seconds.
Prefix dict has been built successfully.
Dumping model to file cache /tmp/jieba.cache
Loading data from /mnt/user/weiyihao/EVA-main/data2/train.txt:   0%|          | 1/37729 [00:02<25:13:41,  2.41s/it]Dump cache file failed.
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/site-packages/jieba/init.py", line 154, in initialize
_replace_file(fpath, cache_file)
PermissionError: [Errno 1] Operation not permitted: '/tmp/tmp2osfm1gz' -> '/tmp/jieba.cache'
Loading model cost 2.570 seconds.
Prefix dict has been built successfully.
Dump cache file failed.
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/site-packages/jieba/init.py", line 154, in initialize
_replace_file(fpath, cache_file)
PermissionError: [Errno 1] Operation not permitted: '/tmp/tmpp7s06t9k' -> '/tmp/jieba.cache'
Loading model cost 2.602 seconds.
Prefix dict has been built successfully.
Loading data from /mnt/user/weiyihao/EVA-main/data2/train.txt:   0%|          | 14/37729 [00:02<17:40:42,  1.69s/it]Dump cache file failed.
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/site-packages/jieba/init.py", line 154, in initialize
_replace_file(fpath, cache_file)
PermissionError: [Errno 1] Operation not permitted: '/tmp/tmp0rz_t63a' -> '/tmp/jieba.cache'
Loading model cost 2.580 seconds.
Prefix dict has been built successfully.
Dump cache file failed.
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/site-packages/jieba/init.py", line 154, in initialize
_replace_file(fpath, cache_file)
PermissionError: [Errno 1] Operation not permitted: '/tmp/tmp8b7xz6ug' -> '/tmp/jieba.cache'
Loading model cost 2.694 seconds.
Prefix dict has been built successfully.
Loading data from /mnt/user/weiyihao/EVA-main/data2/train.txt: 100%|██████████| 37729/37729 [04:54<00:00, 128.23it/s]
Cache path is None, no cache saved
Path: /mnt/user/weiyihao/EVA-main/data2/train.txt | Ratio:1.0 | Max enc len: 256 | Max dec len: 128 | Data num: 37053
No cache, processing data
Loading data from /mnt/user/weiyihao/EVA-main/data2/valid.txt: 100%|██████████| 5992/5992 [01:00<00:00, 98.87it/s]
Cache path is None, no cache saved
Path: /mnt/user/weiyihao/EVA-main/data2/valid.txt | Ratio:1 | Max enc len: 256 | Max dec len: 128 | Data num: 5761
Total train epochs 5 | Total train iters 3859 |
building Enc-Dec model ...
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 14 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 15 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 17 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 18 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 19 closing signal SIGTERM
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -9) local_rank: 2 (pid: 16) of binary: /opt/conda/bin/python
Traceback (most recent call last):
File "/opt/conda/bin/torchrun", line 8, in
sys.exit(main())
File "/home/weiyihao/.local/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/errors/init.py", line 345, in wrapper
return f(*args, **kwargs)
File "/home/weiyihao/.local/lib/python3.8/site-packages/torch/distributed/run.py", line 724, in main
run(args)
File "/home/weiyihao/.local/lib/python3.8/site-packages/torch/distributed/run.py", line 715, in run
elastic_launch(
File "/home/weiyihao/.local/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/home/weiyihao/.local/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 245, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
/mnt/user/weiyihao/EVA-main/src/eva_finetune.py FAILED
Failures:
<NO_OTHER_FAILURES>
Root Cause (first observed failure):
[0]:
time      : 2022-06-17_03:32:24
host      : iZ2zecyh456naae68lp3swZ
rank      : 2 (local_rank: 2)
exitcode  : -9 (pid: 16)
error_file: <N/A>
traceback : Signal 9 (SIGKILL) received by PID 16

PermissionError: [Errno 1] Operation not permitted: '/tmp/tmp5h9gvay8' -> '/tmp/jieba.cache'
please fix this first

PermissionError: [Errno 1] Operation not permitted: '/tmp/tmp5h9gvay8' -> '/tmp/jieba.cache'
please fix this first

I don't think it's the cause. The permission issue happened when I run interactive mode with a single GPU, but it is still able to proceed to build the model and etc.. The above error happened only when running in a distributed setting, and it seems there is no clue in the log to pinpoint the exact issue.

Update: The issue is solved. It was caused by RAM not enough during building the model.


GUID alias support

Feature Request
Desired behaviour
See https://github.com/ga4gh/tool-registry-service-schemas/issues/38
This is a bundle of a few issues

it should be possible to add an alias to tools and workflows in Dockstore (start with CLI, then GUI)
those aliases should be displayed as an array in TRS
aliases should probably be elastic search searchable and displayed in GUI2
aliases should be trivially searchable using the TRS endpoint

┆Issue is synchronized with this Jira Story
┆Issue Type: Story
┆Fix Versions: Dockstore 1.9
┆Sprint: Backlog
┆Issue Number: DOCK-342
┆Epic: Create DOIs

Adding works with Dockstore PUT endpoint, retrieving works with TRS endpoint
Doesn't appear in an obvious manner in UI2 or CLI

I think that's probably enough for 1.5.0 albeit with documentation.
Punting the rest of the list to 1.6.0

Added some docs in https://github.com/ga4gh/dockstore/pull/1786.

Same state as previous comment.


Vacuum for redis broken

As the title suggests, my redis instance ramps up memory and it does not get vacuumed. I have tested it locally and it produces about 1MB of data every 5 minutes and it does not get vacuumed. I cannot provide a reproduction but can try to make one upon request.

Vacuuming is not done automatically. Consider setting up a cron or interval and spawn that task on the side.
The vacuum method exists in the storage but you have to decide how often it happens.

Sorry, I should have been clearer. I troubleshooted it yesterday and it is the result of it only cleaning dead jobs and not done jobs

Thats possibly a bug then, let me try to replicate.

It would probably be worth documenting that vacuuming doesn't happen automatically. For some time I wondered about memory growth but it was manageable so I did not prioritize looking into it but now it makes sense.


not support packages that have a scope

If the package has a scope like @scope/package-name, it will generate scripts like
module.exports = @scope/package-name;

which will raise errors.

@njzjz thanks for reporting, can you please post that error so we have a reference of it and come up with a possible fix.

The webpack-cdn-plugin raises no errors, but the scripts it generates should be
module.exports = "@scope/package-name";

It is wrong if it does not has ".

Oh I see, I can manually assign the name.


Mobile/Web version - bugs

About US page - Join us section.
In the mobile version there is no padding and the text sticks to edges.

Underlined text should be consistent across the website. Example:

How in looks in the mockups:

It applies to both versions (mobile and web).
In the mobile version some titles doesn't have an underline (for example, About us page).

Make DIF logo (Partners section) be not colored, but in a black/white stile to keep it consistent with other logos

1,3 were fixed before

perfect! thanks a lot!


feat: add rocksdb block_size to config

Summary of changes
Changes introduced in this pull request:

add rocksdb block_size to config
sets default block_size to 2KB according to metrics from https://github.com/ChainSafe/forest/pull/2202 This PR just uses default block size
Slightly improve DB exists and get API by using key_may_exist

@elmattic FYI DB size remains the same on calibnet
DB size from CI job: 242.7MB
Previous DB size from CI job: 242.7MB
Ref: wiki

block_size -- RocksDB packs user data in blocks. When reading a key-value pair from a table file, an entire block is loaded into memory. Block size is 4KB by default. Each table file contains an index that lists offsets of all blocks. Increasing block_size means that the index contains fewer entries (since there are fewer blocks per file) and is thus smaller. Increasing block_size decreases memory usage and space amplification, but increases read amplification.

Reference issue to close (if applicable)

Closes
Other information and links

@elmattic FYI DB size remains the same on calibnet

What about mainnet db size? Did overall Get latency improved?
From my experience it's not easy to reason on performances using calibnet, since most of the data will fit L1 and L2.

What about mainnet db size?

I will update numbers on mainnet later, given that both 4KB and 2KB are larger than most (P95) of the values we store, my assumption is this change has no impact on any of read amplification, write amplification, and space amplification.  The only thing it would improve is cache memory usage.

The downside of using large blocks is that RAM is wasted in the block cache.

I don't think we should merge this until we have concrete evidence of the benefits. Let's measure if it really decreases memory usage.

Any performance gains due to usage of key_may_exist function

@elmattic It makes read performance much worse, by playing with the bench funtion here. Should revert the change

@hanabi1224 What do we do with this? Is this a draft?

@LesnyRumcajs I have repurposed this PR to not include the block_size change so now it only exposed it to the config.  It should be safe to merge but I'm not sure if we still need this as we might want to migrate to other DB soon.
Regarding numbers, I don't see any notable change in DB size or import speed by change block size to 2KB

@LesnyRumcajs I have repurposed this PR to not include the block_size change so now it only exposed it to the config. It should be safe to merge but I'm not sure if we still need this as we might want to migrate to other DB soon.
Regarding numbers, I don't see any notable change in DB size or import speed by change block size to 2KB

If it doesn't have an effect on performance then I don't think we should offer it as an option. I would be quite surprised if it would be beneficial to change the default block size in the future.

Shall we close it then?


otel-collector + AWS X-Ray: large gaps in tracing blocks

[X] This may affect other libraries, but I would like to get opinions here first

Hey all, I use Opentelemetry on Hapi, and I'm currently doing AWS X Ray via the otel-collector sidecar on ECS. This might be something I need AWS support for, but I wanted to check here first, as I am not sure how to see what I'm missing. Basically, I get these inconsistent untraced segments of my request. This doesn't happen on most requests, but the sampling picks up a few of them about every hour.
What I'm confused about here is that the top level trace from the otel-collector sidecar clocks the request at 2.7 seconds, and then the application level traces only account for maybe 700 ms. So, my confusion is, where are the other 2 seconds going?
If someone has an answer, I'd appreciate the heck if you shared, but I'm also trying to figure out what I can do to get more context here. Also, if I should instead be having this conversation with AWS I will, but publicly having this conversation might benefit the next person in my spot.
Thanks for the awesome libraries! You guys are amazing

Basically, I get these inconsistent untraced segments of my request

That made me think of a similar issue we faced.
In our case, we were using a remote sampler and we were sampling spans, that's why some spans would appear and others would not.
We then moved to sample all spans if a trace is sampled, the way we achieved that in go was like this:
sdktrace.WithSampler(sdktrace.ParentBased(sampler)),

Not sure if this is of help, but worth a try.

@toddtarsi does the 👍 mean this helped you solve your problem? If so can you close this or comment?

@dyladan - Not sure yet, but I am extremely appreciative of a helpful suggestion like this, and I'm definitely going to implement my sampling rules more deliberately. Sorry to leave this open here.

No worries I just wanted to make sure the status was clear


Install postgis 2.4 for postgresql 9.5

DOPS-121

should not be done in this repo... as it's not wide required extension


Non-zero tcp re-transmission rate from a cilium pod

I have two physical machines. One (lets call it A) is running the iperf3 server. The other (called B) is running Kubernetes pods using cilium networking. If I run a performance test directly on the host on B, I don't see a high re-transmission rate. If I run the same test from inside of a container on Host B then I see a re-transmision rate which seems to affect the bandwidth.

What kernel are you running?
Can you provide a pcap sample?
Can you send a sample of cilium monitor?

I can answer some of those questions right now, I'm working on the rest.

What version of cilium are you running?

Cilium 0.9.90 9f093ac Sun, 11 Jun 2017 07:16:24 +0200 go version go1.8.3 linux/amd64

Are you using IPv6 in the host test as well?
Yes, IPv6 everywhere.
What kernel are you running?
4.9.30
What container image is being used to run iperf?
Based off ubuntu ubuntu@sha256:ea1d854d38be82f54d39efe2c67000bed1b03348bcc2f3dc094f260855dff368, with some networking tools installed including iperf.

(Have it reproduced locally as well.)

Some info for myself.
Host to host (w/o container):
# iperf3 -c f00e::1 -t 60
Connecting to host f00e::1, port 5201
[  4] local f00e::2 port 57002 connected to f00e::1 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  1.07 GBytes  9.23 Gbits/sec   23    629 KBytes
[  4]   1.00-2.00   sec  1.08 GBytes  9.28 Gbits/sec    0    636 KBytes
[  4]   2.00-3.00   sec  1.08 GBytes  9.29 Gbits/sec    0    637 KBytes
[  4]   3.00-4.00   sec  1.08 GBytes  9.28 Gbits/sec    0    641 KBytes
[  4]   4.00-5.00   sec  1.08 GBytes  9.28 Gbits/sec    0    644 KBytes
[  4]   5.00-6.00   sec  1.08 GBytes  9.29 Gbits/sec    0    644 KBytes
[  4]   6.00-7.00   sec  1.08 GBytes  9.29 Gbits/sec    0    644 KBytes
[  4]   7.00-8.00   sec  1.08 GBytes  9.28 Gbits/sec    0    644 KBytes
[  4]   8.00-9.00   sec  1.08 GBytes  9.29 Gbits/sec    0    644 KBytes
[  4]   9.00-10.00  sec  1.08 GBytes  9.27 Gbits/sec    0   1.02 MBytes
[  4]  10.00-11.00  sec  1.04 GBytes  8.94 Gbits/sec    0   1.02 MBytes
[  4]  11.00-12.00  sec  1.06 GBytes  9.15 Gbits/sec    0   1.31 MBytes
[  4]  12.00-13.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.31 MBytes
[  4]  13.00-14.00  sec  1.05 GBytes  8.98 Gbits/sec    0   1.31 MBytes
[  4]  14.00-15.00  sec  1.05 GBytes  9.02 Gbits/sec    0   1.31 MBytes
[  4]  15.00-16.00  sec  1.07 GBytes  9.19 Gbits/sec    0   1.99 MBytes
[  4]  16.00-17.00  sec  1.05 GBytes  8.99 Gbits/sec    0   1.99 MBytes
[  4]  17.00-18.00  sec  1.07 GBytes  9.20 Gbits/sec    0   1.99 MBytes
[  4]  18.00-19.00  sec  1.05 GBytes  9.03 Gbits/sec    9   1.39 MBytes
[  4]  19.00-20.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  20.00-21.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  21.00-22.00  sec  1.06 GBytes  9.09 Gbits/sec    0   1.39 MBytes
[  4]  22.00-23.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  23.00-24.00  sec  1.06 GBytes  9.11 Gbits/sec    0   1.39 MBytes
[  4]  24.00-25.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  25.00-26.00  sec  1.06 GBytes  9.14 Gbits/sec    0   1.39 MBytes
[  4]  26.00-27.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  27.00-28.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  28.00-29.00  sec  1.07 GBytes  9.17 Gbits/sec    0   1.39 MBytes
[  4]  29.00-30.00  sec  1.04 GBytes  8.94 Gbits/sec    0   1.39 MBytes
[  4]  30.00-31.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  31.00-32.00  sec  1.07 GBytes  9.15 Gbits/sec    0   1.39 MBytes
[  4]  32.00-33.00  sec  1.08 GBytes  9.29 Gbits/sec    0   1.39 MBytes
[  4]  33.00-34.00  sec  1.08 GBytes  9.29 Gbits/sec    0   1.39 MBytes
[  4]  34.00-35.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  35.00-36.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  36.00-37.00  sec  1.08 GBytes  9.27 Gbits/sec    0   1.39 MBytes
[  4]  37.00-38.00  sec  1.04 GBytes  8.95 Gbits/sec    0   1.39 MBytes
[  4]  38.00-39.00  sec  1.08 GBytes  9.29 Gbits/sec    0   1.39 MBytes
[  4]  39.00-40.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  40.00-41.00  sec  1.07 GBytes  9.20 Gbits/sec    0   1.39 MBytes
[  4]  41.00-42.00  sec  1.04 GBytes  8.97 Gbits/sec    0   1.39 MBytes
[  4]  42.00-43.00  sec  1.08 GBytes  9.29 Gbits/sec    0   1.39 MBytes
[  4]  43.00-44.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  44.00-45.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  45.00-46.00  sec  1.08 GBytes  9.29 Gbits/sec    0   1.39 MBytes
[  4]  46.00-47.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  47.00-48.00  sec  1.08 GBytes  9.29 Gbits/sec    0   1.39 MBytes
[  4]  48.00-49.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  49.00-50.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  50.00-51.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  51.00-52.00  sec  1.08 GBytes  9.29 Gbits/sec    0   1.39 MBytes
[  4]  52.00-53.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  53.00-54.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  54.00-55.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  55.00-56.00  sec  1.08 GBytes  9.29 Gbits/sec    0   1.39 MBytes
[  4]  56.00-57.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  57.00-58.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  58.00-59.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.39 MBytes
[  4]  59.00-60.00  sec  1.08 GBytes  9.28 Gbits/sec    0   1.85 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-60.00  sec  64.4 GBytes  9.22 Gbits/sec   32             sender
[  4]   0.00-60.00  sec  64.4 GBytes  9.22 Gbits/sec                  receiver

iperf Done.

# tc -s qdisc show dev enp1s0f0 | grep requeues
 Sent<PHONE_NUMBER>2 bytes 48446901 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 Sent<PHONE_NUMBER> bytes 3265306 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 Sent 65205812 bytes 43128 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 Sent<PHONE_NUMBER>5 bytes 21893267 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 Sent 3842754 bytes 2569 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 Sent<PHONE_NUMBER> bytes 6134845 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 Sent<PHONE_NUMBER>5 bytes 17106637 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 Sent 994784 bytes 658 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 Sent 686106 bytes 491 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0

Container to host:
# docker exec -i client iperf3 -c f00e::1 -t 60
Connecting to host f00e::1, port 5201
[  4] local f00e::c0a8:17d:0:71d4 port 37128 connected to f00e::1 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  1.04 GBytes  8.97 Gbits/sec  2752    577 KBytes
[  4]   1.00-2.00   sec  1.05 GBytes  9.06 Gbits/sec  467    737 KBytes
[  4]   2.00-3.00   sec  1.02 GBytes  8.79 Gbits/sec  761    563 KBytes
[  4]   3.00-4.00   sec  1.01 GBytes  8.64 Gbits/sec  811    571 KBytes
[  4]   4.00-5.00   sec  1.05 GBytes  9.06 Gbits/sec  1040    682 KBytes
[  4]   5.00-6.00   sec  1.06 GBytes  9.07 Gbits/sec  798    764 KBytes
[  4]   6.00-7.00   sec  1.03 GBytes  8.85 Gbits/sec  1101    676 KBytes
[  4]   7.00-8.00   sec  1.08 GBytes  9.26 Gbits/sec    0   1.39 MBytes
[  4]   8.00-9.00   sec  1.05 GBytes  9.05 Gbits/sec  1059    713 KBytes
[  4]   9.00-10.00  sec  1.01 GBytes  8.71 Gbits/sec  1112    575 KBytes
[  4]  10.00-11.00  sec  1.04 GBytes  8.93 Gbits/sec  738    781 KBytes
[  4]  11.00-12.00  sec  1.04 GBytes  8.94 Gbits/sec  616   1.04 MBytes
[  4]  12.00-13.00  sec  1.02 GBytes  8.76 Gbits/sec  2415    410 KBytes
[  4]  13.00-14.00  sec  1.07 GBytes  9.15 Gbits/sec  157   1.23 MBytes
[  4]  14.00-15.00  sec  1.08 GBytes  9.26 Gbits/sec    0   1.75 MBytes
[  4]  15.00-16.00  sec  1.03 GBytes  8.84 Gbits/sec  2412    833 KBytes
[  4]  16.00-17.00  sec  1.05 GBytes  9.04 Gbits/sec  1869    681 KBytes
[  4]  17.00-18.00  sec  1.04 GBytes  8.92 Gbits/sec  1366    607 KBytes
[  4]  18.00-19.00  sec  1006 MBytes  8.44 Gbits/sec  2712    420 KBytes
[  4]  19.00-20.00  sec  1021 MBytes  8.57 Gbits/sec  797    697 KBytes
[  4]  20.00-21.00  sec  1.05 GBytes  9.05 Gbits/sec  1399    949 KBytes
[  4]  21.00-22.00  sec  1.08 GBytes  9.26 Gbits/sec    0   1.55 MBytes
[  4]  22.00-23.00  sec  1.06 GBytes  9.09 Gbits/sec  1247    798 KBytes
[  4]  23.00-24.00  sec  1.06 GBytes  9.09 Gbits/sec  875    545 KBytes
[  4]  24.00-25.00  sec  1.06 GBytes  9.08 Gbits/sec  447    789 KBytes
[  4]  25.00-26.00  sec  1.04 GBytes  8.93 Gbits/sec  614    666 KBytes
[  4]  26.00-27.00  sec  1.06 GBytes  9.07 Gbits/sec  1734    690 KBytes
[  4]  27.00-28.00  sec  1.06 GBytes  9.13 Gbits/sec  802    754 KBytes
[  4]  28.00-29.00  sec  1.05 GBytes  9.03 Gbits/sec  768    764 KBytes
[  4]  29.00-30.00  sec  1.05 GBytes  9.05 Gbits/sec  694    731 KBytes
[  4]  30.00-31.00  sec  1.05 GBytes  9.03 Gbits/sec  1063    791 KBytes
[  4]  31.00-32.00  sec  1.05 GBytes  9.05 Gbits/sec  625   1017 KBytes
[  4]  32.00-33.00  sec  1.03 GBytes  8.87 Gbits/sec  2018    420 KBytes
[  4]  33.00-34.00  sec  1.05 GBytes  9.00 Gbits/sec  656   1.11 MBytes
[  4]  34.00-35.00  sec  1005 MBytes  8.43 Gbits/sec  3386    420 KBytes
[  4]  35.00-36.00  sec  1.03 GBytes  8.82 Gbits/sec  786    401 KBytes
[  4]  36.00-37.00  sec  1.05 GBytes  8.98 Gbits/sec  725    778 KBytes
[  4]  37.00-38.00  sec  1.07 GBytes  9.21 Gbits/sec  658   1.12 MBytes
[  4]  38.00-39.00  sec  1.08 GBytes  9.26 Gbits/sec    0   1.68 MBytes
[  4]  39.00-40.00  sec  1.06 GBytes  9.10 Gbits/sec    0   1.99 MBytes
[  4]  40.00-41.00  sec  1.03 GBytes  8.87 Gbits/sec   57   1.59 MBytes
[  4]  41.00-42.00  sec  1.06 GBytes  9.12 Gbits/sec  1274   1.01 MBytes
[  4]  42.00-43.00  sec  1.07 GBytes  9.23 Gbits/sec  848   1.22 MBytes
[  4]  43.00-44.00  sec  1.08 GBytes  9.24 Gbits/sec    0   1.75 MBytes
[  4]  44.00-45.00  sec  1.03 GBytes  8.88 Gbits/sec    0   1.89 MBytes
[  4]  45.00-46.00  sec  1.07 GBytes  9.19 Gbits/sec    0   2.23 MBytes
[  4]  46.00-47.00  sec  1002 MBytes  8.41 Gbits/sec  1294    431 KBytes
[  4]  47.00-48.00  sec  1.05 GBytes  9.03 Gbits/sec  1076    911 KBytes
[  4]  48.00-49.00  sec  1.04 GBytes  8.95 Gbits/sec  789   1.06 MBytes
[  4]  49.00-50.00  sec  1.03 GBytes  8.83 Gbits/sec  607    993 KBytes
[  4]  50.00-51.00  sec  1.05 GBytes  9.03 Gbits/sec  1419    588 KBytes
[  4]  51.00-52.00  sec  1.03 GBytes  8.81 Gbits/sec  1422    589 KBytes
[  4]  52.00-53.00  sec  1.05 GBytes  9.05 Gbits/sec  594   1.15 MBytes
[  4]  53.00-54.00  sec  1.03 GBytes  8.87 Gbits/sec    0   1.37 MBytes
[  4]  54.00-55.00  sec  1.08 GBytes  9.26 Gbits/sec    0   1.84 MBytes
[  4]  55.00-56.00  sec  1.08 GBytes  9.26 Gbits/sec    0   2.23 MBytes
[  4]  56.00-57.00  sec  1.08 GBytes  9.26 Gbits/sec    0   2.56 MBytes
[  4]  57.00-58.00  sec  1.04 GBytes  8.90 Gbits/sec  3252    409 KBytes
[  4]  58.00-59.00  sec  1001 MBytes  8.40 Gbits/sec  528    561 KBytes
[  4]  59.00-60.00  sec   998 MBytes  8.37 Gbits/sec  1355    564 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-60.00  sec  62.6 GBytes  8.96 Gbits/sec  55995             sender
[  4]   0.00-60.00  sec  62.6 GBytes  8.96 Gbits/sec                  receiver

iperf Done.

# tc -s qdisc show dev enp1s0f0 | grep requeues
 Sent<PHONE_NUMBER>1 bytes 56881513 pkt (dropped 70, overlimits 0 requeues 10094)
 backlog 0b 0p requeues 10094
 Sent<PHONE_NUMBER>2 bytes 7774287 pkt (dropped 30, overlimits 0 requeues 47)
 backlog 0b 0p requeues 47
 Sent<PHONE_NUMBER>6 bytes 12285016 pkt (dropped 40, overlimits 0 requeues 347)
 backlog 0b 0p requeues 347
 Sent<PHONE_NUMBER> bytes 4186166 pkt (dropped 0, overlimits 0 requeues 41)
 backlog 0b 0p requeues 41
 Sent<PHONE_NUMBER>2 bytes 20177249 pkt (dropped 0, overlimits 0 requeues 4445)
 backlog 0b 0p requeues 4445
 Sent<PHONE_NUMBER> bytes 4186755 pkt (dropped 0, overlimits 0 requeues 57)
 backlog 0b 0p requeues 57
 Sent<PHONE_NUMBER> bytes 5552855 pkt (dropped 0, overlimits 0 requeues 5146)
 backlog 0b 0p requeues 5146
 Sent<PHONE_NUMBER> bytes 1774587 pkt (dropped 0, overlimits 0 requeues 9)
 backlog 0b 0p requeues 9
 Sent<PHONE_NUMBER> bytes 944598 pkt (dropped 0, overlimits 0 requeues 2)
 backlog 0b 0p requeues 2

pcap shows a lot of ofo packets from TCP side.
requeues counter indicate that we're hitting dev_requeue_skb() on xmit meaning we got NETDEV_TX_BUSY return code either somewhere from sch_direct_xmit()/dev_hard_start_xmit() or from driver itself.
Driver itself (ixgbe) I would exclude right now since tx_busy count doesn't increase, which it would otherwise from ixgbe_xmit_frame_ring() which is the only location that could return NETDEV_TX_BUSY:
# ethtool -S enp1s0f0 | grep busy
     tx_busy: 0

Turning off GRO/GSO/TSO definitely makes the problem worse and roughly doubles the retrans rates (69719 -> 119905 for a 60sec run). On host -> host stable range of 40-90 retr/sec, and container -> host 1500-2500 retr/sec.
Host facing veth shows that bpf_lxc is not dropping packets during iperf test run (dropped counter here unrelated):
# tc filter show dev lxc1d261 ingress
filter protocol all pref 1 bpf
filter protocol all pref 1 bpf handle 0x1 bpf_lxc.o:[from-container] direct-action
# tc -s qdisc show dev lxc1d261
[...]
qdisc clsact ffff: parent ffff:fff1
 Sent<PHONE_NUMBER>477 bytes 825267238 pkt (dropped 18, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0

All ip6tables rules were flushed to make sure it doesn't mess with us.
Debugging is disabled; drop monitor doesn't show anything from cilium side:
# cilium config
Conntrack                Enabled
ConntrackAccounting      Enabled
ConntrackLocal           Disabled
Debug                    Disabled
DropNotification         Enabled
Policy                   Disabled
PolicyTracing            Disabled
# cilium endpoint list
ENDPOINT   POLICY        IDENTITY   LABELS (source:key[=value])   IPv6                    IPv4            STATUS
           ENFORCEMENT
29140      Disabled      285        container:id.client           f00e::c0a8:17d:0:71d4   <IP_ADDRESS>   ready
# cilium endpoint config 29140
Conntrack                Enabled
ConntrackAccounting      Enabled
ConntrackLocal           Disabled
Debug                    Disabled
DropNotification         Enabled
NAT46                    Disabled
Policy                   Disabled

First glance on inflight BQL seems ok. Further investigating ...

Testing with nfp (40G) on the same setup, single stream; looks much better to me:
[root@linux-1 tests]# ip netns exec blue iperf3 -c f00e::1 -t60
Connecting to host f00e::1, port 5201
[  4] local f00d::1 port 57486 connected to f00e::1 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  1.39 GBytes  11.9 Gbits/sec   95   1.33 MBytes
[  4]   1.00-2.00   sec  1.40 GBytes  12.0 Gbits/sec    0   1.33 MBytes
[  4]   2.00-3.00   sec  1.44 GBytes  12.3 Gbits/sec    0   1.33 MBytes
[  4]   3.00-4.00   sec  1.61 GBytes  13.8 Gbits/sec  386   1.34 MBytes
[  4]   4.00-5.00   sec  1.62 GBytes  13.9 Gbits/sec    0   1.37 MBytes
[  4]   5.00-6.00   sec  1.58 GBytes  13.6 Gbits/sec    0   1.37 MBytes
[  4]   6.00-7.00   sec  1.59 GBytes  13.7 Gbits/sec    0   1.37 MBytes
[  4]   7.00-8.00   sec  1.59 GBytes  13.7 Gbits/sec    0   1.37 MBytes
[  4]   8.00-9.00   sec  1.57 GBytes  13.5 Gbits/sec  164   1.41 MBytes
[  4]   9.00-10.00  sec  1.63 GBytes  14.0 Gbits/sec  238   1.41 MBytes
[  4]  10.00-11.00  sec  1.62 GBytes  13.9 Gbits/sec    0   1.41 MBytes
[  4]  11.00-12.00  sec  1.61 GBytes  13.8 Gbits/sec   53   1.41 MBytes
[  4]  12.00-13.00  sec  1.47 GBytes  12.7 Gbits/sec  773   1008 KBytes
[  4]  13.00-14.00  sec  1.59 GBytes  13.6 Gbits/sec    0   1008 KBytes
[  4]  14.00-15.00  sec  1.51 GBytes  12.9 Gbits/sec    0   1008 KBytes
[  4]  15.00-16.00  sec  1.50 GBytes  12.9 Gbits/sec    0   1008 KBytes
[  4]  16.00-17.00  sec  1.64 GBytes  14.1 Gbits/sec    0   1008 KBytes
[  4]  17.00-18.00  sec  1.62 GBytes  13.9 Gbits/sec  135   1.54 MBytes
[  4]  18.00-19.00  sec  1.61 GBytes  13.8 Gbits/sec    0   1.54 MBytes
[  4]  19.00-20.00  sec  1.58 GBytes  13.5 Gbits/sec    0   1.54 MBytes
[  4]  20.00-21.00  sec  1.61 GBytes  13.8 Gbits/sec    0   1.54 MBytes
[  4]  21.00-22.00  sec  1.63 GBytes  14.0 Gbits/sec    0   1.54 MBytes
[  4]  22.00-23.00  sec  1.52 GBytes  13.1 Gbits/sec    0   1.54 MBytes
[  4]  23.00-24.00  sec  1.59 GBytes  13.7 Gbits/sec    0   1.54 MBytes
[  4]  24.00-25.00  sec  1.61 GBytes  13.9 Gbits/sec    0   1.54 MBytes
[  4]  25.00-26.00  sec  1.62 GBytes  13.9 Gbits/sec    0   1.54 MBytes
[  4]  26.00-27.00  sec  1.53 GBytes  13.2 Gbits/sec    0   1.54 MBytes
[  4]  27.00-28.00  sec  1.52 GBytes  13.1 Gbits/sec    0   1.54 MBytes
[  4]  28.00-29.00  sec  1.58 GBytes  13.6 Gbits/sec    0   1.54 MBytes
[  4]  29.00-30.00  sec  1.63 GBytes  14.0 Gbits/sec  113   1.54 MBytes
[  4]  30.00-31.00  sec  1.62 GBytes  13.9 Gbits/sec    0   1.54 MBytes
[  4]  31.00-32.00  sec  1.62 GBytes  13.9 Gbits/sec    0   1.54 MBytes
[  4]  32.00-33.00  sec  1.59 GBytes  13.7 Gbits/sec    0   1.54 MBytes
[  4]  33.00-34.00  sec  1.59 GBytes  13.7 Gbits/sec  670   1.08 MBytes
[  4]  34.00-35.00  sec  1.63 GBytes  14.0 Gbits/sec    0   1.08 MBytes
[  4]  35.00-36.00  sec  1.60 GBytes  13.7 Gbits/sec   92   1.45 MBytes
[  4]  36.00-37.00  sec  1.58 GBytes  13.6 Gbits/sec    0   1.45 MBytes
[  4]  37.00-38.00  sec  1.64 GBytes  14.1 Gbits/sec    0   1.45 MBytes
[  4]  38.00-39.00  sec  1.59 GBytes  13.7 Gbits/sec    0   1.45 MBytes
[  4]  39.00-40.00  sec  1.62 GBytes  14.0 Gbits/sec    0   1.45 MBytes
[  4]  40.00-41.00  sec  1.60 GBytes  13.8 Gbits/sec    0   1.45 MBytes
[  4]  41.00-42.00  sec  1.49 GBytes  12.8 Gbits/sec    0   1.45 MBytes
[  4]  42.00-43.00  sec  1.58 GBytes  13.6 Gbits/sec    0   1.45 MBytes
[  4]  43.00-44.00  sec  1.61 GBytes  13.9 Gbits/sec  122   1.45 MBytes
[  4]  44.00-45.00  sec  1.49 GBytes  12.8 Gbits/sec    0   1.45 MBytes
[  4]  45.00-46.00  sec  1.64 GBytes  14.1 Gbits/sec    0   1.45 MBytes
[  4]  46.00-47.00  sec  1.61 GBytes  13.9 Gbits/sec    0   1.45 MBytes
[  4]  47.00-48.00  sec  1.63 GBytes  14.0 Gbits/sec  213   1.45 MBytes
[  4]  48.00-49.00  sec  1.61 GBytes  13.8 Gbits/sec   35   1.45 MBytes
[  4]  49.00-50.00  sec  1.62 GBytes  13.9 Gbits/sec    0   1.45 MBytes
[  4]  50.00-51.00  sec  1.61 GBytes  13.8 Gbits/sec    0   1.45 MBytes
[  4]  51.00-52.00  sec  1.60 GBytes  13.8 Gbits/sec    0   1.45 MBytes
[  4]  52.00-53.00  sec  1.51 GBytes  13.0 Gbits/sec  148   1.45 MBytes
[  4]  53.00-54.00  sec  1.58 GBytes  13.6 Gbits/sec    0   1.45 MBytes
[  4]  54.00-55.00  sec  1.61 GBytes  13.8 Gbits/sec    0   1.45 MBytes
[  4]  55.00-56.00  sec  1.62 GBytes  13.9 Gbits/sec    0   1.45 MBytes
[  4]  56.00-57.00  sec  1.60 GBytes  13.8 Gbits/sec    0   1.45 MBytes
[  4]  57.00-58.00  sec  1.63 GBytes  14.0 Gbits/sec    0   1.45 MBytes
[  4]  58.00-59.00  sec  1.61 GBytes  13.8 Gbits/sec    0   1.45 MBytes
[  4]  59.00-60.00  sec  1.61 GBytes  13.8 Gbits/sec    0   1.45 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-60.00  sec  94.9 GBytes  13.6 Gbits/sec  3237             sender
[  4]   0.00-60.00  sec  94.9 GBytes  13.6 Gbits/sec                  receiver

iperf Done.

So perhaps indeed related to ixgbe/virtio_net as mentioned on the netdev discussion.

@borkmann, great debugging! @rlguarino was doing his tests on SolarFlare NICs, so it's not ixgbe.

Ok, thanks for letting me know @bobrik. I'll continue to debug this next week, there's likely still something I'm missing. ;) Will get back once I have some more details.

Not sure if that matters, but we're running on boded interface (2x10G):
$ cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2+3 (2)
MII Status: up
MII Polling Interval (ms): 1000
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: fast
Min links: 0
Aggregator selection policy (ad_select): stable

...

( Potentially related: https://github.com/moby/moby/issues/33115 )

@rlguarino Can you observe the same issue with docker when cilium is not used at all?

This issue appears to be stale, I will close it. If there is still an issue, please reproduce with the latest Cilium and consider reopening this issue.


blueprint with two/more tiers

Reviewers
@hmunfru @jframos
Description
https://jira.tid.es/browse/CLAUDIA-4678
Test
Tested locally with experimentacion testbed

LGTM

Coverage increased (+0.11%) to 43.31% when pulling 75b686208e1639814ebe15bc7537d200257fe64f on jesuspg:bug/4678_two_tiers into 7ed15999d531d68884383262cadc35e07299ac56 on telefonicaid:hotfix/1.3.1.

LGTM


Add ability to style "go to" link without underlines

VSCode Version: Version 1.10.0-insider (1.10.0-insider)
OS Version: 10.12.3

It would be excellent if, instead of the normal link underlines on detected "go to" links:

We could have — perhaps as part of a user-created theme — a border-bottom: dotted sort of underline.

To keep the number of issues in our inbox at a manageable level, we're closing issues that have been on the backlog for a long time but haven't gained traction: we look at the number of votes the issue has received and the number of duplicate issues filed. Thank you for your time and understanding.
If you disagree and feel that this issue is crucial: we're happy to listen and to reconsider.


Documentation of current capabilities

Improve documentation before the next release.

I think maybe the README.md file could be split up into other documentation pages; right now it has a lot of detail about Git data collection that I'm not sure every casual user needs to read right off the bat.
And we'd want to update the gh-pages branch to include a description of the next release and the current set of functionality.

[ ] add Gitter.im badge to the README
[ ] move Git data collection to separate wiki page

[ ] fix installation instructions in README to refer to main repository

#329 is for the website / gh-pages update. Otherwise, this ticket is complete. Thanks @npdoty for the excellent work.


Fix some API tests

Summary of changes
Asana Ticket: Fix API test fails
Fixed a few tests that fail due to updated API information.

Assigned to: @meagonqz

This looks good but the semaphore execution timed out for some reason. Rerunning it now to make sure the elixir tests pass now.

@meagonqz I can't seem to get a clean(-ish) CI, I will look more closely into this and see if there might be a real bug going on after all.

@meagonqz victory: https://semaphoreci.com/mbta/dotcom/branches/pwd-fix_api_test_failures/builds/9


Fix: Sanitise Form 429 Error

:tada: This PR is included in version 2.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 2.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Don't document VirtualizedList context

VirtualizeList internally uses contexts to allow buidling tree of nested VirtualizedList refs. This is an API really meant to only be called by VirtualizedList, but is public so other VirtualizedList instances can call it. The object shape, parameters, etc can change (and did change a bit in 0.71). Remove it from documentation, since folks should not rely on it.

LGTM, thank you for providing a context for this removal! 👍
Maybe we should remove this method also from the versioned docs, wdyt?

I wouldn't be against it. Some of the other imperative methods here like recordInteraction() are also a bit private, so it might not hurt for me to dig through everything listed for the component and past revisions.


Roslyn generator cannot generate multi components files

Hi,
I  have two .cs components file , the first one is HpComponent  and it can be generated in GameContext
using Entitas;
public class HpComponent : IComponent
{
    public int value;
}

But the second one (ApComponent) cannot be generate in Context, even thougth i delete the HpComponent .
using Entitas;
public class ApComponent : IComponent
{
    public int value;
}

I use  "Jenny/Jenny.exe gen "   to generate code.
I hope someone can help me , thanks.

I change proj-path from projectName.csproj to Assembly-CSharp.csproj fixed my problem


update the edgeweight of undirected graph

Hello,
I have a gnn that will assign edge weight based on the topology of the subgraph. For each batch, I will update say 16 edge weights of one graph. Since the graph is undirected, it is expected that when edge weight (stored in the g.edge_weight as a tensor of size (n_edge)) corresponding to edge [1, 2] is updated, the weight of edge [2, 1] will be 'automatically' updated.
Do you know what's a good way to force the weight of two edges to be the same all the time? If this can not be done, I probably will have to find the location of edge [2, 1] when updating edge [1, 2] and it seems quite awkward.
Thank you very much!

One possible way to achieve this is to only save one direction in edge_index and use a custom GNN operator where you propagate twice (with different flows):
self.flow = 'source_to_target'
h= self.propagate(edge_index, x=x, edge_weight=edge_weight)
self.flow = 'target_to_source'
h= h + self.propagate(edge_index, x=x, edge_weight=edge_weight)

Thank you very much!


So.. I did a thing. Well another thing. BATCH INSTALLS!

@laetemn Batch Installs are HERE!
https://github.com/AxionDrak/GameCube-Backup-Manager/commit/0963467a76ce4c66bf0b74d0559e93314451207d
TODO:

Multithreading for 1:1
Clean up code
Options for how many threads to use (games to transfer at a time)
Progress bar(s) for each game? or Global?

Also fixed the checkboxes


ci: fix broken CI

@JohnGriffiths please be more careful next time, heh: https://github.com/NeuroTechX/eeg-notebooks/commit/3efcb158f5cb2111072320a67046315789c3101b
Changes

Updated psychopy
Updated minimum Python to 3.8

Required because psychopy now depends on tobii-research (another psychopy dep we wont use, sigh...) which is Python>=3.8

Updated seaborn to latest

Replaced use of removed tsplot with custom function

Fixed some examples

And @JohnGriffiths why is psychtoolbox even needed? Why is it not in the requirements file? What does the JG_ADD comment mean?

Ah, I assume psychtoolbox was to fix https://discourse.psychopy.org/t/missing-sound-libraries-for-psychopy3-standalone-release-3-2-3-for-win32/9162
However, it was already installed, so it had no effect. Added pygame instead.

I'd also like to note that the dependency hell that caused this would have been handled by poetry, were we to use it.

Alright, so this issue got a bit messier than first thought. Looks like a bunch of issues arising from having a >2yr old version of psychopy... Hoping psychopy has decent backwards compatibility.

Ugh, apparently psychopy now depends on tobii-research, which is only available for Python 3.8+
Python 3.7 is EOL in ~1 year anyway, might be time to bump it.

Now only macOS is failing, needs something like https://github.com/bambocher/pocketsphinx-python/pull/71

Fixed seaborn>=0.11 compatibility (which dropped tsplot).
Building docs is still failing, but I don't think that's related to the changes.

@JohnGriffiths This PR should be an overall improvement, but still some minor stuff needed (fix building docs & building on macOS)
There may still be bugs or more testing needed, esp with the updated versions of stuff, but it's not sustainable to stay on the old versions anyway (as that'll lead to its own problems) so better get this out of the way.

Merging.

Thanks for doing the work on this Erik. Timing is not great though as we have just launched NTCS this week with e-mail out to thousands of of openbci users and this merge has broken installation.

Our Installation instructions currently say to use py37 and install wxpython.
conda create -n "eeg-notebooks" python=3.7 git pip wxpython
conda activate "eeg-notebooks"
git clone https://github.com/NeuroTechX/eeg-notebooks
cd eeg-notebooks
pip install -e .

I am troubleshooting this now but definitely just switching to python=3.8 doesn't work as there's no wxpython for py37.
Trying to recall now why we settled on that wxpython instruction (was definitely an interim fix). We did go round the houses between py37, py38, and py39 last time we tackled this and settled on 3.7.

Also important = any installation instruction updates need to be reflected in the docs instructions, and currently we have a build failure in the docs, so can't actually change the documentation instructions.
I have tried several times to resolve this problem but haven't been able to yet. The doc build seems to work fine on all local machines I've tried so a bit stumped.

install with py38 is failing on two out of two windows machines for me now. have you tested this? are there extra install instructions?
might need to revert this PR.
@oreHGA @JadinTredup if you got a mo to check on this would be great as is pretty time critical.

I'll jump on it

There are wxPython wheels for Python 3.8 here, as detailed in requirements.txt: https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-20.04/

@JohnGriffiths I have a revert for this ready to go, just give me the word!

Yeah I think revert for now and we can work on the PR as a priority matter
On Sun., Mar. 27, 2022, 1:53 p.m. Erik Bjäreholt, @.***>
wrote:

@JohnGriffiths https://github.com/JohnGriffiths I have a revert for
this ready to go, just give me the word!
—
Reply to this email directly, view it on GitHub
https://github.com/NeuroTechX/eeg-notebooks/pull/167#issuecomment-1079983734,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AADBEAWNUY6FRKNL6J3EENDVCCVA5ANCNFSM5RTYZBGA
.
You are receiving this because you were mentioned.Message ID:
@.***>


Ignore .stylelintrc

Description

lint-staged failed to search for the configuration file of stylelint (following current documentation might not be correct). However, running solely stylelint seem not to cause any issue.
Steps to reproduce

Run with lint-staged with a config file as:
{
  "*.scss": "stylelint src/**/*.scss --fix"
}

Error: No configuration provided

And even if I provide the already defined configuration fo stylelint:
{
  "*.scss": "stylelint --config .stylelintrc src/**/*.scss --fix"
}

Error: No (config) file not found

Environment

OS:  macOS Catalina
Node.js:  v10.15.3
lint-staged:  v10.0.8
stylelint: v13.2.1

Please post debug logs by running lint-staged --debug.

you should change code below
{
  "*.scss": "stylelint src/**/*.scss --fix --config .stylelintrc"
}

or
// fix only lint-staged file
{
  "*.scss": "stylelint --config .stylelintrc --fix"
}

see https://stylelint.io/user-guide/usage/cli#example-g---specifying-a-config
https://github.com/okonet/lint-staged#stylelint-for-css-with-defaults-and-for-scss-with-scss-syntax

Stylelint assumes you're overriding it's ignored files when lint-staged passes explicit arguments. The documentation explains how to get around this for eslint, but not for stylelint. The problem is that a similar approach does not work with stylelint, because stylelint --fix '' (passing an empty string) tries to run the linter against all of your files, instead of matching eslint behaviour which is a no-op.
Something like
{
  '*.scss': (staged) => {
    const files = staged
      .map((to) => path.relative(process.cwd(), to))
      .filter((file) => /^src\//.test(file))

    return files ? `stylelint --fix ${files}` : 'echo'
  },
}

might work for you. I couldn't really think of a better no-op than echo.


Fix file argument

This fix should allow omitting --file argument (prints the help message if so), while still allowing to pass arguments like --verbose or --version without crashing.

@mwkm00, it looks like the black formatting linter is failing.
More info on black is here: https://github.com/psf/black
Would you mind fixing that before I review this?

Should be good now, I assumed exit code 1 would be acceptable after not delivering file arg, but the program goes to the help message and exits gracefully so exit code 0 is probably the way to go. Tests should be fixed now.

My bad, formatting. Thanks for patience.

@vargenau, is that you were thinking?

Hi,
Yes, it works correctly now.
Thank you for the quick fix.

@goneall, would you please take a quick look too? 🙏


pull request for e2e test at 02/05/2017 12:45:07

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


ForbiddenError: invalid csrf token

Can't upload or access characters

My PC's clock was messed up


Change the License to OpenSource

Currently the repository LICENSE file in repo and packages lists Extreme Workflow Composer EULA, including Homepage: https://www.extremenetworks.com/product/workflow-composer/, EWC naming and other Extreme Networks references.
This should be changed to Apache 2 OSS license.
Additionally, every file needs a new Apache 2 license/copyright header instead of Extreme proprietary, including README and repository descriptions.

Resolved via https://github.com/StackStorm/st2-enterprise-sso-backend/pull/10


🐛 Manual Javascript injection doesnt work as docs describe

Bug report
The docs found at https://firebase.google.com/docs/flutter/setup?platform=web#disable-auto state that the JS auto injection can be disabled using window.flutterfire_ignore_scripts which is true however it also states they can be manually loaded by just including a script tag. Ie:
<script src="https://www.gstatic.com/firebasejs/10.5.0/firebase-analytics.js"></script>
<script src="https://www.gstatic.com/firebasejs/10.5.0/firebase-firestore.js"></script>

This doesnt appear to work.
After reading through the source I figured out that you can import them as follows:
      window.firebase_analytics = await import("./libs/firebase/firebase-analytics.js");
      window.firebase_firestore = await import("./libs/firebase/firebase-firestore.js");

However all the firebase-x.js scripts have the path to firebase-app.js hardcoded within the import statements preventing any override of app/core.
Steps to reproduce
Steps to reproduce the behavior:
In index.html place:
window.flutterfire_ignore_scripts = ['analytics', 'firestore'];
<script src="https://www.gstatic.com/firebasejs/10.5.0/firebase-analytics.js"></script>
<script src="https://www.gstatic.com/firebasejs/10.5.0/firebase-firestore.js"></script>

Errors thrown in the console
Expected behavior
That analytics and firestore are fetched from the path given and firebase to initialize correctly
Additional context
The ability to prevent the JS auto injecting allow control of what files are downloaded, in our case we need to run fully offline this prevents that. Even if the manually loading worked as documented it would still mean firebase-app.js is always fetched from gstatic.com

Thanks for the report @willcalderbank
I used the sample index.html snippet you shared, in plugin's example and ran on web, but it didn't give me any errors in the console or in browser console. Can you share the error you get ?

Apologies @darshankawar i've been away for a week. Thanks for looking into it.
Here's a more complete sample (which follows the doc's instructions):
<!DOCTYPE html>
<html>
  <head>
    <script src="flutter.js" defer></script>
    <script src="https://www.gstatic.com/firebasejs/10.5.0/firebase-analytics.js"></script>
    <script src="https://www.gstatic.com/firebasejs/10.5.0/firebase-firestore.js"></script>

  </head>
  <body>
    <script>
      window.flutterfire_ignore_scripts = ['firestore', 'analytics'];

      window.addEventListener('load', function (ev) {
        _flutter.loader.loadEntrypoint().then(function (engineInitializer) {
          return engineInitializer.initializeEngine();
        }).then(function (appRunner) {
          return appRunner.runApp();
        });
      });
    </script>
  </body>
</html>

If causes the following errors in the console:
firebase-analytics.js:1 Uncaught SyntaxError: Cannot use import statement outside a module
firebase-firestore.js:1 Uncaught SyntaxError: Cannot use import statement outside a module

As the error states the modulular versions of the JS cant be imported like this, so instead we can try and use namespaced versions:
<!DOCTYPE html>
<html>
  <head>
    <script src="flutter.js" defer></script>
    <script src="https://www.gstatic.com/firebasejs/10.5.0/firebase-analytics-compat.js"></script>
    <script src="https://www.gstatic.com/firebasejs/10.5.0/firebase-firestore-compat.js"></script>

  </head>
  <body>
    <script>
      window.flutterfire_ignore_scripts = ['firestore', 'analytics'];

      window.addEventListener('load', function (ev) {
        _flutter.loader.loadEntrypoint().then(function (engineInitializer) {
          return engineInitializer.initializeEngine();
        }).then(function (appRunner) {
          return appRunner.runApp();
        });
      });

    </script>
  </body>
</html>

In which case the following error is thrown:
firebase-analytics-compat.js:1 Uncaught TypeError: Cannot read properties of undefined (reading 'INTERNAL')
    at firebase-analytics-compat.js:1:294
    at firebase-analytics-compat.js:1:316
firebase-firestore-compat.js:1 Uncaught TypeError: Cannot read properties of undefined (reading 'INTERNAL')
    at firebase-firestore-compat.js:1:294
    at firebase-firestore-compat.js:1:316

The only way ive been able to get it to work is to use the following:
<!DOCTYPE html>
<html>
  <head>
    <script src="flutter.js" defer></script>
  </head>
  <body>
    <script>
      window.flutterfire_ignore_scripts = ['firestore', 'analytics'];

      window.addEventListener('load', async function  (ev) {

        window.firebase_firestore = await import("https://www.gstatic.com/firebasejs/10.3.1/firebase-firestore.js");
        window.firebase_analytics = await import("https://www.gstatic.com/firebasejs/10.3.1/firebase-analytics.js");

        _flutter.loader.loadEntrypoint().then(function (engineInitializer) {
          return engineInitializer.initializeEngine();
        }).then(function (appRunner) {
          return appRunner.runApp();
        });
      });

    </script>
  </body>
</html>

Note the version change to 10.3.1. At one point during testing a warning showed that this was the version to use with the flutter sdk. Version 10.5.0 did not work with the above await import. Changing the other 2 examples to 10.3.1 didnt seem to have an effect the error was the same.
Hope this helps.

Thanks for the detailed update. Seeing the same behavior as reported.

Thanks for the doc change but I fear its not correct {{web_sdk_version}} is the latest version, it doesn't work with that version only 10.3.1.
It also doesn't address the problem that all of the js libs force firebase-app.js to be downloaded via gstatic.com


Make kubernetes dns addon more configurable

Motivation for this change
Sometimes there is need to have greater control over the cluster DNS, for example augmenting it with additional service discovery (i.e. Consul), or adding DNS records for internal subdomains. With mode Reconcile, all local changes to the maps are instantly overwritten. Changing the Deployment and ConfigMap to EnsureExists allows the user the change the maps after installation.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

Wouldn't this make any subsequent changes in Nix be ignored silently?
cc @johanot

@srhb Probably yes, but that's what you want if you have local changes. It's quite easy to track upstream changes:

Save the relevant maps
Delete them
Let addon-manager recreate them
Merge in local changes

I just worry that it's unintuitive from the perspective of NixOS. I'll let @johanot chime in though, since he's actually using Kubernetes in production, and I'm not. :)

@srhb I'm running kubernetes in production, and I spent quite a few hours trying to come up with a sensible and stable workaround (like using dnsmasq on the host(s)) which would not require messing with the defaults. Nothing really worked out. So unfortunately this was the only practical solution left in the end.

@juselius Isn't your usecase an argument for opting out of the kubernetes.dns module completely, i.e. services.kubernetes.addons.dns.enable = false and then deploy the dns service manually or add a custom addon to the addon manager with EnsureExists? I'd argue that Reconcile is the intuitive choice from a declarative point of view.

@johanot Well, yes and no. It's much easier to track nixos when a) all you have to do is delete the deployment and configmap to get the new defaults b) ad you have a patch yaml for local  changes. But I see the problem now. Everybody without local modifications end up in dire straits.  Would a an option e.g. "allowLocalChanges" make the patch more palatable?

Well yes - especially if the default behavior is preserved.
I think it would be slightly prettier with e.g. reconcileMode, type = enum [ "Reconcile" "EnsureExists" ];, default = "Reconcile", but that's just opinion. :)

@johanot Good idea! I'll post a new PR.

Thanks @johanot!

@GrahamcOfBorg eval

@juselius Can you please squash the entire changeset down to one commit?

@johanot done!

Oh.. Sorry I didn't see this before. The commit message should be prefixed with nixos/kubernetes:, instead of kubernetes:, since it involves changes to the NixOS module, not the package.

I find usecases in PR description more compelling than current option description. For a new guy like me, it is not clear why this once setting is exported as NixOS option, but others are not.
Another problem is that only some of "addons" allow this override. That looks a bit ad-hoc. Maybe note about that in option description as well? Otherwise it would require looking into module code to find out what this is for.

@danbst Good suggestion! I have added a line in the description about a use case. My opinion regarding the "ad-hocness" of the override,  is that if you need it in the future, do it in the future. There is little point in adding options for things nobody will ever need, just for symmetry's sake.
@johanot Shall we try to get this merged in, before it goes stale again?

@juselius Fine by me, but I don't have merge access :-)
@srhb Do you have any further comments? If not, will you please assist in merging and back-porting?

Thanks @danbst !

@juselius as for me PR description is still more clear about what this option is for. But maybe you k8s guys use some other language, so :man_shrugging: problem with me I guess.

backported

First NixOS PR! Time to celebrate :)
@johanot Oh, I thought you had merge access.
@danbst Thanks! Perhaps you are right regarding the description.  But the PR description is just one of many uses. This is nonetheless not an option a k8s newbie normally would find useful (I believe).


Language Retrieval

This PR aims to define how the rest of the system should get available languages and the relevant information for those languages to be displayed to the user.

This change is

src/main/kotlin/org/bibletranslationtools/fetcher/application/PortLanguageRepository.kt, line 10 at r1 (raw file):
Previously, jsarabia (Joe) wrote…
how about we change this to
PortGatewayLanguageCatalog : LanguageCatalog
The SourceAudioLanguages class isn't well named (the name doesn't tell me anything meaningful, or worse, leads me to believe it's something different)
What you probably want there is to start constructing a more "true" "repository" where you get the content. It might have things like getLanguages(), getBooks(languageCode: String), etc...

@jsarabia  do you mean that a repo that provides many functionalities like getLanguages(), getBooks().... and such?

src/main/kotlin/org/bibletranslationtools/fetcher/application/PortLanguageRepository.kt, line 36 at r1 (raw file):
Previously, jsarabia (Joe) wrote…
I think you can just call getResource directly and don't need [this...]?

Done.

src/main/kotlin/org/bibletranslationtools/fetcher/application/SourceAudioRepository.kt, line 9 at r5 (raw file):
Previously, jsarabia (Joe) wrote…
@anonymouswalker Is this an entity (noun) or an action (verb)?

Oh I forgot that the use case should be verb


AppDomain Security vs Reflection Emit

The malicious client example app doesn't work as it used to.
Seems that Reflection Emit is demanding full trust now. Qactive only asserts Reflection permission but apparently that's not enough. Did Emit always demand full trust or did something change in Rx?
Unfortunately, if full trust is required by Rx's Qbservable Provider (the call to Subscribe) then CAS security may be useless here and all of the AppDomain code may be removed.
Alternatively, consider writing a custom Qbservable Provider just like Rx but relaxing the permission set, if possible. Perhaps one way to do this is to ensure that the expression tree is compiled outside of the call to Subscribe and therefore full trust can be asserted? (Note that asserting full trust around the call to Subscribe entirely defeats the purpose of using CAS because it means that the client's entire query will be running with full trust!)

Fixed by wrapping Rx's queryable implementation and ensuring that the expression is compiled with full trust but the call to Subscribe is not. I'm using reflection, so it's a fragile solution; however, I've also added a new ISecureQbservable interface that theoretically Rx could adopt (as can custom queryable providers) and then the reflection can be removed.


Streaming with request

I'm trying to use nock to test streaming an image from one url into another. It goes something like this:
const request = require('request');
const stream = request.get('http://foo.com/images/image');

const options = {
  formData: {
    image: stream
  },
  url: 'http://bar.com/images'
};

request.post(options, (err, res) => {
  // do stuff
});

When I'm mocking the GET request, what should I do with reply() to make sure the POST actually gets through (maybe using nock for that endpoint itself)? I've tried some alternatives like replyWithFile(), reply() with a readable file stream and others but it seems to hang on every one of them.
Maybe I'm missing the point and this is some idiosyncrasy in request, but the code works in and of itself.
Thanks!

It seems to be some problem with the implementation after all. Sorry.


Missing info in the history of a package uploaded using jamfCPR

Looks like "Add information" (when it was uploaded, the account used to upload, and that is was uploaded with jamfCPR) to the Notes section for a package in Jamf Pro is not happening in beta 5

Only showing the username as the API client ID. no note that it was uploaded by jamfCPR.

Certainly won’t show up in the history, should see it back on the General tab of a package.  Notes is just below the Info box.  Let me know if it’s not there.

Added additional info.


Use lychee in updated base image

Thank you for helping out with embedded-graphics development! Please:

[x] Check that you've added passing tests and documentation
[x] Add a CHANGELOG.md entry in the Unreleased section under the appropriate heading (Added, Fixed, etc) if your changes affect the public API
[x] Run rustfmt on the project
[x] Run just build (Linux/macOS only) and make sure it passes. If you use Windows, check that CI passes once you've opened the PR.

PR description
Paired with https://github.com/embedded-graphics/build-tools/pull/4

Thanks :)


Removed isDarkModeEnabled prop and formatting

Overview

Test Plan

:tada: This PR is included in version 8.9.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

This is definitely a semver breaking change!
Please note I contribute very heavily to open source and I'm using this package which I'm not maintaining (thank you! seriously) so I'm not really complaining, but semver breaks are worth noting

Ouch! Thanks @mikehardy , you are 100% right, I was trying to release from mobile and did a mess. Any suggestion on how we can make improve the current state?

I'd just bump it again and say "sorry folks :-), use  if you are not ready for the break yet", that's what I've done in the past when I've accidentally semver-violated


Interaction between gil and tokio causing a deadlock

🐛 Bug Reports
We have a multi-threaded Rust application that calls Python from time to time within (tokio) threads. However, I am observing a deadlock on the code.
I am not entirely sure that what I am doing is correct, but I was able to reduce it to a minimal example (see below).
🌍 Environment

Your operating system and version: MacOS
Your python version: 3.7.4
How did you install python (e.g. apt or pyenv)? Did you use a virtualenv?: brew + venv
Your Rust version (rustc --version): rustc 1.46.0-nightly (3503f565e 2020-07-02)
Your PyO3 version: 0.12.3
Have you tried using latest PyO3 master (replace version = "0.x.y" with git = "https://github.com/PyO3/pyo3")?: not yet

💥 Reproducing
#src/lib.rs
use pyo3::{prelude::*, wrap_pyfunction};
use tokio::runtime::Runtime;
use tokio;

#[pyfunction]
fn compute() -> i32 {
    let mut rt = Runtime::new().unwrap();
    rt.block_on(async {
        let a = tokio::spawn(async {
            let gil = pyo3::Python::acquire_gil();
            let py = gil.python();
            let builtins = PyModule::import(py, "builtins").unwrap();
            let total: i32 = builtins.call1("sum", (vec![1, 2, 3],)).unwrap().extract().unwrap();
            total
        });
        let b = tokio::spawn(async {
            let gil = pyo3::Python::acquire_gil();
            let py = gil.python();
            let builtins = PyModule::import(py, "builtins").unwrap();
            let total: i32 = builtins.call1("sum", (vec![1, 2, 3],)).unwrap().extract().unwrap();
            total
        });
        let r1 = a.await.unwrap();
        let r2 = b.await.unwrap();

        r1 + r2
    })
}

#[pymodule]
fn blabla(_: Python, module: &PyModule) -> PyResult<()> {
    module.add_function(wrap_pyfunction!(compute, module)?)?;
    Ok(())
}

#Cargo.toml
[package]
name = "rust-tokio-pyo3"
version = "0.1.0"
authors = ["Jorge C. Leitao<EMAIL_ADDRESS>edition = "2018"

[dependencies]
tokio = { version = "^0.2", features = ["macros", "rt-core", "rt-threaded"] }
pyo3 = { version = "0.12.3", features = ["extension-module"] }

[lib]
name = "blabla"
crate-type = ["cdylib"]

# test_bla.py
import blabla

print(blabla.compute())

To reproduce, run:
python -m venv venv
source venv/bin/activate

pip install maturin==0.8.2 toml==0.10.1

maturin develop
python -m test_bla

WARNING: this code will not exit via keyboard: kill PID is required.

I would suggest using an async RwLock from Tokio at the beginning of each Tokio Task that contains PyO3 code. While I haven't used PyO3 myself, I've heard issues with it and threading. Using the RwLock, you can use the read lock for the scopes of each Task, and the write lock for the Task with the GIL call. This would ensure as many concurrent non-GIL Tasks as possible, while having a global lock when you're about to do a GIL call.
This is mostly a guess from my async knowledge and my very rudimentary Python knowledge. I hope it helps in some form.

Also, RwLocks are normally prone to unfairness due to how the CPU schedules the locks, but Tokio's fairness mechanisms should compensate.

 I have a lazy runtime that I'm building at the start of the program and I'm trying to create a connection to a grpc service when I call a command.
The problem is, the client says I have reactor core problems.

When interacting with Python, you have to always be aware that the Python GIL is exactly that - a global lock. If one thread holding the GIL is blocked waiting on another thread, you'll get a deadlock if that other thread is waiting to acquire the GIL.
The general rule is that you should always release the Python GIL before performing any other operation that blocks a thread. That could be waiting to lock a mutex, for example.
In your case, you need to remember that the thread calling the #[pyfunction] already has the GIL. So your call to rt.block_on is incorrect, as you are not first releasing the GIL.
You can do this by using Python::allow_threads to temporarily release the GIL before issuing the blocking instruction, like this:
#[pyfunction]
fn compute(py: Python) -> i32 {
    let mut rt = Runtime::new().unwrap();
    py.allow_threads(|| {
        // Inside this closure, this thread has temporarily released the GIL, meaning that
        // the tasks a and b are now actually able to acquire the GIL and progress.
        rt.block_on(async {
            let a = tokio::spawn(async {
                let gil = pyo3::Python::acquire_gil();
                let py = gil.python();
                let builtins = PyModule::import(py, "builtins").unwrap();
                let total: i32 = builtins.call1("sum", (vec![1, 2, 3],)).unwrap().extract().unwrap();
                total
            });
            let b = tokio::spawn(async {
                let gil = pyo3::Python::acquire_gil();
                let py = gil.python();
                let builtins = PyModule::import(py, "builtins").unwrap();
                let total: i32 = builtins.call1("sum", (vec![1, 2, 3],)).unwrap().extract().unwrap();
                total
            });
            let r1 = a.await.unwrap();
            let r2 = b.await.unwrap();

            r1 + r2
        })
    })
}

This modified version of your compute function completes successfully without deadlocking.

@khionu and @davidhewitt , thank you for your help.
@davidhewitt thanks for the solution. It makes sense, I was not aware of allow_threads API to achieve this. It solved the problem.
FYI, the usage of pyo3 here is for data processing with Apache Arrow: https://github.com/jorgecarleitao/datafusion-python. We use threads for parallelism on the rust side, but allow Python to run in the middle, which is why we need to lock the GIL.

@ davidhewitt,  just  a side question referring to your comment:The general rule is that you should always release the Python GIL before performing any operation that blocks a thread.....
If I just want a multi-thread function from rust-side, and hope to blocking it when using the funciton in python scripyt, is it still necessary to use Python::allow_threads?

@dbsxdbsx I'm not entirely sure what you're asking, but here's my attempt to answer:

You can make as many Rust threads as you like. If they don't run Python code, they won't get blocked by the GIL.
If the threads are just running Rust code (without Python), then no need to call Python::allow_threads. But if any of those threads try to call Python code, you'll need to have called Python::allow_threads.

Also, if you want other Python threads to run while the current Python thread is running Rust code, you need to call allow_threads

@davidhewitt @programmerjake , thanks for your reply. It is quite clear now.


bootstrap .form-horizontal incompatible with v4 of bootstrap style sheet

This relies on the development version of bootstrap's style sheet, v4-dev.
Evidence:  if I switch to version 3.3.6, the header in the base template messes up.
However, v4-dev has a bug that makes it not work with bootstrap's form-horizontal class.
Evidence: I used form-horizontal in my code, and it works with version 3.3.6 of the style sheet, but not v4-dev.
There's probably nothing we can do now, except wait for the new release of v4, but I wanted to put this issue here as feedback that might make an impact on future decisions to use development versions of things (style sheets, etc).
Thanks

Cookiecutter Django is partly about using bleeding edge tools. How Bootstrap works between versions is out of scope for Cookiecutter. Hence why I'm closing this issue.
However, if someone wants to create a Bootstrap 3 theme cookiecutter template that works as a direct replacement for Cookiecutter Django, I'm willing to link to it.