Pin plugins to release tags

Interesting. Keeping all vendored plugins on released versions makes good sense. Are we already doing that, but this just makes that clearer?
$ git submodule status
 cb92aa6f507d9cd3527889d89b1c380843be0e70 plugins/docker-login (v2.0.1-40-gcb92aa6)
 a7e99dc4da50fa8dfc5a520b85ec8a28b6a7087a plugins/ecr (v2.3.0-1-ga7e99dc)
 67c4e227b5e591608794776d6330c7fe4ea2273a plugins/secrets (v2.0.0)

The ECR plugin is vendored 1 commit ahead of v2.3.0:
ecr-buildkite-plugin (master)]$ gl | head -n 4
* a7e99dc (HEAD -> master, origin/master, origin/HEAD) Bump README.md for v2.3.0 release  boomper-bot[bot], 3 months ago
*   aa8e3cd (tag: v2.3.0) Merge pull request #56 from tmcg-gusto/master  Jason Jacob, 3 months ago
|\
| * 75bfe8a Add retry to get-login-password  Thomas McGinniss, 3 months ago

secrets is vendored right on v2.0.0:
$ gl | head -n 4
*   67c4e22 (HEAD -> master, tag: v2.0.0, origin/master, origin/HEAD) Merge pull request #37 from buildkite/go  Paul Annesley, 6 months ago
|\
| * ca69f16 go: double-quote GIT_CONFIG_PARAMETERS; eval keeps inner single-quotes  Paul Annesley, 6 months ago
| * d116acb CI: test_credentials.sh provides s3secrets-helper via docker.  Paul Annesley, 6 months ago

docker-login is vendored ~40 commits ahead of v2.0.1, and it seems there isn't a formal post-v2.0.1 release for us to pin to :grimacing:


catch up master

Summary by CodeRabbit

New Features

Enhanced the visual representation of assets and chains across the platform with updated logic for imagery and information display.

Improvements

Refined the component logic for a more consistent and accurate representation of chain and token information.

Refactor

Streamlined import paths for better maintainability and project structure.

Documentation

Added detailed ABIs for various smart contracts to facilitate developer interactions with blockchain operations.

Bug Fixes

Corrected issues with data validation and conditional rendering in several components to improve user experience.

c43c97d1b018b6d5463c60c2d7e581c5d79140b2: explorer-ui preview link
c43c97d1b018b6d5463c60c2d7e581c5d79140b2: synapse-interface preview link

git diff updated-fe-release fe-release
diff --git a/packages/synapse-interface/CHANGELOG.md b/packages/synapse-interface/CHANGELOG.md
index f5914a9b..c5fb02af 100644
--- a/packages/synapse-interface/CHANGELOG.md
+++ b/packages/synapse-interface/CHANGELOG.md
@@ -7,9 +7,6 @@ See [Conventional Commits](https://conventionalcommits.org) for commit guideline

 **Note:** Version bump only for package @synapsecns/synapse-interface

-
-
-
 ## [0.1.208](https://github.com/synapsecns/sanguine/compare/@synapsecns/synapse-interface@0.1.207<EMAIL_ADDRESS>(2023-12-14)

 **Note:** Version bump only for package @synapsecns/synapse-interface

This LGTM - will indeed catch up master with the fe-release branch.
Very important: this absolutely needs to be regularly merged into master (not squashed). This could be achieved by either:

Allowing regular merge strategy in the repo settings (cc @aureliusbtc @trajan0x), or
Performing a merge locally and then pushing it:

Make sure your local master and update-fe-release are up to date
git checkout master --recurse-submodules
git merge updated-fe-release
git push
This should automatically close this PR as merged, while performing a regular merge. This allows master and fe-release to fully catch up (technically, master will be 1 "merge commit" ahead, but the diff will be empty, and branches could be later merged into one another w/o conflicts).

The end result would be the same - the difference is whether the squash merges into master continue to be enforced on the repo level.

Will wait for responses from @trajan0x or @aureliusbtc to see if we want to go with option (1), otherwise will proceed with option (2) tomorrow morning

Closing: to be replaced by #1721


Update workflows

See https://github.com/Senzing/docker-base-image-debian/pull/65

Pull Requests:

https://github.com/Senzing/docker-senzing-base/pull/148


Support for declaration merging

Use declaration merging instead of extending @ColumnOptions.
See https://www.typescriptlang.org/docs/handbook/declaration-merging.html for more info.

Less necessary with Transformers in #14 .


Relaxes key-spacing rule

Relaxes key-spacing rule to warning for consistency with space-in-parens and space-in-bracket settings.
Changes

Changes key-spacing to warning.
Removes unneeded newline.

Testing

Misalignment of property values in an object literal should throw a warning, not an error, and thus won't break a build.

Review

@ascott1
@wpears
@mistergone

Good change. Thanks @anselmbradford!


runtime,internal/godebug: race condition raised with parallel tests with goroutines and -race

Go version
go version go1.21.5 darwin/arm64
What operating system and processor architecture are you using (go env)?
- Local (Apple M2)
- GitHub Actions (x86)

What did you do?
I have a testsuite for my github.com/dolmen-go/rendezvous package.
$ git clone https://github.com/dolmen-go/rendezvous.git
$ cd rendezvous
$ git checkout 44e68d01a216b3755c9ba5dc75835074c3e5e866
$ go version
go version go1.21.5 darwin/arm64
$ go test -v -race -run 'Test(One|Two)' "-gcflags=all=-trimpath=$(go env GOPATH)" github.com/dolmen-go/rendezvous
=== RUN   TestOne
=== PAUSE TestOne
=== RUN   TestTwo
=== PAUSE TestTwo
=== CONT  TestOne
=== CONT  TestTwo
==================
WARNING: DATA RACE
Read at 0x00c000076560 by goroutine 14:
  internal/godebug.(*Setting).IncNonDefault()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/internal/godebug/godebug.go:102 +0x58
  internal/godebug.(*Setting).IncNonDefault-fm()
      <autogenerated>:1 +0x34
  runtime.(*godebugInc).IncNonDefault()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/runtime/runtime.go:111 +0xd0
  github.com/dolmen-go/rendezvous.WaitAll.func1()
      src/github.com/dolmen-go/rendezvous/rendezvous.go:56 +0xec
  github.com/dolmen-go/rendezvous.WaitAll.func2()
      src/github.com/dolmen-go/rendezvous/rendezvous.go:57 +0x44

Previous write at 0x00c000076560 by goroutine 12:
  internal/godebug.New()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/internal/godebug/godebug.go:74 +0x40
  internal/godebug.newIncNonDefault()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/internal/godebug/godebug.go:203 +0x28
  runtime.(*godebugInc).IncNonDefault()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/runtime/runtime.go:108 +0x84
  github.com/dolmen-go/rendezvous.WaitAll.func1()
      src/github.com/dolmen-go/rendezvous/rendezvous.go:56 +0xec
  github.com/dolmen-go/rendezvous.WaitAll.func2()
      src/github.com/dolmen-go/rendezvous/rendezvous.go:57 +0x44

Goroutine 14 (running) created at:
  github.com/dolmen-go/rendezvous.WaitAll()
      src/github.com/dolmen-go/rendezvous/rendezvous.go:47 +0x210
  github.com/dolmen-go/rendezvous_test.TestTwo()
      src/github.com/dolmen-go/rendezvous/rendezvous_test.go:118 +0xe4
  testing.tRunner()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/testing/testing.go:1595 +0x1b0
  testing.(*T).Run.func1()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/testing/testing.go:1648 +0x40

Goroutine 12 (running) created at:
  github.com/dolmen-go/rendezvous.WaitAll()
      src/github.com/dolmen-go/rendezvous/rendezvous.go:47 +0x210
  github.com/dolmen-go/rendezvous_test.TestOne()
      src/github.com/dolmen-go/rendezvous/rendezvous_test.go:104 +0x180
  testing.tRunner()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/testing/testing.go:1595 +0x1b0
  testing.(*T).Run.func1()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/testing/testing.go:1648 +0x40
==================
=== NAME  TestOne
    testing.go:1465: race detected during execution of test
--- FAIL: TestOne (0.00s)
=== NAME  TestTwo
    testing.go:1465: race detected during execution of test
--- FAIL: TestTwo (0.00s)
FAIL
exit status 1
FAIL	github.com/dolmen-go/rendezvous	0.104s

Some characteristics that might be relevant:

The rendezvous package launches goroutines in its tests.
The rendezvous testsuite uses parallel testing at top testing level:

func TestOne(t *testing.T) {
        t.Parallel()
        // ...
}

func TestTwo(t *testing.T) {
        t.Parallel()
        // ...
}

This issue is also visible on GitHub Actions: https://github.com/dolmen-go/rendezvous/actions/runs/7147035848/job/19465833264#step:4:32
What did you expect to see?
Test success like with previous Go versions.
What did you see instead?
Race condition in runtime:
WARNING: DATA RACE
Read at 0x00c000076560 by goroutine 14:
  internal/godebug.(*Setting).IncNonDefault()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/internal/godebug/godebug.go:102 +0x58
  internal/godebug.(*Setting).IncNonDefault-fm()
      <autogenerated>:1 +0x34
  runtime.(*godebugInc).IncNonDefault()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/runtime/runtime.go:111 +0xd0
  github.com/dolmen-go/rendezvous.WaitAll.func1()
      src/github.com/dolmen-go/rendezvous/rendezvous.go:56 +0xec
  github.com/dolmen-go/rendezvous.WaitAll.func2()
      src/github.com/dolmen-go/rendezvous/rendezvous.go:57 +0x44

Previous write at 0x00c000076560 by goroutine 12:
  internal/godebug.New()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/internal/godebug/godebug.go:74 +0x40
  internal/godebug.newIncNonDefault()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/internal/godebug/godebug.go:203 +0x28
  runtime.(*godebugInc).IncNonDefault()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/runtime/runtime.go:108 +0x84
  github.com/dolmen-go/rendezvous.WaitAll.func1()
      src/github.com/dolmen-go/rendezvous/rendezvous.go:56 +0xec
  github.com/dolmen-go/rendezvous.WaitAll.func2()
      src/github.com/dolmen-go/rendezvous/rendezvous.go:57 +0x44

Link to runtime source: https://github.com/golang/go/blob/go1.21.5/src/runtime/runtime.go#L97
A comment says:
		// If other goroutines are racing here, no big deal. One will win,
		// and all the inc functions will be using the same underlying
		// *godebug.Setting.

If this is related, I think this is "big deal" as running "go test -race" in CI is quite common.

$ go test -race -run 'Test(One|Two)' "-gcflags=all=-trimpath=$(go env GOPATH)" -count=1 -parallel=1
PASS
ok  	github.com/dolmen-go/rendezvous	1.271s
$ go test -race -run 'Test(One|Two)' "-gcflags=all=-trimpath=$(go env GOPATH)" -count=1 -parallel=2
==================
WARNING: DATA RACE
Read at 0x00c000076500 by goroutine 14:
  internal/godebug.(*Setting).IncNonDefault()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/internal/godebug/godebug.go:102 +0x58
  internal/godebug.(*Setting).IncNonDefault-fm()
      <autogenerated>:1 +0x34
  runtime.(*godebugInc).IncNonDefault()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/runtime/runtime.go:111 +0xd0
  github.com/dolmen-go/rendezvous.WaitAll.func1()
      src/github.com/dolmen-go/rendezvous/rendezvous.go:56 +0xec
  github.com/dolmen-go/rendezvous.WaitAll.func2()
      src/github.com/dolmen-go/rendezvous/rendezvous.go:57 +0x44

Previous write at 0x00c000076500 by goroutine 12:
  internal/godebug.New()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/internal/godebug/godebug.go:74 +0x40
  internal/godebug.newIncNonDefault()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/internal/godebug/godebug.go:203 +0x28
  runtime.(*godebugInc).IncNonDefault()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/runtime/runtime.go:108 +0x84
  github.com/dolmen-go/rendezvous.WaitAll.func1()
      src/github.com/dolmen-go/rendezvous/rendezvous.go:56 +0xec
  github.com/dolmen-go/rendezvous.WaitAll.func2()
      src/github.com/dolmen-go/rendezvous/rendezvous.go:57 +0x44

Goroutine 14 (running) created at:
  github.com/dolmen-go/rendezvous.WaitAll()
      src/github.com/dolmen-go/rendezvous/rendezvous.go:47 +0x210
  github.com/dolmen-go/rendezvous_test.TestTwo()
      src/github.com/dolmen-go/rendezvous/rendezvous_test.go:118 +0xe4
  testing.tRunner()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/testing/testing.go:1595 +0x1b0
  testing.(*T).Run.func1()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/testing/testing.go:1648 +0x40

Goroutine 12 (running) created at:
  github.com/dolmen-go/rendezvous.WaitAll()
      src/github.com/dolmen-go/rendezvous/rendezvous.go:47 +0x210
  github.com/dolmen-go/rendezvous_test.TestOne()
      src/github.com/dolmen-go/rendezvous/rendezvous_test.go:104 +0x180
  testing.tRunner()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/testing/testing.go:1595 +0x1b0
  testing.(*T).Run.func1()
      /opt/homebrew/Cellar/go/1.21.5/libexec/src/testing/testing.go:1648 +0x40
==================
--- FAIL: TestOne (0.00s)
    testing.go:1465: race detected during execution of test
--- FAIL: TestTwo (0.00s)
    testing.go:1465: race detected during execution of test
FAIL
exit status 1
FAIL	github.com/dolmen-go/rendezvous	0.314s

cc @golang/runtime

Read is here: https://cs.opensource.google/go/go/+/master:src/internal/godebug/godebug.go;l=102;drc=39effbc105f5c54117a6011af3c48e3c8f14eca9
Write is here: https://cs.opensource.google/go/go/+/master:src/internal/godebug/godebug.go;l=74;drc=39effbc105f5c54117a6011af3c48e3c8f14eca9
The write is the initialization of the struct. It must happen before the read. Indeed, I believe the atomic store at https://cs.opensource.google/go/go/+/release-branch.go1.21:src/runtime/runtime.go;l=109;drc=1a9893a969e0a73dc4f1e48ed40ccaf29ec238a6 is publishing the value. But this is a runtime/internal/atomic value, which doesn't include race annotations. So I think we need to manually annotate that path.

@prattmic I was looking at this the other day and I also think we should instrument it manually. So something like this would suffice? I'm wondering how to come up with a test for this 😅
diff --git a/src/runtime/runtime.go b/src/runtime/runtime.go
index 0829a84e43..30e7226e0a 100644
--- a/src/runtime/runtime.go
+++ b/src/runtime/runtime.go
@@ -161,6 +161,7 @@ type godebugInc struct {
 }

 func (g *godebugInc) IncNonDefault() {
+	var racer int
 	inc := g.inc.Load()
 	if inc == nil {
 		newInc := godebugNewIncNonDefault.Load()
@@ -172,7 +173,9 @@ func (g *godebugInc) IncNonDefault() {
 		// *godebug.Setting.
 		inc = new(func())
 		*inc = (*newInc)(g.name)
+		raceacquire(unsafe.Pointer(&racer))
 		g.inc.Store(inc)
+		racerelease(unsafe.Pointer(&racer))
 	}
 	(*inc)()
 }

Has anyone analyzed why this happens with parallel testing? What Go debug events metric is incremented in that case?
Is there a workaround besides:

not using parallel tests
not using the race detector when parallel testing is enabled: go test -parallel=2
not using parallel tests when the race detector is not enabled: go test -race -parallel=1
All those solutions are unfortunately about avoiding parallel testing.

As godebugInc is about incrementing a metric, I did more investigation by dumping the metrics that are effectively incremented:
package rendezvous_test

import (
	"fmt"
	"os"
	"runtime/metrics"
	"strings"
	"testing"
)

func TestMain(m *testing.M) {
	// call flag.Parse() here if TestMain uses flags
	exitCode := m.Run()

	descs := metrics.All()
	var godebugMetrics []metrics.Sample
	for i := range descs {
		if strings.HasPrefix(descs[i].Name, "/godebug/") && strings.HasSuffix(descs[i].Name, ":events") {
			godebugMetrics = append(godebugMetrics, metrics.Sample{Name: descs[i].Name})
		}
	}
	metrics.Read(godebugMetrics)
	for i := range godebugMetrics {
		if godebugMetrics[i].Value.Uint64() > 0 {
			fmt.Printf("%s: %v\n", godebugMetrics[i].Name, godebugMetrics[i].Value.Uint64())
		}
	}

	os.Exit(exitCode)
}

The result:
go test -v -parallel 10 -run TestOne
=== RUN   TestOne
=== PAUSE TestOne
=== CONT  TestOne
--- PASS: TestOne (0.00s)
PASS
/godebug/non-default-behavior/panicnil:events: 1
ok  	github.com/dolmen-go/rendezvous	0.096s

I have in fact a test that runs panic(nil) on purpose. I have to check if I still want that test.

My final workaround is to drop the panic(nil) in my tests as it fails on Go 1.21 and is no longer relevant with Go 1.21+.

I'm having no luck with a reproducer, this is on Darwin amd64, 1.21.5, 1.21.3, also tip.  Here's my current attempt, I'm about to try this on darwin/arm64.  To get an event count I have to specify 1.20 in go.mod, or GODEBUG=panicnil=1, but in no case have I been able to tickle a race failure.  (I absolutely believe this is a bug, but a small test would be darn nice.)
package parpanic_test

import (
	"fmt"
	"os"
	"runtime/metrics"
	"strings"
	"testing"
)

//go:noinline
func panicNil() (e any) {
	defer func() {
		e = recover()
	}()
	panic(e)
}

var tests = []string{"ant", "bat", "cat", "dog", "emu", "fox", "gnu", "hen"}

func TestMany(t *testing.T) {
	t.Parallel()
	for _, tc := range tests {
		t.Run(tc, func(t *testing.T) {
			t.Log(panicNil())
		})
	}

}

func TestMain(m *testing.M) {
	// call flag.Parse() here if TestMain uses flags
	exitCode := m.Run()
	descs := metrics.All()
	var godebugMetrics []metrics.Sample
	for i := range descs {
		if strings.HasPrefix(descs[i].Name, "/godebug/") && strings.HasSuffix(descs[i].Name, ":events") {
			godebugMetrics = append(godebugMetrics, metrics.Sample{Name: descs[i].Name})
		}
	}
	metrics.Read(godebugMetrics)
	for i := range godebugMetrics {
		if godebugMetrics[i].Value.Uint64() > 0 {
			fmt.Printf("%s: %v\n", godebugMetrics[i].Name, godebugMetrics[i].Value.Uint64())
		}
	}
	os.Exit(exitCode)
}

Here is my small reproducer:
package main_test

import "testing"

func Test(t *testing.T) {
	t.Run("One", func(t *testing.T) {
		t.Parallel()
		defer func() {
			recover()
		}()
		panic(nil)
	})
	t.Run("Two", func(t *testing.T) {
		t.Parallel()
		defer func() {
			recover()
		}()
		panic(nil)
	})
}

$ GODEBUG=panicnil=1 go test -run 'Test$' -race

@dr2chase Your TestMany executes in parallel, but there is no parallel test to run. The subtests are not run in parallel.

@gopherbot please consider this for backport to 1.21.  It broke someone's integration tests, and since GODEBUG=panicnil (or go.mod version = 1.20) is a compatibility feature added to 1.21, it is a regression, if you look at it right.

The fix has been released with Go 1.21.6 (see #64757).


3D indicator update needed

I need to update this 3D indicator to an object instead of a gridmap to use less memory and space, preferably in Blender.

done


SWDEV-347383 - Remove code that sets -std and -fms-* flags

Totally agree with -std=c++, but who sets -fms-* now on the Windows path? hiprts, cmake, both?

Per Sam, those -fms-* flags are included in clang so no need to set them separately.


Loss exploding after few steps

`from functools import reduce
from operator import mul
from typing import Tuple
import numpy as np
import torch
import torchvision
import torch.nn as nn
from models.loss_functions.lsaloss import LSALoss
from models.base import BaseModule
from models.blocks_2d import DownsampleBlock
from models.blocks_2d import ResidualBlock
from models.blocks_2d import UpsampleBlock
from models.estimator_1D import Estimator1D
import cv2
class Encoder(BaseModule):
"""
CIFAR10 model encoder.
"""
def init(self, input_shape, code_length):
# type: (Tuple[int, int, int], int) -> None
"""
Class constructor:
    :param input_shape: the shape of CIFAR10 samples.
    :param code_length: the dimensionality of latent vectors.
    """
    super(Encoder, self).__init__()

    self.input_shape = input_shape
    self.code_length = code_length

    c, h, w = input_shape

    print (c,h,w)

    activation_fn = nn.LeakyReLU()

    # Convolutional network
    self.conv = nn.Sequential(
        nn.Conv2d(in_channels=c, out_channels=32, kernel_size=3, bias=False),
        activation_fn,
        ResidualBlock(channel_in=32, channel_out=32, activation_fn=activation_fn),
        DownsampleBlock(channel_in=32, channel_out=64, activation_fn=activation_fn),
        DownsampleBlock(channel_in=64, channel_out=128, activation_fn=activation_fn),
        DownsampleBlock(channel_in=128, channel_out=256, activation_fn=activation_fn),
    )
    self.deepest_shape = (256, h // 8, w // 8)

    # FC network
    self.fc = nn.Sequential(
        nn.Linear(in_features=reduce(mul, self.deepest_shape), out_features=256),
        nn.BatchNorm1d(num_features=256),
        activation_fn,
        nn.Linear(in_features=256, out_features=code_length),
        nn.Sigmoid()
    )

def forward(self, x):
    # types: (torch.Tensor) -> torch.Tensor
    """
    Forward propagation.

    :param x: the input batch of images.
    :return: the batch of latent vectors.
    """
    h = x
    print (type(h))
    h = self.conv(h)
    h = h.view(len(h), -1)
    o = self.fc(h)

    return o

class Decoder(BaseModule):
"""
CIFAR10 model decoder.
"""
def init(self, code_length, deepest_shape, output_shape):
# type: (int, Tuple[int, int, int], Tuple[int, int, int]) -> None
"""
Class constructor.
    :param code_length: the dimensionality of latent vectors.
    :param deepest_shape: the dimensionality of the encoder's deepest convolutional map.
    :param output_shape: the shape of CIFAR10 samples.
    """
    super(Decoder, self).__init__()

    self.code_length = code_length
    self.deepest_shape = deepest_shape
    self.output_shape = output_shape

    print (self.output_shape,"--")

    activation_fn = nn.LeakyReLU()

    # FC network
    self.fc = nn.Sequential(
        nn.Linear(in_features=code_length, out_features=256),
        nn.BatchNorm1d(num_features=256),
        activation_fn,
        nn.Linear(in_features=256, out_features=reduce(mul, deepest_shape)),
        nn.BatchNorm1d(num_features=reduce(mul, deepest_shape)),
        activation_fn
    )

    # Convolutional network
    self.conv = nn.Sequential(
        UpsampleBlock(channel_in=256, channel_out=128, activation_fn=activation_fn),
        UpsampleBlock(channel_in=128, channel_out=64, activation_fn=activation_fn),
        UpsampleBlock(channel_in=64, channel_out=32, activation_fn=activation_fn),
        ResidualBlock(channel_in=32, channel_out=32, activation_fn=activation_fn),
        nn.Conv2d(in_channels=32, out_channels=3, kernel_size=1, bias=False)
    )

def forward(self, x):
    # types: (torch.Tensor) -> torch.Tensor
    """
    Forward propagation.

    :param x: the batch of latent vectors.
    :return: the batch of reconstructions.
    """
    h = x
    h = self.fc(h)
    h = h.view(len(h), *self.deepest_shape)
    h = self.conv(h)
    o = h

    return o

class LSACIFAR10(BaseModule):
"""
LSA model for CIFAR10 one-class classification.
"""
def init(self,  input_shape, code_length, cpd_channels):
# type: (Tuple[int, int, int], int, int) -> None
"""
Class constructor.
    :param input_shape: the shape of CIFAR10 samples.
    :param code_length: the dimensionality of latent vectors.
    :param cpd_channels: number of bins in which the multinomial works.
    """
    super(LSACIFAR10, self).__init__()

    self.input_shape = input_shape
    self.code_length = code_length

    # Build encoder
    self.encoder = Encoder(
        input_shape=input_shape,
        code_length=code_length
    )

    # Build decoder
    self.decoder = Decoder(
        code_length=code_length,
        deepest_shape=self.encoder.deepest_shape,
        output_shape=input_shape
    )

    # Build estimator
    self.estimator = Estimator1D(
        code_length=code_length,
        fm_list=[32, 32, 32, 32],
        cpd_channels=cpd_channels
    )

def forward(self, x):
    # type: (torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor, torch.Tensor]
    """
    Forward propagation.

    :param x: the input batch of images.
    :return: a tuple of torch.Tensors holding reconstructions, latent vectors and CPD estimates.
    """
    h = x

    # Produce representations
    z = self.encoder(h)

    # Estimate CPDs with autoregression
    z_dist = self.estimator(z)

    # Reconstruct x
    x_r = self.decoder(z)
    # print (x_r.shape)
    x_r = x_r.view(-1, *self.input_shape)

    return x_r, z, z_dist

def load_dataset(data_path="/home/jbmai/Downloads/Defect Images-20190705T133320Z-001"):
# data_path = 'data/train/'
torchvision.transforms.Grayscale(num_output_channels=1)
trainTransform  = torchvision.transforms.Compose([
                                torchvision.transforms.Resize(size=(128,128), interpolation=2),
                                torchvision.transforms.ToTensor(),
                                ])

train_dataset = torchvision.datasets.ImageFolder(
    root=data_path,
    transform=trainTransform)

train_loader = torch.utils.data.DataLoader(
    train_dataset,
    batch_size=64,
    num_workers=0,
    shuffle=True
)
return train_loader

net = LSACIFAR10(input_shape=[3,128,128],code_length = 32,cpd_channels =100)
lossFunction = LSALoss(cpd_channels=100)
optimizer = torch.optim.SGD(net.parameters(), lr=0.001, momentum=0.9)
try:
checkpoint = torch.load("savedWeights/enc.pth")
net.encoder.load_state_dict(checkpoint)

checkpoint = torch.load("savedWeights/est.pth")
net.estimator.load_state_dict(checkpoint)

checkpoint = torch.load("savedWeights/dec.pth")
net.decoder.load_state_dict(checkpoint)

except Exception as e:
print (e)
for epoch in range(1000):  # loop over the dataset multiple times
running_loss = 0.0
d = load_dataset()
for i, (data,l) in enumerate(d):
    # get the inputs; data is a list of [inputs, labels]

    # print (data.shape)

    # zero the parameter gradients
    optimizer.zero_grad()

    # forward + backward + optimize
    # print (data)
    x_r,z,z_dist = net.forward(data)
    # print (x_r.shape)
    # print(data.shape)
    loss = lossFunction(data,x_r,z,z_dist)
    loss.backward()
    optimizer.step()

    # print statistics
    running_loss += loss.item()
    if i % 5 == 0:    # print every 2000 mini-batches
        print('[%d, %5d] loss: %.3f' %
              (epoch + 1, i + 1, running_loss / 5))
        running_loss = 0.0
if (epoch % 5)== 0 :
    # print ("--------------------{} epoch-----------".format(epoch))
    # net.encoder.eval()
    # net.estimator.eval()
    # net.decoder.eval()

    # z = net.encoder(data)
    # z_dist = net.estimator(z)
    # x_r = net.decoder(z).permute(0,2,3,1).detach().numpy()
    # out =x_r
    # print (type(out))
    # for i in range(out.shape[0]):
    #     # print (out.shape)
    #     # # out.permute(0,2,3,1)
    #     # print (out.shape)
    #     cv2.imwrite("constructedImages/outDec{}_{}.jpg".format(epoch,i),out[i,:,:,:]*255)
    #     # cv2.waitKey(0)

    #     # cv2.
    # net.encoder.train()
    # net.estimator.train()
    # net.decoder.eval()
    torch.save(net.encoder.state_dict(),("savedWeights/enc.pth"))
    torch.save(net.estimator.state_dict(),("savedWeights/est.pth"))
    torch.save(net.decoder.state_dict(),("savedWeights/dec.pth"))

print('Finished Training')`
Output :
<class 'torch.Tensor'>
[1,     1] loss: 727109273.600
<class 'torch.Tensor'>
[2,     1] loss:<PHONE_NUMBER>514337382531072.000
Hi can you help me rectify the issue.

Managed to correct above by changing hyperparameters,but after few steps i get
[1,    71] loss: 3293.943 <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> [1,    76] loss: 3096.135 <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> [1,    81] loss: 3161.412 <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> [1,    86] loss: 3176.714 <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> [1,    91] loss: 2969.099 <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> [1,    96] loss: 3247.497 <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> [1,   101] loss: 3068.246 <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> [1,   106] loss: 2769.233 <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> [1,   111] loss: 2766.316 <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> [1,   116] loss: 2718.537 <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> [1,   121] loss: 3055.004 <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> [1,   126] loss: 2576.473 <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> [1,   131] loss: 2947.091 <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> [1,   136] loss: 2869.674 <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> <class 'torch.Tensor'> [1,   141] loss: 2717.064 <class 'torch.Tensor'> Traceback (most recent call last):   File "modelTrainer.py", line 261, in <module>     x_r,z,z_dist = net.forward(data)   File "modelTrainer.py", line 188, in forward     z = self.encoder(h)   File "/home/jbmai/DefectsDetection/NoveltyDetection/models/base.py", line 33, in __call__     return super(BaseModule, self).__call__(*args, **kwargs)   File "/home/jbmai/anaconda3/envs/torchenv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__     result = self.forward(*input, **kwargs)   File "modelTrainer.py", line 73, in forward     o = self.fc(h)   File "/home/jbmai/anaconda3/envs/torchenv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__     result = self.forward(*input, **kwargs)   File "/home/jbmai/anaconda3/envs/torchenv/lib/python3.7/site-packages/torch/nn/modules/container.py", line 92, in forward     input = module(input)   File "/home/jbmai/anaconda3/envs/torchenv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__     result = self.forward(*input, **kwargs)   File "/home/jbmai/anaconda3/envs/torchenv/lib/python3.7/site-packages/torch/nn/modules/batchnorm.py", line 83, in forward     exponential_average_factor, self.eps)   File "/home/jbmai/anaconda3/envs/torchenv/lib/python3.7/site-packages/torch/nn/functional.py", line 1693, in batch_norm     raise ValueError('Expected more than 1 value per channel when training, got input size {}'.format(size)) ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 256])
ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 256])

Hi @learnermaxRL
I would advise using Adam for optimization.
As for the latter error you reported, it is likely to be due to a singleton batch (i.e., a batch with only one sample). It may be due to the DataLoader, try setting the drop_last flag to True.
D

Thanks,corrected it,however i can see that reconstructed images have negative values in the tensor,is it desirable?
sample slice of x_r =  [  19.986425    33.786083   109.08704  ]]
[[  49.809772   -32.651962    -1.5576267]
[  53.66301    -72.02914     48.711018 ]
[  39.252117   -81.27754     75.4854   ]
...
[  21.410696   -71.10042     68.18309  ]
[  -8.615957  -179.66095      8.810505 ]
[  44.986786    29.80011     93.024506 ]]
[[ -68.59759     64.74513     51.421898 ]
[ -18.552599    29.491028    69.56346  ]
[ -49.379646    45.368095    29.86158  ]
...
[ -84.07668     41.966274   100.4433   ]
[   1.8273218   58.350666    60.632793 ]
If yes than how to get back RGB image from them?

Negative values are not undesirable per se, as long as it is a concious choice.
What is the range of your input images?
D

input images are RGB images with standard 0-255 range being fed through dataloader pytorch
`def load_dataset(data_path="/home/ji/Downloads/aug/"):
# data_path = 'data/train/'
torchvision.transforms.Grayscale(num_output_channels=1)
trainTransform  = torchvision.transforms.Compose([
                                torchvision.transforms.Resize(size=(128,128), interpolation=2),
                                torchvision.transforms.ToTensor(),
                                ])

train_dataset = torchvision.datasets.ImageFolder(
    root=data_path,
    transform=trainTransform)

train_loader = torch.utils.data.DataLoader(
    train_dataset,
    batch_size=16,
    num_workers=0,
    shuffle=True,
    drop_last=True
    # pin_memory=True
)
return train_loader

`

I would advise standardizing input images.
E.g., try making each channel zero-mean and unit std.
D

that makes sense,however standardization wouldnt guarantee negative output right,?

Also i checked my input data is between 0 and 1 already,
Pytorch default backend 1 for images are Pillow, and when you use ToTensor() 2 class, PyTorch automatically converts all images into [0,1].
So shouldn't the reconstruction output be standardized before calculating reconstruction loss or perhaps use another activation function?

I would advise using a linear activation function for the reconstruction and providing

Are you talking about the x_r from decoder,should i use sigmoid on top of last layer in  decoder output,, i am trying to train the model to learn the reconstruction of the image itself,so GT will be the same image
x_r,z,z_dist = net.forward(data)         # print (x_r.shape)         # print(data.shape)         loss = lossFunction.forward(data,x_r,z,z_dist)         loss.backward()         optimizer.step()
Is there anything else I need to do?

net = LSACIFAR10(input_shape=[3,128,128],code_length = 64,cpd_channels =100) lossFunction = LSALoss(cpd_channels=100)

The sigmoid is not mandatory. I would advise not to use it.
The rest of the code seems fine. Would an [0-1] input deliver those reconstructions?
D

Does the reconstruction loss go down?

How have you done the reconstruction of image,as in what was your input range in cifar images for instance  and  what was the activation function of decoder and range of reconstructed output?

It has high values ~2500,since i am using sigmoid the learning is pretty slow,i suspect the gradients are very less due to higher output before sigmoid.After 100 epochs on dataset of size 1400 with batch size 64 the loss only decrease by approx 50 units.
Can you guide me as to what have you done

range of CIFAR-10 images: [0-1]
decoder activation function: None
range of the reconstruction: approximately [0-1]

I see ,but how come the values are negative in my case,i mean weights arent negative,my input [0,1] no non linearity applied on top of your code,what is going wrong here?

I saw you are using 128x128 images. The number of downsampling in the model is tuned on 32x32 images. You might then have a huge linear layer before the bottleneck of the autoencoder, with many parameters slowing down learning.
As for the negative values, it is really weird. I would try optimizing the reconstruction loss only (plain autoencoder) and see if the problem fades.

 :)

Yeah sure will do,but how do I deal with reconstruction with negative values,using sigmoid is the last option I would prefer.Any help?


Increase max_connections for staging db

There have been problems with jobs on staging since upgrading to release 23.0. It is difficult to debug because the database  keeps running out of available connection slots.

50 has been enough for dev and staging up until now, and dev still seems OK. I'm not really sure how many connections we expect from the galaxy application. It may have increased with the new version. If this doesn't fix it I'll try 100.


go format

format code comments
check error
set is a package. Revise set to txSet
Do we consider using the assert package instead of directly using t.Error in unit testing

@moyrne
1、First of all , thank you for submitting the PR
It is recommended that you submit PRs separately for refactoring and formatting, for easier review and faster merges.
2、 In addition, Revise set to txSet, I didn't understand why it was done
3、Do we consider using the assert package instead of directly using t.Error in unit testing
yes,  unit tests will all use assert instead of the standard library, you can also contribute code if you are interested ：）

@moyrne thanks LGTM 👍


Update the native-fp8 guide with cudnn layer norm

This PR updates the guide to include the new flag to enable the cudnn layer norm.
cc. @ashors1 @terrykong @nouiz

Should we add it to https://github.com/NVIDIA/JAX-Toolbox#environment-variables with the same TE disclaimer?
Thoughts @nouiz ?


chore(DTFS2-NONE): update package-lock for portal

Introduction
package lock in portal was out of date.
Resolution
updated package lock

Codecov Report

Merging #2313 (41a6a0d) into main (2390d16) will increase coverage by 2.06%.
Report is 110 commits behind head on main.
The diff coverage is 4.65%.

@@            Coverage Diff             @@
##             main    #2313      +/-   ##
==========================================
+ Coverage   21.01%   23.08%   +2.06%
==========================================
  Files        1030     1032       +2
  Lines       25229    25613     +384
  Branches     4081     4092      +11
==========================================
+ Hits         5303     5913     +610
+ Misses      17370    17186     -184
+ Partials     2556     2514      -42

Flag
Coverage Δ

ui-test
65.14% <ø> (+0.33%)
:arrow_up:

unit-test
3.78% <4.65%> (+3.06%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files
Coverage Δ

dtfs-central-api/src/helpers/isNumber.js
100.00% <100.00%> (ø)

.../src/v1/routes/middleware/headers/check-api-key.js
0.00% <ø> (ø)

...l-api/src/v1/routes/middleware/headers/security.js
0.00% <ø> (ø)

...entral-api/src/v1/routes/middleware/headers/seo.js
0.00% <ø> (ø)

...-api/src/v1/routes/middleware/remove-csrf-token.js
0.00% <ø> (ø)

dtfs-central-api/src/helpers/index.js
0.00% <0.00%> (ø)

...c/v1/controllers/cron-jobs/cron-jobs.controller.js
0.00% <0.00%> (ø)

.../durable-functions/durable-functions.controller.js
0.00% <0.00%> (ø)

.../v1/controllers/portal/deal/get-deal.controller.js
0.00% <0.00%> (ø)

.../portal/gef-facility/update-facility.controller.js
0.00% <0.00%> (ø)

... and 7 more

... and 405 files with indirect coverage changes
:mega: Codecov offers a browser extension for seamless coverage viewing on GitHub. Try it in Chrome or Firefox today!


Added bog page but table is taking long to load

Firstly pull the latest changes
Deploy on surge at your URI
I'll look into issue


[WIP] Add Kokkos Sparse Operations

Before submitting
Please complete the following checklist when submitting a PR:

[x] All new features must include a unit test.
If you've fixed a bug or added code that should be tested, add a test to the
tests directory!

[x] All new functions and code must be clearly commented and documented.
If you do make documentation changes, make sure that the docs build and
render correctly by running make docs.

[x] Ensure that the test suite passes, by running make test.

[ ] Add a new entry to the .github/CHANGELOG.md file, summarizing the
change, and including a link back to the PR.

[x] Ensure that code is properly formatted by running make format.

When all the above are checked, delete everything above the dashed
line and fill in the pull request template.

Context:
Description of the Change:
Add native support for the calculation of sparse Hamiltonians' expectation values.
Benefits:
Performant calculation with Pennylane-Lightning
Possible Drawbacks:
Not that I'm aware of.
Related GitHub Issues:
Not that I'm aware of.

Hi @AmintorDusko, I think Kokkos does not support MSVC. We may need to try clang-cl instead... I will ping you on slack.

Hi @AmintorDusko, I think Kokkos does not support MSVC. We may need to try clang-cl instead... I will ping you on slack.

Yes. Its looks like it! Thank you very much @chaeyeunpark!
Also, your add_kokkos fork was quite useful!
Thank you for looking into it before me!

Hi Amintor, I also suspect that these lines in a workflow may change the shell to cmd from PowerShell. As we are using clang-cl for Windows, the lines may not be necessary. Could you try after removing these lines?

Hi Amintor, I also suspect that these lines in a workflow may change the shell to cmd from PowerShell. As we are using clang-cl for Windows, the lines may not be necessary. Could you try after removing these lines?

Sure. I will try.

When tested locally, I found that running pip install -e . without cibuildwheel already raises FTK2009 (path too long) error. I could remove this error by adding shell=True, but other errors started to appear (even in Linux). Let me check this more tomorrow.

When tested locally, I found that running pip install -e . without cibuildwheel already raises FTK2009 (path too long) error. I could remove this error by adding shell=True, but other errors started to appear (even in Linux). Let me check this more tomorrow.

Sure! Thank you @chaeyeunpark!

Hi, @mlxd. I brought what I considered to be the essential changes in your PR.
These changes are been incorporated as commits starting with win64_path_fix: .
Windows wheels are building nicely now! Nice job on that!

Awesome work Amintor! I see that you have enabled the serial device. Any chance we can get benchmarks for the OPENMP device? We could potentially see great speedups here.

Awesome work Amintor! I see that you have enabled the serial device. Any chance we can get benchmarks for the OPENMP device? We could potentially see great speedups here.

Hi @trevor-vincent! I made several benchmarks comparing Serial, OpenMP and Threads devices.
Because the calculation of a single expval for Sparse matrix is very fast (even in a single thread), adding any type of multi-threading only incorporates overhead.
The total time is sizeable, because the single expval is calculated hundreds of times, and because of other device operations.

Just one more question as I am not sure how Kokkos works: Will this functionality also use GPU when we set Kokkos to use GPU?

Hi @chaeyeunpark. In theory, functionalities implemented with Kokkos are device-independent.
Having said that, maybe we would need to add some extra steps (memory-related) to accomplish this.
Without access to GPUs, it is difficult to be 100% sure.

Just one more question as I am not sure how Kokkos works: Will this functionality also use GPU when we set Kokkos to use GPU?

Hi @chaeyeunpark. In theory, functionalities implemented with Kokkos are device-independent. Having said that, maybe we would need to add some extra steps (memory-related) to accomplish this. Without access to GPUs, it is difficult to be 100% sure.

I think this is a good one to come back to later once @trevor-vincent has finished his work with Kokkos. It may be straight forward to take advantage of any other devices, since we can technically target an entire collection of backends (CUDA, HIP, SYCL, HPX, OpenMP and C++ threads). We can try to target SYCL as the backend and it may pick-up Intel GPUs for some testing locally.

Thanks, @mlxd @AmintorDusko, for the answers. However, I was asking as I wondered whether it is possible to use GPU backends for quantum gate operations but CPU for sparse operations (or vice versa). I was thinking about the strategy to integrate Kokkos kernels (for gates) with the current codebase. Of course, as @mlxd said, we should discuss it more when the Kokkos kernel is available, but we may need to decide who is responsible for choosing the device for Kokkos (we or users?).

Thanks, @mlxd @AmintorDusko, for the answers. However, I was asking as I wondered whether it is possible to use GPU backends for quantum gate operations but CPU for sparse operations (or vice versa). I was thinking about the strategy to integrate Kokkos Kernels (for gates) with the current codebase. Of course, as @mlxd said, we should discuss it more when the Kokkos kernel is available, but we may need to decide who is responsible for choosing the device for Kokkos (we or users?).

Hi @chaeyeunpark, thanks for your clarification.
Yes, one can design the code so that part of it is in the CPU (host), and the other part is in the GPU.
You can define more than one device. Maybe toasters in the future (@trevor-vincent ?)


updated all packages to use @solana/web3.js as a peer dependency

Going off the same issues as outlined in #340, this PR updates all packages to use @solana/web3.js as a peer dependency.
I don't see anything breaking with this change as the readme directs users to install @solana/web3.js as a dependency.

So this looks good, but I think these packages now need web3.js as a dev dependency so it gets installed when building each one from source.

Good catch, updated! Thanks @jordansexton

Looks great, I made the order of peer/deps/devDeps consistent. I'm wondering why the yarn lockfile changes when running yarn install in this branch though.

Nevermind the yarn.lock issue, it looks like it's just a result of some nested dependency update. Thanks for this!


7.0.0-beta.9 conflicts symbol.observable to redux (and other have old symbol-observable version)

Bug Report
Current Behavior
A clear and concise description of the behavior.
repro at https://github.com/kwonoj/rxjs-symbol-test, npm i && npm run build will raise compilation error.

https://github.com/ReactiveX/rxjs/pull/5874 introduces changes to symbol, which also updated in symbol-observable to support this (https://github.com/benlesh/symbol-observable/blob/master/CHANGELOG.md#300-2020-11-02). But some of packages, for examble redux (https://github.com/reduxjs/redux/blob/v4.0.5/package.json#L53) relies on older version of symbol-observable conflicts with latest rxjs changes.
This is particularly tricky as there's no easy workaround instead of trying to mutate module resolution somehow.
Expected behavior
A clear and concise description of what you expected to happen (or code).
Environment

Runtime: [e.g. Node v${x}, Chrome v${x}]
RxJS version:
(If bug is related) Loader, build configuration: [e.g webpack, angular-cli version, config]

Possible Solution

Additional context/Screenshots
Add any other context about the problem here. If applicable, add screenshots to help explain.

Ugh. I guess we can change it back to symbol. I'm just not sure how we can ever get that type right. I guess we can't. :\

Or we could upstream other changes... maybe deprecate symbol-observable and try to get people to move to 3.x of that library?

get people to move to 3.x of that library

That's only pass I can see as well, since I don't see easy workaround. But problem is it's really long tail, i.e we can't say for sure how long will it take the majority of redux user upgrade to latest to get away with old symbol-observable.
Maybe it's worth to ask redux upstream at least to publish latest version to remove / or update symbol-observable? Not immediate solution but will help.

redux upstream at least to publish latest version to remove / or update symbol-observable? Not immediate solution but will help.

at least if we have published version we can recommend to upgrade redux as well.

Well, really the issue is if they're using redux AND rxjs, (so perhaps worth pulling in @jayphelps because redux-observable).

redux-observable is safe. I specifically meant redux cause latest release have symbol-observable (https://github.com/reduxjs/redux/blob/v4.0.5/package.json#L53) and looks like master seems safe from it (https://github.com/reduxjs/redux/blob/master/package.json#L53), but there isn't version published with these changes.

At this point I think maybe the best answer is that RxJS just sort of "shadow supports" Symbol.observable. In other words, since it's a crap shoot that TypeScript will even figure out the right thing to do, perhaps we don't add the type or type handling, and instead we only handle the proper runtime behavior. I just don't know.
I seriously doubt anyone has ever gotten this working properly in TypeScript haha.

redux-observable is safe. I specifically meant redux cause latest release have symbol-observable (https://github.com/reduxjs/redux/blob/v4.0.5/package.json#L53) and looks like master seems safe from it (https://github.com/reduxjs/redux/blob/master/package.json#L53), but there isn't version published with these changes.

Specifically, I mean people using redux-observable will have this issue because they're definitely using both redux and rxjs.

If redux is willing to patch, I don't personally see it as a particularly egregious issue that people would have to update their redux to be able to use the latest rxjs. Things like that happen.

I mean people using redux-observable will have this issue because they're definitely using both redux and rxjs

that is true. it was my case 😭

Yeah, certainly not ideal if it can be avoided. I don't have full background to comment on that, though.

We should revert to whatever we were doing before. Which is just having symbol and not unique symbol. ... symbol-observable will also need to be rolled back.


Instead of seen for error, check for stream being cancelled

Running a closure just on seeing an error is too early. Instead, as intended, let's run it when the stream is closed.

****************************************************************

libgrpc.so

     VM SIZE                                                                   FILE SIZE
 ++++++++++++++ GROWING                                                     ++++++++++++++
  [ = ]       0 [None]                                                          +32  +0.0%
  +0.1%     +32 src/core/ext/transport/chttp2/transport/chttp2_transport.cc     +32  +0.1%
      +7.7%     +49 grpc_chttp2_complete_closure_step                               +49  +7.7%

  +0.0%     +32 TOTAL                                                           +64  +0.0%

****************************************************************

libgrpc++.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

[trickle] No significant performance differences

[microbenchmarks] Performance differences noted:
Benchmark                                                                                atm_cas_per_iteration    nows_per_iteration
-----------------------------------------------------------------------------------------  -----------------------  --------------------
BM_StreamingPingPongWithCoalescingApi<InProcessCHTTP2, NoOpMutator, NoOpMutator>/0/0/0     +4%
BM_StreamingPingPongWithCoalescingApi<InProcessCHTTP2, NoOpMutator, NoOpMutator>/0/0/1     +4%
BM_StreamingPingPongWithCoalescingApi<MinInProcessCHTTP2, NoOpMutator, NoOpMutator>/0/0/0  +4%
BM_StreamingPingPongWithCoalescingApi<MinInProcessCHTTP2, NoOpMutator, NoOpMutator>/0/0/1  +4%
BM_UnaryPingPong<InProcessCHTTP2, NoOpMutator, NoOpMutator>/2097152/2097152                                         -12%
BM_UnaryPingPong<MinInProcessCHTTP2, NoOpMutator, NoOpMutator>/2097152/2097152                                      -12%

****************************************************************

libgrpc.so

     VM SIZE                                                                   FILE SIZE
 ++++++++++++++ GROWING                                                     ++++++++++++++
  +0.0%     +16 [None]                                                         +200  +0.0%
  +0.1%     +16 src/core/ext/transport/chttp2/transport/chttp2_transport.cc     +16  +0.1%
      +0.7%     +16 grpc_chttp2_cancel_stream                                       +16  +0.7%

  +0.0%     +32 TOTAL                                                          +216  +0.0%

****************************************************************

libgrpc++.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

Objective-C binary sizes
*****************STATIC******************
  New size                      Old size
 1,980,691      Total (>)      1,980,683

 No significant differences in binary sizes

***************FRAMEWORKS****************
  New size                      Old size
10,811,763      Total (<)     10,811,768

 No significant differences in binary sizes

[microbenchmarks] Performance differences noted:
Benchmark                                                                              cpu_time    real_time
-------------------------------------------------------------------------------------  ----------  -----------
BM_PumpStreamClientToServer<InProcess>/2097152                                         +5%         +5%
BM_PumpStreamClientToServer<InProcess>/262144                                          +6%         +6%
BM_StreamingPingPongMsgs<InProcess, NoOpMutator, NoOpMutator>/32768                    +10%        +10%
BM_StreamingPingPongMsgs<MinInProcess, NoOpMutator, NoOpMutator>/262144                +6%         +6%
BM_StreamingPingPongWithCoalescingApi<InProcess, NoOpMutator, NoOpMutator>/262144/1/1  +6%         +6%
BM_UnaryPingPong<InProcess, NoOpMutator, NoOpMutator>/262144/262144                    +7%         +7%
BM_UnaryPingPong<MinInProcess, NoOpMutator, NoOpMutator>/0/262144                      +4%         +4%
BM_UnaryPingPong<MinInProcess, NoOpMutator, NoOpMutator>/32768/32768                   +4%         +4%

This is no longer relevant


Updated otel collector dependencies to address vulnerabilities.

Issues addressed:
https://github.com/advisories/GHSA-mvff-h3cj-wj9c
https://github.com/advisories/GHSA-77vh-xpmg-72qh

LGTM


Clean up evaluation demo notebook and utils.py file.

rail/examples/evaluation_examples/utils.py -> This stuff is only used by the evaluation demo notebook. I think the answer here is to clean up that notebooks and either get stuff from somewhere else, or just add it to the notebook. If we have other notebooks that start needed some of this stuff, we can think about where it belongs in the wider RAIL ecosystem. This should be it's own issue.

Though, if we refactor the Evaluation code, it might be better to wait...

Util refactoring is done in PR #138, closing issue.


Change layout of Endpoint Explorer

The current layout of Endpoint Explorer is wasting a lot of space and requires users to scroll every time they want to change the endpoint:

I propose to merge "resource" and "endpoint" menus into one, expandable menu (like the one in the devportal). Endpoint options and response can be moved to the right side of the menu:

hmm idk. I'd rather use that space a place to put a description of the chosen endpoint

In my original mockups of the laboratory, there was some docs stuff on the right side of the page.
I think moving it there would not be helpful because it would cram everything into a small space.

Agree with @jedmccaleb and @irisli, though I do think it's a problem that, once the user selects an endpoint, a new interactive form appears at the base of the screen. We might want to move them down automatically or otherwise highlight this section more aggressively when it appears.


[server] Rewrite gRPC read Service to align with Netty-Based HTTP Server

Rewrite gRPC read Service to align with Netty-Based HTTP Server

Refactored the Netty handler to enable code reuse for gRPC handlers
Added missing APIs to the gRPC service to close feature gaps with the
HTTP implementation.
Fixed the gRPC I/O request path threading model: previously, requests
were handled by the gRPC executor thread pool, but now they are processed
by the StorageReadExecutor thread pool for better performance.

Deprecated the old get and multiGet gRPC services, which relied on
a single schema and encouraged tightly coupled code. This refactor paves
the way for implementing server-side streaming support for batch get
requests more easily.

Simplified the request processing logic by replacing the chain of
responsibility pattern with a single processor class.

How was this PR tested?

WIP (UTs and E2E will be added)
Does this PR introduce any user-facing changes?

[x] No. You can skip the rest of this section.
[ ] Yes. Make sure to explain your proposed changes and call out the behavior change.

I'm still working on client side changes to support streaming within a single route. I've figured out structure for it and working  changes

I'm still working on client side changes to support streaming within a single route. I've figured out structure for it and working changes

Nice, looking forward to it. When you're ready with your next iteration, please rebase, push it up and LMK. I'll take another look.


chore(deps): update renovate/renovate docker tag to v35.25.0

This PR contains the following updates:

Package
Update
Change

renovate/renovate
minor
35.24.3 -> 35.25.0

Release Notes

renovatebot/renovate
v35.25.0
Compare Source
Features

manager/bitbucket-pipelines: support docker image object (#21102) (34b5401)

v35.24.6
Compare Source
Bug Fixes

merge config constraints over package file extracted constraints (#21207) (0b79b3d)

Miscellaneous Chores

use node: protocol imports (#21181) (176282e)

v35.24.5
Compare Source
Build System

deps: update dependency cacache to v17.0.5 (#21214) (c87d84c)

v35.24.4
Compare Source
Bug Fixes

onboarding: force branch creation to prevent diff-check (#21200) (874cfe8)

Miscellaneous Chores

deps: update dependency @types/node to v18.15.10 (#21203) (39493b7)

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

:tada: This PR is included in version 4.4.6 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


[Feature Request] Read cargo fmt configurations from .cargo/config.toml

I would like a single project to use a custom formatter. I have created a .cargo/config.toml with the following contents:
[env]
RUSTFMT = "echo"

however, the usual formatter appears to be called.
Using RUSTFMT="echo" cargo fmt returns /path/to/main.rs --edition 2021 as expected.

Thanks for reaching out!
cargo fmt is really just a thin wrapper around rustfmt that reads metadata from your Cargo.toml file, and rustfmt only reads configurations values from the .rustfmt.toml or rustfmt.toml files.
@calebcartwright correct me if I'm wrong, but I don't think there's been any discussion about reading configurations / env variables from the .cargo/config.toml. Is it common for other tools to read configuration values from there?
@aDogCalledSpot to the best of my knowledge, and the Contributing guide the RUSTFMT environment variable is meant as a testing utility when developing and working on rustfmt locally.
As you've demonstrated above, you can certainly use the RUSTFMT environment variable to point to an arbitrary binary but that's not the intended use case.

Interesting, I wasnt aware of the intended use case. Sorry.
Since this is a feature request now, Ill give some more context of what I'm trying to achieve.
I have a project using Yew with some other collaborators. I recently found yew-fmt which fomats the code in Yew's html macro which is otherwise left untouched by rustfmt, and otherwise hands the rest of the code off to rustfmt.
It makes sense to use this for the project but I dont want to set RUSTFMT globally in my shell.

Looks like this is where the RUSTFMT environment variable comes from: https://github.com/rust-lang/rustfmt/pull/4419, so maybe it's not just for testing as I had assumed.
Thanks for sharing your usecase. In order to support it we'd need to read configurations from the .cargo/config.toml. I just don't know if that's something other tools do or if that's something we want to support in rustfmt.

I created a shell script called my_custom_formatter.sh with the following content:
echo "Hey There you're using my custom formatter!!!"
echo "passing the following to rustfmt!"
for var in "$@"
do
    echo "$var"
done

Running the cargo-fmt binary built from source with
cargo --config="env.RUSTFMT= \"path/to/my_custom_formatter.sh\"" run --bin cargo-fmt -- --version

prints
Hey There you're using my custom formatter!!!
passing the following to rustfmt!
--version

Running RUSTFMT="/path/to/my_custom_formatter.sh" cargo fmt -- --version also prints
Hey There you're using my custom formatter!!!
passing the following to rustfmt!
--version

However running the cargo-fmt that I have installed on my machine with
cargo --config="env.RUSTFMT= \"path/to/my_custom_formatter.sh\"" fmt -- --version

prints out rustfmt 1.6.0-nightly (1c05d50c 2023-10-21).
From what I can tell this doesn't seem to be an issue with the rustfmt or cargo-fmt binaries. It seems that cargo has special knowledge of the RUSTFMT environment variable: https://github.com/rust-lang/cargo/blob/363a2d11320faf531f6aacd1ea067c6bc08343b9/src/doc/src/reference/environment-variables.md?plain=1#L78-L80

I just tried this in my test project where have the env var set in the config.toml, and to my surprise I get:
error: failed to merge --config key `env` into `/home/ari/code/alpico/app/frontend/.cargo/config.toml`

Caused by:
  failed to merge key `RUSTFMT` between /path/to/.cargo/config.toml and --config cli option

Caused by:
  failed to merge config value from `--config cli option` into `/path/to/.cargo/config.toml`: expected table, but found string

The values from env are only passed through to the spawned processes by cargo - not handled directly by cargo.

Yeah. I was just looking at these docs: https://doc.rust-lang.org/cargo/reference/config.html#env, but like I said before cargo reads the RUSTFMT environment variable.
It seems that everything is working as expected, and you'll need to make sure that RUSTFMT is set if you want to leverage it to call a different formatting binary.


[SYSTEMML-573] Frame CSV Spark read/write

thanks @asurve  but this requires some work - please keep the intended distributed frame representation (https://issues.apache.org/jira/browse/SYSTEMML-560) in mind.

binaryBlock2CSV slice: The slicing of binary blocks to rows is not required for frames as each frame block covers all columns.
csv2BinaryBlock nnz: Without the CSVAnalysisFunction, the accumulator from which you try to obtain the nnz will always return 0. Furthermore, for frames we don't even maintain the nnz, so please remove the related code.
csv2BinaryBlock shuffle: The whole point of the variable length frameblocks is to prevent shuffle. Currently, the mergeBykey causes unnecessary shuffle and should be removed. Please, also delete the related MergeBlocksFunction and mergeByKey primitives.
Blocksize: Again, we want to create variable size blocks at the end of partitions, so we cannot use UtilFunctions.computeBlockSize which would always give you 1k, even at the end of partitions.
Tests: Please factor out your new testcases into a separate converter test class as it has nothing to do with frame read/write. Given this refactoring you could also drop the execution type as it is misleading because these are component tests not integration tests on dml script level.

I have addressed the changes.

Binary block keys: on csv to binary block, it seems that we output the wrong keys (block indexes instead of block row offset) which is wrong with variable-sized blocks.
Formatting: Finally, there are also some formatting issues, e.g.,  FrameRDDConverterUtils, line 295.

addressed the comment.

LGTM


Revert "Running linter with all tags in .md files (#1491)"

This reverts commit 7ef91d6b797c1cb892c33f3c5d3431e7ee516de2.
Issues with a dependency package causes error as here
reverting until package is fixed and republished.
This checklist is used to make sure that common issues in a pull request are addressed. This will expedite the process of getting your pull request merged and avoid extra work on your part to fix issues discovered during the review process.
PR information

[ ] The title of the PR is clear and informative.
[ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For information on cleaning up the commits in your pull request, see this page.
[ ] Except for special cases involving multiple contributors, the PR is started from a fork of the main repository, not a branch.
[ ] If applicable, the PR references the bug/issue that it fixes.
[ ] Swagger files are correctly named (e.g. the api-version in the path should match the api-version in the spec).

Quality of Swagger

[ ] I have read the contribution guidelines.
[ ] My spec meets the review criteria:

[ ] The spec conforms to the Swagger 2.0 specification.
[ ] The spec follows the guidelines described in the Swagger checklist document.
[ ] Validation tools were run on swagger spec(s) and have all been fixed in this PR.

Requested a fix for the package here


global classes for dev version of abapGit

something we have been discussing in the slack channel
it would make it easier to do integration to Eclipse/ADT and other tools
Still want to have a single automatic generated report for copy/paste install
this will require some steps and a lot of trial and error,
1: get rid of global and cross class macros, macros inside methods are ok
2: get rid of global types(move to interface? ZABAPGIT_DEFINITIONS)
3: get rid of global constants(move to interface? ZABAPGIT_DEFINITIONS)
4: tool https://github.com/larshp/local-to-global for help converting

constants, todo:
_object_check_timestamp - difficult due to the RETURN
_reduce_state - new method?
_add_dialog_fld - many fields
_add_rule - move constants into code and call a method with table parameter to initialize rules?
related to unit tests:
_generate_parse
_generate_order
_generate_extend
_append_local
_append_remote
_append_state
_append_result
_new
_old
_expected

first PR merged, more to follow

Wow you are actually going through with this 👍
So the general idea would be to have some (loosely coupled) functionality depend on the existence of the "global class installation" and that installation can be done using the local copy paste abapGit installation? And abapGit development itself would then be done using the local or global classes?

installation: via copy pasting 1 report with local classes
development: global classes
And still having abapmerge building the copy-paste report, there is a lot of issues that needs to be handles, currently just trying out some stuff

woudl abapmerge convert Global classes to local classes?

yeah, its the plan, lets see if it works out

see https://github.com/larshp/abapGit-global for WIP

todo: macros from above and rename the local class of the following so the name fits into 30 chars,
rename: zcx_abapgit_2fa_token_gen_failed
rename: zcx_abapgit_2fa_token_del_failed
rename: zcx_abapgit_2fa_communication_error
rename: zcl_abapgit_2fa_authenticator_base
rename: zcl_abapgit_2fa_github_authenticator
rename: zcl_abapgit_2fa_authenticator_registry
rename: zcl_abapgit_proxy_authentication
rename: zcl_abapgit_object_oriented_class
rename: zcl_abapgit_object_enho_wdyconf
rename: zcl_abapgit_object_enho_interface
rename: zcl_abapgit_object_oriented_interface
rename: zcl_abapgit_object_oriented_factory
rename: zif_abapgit_object_oriented_object_fnc
rename: zcl_abapgit_oo_object_serializer
rename: zcl_abapgit_object_oriented_base
rename: zcl_abapgit_object_tabl_validation
rename: zcl_abapgit_tabl_validation_dialog
rename: zif_abapgit_object_comparison_result
rename: zcl_abapgit_null_comparison_result
rename: zcl_abapgit_gui_page_background_run
rename: zcl_abapgit_gui_page_background
rename: zcl_abapgit_gui_page_branch_overview
rename: zcl_abapgit_gui_page_db_display
rename: zcl_abapgit_gui_page_repo_settings
rename: zcl_abapgit_persistence_migrate
rename: zcl_abapgit_persistence_background
rename: zcl_abapgit_persistence_settings
rename: zcl_abapgit_repo_content_browser
rename: zcl_abapgit_services_background
rename: zcl_abapgit_transport_to_branch
rename: zcl_abapgit_gui_view_repo_content

Just fyi, do you know about this in SE24?

I have personally never used it but seems like the exact use case (initial local to global conversion).

nope, didnt know, cool feature
https://github.com/larshp/local-to-global will work with very little manual work

dynamic references, pages and objects?

[ ] move 'GC_ABAP_VESION' to somewhere else
[ ] fix/test FORM callbacks for popups, see zabapgit_forms.prog.abap
[ ] macro _REDUCE_STATE, new method?
[ ] macro _ADD_RULE, move constants into code and call a method with table parameter to initialize rules?
[ ] macro _OBJECT_CHECK_TIMESTAMP - difficult due to the RETURN
[ ] macro _ADD_DIALOG_FLD refactoring, current approach does not work for global classes
[ ] selection screen password popup, ZCL_ABAPGIT_PASSWORD_DIALOG
[ ] rename method 'serialize_abap_czif_abapgit_source', longer than 30 chars

It is difficult to refactor all the code in one go, so perhaps a new plan is needed.
Instead of moving everything in 1 commit, then move to global classes over time.
A good starting point could be moving https://github.com/larshp/abapGit/blob/master/src/zabapgit_zlib.prog.abap to a global class, as it does not have any requirements to LCL_ classes.
This would get it started, and the infrastructure needed for the refactoring would be built, after this it would just be a lot of hard work 😄

https://github.com/larshp/abapGit-global updated

Another plan, have abapmerge create the merged file in the following order:
A: all DEFINITIONs DEFERRED
B: all DEFINITIONs
C: all IMPLEMENTATIONs
D: all testclasses, note that local test class names might collide
E: concatenated PROG
Assumptions:

no macros in global classes
no local classes in global classes
no calls from global classes to local classes

Example:
CLASS lcl_bar DEFINITION DEFERRED.
CLASS lcl_foo DEFINITION DEFERRED.

CLASS lcl_foo DEFINITION.
  PUBLIC SECTION.
    METHODS: foo IMPORTING bar TYPE REF TO lcl_bar.
ENDCLASS.

CLASS lcl_bar DEFINITION.
  PUBLIC SECTION.
    METHODS: bar IMPORTING foo TYPE REF TO lcl_foo.
ENDCLASS.

CLASS lcl_foo IMPLEMENTATION.
  METHOD foo.
    NEW lcl_bar( ).
    RETURN.
  ENDMETHOD.
ENDCLASS.

CLASS lcl_bar IMPLEMENTATION.
  METHOD bar.
    NEW lcl_foo( ).
    RETURN.
  ENDMETHOD.
ENDCLASS.

note that additional classes will not be made global until #904 is solved

I think for further globalization e.g. #964 we should also make the local interface lif_defs (ZABAPGIT_DEFINITIONS) global. I'm not quite sure whether abapmerge handles this correctly.
This would be a first try.

try opening a PR for it, then I'll try running abapmerge on it
alternatively just npm install abapmerge and run it locally

I'll submit the PR tomorrow. If I run abapmerge locally I get this error.

Any idea?

something went wrong in the regex, https://github.com/larshp/abapmerge/blob/master/src/class_list.ts#L19, would have to start the debugger, but perhaps some Windows specific stuff

yep, it's a windows thing. Line breaks are \r\n not just \n. This regex would work.
/^((.|\r\n)*ENDCLASS\.)\s*(CLASS(.|\r\n)*)$/i
Anyhow how do I call abapmerge correctly? I'm now quite sure which arguments I have to supply.

Correctly: yes, see https://github.com/larshp/abapGit/blob/master/travis.sh#L2 it just pipes the output to file

As you already guessed global interfaces don't work.

Next steps:

lcl_gui_asset_manager to global, note that this might require changes in abapmerge, due to the following abapmerge commands

        " @@abapmerge include zabapgit_css_common.w3mi.data.css > _inline '$$'.
      WHEN 'JS_COMMON'.
        " @@abapmerge include zabapgit_js_common.w3mi.data.js > _inline '$$'.

Plus we have to move lcl_object_* classes to global, this will have to be done in one step. Be sure to check performance when installing this many classes, the new CLAS deserialization should make it possible, but who knows

Current core dependencies:
lcl_objects <-> lcl_repo*
lcl_repo -> lcl_tadir -> lcl_objects
lcl_repo also uses lcl_exit, need some kind of new solution for the exits setup

Done! closing ✨


don't add empty tokens (due to trimming) to the index to avoid errors while searching

The text 'test ???' will lead to the following tokens:
['test', '???']
After trimming this becomes:
['test', '']
The empty string then causes problems while searching the index. This fix avoids retaining the empty string after trimming.

Please ignore the second commit. I'm not sure why it was auto-added to this pull request

Thanks for taking a look at this. Perhaps I'm missing something, but what is the issue here? I was unable to reproduce the error but maybe I'm not understanding what the issue is.
I did the following:
var idx = lunr(function () {
  this.field('foo')
})

idx.add({id: 1, foo: 'foo ???'})

idx.search('foo') // returns the right document
idx.search('???') // returns no documents

idx.tokenStore.length //= 1
idx.corpusTokens.toArray() //= ["foo"]

This is the behaviour I would expect. What behaviour did you expect and what are you seeing?

The empty token I described lead to an error while searching (inout irrelevant). I can't give you the exact error message as I'm not at work right now, but it was something like a null-pointer equivalent while iterating over the tokens.
It could be that we're only seeing this issue because we've removed both the stemming and stop word filter from the pipeline, giving us a different behaviour than what you are seeing.

Ah yes, that's probably it. The stop word filter rejects empty tokens. And so removing it means that those empty tokens make it into the index and then you get the issue you see.
It seems strange to have the stop word filter include the empty token, its not really a word! I think it is safe to have that check in the trimmer and then not have it as part of the stop word filter. Would you mind updating this pull request with the following:

Remove this commit
Remove the empty word from the stop word filter

The language extensions also include the empty token in the stop word filters, but from looking at the code they also remove the trimmer function. I think this is fine though because they cannot use the built in trimmer as it is really focused on English and doesn't work well with non ascii characters.

Also, out of interest, what use case do you have where you need to remove the stemmer and stop word filter? Just trying to understand how you are using lunr and whether there is something lunr can do to make your implementation easier.

Ah okay, that makes sense. You'll receive another patch within the next couple of days.
We've removed the stop-word filter because we've had some issues with finding some words that were removed. For simplicity, let's just say that the application allows to assemble a book using pre-defined chapters that can be searched for using lunr. One section was named 'in-depth something' and lunr didn't find anything for 'in-', a lot for 'in-d' and I think you had to type in a lot more characters to actually limit the search result to 'in-depth'. For our use case I think we're better off without the stop word filter, as the search is not used for large documents, but rather short section headings.
As for the stemmer, I believe the issue was with words like 'companies' where you'd find the entry when typing up to 'compan', but would suddenly not find it anymore if you appended an 'i'. Of course typing in the whole word you'll find it again.
Since the results are shown instantaneously the user experience was a bit weird when the entry you were searching for appears, disappears and appears again as you type. We have therefore removed it as well, as our users know what they are searching for anyway - it's just meant to speed up the process of finding something for them (there are a lot of headings in the index).
Regarding the stemming issue: I don't exactly know how you could improve upon it, as you can't really know that 'compani' will be 'companies' when the whole word is typed and therefore can't stem it yet, can you?

Yeah I think the stemmer has caused issues like this for other people, I don't really have a good suggest to fix this I'm afraid.

Please have another look at the changes I have committed.

I've pushed your changes in the latest version of lunr. Thanks again for you help!


Validate shard-WIP

Overview
In order to provide for potential read-only clusters, we needed to be able to point a pilosa cluster at some shared space and only load the shards applicable to each node. Integeration testing is working, but some internal testing needs to be added.  Just wanting a bit of feedback before i continue
Fixes #1808

this was merged in #1900


some issue about the size of input image

As we now,the size of input image in Inception-ResNet-v1 is 299X299X3.However,i see the code in train_softmax.py and i find that your size of input image is 160X160X3.I wonder konw why not resize 299X299X3?

You can do with 299x299x3, the performance would be slightly better, but the batch size should be much smaller and the training time would be much longer.

OK,thank you for your replay


Timecode Subframes in output metadata

Most of the time, positions of markers will be on exact frame starts (frame boundaries) and have a subframes component of 0. However in some projects markers may have subframe position information.

Having timecode output with truncated subframes is fine for most use cases.
We could however allow subframes to be output in the CSV very easily, perhaps with a --show-subframes CLI option.
Or perhaps we enable subframes output in CSV automatically but only if at least one marker in the project has subframes > 0. The reason is that many projects will not have subframes in their timecode since FCP defaults to inserting markers at frame starts. The only time you run into this odd situation we have here is when mixing frame rates or converting projects between frame rates.

We could however allow subframes to be output in the CSV very easily, perhaps with a --show-subframes CLI option.

Yes. I think is is best to provide users the option. Some users may or may not like dealing with subframes.

That's my instinct too. Hide them by default but provide an opt-in flag to enable them.

Done. --enable-subframes CLI flag now adds subframes to all timecode strings.


seventh fall

Thank you for the report. Hopefully is resolve with https://github.com/ZhengPeiRu21/mod-playerbots/commit/246b5b021dc3f4342c34ab4af850e0318294c4b8.


I source the corresponding path about intel (icc, mkl, mpi), but there are no results.

I do these:
source .../compilervars.sh intel64
etc.

Hi, you would need to source from the directory where the Intel tools are installed.
E.g: default install path would be /opt/intel/bin/compilervars.sh intel64

@wuzhijiexia , please try Dheevatsa's suggestion and reopen the issue if needed.


Doesn't work with FontAwesome

I tried this
.babelrc
{
  "plugins": [
    ["direct-import",
      {
        "modules": [
          "@fortawesome/free-brands-svg-icons",
        ]
      }
    ]
  ]
}

some.js
import { faGithub } from '@fortawesome/free-brands-svg-icons'

It imports entire svg suit.
It works with the other plugin though, however this one poorly handles @material-ui imports. Would be nice to handle everything with one plugin.
    ["transform-imports", {
      "@fortawesome/free-brands-svg-icons":  {
        "transform": "@fortawesome/free-brands-svg-icons/${member}",
        "preventFullImport": true,
        "skipDefaultConversion": true
      }
    }]

this transforms import to
import { faGithub } from '@fortawesome/free-brands-svg-icons/faGithub'

Not that icon is not a default expor.

@ksinas hey! thanks for the report, but unfortunately it's impossible to make it work with the @fortawesome/free-brands-svg-icons.
The only thing that plugin does is a static check of the module entry of the package, find reexports, and redirects your imports. It works well with @material-ui/icons because they reexport all the icons: https://unpkg.com/browse/<EMAIL_ADDRESS>But font awesome exports a single file: https://unpkg.com/browse/<EMAIL_ADDRESS>So if you're using any modern bundler, you don't need a babel plugin, the bundler will tree shake everything for you.


desktop-style MenuFlyout

This sample creates a desktop-style context menu, but when I right-click on Windows10 Desktop or TaskBar, there is no animation which is ugly. So how to remove that from MenuFlyout?

Can you be more specific which sample you are referring to?

I mean the project : Context Menu(Xaml), I want the menu to show up like right-click on the Windows desktop without the animation like scrolling.

This doesn't appear to be a bug in the sample. The animations for MenuFlyout can be customized the same way as any other XAML animation. This Web page has an example.
If you need further assistance customizing the animation, you can open a support case with our Developer Support team. Please use these steps below for further help:

Visit http://aka.ms/storesupport and sign-in with your Microsoft account
Under the “App Development” section choose the appropriate type – such as Building apps and games, and then choose from the Problem type/Category to get engaged through a formal support channel


fix(v2): use absolute path to manifest file

Motivation
If you visit any page other than the home page, for example https://v2.docusaurus.io/docs/, then you can see the following error in the console:

Failed to load resource: the server responded with a status of 404 ()   manifest.json

Or something like this:

Manifest: Line: 1, column: 1, Syntax error.

Have you read the Contributing Guidelines on pull requests?
Yes
Test Plan
Preview (see motivation section).
Related PRs
(If this PR adds or changes functionality, please take some time to update the docs at https://github.com/facebook/docusaurus, and link to your PR here.)

Hmmm that's annoying 😅
I tried to keep relative paths because I wanted the PWA to work in deploy previews, using baseurl. Seems it does not work on the deploy preview now :'(

I don't get what you mean. Right now the browser cannot find the manifest file because a relative URL is specified.

I mean, we want the (+) icon to be there in all cases

Currently, it only works on the homepage (which is the problem your PR fixes, as it will work from /docs/ too)
But your PR breaks the (+) on deploy previews: https://deploy-preview-3554--docusaurus-2.netlify.app/classic/

While it used to work on PRs. (but only on PR classic homepage 🤪 ):

The ideal solution should make it work:

for all subpaths: / or /docs/myDoc
for deploy previews: /classic/ or /classic/docs/myDoc

This way it's easier to review PWA issues in deploy previews

Aha, I figured it out now, so I changed to use baseUrl const, it should be OK, I guess.

LGTM, we'll see if main site still works after :D


castling not working

hi! I cannot figure out how to castle. i've tried to type o-o and o-o-o, but it says invalid, try again. any idea why?

 Have you tried O-O or O-O-O?


Add Count quantity type

Is Fraction the standard unit for "count"?
See https://developer.apple.com/library/ios/documentation/HealthKit/Reference/HKUnit_Class/#//apple_ref/occ/clm/HKUnit/unitFromString:

Should a count quantity be an integer?
https://developer.apple.com/library/ios/documentation/HealthKit/Reference/HKUnit_Class/#//apple_ref/occ/clm/HKUnit/countUnit
The Fraction quantity type is like the percent unit.
https://developer.apple.com/library/ios/documentation/HealthKit/Reference/HKUnit_Class/#//apple_ref/occ/clm/HKUnit/percentUnit

Closing this issue due to no response. I think int is a good type for countable quantities! You could use a validation attribute to ensure it is >=0.


tenv managed terraform causes illegal characters for terraform output -json

Describe the bug
When the terraform version is managed with tenv, it is adding a Resolved version from ...` message at the beginning of the output. This causes JSON (or any structured terraform output) to have illegal characters in it.
To Reproduce
Steps to reproduce the behavior:

Install a version of terraform with tenv tf install 1.5.0 (or any other version).
Run terraform apply
Run terraform output -no-color -json
See error in the output

❯ terraform output -no-color -json
Resolved version from /Users/<user_name>/.tenv/Terraform/version : 1.5.0
{
  "test_output": "Hello world!"
}

Expected behavior
tenv shouldn't be prefixing the terraform resolution message: Resolved version from /<user_home>/.tenv/Terraform/version : 1.5.0 or should at least have a documented way to suppress this warning.
Screenshots
Environment (please complete the following information):

OS: MacOS Catalina
tofuenv version v1.0.15

Additional context
This error doesn't occur with tfenv.

There is a documented way to suppress informative display : TENV_QUIET=true
Also, as stated in #63 , i will probably change to redirect all informative display to stderr


endpoint for searching user_no with psn-name ?

Hi,
Do you know the endpoint to search for the user_no, using the psn-profile-name ?

To my knowledge, none exists.
Services like kudosprime have built and maintain their own databases that they use to do lookups.

I wonder how they get that information. Is there a public pool of user data
they scrape?
On Sun, 12 Apr 2020 at 20:33, Matthew Williams<EMAIL_ADDRESS>wrote:

To my knowledge, none exists.
Services like kudosprime have built and maintain their own databases that
they use to do lookups.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/alexpersian/gt7sp-api/issues/9#issuecomment-612657766,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJOEOB3DNY5345EXSUW72YLRMICPFANCNFSM4MGQLMJQ
.

I believe that they have a setup that simply accesses the user profile endpoint by number and records the psn username to a massive database.
Due to the number of records and queries required, I’m guessing they use polling or some sort of method to check and update user records.
I have noticed over time that often usernames are not immediately updated to reflect psn name changes. Usually it takes a few days to reflect changes.

Aha I see.. Thanks for that!


Add a 'timer' function

The 'timer function' can be used to record the amount of time a function takes, and then send that value using the 'timing' function.

+1

These changes have been merged into master in the upstream repo (brightcove). Closing.


[WIP][PT Run][New Plugin] GUID/UUID plugin

TO-DO:

it's almost done, wait for your comments
Need new symbols (ping  @niels9001)
installer, sign, unit-test, docs

Summary of the Pull Request
What is this about:

New PT Run plugin to generate GUID/UUID
Three formats

Simple
In Groups, format: 8-4-4-4-12
Uniform Resource Name (URN)

Enclosing GUID/UUID with entered characters.
Automatically rotates brackets at the end  of the line (for ( , [,  { )
Automatically rotates brackets misspelled at the beginning of the line (for ) , ],  })
Context menu:

Copy as uppercase to clipboard
Copy as lowercase to clipboard

What is included in the PR:

New PT Run
Localization (.resx)

How does someone test / validate:

Manual testing

Quality Checklist

[x] Linked issue: inspired by #3200
[ ] Communication: I've discussed this with core contributors in the issue.
[ ] Tests: Added/updated and all pass
[ ] Installer: Added/updated and all pass
[x] Localization: All end user facing strings can be localized
[ ] Docs: Added/ updated
[ ] Binaries: Any new files are added to WXS / YML

[ ] No new binaries
[ ] YML for signing for new binaries
[ ] WXS for installer for new binaries

Contributor License Agreement (CLA)
A CLA must be signed. If not, go over here and sign the CLA.

@TobiasSekan You only need 1 icon, right? Any ideas what kind of icon would make sense :-)?

Here is an icon which might be interesting: https://github.com/microsoft/PowerToys/issues/3200#issuecomment-1004905377

@TobiasSekan You only need 1 icon, right? Any ideas what kind of icon would make sense :-)?

Yes only one icon, but a dark and a light version

@niels9001 something like this
https://store-images.s-microsoft.com/image/apps.14092.13510798883190732.87c5f44c-cf37-418a-aeb1-04ee609ddd62.8bd3be3f-208a-498d-9a34-ad7cb339c686

@niels9001 something like this

Isn't this the on we have for vs code plugin.

I would suggest F413 as icon

Added hexadecimal format (for the c/c++ programmers)
All results now in lower case
Subtitle now describe the format of the Guid/UUID
Address feedback

we could rename the plugin to value generator and add "random number" and "temp folder name" and othe things that programmers need. That would reduce the number of different plugins.
Possible syntax:

{ guid "[{ => as it is
{ random 5 => number with 5 digits
{ temp 7 alphanum => 7 alphanumeric chars
{ mac => random mac address (when creating vms)
{ ipv6 <network part>/<cidr>
Other ideas ...

@crutkas can you set this PR back to DRAFT state

@crutkas can you set this PR back to DRAFT state

Can't you do this in the review menu/section?

done

@TobiasSekan what is needed to wrap this up?  Just a review?

@TobiasSekan what is needed to wrap this up?  Just a review?

We decided in the past to create a more general value creator plugin. If still wNt to do this there is a bit more work needed.


Component Inheritance Performance

Because of the known inheritance performance issues, I propose that we change the base Component class to be factory functions instead. Once you start inheriting more than 1 or 2 levels there is a performance hit, so I'd avoid creating small classes that save minimal lines of code that are inherited by all components.  A solution would be to create factory functions that can be used by components to generate this repeated code without actually inheriting. When creating a new component instead of writing:
class MyComponent extends Component {
...
}

// Common JavaScript
class Component {
   init() {...}
   create() {...}
   ....
}

Developers would call the factory function instead the class:
class MyComponent {
   Component.createAPI(MyComponent);
}

// Common JavaScript for components
Component = {
   createAPI(proto) {
        proto.init = () => { ... }
        proto.create = () => { ... }
        ...
   }
}

The factory functions support both static and non-static methods/properties.
This would not change the end-user experience of consumers but will take in to consideration the performance hit of inheritance and be more friendly to consumers as this framework is absorbed by other frameworks/applications.
Working example: https://github.ibm.com/Watson-Health/design-system/tree/develop/src/components/tooltip and https://github.ibm.com/Watson-Health/design-system/blob/develop/src/global/scripts/component.js

Is this issue related to a specific component?
No

What offering/product do you work on? Any pressing ship or release dates we should be aware of?
WHDS - Initial launch is end of Q1

References around issue:
From MDN:
The lookup time for properties that are high up on the prototype chain can have a negative
impact on the performance, and this may be significant in the code where performance is
critical. Additionally, trying to access nonexistent properties will always traverse the
full prototype chain.

There's another article that goes in to more depth: https://www.toptal.com/javascript/javascript-prototypes-scopes-and-performance-what-you-need-to-know

I do not have benchmark/side-by-side comparisons to show the improvements as I’ve known about the issues for a while (i’ve been writing prototype JS for years) and just didn’t do it based on what I read. I also think that historically inheritance with prototype was not as straight forward and due to it’s learning curve some developers stayed away from it. I think that changed with the introduction of classes and people starting to think that JavaScript is now an Object orientated language causing more conversation about the topic beyond the two references above.
However with using the inheritance approach, that means every component has a minimum of 3 levels of inheritance from the get go (Object > Base Component > Specific Component). I would recommend saving inheritance for components that have a lot more similarities to avoid getting even bigger chains. Those base component properties are ones that are accessed more often too. I feel like it does not save the developer that much work to not have the base component versus calling a function that generates most of the properties/methods.
As a framework that is consumed by other applications/frameworks, performance could become an even bigger issue as carbon classes get extended and only adding to the level of nesting.  Currently within WH we are looking at a chain like: Carbon > WHDS > WHDS Payer Provider > Applications.
This chain is necessary to provide the needed support for the applications and has been discussed at length across teams. This means that there is a lot of areas where there can be added inheritance along that pipeline if someone needs to extend a class. I think Carbon needs to be VERY aware of how people intend to use Carbon and it needs to be as performant as possible. Our applications deal with a lot of big data so we don't want to limit applications or add performance hits knowingly. I think with proper documentation the factory function is quite easy to pick up and implement.

Carry over comments from original Issue from @asudoh:

My knee-jerk reaction was that why Justin Fagnani has the statge1 proposal for mix-in if the prototype inheritance has performance problems.

Refs: https://draft.li/blog/2016/12/22/javascript-engines-hidden-classes/ https://github.com/davidmarkclements/v8-perf/blob/master/readme.md#removing-properties-from-objects

Hi @elizabethsjudd! Thanks so much for taking the time to copy this over from the old repo 😄  really appreciate this.
Just to frame how we look to evaluate performance suggestions, we take the following items into account:

Is the performance regression identifiable and demonstrable
Is the performance regression a common path for a library consumer
Does the suggested fix appropriately address the first part of this list without significant drawbacks to other parts of the code
Does the suggested fix align with best practices and can be easily understood and contributed

An aspect of the issue that you've raised is that we would need to verify the first part of this checklist as we look to evaluate the component base class performance characteristics. This is where the benchmarks come in as it's important to identify the following characteristics:

The prototype chain impacts some aspect of performance during some part of an application lifecycle, whether that's parsing, initializing, mounting, unmounting, or other runtime characteristics
The impact is something that causes a poor experience for the end user, the developer, or both
The impact is something that occurs during typical/standard usage for a carbon adopter

Benchmarks also help us verify that browsers aren't already optimizing typical JS usage patterns under the hood. We've often seen theoretical performance cases (like O(n) in a prototype chain lookup) that often don't match the implementation.
@asudoh did some interesting research here, namely with respect to v8 internals that suggest that if the prototype lookup path is hot for a component that the lookup cost ends up being O(1) Link. If that's the case, then thankfully this shouldn't be a realistic bottleneck in an application that uses Carbon 😄
However, as we stated above, we need numbers to verify that the cost associated with dynamic property access is significant. This cost also needs to be something that's incurred regularly enough to flag this as a potential bottleneck in a typical application.

@asudoh was auditing IBM Cloud console last night on his older Mac to see what the initialization cost was and ended up sharing the screenshot below. I believe (correct me if I'm wrong here, Akira) that the highlighted section points to the actual initialization cost of the components on that page, which in this case is around ~32ms, or two frames.
Would be interesting to see runtime performance characteristics as well 😮

To my understanding the hit would show up more during runtime.  I'd also be curious as to know what components where on the page, how many, etc as that would also have a large input on the initialization time.
I was thinking about this the other day and I really would like to do some comparison tests but we don't really have WHDS set with enough for it to be a real comparison. I could test Ink, as it also doesn't use inheritance, but I also know there is some bloat with patches currently and again don't know how accurate the results would be. I think if we had just carbon tooltips we could compare it with WHDS tooltips.

Another good resource for this discussion: https://medium.com/@tverwaes/setting-up-prototypes-in-v8-ec9c9491dfe2
Namely:

Prototypes have 2 main phases: setup and use. Prototypes in the setup phase are encoded as dictionary objects. Stores to prototypes in that state are really fast, and do not necessarily need to enter the C++ runtime (a boundary crossing which is pretty expensive).

Any direct access to the prototype, or access through a prototype chain, will transition it to use state, making sure that all such accesses from now on are fast.

In short, the ideal way to setup prototypes is as follows:
var proto = func.prototype = Object.create(…);
proto.method1 = …
proto.method2 = …


Fix X icon

Provide the exact quote of the error or issue
The X icon should be X instead of blue bird.
Does the feature already exist? Please link to it.
No response
Are you willing to participate in fixing this issue and create a pull request with the fix?
None

Fixed


Fix for loop in EventLoop::run optimised out in a release build on macOS

Fix in relation to #148, doesn't feel very elegant, so am open to suggestions for something nicer.

cc https://github.com/rust-lang/rust/issues/28728

Added a commit to this which addresses tomaka/rodio#107

I don't know how coreaudio works, but I guess I can merge this anyway?

Yeah I think so, just gets things working again

Thanks


Mobile App

Now that PWA is on React, is there any plan to introduce mobile app (android, ios, react-native) which consumes the backend GraphQL API?

There are no such plans. That exactly the role of PWA - it is supposed to give you the experience of a native mobile app with a mobile-first design, offline mode and notifications.

Hi !
Can i get open source code of Saleor Android App ?
Thanks

There is no Android app, the storefront app is built in JavaScript: https://github.com/mirumee/saleor-storefront/

@maarcingebala  Hi sir !
Can you please me ?
I am generating PWA apk of saleor but it's throwing following error.
Please tell me solution.
Thanks


Add query_parameter_to_metadata and query_parameter_to_header http filters

Title: Add query_parameter_to_metadata and query_parameter_to_header http filters
Description:
Currently there is a header -> metadata filter (header_to_metadata) and similar filters for other mapping can be useful:

query param -> metadata
query param -> header

Relevant Links
https://github.com/envoyproxy/envoy/issues/5281

@lizan @qiannawang @bochunz

is it possible to rename that existing filter? If not, seems like adding the additional config that handles query param will make the filter name awkward.