Is  changing MAXVIFS from 32 to 64 in the kernel an issue?

There is an pfSense activity related to "the possible to pass MAXVIFS from 32 to 64 in the kernel and in pimd".
It that an issue for the actual pimd v3 build?

This used to be a HARD kernel limitation, the bits that encode VIFs are in a 32-bit unsigned integer.  This is a heritage from the MROUTING stack that both *BSD and Linux have. The pimd user space daemon should be capable of handling more, but that has not been tested for a few years.

The kernel define MAXVIFS must be changed so any user space application can see and utilize this
For pimd there's a configure option called --with-max-vifs=NUM, where NUM is the max

I surched for "max-vif" in the "git files" and found
"pimd is limited to the number of MAXVIFS interfaces listed in the
kernel headers.  In Linux see /usr/include/linux/mroute.h. "
Does that imply that under normal conditions pimd automatically reads/gets!! the number from the OS!?
So that the possible --max-vifs settings is not necessary at all under normal conditions!? (I specially refer to FreeBSD of course)
(If not is there a default value?)

Yes, that's correct.  If the OS header files list 33 as MAXVIFS then pimd will use 33 ... be aware though, this was last tested in 2015 and then only on Linux.  I would be surprised if it just worked out of the box.

If you tell me where it is read I can set a trece and test it

You asked for it ... it's A LOT!  I would definitely recommend against going into this as a
beginner in C code, PIM protocol, or kernel knowledge.  It is not for the faint of heart.
The MAXVIFS constant is defined in the operating system headers, pimd carries some
replacement headers for really old systems with the MROUTING stack but without the
headers -- this is not the case for most modern systems, so do not use or rely on the
values in the pimd/include directory.

DISCLAIMER: I will not spend any time helping to debug changes related to MAXVIFS.
I will only accept patches/pull-requests if the submitter has taken the time to test their patch
against at least Linux and *BSD systems, and preferably using the Ixia ANVL protocol suite.

src/kern.c:    char           input[IFNAMSIZ], output[MAXVIFS * (IFNAMSIZ + 2)] = "";
src/rsrr.c:	char oifs[MAXVIFS + 1];
src/rsrr.h:    uint8_t  out_vifs[MAXVIFS];		/* outgoing vif bitmask */
src/route.c:    uint8_t old_oifs[MAXVIFS];
src/route.c:    uint8_t new_oifs[MAXVIFS];
src/route.c:    uint8_t new_leaves[MAXVIFS];
src/route.c:    uint8_t new_oifs[MAXVIFS];
src/route.c:    uint8_t old_oifs[MAXVIFS];
src/route.c:    uint8_t new_leaves[MAXVIFS];
src/route.c:    uint8_t oifs[MAXVIFS];
src/route.c:    uint8_t new_joined_oifs[MAXVIFS];  /* The oifs for that particular mrtentry */
src/route.c:    uint8_t old_joined_oifs[MAXVIFS] __attribute__ ((unused));
src/route.c:    uint8_t old_pruned_oifs[MAXVIFS] __attribute__ ((unused));
src/route.c:    uint8_t old_leaves[MAXVIFS] __attribute__ ((unused));
src/route.c:    uint8_t new_leaves[MAXVIFS];
src/route.c:    uint8_t old_asserted_oifs[MAXVIFS] __attribute__ ((unused));
src/route.c:    uint8_t new_real_oifs[MAXVIFS];    /* The result oifs */
src/route.c:    uint8_t old_real_oifs[MAXVIFS];
src/route.c:		uint8_t oifs[MAXVIFS];
src/route.c:    uint8_t new_pruned_oifs[MAXVIFS];
src/vif.c:struct uvif	uvifs[MAXVIFS]; /* array of all virtual interfaces          */
src/vif.c:    for (vifi = 0, v = uvifs; vifi < MAXVIFS; ++vifi, ++v)
src/vif.c:    if ((numvifs + 1) == MAXVIFS) {
src/pim_proto.c:    uint8_t oifs[MAXVIFS];
src/pim_proto.c:    uint8_t pruned_oifs[MAXVIFS];
src/pim_proto.c:    uint8_t entry_oifs[MAXVIFS];
src/debug.c:    char oifs[MAXVIFS+1];
src/debug.c:    char joined_oifs[MAXVIFS+1];
src/debug.c:    char pruned_oifs[MAXVIFS+1];
src/debug.c:    char leaves_oifs[MAXVIFS+1];
src/debug.c:    char asserted_oifs[MAXVIFS+1];
src/debug.c:    char incoming_iif[MAXVIFS+1];
src/igmp_proto.c:    if (vifi >= MAXVIFS)
src/ipc.c:	char asserted_oifs[MAXVIFS+1];
src/ipc.c:	char incoming_iif[MAXVIFS+1];
src/ipc.c:	char joined_oifs[MAXVIFS+1];
src/ipc.c:	char pruned_oifs[MAXVIFS+1];
src/ipc.c:	char leaves_oifs[MAXVIFS+1];
src/ipc.c:	char oifs[MAXVIFS+1];
src/config.c:	if (numvifs == MAXVIFS) {
src/config.c:    for (vifi = 0, v = uvifs; vifi < MAXVIFS; ++vifi, ++v)
src/config.c:    for (vifi = 0, v = uvifs; vifi < MAXVIFS; ++vifi, ++v)
src/mrt.h:    uint8_t		  oifs[MAXVIFS];		/* The current result oifs	    */
src/mrt.h:    uint8_t		  joined_oifs[MAXVIFS];		/* The joined oifs (Join received)  */
src/mrt.h:    uint8_t		  pruned_oifs[MAXVIFS]; 	/* The pruned oifs (Prune received) */
src/mrt.h:    uint8_t		  asserted_oifs[MAXVIFS];	/* The asserted oifs (lost Assert)  */
src/mrt.h:    uint8_t		  leaves[MAXVIFS];		/* Has directly connected members   */
src/defs.h:extern struct uvif	uvifs[MAXVIFS];
src/vif.h:#define PIMD_VIFM_CLRALL(m)		(memset(m, 0, MAXVIFS))
src/vif.h:#define PIMD_VIFM_COPY(mfrom, mto)	(memcpy(mto, mfrom, MAXVIFS))
src/vif.h:    return memcmp(m1, m2, MAXVIFS) ? 0 : 1;
src/vif.h:    for (n = 0; n < MAXVIFS; n ++) {
src/vif.h:    for (n = 0; n < MAXVIFS; n ++) {
src/vif.h:    for (n = 0; n < MAXVIFS; n ++)
src/vif.h:    for (i = 0; i < MAXVIFS; i++) {
src/vif.h:#define NO_VIF		((vifi_t)MAXVIFS)  /* An invalid vif index */

I took a look at the code and did some build-tests. Some findings, I do not guarantee that all my conclusions are correct! Never the less I hope it helps :)   Some remarks based on my actual knowledge:

I am far from sure that pimd will work for an OS having more than 32 VIFS
I think that there should be a second dynamic act_max_value de-terminated at runtime (at the start of the program (or more))
I think that static structure sizes are not the best idea (there is history of course)

Some findings
MAXVIFS is a constant set during the build process by configure.ac. the result is used to allocate static vif related arrays/structures (: ) (e.g. vif.h is calculating static structs)
This implies that if the destination os is compiled with a different maxvif number, there is a problem!
Some test results related to the build process (FreeBSD)

./configure --with-max-vifs=64		# force the max number of vifs to 64 (from the default for most os 32) has no effect (MAXVIFS=32). Note here that Only include\linux\netinet\mroute.h has a variable called CUSTOM_MAX_VIFS
changing include/feebsd/netinet/ip.mroute.h #define MAXVIFS 32 t0 21 has no effect
changing include/feebsd2/netinet/ip.mroute.h #define MAXVIFS 32 t0 22 has no effect

I am not sure where the 32 config.ac generates is coming from. 	AC_DEFINE_UNQUOTED(CUSTOM_MAX_VIFS, $max_vifs, [Custom MAX VIFs in kernel.])], [max_vifs=32]) ??????
configure.ac

it would be nice if configure.ac could list which OS it detected and related which include it did use (under Summary)
AS_HELP_STRING([--with-max-vifs=NUM], [Kernel maximum number of allowed VIFs, default: 32.
However I think that 32 can be different is specified different in related ip.mroute,h (if that would work as original intended)
generating a build number including some version indicator and a build date would be nice too (shown by pimctl --version?)

definition files

there are nested includes, which makes it hard to understand what is really referenced from within a certain .c file
defs.h seems to have additional definitions in reality belonging in another .h !! ==> vif.h !!!! (/* vif.c */). Is that a correct conclusion ?

I could do (and are willing to do) some test with debug statements with a running pimd, but that would only be helpful if maxvifs is dynamic and my actual feeling is that it is not ......

I understand you're interested in working on this, but please read the disclaimer I posted previously!
Any work on this is bound to run into major problems and cause changes I'm not prepared to take any part in for the foreseeable future. I have other priorities, both in my life personally, and in this project (or other projects I also maintain).
Thank you for understanding.

Perhaps the temporarily solution is quite simple ...... just raising the default from 32 to 64 !?

Possibly, but again, that is a kernel/OS define.

I have to better understand config.ac. I do not yet track down how and where config.ac terminates os-type and the actual maxvifs.

MAXVIFS is set in a system/kernel/OS header, usually somewhere down in /usr/include/*.
The workaround in configure.ac is just for Linux.  Where all open source pieces come from different developers/teams and are glued together in various distributions that often forget to update their system header files.
On the BSDs, when adding support for more than 32 kernel VIFs, adjust the kernel header file, which is part of the system and included automatically by userspace.  This is the ideal scenario and does not include changing pimd sources at all.

Yep on my development system ip-mroute.h is in /usr/include/netinet
configure.ac host-os is determinate my computer as freebsd12.2 so I added "freebsd12*)" with no action for the moment
(I did try CPPFLAGS="-I../usr/include/netinet/", which does not work  (conftest.c:11:10: fatal error: 'ac_nonexistent.h' file not found)
Using a script I am executing among other things following commands:
Start with building using git: ./autogen.sh (1), ./configure (with options) (2), make clean all, make dist
Then building with port (usiung distribution file): make makesum, make stage (3), make stage-qa, make package
The numbered actions (1,2,3) do generate MAX VIF values.
I used four ^MAX VIFS test^ settings

MAX VIFS in ip-mroute.h = 32
configure override (if active) with --with-max-vifs=64
config.ac; AC_DEFINE_UNQUOTED = 99
CPPFLAGS="-I../usr/include/netinet/" yes/no active

With those settings I did three tests
test-1: mroute = 32, --with-max-vifs=64,  AC_DEFINE_UNQUOTED = 99, CPPFLAGS active
result: 1) = 64 2) = 64 3) = 99
test-2A: mroute = 32, no override,  AC_DEFINE_UNQUOTED = 99, CPPFLAGS active
result: 1) = 64 2) = 99 3) = 99
test-2B: mroute = 32, no override,  AC_DEFINE_UNQUOTED = 99, CPPFLAGS deactivated
result: 1) = 64 2) = 99 3) = 99
Result 1) seems to relay on what happened in the preceding run (not sure)
As noted before the mroute.h MAX VIFS value is dubious, since it is likely, but no one can grantee that the value used for the package target machine is the same.
So as far as I know:

when using git the Kernel MAX VIFs can be set using  --with-max-vifs=64
however packages for FreeBSD are normally build using port. only option there seems to be "AC_DEFINE_UNQUOTED=NUM"

Note here that I think that for FreeBSD MAX VIFS is normally determined by config.ac AC_DEFINE_UNQUOTED standard having a value of 32.
So for the moment based on what I know, I suggest to recompile with a modified  AC_DEFINE_UNQUOTED you are running FreeBSD and you need a package supporting more vifs than the default 32. I know a bit dirty ..
Louis
PS doing tests I noticed some errors in config.log
onftest.c:11:10: fatal error: 'ac_nonexistent.h' file not found
#include <ac_nonexistent.h>
configure: failed program was:
| /* confdefs.h */
conftest.c:54:6: warning: incompatible redeclaration of library function 'strlcpy' [-Wincompatible-library-redeclaration]
char strlcpy ();
conftest.c:55:6: warning: incompatible redeclaration of library function 'strlcat' [-Wincompatible-library-redeclaration]
char strlcat ();
^
conftest.c:55:6: note: 'strlcat' is a builtin with type 'unsigned long (char *, const char *, unsigned long)'
1 warning generated.

Related to non-Linux system, closing.


After upgrading virtualenv (13.0.1) I get a warning _markerlib

I now get this error when I use vex:
/usr/local/bin/vex:5: UserWarning: Module _markerlib was already imported from /Library/Python/2.7/site-packages/distribute-0.6.49-py2.7.egg/_markerlib/__init__.pyc, but /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python is being added to sys.path
  from pkg_resources import load_entry_point

Any ideas?

I don't know from this what you are doing. vex doesn't use _markerlib. Literally, the message is coming from the process launched by vex, not from vex. Your error message suggests this is coming from pkg_resources and has something to do with distribute.


it shows the error "bin may be padded larger" while using at esp32s3-korvo-2 (AIV-569)

I trained a simple network like

and it is slim and quantized, then i rebuild the model following your instructions step by step.
like:

and the csv is

it show errors like :

and my idf's version is esp-idf-v4.4.3

and when it launch at the board,  like:

Ths.
It's ok when i follow your suggestion 3.


fix edit on github button

Summary

Currently, on the Paketo site, there's an Edit on Github button at the top and bottom of each docs page. Only the bottom one on each page works.
(Try the buttons on https://paketo.io/docs/concepts/buildpacks/ for instance)
This is because the edit on github partial was relying on receiving certain manually-set variables from its caller. Instead, the partial has been refactored to pull the information it needs from the provided page context.
Use Cases

Now the partial can be safely used on any Markdown page without the caller needing to set up additional scratch variables. And site users don't run into a broken link at the top of each docs page.
Checklist

[x] I have viewed, signed, and submitted the Contributor License Agreement.
[ ] I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)
[ ] I have added an integration test, if necessary.
[ ] I have reviewed the styleguide for guidance on my code quality.
[x] I'm happy with the commit history on this PR (I have rebased/squashed as needed).

PR DEPLOYED TO: https://paketo-staging--fix-github-edit-button-dqtlaut1.web.app
SITE EXPIRES AT: 2021-07-17T22:21:44.855909504Z (in 72h)


Pulsar dashboard support getting authorization token from a file

Describe the solution you'd like
According to administration-dashboard.md, dashboard container get authorization token from JWT_TOKEN environment.
If the token was generated by Public/Private keys, it will be too long for JWT_TOKEN, It could be better to  support getting authorization token from a file for dashboard.

I found JWT_TOKEN is good enough, this issue will be closed.
For docker run command:
export JWT_TOKEN=`cat token-value.txt`

For kubernetes cluster:
name: JWT_TOKEN
valueFrom:
  secretKeyRef:
    name: token-name
    key: token-value.txt


Wrong format in docs in GET /players/{account_id}/ratings

The documentation here says that GET /players/{account_id}/ratings returns
[
  {
    "hero_id": "string",
    "rank": 0,
    "card": 0
  }
]

but it returns
[
  {
    "hero_id": "string",
    "score": 0,
    "percent_rank": 0,
    "card": 0
  }
]

I think we just have to update the spec.js file


Feedback 'I want to see my analytics logs, so I'm sending...'

Message from 56ec4863139b21562100026e sent on 2016-03-18 18:30:16 UTC###

I want to see my analytics logs, so I'm sending myself feedback to see them - orta

Link to HockeyApp

https://rink.hockeyapp.net/manage/apps/37020/feedback/465705

This should auto-close in hockey too, I think.

Noice! Do we need to do anything special to make feedback show up here?


pass an interface containing the callbacks to the stream

Will be need for future optimisations. Also fixes #1041.

Codecov Report

Merging #1048 into master will decrease coverage by 0.09%.
The diff coverage is 94.77%.

@@            Coverage Diff             @@
##           master    #1048      +/-   ##
==========================================
- Coverage    90.8%   90.71%   -0.09%
==========================================
  Files         106      106
  Lines        7225     7222       -3
==========================================
- Hits         6560     6551       -9
- Misses        430      435       +5
- Partials      235      236       +1

Impacted Files
Coverage Δ

packet_packer.go
90.35% <ø> (ø)
:arrow_up:

crypto_stream.go
100% <100%> (ø)
:arrow_up:

internal/flowcontrol/base_flow_controller.go
100% <100%> (ø)
:arrow_up:

stream_framer.go
96.88% <100%> (ø)
:arrow_up:

streams_map.go
92.22% <100%> (ø)
:arrow_up:

server_tls.go
73.83% <100%> (ø)
:arrow_up:

session.go
82.89% <64.71%> (-0.98%)
:arrow_down:

stream.go
88.24% <89.47%> (ø)
:arrow_up:

send_stream.go
95.68% <95.68%> (-0.1%)
:arrow_down:

receive_stream.go
97.22% <97.22%> (-0.04%)
:arrow_down:

... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 44cff87...fc8fafd. Read the comment docs.


No mention of trigger:

The trigger: node is documented elsewhere, but a page titled "Reference/YAML schema", which introduces a semi-formal grammar syntax, should be exhaustively complete.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7458d5b2-1bb7-7d87-7ce6-2b0c430b0155
Version Independent ID: edf5bbd3-a3f0-5426-e985-31290f8afd05
Content: YAML schema - Azure Pipelines
Content Source: docs/pipelines/yaml-schema.md
Product: devops
GitHub Login: @erickson-doug
Microsoft Alias: macoope

Thank you, good catch. Adding it now.


EmitFlags.NoLeadingComments is not propagated down to tokens

Background: I'm trying to emit an import declaration without its leading trivia (as part of OrganizeImports).  I can set the emit flag on the declaration and that's sufficient to prevent emitNodeWithComments from emitting the leading trivia.  However, emitNodeWithNestedComments emits the leading trivia anyway as part of emitTokenWithComment(SyntaxKind.ImportKeyword in emitImportDeclaration.
For context, here's the call stack:
emitImportDeclaration (src\compiler\emitter.ts:2136)
pipelineEmitUnspecified (src\compiler\emitter.ts:756)
pipelineEmitWithHint (src\compiler\emitter.ts:539)
pipelineEmitWithSourceMap (src\compiler\emitter.ts:529)
emitNodeWithNestedComments (src\compiler\comments.ts:191)
emitNodeWithSynthesizedComments (src\compiler\comments.ts:133)
emitNodeWithComments (src\compiler\comments.ts:95)
pipelineEmitWithComments (src\compiler\emitter.ts:517)
Writer.onEmitNode (src\services\textChanges.ts:743)
pipelineEmitWithNotification (src\compiler\emitter.ts:507)
print (src\compiler\emitter.ts:458)
writeNode (src\compiler\emitter.ts:404)
getNonformattedText (src\services\textChanges.ts:689)
getFormattedTextOfNode (src\services\textChanges.ts:665)
format (src\services\textChanges.ts:654)
(anonymous function) (src\services\textChanges.ts:656)
computeNewText (src\services\textChanges.ts:656)
(anonymous function) (src\services\textChanges.ts:643)
(anonymous function) (src\services\textChanges.ts:642)
getTextChangesFromChanges (src\services\textChanges.ts:633)

@weswigham can probably more clearly articulate what should be happening.

I'm going to hack around this problem to fix a group of OrganizeImport bugs, including #22731.  I'll tag the hack with the URL of this issue.

Hmmmmmmmmmmmmmmmmmm, thinking on it more, the correct behavior is probably to move emitTokenWithComment into the comment emitter, then have it track both the context node, and weather anything has been written to indicate that the token's leading/trailing comments are not the node's (and also respect the flag on the node if they are). @rbuckton do you have preferences here?

I was looking at reducing the stack depth in another PR, which has changes how we emit comments for tokens (though I also need to address some performance issues before that can go in). It might be easier to indicate whether a token is the first or last token and have the comment emitter use that to check for the related EmitFlag on the context node.


[Bug]: Go SDK: Parquetio does not work on Google Cloud Dataflow.

What happened?
Using Go SDK's parquetio.Write on Google Cloud Dataflow does not appear to work. Run the code example below to reproduce the error.
This code will work with the direct runner.
// Test code for writing a parquet file using the example structure in Beam's parquetio package.
// Based on the example presented at<EMAIL_ADDRESS>// go run cmd/parquet/parquet.go -dataflow_service_options enable_prime -async -runner dataflow -project $PROJECT -region us-west1 -staging_location gs://$BUCKET/staging/
// go run cmd/parquet/parquet.go -async -runner dataflow -project $PROJECT -region us-west1 -staging_location gs://$BUCKET/staging/
package main

import (
	"context"
	"flag"
	"fmt"
	"reflect"

	"github.com/apache/beam/sdks/v2/go/pkg/beam"
	"github.com/apache/beam/sdks/v2/go/pkg/beam/io/parquetio"
	"github.com/apache/beam/sdks/v2/go/pkg/beam/log"
	"github.com/apache/beam/sdks/v2/go/pkg/beam/x/beamx"

	_ "github.com/apache/beam/sdks/v2/go/pkg/beam/io/filesystem/gcs"
)

const (
	BUCKET = "REPLACE_WITH_YOUR_GCS_BUCKET"
)

type Student struct {
	Name    string  `parquet:"name=name, type=BYTE_ARRAY, convertedtype=UTF8, encoding=PLAIN_DICTIONARY"`
	Age     int32   `parquet:"name=age, type=INT32, encoding=PLAIN"`
	Id      int64   `parquet:"name=id, type=INT64"`
	Weight  float32 `parquet:"name=weight, type=FLOAT"`
	Sex     bool    `parquet:"name=sex, type=BOOLEAN"`
	Day     int32   `parquet:"name=day, type=INT32, convertedtype=DATE"`
	Ignored int32   //without parquet tag and won't write
}

func main() {
	flag.Parse()
	ctx := context.Background()
	beam.Init()

	p, s := beam.NewPipelineWithRoot()

	oneStudent := beam.CreateList(s, []Student{
		{
			Name:    "Apache Beam",
			Age:     5,
			Id:      9000,
			Weight:  350,
			Sex:     false,
			Day:     2,
			Ignored: 101,
		},
	})
	parquetio.Write(s, fmt.Sprintf("gs://%s/parquet/student.parquet", BUCKET), reflect.TypeOf(Student{}), oneStudent)

	if err := beamx.Run(ctx, p); err != nil {
		log.Exitf(ctx, "Failed to execute job: %v", err)
	}
}

2023/03/05 22:51:35 No environment config specified. Using default config: 'apache/beam_go_sdk:2.45.0'
2023/03/05 22:51:35 Failed to execute job:      generating model pipeline
failed to add scope tree: &{{parquetio.Write root/parquetio.Write} [{github.com/apache/beam/sdks/v2/go/pkg/beam.addFixedKeyFn 3: ParDo [In(Main): T <- {2: main.Student/R[main.Student] GLO}] -> [Out: KV<int,T> -> {3: KV<int,main.Student>/KV<int[varintz],R[main.Student]> GLO}]} {CoGBK 4: CoGBK [In(Main): KV<int,main.Student> <- {3: KV<int,main.Student>/KV<int[varintz],R[main.Student]> GLO}] -> [Out: CoGBK<int,main.Student> -> {4: CoGBK<int,main.Student>/CoGBK<int[varintz],R[main.Student]> GLO}]} {github.com/apache/beam/sdks/v2/go/pkg/beam/io/parquetio.parquetWriteFn 5: ParDo [In(Main): CoGBK<int,interface {}> <- {4: CoGBK<int,main.Student>/CoGBK<int[varintz],R[main.Student]> GLO}] -> []}] []}
        caused by:
failed to add input kind: {github.com/apache/beam/sdks/v2/go/pkg/beam/io/parquetio.parquetWriteFn 5: ParDo [In(Main): CoGBK<int,interface {}> <- {4: CoGBK<int,main.Student>/CoGBK<int[varintz],R[main.Student]> GLO}] -> []}
        caused by:
failed to serialize 5: ParDo [In(Main): CoGBK<int,interface {}> <- {4: CoGBK<int,main.Student>/CoGBK<int[varintz],R[main.Student]> GLO}] -> []
        caused by:
        encoding userfn 5: ParDo [In(Main): CoGBK<int,interface {}> <- {4: CoGBK<int,main.Student>/CoGBK<int[varintz],R[main.Student]> GLO}] -> []
bad input type
        caused by:
        encoding full type interface {}
bad type
        caused by:
unencodable type 'interface'
exit status 1

go version go1.20.1 linux/amd64
Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
Issue Components

[ ] Component: Python SDK
[ ] Component: Java SDK
[X] Component: Go SDK
[ ] Component: Typescript SDK
[ ] Component: IO connector
[ ] Component: Beam examples
[ ] Component: Beam playground
[ ] Component: Beam katas
[ ] Component: Website
[ ] Component: Spark Runner
[ ] Component: Flink Runner
[ ] Component: Samza Runner
[ ] Component: Twister2 Runner
[ ] Component: Hazelcast Jet Runner
[X] Component: Google Cloud Dataflow Runner

Nevermind I think a change since the latest release fixes this, https://github.com/apache/beam/pull/25203.


Error in starting demo server

><EMAIL_ADDRESS>start
> npm run minify && npm run render && npm run compile && npm run serve

><EMAIL_ADDRESS>minify
> ./scripts/minify.js

'.' is not recognized as an internal or external command,
operable program or batch file.

Might be wrong from my computer . can anyone suggest me where i am wrong?
Here is the Package.json :
{
  "name": "create-nue",
  "type": "module",
  "version": "0.1.3",
  "homepage": "https://github.com/nuejs/create-nue",
  "description": "A recommended way to start a Nue project",

  "repository": {
    "url": "https://github.com/nuejs/create-nue.git",
    "type": "git"
  },
  "bin": {
    "create-nue": "./scripts/init.js"
  },

  "scripts": {
    "start": "npm run minify && npm run render && npm run compile && npm run serve",
    "serve": "cd www && ../scripts/server.js",
    "compile": "./scripts/compile.js",
    "minify": "./scripts/minify.js",
    "render": "./scripts/render.js"
  },
  "dependencies": {
    "nuejs-core": "^0.1.0",
    "js-yaml": "^4.1.0"
  }
}

//resolved

how?

how?

Here is how you have to edit package.json "scripts" part:
"scripts": {
    "start": "npm run minify && npm run render && npm run compile && npm run serve",
    "serve": "cd www && node ../scripts/server.js",
    "compile": "node ./scripts/compile.js",
    "minify": "node ./scripts/minify.js",
    "render": "node ./scripts/render.js"
  }


OkHttp + HTTP2 + ExoPlayer (media3): only the first 16 MiB of audio data is audible

Summary
We build and maintain several apps that offer live and on-demand audio playback capabilities. For this, we heavily rely on OkHttp and ExoPlayer (now media3). It took us a while to narrow this down, but in a nutshell we've discovered that there's a combination of OkHttp, ExoPlayer and (some) hosts serving content over HTTP2 that causes:

OkHttp to gradually load audio data until it reaches 16 MiB, after which the request stops.
Within this 16 MiB of audio data, ExoPlayer will happily play audible audio. Once it gets beyond that 16 MiB, ExoPlayer continues playback, progress keeps progressing, but there will be no audible audio.

We're not 100% sure if it's OkHttp or ExoPlayer, but given our investigation (see further below) so far, we feel this is the best place to start looking into this issue in more detail.
Reproducer
A sample to reproduce this can be found here: https://github.com/mhelder/okhttp-media3-http2-16mb-issue
Follow these steps:

Build and deploy sample app
Hit the button labeled "❌ Player w/ OkHttp stack (HTTP2)"
Playback will start at high speed. It'll be gibberish, but the point is you can hear audio.
Wait for a bit for the progress to reach ~17:25 min. @128kbps, this is the 16 MiB mark. (Don't seek/scrub!)
Playback seemingly continues, but you'll hear no more audio. You can even pause and resume, after which the silence remains.

Also worth mentioning:

If you run everything through a proxy (e.g. Charles), at around 6:15 min of playback, 16 MiB of data will have been loaded. This is the point where the proxy reports the request as 'completed' normally. But the actual audio file size is ~31.1 MiB (33:57 min), so there's more data to load. The content-length header correctly reflects that. (At this point, nothing is reported (yet) when I attach an EventListener),
At around 15:25 min of playback, the buffer indicator on the progress bar jumps all the way till the end. This seems to be the point where the audio is suddenly considered to be 16 MiB in size, instead of the actual 31.1 MiB. With an EventListener attached, this is where responseBodyEnd -> connectionReleased -> callEnd are reported. No failures or anything else out of the ordinary seems to happen.
Seeking or scrubbing affects everything described above. In fact, it's how you - as a user - can 'fix' hearing no more audio.

Investigation
The sample app contains a few things we've investigated using the same audio file. Each of these options matches one of the buttons in the sample app, so you can also try yourself:

❌ Use ExoPlayer with OkHttp, and allow HTTP2 - the default. This doesn't work, as described above.
✅ Use ExoPlayer with OkHttp, but limit protocol to HTTP1.1. This works.
✅ Use ExoPlayer with default network stack (UrlConnection-based, uses HTTP1.1). This works.
✅ Use ExoPlayer with Cronet (Play services-based): This works.

On top of that, we also verified the following:

❌ Use OkHttp5: 5.0.0-alpha.14. Same problem.
❌ Use ExoPlayer from before it became Media3. Same problem.
❌ Use latest Media3 alpha: 1.4.0-alpha01. Same problem.
✅ Use a different audio file (with same bitrate) from a different host that also serves over HTTP2: link. Works! (Note: this file might become unavailable at some point)

And finally, we tried cutting out Media3 and directly use OkHttp to make a request to the audio file:
okHttpClient.newCall(
    Request.Builder().get().url(mediaUrl).build()
).enqueue(object: Callback {
    override fun onFailure(call: Call, e: IOException) { ... }
    override fun onResponse(call: Call, response: Response) {
        // do something
    }
})

The result depends on what we do in onResponse:

If we don't read from the response stream, we observe the same behaviour as with Media3: the request completes after ~47 seconds.
If we do read from the response stream, the whole audio file (all 31.1 MiB) gets downloaded.

Conclusion (so far)
Given above, what we've been able to establish so far is that it appears to be a combination of OkHttp and HTTP2 on some hosts: e.g. it fails for content from tritondigital.com, but not art19.com. Could this imply the host misbehaving, or perhaps timing out because no data is being read for too long? Would that warrant looking into Media3's OkHttpDataSource? But since it does work correctly with Cronet, hopefully there also a way to make it work with OkHttp, and still leverage HTTP2.
The workaround we use for now in our apps is to derive an OkHttp instance from our app-wide singleton and force it to HTTP1.1. But we're not really sure where to go from here. Any help, pointers and ideas would be appreciated!

Wow, thanks for the detailed repro. I'll take a look briefly on the weekend.
I'm familiar with exoplayer, so hopefully can poke around. The repro with only okhttp looks helpful.

OkHttp generally prefers you to consume the responses if you want more data to be read. If you don't read, everything should work but eventually stream flow control should mean the server stops sending fresh data.
But that doesn't explain why it doesn't resume sending after it's evetually consumed.

I'm exploring on this branch https://github.com/square/okhttp/pull/8419/files
It shows the logging we have for Http2 frames and also flow control. But probably nothing more until the weekend. And the logging may be confusing.

It looks like an issue with the server.
Testing on an OkHttp branch showed expected behaviour. OkHttp doesn't update the flow control window, as the client hasn't consumed the bytes.  Once the window is exhausted (16mb) the server should stop sending and wait for the client to request more.
12:44:59.296  D  >> CONNECTION 505249202a20485454502f322e300d0a0d0a534d0d0a0d0a
12:44:59.297  D  >> 0x00000000     6 SETTINGS
12:44:59.297  D  >> 0x00000000     4 WINDOW_UPDATE 16711681
...
12:45:01.741  D  << 0x00000005     9 DATA
12:45:01.741  D  << 0x00000005  4570 DATA
12:45:01.742  D  >> 0x00000000     4 WINDOW_UPDATE 8389495
12:45:46.441  D  << 0x00000005     0 DATA          END_STREAM
12:45:50.208  D  >> 0x00000005     4 WINDOW_UPDATE 8396084
12:45:50.255  D  << 0x00000005     4 RST_STREAM

But it looks like the server closed the stream cleanly as if the end of the data.

A branch with some extra logging. I should have forked but wanted to try to avoid creating lots of repos, it didn't work well.
https://github.com/yschimke/reproduce/tree/okhttp_logging

I'm going to close this, but happy to continue the discussion on the closed issue.
Thanks for the great repro, it was amazingly helpful.

Apologies for the delayed reply - many thanks for looking into this @yschimke!
Any theories on why we don't see the same happening with Cronet (which will also leverage HTTP2)? Is it related to the size of control flow window? Any other 'controls' that could affect why we're not seeing consistent behaviour with different clients using the same protocol?
I would love to dig a little deeper and figure out what's different, but wouldn't really know where to start. Is there tooling that would allow me to inspect communication at the protocol layer? Perhaps some sort of proxy?
I've also been trying to get Triton Digital involved, but - as I'd feared - it's proven difficult to get past the first line of support.

No, but you should be able to see via Wireshark or NetLog?
I'd speculate that Cronet's Request Callback means the app is driving the flow control more than OkHttp allows
https://developer.android.com/develop/connectivity/cronet/lifecycle
https://developer.android.com/develop/connectivity/cronet/reference/org/chromium/net/UrlRequest.Callback.html#onReadCompleted(org.chromium.net.UrlRequest, org.chromium.net.UrlResponseInfo, java.nio.ByteBuffer)
For tools you want something that can decrypt a TLS connection, and store a PCAP, or hook into Wireshark. Maybe something like https://docs.mitmproxy.org/stable/howto-install-system-trusted-ca-android/ ?


merge from arlac77/template-github-action,arlac77/template-kronos-component

.github/workflows/ci.yml

chore(deps): add 15.0.1 remove 15.0.0 (jobs.build.strategy.matrix.node-version)

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@e534e56). Click here to learn what that means.
The diff coverage is n/a.


The ALEXA UI doesn't synchronize audio when playing music.

IMPORTANT: Before you create an issue, please take a look at our Issue Reporting Guide.
Briefly summarize your issue:
The ALEXA UI doesn't synchronize audio when playing music.
What is the expected behavior?
The ALEXA UI synchronizes audio when playing music as soon as possible.
What behavior are you observing?
After stopping the first time music play, we waited few minutes for screen refresh.
Then asked Alexa to start music play once again.
The ALEXA UI didn't display the playcard until the audio had played about 10 seconds.
Does this behavior match your expectation?
The log is that we recorded after stopping music play.
sssdk240_playmusic_after.zip
Provide the steps to reproduce the issue, if applicable:

Run smart screen sdk v2.4 on RPi4.
Say ‘Alexa, play music’. Play music the first time.
Say ‘Alexa, stop’.
Wait 5 minutes.
Say ‘Alexa, play music’. Play music the second time.
Record the time interval between audio and playcard.

Tell us about your environment:

What version of the AVS Device SDK are you using?
  AVS Device SDK: <1.21.0>
  APL core library: <1.4.1>
  Alexa smart screen SDK: <2.4.0>

Tell us what hardware you're using:

[ ] Desktop / Laptop
[*] Raspberry Pi
[ ] Other - tell us more:

Tell us about your OS (Type & version):

[ ] Linux
[ ] MacOS
[*] Raspbian Stretch
[ ] Raspbian Jessy
[ ] Other - tell us more:

Have you tried the same use case with AVS Device SDK SampleApp?

[ ] The issue is not applicable to the AVS Device SDK SampleApp.
[*] The issue is not reproducible on the AVS Device SDK SampleApp.

Hi @eytu0233,
Alexa-smart-screen-sdk receives RenderPlayerInfo presentation directive in addition to Play directive from AVS asynchronously. Due to this, there may be a delay in the presentation processing and rendering, especially on platform with low processing power.
Thank you for your contribution to the Alexa Smart Screen SDK!

Hi @VaruPan,
We found the  RenderPlayerInfo and Play directive almost send to device simultaneously
2020-12-09 05:45:10.515 [  2] I DirectiveSequencer:onDirective:directive=namespace:AudioPlayer,name:Play,messageId:2d5964c5-1e6e-44a3-b7f0-6fcb772619cf,dialogRequestId:f6059e5b-484a-4b4d-9e4c-0e6b85389387,correlationToken:,eventCorrelationToken:,payloadVersion:,instance:
2020-12-09 05:45:10.508 [  2] I DirectiveSequencer:onDirective:directive=namespace:TemplateRuntime,name:RenderPlayerInfo,messageId:69834d84-fb60-4478-b650-84dcb45ddb00,dialogRequestId:f6059e5b-484a-4b4d-9e4c-0e6b85389387,correlationToken:,eventCorrelationToken:,payloadVersion:,instance:
But the UI still be too late. How could we know about presentation processing and rendering time delay?

Sure,

Hi @VaruPan,

Does the render time is about 17.728s on RPi4? It doesn't make sense why it take too long time
2020-12-09 05:45:10.511 [  c] 5 TemplateRuntime:handleRenderPlayerInfoDirective
2020-12-09 05:45:28.239 [  2] 3 GUILogBridge:GUILog:component=WSClient:message sent, type: renderComplete

2.How to use MetricsRecorder to record it?


README.md

title: Renaming an organization
intro: 'If your project or company has changed names, you can update the name of your organization to match.'
redirect_from:

/articles/what-happens-when-i-change-my-organization-s-name/
/articles/renaming-an-organization
versions:
free-pro-team: ''
enterprise-server: ''
github-ae: '*'

{% tip %}
Tip: Only organization owners can rename an organization. {% data reusables.organizations.new-org-permissions-more-info %}
{% endtip %}
What happens when I change my organization's name?
After changing your organization's name, your old organization name becomes available for someone else to claim. When you change your organization's name, most references to your repositories under the old organization name automatically change to the new name. However, some links to your profile won't automatically redirect.
Changes that occur automatically

{% data variables.product.prodname_dotcom %} automatically redirects references to your repositories.  Web links to your organization's existing repositories will continue to work. This can take a few minutes to complete after you initiate the change.
You can continue pushing your local repositories to the old remote tracking URL without updating it. However, we recommend you update all existing remote repository URLs after changing your organization name. Because your old organization name is available for use by anyone else after you change it, the new organization owner can create repositories that override the redirect entries to your repository. For more information, see "Changing a remote's URL."
Previous Git commits will also be correctly attributed to users within your organization.

Changes that aren't automatic
After changing your organization's name:

Links to your previous organization profile page, such as https://{% data variables.command_line.backticks %}/previousorgname, will return a 404 error. We recommend you update links to your organization from other sites{% if currentVersion == "free-pro-team@latest" %}, such as your LinkedIn or Twitter profiles{% endif %}.
API requests that use the old organization's name will return a 404 error. We recommend you update the old organization name in your API requests.
There are no automatic @mention redirects for teams that use the old organization's name.

Changing your organization's name
{% data reusables.profile.access_profile %}
{% data reusables.profile.access_org %}
{% data reusables.organizations.org_settings %}
4. Near the bottom of the settings page, under "Rename organization", click Rename Organization.

Further reading

"Changing a remote's URL"
"Why are my commits linked to the wrong user?"

Why:

What's being changed:

Check off the following:

[ ] I have reviewed my changes in staging. (look for the deploy-to-heroku link in your pull request, then click View deployment)
[ ] For content changes, I have reviewed the localization checklist
[ ] For content changes, I have reviewed the Content style guide for GitHub Docs.


Serializer/Deserializer implementation for FreeChunkMap

Description
Adds an implementation for the serializer/deserializer for FreeChunkMap. Pending:

invoking the serializer during checkpointing, and deserializer during restoration from checkpoint
testing

Contributor agreement

[x] I have read and agree to the Contributor Agreement.

This PR was not intended for the main repository, hence closed.


Question: xruns with 0ms delay

When I catch JACK xruns on certain clients via the API, what does it mean if the reported delay is 0ms?

Sorry, I don't know. The delay value comes directly from the JACK library. You should probably ask on the JACK mailing list: http://jackaudio.org/community.html.

Okay, will do and report potential results. :)

My question does not seem to find anybody knowing more about this. :/

I'm sorry your question (http://lists.jackaudio.org/pipermail/jack-devel-jackaudio.org/2019-May/001973.html) hasn't been answered (yet).
Maybe another option would be to try to ask on the "Linux Audio User" mailing list?

@HaHeho Any news on this?


Question about result

@hanoonaR @mmaaz60
Hello, Thank you for sharing excellent work.
I have confirmed the output of model.
Even though I input the same video, the results are different like below
('The person in the video is using a cellphone' or 'The person in the video is holding a cellphone')
Can you tell me how to control the result? I mean I wanna get the same result
Thank you in advance

Hi @jhj7905,
Thank you for your interest in our work. Video-ChatGPT uses Vicuna for generating text output. As Vicuna is a generative LLM, it is expected to have somewhat different output with similar meaning in different runs. I hope I answer your question. Thanks


Getting "Danger does not have write access to the PR to set a PR status." but can post to GitHub using a test script

Report
We are using Danger with Jenkins for PR on our Github enterprise instance.
Since some days - which I'm afraid coincides with our update to Github Enterprise 2.16.0 we get the the error message "Danger does not have write access to the PR to set a PR status.".
I created a sample ruby script using the same access token and am able to update the status of our PRs.
I have the feeling that setting the status sometimes work in the first run, but not in later cases.
What did you do?
Run bundle exec danger as last step in our Jenkins task.
We pinned to the most recent danger version in our Gemfile (5.12.0)
What did you expect to happen?
That danger is able to correctly update my PRs.
What happened instead?
I get this output in my Jenkins console:

bundle exec danger --verbose
All green. Yay.

Danger does not have write access to the PR to set a PR status.
Your Environment
Jenkins
Danger 5.12
swiftlint.lint_files

failure "PR title should match Angular JS conventions - see https://gist.github.com/stephenparish/9941e89d80e2bc58a153 - e.g. 'feat: GG-1234 add kitchen sink'" if github.pr_title.match(/^(feat|fix|chore|impr)(\([a-zA-Z]+\))?:( [a-zA-Z][a-zA-Z0-9 ,'\-\._!\u2019]*[^\.])$/) == nil

failure "Please add \"ready to merge\" label." if github.pr_labels.detect{|l| l.include? "ready to merge"} == nil

failure "Please remove \"blocked\" label." if github.pr_labels.detect{|l| l.include? "blocked"} != nil

@bibendi where is that page? I'm facing a similar issue but couldn't find that specific page of your screenshot 😅

Same error, I granted permission for statuses: write and it worked.
ref: https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token

I don't think this is a bug in Danger, and I'd vote for closing this issue.
Perhaps, the documentation could be improved to mention that when Danger is running in Github actions, it needs the appropriate permissions.
permissions:
  pull-requests: write # required for Danger to post comments
  issues: write # required for Danger to post comments
  # potentially other permissions, as needed


Unclear about custom environment

It is unclear to me from the docs what the custom environment is actually doing. Is this a necessary process for running a production site, or is it ok to run a production site with the environment set to local?
Should we update the docs at https://islandora.github.io/documentation/installation/docker-custom/ to explain what it means to create a custom Drupal container vs bind mounting the codebase folder, why it is a good idea, etc?

The repository probably needs a revisit in design at some point to align with actual use cases. The original intention is to have a way to package a Drupal site into a image that could then be deployed elsewhere. Though as you've found no one used it and overtime things changed. By and large I think a production site should be managed under its own repository. Probably have some layout like (this isn't a working example just a vague outline).
├── Dockerfile
├── drupal
│   ├── composer.json
│   ├── composer.lock
│   ├── config
│   │   ├── splits
│   │   └── sync
│   └── web
│       ├── modules
│       ├── profiles
│       └── themes
├── README.md
├── docker-compose.yml
└── rootfs

Where in the Dockerfile extends the islandora/drupal:${tag} image adding any additional packages etc needed, as well as copies the drupal folder into the image to /var/www/drupal/ and performs a composer install.
The modules, profiles, themes are for any custom Drupal code to the production site that do not merit their own repositories (i.e. no one in the wider community would use them).
The config folder is for keeping the drupal production site configuration under source control. Allowing one to bring up a site locally change configuration and export it, building a new image that could then be deployed to production.
The docker-compose.yml file could be many or just a single file derived from the services one wants to include, by say looking at isle-dc. Production may use docker compose for running the site, it's up to the use case.
Truth is at the end of the day we've not got a turn-key suits all solution; ready for production system. It's more like an a la-cart menu of things that can be used to customise and build something to fit ones needs. As the need for customisation is a big driver for many.

I'm very much in favor of this reorg. In our internal repositories (which diverged from isle-dc a long time ago) we have essentially done this.

My initial confusion should be cleared up by updates to documentation for setting up a production server - https://github.com/Islandora/documentation/pull/2225
Nigel has also created a non-isle version of the structure he outlined above - https://github.com/Islandora-Devops/isle-site-template


Exploration of preact bindings, logger and functions as actions

This is just a draft to see if there's any interest in any of the following features:
Default logger
{ log: true} option that turns console logging for all actions, looks like this

Preact bindings
{ Provider, connect } for Preact, means you can do this:
const App = (atom, split) => (

 split('inc')}>
    {atom.count}

)
export default connect()(App)

  <Provider atom={atom}>
    <App />
  </Provider>

Functions as actions
Functions as actions, this would mean you can organise your actions in modules and import them where needed. This one's a bit more controversial.
const inc = () => split((get, split) => split({ count: get().count + 1})
export default connect()((atom, split) =>
atom.count
)

For me here are the pros/cons:
Pros

you can keep your actions local to the component

Cons

weird ceremonial syntax :-/ there are like 3 splits in that line
3 ways of doing stuff now, dispatching updates, dispatching actions and dispatching fns
you lose the "action stream" concept, like tiny-atom follows this idea that there's a stream of actions which are plain objects and evolve consumes them and does stuff, here we break that...

On that last one, for me it's appealing the idea of keeping actions close to the component and plugging them in somehow. But not sure the proposed way here is the best. The only other way is to sort of "register" functions by importing them in your central actions.js file like so
module.exports = Object.assign(
  {},
  require('./my-subapp/actions'),
  require('report/actions')
)

But that's fairly weird and you might have clashes. So then you like create constants file, so then you basically have redux, hm. So the nice thing about using functions as actions that it like doesn't matter, but then you lose this event stream idea basically unless you pretend the fn name is the action name.... but then you don't see the payloads, so yeah, not nice. Hm.

I've never really been a fan of function parameter overloading. I would've prefered explicit functions for mutating state and dispatching actions, so not a fan of adding a third way of using split

const inc = () => split((get, split) => split({ count: get().count + 1})
export default connect()((atom, split) =>
atom.count
)

I'm not really following this, where is that first split coming from?

Yeah, should have been:
const inc = (get, split) => split({ count: get().count + 1})
export default connect()((atom, split) =>
 split(inc)}>atom.count
)

But basically, not sure this is a good idea.. I think if you wanna go that route - you should just use redux-zero.
I think tiny-atom should remain closer to redux in the sense that it operates on a stream of actions and reduces them via it's singular evolve function. So no splitting of functions, only splitting of patches/actions is possible. I think that's fine.
The preact bindings are really nice idea, I've seen your preact-tiny-atom, but I'm basically thinking about just rolling it into this lib to make it more useful out of the box. Thinking of this now:
const mapStateToProps = state => ({
  count: state.count,
  doubleCount: state.count * 2,
})

const mapSplitToProps = split => ({
  increment: x => split('increment', x),
  decrement: x => split('decrement', x)
})

const App = ({ count, doubleCount, increment, decrement }) => (

{count}

{doubleCount}

    <button onClick={increment}>+</button>
    <button onClick={decrement}>-</button>

)

export default connect(mapStateToProps, mapSplitToProps)(App)

And also thinking about introducing options.debug that gets called on both each action being split and on each state update, it should make a) debuggers/loggers easier to build (see #8) and b) make it clearer that it's just for debugging and c) make render signature cleaner.
Also - because tiny-atom is using the concept of actions just like redux, redux dev tools just work via remotedev integration, including time travel:

I want to include that out of the box as well via tiny-atom/devtool or smth.

Like, redux is just .. bad api, it's just too confusing for everyone.. and not great defaults and DX..
Redux-zero is very cool, it just loses this mental model of actions. The benefit is that you bring actions closer to your components, which might be nice.
And tiny-atom, is closer to redux in that action stream sense, but it means actions are closer to the global store, actions are global. But I think that's ok. What you can do is import actions from your individual modules (just like combining redux reducers from multiple locations) or you can also create custom connector that registers actions for the component into tiny atom via context, it's doable as an extra, if that's your thing, while keeping the rest of tiny atom the same. Just need to write some docs for that..

Will close this to keep for the record. Will reopen a next branch.


Training Hangs

When it hangs
I'm trying to train an instance segmenting model using Mask2Former with a Swin-B (IN21k) backbone.
/home/ubuntu/.local/lib/python3.8/site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument.  It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  max_size = (max_size + (stride - 1)) // stride * stride
/home/ubuntu/Mask2Former/mask2former/modeling/transformer_decoder/position_encoding.py:41: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  dim_t = self.temperature ** (2 * (dim_t // 2) / self.num_pos_feats)

(full logs)
There are no new [03/31 00:28:13 d2.utils.events]:  eta: ... logs that come after, and I waited about an hour to see. I've tried multiple times.
One time it hanged after iteration 839, another time it hanged on iteration 59. In the above logs it hanged after iter 219.
GPU Utilization
Before it hangs, the 8 GPUs are utilized between 25% and 90% and there is a python3 process associated with each GPU.
When it hangs, 7 GPUs are utilized 100% and one GPU is utilized 0%. There is a python3 process associated with only 7/8 of the GPUs.

**Click to expand GPU info**
GPU Utilization Before it Hangs
GPU                 Memory                   GPU-Util
Tesla V100-SXM2...  13198MiB / 16384MiB |     69%
Tesla V100-SXM2...  13226MiB / 16384MiB |     74%
Tesla V100-SXM2...  13128MiB / 16384MiB |     90%
Tesla V100-SXM2...  13134MiB / 16384MiB |     58%
Tesla V100-SXM2...  13206MiB / 16384MiB |     44%
Tesla V100-SXM2...  13222MiB / 16384MiB |     27%
Tesla V100-SXM2...  13170MiB / 16384MiB |     25%
Tesla V100-SXM2...  13142MiB / 16384MiB |     52%

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     18684      C   /usr/bin/python3                13195MiB |
|    1   N/A  N/A     18685      C   /usr/bin/python3                13223MiB |
|    2   N/A  N/A     18686      C   /usr/bin/python3                13127MiB |
|    3   N/A  N/A     18687      C   /usr/bin/python3                13131MiB |
|    4   N/A  N/A     18688      C   /usr/bin/python3                13203MiB |
|    5   N/A  N/A     18689      C   /usr/bin/python3                13219MiB |
|    6   N/A  N/A     18690      C   /usr/bin/python3                13167MiB |
|    7   N/A  N/A     18691      C   /usr/bin/python3                13139MiB |
+-----------------------------------------------------------------------------+

GPU Utilization When it Hangs
GPU                 Memory                   GPU-Util
Tesla V100-SXM2...  13198MiB / 16384MiB |    100%
Tesla V100-SXM2...  13226MiB / 16384MiB |    100%
Tesla V100-SXM2...  13130MiB / 16384MiB |    100%
Tesla V100-SXM2...  13134MiB / 16384MiB |    100%
Tesla V100-SXM2...  13206MiB / 16384MiB |    100%
Tesla V100-SXM2...  13222MiB / 16384MiB |    100%
Tesla V100-SXM2...  339MiB / 16384MiB |      0%
Tesla V100-SXM2...  13142MiB / 16384MiB |    100%

INTERESTING TO NOTE: GPU 6 process is no longer is shown!
(dmesg doesn't show anything interesting regarding why this process ended, so no OOM I assume)
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     18684      C   /usr/bin/python3                13195MiB |
|    1   N/A  N/A     18685      C   /usr/bin/python3                13223MiB |
|    2   N/A  N/A     18686      C   /usr/bin/python3                13127MiB |
|    3   N/A  N/A     18687      C   /usr/bin/python3                13131MiB |
|    4   N/A  N/A     18688      C   /usr/bin/python3                13203MiB |
|    5   N/A  N/A     18689      C   /usr/bin/python3                13219MiB |
|    7   N/A  N/A     18691      C   /usr/bin/python3                13139MiB |
+-----------------------------------------------------------------------------+

Environment

Based on "Deep Learning AMI GPU CUDA 11.3.1 (Ubuntu 20.04) 20220303"
torch==1.10 cu113
detectron2 for torch1.10 cu113

Config
My config is based on the provided COCO maskformer2_swin_base_IN21k_384_bs16_50ep.yaml with the following changes:

SOLVER.STEPS removed from config
MODEL.SEM_SEG_HEAD.IGNORE_VALUE changed to 323
MODEL.SEM_SEG_HEAD.NUM_CLASSES changed to 323
INPUT.DATASET_MAPPER_NAME changed to "mask_former_instance"
TEST.EVAL_PERIOD changed to 1000
INPUT.CROP was added:

  CROP:
    ENABLED: True
    TYPE: "absolute"
    SIZE: (640, 640)
    SINGLE_CATEGORY_MAX_AREA: 1.0

Run Command
DETECTRON2_DATASETS=/home/ubuntu/detectron_datasets python3 train_net.py --num-gpus 8 \
  --config-file configs/food/instance-segmentation/swin/maskformer2_swin_base_IN21k_384_bs16_160k.yaml

Dataset
https://www.myfoodrepo.org/
Things I've Tried
None of the following changes have fixed the error:

Change DATALOADER.NUM_WORKERS from 4 to 0
Change SOLVER.BASE_LR from 0.0001 to 0.00001
Change SOLVER.AMP.ENABLED from True to False
Use 8 A100 GPUs
Use 8 V100 GPUs

What's the problem? Any idea how to fix?
Any idea what's going wrong here?
It appears that detectron2 already defaults to logging.DEBUG. Is there any way I can force detectron2 / Mask2Former to produce more logging details?
Is it getting stuck in a training iteration? SOLVER.MAX_ITER and SOLVER.STEPS are relatively high numbers. Although, I don't understand why this would result in termination of one of the GPU processes. However I don't understand pytorch internals well.
What further debug steps would you recommend?

User with similar issue (getting stuck, one GPU at 0%, other GPUs at 100%)
https://discuss.pytorch.org/t/nn-dataparallel-gets-stuck/125427/12
Neither solution applies as IOMMU is disabled on my instances and I'm not using a container.

https://github.com/pytorch/pytorch/issues/22259#issuecomment-1086911791


0010-MARG-margin_orchestration.md

We feel we need test coverage for the full range of margin settings which can be included in market proposal
Could we also also have examples party margin calcs for a range of expected used settings.
there are numerous feature tests but not all have ACs
integration tests tend to use the same market setup
| ETH/DEC19 | BTC        | BTC   | 1                | default-log-normal-risk-model | default-margin-calculator | default-none | default-eth-for-future | default-none     |

is there scope for adding other calculators for different margin setups apart from default-margin-calculator?

So do you mean like the margin is correctly calculated with lognormal risk model with parameters x, y, z and also for log normal risk model with parameters X, Y, Z ?

not sure what you meant by that but we do need to test for alternative margin / risk settings


Add mattermost-push-proxy as an optional service in docker-compose.yml

It would be nice if we could have the push notifications service as part of the services setup and managed by docker. At best we would only have to provide it with the location of the public/private keys from Apple as well as the Android API from Google.

Thanks @tomzx for the suggestion. If you are interested, A PR is very welcome.

Closing this for now due to no activity. Feel free to reopen it for more discussion.


Refactor connectorId as ModelId for sync methods.

Description

This PR refactors the connectorId used in the SyncConnector type to use ModelId over string.
Risk
Pretty low risk, as this code does not change the activities prototypes.

Deploy Plan

:fire:


Add change activity for Gardens

On using the "Advance Bastion Turn", I'm wondering if we want to change the message from "No orders issued to special facilities this turn." to "No special facilities have completed their orders this turn." if ongoing orders are in effect. We could even show progress for ongoing orders, but that might be a future UI improvement.

It's important to note if no orders were issued, because that might be because the characters were absent, in which case a bastion event is triggered. But I agree we could still note that progress was made on ongoing orders so that the message doesn't get confused for 'nothing was done'.


Handle payload differently in background or app-closed state

Following up on #1596
When a notification is received the code currently does not distinguish between a background state or closed-state. The notification payload is sent to the app and then either delivered or cached for delivery on next init.
Adding an extra check for this means the payload gets discarded when the app is not running. We can either add a force-cache or no-cache flag to the payload and handle it accordingly. Depending on preference of the default action, which one should we choose?

@laagland is this a comment on #1596 or a new issue? If it is a new issue please fill out the issue template.

@macdonst this is a new issue, discussed in the comments of #1596. @fredgalvao advised to split the issues mentioned there. The issue template is basically the same. But to recollect:
Expected Behaviour
According to the payload flowchart and the text beneath it, when the plugin receives the notification with content-available set to 1 and the app is not running (force-closed), no further processing should occur.
Actual Behaviour
In the above situation, the payload is cached and sent to the app on initialisation.
While this could be intended behavior, with the current code there is no method to prevent the caching if you want to.

@macdonst what should I do to move this issue forward. I've committed some changes to my fork that I think will handle this use-case.

@laagland well, I think you've done the best thing to move this forward by send a pull request. That is the best way to get anything done in an open source project.

This will be part of 1.10.0


multiple tags as control code

Does the following mean one training record for each tag of the multiple tags? Say, if my average number of multiple tags is 10, the data size for fine-tuning will become 10x of the original? Is this understanding correct? If correct, I plan to give it a try & thank you for your advice.

The way it's trained, the current checkpoints don't support that. However, there is nothing preventing one from fine-tuning (or re-training) CTRL to do that. I'm fairly sure that the model will learn to pick it up.

Originally posted by @keskarnitish in https://github.com/salesforce/ctrl/issues/33#issuecomment-536637383

It's up to you really; it depends on what you want to do at the end.
If it is a hierarchy (like, [Books, Author, Title]), then you don't need to replicate the data.
If it is a label for the data but the data has multiple labels (like, Wikipedia Stoicism is ... and Philosophy Stoicism is..), then you probably would benefit from the replication.
Closing for now, reopen as needed.


Zarr-Python endorses SPEC0

Hi all. 👋🏻
After multiple discussions with Zarr-Python Core-Devs and yesterday's (2/6) SPEC meeting, I'm submitting this PR on behalf of Zarr-Python to endorse SPEC0.
@joshmoore, @jhamman and @d-v-b - please approve or react with 👍🏻.
Thanks!

Thanks, all!


Mayhem

Description of change

Integrated Mayhem a Fuzzing tool.
Links to any relevant issues

Type of change

Bug fix (a non-breaking change which fixes an issue)
Enhancement (a non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Documentation Fix

How the change has been tested

Change checklist

[ ] I have followed the contribution guidelines for this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have checked that new and existing unit tests pass locally with my changes
[ ] I have updated the CHANGELOG.md, if my changes are significant enough

LGTM! And builds are passing now!


No Buttons on local Server

Hi Guys,
I got my local Server running and ready for the field. It has an internet connection and I can login.
But for some reasons - I don’t understand - there are none of those buttons that should be on the web-app.
So no add plants button, none of the calibration buttons/field, none on the farmware page and none in the settings.
How can I get this buttons there? I am using edge as a browser which works fine when I am using the official farmbot server.
I just saw, it also says invalid date on my local server. Maybe this is connected to the missing buttons issue?
Also Version says “none” which should be different I guess
First posted in the forum, but I guess it should be here. Nevertheless, I'll crosslink the threads for others to find the solution faster:
https://forum.farmbot.org/t/no-buttons-on-local-server/3669/2

@derletztename

there are none of those buttons that should be on the web-app

What version of FBOS is your device running?
Is the device online?
Do you see any errors in the javascript console? (ctrl + shift + j)

I am using edge as a browser

We don't test against Edge. It's always best to try against Chrome (preferred) or Firefox (seems to work OK).

says invalid date on my local server

You are completely off grid, correct? We are aware of a bug in off-grid systems due to an inability to contact an NTP server. We are working on a fix for a future version of FBOS.

Version says “none”

This should not create an issues. The version is just there for convinience.

Thanks for answering fast RickCarlino,
I have no idea what FBOS is and how I find out which version I am using. I use Ubuntu Server 18.04 with just the things added your manual wanted me to do.
Googling hadn't helped how to find out what FBOS I am using. Could you maybe tell me how to find out?
The device is still on my desk, till it runs smoothly ( I have to rebuild the plants via the web app as I am not able to perform the coding wizardry you told me to do ;)
then it will move to the field, but still keep an internet connection via a mobile LTE hotspot.
My personal experience was that firefox did not work very good. I had issues drag and dropping plants with firefox, but none with edge.

@derletztename FBOS = "FarmBot OS". Pardon my acronym usage. 6.4.3 is the latest version.
Do you know which version you are running? Have you had any indication that the device was able to connect yet?

I googled for quite some time to understand what FBOS might be :D
I have not yet connected the local server to the farmbot. Farmbot is still connected to your public server, so I can use it for watering and stuff. I don't wanted to make the move before the server is running properly.
I think, I should be able to drag and drop the plants onto the field without the farmbot is connected. Also the buttons should be visible, shouldn't they?

@derletztename It sounds like a browser rendering issue then. Please install Chrome and try again (it would help if we could see a screenshot).

Yeah, this is crazy. So when I use your server, edge is working perfectly fine. But on my local server with edge there are not button, but with chrome they are there...
I am using chrome for farmbot work from now on... just to be sure ;)

Thanks for the support

Glad you got it worked out, @derletztename !
Chrome is, unfortunately, our only supported browser at the moment so it is best to avoid Edge where possible.


chore: linarith only needs ring1

I appreciate that this is "going backwards" in the sense of requiring more explicit imports of tactics (and making it more likely that users writing new files will need to import tactics rather than finding them already available).
However it's useful for me while I'm intensively working on refactoring and upstreaming tactics if I can minimise the dependencies between tactics. I'm very much aware that in the long run we don't want users to have to manage imports of tactics, and I am definitely going to tidy this all up again once I'm finished with this project!

bors d+

bors merge


Remove some @title assignments from controllers

The @title is used both in <html><head><title> and in the page content.
This PR removes assignments of @title from some controller actions into corresponding view templates, where it is already assigned in most cases. It seems like it should be assigned in views everywhere, but this PR does not take that step.
It's still assigned in controllers:

for all indexes, by ApplicationController#show_index_of_objects. args sent to show_index_of_objects may send a no_hits_title, or the view may supply one. Ideally I'd like to move this assignment into the index views, but i'm not sure how to do it without creating a lot of redundancy.
for the Edit Project form reload, by ProjectsController#update
for all mailers

coverage: 93.687% (-0.02%) from 93.706% when pulling 4f9092b28032f2abc50f7b2b9365635f2b5d7809 on nimmo-rubric into e76ec814b66e53bbf51c8dfbbea0004456d36c60 on main.


Fix boxplot redraw issues

The boxplot mark was previously not responding to various state changes (new data, domain changing, etc), as not all of the selections were rebinding the data to the DOM elements.
Fixes issue in #710, and some issues noted in #698.

Just noted a needed tweak in the outliers, will follow up with another commit soon.


Convert to composite action?

Might allow for better caching... to investigate.

Would require building clang-format from source. See https://github.com/Unibeautify/docker-beautifiers/blob/master/clang-format/Dockerfile

Would require building clang-format from source. See https://github.com/Unibeautify/docker-beautifiers/blob/master/clang-format/Dockerfile

[ ] set up composite action
[ ] refactor script to pull Docker image and run from there
[ ] modify Dockerfile to use clang-format as CMD
[ ] refactor testing to call script on directory
[ ] modify README to indicate new version support
[ ] new release


Fix Broker Controller discovery w/ LocalLogManager

Verified by running
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

LGTM


fixed ES2+ compatibility in transport address regex.

elasticsearch 2+ http API (nodes/_all/clear) does not reply with inet[/ip:port] but plain ip:port.
the regex now matches "ip:port" and "inet[/ip:port]"

thanks


UC|debug doesnt work with overcast

Disabling debug as the code within debug function will not work with overcast.
Fix wrong flavor reference
unset env_http[s]_proxy before running make_userdata.sh
Added validation stuffs for undercloud to .overcast.yaml
it seems overcast has a bug because of which the variable is not getting
extracted properly on remote execution commands, because of which verify_hosts,
trigger_update etc are failing using overcast. Added an issue for that, for now
using python jiocloud directly from deploy.sh.
save cleanup files in ~/ so they can be used by delete job.
get_ip_of_node once rather than in a loop - ip will be assigned before that
get exeecuted

please test this

please test this

Test FAILed

please test this

Test FAILed

please test this

please test this

looks good as soon as it passes.

Test FAILed

Test PASSed


Introducing ros2_shm_msgs repo to support pointcloud and image zero copy transport.

Hello,
I am here to introduce a repo ros2_shm_msgs for zero copy with pointcloud and image, and just finished a point cloud transport demo.
Issues are welcomed!
Thanks.

@ZhenshengLee thanks for the heads up! Are you looking to eventually merge that into our project, or just raising awareness of your project with the additional message definitions? Either way works for me.
cc @MatthiasKillat

@ZhenshengLee thanks for the heads up! Are you looking to eventually merge that into our project, or just raising awareness of your project with the additional message definitions? Either way works for me.
cc @MatthiasKillat

I am happy to merge this into your project when all demos are finished.
Thanks.

@ltbj @MatthiasKillat
Hi, I am glad to tell you that the main object of https://github.com/ZhenshengLee/ros2_shm_msgs has been achieved, and you can use this lib for a more easy way to use zero copy transport with image(opencv) and pointcloud(pcl and open3d).
Feel free to check the readme https://github.com/ZhenshengLee/ros2_shm_msgs/blob/master/README.md


Error in makeSummary: invalid json response body at https://www.google.com/sorry/index?continue=https://www.youtube.com/watch%3FprettyPrint%3Dfalse%26v%3D5A8UUovaSb8%26pbj%3D1&q=EgQDgW_cGKyZ9bEGIjDIbez6kcN4Ofhrrbcfy2T6brx1CtlOvEP6CvOF0rtGbElS0CP3ISqeUAkwFsUF2MgyAXJaAUM reason: Unexpected token '<', "<!DOCTYPE "... is not valid JSON

Describe the bug
Error in makeSummary: invalid json response body at https://www.google.com/sorry/index?continue=https://www.youtube.com/watch%3FprettyPrint%3Dfalse%26v%3D5A8UUovaSb8%26pbj%3D1&q=EgQDgW_cGKyZ9bEGIjDIbez6kcN4Ofhrrbcfy2T6brx1CtlOvEP6CvOF0rtGbElS0CP3ISqeUAkwFsUF2MgyAXJaAUM reason: Unexpected token '<', "<!DOCTYPE "... is not valid JSON
Until a few days ago, it was working fine, but since 4-5 days ago, this error has been coming up. If you remove youtubei, that error does not appear. However, because I need a transcript, I have to use youtubei, so I am reporting this issue.
my function
const youtube = new Client();
async function makeSummary(req,res){
const { videoId, lang, ask, email } = req.body;
.....
const transcript = await youtube.getVideoTranscript(videoId);
console.log(transcript,'transcript')
}
result
Error in makeSummary: invalid json response body at https://www.google.com/sorry/index?continue=https://www.youtube.com/watch%3FprettyPrint%3Dfalse%26v%3D5A8UUovaSb8%26pbj%3D1&q=EgQDgW_cGKyZ9bEGIjDIbez6kcN4Ofhrrbcfy2T6brx1CtlOvEP6CvOF0rtGbElS0CP3ISqeUAkwFsUF2MgyAXJaAUM reason: Unexpected token '<', "<!DOCTYPE "... is not valid JSON

This works fine on my end
const client = new Client();
const transcript = await client.getVideoTranscript("5A8UUovaSb8");
console.log(transcript);

It seems like your error is caused by YouTube detecting your IP as malicious (can be caused by sending too many requests with the same IP, using a VPN, or other stuff). Unfortunately, there's nothing I can do (of which I'm aware) to work around this.

I'm getting this on all requests now too. I wonder if Google recently deployed something for this.

I'm getting this on all requests now too. I wonder if Google recently deployed something for this.

> await client.getVideoTranscript("5A8UUovaSb8");
Uncaught:
[FetchError: invalid json response body at https://www.youtube.com/api/timedtext?v=5A8UUovaSb8&ei=sIFDZq6xFOy52roPnd2MkAc&caps=asr&opi=112496729&exp=xbt&xoaf=5&hl=en-GB&ip=<IP_ADDRESS>&ipbits=0&expire=1715725344&sparams=ip%2Cipbits%2Cexpire%2Cv%2Cei%2Ccaps%2Copi%2Cexp%2Cxoaf&signature=CA50B3C8AC657AF47C19E376E92F4A3223DFD927.604DB09FF45AD34E2CE5F96AA9BAAEB2385D5828&key=yt8&kind=asr&lang=en&prettyPrint=false&fmt=json3 reason: Unexpected token < in JSON at position 0] {
  type: 'invalid-json'
}

"I think I almost fixed it by adding const { OAuth2Client } = require('google-auth-library');. It seems to work now, but I am still testing, so I can't say it's completely fixed yet."

I'm going to close this since there's not much I can do to solve this issue, however, you can try using OAuth: https://github.com/SuspiciousLookingOwl/youtubei/issues/113#issuecomment-2439351189


VSHA-267 Changes to RTS values.yaml to make RTS work on vShasta

Summary and Scope
Restore chart settings from the distant past that make it possible to enable Google mode for RTS to allow it to work on Virtual Shasta. Specifically, this adds a volume mount to the cray-hms-rts container that places the GCP service account credentials in the right place for RTS to use them in Google mode. The volume is defined as optional, so, if the underlying secret is not present it will be mounted as an empty directory.
This is a backward compatible change.
Issues and Related PRs

Relates to VSHA-267

Testing
Tested on:

Virtual Shasta

Test description:
Deployed chart with appropriate overrides in customizations.yaml and verified that the resulting RTS came up using the Google helpers and discovered nodes. Deployed the same chart without the customizations.yaml overrides and verified that RTS came up as it has in the past on vShasta.

Were the install/upgrade-based validation checks/tests run (goss tests/install-validation doc)? y
Were continuous integration tests run? If not, why? n
Was upgrade tested? If not, why? N/A
Was downgrade tested? If not, why? N/A

Risks and Mitigations
There are no known risks associated with this change.
Pull Request Checklist

[x] Version number(s) incremented, if applicable
[x] Copyrights updated
[x] License file intact
[x] Target branch correct
[x] CHANGELOG.md updated
[x] Testing is appropriate and complete, if applicable

These changes should be made to the v3.0 chart instead of the v2.0 chart

You should also update cray-hms-rts.compatibility.yaml


Does it give line&column info for the parsed tokens?

I am pretty sure it doe snot, as a search in the project for the word "line" only finds stuff in the tests/ directory.

Would love to see this as well :-) (hi again @hoijui, seeing you're going through the same options :-P https://github.com/servo/html5ever/issues/492)


Adds Github Actions as CI

Pulls Substratee Docker Image
Install Rust Toolchain
Prints Information about the environment
Builds the project in hardware mode

Looks neat! For now this does a build only - do we want to run the tests too, using this Action? Or do you want to have that in a separate Action?

Looks neat! For now this does a build only - do we want to run the tests too, using this Action? Or do you want to have that in a separate Action?

Yes, I will implement Tests shortly


Creating a .select() store for each item in an array

Perceived Problem
I want to dynamically iterate over each item in an array (within an ImmerStore) and create a store for that array element (via .select())
Ideas / Proposed Solution(s)
I was able to achieve this manually like so, but it feels like a hack...
export let store: ImmerStore<any[]>;

const childStores = derived(store, (current) => {
  return current.map((_value, index) => {
    return store.select((current) => current[index]);
  });
});

Now each childStores member can be manipulated as expected 🎉
I'm not sure what kind of API would be possible to support my use-case, but it would be awesome if ImmerStore<any[]> was iterable, where each member was the same as having called .select with the index, so that something like this would "just work"
<script>
  export let store: ImmerStore<string[]>;

  function addString() {
    store.update(current => [...current, '']);
  }
</script>

<button on:click={addString}>Add another one</button>

{#each store as stringStore}
  <input type="text" bind:value={$stringStore} />
{/each}

To iterate over the store contents, you'll need to get the store contents. (We can't know the length of the contents without interrogation).
To update a specific store value, you'll need a store (via select) for that specific item.
Combining this, I first imagined something like this:
https://svelte.dev/repl/d3fff15708a7453590363498ef80ffb8?version=3.47.0
But you'll quickly see the error - svelte still has some limitations with respect to stores :-(
To workaround this, you'll need to create a component for each instance such as:
https://svelte.dev/repl/026c5e6e35f0481eb7757d2c2de905c4?version=3.47.0
I'm hoping they remove this limitation, but for now this should be able to get you over the hump.
If you find a better workaround do let me know.


Tutorial interface can't restore after changing size of page

Describe the bug
Tutorial interface can't restore after changing size of page
Steps to reproduce the behavior

Navigate to https://arcade.makecode.com/beta#
Open any tutorial
Change size of tutorial page

Actual behavior
Tutorial interface can't restore after changing size of page
Expect behavior
Tutorial interface can restore after changing size of page
Screenshot

Additional context
1.OS: Windows11
2.arcade version: 1.13.27
3.Microsoft MakeCode version: 9.1.3

Verified this bug was fixed in the latest version, so close this bug as fixed:
arcade version: 1.13.53
MakeCode version: 9.3.9
Screen Recording:


Fix reader for large index files.

Currently the offsets are cast into uint32 even though the index can
grow larger than 4GiB.
Test is currently OOMing, will fix.

This change is

Removed test as it OOMs. @fabxc take a look?

Is there a workaround for users currently affected by this bug? Removing or splitting files maybe?

👍

LGTM

This doesn't seem to make it into 2.2.0-rc.0, is there a chance for it to make it for final?

👍

@fabxc Will this make it into 2.2.0 as it didn't make the cut to rc1?

Mh, what makes you think this didn't make it into rc.1? Is the underlying issue not resolved.
It should be in:

https://github.com/prometheus/prometheus/blob/v2.2.0-rc.1/vendor/vendor.json#L805
https://github.com/prometheus/tsdb/commit/494acd307058387ced7646f9996b0f7372eaa558

@fabxc Ah, stupid me, I was looking at the release/2.2 branch not the tag :/

After trying Prometheus 2.2.0-rc.1 which has this fix, I can confirm that large index files are handled correctly so far (1 full day of testing). I have multiple 9-15GB index files and no errors. Thank you for making this fix available.


Welcome to join opencompass

Great works.
Welcome to contribute you benchmark into OpenCompass.

Thank you for your acknowledgment. We will consider contributing our work to OpenCompass in the future.

Great works.
Welcome to contribute you benchmark into OpenCompass.

We would like to integrate UHGEval into OpenCompass. Could you provide a contact information for further discussion on the details?