proximity_thresh parameter problem

I found that parameter proximity_thresh belongs to the reid part in the YOLO8 project, and in the YOLO8 project, get_ Dist only uses iou_ Distance and fuse_ Score, you can get a good tracking effect, but in this CPP project, I found that the proximity_thresh parameter appeared in IOU_ In distance and not using iou in yolo8_ Dists_ Steps for masks and related masks

The proximity_threshold parameter in this project represents the Intersection over Union (IoU) distance between two bounding boxes, as outlined in the configuration file.
When two bounding boxes have an IoU value within the range defined by proximity_threshold, they are chosen for association. Following this, their scores are fused as showcased in the implementation here.
It's worth noting that the naming conventions utilized in this project and the YOLOv8 implementation of BotSORT may differ, which could potentially lead to confusion.
I hope this clears up the functionality of the proximity_threshold parameter

Closing due to inactivity.


Make RuleSetProvider easier

This is a sample RuleSetProvider of Detekt:
class CustomRuleSetProvider : RuleSetProvider {
  override val ruleSetId: String = "detekt-custom-rules"

  override fun instance(config: Config)
      = RuleSet(ruleSetId, listOf(NoInternalImportRule(config)))
}

I don't like it since the ruleSetId is being used twice. ktlint's RuleSetProvider looks like this:
class CustomRuleSetProvider : RuleSetProvider {
  override fun get() = RuleSet("custom-ktlint-rules", NoInternalImportRule())
}

Which is much better. Can we also squeeze the ruleSetId together?

We could change the list to a vararg BaseRule and remove the explicit RuleSet(ruleSetId... by using a SingleAssign-Delegate and set the ID in buildRuleset function.
The ruleSetId as a property has the benefit of lazily creating a rule set only if it is active in the config.

Linking to #2680

Closing this issue in favor of #3816


test_sirs.py problem when i test my own data ,after precessing to save the processed image got this wrong

File ~/Downloads/DSRNet-main/models/dsrnet_model_sirs.py:143 in test
Image.fromarray(tensor2im(torch.clip((output_rr + 1) / 2, 0., 1.)).astype(np.uint8)).save(
TypeError: clip() received an invalid combination of arguments - got (numpy.ndarray, float, float), but expected one of:

(Tensor input, Number min, Number max, *, Tensor out)
(Tensor input, Tensor min, Tensor max, *, Tensor out)

Thanks for reporting it, and it's fixed now.


TypeError: this.require is not a function

write demo according to the readme , get exception:
TypeError: this.require is not a function
    at VueComponent.mounted (vue-monaco.common.js?70e3:75)
    at callHook (vue.esm.js?efeb:2921)
    at Object.insert (vue.esm.js?efeb:4158)
    at invokeInsertHook (vue.esm.js?efeb:5960)
    at Vue.patch [as __patch__] (vue.esm.js?efeb:6179)
    at Vue._update (vue.esm.js?efeb:2660)
    at Vue.updateComponent (vue.esm.js?efeb:2788)
    at Watcher.get (vue.esm.js?efeb:3142)
    at new Watcher (vue.esm.js?efeb:3131)
    at mountComponent (vue.esm.js?efeb:2795)

demo code:

App.vue

<template>

    <router-view/>

</template>

2.helloworld.vue
<template>
  <monaco-editor
    class="editor"
    v-model="code"
    language="javascript">
  </monaco-editor>
</template>

@egoist same here. What went wrong?

same here

Same here.
It is probably related that we use webpack to bundle our application.
To my understanding webpack is not able to resolve the AMD require statement if we dynamically assign window.require to a property
    require: {
      type: Function,
      default: window.require
    }

@MaChristmann nice, do you have an example package.json, how to integrate your version in a project? If I use it in a project (vue-cli), like this:
"vue-monaco": "MaChristmann/vue-monaco#da7b977",
"monaco-editor": "^0.13.1",
I get the following error:
This dependency was not found:

* vue-monaco

To install it, you can run: npm install --save vue-monaco                                                                                No type errors found
Version: typescript 2.9.2

Try the latest version, btw require prop is renamed to amdRequire.


support setting timezone in docker

It would be easier to set local time in CRON_SCHEDULE.
Probably need to add RUN apk update && apk add tzdata to Dockerfile and then set TZ env variable.

I'll add the package but it's up to the user to set TZ. There's no one-size-fits-all for time zone.


Scrollable equations

Would it be possible to make equations that are two long "scrollable" without hosting my own css?
One big advantage of mdbook, in comparison to pdf files, is that I can scale the page to the size of my screen. Unfortunately by default mdbook-katex does not do this. The fix is rather simple and even described in the CSS customization section of KaTeX's documentation. All that has to be done is to make a single modification to katex.css
.katex-display { overflow: auto hidden }

I wanted to ask if adding it by default or as an option to mdbook-katex would be worth considering. It's a bit of a hassle, downloading the css and font just for this...

I imagine that the easiest way is to have an additional-css file with
.katex-display { overflow: auto hidden !important; }

Could you please try this, and if it doesn't work, research a bit on how to use CSS?

Cool, it works.
I have no real experience with css or html but I could have figured, that you can simply add css like this.
Thank you.


Fixed generation of complex data types in Strawberry Shake

Hello,
sending this PR in context of this Michael's post
I got CSC : error SS0006: An error occurred during generation: Specified argument was out of the range of valid values.
EDIT:
fix: value types, references and complex data types

All references built with the TypeReferenceBuilder now reference with
the full namespace
Fxi nullability issues of value types in complex data types
Fix generation of nested complex data types

Thanks for your help

/AzurePipelines run


proposal: Go 2: type assertion extensions, defined type assertions and generics

Proposal: type assertion extensions, defined type assertions and generics
Introduction
Currently, type assertions can be used on interface values to determine the dynamic type of a value:
var val interface{}
//...
x, ok := val.(int)                  // an int type assertion: x is of type int
s, ok := val.(strings.Stringer)	    // a satisfaction test: s if op type strings.Stringer and not nil if ok

Type switches borrow the type assertion syntax:
switch x := val.(type) {
case string:            // x is a string
case bool:              // x is a bool
case float32, float64:  // x is an interface{} value containing an float32 or float64
default:
	// x is an interface{} value containing any type
}

This document contains a set of 6 proposals to extend on this idea.
Proposal 1 - Allow multiple types in type assertions
Extend the type assertion syntax to allow multiple types, just as in the switch case statements:
x, ok := val.(float32,float64)	// if ok, x has dynamic type float32 or float64, otherwise it is nil

This works just like in a type switch case: if val holds an int or float64 ok will be true and x will have a the dynamic type of float32 or float64.
Proposal 2. Allow defined type assertions
A defined type assertion is declared using the new keyword meta:
meta Float {float32,float64}
meta Int {int,int8,int16,int32,int64}

meta {} // all types

Because of the keyword meta a defined type assertion is also called a meta type. A defined type assertion or meta type can be used anywhere where an "inline" type assertion can be used.
var val interface{} = 1
x, ok := val.(Int)	// ok = true
x, ok = val.(Float) 	// ok = false

swich x := val.(type) {
case Float:
	// x has dynamic type of any type in Float
case Int:
    // x contains and int (of any type in Int)
	// x is int
case int:
    // x is an int
}

Like types, meta types are exported if starting with a capital character. Meta types can include other meta types:
package types

meta SignedInt {int,int8,int16,int32,int64}             // all signed integer types
meta UnsignedInt {uint,uint8,uint16,uint32,uint64}      // all unsigned integer types
meta Int {SignedInt,UnsignedInt}                        // all integer types
meta Float {float32,float64}                            // all floating point types
meta Complex {complex64,complex64}                      // all complex types
meta Real {Int,Float,Complex,rune,byte}                 // all real number types
meta Number {Real,Complex}                              // all number types

meta Meta{int,bool,Meta}                                 // error: meta type self inclusion
meta Meta{int,bool,Meta}                                 // error: meta type self inclusion

Proposal 3. Extended type assertions to include type kinds
The reflect packages uses the concept of type kinds to identify types as kind of types such as functions, channels, etc. Type assertions are extended to support the same idea.
f, ok := val.(func) // true if f is a function, f has as dymanic type some function
c, ok := val.(chan) // true if c is a channel, c has as dymanic type some channel type

The full list is illustrated using a type switching statement:
switch x := val.(type) {
    case func:
    case chan:
    case struct:
    case map:
    case []:		// slice types
    case [3]:		// array types of length 3
    case [.]:		// array types of any length
    case *:	        // pointer types
}

// type kinds can be used in defined type assertions as well
meta F {func}       // a meta type containing the set of all functions
meta S {struct}     // all structs
meta PS {[]*struct} // all types that are a slice of pointer to struct

// like types, meta types can be recursively defined
meta R {Float,*R} // a float, a pointer to a float, a pointer to a pointer to a float, etc

Meta types using type kinds may help to write more concise and clearer documentation and can be used to validate function arguments of type interface{}:
package json

// M is the set of all types that are marshallable to JSON.
// NOTE: compare this with the description in the JSON package of marshallable types
meta M {
	*M,
	bool,
	int,
	float64
	string,
	[]M,
	struct,
	Marshaller,
}

func ParseJSON(r io.Reader, dest interface{}) error {
    if _, ok := dest.(*M); !ok {
        return fmt.Errorf("dest is not a pointer or the value pointed to is not marshallable to JSON", dest)
    }
    ... // use type switching (and package reflect for structs) for rest of implementation
}

More intricate examples of using type kinds are given after generics have been discussed.
Proposal 4. Extend type assertions to include comparison operators
if _, ok := val.(<); !ok {
    fmt.Println("val is not comparable")
}

meta Equatable {==}
// Find is a silly function to showcase the use of comparison operator type assertions
func Find(needle interface{}, haystack ...interface{}) (int, error) {
	if _, ok := needle.(Equatable); !ok {
		return 0, ...
	}
}

More intricate examples are given in the generic section.
Proposal 5. Generics can be based on meta types
A generic function is declared by using a meta type in its signature.
meta C {<} // C is the set of all comparable types
meta E {==}

// a generic function: it uses the meta type C in the function signature
func Min(a, b C) C {
	if a < b {
		return a
	}
	return b
}

// Find is also generic because it uses meta type E
func Find(needle E, haystack []E) int {
    for i, value := range haystack {
        if value == needle {
            return i
        }
    return len(haystack)
}

A generic type is defined by using a meta type in its definition.
meta T {}

// a generic struct: it uses the meta type T in its field definitions
type ListNode struct {
	prev *ListNode
	next *ListNode
	Payload T
}

type MyGenericChannel chan T

meta K (==)
meta V {}
type Pair struct {
    Key K
    Value V
}

type HashTable struct {
    Table []Pair(K,V)
    HashFn func(value V) int
}

A generic type may include a meta type signature directly after its name.
// same definition of the generic Pair type but with explicit meta type signature (K,V)
type Pair(K,V) struct {
    Key K
    Value V
}

If the meta type signature is omitted it defaults to the order of appearance of the used meta types.
// Min is a generic function with a meta type signature (T) consisting of a single meta type T
func Min(T)(a T, b T) T {...}	        // (T) is the explicit meta signature.

// these declarations all define the same generic function type
type MinFn(T) func(a, b T)              // meta signature (T) is inferred here automatically
type MinFn func(a, b T)

func DotProduct(a, b Vector(T)) T       // meta signature (T) is inferred here automatically
func DotProduct(T)(a, b Vector(T)) T    // explicit meta sig (T)

func FanIn(chans []chan T) chan T  {...}

// example with multi meta types
func Keys(m map[K]V) []K {...}          // meta signature (K,V) is inferred here automatically
func Keys(K,V)(m map[K]V) []K {...}     // same thing but with an explicit meta type signature.

// struct examples
type LiseNode(T) struct {               // explicit meta signature
    prev *ListNode
    	next *ListNode
    	Payload T
}

type Bimap(K,V) struct {		// meta type signature (K,V) is required here - see below
	forward map[K]V
	reverse map[K]V
}

In general, the meta type signature will be omitted whenever possible because it is shorter and therefore more readable. A common case where the type signature is required is when a meta type is used only in non-exported struct fields, but the type self is exported. This is case for the Bimap(K,V) example above.
Some less common cases where meta type signature is required are:

when a meta type is used only by the function's implementation and is not present in the function's signature
for backwards compability; for example the order of struct fields that uses meta types have changed

A common reason to include the meta signature although not required would be clarity, for example because a different meta type order is more natural.
If a meta signature is provided, all meta types must be listed. Partial lists are not allowed.
Generic struct types cannot be specialized for specific types such as for example in C++.
It is not necessary to repeat the meta type signature on generic type methods, but it may be done so for reason of clarity.
func (b *Bimap) GetValueByKey(key K) V {
    return b.foward[key]
}

func (b *Bimap(K,V)) GetKeyByValue(value Value) V { // ok - but not needed
    return b.reverse[value]
}

meta T {K}

func (b *Bimap(T,V)) GetKeyByValue(value Value) V { // error - illegal type signature (T,V) (expected (K,V)
    return b.reverse[value]
}

It is recommended that a package that exports generics types use one or two type meta types at most. They should be name with single capital letter like T, K, V etc.  This would make than easily recognisable.
An exception would be a types packages for example that defines Ints, Floats, etc. These meta types could be used for both generics, type assertions and type switching.
When using a generic type or or function, each meta type identifier must be bound to a concrete type. This process is called type binding.
In many case, type binding can be done automatically by the compiler using the context in which an generic type is used. In cases where this is not possible, or where the inferred binding is not desired, the binding must be given explicitly.
package stats
meta T {int,float32,float64}

func Sum(a, b T) T {
	return a + b
}

stats.fsum(1, 1)                    // ok - T is inferred to type int
stats.fsum(1.0, 1.0)                // ok - T is inferred to type float64
stats.fsum(float32)(1.0, 1.0)       // ok - T is explicitly set to type float64

node := ListNode{Payload: "Gophers!"}                   // ok - T is inferred to type string
var node ListNode(string)                               // ok - T is explicitly set
type MyNode ListNode(string)                            // ok - MyNode has type struct { ..., Payload string }

When instantiating a generic type, the types that are (automatically or implicitly) bound to the meta types are compile-time type asserted using the meta type by the compiler.
stats.Sum("go", "c++")              // compile error: type string is not in stats.T
stats.Sum(byte)                     // compile error: type byte is not in stats.T

meta S {io.Reader}
fund MyReader(val S) {...}

stringReader := MyReader(string)        // compile error: string does not satisfy io.Reader
fileReader := MyReader(file)            // ok - fileReader has type func(*os.File)

A meta type can only bind to a single concrete type.
stats.Sum(1, 1.0)                   // compile error: attempt to bind stats.T to both int and float64.

fsum := stats.Sum                   // compile error: no type is provided for stats.T
fsum := stats.Sum(float64)          // ok - fsum is a function with signature (a, b float64) float64

When implementing generic functions, a compilation error occurs when an operation is attempted that is not supported by all types in the meta type.
meta T {}
meta C {<}

func Max(a, b T) T {
    if a > b {		// compile error: operator < is not supported by all types in T
        return b
    }
    return a
}

func Min(a, b C) C {
    if a < b {		// ok - all types in C support < operator (as per its definition)
        return b
    }
    return a
}

func Max(a, b C) C {
    if a > b {		// also fine: the compiler knows that all types that support <, also support >
        return b
    }
    return a
}

func MyFunc(val C) {
        val.String()  // compile error: method String is not supported by all types in C
}

meta S {fmt.Stringer}  // S is the set of all types that satisfy interface fmt.Stringer
func Concat(values []S) string {
    var result s
    for _, v := range values {
        s += v.String()         // ok
    }
    return s
}

type interface One {
    A()
    B()
}

type interface Two {
    B()
    C()
}

meta Q {One,Two}

func contrived(q Q) {
    q.B()               // ok: all types in Q have B
    q.A()               // compile error: A() is not supported by all types in Q
}

Besides the common applications of generics, in Go generics can also be used to provide a type safe signature to functions that are otherwise implemented using reflection.
package sql
meta R {struct}

// ScanRow uses generics only to provide a type safe function signature
func ScanRow(row *sql.Row, dest *R) error {
	return scanRow(row, dest)	// scanRow is the non-generic implementation, separated out to prevent unnecessary code duplication.
}

var person Person
sql.ScanRow(r, person)	// compile error: argument person of type Person is not of meta type *sql.R
sql.ScanRow(r, &person)	// ok

var count int
sql.ScanRow(r, &count)	// compile error: argument count of type *int is not of meta type *sql.R

// package json
// M is the set of all types that are marshallable to JSON.
meta M {
	*M,
	bool,
	int,
	float64
	string,
	[]M,
	struct,
	Marshaller,
}

func Unmarshall(r io.Reader, dest *M) error { // type safe interface
    return parse(r, m) // function parse is not generic and is implemented using reflection
}

json.Unmarshall(..., person)    // compile error -  person (type Person) is not of meta type *json.Marshallable
json.Unmarshall(..., &person)   // ok

c := make(chan int)
json.Unmarshall(..., &c)       // compile error - c (type chan int) is not of meta type *json.Marshallable c

Proposal 6. compile blocks in generic functions compile different code depending on type
This is best explained using an example.
package linalg

// Field is an interface to types that support arithmetic using +,-,* and /.
// NOTE: big.Int/Rat/Float could implement Field
type Field interface {
	// in the field operations below, f is the right operand
	AddField(f Field) Field
	SubField(f Field) Field
	MultField(f Field) Field
	DivField(f Field) Field
}

meta N {types.Ints,types.Float,types.Complex}
meta T {N,Field}

type Vector {
	Elems []T
}

func DotProduct(a, b Vector(T)) T {
    var result T
    for i := range len(a.Elems) {

        // compile different code depending on type
        compile T.(type) {
        case Field:
            // code in this block is included only for types that satisfy Field

            prod := a.Elems[i].MultField(b.Elems[i])
            result = result.AddField(prod).(T)

        case N:
            // code is included for types in N
            result += a.elems[i] * b.elems[i]
        }
    }
    return result
}

func (* Matrix) Mult(other Matrix(T)) Matrix(T) {
    // compile blocks are against used to provide (optimized) implementation
}

Another example.
// package goclapack
import la "math.linalg"

type N {float64,float32,complex64,complex128}

func SVD(m la.Matrix(N)) (la.Matrix(N), la.Matrix(N), la.Matrix(N), error) {
	s := m la.NewMatrix(N)(...)
    v := m la.NewMatrix(N)(...)
    d := m la.NewMatrix(N)(...)

    var err error

    // compile blocks used to call correct routine directly on the elements in matrix m, s,v and d.
    compile N.(type) {
    case float32:
    	C.fgesvd('A', 'A', c.size_t(m.rows), c.size_t(m.cols), unsafe.Pointer(&m.elem[0]))
        // detect err
        err = ...
    case float64:
        C.dgesvd('A', 'A', c.size_t(m.rows), c.size_t(m.cols), unsafe.Pointer(&m.elem[0]))

    // etc
    }

    return s, v, d, err

}

Proposal 3, as currently stated, is inherently not backwards compatible. For example:
Playground
package main

type CustomInt int

func Example(v interface{}) {
	switch v.(type) {
	case int:
		println("built-in")
	case CustomInt:
		println("custom")
	}
}

func main() {
	Example(CustomInt(0))
}

Currently, this will print custom, but it sounds like your change, as stated, would make it print built-in since it would now treat built-in types as kinds in type switches.

In the new generics draft, interfaces can contain a list of valid underlying types.
type Int interface {
  type int, int8, int116, int32, int64, uint, uint8, uint16, uint32, uint64
}

However, these can only be used as type parameters. A simpler proposal would be to allow using these in type assertions and type switches.
x := v.(Int)

switch x := v.(type) {
    case Int:
}

Proposal 3, as currently stated, is inherently not backwards compatible. For example:
Playground
package main

type CustomInt int

func Example(v interface{}) {
	switch v.(type) {
	case int:
		println("built-in")
	case CustomInt:
		println("custom")
	}
}

func main() {
	Example(CustomInt(0))
}

Currently, this will print custom, but it sounds like your change, as stated, would make it print built-in since it would now treat built-in types as kinds in type switches.

@DeedleFake This doesn't seem to be the case, I think you may have misinterpreted. int is still a built-in type, not a kind. It seems that the only thing that Proposal 3 states is that things like someVal.(func) (which is not currently valid) can be used in type-switches to describe any function.

If that was true, then the example M type wouldn't, as it claims, be all of the possible marshallable types:

// M is the set of all types that are marshallable to JSON.
// NOTE: compare this with the description in the JSON package of marshallable types
meta M {
	*M,
	bool,
	int,
	float64
	string,
	[]M,
	struct,
	Marshaller,
}

If int isn't a kind, then if I do type A int, M won't match A. Although, this example also doesn't cover int8, int16, and so on, or any of the uint types.


llvm redefinition of type for 2 dictmergers with different BinOps

The following code produces:
REPL: Compile error: Compile error: module:926:1: error: redefinition of type
%d0.bld = type %d0
^

|x:vec[i64]|
let r = for(x, {dictmerger[i64, i64, *], dictmerger[i64, i64, +]}, |b, i, n|
	{merge(b.$0, {n, i}), merge(b.$1, {n, 1L})}
);
{result(r.$0), result(r.$1)}

But works if I use the same merging operation in both builders.

This has been fixed in the llvm-st branch: no fix planned for master.


implement checksum error detection.

A checksum is added in the RealCommunicationChannel class using CRC32. This method was chosen because it helps detect errors in the data in a quick and reliable way. When a command is sent, the checksum is calculated and added to the packet. When the response is received, the checksum is checked again to make sure the data hasn't been changed or corrupted. This method works well to catch mistakes that might happen while the data is being sent.

Implemented checksum by creating a ChecksumHandler class. We use SHA-256 algorithm for calculating the checksum.


Design Meeting Notes, 6/2/2023

Import Assertions / Import Attributes
https://github.com/microsoft/TypeScript/issues/53656
https://github.com/microsoft/TypeScript/pull/54242
import Resource from "./some-json-file.json" with { type: "json" };

import Resource from "./some-json-file.json" assert { type: "json" };

At last TC39, progressed conditionally - some paperwork to be done.
Babel and at least one other implementer has it.
Node merged (shipped?) this already.
Probably safe - but we want concrete stage 3 to make sure.
Feels like we need confidence for at least syntax - should be able to use the with syntax instead of assert as soon as it hits stage 3.
But new semantics are tougher - this is a feature that is largely implementation defined. Unless we have a strong vision or the community coalesces on semantics, it is hard for us to figure out.
Will wait on true stage 3.

Plugins!
https://github.com/microsoft/TypeScript/issues/16607

3 things we've been talking a lot about

Watch plugins
Module resolution
Transformer (emit)

Also, module resolution has some asymmetry with the language service - generating a .d.ts on the fly for a GraphQL schema/query would mean you have to save a file on disk.

The most common thing is for CSS.
Ties into issues around the System interface expecting strings instead of binary buffers.

How do you map back an error span back to a binary?

Why did we start exploring transformer plugins?

Wanted people to stop patching TypeScript.
Patchers said they'd keep maintaining ts-patch.
Arguable that transformer plugins just fragment things more. esbuild, swc, Babel, Bun, Deno - none of them could leverage these transformers.

Feels like one of the concerns is type-driven emit, right?

Maybe, but also having parity mismatch is its own issue between all of these compilers.

People do use our output, but the marketplace for TypeScript compilers is broad.

Really what you'd want to do is describe how the code will be changed before it hits the checker.

e.g. transforms over a tagged template string to give better types
Mixed feelings about supporting that.
Lose invariants - need to know which kinds of nodes occur in the tree.

Could do a walk of the tree.
Really prefer having a source of truth on disk if possible (i.e. a .d.ts file or something)

"Running" things in the type system is slow and hits recursion limiters. Eventually might need to think about a way to enable these scenarios.

We probably would be best-served by splitting up the umbrella "compiler plugins" issue into 6 new issues so we can discuss them with some more clarity.

Watch
Transform
Resolution
Post-Execution (e.g. linting)
Post-parse
Mid-Check (?)
...?

"Running" things in the type system is slow and hits recursion limiters. Eventually might need to think about a way to enable these scenarios.

Could you expand on this one? I wonder about more details about what's hiding behind this point

At least once a month we have this conversation:

Our type system is Turing-complete and people are writing bananas types to compute wild stuff, and it's very slow
Well we should give them a way to just "write" code in the type system
That's only going to make the problem worse; every optimization we put in TS moves up the complexity of things people are writing instead of moving up the overall performance
Sure, but now they can write more complex stuff
We started this conversation because we were trying to make things faster, not more complex


New tools and slim down Docker image

fyi @Altonhe. please let me know if you have any concerns about these changes.

fyi @Altonhe. please let me know if you have any concerns about these changes.

Looks good to me


Doc for distribution of analyzer config in nuget

Summary
This commit adds documentation for the distribution of .globalconfig files with nugets.

Internal previews

📄 File
🔗 Preview link

docs/fundamentals/code-analysis/configuration-files.md
Configuration files for code analysis rules

@dotnet-policy-service agree

The information comes from this comment https://github.com/dotnet/roslyn/issues/47707#issuecomment-1075412726 made by @jmarolf on Roslyn.

I fixed the hardcoded locale, but I don't know how to make a relative link to the documentation of nuget. Alternatively I could remove the link.

@gewarren
It looks like you reviewed this with one suggestion. Should we accept the suggestion and merge?


multiple reload of page cause intervention to be disruptive on favicon.ico

I was testing my app in Golang with ModSecurity 3.0.10.  I was refreshing the page multiple time to test something and my server start giving me a 500 internal error returned but my inspection from my modsecurity implementation. After some test I found in my modsecurity_debug.log the error that procude the intervention.  After 20+ reload, the crash occur on the following rule. I was only reloading the login page of the application. Nothing to block there
The questions are:

Even after a reload of my executable of the app. The blocking still occurs and always related to that favicon.ico. Why and where are the memory of that blocking are stored.
Why this rule trigger only after multiple reload and is kept blocked after a restart of the application.

[169308372743.594574] [/favicon.ico] [9]  T (0) t:urlDecodeUni: "GA1.1.849814932.1680637192"
[169308372743.594574] [/favicon.ico] [9] Target value: "GA1.1.849814932.1680637192" (Variable: REQUEST_COOKIES:_ga)
[169308372743.594574] [/favicon.ico] [9]  T (0) t:urlDecodeUni: "GS1.1.16917726<IP_ADDRESS>17735<IP_ADDRESS>"
[169308372743.594574] [/favicon.ico] [9] Target value: "GS1.1.16917726<IP_ADDRESS>17735<IP_ADDRESS>" (Variable: REQUEST_COOKIES:_ga_CZJH9BPMEF)
[169308372743.594574] [/favicon.ico] [9]  T (0) t:urlDecodeUni: "MTY5MzA4MjkwOXxOd3dBTkVJeVRWZFhWRkZKU1VaRVVWTlhTRlZTTmxOVVZGZEJSRWRNUXpWVFNVbzBU (90 characters omitted)"
[169308372743.594574] [/favicon.ico] [9] Target value: "MTY5MzA4MjkwOXxOd3dBTkVJeVRWZFhWRkZKU1VaRVVWTlhTRlZTTmxOVVZGZEJSRWRNUXpWVFNVbzBU (117 characters omitted)" (Variable: REQUEST_COOKIES:session)
[169308372743.594574] [/favicon.ico] [7] Added regex subexpression TX.0: -QSnDdXzHKPXy-i-
[169308372743.594574] [/favicon.ico] [7] Added regex subexpression TX.1: -QSnDdXzHKPXy-i-
[169308372743.594574] [/favicon.ico] [9] Matched vars updated.
[169308372743.594574] [/favicon.ico] [4] Running [independent] (non-disruptive) action: setvar
[169308372743.594574] [/favicon.ico] [8] Saving variable: TX:anomaly_score_pl4 with value: 0
[169308372743.594574] [/favicon.ico] [4] Running [independent] (non-disruptive) action: setvar
[169308372743.594574] [/favicon.ico] [8] Saving variable: TX:sql_injection_score with value: 0
[169308372743.594574] [/favicon.ico] [9]  T (0) t:urlDecodeUni: "_ga"
[169308372743.594574] [/favicon.ico] [9] Target value: "_ga" (Variable: REQUEST_COOKIES_NAMES:_ga)
[169308372743.594574] [/favicon.ico] [9]  T (0) t:urlDecodeUni: "_ga_CZJH9BPMEF"
[169308372743.594574] [/favicon.ico] [9] Target value: "_ga_CZJH9BPMEF" (Variable: REQUEST_COOKIES_NAMES:_ga_CZJH9BPMEF)
[169308372743.594574] [/favicon.ico] [9]  T (0) t:urlDecodeUni: "session"
[169308372743.594574] [/favicon.ico] [9] Target value: "session" (Variable: REQUEST_COOKIES_NAMES:session)
[169308372743.594574] [/favicon.ico] [4] Rule returned 1.
[169308372743.594574] [/favicon.ico] [9] Running action: log
[169308372743.594574] [/favicon.ico] [9] Saving transaction to logs
[169308372743.594574] [/favicon.ico] [9] Running action: auditlog
[169308372743.594574] [/favicon.ico] [9] Saving transaction to logs
[169308372743.594574] [/favicon.ico] [9] Running action: status
[169308372743.594574] [/favicon.ico] [4] Running (non-disruptive) action: tag
[169308372743.594574] [/favicon.ico] [9] Rule tag: application-multi
[169308372743.594574] [/favicon.ico] [4] Running (non-disruptive) action: tag
[169308372743.594574] [/favicon.ico] [9] Rule tag: language-multi
[169308372743.594574] [/favicon.ico] [4] Running (non-disruptive) action: tag
[169308372743.594574] [/favicon.ico] [9] Rule tag: platform-multi
[169308372743.594574] [/favicon.ico] [4] Running (non-disruptive) action: tag
[169308372743.594574] [/favicon.ico] [9] Rule tag: attack-sqli
[169308372743.594574] [/favicon.ico] [4] Running (non-disruptive) action: tag
[169308372743.594574] [/favicon.ico] [9] Rule tag: OWASP_CRS
[169308372743.594574] [/favicon.ico] [4] Running (non-disruptive) action: tag
[169308372743.594574] [/favicon.ico] [9] Rule tag: capec/1000/152/248/66
[169308372743.594574] [/favicon.ico] [4] Running (non-disruptive) action: tag
[169308372743.594574] [/favicon.ico] [9] Rule tag: PCI/6.5.2
[169308372743.594574] [/favicon.ico] [4] Running (non-disruptive) action: tag
[169308372743.594574] [/favicon.ico] [9] Rule tag: paranoia-level/4
[169308372743.594574] [/favicon.ico] [9] This rule severity is: 4 current transaction is: 255
[169308372743.594574] [/favicon.ico] [9] Saving msg: Restricted SQL Character Anomaly Detection (cookies): # of special characters exceeded (3)
[169308372743.594574] [/favicon.ico] [4] Running (disruptive)     action: block.
[169308372743.594574] [/favicon.ico] [8] Marking request as disruptive.
[169308372743.594574] [/favicon.ico] [8] Running action deny
[169308372743.594574] [/favicon.ico] [8] Skipping this phase as this request was already intercepted.
Expected behavior
A way to clear the memory within application or at least on every restart
Server (please complete the following information):

ModSecurity version (and connector): 3.0.10
WebServer: golang 1.20.7 with Echo Framework 4.11.1
OS (and distro): Wubuntu 11.3 last updates

Rule Set (please complete the following information):
odsecdefault.conf
crs-setup.conf
REQUEST-913-SCANNER-DETECTION.conf
REQUEST-942-APPLICATION-ATTACK-SQLI.conf
Additional context
This intervention seems also not to return any log output from the SetServerLogCallback function.
modsecurity_debug.log

crs-setup.conf modifiied lines
SecDefaultAction "phase:1,log,auditlog,deny,status:403"
SecDefaultAction "phase:2,log,auditlog,deny,status:403"

Hello @ampminfo ,
You may wish to inquire with the rule publisher about this rule, as they would know more about any issues with it that have been experienced by other users.  I'll note, in general, however:

942421 is a CRS rule with paranoia level 4; rules with paranoia levels of 3 or 4 are known to be highly prone to false positives
it appears that the tested string is truncated, so it is entirely possible that there are 3+ 'forbidden' characters (as defined by the rule) in the string
given that it is a cookie value that could explain why it only occurs on some requests but not all. Cookie values, by their nature vary, and may sometimes have a certain combination of characters but not other times
the other thing that is notable about the rule is that there are characters in the regular expression that are not part of the ASCII set (0x00-0xF7). That is unusual and it might be possible that that is causing some anomalous effects.

With no further activity, I'll go ahead and close this.


Red Hat 9 fails

FAILED! => {"changed": false, "failures": ["No package python38-psycopg2 available."], "msg": "Failed to install some of the specified packages", "rc": 1, "results": []}

Thank you. I have merged your fix.


My problem is the login

Steps to Reproduce

Run flutter create bug.
Update the files as follows: ...
...

Expected results:
Actual results:

Logs

Hi @Elian28
can you please provide flutter run -v, flutter doctor -v and, minimal reproduction code?
Thanks


Contact

Sir i just need to talk to you on this. I dont find any other way to through u my message. Kindly contact me i need help regarding this code of NLP.

Sir i just need to talk to you on this. I dont find any other way to through u my message. Kindly contact me i need help regarding this code of NLP

Can you send the data?


Add KubeVirt platform type

This is the first PR adding KubeVirt provider to Openshift.
Please find the related enhancement for this change:
https://github.com/openshift/enhancements/pull/417
In addition to this repository, there are also changes in repositories (WIP):

openshift/installer (https://github.com/nirarg/installer/tree/kubevirt-provider)
openshift/machine-api-operator (https://github.com/nirarg/machine-api-operator/tree/kubevirt-provider)
openshift/cloud-credential-operator (https://github.com/bardielle/cloud-credential-operator)
openshift/machine-config-operator (https://github.com/ravidbro/machine-config-operator/tree/add_ocp_platform)

Also added new repository for machine-api-operator access the new provider (WIP):

https://github.com/ravidbro/cluster-api-provider-kubevirt

/test verify

/assign @knobunc

/lgtm

@knobunc this PR is not going to be downported to release-4.6, do we still need to open a bugzilla for that?


Creating a plugin from this

@alex720 and i want to create a Teamspeak plugin based on TeamspeakHasher's functions, if that's okay i'd like to keep some of the thoughts here so you can give your opinion if you want ❤
Roadmap:

[ ] Read and parse Identities from %appdata%\TS3Client\settings.db's ProtobufItems table with a QSqliteManager.
[x] Build Qt UI with a listwidget to switch between Identities and a section for info's/actions
[ ] Build backend for UI (kms)
[ ] Multi-instance support (increasing multiple IDs at once)
[ ] Exporting back to *.ini file for teamspeak import

So the plugin base should be easy since Teamspeak's plugin API is C++ compatible

[ ] Read and parse Identities from %appdata%\TS3Client\settings.db in the ProtobufItems table.
[ ] Build Qt UI with a listwidget to switch between Identities and a section for info's/actions

Note that extracting the public key from the database string is exactly what TSIdentityTool implements. So you probably want to integrate this as well into the plugin.
Moreover, I recommend not to display the private key in the GUI, since this will definitely cause users to leak their private key. Obviously, the same applies to the obfuscated identity string as well. If you really want to display this, please add a warning to the GUI.
Finally, I would strongly prefer the term "counter" as opposed to "offset", since the latter is misleading.

Right, it can "decode" the final identity string containing the "counter" and the private key but we'll need to read all the identity strings from that giant blob of protobuf either byte by byte or using a protobuf "de-obfuscator" first. That's going to be the tricky part. (putting it back in with the new counter wil, be even more tricky but that's low prio.)

users of the plugin are not supposed to screenshot this but yeah i can add a warning messagebox that shows up on first use

i'll try but i cant promise anything cause it was always refered to as offset for me since i started learning about teamspeak IDs from @Splamy

Heh, it might be misleading but teamspeak itself calls it client_key_offset


Fix stuck on Loading StaticQuery

StaticQuery doesn’t seem to play well with query coming from a variable.

Fascinating, what a strange bug. Thanks @ryanditjia !


[Web][SIMS-#1115]UI updates ministry - Part 1 (Login and Sidebar)

@guru-aot Hi guru, looking good. Just missed this one (removing the uppercase formatting). Thanks.


M #-: better English wording

I'm not an English expert, neither a native speaker but a comma , before which hurts me a lot.
In the opposite, the comma should be present to separate the condition to the action.

THANKS merged as per b186d196a69802f9eca5b8fbaf070c77e07dd342


don’t minify files that already advertise as minified.

@ef4 r?

:+1:

@ef4 do you think its worth it, to read the contents of files and see if they look minified?

Broke Ember's build :crying_cat_face:

Basically, in emberjs-build we copy ember.prod.js to ember.min.js then we minify.  I'll work on getting that updated (because its pretty crazy now ;P).


Allow JdbcMigrations to be non-transactional

Currently it is not possible for JdbcMigrations (and SpringJdbcMigrations) to be non-transactional.
This PR allows to override JdbcMigrations (and SpringJdbcMigrations) to mark them as non-transactional.
Please let me know if there are any questions. Documentation PR: https://github.com/flyway/flywaydb.org/pull/103

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

+1 badly needed

Closing as this has now been implemented as part of a bigger API change for Java-based migrations.

Axel,  bit of a noob question:  I probably misunderstood the above, and we are running with JDBC driver against MSSQL2017  but  via   CommandLine.   The above improvement in 5.2 has no effect on CommandLine exec of FLYWAY, does it ?   Any plans to make  CommandLine based executions honour the MIXED flag (or at least allow  before and after callbacks to run in nonTran mode) ???   Danke.

@seb-urbaniak This works with all clients, but you must first migrate your JdbcMigrations to the new JavaMigration interface (it's trivial). See https://github.com/flyway/flyway/issues/2164
Please raise a separate issue for the callbacks.

@axelfontaine lol, kapiere ich nicht.  We call flyway.cmd  directly from DOS. #2164 seems a different barrel of fish. :)


Update mobile-appconfig.rst to include new AppConfig timeout property

LGTM, thanks @sadohert!

@enahum - My Bad... The customer was looking for docs from me and I dove in here.
Do we you want me to pull it back?  Woudl adding heavy caveats be okay?  "Not recommended for most deployments.  This parameter helps to address issues with infrastructure performance"  or something like that?

@sadohert I want us to discourage people from using it.. not sure about the words to use, but 5/5 on the intention.

@sadohert, @enahum: should this PR be closed?

I'd say we should close it

I understand we really don't want it to be used, but my concern would be that the option gets lost.  There's a customer that needs it now, and there could be others in the future.  If this isn't surfaced in any way it would be an unfortunate lost opportunity to help another customer that might need it.
I would like to hold onto it with strong caveats against it.

@sadohert maybe add a WARNING saying that this is something that we discourage as it could be hiding underlaying issues blah blah blah, this is something that the current customer that is using it should really look into and not relay on this.

I agree with a warning and retaining the info. This could also work well as a forum post, if we don't specifically want to commit to/support this and if we'd like feedback from the community in terms of usage and improvement?

+1 on putting this into a forum post instead.

@svelle, would this be beneficial to Support in a different format/place than the docs?

I recommend that this ticket for documentation updates be converted to a ticket to create a new article in the Support KB. @svelle - If you agree, can a member of the Support team take this one on, please?

thanks @cwarnermm good point about this not needing to live in the docs.
@sadohert seeing as this is a relatively old PR do you see any use in us adding this to the KB?

@svelle  - I think its worth adding to the KB.  I would really prefer there was SOME artifact that can be discovered to help someone in teh future.


Fix link to instructions for installing kpack

Is there a related GitHub Issue?
No related Issue.
What is this change about?
The link to the instructions for installing kpack was broken and has been corrected.
Does this PR introduce a breaking change?
No breaking change.
Acceptance Steps
NA
Tag your pair, your PM, and/or team
NA
Things to remember
NA

Kpack used to put very convenient ready-to-run commands in their release notes (see v0.9.1) but they didn't for their last release. I was hoping they'd start doing it again in the next release 😅 which is why I was waiting. The generic instructions are not great unfortunately 😕 but they're still better than releases with no instructions at all, so we should probably merge this and wait for something better.

@georgethebeatle Thanks for suggesting good ideas.
I have modified the commit.

Thank you @kei-yamazaki


Encourage to overwrite CommitScheduler.push_to_hub

Demo Space for working implementation examples: https://huggingface.co/spaces/Wauplin/demo_commit_scheduler

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


shap analysis issues

Hi, I just got ideas from your paper and code: i used my own model which accepts two modality image mri and pet to perform model_wrappers in your code. And I got nice AUC、PR curves, but when i started to do shap analysis, the shap_values  array constains a lots of zero. This is my first time using shap analysis, so I want to know what causes such problems.
looking forward to your reply!
您好，受了您的论文和代码启发之后，我尝试自己构建的模型使用MRI与PET两种模态影像联合诊断疾病阶段，在ROC与PR结果方面表现很好，但是我在进行shap分析生成于FigureTable/NeuroPathRegions/shap_csvfiles中的csv文件包含了很多0值，这导致我在后续进行您代码中FigureTable/brainNetwork时出现了问题*，所以我想知道在模型表现优异的情况下，shap分析得到的结果不理想的原因是什么。
期待您的回复!

I've solved the problem, thank you


Global Bans & PHP 5.6

Hello,
Yesterday I upgraded my Apache 2 web server's PHP version to 5.6, as the version provider through the default YUM repositories was too old. Since then, I am unable to add any servers to my BanManager Web UI. It receives an "invalid response" from the MySQL server. The MySQL server is tries to connect to is not locally hosted, it's a MariaDB instance on the same node as the Minecraft server. I have setup IPTables and file/directory permissions on both servers correctly. Is it my PHP version? Can somebody look into this? I'm also using the mod_security and mod_evasive Apache add-ons.
I'd also like to request support for global bans in the Web UI.
Regards,
John

Check your apache error log

Nothing out of the ordinary I can see.

Invalid response means a JSON response wasn't sent back from the server, suggesting a PHP error occurred.
Double check error reporting is enabled within your settings.php file.
Make sure you have the php mysql libraries installed.
Please make a separate issue for your feature request, otherwise it'll get polluted with your current problem.

http://pastebin.com/kMXCQUp8

@MeMyselfMC Did you enable debug mode in the settings file? Also try to comment out line 13 and 14 in the ajax/updatesettings.php file. Then check your web server error logs again and paste the output :)

@frdmn http://pastiebin.com/5599ac717c6a2

yum install php-mysql
As I said previously, ensure you have the php mysql libraries

k

@MeMyselfMC Could you resolve the problem?

@frdmn Yeah lol, sorry for bothering you guys.

@MeMyselfMC No worries, I'm here to help :)


Add patch remove feature

Currently, benchcab does not support removing namelist parameters from the base namelist file. For example, legacy CABLE branches may not be compatible with a namelist parameter in the base namelist file.
This change adds a patch_remove key which will remove a given set of namelist parameters for a specific branch.
Fixes #92

Integration tests
Test patch remove feature:
#!/bin/bash
bench_example_dir='bench_example_test_patch_remove'
rm -rf $bench_example_dir
git clone git@github.com:CABLE-LSM/bench_example.git $bench_example_dir
cd $bench_example_dir
git reset --hard 6287539e96fc8ef36dc578201fbf9847314147fb
cat > config.yaml << EOL
project: tm70

experiment: AU-Tum

realisations: [
  {
    path: "trunk",
  },
  {
    path: "branches/Users/mm3972/CABLE_GW_into_trunk_version2",
    revision: 9571,
    build_script: offline/build.ksh,
    patch_remove: {
      cable: {
        soilparmnew: nil
      }
    }
  }
]

modules: [
  intel-compiler/2021.1.1,
  netcdf/4.7.4,
  openmpi/4.1.0
]

science_configurations:
  - cable:
      cable_user:
        CONSISTENCY_CHECK: False
EOL
benchcab run -v

Job standard output:
Running fluxsite tasks...
Running task AU-Tum_2002-2017_OzFlux_Met_R1_S0... CABLE standard output saved in /scratch/tm70/sb8430/benchcab_integration_tests/bench_example_test_patch_remove/runs/fluxsite/tasks/AU-Tum_2002-2017_OzFlux_Met_R1_S0/out.txt
./cable cable.nml
Adding attributes to output file: /scratch/tm70/sb8430/benchcab_integration_tests/bench_example_test_patch_remove/runs/fluxsite/outputs/AU-Tum_2002-2017_OzFlux_Met_R1_S0_out.nc
Running task AU-Tum_2002-2017_OzFlux_Met_R0_S0... CABLE standard output saved in /scratch/tm70/sb8430/benchcab_integration_tests/bench_example_test_patch_remove/runs/fluxsite/tasks/AU-Tum_2002-2017_OzFlux_Met_R0_S0/out.txt
./cable cable.nml
Adding attributes to output file: /scratch/tm70/sb8430/benchcab_integration_tests/bench_example_test_patch_remove/runs/fluxsite/outputs/AU-Tum_2002-2017_OzFlux_Met_R0_S0_out.nc
Successfully ran fluxsite tasks

Running comparison tasks...
Comparing files AU-Tum_2002-2017_OzFlux_Met_R0_S0_out.nc and AU-Tum_2002-2017_OzFlux_Met_R1_S0_out.nc bitwise...
nccmp -df /scratch/tm70/sb8430/benchcab_integration_tests/bench_example_test_patch_remove/runs/fluxsite/outputs/AU-Tum_2002-2017_OzFlux_Met_R0_S0_out.nc /scratch/tm70/sb8430/benchcab_integration_tests/bench_example_test_patch_remove/runs/fluxsite/outputs/AU-Tum_2002-2017_OzFlux_Met_R1_S0_out.nc
Failure: files AU-Tum_2002-2017_OzFlux_Met_R0_S0_out.nc AU-Tum_2002-2017_OzFlux_Met_R1_S0_out.nc differ. Results of diff have been written to /scratch/tm70/sb8430/benchcab_integration_tests/bench_example_test_patch_remove/runs/fluxsite/analysis/bitwise-comparisons/AU-Tum_2002-2017_OzFlux_Met_S0_R0_R1.txt
Successfully ran comparison tasks

======================================================================================
                  Resource Usage on 2023-08-08 12:14:45:
   Job Id:             92172069.gadi-pbs
   Project:            tm70
   Exit Status:        0
   Service Units:      1.18
   NCPUs Requested:    18                     NCPUs Used: 18
                                           CPU Time Used: 00:03:13
   Memory Requested:   30.0GB                Memory Used: 1.72GB
   Walltime requested: 06:00:00            Walltime Used: 00:01:58
   JobFS requested:    100.0MB                JobFS used: 0B
======================================================================================


How to install it manually?

I refer to the tutorial and I know that I should run the react-native link command after installation. However, due to the different structure of our project directory, running react-native link does not make it work. So I want to know how to manually install this library to make it work?
Thanks!

@suuzee linking the library is the same as documentation describe it https://facebook.github.io/react-native/docs/linking-libraries-ios, but you're right it should be available in the readme, will add documentation for manual installation soon, thanks!

any idea how to solve this https://github.com/react-native-community/react-native-text-input-mask/issues/73

following this https://github.com/react-native-community/react-native-text-input-mask/issues/54#issuecomment-406718514 does not solve


feat(profile): ProfileTwitterLink component

closes #13

Before submitting this PR, please make sure:

[x] Code builds clean without any errors or warnings
[x] I've merged recent default branch -- main and I've no conflicts
[x] I didn't break any original functionality

PR type

[x] Feature

Screenshot

I don't know if it's a good way to duplicate "Identity" class.
Let me know if I have to import it instead.

@roiLeo UI is really nice !
just some things:
Afaik ProfileLink is composed of identity and link to subscan
If I were you I would add new boolean prop like showTwitter or withTwitter which would show the tw handle

Screenshots
GalleryItem

CollectionItem

LGTM


Simplify the testing of the toml extra, fixing #1084

@portante You are really on top of these pull requests! :)

@portante You are really on top of these pull requests! :)

@portante You are really on top of these pull requests! :)

Seems like best way to learn about trips and tips for coverage to review the work being done to coverage itself.

@portante You are really on top of these pull requests! :)

Seems like best way to learn about trips and tips for coverage to review the work being done to coverage itself.


Remove journal backup flag from start master

parity with the bash side change in https://github.com/Alluxio/alluxio/pull/17933

alluxio-bot, merge this please


jsonpb: No way for types with UnmarshalJSONPB to unmarshal to nil

Any message type that implements the UnmarshalJSONPB function will be initialized during object unmarshal with no apparent workaround.
For example, consider a special type that must contain a valid UUID string:
message UUIDValue {
    string value = 1;
}

When unmarshalling, we'll just check that it matches some accepted UUID format.
func (m * UUIDValue) UnmarshalJSONPB(_ *jsonpb.Unmarshaler, data []byte) error {
    if !validateUUID(data) {
        return fmt.Errorf(`invalid uuid '%s' does not match accepted format`, string(data))
    }
    m.Value = strings.Trim(string(data), `"`)
    return nil
}

But now when unmarshaling any object with a UUIDValue in it, providing a JSON value of null will cause the UUIDValue field to be initialized (and error) instead of being set to nil like any standard message or WKT. I don't see anything that can be done in the UnmarshalJSONPB method that can cause the field to be niled either.
Note that providing no JSON value at all will result in the field becoming nil, which feels inconsistent and unintuitive.

UnmarshalJSONPB and MarshalJSONPB were not really well-thought out and will probably be deprecated in the future. It's interacted poorly in the past with regard to the JSON<->PB specification and we probably want to diminish rather than increase support for it

UnmarshalJSONPB and MarshalJSONPB were added so that custom protobuf types can support the JSON<->PB specification. However, the root of the issue is the lack of protobuf reflection support. If we had that, the jsonpb would not care whether a message was a "custom" type as long it properly implemented the reflection API. Since we are moving to a world with first-class support for reflection, we're going to decline more changes to those interfaces. They will be deprecated in the near future.
See #364.


Bump grpc, protos dependencies

Bumping up various grpc-related dependencies.
grpc 1.35.0 itself bumped re2j and google common protos (which are matched in this PR).

Thanks! Can you please sign your commit and re-push? See https://github.com/envoyproxy/envoy/blob/main/CONTRIBUTING.md#fixing-dco and https://docs.pi-hole.net/guides/github/how-to-signoff/.

@akonradi Fixed!

Thanks!


This is for the Pi4 base unit, or the PiZero board that slot into the cluster hat?

CBRIDGE is the base image which it always creates.
The USBBOOT Px images are then made from the CBRIDGE image (if USBBOOTLITE/USBBOOTSTD/USBBOOTFULL have been set to something other than 0 in config-local.sh).
CNAT/P1/px are again made from the CBRIDGE image by updating the "init=.." part in /boot/cmdline.txt to reconfigure on first boot.

CBRIDGE is the base image which it always creates.
The USBBOOT Px images are then made from the CBRIDGE image (if USBBOOTLITE/USBBOOTSTD/USBBOOTFULL have been set to something other than 0 in config-local.sh).
CNAT/P1/px are again made from the CBRIDGE image by updating the "init=.." part in /boot/cmdline.txt to reconfigure on first boot.

I'm sorry, I didn't quite understand that. Maybe I'm not the right audience for this software.

I'm sorry, I didn't quite understand that. Maybe I'm not the right audience for this software.


Q: Scale Master Nodes?

Hej!
I've spin up a fresh Cluster on Hetzner. Works great so far.
I found some guides on how to scale and size the worker nodes, but no for master nodes.
Because i've choosen "relatively" big master-nodes when creating the cluster, i would like to scale the down a bit.
How can i scale down/up the VM-Size of all three master nodes?

Vertical Scale Masters

Scaling up the control plane nodes is a bit more complicated. That's because the worker nodes can be easily replaced and/or powered off, but that's not the case for the control plane nodes.
In this case, you can't rely on Terraform to scale up the control plane nodes for you, because Terraform would power off all control plane nodes at the same time, which could make etcd lose the quorum (in that case your cluster would be corrupt/lost).
I haven't tested it, but I think you might have success if you do something like this:

Drain one of the control plane nodes (kubectl drain <node-name>)
SSH to that node and power it off (sudo poweroff)
Login to Hetzner Control Panel and scale up/down the instance
Power on the instance after scaling it
Wait for the node to come up

Make sure kubectl get nodes returns Ready for that node
Just to make sure, check etcd logs for that node (run kubectl get pods -n kube-system and then find the etcd pod for the node you restarted (each etcd pod has the node hostname in its name))

Repeat the steps for the remaining two control plane nodes (make sure that you do it for only one node at a time)
After completing all steps for all control plane nodes, update the instance size in terraform.tfvars file
Run terraform refresh to update your Terraform state with the changes you did manually
Run terraform plan and make sure that it doesn't propose any additional changes

Note that some providers might not allow you to scale down beyond the size you used when creating the instance. I think Hetzner might allow you to scale down, but if that's not the case, you'll have to use a different approach.

can confirm, the procedere does work!


[core.ACL] ACL's

Not sure how I want to implement this

blocked by #9
I'll need a way to store user id and access amount to disk

Will need to load presistant user data when bot loads
will then need to load ACL's from disk
Put ACL defaults in config, so commands could do @command("^ping", access=ACL.admin), would make it strictly admin command

Here are some considerations: (furgo storage for now)

How do I go from username in channel to uid? This could be challenging to do; would need a way to find and resolve conflicts
How much flexibility to give to the bot owner. Should access levels be restricted to a range?

Owner getting 1000; and all other ranks are from 0-999
Should there even be considered ranks?

Command decorators should allow for access=123, but how could a bot owner configure commands in his bot? Should we leave it to going through each plugin and modifying access on a per command basis? This brings up another point; could we centralize plugin configs somewhere?
Some users could be white listed for certain commands? So they may not have access but are allow to use it

Blocked by #22

Implemented in 3fdbf1b


Updated to exo v2.5.1

@thasso I would ask for the generic review, @jtreveset-castlabs - dash parser essentials stuff


Pymbar4 bootstrapping

Adding support for bootstrapped errors to all expectations.
Computes bootstrap uncertainties and covariances for:

compute_free_energy_differences
compute_expectations
compute_multiple_expectations
compute_perturbed_free_energies
compute_entropy_and_enthalpy

These are accessed by using the uncertainty_method = "bootstrap". Note that MBAR needs to be initialized with "n_bootstrap=(some integer)" samples in order to do use this method - an error is omitted otherwise.  Possibly could be done differently, but the logic is simpler this way for now.
I could use a little help in rewriting the pytests.  Currently, to get bootstrap uncertainties from various functions, as one will now want to check both the default uncertainties and the bootstrap uncertainties, and MBAR needs to be initialized differently each way, which the pytests don't seem to handle.
Note this is built off of #444, so many of the differences are from that branch.

@mrshirts : I should be able to test this out in the next couple days.
What's your expected timeline for getting this merged and pymbar 4 released? The reason I ask is because I am generating data for a paper and ideally I would be able to use a stable version of pymbar to do the bootstrapping.
@mikemhenry is planning on fixing the pymbar 3 unit tests (which will take about a day) so that we can get https://github.com/choderalab/pymbar/pull/302 merged in, but if you're planning on having pymbar 4 released very soon, then maybe its not worth his effort.

@mrshirts : I had a discussion today with @jchodera  and the perses / openmmtools developers -- since our tools depend on pymbar3, we are willing to take over maintaining pymbar3, so I think @mikemhenry will work on cutting a bugfix release that fixes the CI and contains the bootstrap branch you wrote awhile ago (https://github.com/choderalab/pymbar/pull/302).
I'll make sure to carefully compare the code in this branch with that of the older branch and will fix any bugs in the old branch. I will also test both branches to make sure they yield similar results. Will post an update once I've done this!

I have concerns about maintaining multiple branches.  I think that it's fine to keep pymbar3 alive for a while to make sure that toolchains have time to adapt, and port any bugfixes back, but I think it's better to keep new development linear.  The output formats of pymbar3 are really restrictive in addition.
Note that #302 is incorrect in at least one way (randomly selects from all samples, instead of ones from one point), and it doesn't support bootstrapping of expectations, etc.
@zhang-ivy, shoot me an email at Colorado address and we can set up a time to talk over things with @jchodera / perses developers?  I'd rather allocate time in helping people to move forward rather than supporting old, out-of-date code.

@zhang-ivy are you going to make a new PR with just the bootstrapping stuff, or take overt this one? I just want to make sure either on this PR or a future PR, the correct branch pymbar3 is selected.

Yes, so what we want is once #302 is merged in to master, and a 3.0.8 version is released, then afterwards master should point to pymbar4, and what is currently master should be pymbar3_ltm or something like that.  @zhang-ivy , @jchodera , is that your understanding?

The failing histogram test just seems to be that a histogram with random samples is not getting data in that histogram.  I will check over that problem again - maybe the way it's handled needs to be tweaked so this is less likely to happen.

Yes, so what we want is once https://github.com/choderalab/pymbar/pull/302 is merged in to master, and a 3.0.8 version is released, then afterwards master should point to pymbar4, and what is currently master should be pymbar3_ltm or something like that. @zhang-ivy , @jchodera , is that your understanding?

Yes!

@Lnaden, thoughts about adjusting the pymbar tests relating to uncertainties?  Specifically, what would you look at for the fact that we can't just add 'bootstrap' as an uncertainty_method variable, since MBAR needs to be initialized with bootstraps for this to happen? So therefore, when uncertainty_method = bootstrap is called, we need to  those MBAR objects to be initialized with bootstrapping - but we don't want ALL of them to be initialized with bootstraps, because that takes a lot of extra computational time.

Currently, to get bootstrap uncertainties from various functions, as one will now want to check both the default uncertainties and the bootstrap uncertainties, and MBAR needs to be initialized differently each way, which the pytests don't seem to handle

thoughts about adjusting the pymbar tests relating to uncertainties?

A couple of options. The simplest is to do nothing special and just initialize the n_bootstraps on the mbar_and_test fixture. Because the mbar_and_test fixture is a scope=module fixture, it only ever gets initialized once for the whole module of tests per parameter, a total of twice. At n_bootstrap=1000, this adds about 15 seconds to the total test suite time on my local machine.
There is a separate option where we can rebuild the tests to pass bootstrap parameters, but I think that adds far more complexity than is needed here.
Let me do a quick tinkering to test bootstrap uncertainty method and to add bad initialization options to make sure things fail correctly as well.

it only ever gets initialized once for the whole module of tests per parameter, a total of twice.

Ah, I didn't realize that!
And 1000 bootstraps are not needed.   200 should be plenty for decent uncertainties (so 3 seconds?) - maybe even 50, but then you might get a few statistically off samples.

So, it's not doing the automated testing because there's now a merge conflict (I changed the base from the "pymbar4 at the time of fork", to "pymbar4 now", which I'm not sure was the correct thing to do, correct?

Also @zhang-ivy I noticed that 'BAR' initialization is just set up for the very beginning not as one of the solvers.  I'm not sure if would help THAT much to start from BAR (versus starting from the previous step) but I can test and see (though in pymbar 4, by moving that into a separate function that can be called.

pymbar4 at the time of fork", to "pymbar4 now", which I'm not sure was the correct thing to do, correct?

What you probably wanted to do was rebase this, or not have changed it at all. Git is pretty good about 3-way merge (normal) and then fastforwarding commits. Since we're already here though, don't do anything else, we just have to fix the merge conflict.

OK, so the following does appear to make bootstrapping faster (If the states overlap in order).  See the latest commit for the code.

Initialize each bootstrap iteration with the "original" f_k.
The run BAR (with the bootstrapped energies), starting from these original f_k.  BAR gives results that are closer (at least in some representative cases).
Pass these to adaptive, without having a minimum number of self-consistent iteration.
Set tolerance to 10-6.

At least for one relatively difficult data set, it seems to take only 1-3 iterations per bootstrap.
I can potentially put the changes in #302 if you can in exchange do some more hard testing on pymbar4 :)

"You" is @zhang-ivy who wants the features, not @Lnaden!

How many iterations does are required per bootstrap for that difficult dataset if you don't use any of the 4 improvements you mentioned?

For this particular data set, ~1500 steps for adaptive, or a run of L_BFGS + 2.
Without bar, just initializing from the true number, 4-5 steps.  requiring 2 steps of self-consistent iteration (which helps adaptive converge), then 4-7 steps.  Increasing the tolerance adds a step usually, but there's a decent difference between 3 and 2 steps!

Sounds good. Can you elaborate on what you mean by hard testing?

Try it on the variety of calculations you would use pymbar3 for, basically, plus trying a few combinations of inputs for each one.

@mrshirts : I just realized I don't think I understand min_sc_iter in this branch. I thought it was telling the solver the minimum number of self consistent iterations to run. However, it doesn't seem to actually control whether the self consistent iteration is computed (as both self consistent and newton raphson are computed either way here) -- it only controls which one is chosen. Do you think it would make more sense to refactor this so that if sci_iter < min_sc_iter, the self consistent gradient/free energy is not computed? I would guess this would speed things up quite a bit.

Sorry for not being clear.  It does computes both, but requires it to select the self_consistent_iteration to be chosen for the first min_sc_iter iterations. This is because in rare situations, NR diverged.  A couple of SC seemed to always resolve the problem.   That is an empirical finding, though.

@mrshirts : Ok, so why would we expect that skipping self consistent iterations would speed things up? If both self consistent and NR are being each iteration computed either way, then we wouldn't expect skipping the sc iterations to speed anything up, right?

If we are far away, then SC is safer, but close, it is significantly slower.  If we are close to the answer, then each NR iteration should be a better solution, and should converge faster, and SC iterations are wasted.  This is the theory, and it appear to be true in practice - finishing in 1-3 iterations, instead of 3-5.

Even though we know that for bootstrapping, we only want the NR solution, we are wasting computational effort each time we compute the SC solution.

We don't actually know that!

While I haven't actually timed how long the self consistent computation takes, I assume its a bottleneck

It involves fewer calculations than NR, so is faster; I have not timed to see how much faster (1/2 the time, maybe?).

I don't think it's a good idea, as if the bootstrap DOES happen to be too far out, you don't necessarily want it to crash. That loses a lot more time that the simulation going slowly. SC will pull it back if it goes off. It's somewhat more important it be robust (and completes) than it be fast (and crashes 1% of the time).

Ah, makes sense. Thanks!

Can you send me an example data file, and I can play around with what should speed it up?
Here are np arrays of the data I used to generate these results. The only problem is that github doesn't allow me to attach files > than a certain limit so I cut the number of samples in half (shape of u_kn is now (36, 90000)) and saved the arrays for you.
bootstrapping_input.zip

It involves fewer calculations than NR, so is faster; I have not timed to see how much faster (1/2 the time, maybe?).

Ah good to know! My guess was wrong then, thanks for clarifying.

With pymbar3 (out of the box), I'm getting 188 seconds for 100 bootstraps.  I tried switching to "L-BFGS-B" for the bootstrapped solver, and it took 194 second, so pretty comparable.
With pymbar4, I'm getting 80 seconds to do 100 bootstraps.   Here, we are skipping the forced SC iterations, and have a larger tolerance (10^-6 vs 10^-12), and I think that's the main effect.   That seems to be a little faster that you are getting, but I don't know why.
Interestingly, for this system, the MBAR solutions for the "original" data are closer than the the BAR solutions for the bootstrapped sample, which is interesting. What is the degree over overlap for this data set?  My other dataset perhaps had less overlap, so the initialization was not as good.
I think using a larger tolerance for bootstrap and removing the "force-take-SC" on the initial steps will help.
This is getting dangerously into the area of optimizing both pymbar3 and pymbar4 simultaneously, which gets messy.   At a certain point, it gets easier to use that time to switch to pymbar4.
I think there are some fundamental limits to how fast it has to go since one has to set things up, etc. What amount of time do you need it to take, to understand what is necessary?  Note that 50 bootstraps is probably decent for computing standard deviations.
Now, pymbar4 will have jax as well, so that may help, we will see (still need to merge those branches, so nothing above reflects it).

With pymbar3 (out of the box), I'm getting 188 seconds for 100 bootstraps. I tried switching to "L-BFGS-B" for the bootstrapped solver, and it took 194 second, so pretty comparable.

Whoa, that's super fast. When I run the default pymbar3 bootstrapping for 100 bootstraps, it takes 565 seconds. When I run 10 bootstraps, it takes 66 seconds. I'm running on a GTX 1080Ti with the following code:

with open("u_kn.npy", "rb") as f:
    decorrelated_u_ln = np.load(f)['arr_0']

with open("n_k.npy", "rb") as f:
    decorrelated_N_l = np.load(f)['arr_0']

initial_time = time.time()
mbar = MBAR(decorrelated_u_ln, decorrelated_N_l, nbootstraps=100, solver_tolerance=1e-12)
results = mbar.getFreeEnergyDifferences(compute_uncertainty=True, uncertainty_method='bootstrap', return_dict=True)
print(results['Delta_f'][0, -1], results['dDelta_f'][0, -1])
print("Time: ", time.time() - initial_time)

Note I added solver_tolerance as an argument to the API but haven't pushed to #302 yet.

Interestingly, for this system, the MBAR solutions for the "original" data are closer than the the BAR solutions for the bootstrapped sample, which is interesting. What is the degree over overlap for this data set? My other dataset perhaps had less overlap, so initialization with BAR helps. I will switch back to pymbar4 and look at this a bit more.

The overlap is likely pretty poor as the trajectory represents a relative binding free energy calculation for mutating an interfacial residue in a protein:protein system (barnase:barstar).

I think using a larger tolerance for bootstrap and removing the "force-take-SC" on the initial steps will help.

Yep, agreed. I'm working on making the changes to #302 now.

This is getting dangerously into the area of optimizing both pymbar3 and pymbar4 simultaneously, which gets messy. At a certain point, it gets easier to use that time to switch to pymbar4.

Agreed!

What amount of time do you need it to take, to understand what is necessary? Note that 50 bootstraps is probably decent for computing standard deviations.

I don't have a good answer to this -- the longer the analysis takes, the more time it'll take for me to generate plots, and I assume I'll have to re-do this many times as I write up my manuscript, so I was just trying to see how much I could speed it up by playing around with things for a day or so.  I'm happy with where we're at and like you said, I don't think we want to spend any more effort optimizing pymbar3 anyway.
As for the number of bootstraps necessary -- this was something I had been thinking about today -- I had assumed we would need 1000 bootstraps to be confident in our uncertainty estimate, but maybe that's not necessary. Why do you think 50 is enough?

For this particular data set, BAR is having a hard time converging - it's hitting the maximum iterations.  Not sure why, but that explains why it doesn't do so well.  I will investigate later, but I don't know that that's really the medium term solution for the speedup.

Whoa, that's super fast. When I run the default pymbar3 bootstrapping for 100 bootstraps, it takes 565 seconds. When I run 10 bootstraps, it takes 66 seconds. I'm running on a GTX 1080Ti with the following code:

I'm just running on my MaBook Pro laptop. Maybe it uses the cores effectively? I tried rerunning the exact code you just sent (same settings) on pymbar 3 , #302 PR, and it completed in 128 seconds for 100 bootstraps (that was on the shorter 36x90000 matrices you sent).

As for the number of bootstraps necessary -- this was something I had been thinking about today -- I had assumed we would need 1000 bootstraps to be confident in our uncertainty estimate, but maybe that's not necessary. Why do you think 50 is enough?

Brad Ephron's book says 50-200 is enough for a good covariance.  If you wanted a rough estimates, I think 20 should be plenty, then use 200 for final results.   The uncertainty might be off by a factor of 50% or so with just 20 samples, but it's enough to give you a basic understanding of what is going on.   1000 is almost certainly overkill.

I'm just running on my MaBook Pro laptop. Maybe it uses the cores effectively? I tried rerunning the exact code you just sent (same settings) on pymbar 3 , https://github.com/choderalab/pymbar/pull/302 PR, and it completed in 128 seconds for 100 bootstraps (that was on the shorter 36x90000 matrices you sent).

Oh my bad, I really don't need to be running this on a GPU, since its not GPU accelerated. I just used a CPU and requested 24 threads and it runs MUCH faster, like 3-4x faster, which is what I was originally aiming for. Thanks for helping me catch that mistake!

Brad Ephron's book says 50-200 is enough for a good covariance. If you wanted a rough estimates, I think 20 should be plenty, then use 200 for final results. The uncertainty might be off by a factor of 50% or so with just 20 samples, but it's enough to give you a basic understanding of what is going on. 1000 is almost certainly overkill.

Ah ok good to know, thanks! I'll see if I can get my hands on his book to learn more about this but will use your suggestion moving forward.

I just used a CPU and requested 24 threads and it runs MUCH faster, like 3-4x faster, which is what I was originally aiming for.

There we go!

Ah ok good to know, thanks! I'll see if I can get my hands on his book to learn more about this but will use your suggestion moving forward.
https://www.hms.harvard.edu/bss/neuro/bornlab/nb204/statistics/bootstrap.pdf

I made the "use BAR to initialize each bootstrap" more robust, but in the end, it wasn't as fast as just initializing from the original guess. I'll think about the best logic!

Ugh, getting the issue again where black locally doesn't give the same results as black in github for the four files, so I don't know what to fix.

Hmm, I think the lint issue was actually with pymbar4, not pymbar4_bootstrapping.  Trying again.

Going to merge this now, and figure out lint later.

Any idea why this is happening? I would've thought it'd be resolved by https://github.com/choderalab/pymbar/pull/443 but then I realized this error occurs at line 730 in mbar.py, and the PR fixes a later line.

The fix was in #441, #443 was a bugfix to #441, so maybe it's still #441?

One other question that would be great to know on a hard system - how does the pymbar4 bootstrapping work for entropy and enthalply errors?  (n_bootstraps in init and uncertainty_method = "bootstrap").

Is it returning a successful result in the end? Then it is working as intended (it tried 'hybr', then something else). Maybe there should be an additional message saying what method succeeded?

Yes, its returning a successful result. I think adding an additional message with the method that succeeded would be nice!

The fix was in https://github.com/choderalab/pymbar/pull/441, https://github.com/choderalab/pymbar/pull/443 was a bugfix to https://github.com/choderalab/pymbar/pull/441, so maybe it's still https://github.com/choderalab/pymbar/pull/441?

Just saw this comment: https://github.com/choderalab/pymbar/pull/441/files#diff-39154a13f665187718c90bf1b7a14a9edb282028dd0fbd94a8dfb02168e42580R648-R652
Do you think we should change logfactor to small number like 0.001?

One other question that would be great to know on a hard system - how does the pymbar4 bootstrapping work for entropy and enthalply errors? (n_bootstraps in initializations and uncertainty_method = "bootstrap" in compute_entropy_and_enthalpy).

Sure I can check this later tonight!

Do you think we should change logfactor to small number like 0.001?

In the current code, it's something like 4x(the smallest mantissa accessible to numpy, like 1e-15).  That has worked so far.  0.001 is not so good, since then numbers near in magnitude 0.001 can't be averaged.

Sure I can check this later tonight!

Great!

@mrshirts : I just tested bootstrapping for compute_free_energy_differences() and compute_entropy_and_enthalpy() with this code (with pymbar4 only, since pymbar3 doesn't do bootstrapping for compute_entropy_and_enthalpy()):
    # Test compute_free_energy_differences()
    decorrelated_u_ln, decorrelated_N_l = analyzer._compute_mbar_decorrelated_energies()
    mbar = MBAR(decorrelated_u_ln, decorrelated_N_l)
    results = mbar.compute_free_energy_differences(compute_uncertainty=True)
    print("compute_free_energy_differences()", results['Delta_f'][0, -1], results['dDelta_f'][0, -1])

    # Test compute_entropy_and_enthalpy()
    results = mbar.compute_entropy_and_enthalpy()
    print("compute_entropy_and_enthalpy()", results['Delta_f'][0, -1], results['dDelta_f'][0, -1], results['Delta_u'][0, -1], results['dDelta_u'][0, -1], results['Delta_s'][0, -1], results['dDelta_s'][0, -1])

    # Test compute_free_energy_differences() with bootstrapping
    mbar = MBAR(decorrelated_u_ln, decorrelated_N_l, n_bootstraps=200)
    results = mbar.compute_free_energy_differences(compute_uncertainty=True, uncertainty_method='bootstrap')
    print("compute_free_energy_differences() bootstrapped", results['Delta_f'][0, -1], results['dDelta_f'][0, -1])

    # Test compute_entropy_and_enthalpy() with bootstrapping
    results = mbar.compute_entropy_and_enthalpy(uncertainty_method='bootstrap')
    print("compute_entropy_and_enthalpy() bootstrapped", results['Delta_f'][0, -1], results['dDelta_f'][0, -1], results['Delta_u'][0, -1], results['dDelta_u'][0, -1], results['Delta_s'][0, -1], results['dDelta_s'][0, -1])

Here are the results.
For case 1 (barnase:barstar mutation):
compute_free_energy_differences() -52.62531138448979 0.04448291789808753
compute_entropy_and_enthalpy() -52.62531138449739 0.044482917898087226 -58.26693039969541 9.695425671134956 -5.64161901519401 9.693498249036294
Failed to reach a solution to within tolerance with hybr: trying next method
compute_free_energy_differences() bootstrapped -52.62531138448979 0.10343092206869589
compute_entropy_and_enthalpy() bootstrapped -52.62531138449739 0.10343092206869589 -58.26693039969541 69.68171363279014 -5.64161901519401 69.67825680667228

For case 2 (ala dipeptide mutation):
compute_free_energy_differences() -52.27795853969293 0.056782925355530894
compute_entropy_and_enthalpy() -52.277958539692946 0.05678292535553127 -57.77404736274184 1.8697858250872512 -5.496088823048922 1.8701373837035669
Failed to reach a solution to within tolerance with hybr: trying next method
compute_free_energy_differences() bootstrapped -52.27795853969293 0.16964559913815944
compute_entropy_and_enthalpy() bootstrapped -52.277958539692946 0.16964559913815944 -57.77404736274184 26.242413812942953 -5.496088823048922 26.239604799661123

Bootstrapping for free energy differences looks good (as I already found in my tests last week), but the bootstrapped uncertainty for the entropy and enthalpy looks much larger than expected...

In the current code, it's something like 4x(the smallest mantissa accessible to numpy, like 1e-15). That has worked so far. 0.001 is not so good, since then numbers near in magnitude 0.001 can't be averaged.

I just tried copying this line from pymbar4 mbar.py:
logfactor = 4.0 * np.finfo(np.float64).eps
        # make sure all results are larger than this number.
        # We tried 1 before, but expecations that are all very small (like
        # fraction folded when it is low) cannot be computed accurately.
        # 0 causes warnings in the test with divide by zero, as does 1*eps (though fewer),
        # and even occasionally 2*eps, so we chooose 4*eps

to replace logfactor here.
But I'm still seeing the same warning:
/home/zhangi/miniconda3/envs/perses-paper/lib/python3.9/site-packages/pymbar-3.0.5-py3.9.egg/pymbar/mbar.py:777: RuntimeWarning: divide by zero encountered in log
  Log_W_nk[:, sa] = np.log(A_n[i, :]) + Log_W_nk[:, l]

but the bootstrapped uncertainty for the entropy and enthalpy looks much larger than expected...

Interesting, can you post the data set somewhere I can look at it?  (like google drive?)  It was looking OK for my tests, but maybe there's an error that didn't show up with that relatively easy test.

But I'm still seeing the same warning with pymbar3:

Yeah, I'd have to take a look at the data set to see.

The data for these experiments isn't too big, so i'm just going to attach them here. I included the u_kn and n_k for barnase:barstar mutation as well as for ala dipeptide mutation.
pymbar3_divide_by_zero.zip

Note that barnase:barstar and ala dipeptide have 24 and 12 thermodynamic states, respectively, but I've added unsampled endstates for lambda = 0 and lambda = 1 such that the states are ordered like [unsampled endstate for lambda 0, sampled endstate for lambda 0, ... sampled endstate for lambda = 1, unsampled endstate for lambda = 1], so the matrices have K = 26 and 14, respectively. Not sure this matters, but just noted it in case it does.

Hmm.  I can't seem to load them.  When I unzip and run I get:
with open("u_kn.npy", "rb") as f:
u_kn = np.load(f)['arr_0']
I get: ValueError: Cannot load file containing pickled data when allow_pickle=False
And when I try
u_kn = np.load(f,allow_pickle=True)['arr_0']
I get:
*** _pickle.UnpicklingError: Failed to interpret file <_io.BufferedReader name='u_kn.npy'> as a pickle
Any thoughts?

Hmm, this is working for me:
with open("bnbs/u_kn.npy", "rb") as f:
    u_kn = np.load(f, allow_pickle=True)

with open("bnbs/n_k.npy", "rb") as f:
    n_k = np.load(f, allow_pickle=True)

Hah, figured out what was doing wrong. Back to work!

Ah, I figured out what went wrong with the expectations bootstrapping. A couple of arrays that needed to be bootstrapped were not. It appears to give very close results now.

@mrshirts : Nice!
Did you have a chance to look into why this warning is appearing for pymbar3 and not pymbar4?
/home/zhangi/miniconda3/envs/perses-paper/lib/python3.9/site-packages/pymbar-3.0.5-py3.9.egg/pymbar/mbar.py:777: RuntimeWarning: divide by zero encountered in log
  Log_W_nk[:, sa] = np.log(A_n[i, :]) + Log_W_nk[:, l]

If not, no worries, I will look into that this week.

I'll try to take a look later today.

There's one really weird point - 24090 is much lower than the other points, and is giving problems.  Not sure if that affects things.

So it fails because with the logic, there always be a number that is zero.  So the idea is when subtracting the minimum value for each row, you actually subtract (minimum_value - small amount), which will result of all entries in that line to be greater than zero.  So setting a slightly larger value for logfactor should solve the problem.

There's one really weird point - 24090 is much lower than the other points, and is giving problems. Not sure if that affects things.

Can you clarify what you're referring to when you say 24090? And what specifically is lower? the free energy?

So it fails because with the logic, there always be a number that is zero. So the idea is when subtracting the minimum value for each row, you actually subtract (minimum_value - small amount), which will result of all entries in that line to be greater than zero. So setting a slightly larger value for logfactor should solve the problem.

Hmm I set logfactor to  logfactor = 4.0 * np.finfo(np.float64).eps in my local version of pymbar3's bootstrap branch and I'm still seeing the same warning..

Ah, OK, I understand better now, and why it was solved in pymbar4. When we subtract out the minimum, in order for it not to be zero, we need to subtract np.min(A) + some factor such that (A-np.min(A))/np.min(A) is greater than eps.  This logic is fixed in pymbar 4, but not pymbar 3 - i.e. the factor we subtract is not too small in absolute terms, but not in relative terms; so technically A-(A_min-eps) = eps for the lowest value, but that is not enough, since if A is too big, then A-(A_min-eps) will round to 0.   See improved logic in pymbar 4 on https://github.com/choderalab/pymbar/blob/e8a68ff0a6e9ef56cf5afed08f5090679e20fefd/pymbar/mbar.py#L872 and the next few lines.
I don't fully understand why the code was not working for very small expectations (expectations of occupation probabilities that were much lower than 1), but it clearly was not, and the new changes seem to fix this.  I will continue to test with the new code to see if there are any issues we didn't get to.

Note that the particular data set is not great, because there's one data point that is 1400 units below the rest of the data points, leading to highly likely overflow.

Thanks for the explanation! I ported the pymbar4 changes you mentioned to my local branch of pymbar3 and the divide by 0 warning goes away.

But at least you see why the pymbar 4 one does work, and can decide how much to port over.

It seems like its a pretty easy fix to port the changes over (only involves changing a few lines), but I'm curious how bad is it to see the divide by 0 warning? In other words, are the returned expectations wrong when we see the warning? I would assume they aren't, but i'm not sure what the implications of dividing by 0 are on the resulting expectations.

but I'm curious how bad is it to see the divide by 0 warning? In other words, are the returned expectations wrong when we see the warning? I would assume they aren't, but i'm not sure what the implications of dividing by 0 are on the resulting expectations.

My understanding is that give the error, the point that is zero (which is the minimum) is left out, which may or many not affect the results.  If the lowest point is very non-representative it would affect it a lot, if it was relatively representative then it would not affect things much.

Ah I see. It sounds like it would definitely be good to port the changes to pymbar3 then. I'll open a new branch for that fix.


Document classification

Describe the feature you'd like
I think it would be a really nice feature if you can implement a simply document classification in the sites of bookstack.
Or maybe in a whole book?
The Idea would be if you create a new Book or a New Sheet you select through a drop-down or something the classification and fillout the needet parts.
And if you export an book or site you had the classification in the footer oder header of the document.
Describe the benefits this would bring to existing BookStack users
Benefit? More Use-Cases for Bookstack, specually if it would be used in scopes where classicfication is required
Can the goal of this request already be achieved via other means?
No idea, but the request with the idea to use tags didn't handle that, because if you export documents that won't be visible.
Have you searched for an existing open/closed issue?

[X] I have searched for existing issues and none cover my fundemental request

How long have you been using BookStack?
Not using yet, just scoping
Additional context
No response

Thanks for the request @tweak19, although I think this is fundamentally the same as issue  #3026.
As mentioned in #1940, I don't see the benefit of creating and maintaining another focused subsystem just for what is essentially a tag/label, especially as different environments would have different expectations upon some kind of classification system.
If you have some dev skills, you could use our visual theme system to customize the export templates to add some indicator for your tag.

Benefit? More Use-Cases for Bookstack, specually if it would be used in scopes where classicfication is required

With regards to benefits in feature requests, I generally don't see "more use-cases" as a viable justification as that could be applied to almost any feature addition, focus is instead on our existing audience.

Since there's been no follow-up or progression here, I'll close this off.


istio-ingressgateway as DaemonSet using istioctl

Describe the feature request
It's already has been opened
https://github.com/istio/installer/pull/398
Describe alternatives you've considered
At the moment manual editing of YAML files are needed to have DaemonSet
[ ] Docs
[X] Installation
[X] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[X] User Experience
[ ] Developer Infrastructure
Additional context
AWS Target groups sees nodes as "Unhealthy" if they do not have istio-ingressgateway pod running on them, manual editing is needed to split pods accross nodes, allowing to set kind Deployment ir DaemonSet for istio-ingressgateway with istioctl would limit possible human errors, less custom interaction with istio install.

Yes, please.  Not being able to get source IP address easily causes a lot of trouble.  Istio supporting this mode out of the box would be incredibly helpful.

any update?

Seems like the issue was ignored, anyway it is achievable with k8s overlays, leaving code snippet for reference
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    ingressGateways:
    - enabled: true
      name: istio-ingressgateway
      k8s:
        overlays:
        - kind: Deployment
          name: istio-ingressgateway
          patches:
            - path: kind
              value: DaemonSet

Would be great if that would also be possible for people who prefer using the helm chart instead of the operator
https://github.com/istio/istio/blob/master/manifests/charts/gateways/istio-ingress/templates/deployment.yaml

Seems like the issue was ignored, anyway it is achievable with k8s overlays, leaving code snippet for reference
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    ingressGateways:
    - enabled: true
      name: istio-ingressgateway
      k8s:
        overlays:
        - kind: Deployment
          name: istio-ingressgateway
          patches:
            - path: kind
              value: DaemonSet

For anyone who faced the issue that the Daemonset is not created, you'd also want to delete the spec.strategy config because it's not supported in DaemonSets:
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    ingressGateways:
    - enabled: true
      name: istio-ingressgateway
      k8s:
        overlays:
        - kind: Deployment
          name: istio-ingressgateway
          patches:
            - path: kind
              value: DaemonSet
            - path: spec.strategy

Any solution for users who took istioctl install path by mistake?


!! IMPORTANT - Is this retired?

Yes.

Rip in pepperino, no more themerino

why did you reopen? lol if you got your answer close it at least keep the place clean


A fatal error has been detected by the Java Runtime Environment:

Please provide a brief summary of the bug
hs_err_pid86419.log
Did you test with the latest update version?

[ ] Yes

Please provide steps to reproduce where possible
No response
Expected Results
Eclipse should not crash
Actual Results
Eclipse crashing
What Java Version are you using?
openjdk version "11.0.22" 2024-01-16 OpenJDK Runtime Environment (build 11.0.22+7-post-Ubuntu-0ubuntu222.04.1) OpenJDK 64-Bit Server VM (build 11.0.22+7-post-Ubuntu-0ubuntu222.04.1, mixed mode, sharing)
What is your operating system and platform?
Ubuntu 22.04.4 LTS
How did you install Java?
No response
Did it work before?
No response
Did you test with other Java versions?
No response
Relevant log output
No response

See #1060


error: cast from 'unsigned char*' to 'SQLUINTEGER' loses precision

Environment

Python: 2.7
pyodbc:  4.0.23
OS: Redhat 5
DB: None - failed to install
driver:  None - failed to install

Issue
pyodbc fails to build with the below error:
 cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++   src/params.cpp: In function 'bool ExecuteMulti(Cursor*, PyObject*, PyObject*)':   src/params.cpp:1600: error: cast from 'unsigned char*' to 'SQLUINTEGER' loses precision   error: command 'gcc' failed with exit status 1

What is the output of gcc --version ?

gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-55)

What version of unixODBC do you have (use odbcinst -j to check), and is this a 32 or 64-bit Python/Linux?
The line in question: https://github.com/mkleehammer/pyodbc/blob/master/src/params.cpp#L1600 has a cast to SQLULEN, not SQLUINTEGER, so unless your ODBC headers are in a strange state, I don't see how such an error could occur.

unixODBC info:
odbcinst -j
unixODBC 2.2.11
DRIVERS............: /etc/odbcinst.ini
SYSTEM DATA SOURCES: /etc/odbc.ini
USER DATA SOURCES..: /odbc.ini
Python is Python 2.7.14 x64-bit
Red Hat Enterprise Linux Server release 5.10 (Tikanga) x64
Could this be due to my system being too old?

This does seem to have to do with unixODBC, for the version 2.2.11 it has to be compiled with BUILD_REAL_64_BIT_MODE defined in order for this error to not occur, you can also try installing the newest unixODBC (2.3.6) as it contains a LOT of bugfixes and uses the "real 64 bit mode" by default

I don't have sudo access so in order to compile and install unixODBC, I will need to specify a different make prefix. Through the pyodbc installation, is there a way to specify a non-default path for unixODBC?

I'm afraid I don't know if there is such a way, it looks like pyODBC uses 'odbc_config --cflags' and 'odbc_config --libs' to determine unixODBC paths, so it looks like it relies on default installation

You can compile and install unixODBC in a different prefix, as long as the libraries and executables are in your PATH before the existing installation, it should work.

That worked. Thanks!


Padding property doesn't take space into account when hasCustomPadding property is true

Describe the bug
For bar graph, when the padding prop is provided(no matter what the value is), the Y-axis label becomes closer to Y-axis. Looks like the default white space between label and Y-axis is gone once you provided a customized padding prop.
When no padding props provided by the consumer:

When provided costomized padding as {left: 30, right: 50}:

To Reproduce
Steps to reproduce the behavior:

provide the padding props in code like

    axisData.padding = {
        right: 50
    };

or
    axisData.padding = {
        left: 30,
        right: 50
    };

Expected behavior
Expected to keep the white space between Y-axis label and Y-axis when padding prop is provided.

Screenshots
see above.
Desktop (please complete the following information):

OS: mac OS
Browser: Chrome

Additional context
Add any other context about the problem here.

We're seeing an issue where the padding right attribute not being honored for the timeline. Not sure if this padding issue will include the resolution for that or if it would be scoped in another fix.

@abhijit945 / @ryanthemanuel - Any thoughts for this issue on when it will be fixed?

@abhijit945 / @ryanthemanuel - Any thoughts on this issue when it will be fixed?

@pranav300 is working on the changes as of this moment. We are trying to figure out the conditions we are trying to solve along with the best possible method to achieve it passively. Since this is core to the functioning of each graph we are being very cautious.
It is our top priority as of now. Any progress will be reflected as part of PR https://github.com/cerner/carbon-graphs/pull/83.

:tada: This issue has been resolved in version 1.12.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Work Item and Build Support

It would be nice to be able to wok with work items and builds in Intellij based IDE's.

@James-Parsons: Thanks for the feedback. We are looking into some workflows around work item and build integration in the IDE. Could you please share with us what you would consider high priority for your daily work to be inside the IDE vs going to the browser?
Thanks,
Madhuri