proposal: Go 2: require explicit type cast to create "nil receiver" interface

Summary
Go's "fat" interface pointers can contain nil data pointer and proper method table pointer. While this property is useful, it may easily become source of undesired bugs - because typed nil is automatically converted to such "half-nil" interface, even if methods on source type do not support nil receivers. Such implicit conversion must result in nil interface pointer. Although, there should still be a way to create nil-receiver interface explicitly
Motivation
Simplest code snippet which demonstrates problem. Please note that error case is not specifically tied to this problem. Any typed nil to interface conversion may cause this
// Library-local error type
type MyError struct {
    // some data fields
}

func (s *MyError) Error() string {
    // Use s and return string
}
// Library-local function
func MyOperation() string, *MyError {
    // Either return string result or MyError error by-pinter
}
// Some func which uses library
func DoStuff() SomeComplexResult, error {
    // Bunch of computations
    if result, err := MyOperation; err != nil {
        return nil, err // typed nil to interface happens here
    }
    // Other bunch of computations
}
// Some error reporter
func TopLevel() {
    // ...
    if result, err := DoStuff(); err != nil {
        print(err.Error()) // nil dereference panic happens here
    }
}

Proposal
Change existing behavior such that implicit conversion of typed nil to interface produces nil interface pointer. Allow create "nil receiver interface" pointer using explicit syntax like nil.(MyError*).(error). This would make most expected behavior the default one, with fallback available.
Drawbacks
Changes semantics, which may result in code breakage in some places where code relies on existing conversion rules.
Alternatives

Introduce lint
Leave as-is

References
Previous proposals which touched this topic:

#24635
#22729
#21538
#27890

Just to clarify what you mean, take the following code:
func main() {
  var me *MyError // Implements error
  var e error

  e = me  // Here e's type would still be converted to error(nil), rather than *MyError(nil)?
}

This would work for me too.  I can't really see a use of keeping track of the type of nil pointer you're carrying around; but I figured someone must want that behavior, or this would have been changed already.
Anyone know of any cases where doing this sort of automatic type conversion would be problematic?
Also, if we did this, and someone wanted the old behavior, how would you specify that?
An advantage of this way is that it silently "fixes" most code to do what people expected anyway.
A disadvantage is that if anyone was relying on the old behavior, this silently breaks it in a way that's 1) nearly impossible to detect programatically, and 2) difficult to restore to functional equivalence.
As I said, I can't immediately see a use for the behavior so the last two points don't bother me.

@target-san If you implement the (*MyError).Error method like the following shows, then the call print(err.Error()) will not panic.
func (s *MyError) Error() string {
    if s == nil {
    	return "<nil *MyError>"
    }
    // Use s and return string ...
}

@gwd
I raised this same question on Go groups quite some time ago and was told nil receivers are actually useful in cases they don't need state. An example I remember is a dummy Read or Write implementation.
And yes, you're right in your example. Implicit conversion under this proposal will result in nil e. Explicit type cast may be used to obtain typed nil - like nil.(MyError).(error). Though syntax may be different.
@go101
Thanks, I know this. But AFAIK not all methods are nil receiver safe even in standard library. My point is, it's very easy to create typed nil accidentally. And normal nil checks cannot spot it.

@go101 Is it? I may be not that knowledgeable in Go, but most other languages assume "receiver" (this, self etc.) is not nil unless type explicitly specifies otherwise. Could you please point to some recommendation like "always check receivers for nil inside methods"?

@target-san Re nil receivers that don't need state: It seems like you could achieve the same effect by just having dummy state; passing back a type based on an int, for example, rather than a type with a nil pointer.
@go101 The problem with your nil-safe error type is that your check to see whether there's an error or not still thinks there was an error (and presumably will abort doing whatever it was trying to do), when the call actually succeed.  Where I recently ran across this was in a web app I'd written, where it did a bunch of checking, then did something like the following:
if display == nil {
  // Render not-found template
} else {
  // Pass 'display' to the appropriate template for rendering
}

I'd changed one of the functions returning a display-compatible structure to return nil when I wanted that thing not visible to this particular user; but instead of showing the "not-found" template, it passed the typed nil pointer to the "display" template, resulting in the user seeing a template error.  Nothing crashed, but it still did the wrong thing in a surprising way.

@gwd Yes. This can be achieved by using some stateless static object. And that would be even better solution IMO. Though we have what we have. And what we have is implicit unchecked creation of broken interface pointers.

@target-san @gwd
sorry, my before conclusion is really wrong.
The correct argument should be if you put a nil pointer in an interface value then call a method of the pointer, then you do it wrongly. This has no differences to calling the method on a the nil pointer directly.

I think if you can't make sure whether or not  an interface value is encapsulating a nil pointer, and you know calling a method of the nil pointer will panic, then you should never call the method on the interface value.
How can this situation happen?

That's exactly the problem I am talking about. You can get into this situation with a perfectly correct  code. No arcane tricks are required. Just directly pass some result  - if that result is a typed nil and destination is an interface which it supports. Check initial post and George's one for examples how you can get such "broken" pointer.

I think if you can't make sure whether or not an interface value is encapsulating a nil pointer, and you know calling a method of the nil pointer will panic, then you should never call the method on the interface value.
How can this situation happen?

This is exactly what this issue is about.  Suppose we have package Foo, which includes package Bar, and package Bar includes package Zot:
type ZotError struct {
  error string
}
func (ze *ZotError) Error() string {
  return ze.error
}

func Zot(arg int) (*ZotError) {
   // Do some stuff with arg, everything turns out fine, return success
  return nil
}

func Bar()  error {
  if normal {
    // Do some stuff
    return nil
  } else {
    // Do some stuff for an excepitonal case that's hard to test
    return Zot()
  }
}

func Foo() error {
  // Do thing 1

  // Do thing 2
  if err := Bar(); err != nil {
    // Success
  } else {
    // Clean up
   return fmt.Errorf("Error doing Bar: %v", err)
  }

  // Do thing 3
}

In this situation, if the "excepitonal case" in Bar() happens, Bar() will return a value of type *ZotError(nil), and the err != nil check will return false, executing the second block.  Foo() will then think that Bar() failed, and try to clean up 'thing 1', even though 'thing 2' actually succeeded.  This might be harmless, or it might put the system in an inconsistent state that would cause data corruption down the road.
If Foo(), Bar(), and Zot() are in separate packages, there is no way for the author of Foo() to know that Bar() might return *ZotError(nil) without inspecting the source code.
The only way for Foo() to be sure that err is actually not nil is to do this instead:
func Foo() {
  if err := Bar(); !reflect.ValueOf(err).IsNil() {
    // Success
  } else {
    e := err.Error()
    // Do something with the error string, dereference NULL
  }
}

But none of the guides for how to write good Go code say this; they all recommend err != nil, which is inherently unsafe.

The problem with the code is here:
func Zot(arg int) (*ZotError) {
    return nil
}

It's generally bad practice to return a concrete error type.  If you always return error, this issue does not happen. The Go FAQ states:

It's a good idea for functions that return errors always to use the error type in their signature (as we did above) rather than a concrete type such as *MyError, to help guarantee the error is created correctly. As an example, os.Open returns an error even though, if not nil, it's always of concrete type *os.PathError.

I'd be more amenable to building a tool that detects this FAQ item than changing the language.

Errors are just an example. Replace with any optional object which gets cast to interface.

@randall77  Indeed that is bad practice given how nil interfaces work at the moment; but it's very far from obvious why.  Humans are very poor at consistently following rules; every "do this for safety" rule we can get rid of will make Go a safer language.

@randall77 Also, this statement isn't quite true:

If you always return error, this issue does not happen.

The following is a pattern that a reasonable developer might use:
// Tries all steps regardless of error; returns an error describing the first issue encountered
func Zot() error {
  var ze *ZotError

 // Do something
 if  error1 && ze != nil {
   ze = &ZotError{error1}
 }

  if error2 && ze != nil {
    ze = &ZotError{error2}
  }
  // and so on

  return ze
}

Here we've followed the advice to return error, but still get something of type *ZotError(nil)

@gwd
The Zot function in your last comment is not implemented correctly.
You should fix it here, instead of checking if an error value encapsulates a nil pointer later.
You should correct the source, instead of remedying the result.

@go101
Sorry for interfering, but "go fix the source" may be rather hard in real production code. The pointer in question may be passing dozens of stack frames and quite few modules on its way. It may be inserted into some map and checked for nil significantly later. And Shroedinger conversion may happen anywhere on that path.

@go101 You'll have to forgive me if that kind of argument makes me a bit angry; my main job involves writing in C, which is absolutely strewn with hidden landmines of undefined behavior that can make your perfectly normal-looking program suddenly full of security holes; and when you go to the compiler people and tell them that obvious-looking code results in bizarre behavior, they tell you you're just using the language wrong, you should fix your program.
One of the things that made me so happy when I first learned Go was that there was no undefined behavior.  All values are initialized by default; no conversions even from int32 to int64 without an explicit cast.  The compiler even prevents you from using goto over a variable declaration, just in case something should accidentally end up uninitialized.
Go's behavior with regard to nil pointers and interfaces is a giant black mark on an otherwise very nice language.  It's not only dangerous and unnecessary: it's completely out of character.

There are really some interface related confusions when I started learning Go. However, when I realized that an interface value is just a box to encapsulate non-interface values which satisfy some requirements (have all the methods the interface specifies), I never get confused again.
There are two categories of interfaces, blank ones and non-blank ones. A blank interface doesn't specify any methods, they are mainly used for reflection. A non-blank interface specifies several methods, they are mainly used for polymorphism.

Go's behavior with regard to nil pointers and interfaces is a giant black mark on an otherwise very nice language. It's not only dangerous and unnecessary: it's completely out of character.

I feel your frustration comes from some bad practices in using interfaces. If you want to return a nil pointer as a non-blank interface value (aka, encapsulate a nil value in an interface value), you should expect users of the return interface value will call the method specified by the interface and declared for the pointer type, then you should guarantee that calling the method on the nil pointer will not panic. This is your responsibility. Otherwise, just don't encapsulate the nil pointer in an interface value. I think the principle is simple and clear.
BTW, Go does have some undefined behaviors, though much less than C.
And I don't very understand your proposal.

Change existing behavior such that implicit conversion of typed nil to interface produces nil interface pointer. Allow create "nil receiver interface" pointer using explicit syntax like nil.(MyError*).(error). This would make most expected behavior the default one, with fallback available.

Do you mean `nil.(*MyError).(error)? Could you show an example how it is used?
The designs of main Go design elements are very consistent, though there are many inconsistencies in detailed designs and concrete implementations. Nil values are just the zero values of some kinds of types. Except this, they are not special comparing to other non-zero values, so Go doesn't treat them specially.

If you want to return a nil pointer as a non-blank interface value (aka, encapsulate a nil value in an interface value), you should expect users of the return interface value will call the method specified by the interface and declared for the pointer type, then you should guarantee that calling the method on the nil pointer will not panic.

You're effectively saying "expect absolutely any method to be called with nil receiver through typed nil". My point is that interface pointer with valid vptr and nil dptr can be created through implicit cast. Again, I don't see "just write code better" as an argument. Compiler should help us, not force us to decrypt some riddles.

Honestly, I still not very understand your proposal. I think I can understand it better if you can show an example how to use the nil.(*MyError).(error) syntax.

Compiler should help us, not force us to decrypt some riddles.

Sometimes, encapsulating a nil pointer in an interface value is desired, sometimes it is not. How can compilers distinguish them?

Honestly, I still don't very understand your proposal. I think I can understand it better if you can show an example how to use the nil.(*MyError).(error) syntax.

Example is very primitive but should illustrate idea
import "io"

type Null interface{}

func (self *Null) Read(p []byte) (n int, err error) {
    return 0, io.EOF
}

func self(*Null) Write(p []byte) (n int, err error) {
    return 0, io.EOF
}

func NewNull() io.ReadWriter {
    return nil.(*Null)(io.ReadWriter)
}

Sometimes, encapsulating a nil pointer in an interface value is desired, sometimes it is not. How can compilers distinguish them?

Cannot disagree with that. My point is, compilers should catch unsound scenarios. And implicit half-nil interface is a soundness hole to me.

The difficulty I see with this proposal is converting back and forth. For instance:
var x *int
var y interface{} = x
z := y.(*int)

Should the third line fail?  What about
var x *int
var y interface{} = x
z := y.(*float64)

Currently the first example passes and the second one fails.  If the conversion of a nil pointer to an interface makes a full nil interface, then we can't distinguish these two cases - either they both have to fail (a nil interface is convertible to no type, as it is now) or they both have to succeed (a nil interface is convertible to the nil value of every pointer type?).
Either way, I don't like the consequences. The first example seems like the identity operation that should always work, and the second example seems like it could hide type errors (although it wouldn't actually break the type system, because type fluidity only works for nil).
So although I agree your proposal solves the problem you're investigating, it's not without its drawbacks in other areas.

The difficulty I see with this proposal is converting back and forth. For instance:
var x *int
var y interface{} = x
z := y.(*int)

Should the third line fail?

Yes, it should fail. x is a nil. When we upcast it to interface{}, we lose all static type information, i.e. type erasure happens here.
If you want current behavior:
var x *int
var y interface{} = x.(*int).(interface{}) // or some other syntax
z := y.(*int)

What about
var x *int
var y interface{} = x
z := y.(*float64)

Fails as well.

Currently the first example passes and the second one fails. If the conversion of a nil pointer to an interface makes a full nil interface, then we can't distinguish these two cases - either they both have to fail (a nil interface is convertible to no type, as it is now) or they both have to succeed (a nil interface is convertible to the nil value of every pointer type?).
Either way, I don't like the consequences. The first example seems like the identity operation that should always work, and the second example seems like it could hide type errors (although it wouldn't actually break the type system, because type fluidity only works for nil).
So although I agree your proposal solves the problem you're investigating, it's not without its drawbacks in other areas.

Whether I'm right or you is highly dependent on one simple question. What is runtime typed nil in Golang? I mean semantically. What meaning does it carry?
I'll expand a bit on my understanding though will try to keep it short.
First, what is a pointer? It's a special type which holds location of an object of some connected type. And it has special value nil which means "does not point to anything". Although pointer has yet another property called type - which describes what kind of object may reside on the other end - or may not.
And here we have first question. What's the value on the other side when pointer is nil? The answer in most cases is "there's no value". This value is just like any other to pointer itself. But not to operations you perform over that pointer. Specifically, dereference. To me it behaves very similar to division by zero. You can have zero integer, pass it around and do many other things. But you cannot divide by it.
Second question which arises here. What's the type of variable n in n := nil? Go says it's "untyped nil" and cannot be used. It's a special default value literal for any pointer type. So far so good.
Then we come to such concept as interface. Unlike OOP languages, Go's interface is more like trait, a concept more common in FP world. We use interfaces in Go through interface pointers. Which are not the same as normal pointers. In fact, they consist of two pointers, one points to actual object and the other to method table necessary for interface to work properly. Due to Go's runtime, second pointer can be also treated as type information.
If we treat this "fat pointer" as some ordinary type, we can have this rough table of its possible states

Data
Type

nil
nil

nil
non-nil

non-nil
nil

non-nil
non-nil

First and last cases are pretty much obvious. They represent normal nil and normal interface pointer to some object.
Third case doesn't make much sense because it's a data pointer without type pointer. It cannot be created in normal Go unless one uses reflection or unsafe. It's unreachable, so let's leave it as-is.
But there's the last possible state which is nil data pointer with some type information. Remember we concluded nils are typed in Go? So far so good - except nil check tells it's not nil! Moving type information from compile time (concrete object pointer) to runtime (interface pointer) changes whole meaning of the pointer! Which to me is a huge deficiency in type system. Imagine you move some statically sized empty array to slice - and it suddenly starts to return non-zero length!
The problem is, we already have nil receiver interfaces, though used not so often. How are they used and why do they exist? I think they exist to have cheap stateless interfaces like the one I showed above. And they shouldn't be treated as nil.
Who's right is highly dependent here on the semantical meaning of runtime typed nil (RTN) in Go. Based on the answer we have several ways to resolve this strange issue - except leave it as-is of course.
If RTN is just an artifact of cheap stateless interfaces. It would mean that nil is just default value of pointer and does not carry type on its own - but pointer does. Consequently it would mean that nil receiver interfaces are also prohibited state, like nil vtable. And if one wants stateless interfaces, he can use hidden static object.
If RTN is an important part of type system, we must preserve it. But we must also properly express it as nil. So checks against nil receiver will result in "it's empty", with ability to check for its type. Then cheap stateless interfaces must go the same way as above.
A third way is the one described in this proposal. There may be more.

But there's the last possible state which is nil data pointer with some type information. Remember we concluded nils are typed in Go? So far so good - except nil check tells it's not nil!

I think this is the fundamental issue here. nil (and consequently, ==nil) means different things for pointers and interfaces.  You're using ==nil on an interface and expecting the behavior of ==nil on the underlying pointers.  There's good discussion and even a proposal to distinguish the different nils at #22729 (and I would prefer that proposal to what is proposed here).  But the confusion of having syntactically similar constructs behave differently happens in many other places in Go. For instance, +1 means different things for integers and floats. [1] means different things for slices and maps. So ==nil means different things for pointers and interfaces.
I understand that there's confusion here, and that confusion can lead to bugs. But I don't think this proposal is the solution.
Interfaces with nil pointers in them aren't an aberration; they can happen all the time. For example:
package main

import (
	"container/list"
	"fmt"
)

var x, y int

func main() {
	a := []*int{&x, nil, &y}

	// encode into a list
	var l list.List
	for _, p := range a {
		l.PushBack(p)
	}

	// decode from list
	b := []*int{}
	for e := l.Front(); e != nil; e = e.Next() {
		b = append(b, e.Value.(*int))
	}

	fmt.Println(a)
	fmt.Println(b)
}

I'm just using interfaces as a container, and I expect to get back the same thing I put in. I'm not calling methods on it or anything, an interface{} is just a runtime-typed wrapper. Should I have to use l.PushBack(p.(*int).(interface{}) instead? Why do I have to do that for nils but not, say, ints?

@target-san
You says it is this proposal is for compiler, but is it also for runtime?
var x *int
if (aRuntimeCondition) {
	x = new(int)
}
// At compile, we don't know whether x is nil.
var y interface{} = x.(*int).(interface{}) // or some other syntax
z := y.(*int)

BTW, you can't declare methods for *Null, for its base type is an interface type.

@randall77

I think this is the fundamental issue here. nil (and consequently, ==nil) means different things for pointers and interfaces. You're using ==nil on an interface and expecting the behavior of ==nil on the underlying pointers.

Well, kind of yes. Going with this logic, interfaces shouldn't support nil as their value at all. As I described earlier, we in fact have compound entity which tries to mimic itself as ordinary pointer.

There's good discussion and even a proposal to distinguish the different nils at #22729 (and I would prefer that proposal to what is proposed here).

The proposal you mentioned has significant drawback, at least to my sight. It introduces a lot of complexity and different kinds of nils. While most of pointer types mentioned - maps, slices, funcs etc. - don't have such complex nature. They AFAIK either point to some object - or don't point. Their nils are statically typed. The only "black sheep" is interface pointer.
Another issue here would arise when/if generics land on table. We will have to introduce "generic nil" to be able to handle something like generic range loop over some collection of interface pointers.

I understand that there's confusion here, and that confusion can lead to bugs. But I don't think this proposal is the solution.
Interfaces with nil pointers in them aren't an aberration; they can happen all the time.

The example presented shows that runtime typed nils are significant part of type system. I have absolutely no objections against this. Let's just decide - is it nil? Or not a nil? Or should interfaces support nil comparison at all?
My short conclusion. There's a loophole in Go's type system because of attempt to "sit on two chairs", i.e. treat interface as both simple pointer and aggregate data structure at once. I'm leaning towards treating it like simple pointer and thus deciding whether it's nil based only on its data pointer.
The proposal itself happened because of certain doubts that Go core team would change behavior so significantly.
@go101

You says this proposal is for compiler, but is it also for runtime?

I think yes. Though I'm in doubts now due to Keith's feedback on typed nils whether this whole approach is right.

BTW, you can't declare methods for *Null, for its base type is an interface type.

My bad.

If it is also for runtime, then the syntax is not much useful, for it can't prevent nil interfaces with concrete nil values from happening at run time.
BTW, I think using the terminology  typed nil to represent non-interface nil is confusion. An interface nil is also a typed nil, its type is an interface type. Maybe, concrete nil is a better terminology for your typed nil.

There's a loophole in Go's type system because of attempt to "sit on two chairs", i.e. treat interface as both simple pointer and aggregate data structure at once.

I want to be clear that in my opinion there is no loophole in Go's type system.  I find that to be a perplexing statement.
What is happening here is that the builtin identifier nil is untyped and overloaded.  Just as you can compare an int, int8, float64, etc., value to 0, you can compare a pointer, channel, map, slice, interface value to nil.  The untyped 0 or nil takes on the type of the value on the other side of the comparison.  That is not a loophole.  It's the nature of how untyped constants work in Go.
People new to the language find it confusing that a pointer that is equal to nil can be stored in an interface variable and that the interface variable is then not equal to nil.  I agree that that is confusing and it would be nice if there were something we could do to fix it.  But it's not a loophole.

People new to the language find it confusing that a pointer that is equal to nil can be stored in an interface variable and that the interface variable is then not equal to nil.

The confusion here stems from the nil pointer suddenly becoming non-nil through implicit cast. That's all. Again, what would you suggest to resolve issue mentioned in opening post and this one by @gwd , taking into account this issue is long-range and cannot be always resolved by just "not doing that"?

If you know the pointer type, assume its is *T, just use type assertion:
if p, _ := display.(*T); p == nil {
  // Render not-found template
} else {
  // Pass 'display' to the appropriate template for rendering
}

And if I don't know it? Or it causes abstraction leak? Or the set of possible types is open?

If you don't know it, then you can only use its methods (through the interface proxy),
then please ensure calling the methods will not panic, even if their receivers are nil pointers.

The confusion here stems from the nil pointer suddenly becoming non-nil through implicit cast.

We are talking about language semantics here, so I think it's important to be pedantic about what we are saying.  A "nil pointer" is a value of pointer type that is equal to nil.  In Go, any type can be implicitly converted to an interface type.  When a nil pointer is implicitly converted to an interface type, it does not "become non-nil".  It is still a nil pointer.  The confusion that some people encounter is that the value of interface type, although it is equal to a nil pointer of the original pointer type, is not equal to nil.  That is because nil is untyped and overloaded.

Again, what would you suggest to resolve issue mentioned in opening post and this one by @gwd , taking into account this issue is long-range and cannot be always resolved by just "not doing that"?

First, let me say that this since this is indeed a long-standing issue that we should take the time to find a good solution.  We are not in a hurry here.
Second, I'm already on record as proposing #22729.  I don't think that is the answer, but I'm hopeful that it may gesture toward the answer.

When a nil pointer is implicitly converted to an interface type, it does not "become non-nil".

Effectively it becomes :D. Because it no longes compares equal to nil.

First, let me say that this since this is indeed a long-standing issue that we should take the time to find a good solution. We are not in a hurry here.

You may consider making pointers non-nilable, then add "optional value" concept. Ta-da! No need to resolve half-nil values, as nil will always be "outside" the value :D. Mostly joking actually.

Second, I'm already on record as proposing #22729. I don't think that is the answer, but I'm hopeful that it may gesture toward the answer.

Read through the thread more carefully than the first time. I must say I'm more sided with Dave and the others who suggest making nil-comparison independent of runtime type pointer stored in interface. I see that these "runtime typed nils" are already a thing.
Anyway, my proposal was aimed to be "least change" by fixing only ambiguous conversion and leaving everything else as-is.

My apologies, but, again, I think it's necessary to be pedantic about language issues.

Effectively it becomes :D. Because it no longes compares equal to nil.

The pointer remains equal to nil.  It has been converted to a value of interface type that is not equal to nil.  It is not accurate to say that the pointer has become non-nil.  The pointer is still nil.

Anyway, my proposal was aimed to be "least change" by fixing only ambiguous conversion and leaving everything else as-is.

The conversion is not ambiguous.  It is precisely defined.  It is nil itself that could be described as ambiguous, because it is untyped.

The conversion is not ambiguous. It is precisely defined.

I think perhaps @target-san means "confusing" rather than "ambiguous".  What happens when you assign a variable with an interface type to a pointer is, as you say, unambiguous and well-defined; but the result is still rather surprising, even to people who theoretically understand what's going on.

"Confusing" comes from not well understanding what are interface values.
An interface value is just a box used to encapsulate non-interface values.
An interface value boxing nothing is called a nil interface value.
An interface value boxing any value, including nil pointer, does box something, it doesn't box nothing.
An interface value boxing something is absolutely not equal to an interface value boxing nothing.
I feel your confusion comes from not familiar with how type deduction works in Go.
When you comparing a value x with an untyped (aka, bare) nil, the nil will be deduced as the type of x, whether or not x is an interface value. That means:

if x is an interface value, the bare nil will also be deduced as an interface value. A nil interface value boxes nothing. If x boxes a nil pointer, then it doesn't nothing, so x is not equal to the bare nil.
if x is an non-interface value, assume its type is T, then the bare nil will also be deduced as a T value.

The type deduction rules are straightforward and intuitive.

@go101

"Confusing" comes from not well understanding what are interface values.

Thank you very much but I'm well aware what are "fat pointers" and how they are usually implemented. As well as usual vtable implementations in OOP languages. And one of sources of my confusion comes from the fact that in other languages with fat pointers both pieces of such "value" are kept coherent. They're either both null or both point to something. But Go has runtime-typed nils, and here's where things get complicated.

@go101 We have very different attitudes on this.
In my ergonomics class, they said that it's your job as a designer to design things that are easy to use; and that 95% of the time when people misuse your product, it's due to poor design.
In my documentation class, they said that it's your job as a document writer to make the documentation understandable; 95% of the time when people are confused by the documentation it's due to poor design.
In my music conducting class, they said that it's your job as a conductor to communicate when the orchestra should start; 95% of the time when your group doesn't come in together, it's because you as a conductor sent mixed signals.  (And he videoed everyone conducting, so that he could prove that it was our fault.)
I could go on and on -- I see this situation in parenting, in economics, in leadership, in school -- situations where people do the "wrong thing" because they were prompted to by the circumstances around them, and then blamed for doing the natural, obvious thing.
Sometimes there's no way to change the design, and education / exhortation to do better is the only answer.  But our first resort, when we find this sort of "anti-pattern" happening, should be to see if there's a way we can re-design the system to make it better.
var x InterfaceType
var y *InterfaceInstance
y = nil
x = y
if x == nil {
  // Not executed
} else {
 // Executed
}

The code above is confusing. It is not natural or normal.  It can be comprehended, but not without extra effort, and it's prone to error.  Rather than spend so much effort trying to make people understand it, and dealing with the consequences of making mistakes, wouldn't it be better to redesign the language so that it doesn't point you in the wrong direction?  Or at least, so that the compiler would help you avoid this sort of situation?
I mean, if "just be more careful" were an effective strategy, why bother using a typed language like Go at all?

It is true that, many new Go programmers, including me when starting learning Go, with experiences of some other popular languages may view nil as the counterpart of null (or NULL) in other languages. It is wrong.
In Go, nil is not a single value, it is just a symbol to represent many many values.
Go is Go, other languages are other languages. Why do you have to let Go become other languages?

But Go has runtime-typed nils, and here's where things get complicated.

No, for every nil, whether it is typed or untyped can be determined at compile time.
@gwd
If you don't plan to adapt the rules of Go, and have to use Go as other languages. I think that is your problem, not of Go.

@gwd
You last example is wrongly written. It should be written as the following to satisfy your expectation.
var x InterfaceType
var y *InterfaceInstance
y = nil
x = y
if x == (*InterfaceInstance)(nil) {
  // Not executed
} else {
 // Executed
}

otherwise, it is equivalent to
var x InterfaceType
var y *InterfaceInstance
y = nil
x = y
if x == InterfaceType(nil) {
  // Not executed
} else {
 // Executed
}

This is the last attempt to make an explanation in this thread.

@go101
We're perfectly aware how it works and why. What we're trying to say is that current behavior is counter-intuitive and error-prone.
 Was it logical if you know the rules? Yes. Was it useful and error-proof? Definitely no.

@go101 You keep describing how Go currently works; we're talking about how humans respond to it, and how we can change Go to make them respond better.  If I didn't know how Go actually works, then I wouldn't have been able to write that example.  If you don't care about making Go more intuitive and safe to use, that's fine; but a lot of people do.

Thanks for the suggestion.  I don't think we've seen this one before.
A significant drawback of this proposal is that it would subtly change the behavior of existing code.  The same Go code would behave differently in different versions of Go.  This goes against the plan described in #28221 for language changes.  We can add language features, and we can (where necessary) remove language features, but we want to avoid changing language features such that some code is valid in two different language versions but has different behavior.

@ianlancetaylor Well, that's a very, very strong argument against this proposal which I overlooked. I agree that "behaves exactly the same or does not compile at all" is a reasonably good way to introduce changes. I then leave it up to your decision whether to leave this proposal open or close it. Hope you will find good enough way to resolve this issue. And thanks for your patience and overall efforts on Go lang.

OK, closing this specific proposal.  Perhaps there is something else in this area that could work.


ACM-10734: Remove obsolete hypershift related image references

Description
With https://issues.redhat.com/browse/HYPBLD-249, the following MCE container images are no longer built and packaged in MCE. This PR is to remove references to those images from the hypershift addon installation chart.
Related Issue
https://issues.redhat.com/browse/HYPBLD-249
Changes Made
With https://issues.redhat.com/browse/HYPBLD-249, the following MCE container images are no longer built and packaged in MCE. This PR is to remove references to those images from the hypershift addon installation char
Screenshots (if applicable)
Add screenshots or GIFs that demonstrate the changes visually, if relevant.
Checklist

[ ] I have tested the changes locally and they are functioning as expected.
[ ] I have updated the documentation (if necessary) to reflect the changes.
[ ] I have added/updated relevant unit tests (if applicable).
[ ] I have ensured that my code follows the project's coding standards.
[ ] I have checked for any potential security issues and addressed them.
[ ] I have added necessary comments to the code, especially in complex or unclear sections.
[ ] I have rebased my branch on top of the latest main/master branch.

Additional Notes
Add any additional notes, context, or information that might be helpful for reviewers.
Reviewers
Tag the appropriate reviewers who should review this pull request. To add reviewers, please add the following line: /cc @reviewer1 @reviewer2
Definition of Done

[ ] Code is reviewed.
[ ] Code is tested.
[ ] Documentation is updated.
[ ] All checks and tests pass.
[ ] Approved by at least one reviewer.
[ ] Merged into the main/master branch.

/lgtm


[BUG] grafics

Acknowledgement of preliminary instructions

[X] I have read the preliminary instructions, and I am certain that my problem has not already been addressed.

What problem did you encounter?
if u put the meniu from 2020 and acctivate the 1-21 grafics ur not going to get 1-21 only 1-10

what do you expect from using an abandoned menu

grafics

they fixed it w


Azure SQL LTR Logs

It there any way to pull some Logs from the Azure SQL backups? PITR or LTR.
Kind Regards
Joan

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9492cedc-cea0-af9d-29a0-52cbca8c5e21
Version Independent ID: 969a8654-8f95-d5de-351d-73c8e02e5d3a
Content: Azure SQL Database automatic, geo-redundant backups
Content Source: articles/sql-database/sql-database-automated-backups.md
Service: sql-database
Sub-service: backup-restore
GitHub Login: @anosov1960
Microsoft Alias: sashan

@joanfs Thanks for your feedback. Could you please specify what exactly you need.
As per my understanding you want to know the list of backups that can be achieved using :
Get the list of all LTR backups in a specific Azure region
The backups are grouped by the logical database id.
Within each group they are ordered by the timestamp, the earliest
backup first.
$ltrBackups = Get-AzSqlDatabaseLongTermRetentionBackup -Location $server.Location
Get the list of LTR backups from the Azure region under
the named server.
$ltrBackups = Get-AzSqlDatabaseLongTermRetentionBackup -Location $server.Location -ServerName $serverName
Get the LTR backups for a specific database from the Azure region under the named server
$ltrBackups = Get-AzSqlDatabaseLongTermRetentionBackup -Location $server.Location -ServerName $serverName -DatabaseName $dbName
List LTR backups only from live databases (you have option to choose All/Live/Deleted)
$ltrBackups = Get-AzSqlDatabaseLongTermRetentionBackup -Location $server.Location -DatabaseState Live
Only list the latest LTR backup for each database
$ltrBackups = Get-AzSqlDatabaseLongTermRetentionBackup -Location $server.Location -ServerName $serverName -OnlyLatestPerDatabase
Hope this helps. Please let me know if you need some other information.

@joanfs We are going to close this request right now. Hope the above information has helped. Please get back to us if you need any further help.


Stop queueing workers exponentially when bulk updating attachments

This PR moves the call to attachment_updater (which queues up a worker for each attachment in a document) outside of the loop which processes each attachment.
This should avoid exponential queues building up, but still mean that every attachment in the document gets processed by a worker.

Trello: https://trello.com/c/EDPncQak/435-remediation-following-whitehall-incident
⚠ This repo is Continuously Deployed: make sure you follow the guidance ⚠

@kevindew

Scope creeping
I wonder if we can do something like:
      attachment.assign_attributes(attributes.permit(:title))
      needs_update = attachment.attachment_data.changed?
      if attachment.save(context: :user_input)
         ServiceListeners::AttachmentUpdater.call(attachment_data: attachment.attachment_data) if needs_update

and then it'd only update the items that need a change (which is likely zero in most cases), just not sure I'm missing an edge case with that.

My concern with this approach is that it'd introduce coupling between this controller and the AssetManagerAttachmentMetadataWorker worker. I don't think it's this controller's responsibility to pre-empt what that worker does, or how it decides if/when an AttachmentData needs updating.
For the sake of keeping this controller simple, I think it's probably a fair price to pay that every attachment gets a worker. In extreme scenarios, this means that we could be adding a few hundred redundant workers to the queue – but in most cases, that number will be far less.
Whilst it's a bit of a smell, I'd say it's probably not worth over-optimising this path. I expect we'd benefit more from a wider refactor of the way these various updaters and workers interact with assets as a whole.

@kevindew

Scope creeping
I wonder if we can do something like:
      attachment.assign_attributes(attributes.permit(:title))
      needs_update = attachment.attachment_data.changed?
      if attachment.save(context: :user_input)
         ServiceListeners::AttachmentUpdater.call(attachment_data: attachment.attachment_data) if needs_update

and then it'd only update the items that need a change (which is likely zero in most cases), just not sure I'm missing an edge case with that.

My concern with this approach is that it'd introduce coupling between this controller and the AssetManagerAttachmentMetadataWorker worker. I don't think it's this controller's responsibility to pre-empt what that worker does, or how it decides if/when an AttachmentData needs updating.
For the sake of keeping this controller simple, I think it's probably a fair price to pay that every attachment gets a worker. In extreme scenarios, this means that we could be adding a few hundred redundant workers to the queue – but in most cases, that number will be far less.
Whilst it's a bit of a smell, I'd say it's probably not worth over-optimising this path. I expect we'd benefit more from a wider refactor of the way these various updaters and workers interact with assets as a whole.

Can understand your concern but I think we're going to have to go further than that to resolve the incident that occurred.
To cause the incident the user did ~400 delete requests, not a single update many, and as each one of those triggers ~400 workers you end up with ~120,000 jobs queued up. So I think rather than being a smell it's actually the cause of the incident.
This stuff with the update_many action looks worse because in theory one update many of a 400 attachment doc would also cause ~120,000 items to be queued. However in practice I think update_many is probably only used for re-ordering and is rarely used (it'd be of no value to the documents in the incident since they manually reference attachments in copy) so while a bigger concern, not necessarily something that happens in practice.
I wouldn't say that we create a coupling to AssetManagerAttachmentMetadataWorker with the change I suggested as it would still go through the ServiceListeners::AttachmentUpdater.call interface.
As an aside though I don't really think the ServiceListeners::AttachmentUpdater class is of much value. It's not really a listener that goes off and checks whether work needs to be done, it seems to be more of just a worker spam cannon - as it doesn't seem to have insight of the quantity of work that'd need to be done so just does everything.

@ollietreend Is it possible to add a test for this to be sure? i.e. If I have 5 attachments and I delete 4 of them I expect AssetManagerAttachmentMetadataWorker to only be called 4 times etc?


Weekly Package Upgrade

:tada: This PR is included in version 4.0.20 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Need Indic scripts experts to review cleanup code

If you can help for a particular script, please comment below.
Comments from Ray - copied from https://github.com/tesseract-ocr/tesseract/issues/995
Read the thread for full context.

it would be useful to have any experts in any of the following scripts
review the new corpus cleanup code,and make comments:
Devanagari, Bengali, Gurmukhi, Gujarati, Oriya, Tamil, Telugu, Kannada,Malayalam,
Sinhala,
Thai,
Myanmar,
Khmer.
There are script-specific cleanup rules in there.
Since I plan to commit new copies of the training data (unicharsets,
wordlists, training text etc) then at that point they will match.
eg.
The code determines what makes a valid/invalid sequence of unicodes in the
script, for instance, is it allowed to have two matras in a row? It gets
more difficult with questions over what category the additional characters
are.
Major new normalization/text cleanup code in training/validat* The best
help with this would be expertise in the various scripts, as previously
discussed.

https://github.com/tesseract-ocr/tesseract/blob/master/training/validate_grapheme.cpp
https://github.com/tesseract-ocr/tesseract/blob/master/training/validate_grapheme.h
https://github.com/tesseract-ocr/tesseract/blob/master/training/validate_indic.cpp
https://github.com/tesseract-ocr/tesseract/blob/master/training/validate_indic.h
https://github.com/tesseract-ocr/tesseract/blob/master/training/validate_khmer.cpp
https://github.com/tesseract-ocr/tesseract/blob/master/training/validate_khmer.h
https://github.com/tesseract-ocr/tesseract/blob/master/training/validate_myanmar.cpp
https://github.com/tesseract-ocr/tesseract/blob/master/training/validate_myanmar.h
https://github.com/tesseract-ocr/tesseract/blob/master/training/validator.cpp
https://github.com/tesseract-ocr/tesseract/blob/master/training/validator.h

Devanagari - Vedic Accents
https://github.com/tesseract-ocr/tesseract/blob/master/training/validator.cpp#L178
bool Validator::IsVedicAccent(char32 unicode) {
  return 0x1cd0 <= unicode && unicode < 0x1d00;
}

Please see

http://www.unicode.org/versions/Unicode10.0.0/ch12.pdf
http://unicode.org/charts/PDF/U0900.pdf
http://unicode.org/charts/PDF/UA8E0.pdf
http://unicode.org/charts/PDF/U1CD0.pdf

The following should also be included as Vedic Accents.

U+0951..U+0954 are a set of combining marks used in transcription of Sanskrit texts.

Vedic tone marks
0951 $॑ DEVANAGARI STRESS SIGN UDATTA
= Vedic tone svarita
• mostly used for svarita, with rare use for udatta
• used also in Vedic texts written in other scripts
→ 1CDA $᳚  vedic tone double svarita
0952 $॒ DEVANAGARI STRESS SIGN ANUDATTA
= Vedic tone anudatta
• used also in Vedic texts written in other scripts
→ 1CDC $᳜  vedic tone kathaka anudatta

Possibly also
Accent marks
0953 $॓ DEVANAGARI GRAVE ACCENT
→ 0300 $̀  combining grave accent
0954 $॔ DEVANAGARI ACUTE ACCENT
→ 0301 $́  combining acute accent

Devanagari Extended: U+A8E0–U+A8FF
This block of characters is used chiefly for Vedic Sanskrit, although many of the characters
are generic and can be used by other Indic scripts. The block includes a set of combining
digits, letters, and avagraha which is used as a system of cantillation marks in the early
Vedic Sanskrit texts. The Devanagari Extended block also includes nasalization marks (candrabindu),
and a number of editorial marks.

Also include the ranges

A8E0-A8F1 Combining Marks
A8F2-A8F7 Marks of Nasalization

Devanagari - Words cannot begin with
Various signs
0900 $ऀ DEVANAGARI SIGN INVERTED CANDRABINDU
= vaidika adhomukha candrabindu
0901 $ँ DEVANAGARI SIGN CANDRABINDU
= anunasika
→ 0310 $̐  combining candrabindu
0902 $ं DEVANAGARI SIGN ANUSVARA
= bindu
0903 $ः DEVANAGARI SIGN VISARGA

Various signs
093C $़ DEVANAGARI SIGN NUKTA
• for extending the alphabet to new letters
093D ऽ DEVANAGARI SIGN AVAGRAHA

and
the various dependent vowel signs.

Devanagari - Eyelash Ra for Marathi
R5a For compatibility with The Unicode Standard, Version 2.0, if the dead consonant
RAd precedes zero width joiner, then the half-consonant form RAh , depicted as
eyelash-RA, is used instead of RAsup .

Page 13 in http://www.unicode.org/versions/Unicode10.0.0/ch12.pdf
Removal of ZWJ in this case will lead to incorrect results.

@zdenop Please label as help wanted.

@zdenop I can support Kannada language. Please task me anything.
I am Kannada native and I have full knowledge in Unicode for Kannada script.

so have a look at above from @Shreeshrii

@Shreeshrii @zdenop I am writing couple of cases for Kannada. Please review it, if this is what you are looking for, I will compile the full list.

Ra (U+0CB0) with halant/virama U+0CCD from a repha in unicode. However, in Kannada script, repha is formation as particular rules.
a. Repha can't formed when Ra+halant is at the beginning of the word (or in other words when it is not preceded by another Kannada letter)
example: ರ‍್ಯಂಕ್
b. Repha is not formed at the end of the word or with just Ra+halant.
example: ಆರ್ , ರ್
c. Repha is not formed when it is followed by Ra
example: ರ‍್ರ + any vowel sign
d. User can avoid the repha form
example:  ಸೂರ್ಯ ಸೂರ‍್ಯ
All printed Kannada text follows the above rules.

In Unicode:
a. This is achieved by adding ZWJ (U+200D) between Ra and halant to stop the repha formation.
b. This is achieved by shaping engine, no special characters added. Ra+halant
c. This is achieved by the font, no special characters added. Ra+halant+Ra + any vowel sign
d. This is achieved by adding ZWJ (U+200D) between Ra and halant to stop the repha formation.
For the OCR, to bridge the printed text to Unicode correctly.
a. When the text has Ra with a subscript/consonant conjugate with it in printed text. The output shall be Ra + ZWJ + halant + any consonant but Ra.
b. Ra+ halant, no special characters to be inserted.
c. When the text has Ra with Ra as subscript/consonant conjugate with it in printed text. The output shall be Ra + halant + Ra + any vowel sign
d. Same as above case "a". Ra + ZWJ + halant + any consonant but Ra.

Usage of Zero width non-joiner, ZWNJ (U+200C)
Text: ರಾಜ್‌ಕುಮಾರ್
Unicode:  U+0CB0 U+0CBE U+0C9C U+0CCD U+200C U+0C95 U+0CC1 U+0CAE U+0CBE U+0CB0 U+0CCD
Exception: When "any Kannada consonant" "halant (U+0CCD)" "any Kannada consonant", to avoid subscript/consonant conjugate, ZWNJ U+200C is added after the halant (U+0CCD)
If ZWNJ is not present, the above text will be ರಾಜ್ಕುಮಾರ್.

Nukta U+0CBC

Text:
ಜಫ಼ಾರ್
ಜಫ಼್ಲರ್
ಜಫ್ಫ಼ರ್
ಜ಼ಾರ್
ಜ್ಜ಼ಾರ್
Image:

Nukta can be with base letter or the subscript.
In unicode: Nukta is always placed next to the consonant before the vowel matra or halant. When the nukta is in subscript, it is placed after the consonant in the subscript.

@MayuraVerma Thanks for your detailed input.
Please see the rules at https://github.com/tesseract-ocr/tesseract/blob/7cc97c25ca32a9e8e7e991587064abae51b22f65/src/training/validate_indic.cpp#L96
and check whether they are OK for Kannada.

@Shreeshrii @zdenop Please update all outdated links in this issue.


fixes #17 Incorrect platform compatibility check

Thank you for integrating the feedback into this PR! Just as a formality before we merge this: how did you test this change?
Is it too much work for you to test that both with the latest mdb_v8 version and older versions (like version 1.0.0 that you can download using curl -O https://us-east.manta.joyent.com/Joyent_Dev/public/mdb_v8/v1.0.0/mdb_v8_ia32.so)?
If it's too much work, that's fine, I think that this change is fairly safe and I don't really have the time to do that testing myself.
Thanks again!

It's not a problem.  I'll test with 1.0.0 when I have some cycles and let you know.  I tested with 1.3.0 by simply copying mdb.js into /opt/local/lib/node_modules/node-debug-school/lib/mdb/mdb.js, and running node-debug-school as root.
[root@10df76c6-720e-6ab5-d94b-b0a0ca0a1b1b /home/rui]# uname -a
SunOS 10df76c6-720e-6ab5-d94b-b0a0ca0a1b1b 5.11 joyent_20180810T005956Z i86pc i386 i86pc Solaris

For testing mdb_v8 v1.0.0 I:

Copied over the both the amd64 and the ia32 v1.0.0 modules from https://us-east.manta.joyent.com/Joyent_Dev/public/mdb_v8/v1.0.0/ to /root and /root/amd64 respectively.
temporarily modified the mdb v8 load command in the source: https://github.com/joyent/node-debug-school/blob/master/lib/platform/platform.js#L34 to: `var mdbLoadV8Command = 'echo "::load v8" | mdb -L /root ' + coreFilePath;
Ran node-debug-school and confirmed the platform check passed and the main menu was displayed.

Additional details:
[root@10df76c6-720e-6ab5-d94b-b0a0ca0a1b1b ~]# echo "::load v8" | mdb -L /root cores/core.199704
mdb_v8 version: 1.0.0 (release, from c95ca32)
V8 version: <IP_ADDRESS>
Autoconfigured V8 support from target
C++ symbol demangling enabled
[root@10df76c6-720e-6ab5-d94b-b0a0ca0a1b1b ~]# file v8.so
v8.so:          ELF 32-bit LSB dynamic lib 80386 Version 1, dynamically linked, not stripped, no debugging information available
[root@10df76c6-720e-6ab5-d94b-b0a0ca0a1b1b ~]# file amd64/v8.so
amd64/v8.so:    ELF 64-bit LSB dynamic lib AMD64 Version 1, dynamically linked, not stripped, no debugging information available
[root@10df76c6-720e-6ab5-d94b-b0a0ca0a1b1b ~]#


Bokeh colormapping fixes

Two colormapping fixes the first fixes a bug in assigning colors to Bars in the right order, the second allows passing a list of colors to bokeh instead of a matplotlib colormap which should always have been allowed.

This PR now adds support for supplying explicit lists of colors as cmaps and selecting some of the inbuilt bokeh palettes as strings. In doing is I discovered that using bokeh palettes is not sufficient, they contain at most 11 colors and do not interpolate between the colors which is not very nice for continuous values. This means for the time being we still rely on the colormaps shipped with matplotlib. I see a number of solutions:

Add code to interpolate between colors and generate larger colormaps on the fly, the base colors for each colormap would still have to come from somewhere though.
Require colorcet and only offer the colormaps it includes unless matplotlib is installed.
Ship a few colormaps ourselves.

Ready to merge? I would consider putting your question about colormaps into an issue and assigning it to the 1.8 milestone.

I would consider putting your question about colormaps into an issue and assigning it to the 1.8 milestone.

Sounds like a plan. Yes this can be merged.


Projections not working

Options.select is ignored and no projection is made.

mongoose-paginate version? And example

2.11.3
I was using it about like so:
var query = {
	$text: {
		$search: 'test',
		$language: 'en'
	}
};
var options = {
	page: 1,
	limit: 10,
	sort: {
		score: {
			$meta: 'textScore'
		}
	},
	select: {
		score: {
			$meta: 'textScore'
		}
	},
	lean: true,
	leanWithId: true
};

function callback(err, result){

}

Orders.paginate(query, options, callback);

No text score included in results
Also if you setup a blacklist projection it is ignored and all fields are still returned
And the lean flag is also being ignored. I don't get leaned records.
I have resulted to using Mongos native driver and it works like a charm.

Shoot. I didn't realize it was Node 4.2+ now... I think my work machine is still on 0.12.x. Could this be related? Either way I am just going to the native driver.


Use an external repository for bison

In light of https://github.com/kythe/kythe/pull/3516 and given @shahms's new tool for handling cmake files, has this become more feasible than it originally was?

If bison used CMake, potentially, but it's still on autotools from what I can tell.


validation: explicitly use integral division

Division behaviour changed between Python 2 and Python 3.  Explicitly
indicate that we wish to use integral division here.

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

CC: @tachoknight @drexin @tbkka @lorentey

@swift-ci test

@swift-ci please test

Build failed
Swift Test Linux Platform
Git Sha - e1f364320d69f971987706869a79bec2bcfb29ee

Build failed
Swift Test OS X Platform
Git Sha - e1f364320d69f971987706869a79bec2bcfb29ee

@swift-ci please test


Use docker-machine instead of boot2docker

Hi,
I'm using docker version 1.12.0 on a mac and I noticed that the boot2docker command is not used anymore thus making the displayed server URL incorrect upon starting up the container so I updated the startup script with the docker-machine command to fix it.
I've also updated the domain in some files to reflect the latest domain change and also filled out the CLA form.
Thanks,

Many thanks @zabil  👍


Skip to defined point

Hi
This is the first time I've contributed to a GitHub project so I hope that I've done this properly. Apologies if I haven't.
I've created a fork 'gloocomms' that adds a 'skip to slide' function — in my opinion one of the biggest gaps in Prezi.
Simply add a normal link to the html, eg. Skip. Replacing # with the slide number. Obviously you could style these appropriately.
Next step would be to allow relative links, like forward two slides, as well as absolute. I'll have a look at this later if I get a chance.
John

Hi,
when it comes to sharing patches on github when you have a forked repo there is a 'Pull Request' feature (little button next on top, next to 'Fork' button and watchers counter).
It lets you define which commit (or commits) can be given as a patch to other repo and then it's much easier to merge the changes.
Anyway. Your request is a feature that some others already asked for (issue #9) and I'll hopefully finish it some time soon.
So thanks for your interest, watch the repo for the changes.
And if you have any other suggestions or updates don't hesitate to drop them here!
Cheers.


Add 2-way binding to the clr-dg-column Component for Filters and Sorting

Select one ...  (check one with "x")
[ ] bug
[ ] feature request
[x ] enhancement

Expected behavior

Add the ability to detect if a column is currently sorted -or- specify an initial sort state for the clr-dg-column Component.
Add the ability to detect when a column is currently filtered -or- specify an initial filter state for the clr-dg-column Component.

Actual behavior
Users can set an initial State for clr-dg-columns for both Sorting and Filtering because the State of Sorting and FIltering is exposed and accesible. In addition when using Sorting or Filtering on the clr-dg-column Component this binding is two way and persists / is accessible in the datagrid's properties.
Reproduction of behavior
n/a
Environment details

Angular version: n/a

Clarity version: n/a

OS and version: n/a

Browser: n/a

We have similar requirement where user navigates to details page and comes back to grid. I need a way to indicate what column the grid is sorted/filtered on.

Could we get an ETA on this issue? some of our work depend on it.
Thx!

@clementcur it's actually on top of our priorities. Haven't been assigned yet but someone should be looking at it next week and when they do, we'll be able to provide an estimate. Let us know if there is a timeline you need if this is blocking you and we're happy to chat.

Great, sounds good. That's fine, we can wait for a couple of weeks.

@jaffoneh do you have a new milestone in mind?

@clementcur should be there in 0.8.8 next week (PR should be out shortly).

I'm looking for a way to set the default sort column but all issues are closed as duplicates of this one.
So how do I set the default sort column, really?

@duzenko - there is an example for Pre-Sorted Columns in the docs: https://clarity.design/documentation/datagrid/custom-sorting
Basically the app needs to bind to the clrDgSortOrder with the desired ClrDatagridSortOrder:
<clr-dg-column [clrDgField]="'name'" [clrDgSortOrder]="descSort">Name</clr-dg-column>

import {ClrDatagridSortOrder} from '@clr/angular';
...
@Component({ /* ... */ })
class MyComponent {
    this.descSort = ClrDatagridSortOrder.DESC;
}


libtermkey: 0.19 -> 0.20

Semi-automatic update. These checks were performed:

built on NixOS
found 0.20 with grep in /nix/store/4sswlqrcz5y3s241l0vgrbsyzcrl9gn5-libtermkey-0.20
found 0.20 in filename of file in /nix/store/4sswlqrcz5y3s241l0vgrbsyzcrl9gn5-libtermkey-0.20

Success on x86_64-linux (full log)
Partial log (click to expand)
libtool --quiet --mode=finish /nix/store/4sswlqrcz5y3s241l0vgrbsyzcrl9gn5-libtermkey-0.20/lib
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/4sswlqrcz5y3s241l0vgrbsyzcrl9gn5-libtermkey-0.20
shrinking /nix/store/4sswlqrcz5y3s241l0vgrbsyzcrl9gn5-libtermkey-0.20/lib/libtermkey.so.1.14.0
gzipping man pages under /nix/store/4sswlqrcz5y3s241l0vgrbsyzcrl9gn5-libtermkey-0.20/share/man/
strip is /nix/store/b0zlxla7dmy1iwc3g459rjznx59797xy-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/4sswlqrcz5y3s241l0vgrbsyzcrl9gn5-libtermkey-0.20/lib
patching script interpreter paths in /nix/store/4sswlqrcz5y3s241l0vgrbsyzcrl9gn5-libtermkey-0.20
checking for references to /tmp/nix-build-libtermkey-0.20.drv-0 in /nix/store/4sswlqrcz5y3s241l0vgrbsyzcrl9gn5-libtermkey-0.20...
/nix/store/4sswlqrcz5y3s241l0vgrbsyzcrl9gn5-libtermkey-0.20

Success on x86_64-darwin (full log)
Partial log (click to expand)
ln -sf termkey_getkey.3.gz /nix/store/92i4arnbawyl2r9wyxa3qylcfnivzdiy-libtermkey-0.20/share/man/man3/termkey_getkey_force.3.gz
ln -sf termkey_start.3.gz /nix/store/92i4arnbawyl2r9wyxa3qylcfnivzdiy-libtermkey-0.20/share/man/man3/termkey_stop.3.gz
ln -sf termkey_start.3.gz /nix/store/92i4arnbawyl2r9wyxa3qylcfnivzdiy-libtermkey-0.20/share/man/man3/termkey_is_started.3.gz
/nix/store/gsarznl9jdshg7iq18idyj5pzlixmrgr-libtool-2.4.6/bin/libtool --mode=finish /nix/store/92i4arnbawyl2r9wyxa3qylcfnivzdiy-libtermkey-0.20/lib
post-installation fixup
gzipping man pages under /nix/store/92i4arnbawyl2r9wyxa3qylcfnivzdiy-libtermkey-0.20/share/man/
strip is /nix/store/4sdh09gmvl15cy0zb6i7mbvxh5syz206-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/92i4arnbawyl2r9wyxa3qylcfnivzdiy-libtermkey-0.20/lib
patching script interpreter paths in /nix/store/92i4arnbawyl2r9wyxa3qylcfnivzdiy-libtermkey-0.20
/nix/store/92i4arnbawyl2r9wyxa3qylcfnivzdiy-libtermkey-0.20

Success on aarch64-linux (full log)
Partial log (click to expand)
libtool --quiet --mode=finish /nix/store/yxzn26hdqmc67ngmvnh0bdk9h31qapdz-libtermkey-0.20/lib
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/yxzn26hdqmc67ngmvnh0bdk9h31qapdz-libtermkey-0.20
shrinking /nix/store/yxzn26hdqmc67ngmvnh0bdk9h31qapdz-libtermkey-0.20/lib/libtermkey.so.1.14.0
gzipping man pages under /nix/store/yxzn26hdqmc67ngmvnh0bdk9h31qapdz-libtermkey-0.20/share/man/
strip is /nix/store/lvx1acn1ig1j2km8jds5x3ggh3f2wa8v-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/yxzn26hdqmc67ngmvnh0bdk9h31qapdz-libtermkey-0.20/lib
patching script interpreter paths in /nix/store/yxzn26hdqmc67ngmvnh0bdk9h31qapdz-libtermkey-0.20
checking for references to /build in /nix/store/yxzn26hdqmc67ngmvnh0bdk9h31qapdz-libtermkey-0.20...
/nix/store/yxzn26hdqmc67ngmvnh0bdk9h31qapdz-libtermkey-0.20


rook-disk-job: bad dd flag

Describe the bug or feature request
Hi, I've been playing with cluster bootstrapping both with cluster-template and this repo on my own homelab. Came across what seems to be a bug in
.taskfiles/Bootstrap/resources/rook-disk-job.tmpl.yaml
The following:
dd if=/dev/zero of=${disk} bs=1M count=100 oflag=direct,dsync;

results in dd: invalid argument 'dsync' to 'oflag'. It seems like the alpine/coreutils dd does not support this flag (I'm running on amd64 for reference)?
I worked around this by removing the dsync flag.
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Thanks for the info, looks like I added it here https://github.com/onedr0p/home-ops/commit/aa384f65450a774c57fb0ada3135fc641c05946b and probably didn't test it as the cluster was already running.


fix(shared): use default value instead of  undefined when generate default value from spec

Problem
The optional properties' field of newly created component/trait does not show in form. For example, the hidden trait has a visually property and it does not show.

This is because the form is rendered from the defaultValue generated from spec. And the generateDefaultValueFromSpec function will not generate value if a property is optional.
Solution
So I changed the logic of generateDefaultValueFromSpec. It will generate default value for optional property now.

LGTM


Migrate examples to use nested builder pattern

Describe the problem you are trying to solve
All library examples should use a consistent pattern for protobuf builders. In some cases we create the builder for each object separately and call OuterBuilder.setInner(InnerBuilder.newBuilder().setFoo("bar").build()) other times we create one builder and use OuterBuilder.getInnerBuilder().setFoo("bar").
Describe the solution you would like
Converge on the inner builder pattern which is a better paradigm to use.
Describe alternatives you've considered
Additional context

We should also be consistent about whether we call .build() on every builder. Many methods accept either a builder or a built object. I propose that we only call .build():

When required by a method call accepting the object as a parameter.
On the 'top-level' resource. For example, if building AdGroupAd objects, call build() on the AdGroupAd.Builder but not on each one's inner Ad object.

@jradcliff agreed

It seems this commonly comes up as a point of confusion in code reviews so dropping the request.


Error in using Clarify with AWS Jumpstart image classification models

Hi,
I have fine-tuned an image classification model using SageMaker jump start.
I have also deployed my model. This is my code for deploying my model:
sgmkr_clnt = boto3.client("sagemaker")

model_id = 'tensorflow-ic-imagenet-nasnet-mobile'
model_s3_key = <path to my model on s3>
model_version = '2.0.2'

model_name = <some name>

response = sgmkr_clnt.create_model(
    ModelName=model_name,
    PrimaryContainer={"Image": deploy_image_uri, "ModelDataUrl": model_s3_key},
    ExecutionRoleArn=SM_ROLE.ARN.value,
)

prefix = <some prefix>
s3_data_input_path = <some address on S3>

analysis_result_path = <some address on S3>

explainability_data_config = clarify.DataConfig(
    s3_data_input_path=s3_data_input_path,
    s3_output_path=analysis_result_path,
    dataset_type="application/x-image",
)

model_config = clarify.ModelConfig(
    model_name=model_name, instance_type="ml.m5.xlarge", instance_count=1, content_type="image/png"
)

predictions_config = clarify.ModelPredictedLabelConfig(label_headers=['NO', 'YES'])

image_config = clarify.ImageConfig(
    model_type="IMAGE_CLASSIFICATION", num_segments=20, segment_compactness=5
)

shap_config = clarify.SHAPConfig(num_samples=500, image_config=image_config)

I have only one img.png as input.
When I run my code I get this error:
An error occurred (ModelError) when calling the InvokeEndpoint operation (reached max retries: 0): Received server error (500) from primary with message "{"error": "unsupported content type image/png"}"
I have been able to use this error for Batch Inference using png files, before, but it does not work with Clarify.

I found where the problem may be, the following code is the inference.py of sagemaker jumpstart
def input_fn(input_data, content_type):
    """
    Args:
        input_data: the request payload serialized in the content_type format
        content_type: the request content_type
    """
    if content_type == "application/x-image":
        decoded = Image.open(io.BytesIO(input_data))
    else:
        raise ValueError(f"Type [{content_type}] not supported.")

    preprocess = transforms.Compose([transforms.ToTensor()])
    normalized = preprocess(decoded)
    return normalized

Here the only acceptable content_type is "application/x-image" but it is not allowed in ModelConfig. Any idea how to solve this?

Hi,
We consider this as a feature request as here are the current supported content_type.
https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/clarify.py#L1170-L1176
I will consult our PM to put this on our roadmap. Thank you!


nodefaultlibs in Crystal

How do I ask the Crystal compiler not to link with any of the default libraries and startup files?

You can try to pass --prelude=empty, but you won't be able to do much with that. For example if you do [1, 2, 3] that won't compile because the constructor of Array is defined by requiring the regular prelude.
The real question is: why do you want to get rid of them?

@asterite I'm trying out Crystal in kernel mode (ring 0), that's why. Is there any way to go around this?

Crystal is primarily a high-level language, meant to build console apps, games (maybe?), web servers, etc. You won't be able to use it (at least not in a simple way)  to build a kernel, driver, etc. Next thing you'll want is to get rid of the GC, which again won't be easy to achieve.
I strongly suggest using other languages for this, like C, C++ and Rust.

@asterite Oh, ok. Thanks for the clarification.

Thought there is nothing wrong in trying. It could be interesting, even if we don't support it.
Quick question: once you get rid of the core/stdlib, you should have no GC nor anything that needs it?

@ysbaddaden Yup, I haven't given up just yet.

I'll definitely want to see what you come up to.
I'm afraid that it will look like C with a little nicer syntax, but still look exactly like C, with the same issues (memory, file descriptor leaks...).
Maybe blocks could help in having a nicer API, like wrapping POSIX calls to check for Errno, open a file and have it closed automatically, but without exceptions it's somehow useless.

@ysbaddaden I'll definitely post back here.
All the code in ring 0 looks like C until you can get an allocator set up. After that you can use the language in its full glory.

Well, you can certainly do things like tinycr. But what do you mean by "get an allocator set up"?

Oh well, tinycr is an example of manually built Crystal executable where you still need Linux to run the executable. I'm talking about running on bare metal without any operating systems, initially there is no way to dynamically allocate memory in that situation. That is what I meant.

Will watch what interesting problems/solutions arise with interest :-)

Are you trying writing an OS in Rust, but in Crystal? This is gonna be hard (thus incredible) since Crystal doesn't support going bare metal with no runtime like Rust allows to.

A while ago I made a very simple kernel program in Crystal, it is indeed possible to write it with pure Crystal. (ok, I cheated with 5 lines of inline assembly, but that's all). Published so you can take a look: https://github.com/lbguilherme/os-crystal
The problem is that the language itself relies on having a runtime, and mostly on having a GC. So, if you add this to main.cr:
class A
end

a = A.new

It will already fail compilation:
$ make run
Compiling crystal code...
Creating kernel.elf...
main.o: In function `*A::new:A':
main_module:(.text+0x716): undefined reference to `malloc'
main_module:(.text+0x745): undefined reference to `memset'
Makefile:9: recipe for target 'kernel.elf' failed
make: *** [kernel.elf] Error 1

You have to implement your memory management from scratch. I would suggest preallocating some region of memory (say, 16MB) to be your kernel's heap. And use it to create your malloc. The free part I leave as an exercise for the reader. :)

Well, or replace the Boehm GC in Crystal with your own allocation implementation. If it's a non GC-allocation-scheme, just add delete class methods to do the explicit free'ing. I'm toying with my own allocator implementation myself, but since Onyx takes my main focus, I've postponed it.
ensure could be a nice place to put de-allocations in ;-)

@ysbaddaden Yeah something like that. I am trying to work around that problem.
What I have learnt over the past few days toying with Crystal is that getting completely rid of the runtime support routines is a problem until the standard library is completely rewritten (like what @lbguilherme did in his proof-of-concept).


Should show an example of modifying an Attached Property

There's no example here about how to reference an attached property that's part of a named object for a setter.
Like if you wanted to update a Grid Row on the element with the name 'MyGrid' it'd be Target="MyGrid.(Grid.Row)"

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: cb748e04-6d4c-8626-d3c3-086e5b3aef79
Version Independent ID: de8185a7-8b9b-5e28-c8d0-65dc0bccd270
Content: Setter Class (Windows.UI.Xaml) - Windows UWP applications
Content Source: https://cpubwin.visualstudio.com/DefaultCollection/windows-uwp/_git/winrt-api-build?path=%2fwinrt-api-build%2fwindows.ui.xaml.setter.yml&version=GBlive&_a=contents
Product: uwp
Technology: winrt-reference
Git Login: jwmsft
Microsoft Alias: jimwalk

Thanks for the suggestion. The doc has been updated with an example of setting an attached property.


Inference on Custom Point Cloud

Hi,
How can we obtain the inference result for a custom point cloud(say test.ply)?

Hi, How can we obtain the inference result for a custom point cloud(say test.ply)?

Here is how I did it on my own dataset:
https://github.com/meidachen/STPLS3D/blob/main/HAIS/mergeResult.py
Please also refer to the following to change the self.dataAugment parameter.
https://github.com/meidachen/STPLS3D/issues/3

Hi, How can we obtain the inference result for a custom point cloud(say test.ply)?

Here is how I did it on my own dataset:
https://github.com/meidachen/STPLS3D/blob/main/HAIS/mergeResult.py
Please also refer to the following to change the self.dataAugment parameter.
meidachen/STPLS3D#3

Hi seniors, can you please tell me how I should do inference on the Scannet v2 dataset?


⚠ Mobitopup has degraded performance

In 8748ddf, Mobitopup (https://napcard.voz.tech) experienced degraded performance:

HTTP code: 200
Response time: 5208 ms

Resolved: Mobitopup performance has improved in 937b626 after 8 minutes.


Introduce a mangling for multiple opaque types within the declaration.

The Qr mangling is used to refer to the opaque type within the
declaration that produces the opaque type. When there are multiple
opaque types, e.g., due to structural or named opaque result types, it
does not specify which of the opaque type parameters it refers to.
Introduce a new mangling QR INDEX for opaque type parameters after
the first, retaining the Qr mangling for the first opaque type
parameter. This way, existing (non-structural) uses of opaque result
types retain the same manglings, but uses of structural or named
opaque result types (new features) will have distinct manglings.
Note that this mangling within a declaration is only used for the
declaration itself, and not for references to the opaque type of the
declaration, so there is no impact on the runtime demangler.

@swift-ci please smoke test


Linking/importing xlsx file with Access show a blank screen

Hello,
I try to import/link an XLSX file in Access 2013 made with onesheet. At the very first step of the wizard the content of the XLSX file is not displayed. I think Access is puzzled by the structure of the file. Could that be fixed?
Regards,

Hi @lkppo,
I was able to reproduce and in fix the issue (using Access 2016). If you have the ability, please try and see if the fix in the linked branch works for your version as well. I think it should.
Kind regards =)

Hi @nimmneun,
This is an error I encountered at work. I will test Monday.
I have to apply the code of PR 68 on master?
Regards,

Hi @lkppo, either this - or you can just git clone the project, checkout the branch with the fix and create a simple file to test it.

Hi @nimmneun,
I confirm that it works. Now Access is moaning about the version of the XLSX file being newer than the Access version but that's another problem.
Regards,


'List' from 'malib.utils.typing'

Hi I am trying to run a basic MARL setup using MAPPO.
Here's my yaml config file
name: "mappo_payload_carry"

training:
  interface:
    type: "centralized"
    population_size: -1
  config:
    # control the frequency of remote parameter update
    update_interval: 1
    saving_interval: 100
    batch_size: 32
    optimizer: "Adam"
    actor_lr: 5.e-4
    critic_lr: 5.e-4
    opti_eps: 1.e-5
    weight_decay: 0.0

rollout:
  type: "async"
  stopper: "simple_rollout"
  stopper_config:
    max_step: 10000
  metric_type: "simple"
  fragment_length: 100
  num_episodes: 4
  episode_seg: 1
  terminate: "any"
  num_env_per_worker: 1
  postprocessor_types:
    - copy_next_frame

env_description:
  #  scenario_name: "simple_spread"
  creator: "Gym"
  config:
    env_id: "urdf-env-v0"

algorithms:
  MAPPO:
    name: "MAPPO"
    model_config:
      initialization:
        use_orthogonal: True
        gain: 1.
      actor:
        network: mlp
        layers:
          - units: 256
            activation: ReLU
          - units: 128
            activation: ReLU
          - units: 64
            activation: ReLU
        output:
          activation: False
      critic:
        network: mlp
        layers:
          - units: 256
            activation: ReLU
          - units: 128
            activation: ReLU
          - units: 64
            activation: ReLU
        output:
          activation: False

    # set hyper parameter
    custom_config:
      gamma: 0.99
      use_cuda: False  # enable cuda or not
      use_q_head: False
      ppo_epoch: 4
      num_mini_batch: 1  # the number of mini-batches

      return_mode: gae
      gae:
        gae_lambda: 0.95
      vtrace:
        clip_rho_threshold: 1.0
        clip_pg_rho_threshold: 1.0

      use_rnn: False
      # this is not used, instead it is fixed to last hidden in actor/critic
      rnn_layer_num: 1
      rnn_data_chunk_length: 16

      use_feature_normalization: True
      use_popart: True
      popart_beta: 0.99999

      entropy_coef: 1.e-2

global_evaluator:
  name: "generic"

dataset_config:
  episode_capacity: 100
  fragment_length: 3001```

I have a custom environment where I created the env.

env = gym.make("urdf-env-v0", dt=0.01, robots=robots, render=render)
possible_agents = env.possible_agents
action_spaces = env.possible_actions
observation_spaces = env.observation_spaces
env_desc = {"creator": env, "possible_agents": possible_agents, "action_spaces": action_spaces, "observation_spaces": observation_spaces}
run(
    group=config["group"],
    name=config["name"],
    env_description=env_desc,
    agent_mapping_func=lambda agent: agent[
                                     :6
                                     ],  # e.g. "team_0_player_0" -> "team_0"
    training=training_config,
    algorithms=config["algorithms"],
    rollout=rollout_config,
    evaluation=config.get("evaluation", {}),
    global_evaluator=config["global_evaluator"],
    dataset_config=config.get("dataset_config", {}),
    parameter_server=config.get("parameter_server", {}),
    # worker_config=config["worker_config"],
    use_init_policy_pool=False,
    task_mode="marl",
)

I tried to see if malib.utils.typing had the List, Dict types, but it looks like they're non existent there, how do I fix this?

adding the line
from typing import List
resolves this issues

@josyulakrishna MAPPO is still in debugging, you can follow the changes in branch test-cases.

Thank you for the response @KornbergFresnel  is there any example of MADDPG or QMIX on a custom gym-based environment, which I can readily use and understand the library?
I have checked the test-cases branch, and none of the examples really worked for me, I'm not sure what I'm doing wrong. It looks like the project is having several branches and its varied across branches, which is very confusing to me.
Any help is appreciated. Thank you.

@josyulakrishna I've just run the cases under examples, there are no errors raised. Do not use any learner other than "Independent Learner," as the lib is still in reconstruction. Both MADDPG and QMIX are also been removed temporary from our lib. I think the standard version will be ready this month.

I see thank you for the reply @KornbergFresnel


adding -u $UID does not solve permission issues

Trying to run the following command with the -u option to avoid root issues
fmriprep-docker DIR/ DIR/derivatives -u $UID
However, I get the following error  message:
Process Process-2:
Traceback (most recent call last):
File "/usr/local/miniconda/lib/python3.7/pathlib.py", line 1241, in mkdir
self._accessor.mkdir(self, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/out/fmriprep/logs'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):   File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap     self.run()   File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 99, in run     self._target(*self._args, **self._kwargs)   File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/run.py", line 609, in build_workflow     log_dir.mkdir(exist_ok=True, parents=True)   File "/usr/local/miniconda/lib/python3.7/pathlib.py", line 1245, in mkdir     self.parent.mkdir(parents=True, exist_ok=True)   File "/usr/local/miniconda/lib/python3.7/pathlib.py", line 1241, in mkdir     self._accessor.mkdir(self, mode) PermissionError: [Errno 13] Permission denied: '/out/fmriprep'
The command only runs if I DO NOT include the -u $UID option. I've tried $USER and get the following output:
docker: Error response from daemon: unable to find user xxxxx: no matching entries in passwd file. fMRIPrep: Please report errors to https://github.com/poldracklab/fmriprep/issues
Essentially, we'd like the derivatives folder and newly created/files to have the same user and group permissions as the user running the camp, not the root.
Any input would be appreciated.
Thanks
geowk

This line of the error:
PermissionError: [Errno 13] Permission denied: '/out/fmriprep'

make me think the user you are running as doesn't have write access to the output directory.

PermissionError: [Errno 13] Permission denied: '/out/fmriprep'

make me think the user you are running as doesn't have write access to the output directory.

mxgd, that is correct.
This was my command:
fmriprep-docker DIR/ DIR/derivatives -u $UID

I did create the DIR directory, but not the DIR/derivatives. After executing the command, the DIR/derivatives directory is created with root user and root as group . I then created the DIR/derivatives before using fmriprep. Since I had user ownership and group privileges the derivatives folder, fmriprep-docker worked. However, my primary group was not applied to the newly created fmriprep, log,etc. folders. My user ID was also applied to group (GID).

Additionally, if other users follow step 1, the new subfolders created under the DIR/derivatives subfolder have user-specific ownership but the files/folders belong to the root group.  Is there way for the newly created files to not only maintain user-specific ownership but also the specified-user's primary group (GID).

Thanks.

Is there way for the newly created files to not only maintain user-specific ownership but also the specified-user's primary group (GID).

this should be possible by passing in both users / group ids to the --user flag, i.e.
fmriprep-docker --user $(id -u):$(id -g) ...

Thanks mgxd. Your solution worked. As long as the DIR/derivatives directory exists prior to running fmriprep-docker, the appropriate username and group ownership is maintained. No root user or root group ownership of newly created files or folders from the fmriprep-docker output.
If one does not create DIR/derivatives directory before executing the fmriprep-docker command, then the DIR/derivatives folder is created by the fmriprep-docker, but with root owner and root group applied and the fmriprep-docker script fails due to permission issues.
Thanks again mgxd.


Canvas is not updated when user access changes

Describe the bug
When a users access is changed it is not reflected in the canvas. Add card buttons will still be there even if access is removed for example.
To Reproduce
Steps to reproduce the behavior:

Go to any Flyt 2.0 environment
Find/create a process where you have contributor/admin access
Have an admin/owner remove your access
See that you can still see and click add card buttons

Expected behavior
When losing access you should no longer be able to add cards, edit cards or draw lines between cards
Development checklist

[x] Threat Modeling

@SayyareHamit This is ready for test in either poc-dev or poc-qa. Let me know if you need me to remove your access in a process so that you can test it

Tested in poc-qa, works good as required. @nesadrian


Pipeline Optimization

The caching portion is based off
https://github.com/cloudflare/wrangler/actions/runs/36010192/workflow
Pretty much reduces the time consuming, several minutes of compiling to around a minute or so.

The only thing that doesn't seem to be filling the cache is the cargo index. Could be removed.


Public generic methods in TestHandler

Makes TestHandler::hasRecordRecords() and TestHandler::hasRecord() public so they can be used in tests directly just like the other generic methods. Also renames hasRecordRecords to hasRecords because hasRecordRecords is kinda weird.
Not sure the protected to public change would be a BC break for 1.x. If not, I can set the hasRecordRecords method back with a deprecation message. WDYT?

Thanks, merged in 1.x too


forestry service layer updates - WIP

Description
-updated service layer file for forestrysectoropscreening
i'm seeing what else needs to be updated
-also changed numbering on pdfs
Types of changes

Checklist

[ ] I have read the CONTRIBUTING doc
[ ] I have checked that unit tests pass locally with my changes
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have added necessary documentation (if appropriate)

Further comments

SonarQube Analysis: https://sonarqube-vmvfjv-tools.pathfinder.gov.bc.ca/dashboard?id=common-forms-toolkit-pr-84


Use 32-bit registry for .NET SDK locations

Fixes #2047 by always using the 32-bit registry to locate $(SDK35ToolsPath) and $(SDK40ToolsPath).

@AndyGerlicher as we discussed in triage the other day, I considered adding a $(Registry32:) magic property, but it looked fairly complex (changes to the evaluator that could slow all property evaluations) for the benefit, so I'm switching to the existing GetRegistryValueFromView property function.


Auto-rotate + flip/flop fails?

Hi!
I think something has changed regarding flip/flop and rotation between 0.17 and 0.18. Previously, I was using the following code to first auto-rotate the image by EXIF orientation and then flop it:
sharp('some-image.jpg').rotate().flop().toFile(...)

This doesn't seem to work in 0.18. The image is not flopped. The same goes for flipping.
If I explicitly define an angle in the rotate() call, things work as expected.
Any ideas?

Hello, it looks this behaviour was altered by commit 99810c0, which can overwrite existing flip and flop parameter values. I can take a look in a few days, or @ncoden might be able to help also.

I just stumbled upon a non-working rotate() all by itself. I am adding this info to this thread instead of creating another one.
This code is not rotating a picture any longer:
sharp(__dirname + '/mocks/virtualcam/01.jpg')
    .rotate(180)
    .toFile(__dirname + '/rotated.jpg');

I came across this issue on a fresh install of 0.18.1, an older project using 0.17.3 is still somewhat working (I came across rotated and non-rotated images, always with rotate(180), never rotate() but I thought that it had something to do with the camera settings (I am processing captures by a Canon EOS and there is some kind of orientation-EXIF), but seeing it kind of broken right now creates second thoughts.

Commit 49297d6 adds tests for auto-rotate and flip/flop, plus the fix. This will be in v0.18.2. Thanks for reporting.

Sorry for the bug I introduced @lovell

@ncoden Pas de souci, il n'a pas eu un test.

Thanks for the fix :)

@lovell How does everybody find out i'm French ? ;)

v0.18.2 now available.


Does not work on MacOS with IntelliJ IDEA 2016.2

Looks like *.so binary is not compatible with El Capitan ...
I'm not sure if it is config/plugin/IDE issue. It works from cmd:
$ /usr/local/bin/R --no-save --quiet
>

But not from R run-time configuration:
Script: location to *.r script
Working Directory: location to *.r script
/usr/local/bin/R --no-save --quiet
Error in dyn.load("/Users/pranas/Library/Application Support/IntelliJIdea2016.2/TheRPlugin/libtherplugin_device64.so") :
  unable to load shared object '/Users/pranas/Library/Application Support/IntelliJIdea2016.2/TheRPlugin/libtherplugin_device64.so':
  dlopen(/Users/pranas/Library/Application Support/IntelliJIdea2016.2/TheRPlugin/libtherplugin_device64.so, 6): no suitable image found.  Did find:
    /Users/pranas/Library/Application Support/IntelliJIdea2016.2/TheRPlugin/libtherplugin_device64.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00
Execution halted

Process finished with exit code 1

Hello! This binary could be disabled by renaming. It's responsible for graphics integration into IDEA.

Hi Sproshev,
I'm using Intellij Community 2016 and Mac OS X EL Capitan and I have the same error as pranasblk.

I'm having the same issue as well. Renaming libtherplugin_device64.so allows me to run R code again.

Heyoni,
Does the R Graphics work properly in your intellij after renaming the file? It still doesn't work for me, though the error message is gone.

No, graphics don't work after renaming that file.

@HuilinChen @heyoni Graphics integration is based on this file. When plugin can't find .so-file, it disables integration.

Maybe we need to configure
/usr/local/bin/Rscript
as R interpreter instead? However, the script specified in the run configuration seems not to be passed to the interpreter in my case:
/usr/local/bin/Rscript --no-save --quiet

Process finished with exit code 1

Environment: IntelliJ 2016.2.3, OS X 10.11.6, R 3.3.1.

@tuabache The script is passed but not directly.
Could you please setup environment variable as shown below (Run - Edit Configurations) and attach output?

@sproshev Thanks for your message. Still the same output:
/usr/local/bin/Rscript --no-save --quiet

Process finished with exit code 1

with runtime config as suggested:

@tuabache What do you see after running /usr/local/bin/Rscript in terminal? Any GUI?

Hello, nice tool!
(Well, potentially)
But I suffer from the same problem as described above.
ther.debugger.io=true had no visible effect
It seems that, for some reason, the name of the scirpt is not given as argument to Rscript:
/usr/local/bin/Rscript --no-save --quiet --args xxx
file name is missing

Process finished with exit code 1

Expected: /usr/local/bin/Rscript --no-save --quiet myscript.r --args xxx
FYI:
$ uname -a
Xxxxx Xxxxx.local 14.5.0 Darwin Kernel Version 14.5.0: Mon Aug 29 21:14:16 PDT 2016; root:xnu-2782.50.6~1/RELEASE_X86_64 x86_64
$ grep -A1 CFBundleGetInfoString ~/Applications/IntelliJ\ IDEA\ CE.app/Contents/Info.plist
    <key>CFBundleGetInfoString</key>
    <string>IntelliJ IDEA 2016.2.4, build IC-162.2032.8. Copyright JetBrains s.r.o., (c) 2000-2016</string>
$ java -version
java version "1.8.0_92"
Java(TM) SE Runtime Environment (build 1.8.0_92-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.92-b14, mixed mode)
$ Rscript -e "R.version.string"
[1] "R version 3.3.1 (2016-06-21)"

@sproshev There's no GUI coming up when running /usr/local/bin/Rscript in terminal; here's the output (as I'd expect):
$ /usr/local/bin/Rscript
Usage: /path/to/Rscript [--options] [-e expr [-e expr2 ...] | file] [args]

--options accepted are
  --help              Print usage and exit
  --version           Print version and exit
  --verbose           Print information on progress
  --default-packages=list
                      Where 'list' is a comma-separated set
                        of package names, or 'NULL'
or options to R, in addition to --slave --no-restore, such as
  --save              Do save workspace at the end of the session
  --no-environ        Don't read the site and user environment files
  --no-site-file      Don't read the site-wide Rprofile
  --no-init-file      Don't read the user R profile
  --restore           Do restore previously saved objects at startup
  --vanilla           Combine --no-save, --no-restore, --no-site-file
                        --no-init-file and --no-environ

'file' may contain spaces but not shell metacharacters
Expressions (one or more '-e <expr>') may be used *instead* of 'file'
See also  ?Rscript  from within R

same problem ...

@ottbot Here is the source https://github.com/sproshev/TheRPlugin_Device

Same problem here - PhpStorm2016.3 and MacOS 10.12.2 - R installed with homebrew.
Error in dyn.load("/Users/steinmb/Library/Application Support/PhpStorm2016.3/TheRPlugin/libtherplugin_device64.so") :
  unable to load shared object '/Users/steinmb/Library/Application Support/PhpStorm2016.3/TheRPlugin/libtherplugin_device64.so':
  dlopen(/Users/steinmb/Library/Application Support/PhpStorm2016.3/TheRPlugin/libtherplugin_device64.so, 6): no suitable image found.  Did find:
	/Users/steinmb/Library/Application Support/PhpStorm2016.3/TheRPlugin/libtherplugin_device64.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00
Execution halted

If I try to use Rscript as runner the R script files is not passed as argument.

@sproshev

Would you mind if I import the code from https://github.com/sproshev/TheRPlugin_Device into the codbase of https://github.com/holgerbrandl/r4intellij. There is no license in your code at the moment, so I'm not sure about how I could use it

Could you provide some hints how to build it? I've tried on MacOS but struggled. Maybebecuase TheRPlugin_Device/cmake/modules/FindR.cmakedoes not contain configuration branch for MacOS? But even when adding

elseif (APPLE)
    set(R_INCLUDE_DIRS "/Library/Frameworks/R.framework/Versions/3.3_devel/Resources/include/") # Default include directory

    set(R_64_LIBRARIES R_64_LIBRARIES-NOTFOUND)
    set(R_32_LIBRARIES R_32_LIBRARIES-NOTFOUND)

    if (R_BIT EQUAL 64)
        find_library(R_64_LIBRARIES NAMES R HINTS "/Library/Frameworks/R.framework/Versions/Current/Resources/lib") # Default lib directory
    elseif (R_BIT EQUAL 32)
        find_library(R_32_LIBRARIES NAMES R HINTS "/Library/Frameworks/R.framework/Versions/Current/Resources/lib") # Default lib directory
    endif ()
endif ()

make dies with the error
[  8%] Building CXX object CMakeFiles/therplugin_device64.dir/Evaluator.cpp.o
In file included from /Users/brandl/Desktop/TheRPlugin_Device/Evaluator.cpp:2:
In file included from /Users/brandl/Desktop/TheRPlugin_Device/Evaluator.h:6:
/Users/brandl/Desktop/TheRPlugin_Device/ScopeProtector.h:6:10: fatal error: 'Rinternals.h' file not found
#include <Rinternals.h>
         ^
1 error generated.
make[2]: *** [CMakeFiles/therplugin_device64.dir/Evaluator.cpp.o] Error 1
make[1]: *** [CMakeFiles/therplugin_device64.dir/all] Error 2
make: *** [all] Error 2

Any help would be highly appreciated.

@holgerbrandl

Feel free to use it in the same way as this plugin :)

I was reading this CMakeLists.txt and all its dependecies while writing TheRPlugin_Device/cmake/modules/FindR.cmake and TheRPlugin_Device/CMakeLists.txt. It's better to check how RStudio developers setup their environment for MacOS.