proposal: net, os: API to interrupt Read/Write() for polled objects

Problem
Although we can interrupt/pause the I/O poller with a timer via Set*Deadline(), there's no way to do that with a call -- something which is easy to do via native OS APIs.
Consider a message receiver, started via go runRecv(c). It's easy to follow and needs 1 goroutine.
func runRecv(iConn net.Conn) {
   ...
   for {
      iConn.SetReadDeadline(...)
      aLen, err := iConn.Read(...)
      if err != nil {
         // handle errors
         return
      }
      // handle input; may entail Read()
   }
}

Support out-of-band input. Now it's hard to follow and needs 2 goroutines, 2 channels, and a select.
func runRecv(iConn net.Conn, iAlt chan T) {
   aReadFlag := make(chan bool)
   var aLen int
   var err error

   go func() {
      for <-aReadFlag { // wait for handler
         iConn.SetReadDeadline(...)
         aLen, err = iConn.Read(...)
         aReadFlag <- true
      }
   }()
   defer func() { aReadFlag <- false }()

   for {
      aReadFlag <- true // signal to reader
   WaitForMsg:
      select {
      case aMsg := <-iAlt:
         // handle out-of-band input
         goto WaitForMsg
      case <-aReadFlag:
      }
      if err != nil {
         // handle errors
         return
      }
      // handle network input; may entail Read()
   }
}

Proposal
Let us interrupt Read/Write() for polled objects. This is easy to follow and needs 1 goroutine.
func runRecv(iConn net.Conn) {
   ...
   for {
      iConn.SetReadDeadline(...)
      aLen, err := iConn.Read(...)
      if err != nil {
         if err == io.Stopped { // iConn.Stop() was called
            // handle out-of-band input
            continue
         }
         // handle errors
         return
      }
      iConn.SetReadStop(false)
      // handle network input; may entail Read()
      iConn.SetReadStop(true)
   }
}

API
As methods of polled types
func (o *T) Stop() error causes the pending or next o.Read/Write() to return io.Stopped.
func (o *T) SetStop(s bool) error false prevents or postpones io.Stopped result.
func (o *T) SetReadStop(s bool) error false prevents or postpones io.Stopped result from Read().
func (o *T) SetWriteStop(s bool) error false prevents or postpones io.Stopped result from Write().
As functions in package io
func Stop(o interface{}) error
func SetStop(o interface{}, s bool) error
func SetReadStop(o interface{}, s bool) error
func SetWriteStop(o interface{}, s bool) error
Multiple Stop() requests outside of polling or after Set*Stop(false) yield at most a single io.Stopped error.
If Stop() is called after Set*Stop(false), a Set*Stop(true) causes the pending or next Read/Write() to return io.Stopped. Alternatively Set*Stop() could take an argument indicating never, postpone, or asap, where never causes subsequent Stop() requests to be ignored instead of postponed.
The API supports each polled type, e.g. net.TCPConn, net.UnixConn, os.File. The calls return an error if the object is not open/connected.
cc @aclements

Related #20280 - using Context to interrupt Read/Write()

i think this would lead to poor code over using channels. Now you have a reader that must handle the OOB input as well. Also, it complicates partial messages as this might need to be accounted for in handling the OOB message. Seems like it will lead to a lot of concurrency issues.
Contrast with the simple solution that the io reader places completed messages on a channel. OOB messages can be placed on the same channel. A command processor reads the channel. No api changes and a cleaner responsibility structure.

The issue is pausing the I/O poller (a general purpose feature), not out-of-band input for network streams. I've clarified that in the problem statement.
What you describe is just slightly more efficient than what I wrote.

Support out-of-band input. Now it's hard to follow and needs 2 goroutines, 2 channels, and a select.

As far as I can tell, a simpler implementation of that logic requires only one extra goroutine and one channel, although I find it clearer with two channels (https://play.golang.org/p/UMk09ufmAU9). It is straightforward to add a third (1-buffered) channel to eliminate the buffer reallocation in the steady state, if you are so inclined.
Either way, it seems much clearer to put the Read results themselves on the channel instead of using it as at out-of-band control channel. (This is closely related to the recurring “share by communicating” theme of my GC '18 talk.)
Note that with that approach, the Read goroutine may complete an already-pending Read call while the consumer is blocked on an interrupt, but won't begin another Read — and will therefore reduce pressure on the poller — until the consumer has received the result.

I think @bcmills code is very simple to understand, and addresses the 'pausing' requirements, and partial message issues (that can be VERY difficult to handle).

Again, the point is pausing the I/O poller, not implementing out-of-band input per se.
For reference, here's the core of @bcmills concept, which needs 2 goroutines, 3 channels, and 2 selects.
As before, it doesn't help if receiving a complete message requires fully processing it (e.g. the message body is streamed to disk), which I flagged via // handle network input; may entail Read().
func logAll(ctx context.Context, nc net.Conn, ic <-chan interruption) error {
	defer nc.Close()

	var (
		readPacket = make(chan []byte)
		readErr    = make(chan error, 1)
	)
	go func() (err error) {
		defer func() { readErr <- err }()
		for {
			buf := make([]byte, maxPacketSize)
			n, err := nc.Read(buf)
			if err != nil {
				return err
			}
			select {
			case <-ctx.Done():
				return ctx.Err()
			case readPacket <- buf[:n]:
			}
		}
	}()

	for {
		select {
		case err := <-readErr:
			if err != io.EOF {
				return err
			}
			return nil
		case packet := <-readPacket:
			log.Printf("Read: %s", packet)
		case interrupt := <-ic:
			log.Printf("Interrupt: %s", interrupt)
		}
	}
}

Why is it important to pause the poller per se?

I also like @bcmills 's solution.  Why confuse matters with the notion of out-of-band data and stopping the poller?  What do you gain from that?  My best guess for what you mean by "out-of-band data" is that you have two different input streams: one from the network connection and one from something else.  In Go the natural way to handle two different input streams is to put them on a channel and use select.  I don't yet see the compelling argument for doing anything else.

But even then, if pausing the poller is the “goal”, when reading a message and streaming to disk? this would be done in a loop anyway (for arbitrary length messages) - a simple Lock() would allow the poller to be paused (the external interrupter takes the lock, pausing the poller).
I think the solution outlined by @bcmills is the “go” way (and the “proper” way as well).

On Jan 6, 2020, at 4:33 PM, Bryan C. Mills<EMAIL_ADDRESS>wrote:
Why is it important to pause the poller per se?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/golang/go/issues/36402?email_source=notifications&email_token=ABF2U4NHF7LJ2EFPUGV5QVLQ4OWSVA5CNFSM4KDANSZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIHAR7Q#issuecomment-571345150, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABF2U4LRYJE5EIOARE3WCRTQ4OWSVANCNFSM4KDANSZA.

With an OS-native API, it's easy to code anything which requires that certain polled objects aren't in-progress. Virtually inserting a message into a TCP stream is but one example.
Sure, we can approximate this today, but the result is harder to read and less efficient. My proposed example (pasted below) is obviously clearer and more efficient than @bcmills suggestion, and trivially supports very long messages.
func runRecv(iConn net.Conn) {
   ...
   for {
      iConn.SetReadDeadline(...)
      aLen, err := iConn.Read(...)
      if err != nil {
         if err == io.Stopped { // iConn.Stop() was called
            // handle out-of-band input
            continue
         }
         // handle errors
         return
      }
      iConn.SetReadStop(false)
      // handle network input; may entail Read()
      iConn.SetReadStop(true)
   }
}

I either don't understand your example or I don't understand the proposal.  It seems to me that the call to iConn.SetReadStop(true) at the end of the loop means that the call to iConn.Read at the start of the loop will always return io.Stopped.  Can you explain how this is supposed to work?

The SetReadStop(true), means that future Stop() calls will work on reads, etc. If it is false, then Stop() is ignored during read.
@networkimprov I think a problem with your proposal is that you are taking a polled approach. Go uses 'select/poll' behind the scenes to be efficient, but a different implementation might not do it that way. What Go exposes is a simpler serial interface - protecting the developer from having to deal with the intricacies of select/poll - in some ways you are adding that back in. If you could 'select' on a Conn, that would be a different story - and a completely differently interface.
Another problem with your code is that it essentially limits you to a single "interrupter/OOB supplier", or you need some additional advanced state management (bringing the overall complexity up). The @bcmills version does not have this limitation.
As a sidenote, Ian mentioned that the SetDeadline() takes effect for existing Reads - so you could set a flag and set a very small read timeout and accomplish the goal.

@ianlancetaylor my example lacks an iConn.Stop() (mentioned in a comment); it occurs in a different goroutine in this case. Only Stop() causes err == io.Stopped.
The iConn.SetReadStop(true) causes io.Stopped on subsequent Read() only if Stop() occurred since the last Read().
@robaho you can use any means (incl select{...}) to reach an alternate event supplier on io.Stopped. I addressed the c.SetDeadline(long_past) concept in the golang-dev thread:

Unfortunately you have to clear the long-past deadline (or set in future) before each Read/Write() so if the SetDeadline(long_past) call arrives after the I/O op returned, you can't see it. Also the resulting error is the same for long-past deadlines, ordinary deadlines, and keepalive failures.

This is clearly a variation on Set*Deadline(), not a polling API.

@networkimprov upon more detailed reading, this is purely a code structure problem not an API/functionality problem. You can't read a partial message - the data is still in the stream - so it must be read eventually - your problem is that it is delaying the handling of 'external OOB' - but that is only because you have a single routine attempting to process 2 sources - there is no reason to do this in Go, as the routines are cheap - @bcmills code is structured far cleaner.
Like I said, your code because much more complex if there is more than one provider of OOB messages - with @bcmills it is a trivial and straightforward change.
I don't think your code is cleaner. It clearly breaks the single responsibility principle - you have a function performing IO and processing messages from other actors (nested within the IO processing).
As an aside, I've only ever seen ReadDeadline() used to terminate a connection when a heartbeat doesn't arrive (dead TCP stream detection), using it for anything else is troublesome and a poor design choice IMO.

This is intentional: "it is delaying the handling of 'external OOB'". Thanks anyway.

@networkimprov

We can already stop the poller with a timer via Set*Deadline(), so why not with a call?

You're not proposing a way to stop the poller, but to pause it, which is a very different thing.

Although we can interrupt/pause the I/O poller with a timer via Set*Deadline(), there's no way to do that with a call -- something which is easy to do via native OS APIs.

It might be useful to show the actual C code that would be easy to do with a native OS API, and describe why, for your particular use case, the existing solutions aren't sufficient.

Sorry for misunderstanding how this API works.
I don't yet see the point of this.  Go supports having a single goroutine handle two different input sources by using channels and select statements.  What do we gain by providing another mechanism?  The resulting code looks harder to understand and more prone to race conditions.  And the overall approach is hard to explain.  What is the benefit?  We just get to save a goroutine?  But goroutines are relatively cheap.

@ianlancetaylor The only thing I can think of is that there's no way to stop an in-progress read call, issue some other system call on the file descriptor, then continue reading. I'm not familiar with the OOB API, but perhaps the idea is that you get a signal indicating that OOB data is present, stop any existing read call, then issue a different kind of syscall to get the OOB data, and then a normal Read call again. Currently I don't see that there's a way to do that (@bcmills' solution would still involve a race on the underlying fd). This is why I think it would be useful to know the actual problem that the original poster is trying to solve.

If this is about MSG_OOB as per recvmsg(2) and friends, neither of the designs here is appropriate and Go already has the right low-level building blocks:
https://godoc.org/golang.org/x/sys/unix#Recvmsg
https://godoc.org/golang.org/x/sys/unix#Sendmsg

The OPs use of OOB is not related to TCP "out of band" data, nor TCP/UDP "ancillary data (e.g. timestamps)".
By OOB, he is referring to a interruption of serial processing to handle an async event.

Are there any use cases other than OOB data that this API might be useful for?

One consideration here is that setting a mode is racy in a concurrent program, and in general we should avoid modes whenever possible.  So rather than SetStop we should have StoppableRead.  Then I guess Stop would mean that the current or next StoppableRead would return io.Stopped.  But if we express the problem in that way, then Stop seems isomorphic to SetDeadline(time.Now()).

@rogpeppe thanks for thinking about ways this could be used outside my example. Re "not ... a way to stop the poller, but to pause it," this has the same effect as Set*Deadline(), but with the error value io.Stopped.
Re what we can do in C, I haven't used the epoll API for a while, but off the top of my head, you can add a a unix-socket to its fd list and interrupt epoll_wait() by pinging it. Then you can do anything before resuming epoll_wait().
The code in the issue which I dubbed "hard to follow and needs 2 goroutines, 2 channels, and a select" is what I'm actually doing. It works, but it made me wish for a variant of Set*Deadline() which can be triggered by a call.
@ianlancetaylor iConn.StoppableRead(...) is an interesting idea, thanks. Note that Stop() affects either the pending or next StoppableRead().
iConn.SetDeadline(time.Now()) doesn't work like iConn.Stop() because, as mentioned above, "you have to clear the long-past deadline (or set in future) before each Read/Write() so if the SetDeadline(long_past) call arrives after the I/O op returned, you never see it. Also the resulting error is the same for long-past deadlines, ordinary deadlines, and keepalive failures."

This is sounding an awful lot like "interrupts" in Java. I suggest if this proposal was to move forward, it would adopt similar syntax and semantics. e.g. Interrupt(), IsInterrupted() etc. This would work on any blocked Go routine.I will say that a big part of the reason Java needs this is that the threads are heavy, so you need to conserve their usage. Similar needs led to async IO facilities. Go took a different (and better approach IMO), and this proposal seems at odds with the Go way of spinning up another routine. I suggest we don't muddy the waters with another way to accomplish the same thing.-----Original Message-----
From: Liam
Sent: Jan 7, 2020 12:45 PM
To: golang/go
Cc: robert engels , Mention
Subject: Re: [golang/go] proposal: net, os: API to interrupt Read/Write() for polled objects (#36402)
@rogpeppe thanks for thinking about ways this could be used outside my example. Re "not ... a way to stop the poller, but to pause it," this has the same effect as SetDeadline(), but with the error value io.Stopped.
Re what we can do in C, I haven't used the epoll API for a while, but off the top of my head, you can add a a unix-socket to its fd list and interrupt epoll_wait() by pinging it. Then you can do anything before resuming epoll_wait().
The code in the issue which I dubbed "hard to follow and needs 2 goroutines, 2 channels, and a select" is what I'm actually doing. It works, but it made me wish for a variant of SetDeadline() which can be triggered by a call.
@ianlancetaylor iConn.StoppableRead(...) is an interesting idea, thanks. Note that Stop() affects either the pending or next StoppableRead().
iConn.SetDeadline(time.Now()) doesn't work like iConn.Stop() because, as mentioned above, "you have to clear the long-past deadline (or set in future) before each Read/Write() so if the SetDeadline(long_past) call arrives after the I/O op returned, you never see it. Also the resulting error is the same for long-past deadlines, ordinary deadlines, and keepalive failures."
—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

On review, the "interrupts" semantics of Java cannot be applied to Go, as there is no reference available to the routine you would want to interrupt, so Interrupt() would need to be on the Conn, and probably any other blocking resource if this was to be "complete".
Seems way too many changes in this case, when the current method works fine IMO.
(In the OPs case, what if the resource was a network file descriptor (or other "slow" device, or a call that blocked until a "device status change" - which might be indefinite), you would need the same Stop, etc. on all of these resource handles.

The code in the issue which I dubbed "hard to follow and needs 2 goroutines, 2 channels, and a select" is what I'm actually doing. It works, but it made me wish for a variant of Set*Deadline() which can be triggered by a call.

If that works, I don't understand why you need to interrupt the read at all. Why not just issue the syscall concurrently with the read and not bother with any of the mechanism? AFAICS you've always got the risk of that happening anyway.

I'm not making a syscall, so not sure what you mean... I need to either prevent the goroutine that calls Read() from processing received data when the poller wakes it if there's an alternate-source message waiting (what I do now), or force it awake to handle the alternate source by a mechanism like iConn.Stop().
Since the latter might be generally useful, and also satisfy #20280, I've proposed it here.

@networkimprov
Read and RecvMsg both invoke syscalls.
This is why it would help if you could be more specific about what you actually want to do while the connection is stopped (as I suggested, some C code would be good). You said in the original proposal that the code "may call Read". With the goroutine approach, there's no guarantee that if you call Read, the other goroutine might not call Read at the same time. So I don't see (currently) how that approach is better than just calling Read (or RecvMsg) anyway, regardless of the currently running Read call.

To clarify, the other goroutine, which calls iConn.Stop() does not also call iConn.Read(...).
Re actually want to do, you mean what happens at // handle out-of-band input. A channel read, or Read() on different connection; example below.
func runRecv(iConn net.Conn, iAlt <-chan *T) {
   aBuf := make([]byte, ...)
   ...
   for {
      iConn.SetReadDeadline(...)
      aLen, err := iConn.Read(aBuf)
      if err != nil {
         if err == io.Stopped { // iConn.Stop() was called
            handleMessage(<-iAlt, nil)
            continue
         }
         // handle errors
         return
      }
      iConn.SetReadStop(false)
      handleMessage(getMessage(aBuf[:aLen]), iConn) // calls io.CopyN(file, iConn, ...)
      iConn.SetReadStop(true)
   }
}

The whole problem is he is trying to avoid an additional Go routine but he already has one = the one that would be caking Stoo. His code is not structured correctly IMO - that is the only issue here.

@robaho Yes, that does seem to be the case.
This is the nub of the issue IMHO:

I need to either prevent the goroutine that calls Read() from processing received data when the poller wakes it if there's an alternate-source message waiting (what I do now), or force it awake to handle the alternate source by a mechanism like iConn.Stop().

I think the thinking is confused here - instead of processing the received data directly when it receives a message, the connection reader should instead be forwarding that message to a goroutine that is also aware of alternate-source messages.
That is, ISTM that @networkimprov is trying to use the reader goroutine as a multiplexer when it could/should be processing data from a single source, leaving another goroutine to to the muxing.

Re "already has [an additional goroutine,] the one that would be calling stop," that's required by the other example too, for alt_channel <- message. Don't casually accuse ppl of intellectual dishonesty.
@rogpeppe you've repeated @bcmills concept, summarized in https://github.com/golang/go/issues/36402#issuecomment-571344113. I've repeatedly stated that it doesn't work for long messages which must be fully processed (e.g. streamed to disk) to be "complete".
For the nth time, this isn't a proposal re insertion of messages into streams from alternate sources. Could we please discuss the applications and problems of an API to interrupt a polled object?

Could we please discuss the applications and problems of an API to interrupt a polled object?

@networkimprov, it seems that none of the rest of us are seeing applications that require this behavior. Could you provide some examples of concrete, existing programs where this would be useful, where the alternate approach is not satisfactory?

Another use case is #20280 which seeks to interrupt polled objects on Context cancellation. I ref'd it in the first comment above :-)

@networkimprov #20280 also doesn't give any examples of existing programs.  It refers to #18507, and #18507 has been fixed since #20280 was proposed.  So unfortunately that reference isn't too helpful.
This API you are proposing is inherently racy, since it requires coordination between the goroutine calling Read and the goroutine calling Stop; consider what happens if the Read completes just as a goroutine calls Stop.  It can only be used correctly in cases where that doesn't matter, which means that it is relatively subtle.  And it duplicates the existing SetDeadline mechanism which has the same problems.
Given these complexities I think we need a clear explanation of significant benefit that outweighs the cost, and I don't think we've seen one.

I don't see how coordination between goroutines using the API is necessary or useful. Stop() causes io.Stopped from the pending or next Read/Write(); it doesn't promise the earliest conceivable one. It does promise that io.Stopped will occur when calling Read/Write() in a loop, which is by far the common case. That also applies to SetDeadline().
When not calling Read/Write() in a loop, you don't rely on io.Stopped to do something; you simply do it after Read/Write() returns.
There is recent activity on #20280 re https://github.com/navytux/go123,
and this experience report by @zombiezen covers an existing program:
https://medium.com/@zombiezen/canceling-i-o-in-go-capn-proto-5ae8c09c5b29
For Context cancellation, SetReadDeadline() can frequently poll a shall_we_quit flag, but that interferes with using it to monitor connection health or protocol correctness, and creates overhead, i.e.
a) wake the polling thread
b) schedule the blocked goroutine
c) block it again
d) sleep the polling thread

@networkimprov

I've repeatedly stated that it doesn't work for long messages which must be fully processed (e.g. streamed to disk) to be "complete".

Yes, you've repeatedly stated that, but you haven't explained any further and I don't understand why that's the case. Why can't the goroutine that's receiving the data from the network reader also stream that data to disk when appropriate?

It's moot, but streaming is in pkg.HandleMessage() which you would site in the "muxing" thread.

@networkimprov If the goroutines don't coordinate, then this API only works reliably if one goroutine calls Read and Write in a loop, as you say.  That is a subtle point.
The experience report by @zombiezen does not seem to be the what you are talking about.  That article is how safely closing down a connection.  We've worked to eliminate those problems, by making it reliable to concurrently call Close and Read, and by adding support for SetDeadline on *os.File.  Since we have done that work, this proposal would not help with that problem.
With regard to the xio package, I have no problem with using a context.Context to cancel a Read or Write.  That also is not this proposal.

Your quote from https://github.com/golang/go/issues/20280#issuecomment-365434288 (after #18507 was fixed) is one inspiration for this proposal:
"I think we can ... focus on the idea of whether it is possible to arrange for a Read to be interrupted, with an error, when some Context is canceled. We can implement that for descriptors that end up in the network poller ... Can we devise a mechanism for associating a Context with a *os.File or net.Conn?"
I drafted a more general-purpose API, as not everyone uses Context. How does a polling interrupt scheme via Context result in a less "subtle" API?
Thinking aloud, we could define
func (o *T) SetStopper(c <-chan struct{}) error
and do channel send or close to cause interrupt. The Read/Write() returns err == io.Stopped.

this API only works reliably if one goroutine calls Read and Write in a loop

To clarify, either a Read() loop, or Write() loop; not Read() and Write() in same loop.

[reposted with major revisions]
Your quote from https://github.com/golang/go/issues/20280#issuecomment-365434288 (after #18507 was fixed) is one inspiration for this proposal:
"I think we can ... focus on the idea of whether it is possible to arrange for a Read to be interrupted, with an error, when some Context is canceled. We can implement that for descriptors that end up in the network poller ... Can we devise a mechanism for associating a Context with a *os.File or net.Conn?"
I drafted a more general-purpose API, as not everyone uses Context. How could a polling interrupt scheme via Context result in a less "subtle" API?
Thinking aloud, we could define this on pollable types to let channel send or close cause interrupt.
func (o *T) SetStopper(c <-chan struct{}) error
The pending or next Read/Write() after channel send returns err == io.Stopped.
This enables iConn.SetStopper(ctx.Done()), eliminating the extra goroutine described by @zombiezen in the "What Doesn't Work" section of his report.

this API only works reliably if one goroutine calls Read and Write in a loop

To clarify, either a Read() loop, or Write() loop; not Read() and Write() in same loop. And it's entirely reliable, but like SetDeadline() or any I/O routine, it's non-deterministic.

I think we should encourage people to use context.Context.  I don't think we need a different API for cancelation.  One is enough.
Also I suggested associating a context.Context with a file or network connection.  I don't know if that is a good idea.  But it's not the idea in this issue.  I was thinking in terms of "if this context is canceled, we no longer care about this network connection."  But this proposal is about something different.

Without beating a dead horse, every Go routine should have an implicit context (TLS again...), but in order to use that effectively, you need a reference to the Go routine (at least traditionally - you could use the Context reference as a substitute for the routine reference). That is
theContext := go somefunc()

Then you can do things like, theContext.setName(), etc.
But you probably want to set some of these properties at creation time to avoid race conditions, via
theContext := go "somename" somefunc()
``` etc.
Then it is a simple matter of changing the internal runtime code to access the routines context during scheduling/io to know if the operation has been cancelled. Clearly Context.Cancel() would need some internal runtime support to interrupt outstanding IO using that context. Context would also need to be thread safe.

That's off-topic; pls file a new issue, thanks.

It was to clarify that @ianlancetaylor recommendation to use context.Context has some gaps IMO.

@ianlancetaylor it would be strange to add a feature to the runtime that allows polled objects to be interrupted non-terminally and not provide any API to it except context.With*().
But in any case, please add the feature, and in such a way that any reasonable API can trigger it. Then any project can add an app-specific API to it. (Many of us already patch the stdlib to make Go work for our apps.)
Should I file a new issue for a runtime feature?

Should I file a new issue for a runtime feature?

We don't have any agreement on #20280.  I don't think we'll get any agreement on a more general feature.

To clarify, I mean an internal polling-interrupt capability, with public API to be determined later.
This would allow projects to experiment with APIs, and likely lead to consensus re a public API.

We already have an internal polling-interrupt capability: runtime.netpollunblock.
We aren't going to add another one without understanding what the public API will look like first.  What you are suggesting sounds like the opposite of the way we do things.

Is this the mechanism by which a deadline timer unblocks its object on timeout?
//go:linkname poll_runtime_pollUnblock internal/poll.runtime_pollUnblock
func poll_runtime_pollUnblock(pd *pollDesc)
If so, how is the Read/Write() error set to indicate SetDeadline() timeout?
Could I make a patch to set a different error value, and then invoke the above call?

No, runtime.pollUnblock is used when a descriptor is being closed, to wake up reads and writes blocking on that descriptor.
Deadline timers unblock via poll_runtime_pollSetDeadline aka internal/poll.runtime_pollSetDeadline which calls netpollunblock.


[Performance] Optimize the ShadowClassLoader when determining isPartOfShadow

This PR tries to restore the main performance drop on 1.18.1
issue:
I have a multi module maven project with 20 modules using ecj.
With 1.18.1 the build time increased ~7 seconds compared to 1.18.0
The bottleneck was, that each compile task per maven module scans the complete rt.jar if "java/lang/Object" is extended. I also noticed, that every task creates a new ClassLoader (which is unneccessary IMHO)

This PR can be closed if #1771 will be merged


Lint based on file type

@lekkas does this implies that it should also handle Typescript?
Should we PR the UI's tslint configuration into this?
See: https://github.com/resin-io/resin-ui/issues/1418

@thgreasi yeah, the idea behind node-resin-lint was to abstract more linters per file/language type with the benefit of having their linting configuration in this repo. It'd be great if we had this for Typescript as well!


What configuration is possible via AppSettings?

The documentation doesn't say which options are configurable via appSettings.

It seems that inlineFields parameter can't be configured using appSettings :(

There are some changes in dev now from a PR that allows more options to be used. Additional documentation has been added.


Cob behaviour

Please, don't merge this pull request before cob_android is "released"

I think this PR needs to be cleaned up (preferably squashed) after the remaining remarks have been addressed....it currently contains quite a lot of changes not related to cob_behaviour/cob_default_robot_behaviour....

This PR cannot be merged. Please update with latest ipa320 indigo_dev.
Also not all aspects have yet been considered....tbd

I can close this pull request, and open a new one with the actual robot working version.
We have also still the teleop problem , until we have a new teleop node  we should not merge this pull request and the cob4-4 PR https://github.com/ipa320/cob_robots/pull/307.
My suggestion is wait for teleop node, addapt the parameters and launch files , merge with ipa320, resolving merge conflicts, and test it on the robots. And then open pull request from ipa-cob4-X accounts.

Keep this one open until there actually is a superseding PR!
The new PR for cob_default_robot_behavior :wink:  should then contain changes only for that feature...
Features from this PR that are not related to cob_default_robot_behavior should be extracted from this one and send via a separate PR for those changes...

please make sure the changes within cob_hardware_config from this PR do not overwrite/break the driver, controller or elmo configs....in the best case they should be removed from this PR

LGTM


Stop filterUnconnected() modifying its input

Helper function filterInternetAdjacencies() would sometimes copy and sometimes not copy the input, which can lead to concurrent reads and writes as seen in #3219.
Changed to always copy, before calling.
I found this by eyeballing; this is the only case I could see where the input .Nodes can be modified.

@satyamz would you like to review?


Change of default algorithm may cause problems

I just wanted to share with you  my experience that the change in the default signing algorithm from HS256 to HS512 can break things in case of JSONWebSignatureSerializer that need to be persistent (e.g. stored in a db).
On our server, previously generated JWTs started causing BadSignature exceptions, resulting in authentication failure.

Ugh sorry. I did not see that this was changed. @davidism can we roll this back?

So my proposal is to add a fallback_digest_methods to HMACAlgorithm and Signer. Then set that to [sha512] and change the default back to sha1 then make a 1.0.1 release.

Damn. This does not work because our key derivation also uses the hash method. :-/

The only fix would involve moving derive_key into the hash algorithm as well and derive different keys for all fallbacks.

This is impossible to fix now without making a massive mess of the API. I'm considering wiping the 1.0 release right now and hoping not enough people upgraded yet and then try to figure out how to deal with this. Thoughts? I definitely do not want to go forward with 512 hashes by default.

https://github.com/search?q="itsdangerous>%3D1.0"&type=Commits
when nuking the release we should should probably ask those projects to revert to an older version as well

@ThiefMaster yeah. I'm not sure but this is not a great situation right now :( The way I designed (or misdesigned) the key derivation system leaves no good way now to support both hashes that the default was changed without redesigning the entire API.
That said, it does show that generally fallbacks need to be supported anyways so the positive aspect of this is that it forces that conversation now.

I removed the 1.0.0 release. Will figure out the plan tonight.

Maybe it's bad practice, but wouldn't it be possible to just select the decoding algorithm according to the header?

@RR2DO2 so even something like this, with a protection against 'none' algorithm, is not advisable i guess https://github.com/desmoteo/itsdangerous/commit/016705775603190c0d6d4bbaacafbcd5faf48965

Had some problems with the upgrade too, Flask requires >=0.24 so we got some breaking changes, we did adjusted/updated to a non breaking change with the latest versions 1.0.0 but just found out this #112.
:/ Fixing again...

I realize your team is probably not having the best of weekends, but could you perhaps find two minutes to put up a brief notice on the Pallets blog that the package was pulled?


fix(es/transforms/compat): Optioanl call function call result with param

The code of optional chaining is a bit confusing, maybe it could be refactored into more like babel's implementation, but that would require 2 or 3 loop instead of one


[cluster-test] improve compatibility_test report

Motivation
Write better report for compatibility_test. Report text now contains all the upgrade steps in compatibility_test, which is useful to show how backwards compatibility was broken.
Have you read the Contributing Guidelines on pull requests?
Yes
Test Plan
Run cti from command line for both pass and fail cases for land_blocking_compat suite. The following is output from the generated report's "text" field, which is displayed in PR comment.
Pass:
Compatibility test results for land_fb8337ea ==> dev_rustielin_pull_5150 (PR)
1. All instances running land_fb8337ea, generating some traffic on network
2. First validator land_fb8337ea ==> dev_rustielin_pull_5150, to validate storage
3. First batch validators (14) land_fb8337ea ==> dev_rustielin_pull_5150, to test consensus
4. Second batch validators (15) land_fb8337ea ==> dev_rustielin_pull_5150, to upgrade rest of the validators
5. All full nodes (30) land_fb8337ea ==> dev_rustielin_pull_5150, to finish the network upgrade
all up : 1017 TPS, 4444 ms latency, 5500 ms p99 latency, no expired txns

Fail:
Compatibility test results for  master_21768f2 ==>  dev_rustielin_pull_5150 (PR)
1. Generating some traffic
Experiment `Updating [val-21(<IP_ADDRESS>), val-15(<IP_ADDRESS>), val-8(<IP_ADDRESS>), val-28(<IP_ADDRESS>), val-0(<IP_ADDRESS>), val-4(<IP_ADDRESS>), val-9(<IP_ADDRESS>), val-1(<IP_ADDRESS>), val-25(<IP_ADDRESS>), val-26(<IP_ADDRESS>), val-19(<IP_ADDRESS>), val-23(<IP_ADDRESS>), val-16(<IP_ADDRESS>), val-17(<IP_ADDRESS>), val-2(<IP_ADDRESS>), val-3(<IP_ADDRESS>), val-5(<IP_ADDRESS>), val-6(<IP_ADDRESS>), val-7(<IP_ADDRESS>), val-10(<IP_ADDRESS>), val-11(<IP_ADDRESS>), val-12(<IP_ADDRESS>), val-13(<IP_ADDRESS>), val-14(<IP_ADDRESS>), val-18(<IP_ADDRESS>), val-20(<IP_ADDRESS>), val-24(<IP_ADDRESS>), val-27(<IP_ADDRESS>), val-29(<IP_ADDRESS>), ]Updated Config: "dev_rustielin_pull_5150"
` failed: `Failed to run experiment: Failed to generate traffic: query_sequence_numbers on http://<IP_ADDRESS>:8080/ for faucet account failed: [http://<IP_ADDRESS>:8080/] get_accounts failed: JsonRpcError { code: -32601, message: "Method not found", data: None } `

Related PRs
(If this PR adds or changes functionality, please take some time to update the docs at https://github.com/libra/website, and link to your PR here.)

This looks cool.
I think perhaps we should rename names of validators from experiment name, e.g.
Experiment `Updating [val-21(<IP_ADDRESS>), val-15(<IP_ADDRESS>), val-8(<IP_ADDRESS>), val-28(<IP_ADDRESS>), val-0(<IP_ADDRESS>), val-4(<IP_ADDRESS>), val-9(<IP_ADDRESS>), val-1(<IP_ADDRESS>), val-25(<IP_ADDRESS>), val-26(<IP_ADDRESS>), val-19(<IP_ADDRESS>), val-23(<IP_ADDRESS>), val-16(<IP_ADDRESS>), val-17(<IP_ADDRESS>), val-2(<IP_ADDRESS>), val-3(<IP_ADDRESS>), val-5(<IP_ADDRESS>), val-6(<IP_ADDRESS>), val-7(<IP_ADDRESS>), val-10(<IP_ADDRESS>), val-11(<IP_ADDRESS>), val-12(<IP_ADDRESS>), val-13(<IP_ADDRESS>), val-14(<IP_ADDRESS>), val-18(<IP_ADDRESS>), val-20(<IP_ADDRESS>), val-24(<IP_ADDRESS>), val-27(<IP_ADDRESS>), val-29(<IP_ADDRESS>), ]Updated

The part with list of validators/ips can probably be just an info! in the experiment, no need to show it to user

@bors-libra r+

:pushpin: Commit 1592c13 has been approved by andll

:hourglass: Testing commit 1592c13fe8d1b11ba46410ec2a9a07b7c573a100 with merge 17f63538c2965506f6e041d66c164a0caed9d021...

:sunny: Test successful - checks-actions_land_blocking_test, checks-circle_commit_workflow
Approved by: andll
Pushing 17f63538c2965506f6e041d66c164a0caed9d021 to master...


Update to more recent npmlazy 1.14

Would be nice if this Dockerfile its hub.docker.com repo were updated to the last npm_lazy release 1.14.0.
https://github.com/mixu/npm_lazy/releases
Thanks!

@langri-sha Any updates on this? I'm trying to use npm_lazy but the current published image is super old.


LiveView Configuration Issue - LiveView session was misconfigured or the user token is outdated

Has anyone been able to get this working with LiveView or have any recommendations as to how I might debug this issue...
I am starting with https://github.com/dersnek/chirp as a working application
I get the following errors after setting everything up per the instructions... it seems like everything is as it should be
[debug] LiveView session was misconfigured or the user token is outdated.

1) Ensure your session configuration in your endpoint is in a module attribute:

    @session_options [
      ...
    ]

2) Change the `plug Plug.Session` to use said attribute:

    plug Plug.Session, @session_options

3) Also pass the `@session_options` to your LiveView socket:

    socket "/live", Phoenix.LiveView.Socket,
      websocket: [connect_info: [session: @session_options]]

4) Define the CSRF meta tag inside the `<head>` tag in your layout:

    <%= csrf_meta_tag() %>

5) Pass it forward in your app.js:

    let csrfToken = document.querySelector("meta[name='csrf-token']").getAttribute("content");
    let liveSocket = new LiveSocket("/live", Socket, {params: {_csrf_token: csrfToken}});

[info] REFUSED CONNECTION TO Phoenix.LiveView.Socket in 452µs
  Transport: :websocket
  Serializer: Phoenix.Socket.V2.JSONSerializer
  Parameters: %{"_csrf_token" => "Kzo_DhAfHjhVMH19RkUJfxEHICQcIQdqfNqFJyuA9ADH26DLUuWfnbQG", "vsn" => "2.0.0"}

My endpoint.ex file has the following sections:
  @session_options [
    store: PlugRailsCookieSessionStore,
    key: "_chirp_key",
    secure: true,
    signing_with_salt: true,
    signing_salt: "signed cookie",
    encrypt: true,
    encryption_salt: "signed encrypted cookie",
    key_iterations: 1000,
    key_length: 64,
    key_digest: :sha,
    serializer: Poison,
  ]
  ]
  socket "/socket", ChirpWeb.UserSocket,
    websocket: true,
    longpoll: false

  socket "/live", Phoenix.LiveView.Socket, websocket: [ connect_info: [session: @session_options]]

  plug Plug.MethodOverride
  plug Plug.Head
  plug Plug.Session, @session_options
  plug ChirpWeb.Router

The main issue appears to have been using secure: true locally without https. Removing that allows for everything to be working as a standalone. I did learn that my rails 5.2 config has a the signing secret created with :aes_256_cbc and the secret with :aes_256_gcm so figuring that out next.


Missing ability to connect to EXO with serviceprincipal

As legacy authentication for Exchange Online will be unusable nearing the end of next year we need to have a way to authenticate to EXO using a service principal the way Azure/MSOL already support. Can this be added in?

@officedocsbot assign @yogkumgit

@jopgommans Which article are you reffering to? Could you please link it?

@jopgommans We appreciate any feedback that improves the content in the Microsoft docs but we haven't had any response from you so we are going to close this issue.
Please feel free to re-open this issue if you see an improvement is required in the specific area of this documentation. We will certainly update this documentation with that valuable information. Thanks again, for taking out some time to open the issue. Appreciate and encourage you to do the same in future also.


Bad character in module's routing file

See this comment here: https://github.com/angular/angular-cli/commit/ba668a8862725f894da3f9608871756507096bb9#commitcomment-18977226 from @intellix
at this line: https://github.com/angular/angular-cli/blob/master/packages/angular-cli/blueprints/module/files/path/name.routing.ts#L5
RouterModule.forChild(<%= camelizedModuleName %>˝Routes);
There is this character ˝ to much! :)

#2107

export const testRouting = RouterModule.forChild(test˝Routes);
results in ng g module test --routing


Parse Error

I am having this problem.. how I can solve it...?

Errors like this occur when the decompiler failed to create parsable Java code. This prevents right-click context menus from working since there isn't a way to figure out what you're clicking on. However this only applies to decompile mode. You can switch to table mode to bypass the need for valid decompilation.
More info: https://www.coley.software/Recaf-documentation/use-modes.html


Links in api.yaml to other HTTP methods don't resolve in Elements or Studio (but do in Studio edit mode)

Describe the bug
Links in api.yaml to other HTTP methods don't resolve in Elements (but do in Studio)
To Reproduce
Our Studio docs page
Our Elements docs page
The link in the text "Verifications are generally processed asynchronously: they will be created and returned in the response body without any reports." will not resolve properly, though it does in the "Edit" mode of Stoplight Studio.
Also fails to resolve on Stoplight Elements.
https://www.loom.com/share/310f6e7a6683494b9ab5dee47266dce3
Expected behavior
Link is expected to resolve.

@lottamus Another one for you.

Hello!
Our team recently took over the stewardship of the elements repo.
Due the volume of un-triaged issues that are pretty old and the time it would take us to get caught up on all of them, we kindly ask that you open up a new ticket if this is still an issue you're struggling with.
If you open a new issue please include:

clear steps to reproduce that include example OAS specs, application setup, code, etc.
the error or issue you are experiencing
the expected behavior

We will be triaging any new tickets in a timely manner.
Thank you for understanding!


Add lock flags for fcntl, flock & for idn2

As discussed here https://github.com/neomutt/neomutt/issues/1228#issuecomment-392265946

I'll have a look at travis & see if I can figure out this & will also add idn2 flag after your comment here https://github.com/neomutt/neomutt/issues/1228#issuecomment-392320195

Travis fails on brew link python@2 command here https://travis-ci.org/neomutt/homebrew-neomutt/jobs/384098400#L4603-L4615 which seems like a dependency of dependency

@jindraj @sharktamer @alexpearce
Does this look OK to you?
Do you have any ideas how to stop Travis failing?
How do you feel about adding Lua support to the upstream Formula?

Ah, well.  Worth a try.

@flatcap thanks, but I forgot to remove this commit https://github.com/neomutt/homebrew-neomutt/pull/40/commits/eee3b8d7e41b031be0d2a07ac20f751b8e005cc1 was waiting for feedback :) will open a PR to remove it since it's useless

will open a PR

There's no need.  If you're happy with the change, commit it directly to master.


Stylus

Any plans to add Jade and Stylus as options? generator-gulp-angular had those so I'm sad to see they're not included.

Stylus will be present in the next release.
But we have not started to include Html preprocessor.

Stylus added in 0.5.4 😃


Certain reactions schemes will cause the script to yield errors.

I mass downloaded reactions from scifinder and ended up getting a Unicode Decode Error:
"'charmap' codec can't decode byte 0x81 in position 671: character maps to ".
My assumption is there is some sort of special character but looking at the position in the rdf file shows nothing of interest.

Hm, interesting. Sounds like an encoding issue. It might be trying to read utf-8 type and maybe your files were saved as a different encoding depending on your region? Does that happen with every file, or only certain ones?
I don't supposed you could share a file (perhaps a "dummy" search if there is confidentiality)? You could share via the mail function, that should furthermore reduce any potential issues with Scifinder.

This is only a prelim work so I'm fine with sending the rdf file (attached).
The output still does give a "fixed" rdf file, but it cannot seem to give a
csv.
On Mon, Jun 10, 2024 at 3:15 PM Alex M. @.***> wrote:

Hm, interesting. Sounds like an encoding issue. It might be trying to read
utf-8 type and maybe your files were saved as a different encoding
depending on your region? Does that happen with every file, or only certain
ones?
I don't supposed you could share a file (perhaps a "dummy" search if there
is confidentiality)? You could share via the mail function, that should
furthermore reduce any potential issues with Scifinder.
—
Reply to this email directly, view it on GitHub
https://github.com/DocMinus/chem-rdf-fixer/issues/7#issuecomment-2159205068,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BIRLR5NZQJE3AALXZSGSVX3ZGYCN5AVCNFSM6AAAAABJC5OPJSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJZGIYDKMBWHA
.
You are receiving this because you authored the thread.Message ID:
@.***>

Maybe I am blind, but I don't see anything attached...?

Hmm, thats odd then because I can see it from my end (I’m not at work right now so it’s on my IPhone). I’m not sure gmail supports sending rdf files then, and I know for a fact that github doesn’t as I’ve tried. Do you have an alternative method to send it?On Jun 11, 2024, at 03:30, Alex M. @.***> wrote:
Maybe I am blind, but I don't see anything attached...?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

It occurred to me that you are viewing this on github, which does not support rdf files and why I sent it via email, please check your inbox

That didn't work. Besides, come to think of it, I wouldn't open a file via email anyway.
I suggest the following option:
see if you can add the text content of the file via copy/paste into here. alternatively, create a pull request and have the file attached there.

PhAz_alkyne501(txt).txt
I converted the file into a txt file to see if this can work, there's too much to paste into a comment. I will send the pull request if this doesn't work either

got it now.
will be busy the next few days, not sure when I can follow this up.
On Wed, Jun 12, 2024 at 4:48 PM shaqoatmeal420 @.***>
wrote:

PhAz_alkyne501(txt).txt
https://github.com/user-attachments/files/15806885/PhAz_alkyne501.txt.txt
I converted the file into a txt file to see if this can work, there's too
much to paste into a comment. I will send the pull request if this doesn't
work either
—
Reply to this email directly, view it on GitHub
https://github.com/DocMinus/chem-rdf-fixer/issues/7#issuecomment-2163234213,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKSF7X7SXQDRVKIDAFOW3QTZHBNTXAVCNFSM6AAAAABJC5OPJSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRTGIZTIMRRGM
.
You are receiving this because you were assigned.Message ID:
@.***>

I'm in no rush

check this branch, it should work now.
https://github.com/DocMinus/chem-rdf-fixer/tree/fix/issue-%237
I will update the main branch and the pypi version the next few days.

Latest version now pushed to main and pypi updated.


Improve test coverage (>85%)

make sure it shows on the README badge

#139 scraper coverage >95%

91% as of 3/12


Terms and privacy warning overlays window buttons and can't be dismissed

I know this is partially a TweetDeck issue, but I started running ModernDeck today and I get the infamous "Every account you're using... agree to the updated Terms and Privacy..." dialog.
Unfortunately in ModernDeck installed from Flathub, there's no way to dismiss this dialog, and I have never figured out where on twitter.com I'm supposed to "agree" to these conditions.

It does not prevent me from using ModernDeck, but I would really appreciate some tips on how to get rid of it!

Hey, this problem occurs, like it says, you are linked to accounts which haven't agreed to Twitter's updated TOS / Privacy Policy, try logging into those accounts on Twitter Web and see if it prompts you to accept the updated TOS / Privacy Policy.
If this solves your problem, feel free to close the issue.

Make sure your app is up to date @maximuskowalski as that bug should be fixed

Thank you, looks like it's a version behind on flathub, showing 9.44 and appimage is 9.45 - so I'll just either wait it out or flip to appimage.

I have logged into the accounts in question to ensure they had accepted TOS, but I can't confirm that it worked because I have the same issue as @maximuskowalski. Once I can update ModernDeck I'll report back (I'm also using flathub).

@maximuskowalski 9.4.5 was supposed to be published on Flathub but the build borked. I'm not actually the one that puts out the Flathub releases (but I do try to coordinate and help where I can)


Feature request: Lighter DOM Size

I'm using a pretty simple SVG with 1 path for fullSymbol & emptySymbol, for example:
const StarFull = props =>
	<svg width="1em" height="1em" viewBox="0 0 19 19" {...props}>
		<path d="M3.844 18.458a.507.507 0 01-.302-.097.524.524 0 01-.206-.506l.983-5.76L.156 8.018a.51.51 0 01-.129-.53.517.517 0 01.415-.352l5.752-.841 2.573-5.239A.513.513 0 019.23.77a.52.52 0 01.463.287l2.572 5.239 5.756.841a.517.517 0 01.282.882l-4.162 4.077.982 5.76a.521.521 0 01-.205.506.509.509 0 01-.544.04l-5.148-2.718-5.15 2.718a.51.51 0 01-.233.057z" />
	</svg>;

const StarEmpty = (props) =>
	<svg width="1em" height="1em" viewBox="0 0 19 19" {...props}>
		<path d="M15.148 18.462a.494.494 0 01-.237-.061l-5.143-2.718L4.622 18.4a.509.509 0 01-.543-.04.521.521 0 01-.206-.506l.982-5.76-4.16-4.077a.51.51 0 01-.13-.53.517.517 0 01.415-.352l5.754-.841L9.31 1.056A.513.513 0 019.772.77a.52.52 0 01.463.287l2.571 5.239 5.754.84a.517.517 0 01.286.883l-4.16 4.077.98 5.76a.521.521 0 01-.518.607zM2.077 7.892l3.65 3.553c.<IP_ADDRESS>3.151.46l-.863 5.018 4.514-2.37a.511.511 0 01.484 0l4.51 2.37-.863-5.018a.518.518 0 01.15-.46l3.652-3.554-5.043-.732a.51.51 0 01-.39-.285L9.772 2.308 7.514 6.874a.516.516 0 01-.39.285l-5.047.732z" />
	</svg>;

<Rating
  initialRating={3.5}
  emptySymbol={<StarEmpty />}
  fullSymbol={<StarFull />}
  fractions={2}
  readonly
/>

This RatingStars component outputs 45 nodes into the DOM 😮  Let's say you have a page of 30 items that each displayed a rating — the total DOM size is already 1,350 nodes without anything else on the page. You're page will get flagged every time by Lighthouse because it's going to exceed the recommended 1500 nodes.
Amazon uses image sprites but they get away with just 5 nodes per rating element.

Hi @rpastorelle!
I see your point and it seems a fair concern.
This component aimed to be as the most flexible rating. But with flexibility probably comes less efficiency, or in our specific case, more elements/nodes 😄.
I am open to optimizations and I can spot some options, specially for read-only ratings. So I will keep this issue open just in case someone (or myself) had desire and time to contribute.
Now some insights and discoveries...
Rating structure
The rating symbols are composed by:

A parent span element that wraps the entire symbol. Used as a container of each symbol
Two children span elements that wrap the empty and the full symbols. Basically used to manage the visibility and the fraction of the symbol being shown
The variable number of elements of the empty and full elements

There are 3 extra wrapper elements for each symbol.
The formula to know the exact number of elements generated for a rating of n symbols would be something like this:
n symbols * (wrapper elements + number of elements emptySymbol + number of elements fullSymbol)
In your example:
5 * (3 + 2 + 2) = 35
(I don't know how you got 45 😅)
Amazon rating structure
I checked two types of star ratings, let's say the input and the read-only.
In the input rating each symbol is composed by two elements (a label and an input). So for a 5 star rating we have 10 elements.

        <input name="star-rating" type="radio" value="1" class="radio-btn hide">

        <input name="star-rating" type="radio" value="2" class="radio-btn hide">

        <input name="star-rating" type="radio" value="3" class="radio-btn hide">

        <input name="star-rating" type="radio" value="4" class="radio-btn hide">

        <input name="star-rating" type="radio" value="5" class="radio-btn hide">

The read-only symbol is a pre-built image with the corresponding rating selected.

  4.7 out of 5 stars

It uses the following sprite:

As you can see in the image they have all ratings from 0 to 5 (with halves too 😉).
As a curiosity
I created a Lighthouse report on the same Amazon page I was looking into the read-only stars and here is what I got:

A code sandbox used for the investigation.


Personal PR since I'm working on this from my forked version.

Since I'm working on this from my fork, I'll make prs and then merge  them myself instead of directly commiting to the repo.

Black will now run before flake8 so you won't see many of flake8 lints about PEP8


lodash: the signatures of the method _.find (and of the alias _.detec...

...t) have been changed
https://lodash.com/docs#find
https://lodash.com/docs#_ (description of the chainable wrapper)

thanks mate!


strange behavior in scipy.stats.spearmanr with bool and NaNs

scipy.stats.spearmanr gives very wrong results in the specific case that one argument is bool and the other contains NaNs.
Reproducing code example:
>>> # weird behavior in scipy.stats.spearmanr with bools and nans
... import numpy as np
>>> import scipy.stats
>>>
>>> n = 100
>>> x = np.random.rand(n)
>>> m = np.random.rand(n)>0.7
>>>
>>> # bool against float, no NaNs
...
>>> a = (x > .5)
>>> b = np.array(x)
>>> scipy.stats.spearmanr(a, b, nan_policy='omit').correlation
0.86172748443213909
>>>
>>> # bool against float with NaNs
...
>>> a = (x > .5)
>>> b = np.array(x)
>>> b[m] = np.nan
>>> scipy.stats.spearmanr(a, b, nan_policy='omit').correlation
0.0061740457463813625
>>>
>>> # int against float with NaNs
...
>>> a = (x > .5).astype(np.int32)
>>> b = np.array(x)
>>> b[m] = np.nan
>>> scipy.stats.spearmanr(a, b, nan_policy='omit').correlation
0.87068493150684934

Scipy/Numpy/Python version information:
Python 3.6.1 |Anaconda 4.4.0 (64-bit)| (default, May 11 2017, 13:09:58)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys, scipy, numpy
>>> numpy.__version__
'1.12.1'
>>> scipy.__version__
'0.19.0'

and
Python 3.6.2 |Continuum Analytics, Inc.| (default, Jul 20 2017, 13:51:32)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy, scipy
>>> scipy.__version__
'0.19.1'
>>> numpy.__version__
'1.13.1'
>>>

What should be the expected behavior here? I would think that it wouldn't make since to do a correlation with a boolean and numeric type, so the simplest solution would be to raise if the either both types are not numeric or if both types are not boolean.

With gh-9204 the values for these cases are all identical


Suggested updates to PBCore translator

Have option to pass user-defined values into the PBCore document. For example: allow the user to define the Generation or Location for a document

PBCore 1.2 XSD doesn't appear to be correct. Might need to be this: http://v2.pbcore.org/xsd/pbcore-1.2.xsd

PBCore 1.2 XSD doesn't appear to be correct.

Please 1 ticket per issue, else it is difficult to track.
Copied in https://github.com/MediaArea/MediaInfoLib/issues/851
I'll delete the comment.

Have option to pass user-defined values into the PBCore document.

I see a technical difference between passing used-defined values not conflicting with file content or MediaInfo existing fields (including file name), and adding data very specific to an output format.
For example we added support of external metadata for EBUCore, this metadata is not in the file (sponsor needed to inject e.g. identifier, Physical condition, Copyright holder... out of the "ebucore:format" element used by MediaInfo), we did same for NISO Z39.87 output, injecting some EBUCore XML in "mix:Extension" NISO element.
In both cases, we are not replacing content, we are adding content in a separate place, so I need to know if your goal is similar of if you want to replace content. As an example, changing location is not specific to PBCore, as the file name is handled everywhere by MediaInfo. For file name ("Location" in PBCore) replacement, we already have external API for file name information " --File_FileName=(YourPreferedFileNameHere)" but it is limited to "by buffer" interface, when no file name is available (empty file name); limitation is arbitrary (we needed it only in a corner case), we could remove this limitation and permit to change the file name in all cases.
For any other value, we need to check your exact need (adding a pure PBCore thing because it is very specific to PBCore, or generic to MediaInfo so all outputs can use the same API), please provide some examples of what your are trying to do, beside the file name thing.
I flag "SponsorNeeded" as it is a specific need and would not be implemented with free support only.

Generation

I don't see this word in PBCore output.

checked with @dericed about details, I add a generic API for injecting data in MI report and @dericed manages PBCore output (mapping between MI report and PBCore output), let's go.


HDDS-4954. Add replicaIndex to the RPC protocols

What changes were proposed in this pull request?
This is the initial patch to support differentiating between multiple container replicas.
With RATIS/Closed containers all the container replicas are exactly the same but with EC they will be different. We need an additional index for each container replicas to know how it should be handled.
This patch is part of bigger effort to show end2end solution for container indexes. The full picture can be found at https://github.com/elek/ozone/tree/ec with all of these improvements:

replicaIndexes are introduced in proto files and Pipeline
EC repliation type is added
SCM allocates datanodes for new EC pipelines (hard coded 1 as of now)
Client sends the instanceIndex during chunkWrites (required for container creation)
Container files on datanodes persist the instance index
ContainerReports contains the instanceIndex
SCM adds the new field the ContainerReplica instances
RelicationManager is extended to handle EC type containers (with a stub implementation)

This doesn't include all of these modifications, it is just the very first, separated step where the proto files and Pipeline are modified.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-4954
How was this patch tested?
The patch is tested together with the full end2end code (https://github.com/elek/ozone/tree/ec)
Started Ozone cluster and created a key:
./ozone sh key put -t EC /vol1/bucket1/key1 ../README.md

Checked the container yaml file if it contains the instandeIndex.
Checked the SCM log for the message from ReplicationMananger:
2021-03-17 12:30:02,754 INFO  container.ReplicationManager (ReplicationManager.java:processContainer(329)) - TODO: replication management for EC containers are missing. Container #1, <IP_ADDRESS> #7

Thanks for separating the replication type changes into separate JIRA.
Please add the replication index to org.apache.hadoop.hdds.client.BlockID and org.apache.hadoop.hdds.client.ContainerBlockID
This will be needed when we create container from client, we anyway pass blockID to DN. If index available there, then DN know which index.
Current patch seems t be covering the protocols only from SCM and DN. The above change can help to cover client needed changes.
We also discussed yesterday that, let's not make index as the property to chunkInfo. I am more comfortable if we make that as property to ContainerBlockID instead. Thanks for removing them in latest patch.

We also discussed yesterday that, let's not make index as the property to chunkInfo. I am more comfortable if we make that as property to ContainerBlockID instead. Thanks for removing them in latest patch.

Sure, I removed it, but during my git rebase it's not fully moved (I manage 10-12 patches today and trying to sync them one-by-one with different PRs)
Now it should be good:
/**
 * Block ID that uniquely identify a block in Datanode.
 */
message DatanodeBlockID {
  required int64 containerID = 1;
  required int64 localID = 2;
  optional uint64 blockCommitSequenceId = 3 [default = 0];
  optional int32 replicaIndex = 4;
}

Please add the replication index to org.apache.hadoop.hdds.client.BlockID and org.apache.hadoop.hdds.client.ContainerBlockID
This will be needed when we create container from client, we anyway pass blockID to DN. If index available there, then DN know which index.

org.apache.hadoop.hdds.client.ContainerBlockID doesn't need any modification as it contains the generic id (which points to the replication set) and pipelines (where each element can have replicaIndex information)
Same is true for BlockID some cases we need this information in some cases we don't I propose to add the information only when it's required. We will see it in the next patches with adding the client side / datanode side changes, and we can extend BlockID and create a BlockInstanceId when required...

Thanks the review @umamaheswararao, I updated the improved version, PTAL...

@elek thanks for the update. I will take a look at the latest patch.

pipelines (where each element can have replicaIndex information)

this works as well for me.

Can we avoid unrelated import organization changes if possible? Otherwise I am +1 on the changes.

Sorry, I just realized that I didn't push my changes which restored the import orders. Should be there now...

Merging it as CI is green. Thanks the review @umamaheswararao


[Bug] State Snapshots fail with EOF

Repro With Devnet Nodes
Generate a state snapshot
Try to create a state snapshot, accessing the backup-service of a remote.
cargo run --release -p backup-cli --bin db-backup -- one-shot backup --backup-service-address http://<IP_ADDRESS>:6186 state-snapshot --state-version 39170318 local-fs --dir ~/epoch-archive/
Fails with error:
2021-02-15T15:09:13.011788Z [main] ERROR storage/backup/backup-cli/src/bin/db-backup.rs:125 main_impl() failed: State snapshot backup failed: request or response body error: error reading a body from connection: unexpected EOF during chunk size line
Remote configs:
The remote machine used for backups needs a node.yaml configuration with backup-service-address accessible by the public (ip: <IP_ADDRESS>:6186):
storage:
  address: "<IP_ADDRESS>:6666"
  backup_service_address: "<IP_ADDRESS>:6186"
  dir: db
  grpc_max_receive_len: 100000000
  prune_window: 100000
  timeout_ms: 30000

Solved. This was a pruning issue with devnet servers, which had pruned the height being requested.


⚠ St. Lawrence Global Observatory CKAN has degraded performance

In 4623ee9, St. Lawrence Global Observatory CKAN (https://catalogue.ogsl.ca/en/) experienced degraded performance:

HTTP code: 200
Response time: 7079 ms

Resolved: St. Lawrence Global Observatory CKAN performance has improved in 3f27c66 after 5 minutes.


Updates job priority documentation

A docblock in the Job interface still refers to priority as being from 1 to 3.

Perfect! Thanks a lot.


Introduce server-side junit integration tests

sample:
https://github.com/adobe/aem-project-archetype/blob/master/src/main/archetype/it.tests/src/main/java/it/tests/HelloWorldModelServerSideTest.java
docs:
https://sling.apache.org/documentation/bundles/org-apache-sling-junit-bundles.html

not sure if needed, closing, lack of interest


[SUPPORT] Dynamic custom fields

I'm submitting a ...  (check one with "x")
[ ] bug report => search github for a similar issue or PR before submitting
[ ] feature request
[X] support request

Hello, since a long time I didn't play with formly but today I would like to refactor my code ;)
Today, to be able to reuse some field configurations, I am creating services to create them. Then, the form is generated by formly.
But I have some troubles when I want to react dynamically on the "not yet generated form" in my services. I know it's not a good pratice so I started to put everything in field component.
But now, because the fields configuration is not specified from the top form, I have some troubles to modify a field on the fly.
Here a simple example to explain my issue : https://stackblitz.com/edit/dynamic-field-v4
The select-requester-field should be a field that can generate dynamically the fieldGroups that will contain the select-user-field.
The select-user-field is just an alias of a select with default internal configurations. There is two solutions, the select-user-field type could be overwrited by the select type OR the select-user-field could contains the select type and overwrite come configurations.
The goal is not to define a form from the top but be able re-use some "smart" fields that can generate dynamically sub form.
I think it's easy but I can't make it work with the formlyFormBuilder or formlyConfig.getMergedField and I would like the best way of doing.
For the moment I have the ExpressionChangedAfterItHasBeenCheckedError issue :)
I would be interested by a v4 solution and a v5 solution if there are different :)

for v5 check my answer https://github.com/formly-js/ngx-formly/issues/1091#issuecomment-411516592

@aitboudad I adapted my example with the v5 : https://stackblitz.com/edit/dynamic-field-v5
But Error: Cannot find control with unspecified name attribute :s
I don't really how the getMergedField(field) is working

getMergedField it assign the default option to the field if not set, the current solution doens't support nested form using fieldGroup which require to be built first.
for select-requester-field you can pass fieldGroup in defaultOptions or use buildForm see

using defaultOptions: https://stackblitz.com/edit/dynamic-field-v5-yrhqbi
using buildForm: https://stackblitz.com/edit/dynamic-field-v5-dgqn62 (not better since it use an internal function which I don't want to make public for now)

Ok thank you, I'll have to use the second way then because the formGroup will not always contain my component select-user-field, sometimes it could be another one...so I cannot define default configurations

ok sound good, remind me later to make it public :)

@aitboudad Hello, monday morning and I would like to refactor my code with your solution ;)
But I still have an issue when I put a default value... ExpressionChangedAfterItHasBeenCheckedError
See : https://stackblitz.com/edit/dynamic-field-v5-8fwrjl?file=src/app/select-requester-field.component.ts

I am really stuck with that :( I don't know how could I update the model earlier...

@aitboudad Something else, on your exmaple https://stackblitz.com/edit/dynamic-field-v5-yrhqbi?file=src%2Fapp%2Fselect-user-field.component.ts the select-user-field doesn't display the label and the description....but the properties are presents....

ExpressionChangedAfterItHasBeenCheckedError fixed by setTimeout, thx : https://stackblitz.com/edit/dynamic-field-v5-of7jt7?file=src/app/select-requester-field.component.ts
I don't really know how to use the lifecycleFormlyExtension to solve this. I mean, I'll have to put my code oustide of my component...which is my concern for the next point

The goal is to specialize the component then I have some questions because I am a bit lost with this getMergedField...:

What this getMergedField is used for ? Merge configurations + build de fields ?
Why did you add wrapper: undefined ?
I would like to specify the expressionProperties in my custom field and not on the parent...the goal is to re-use this field and not force all developers to duplicate the expressionProperties. In this way, they can just specify the type of the field and the key, then the field will manage all dynamic stuff ;).

I would like to use this new way but I feel that the old way with a formly-form ( https://stackblitz.com/edit/dynamic-field-v5-nvzdhy?file=src/app/select-user-field.component.ts) or with a this.formlyBuilder.buildForm (https://stackblitz.com/edit/dynamic-field-v5-jfnrfn?file=src/app/select-user-field.component.ts) are still easier to put in place

I don't really know how to use the lifecycleFormlyExtension to solve this. I mean, I'll have to put my code oustide of my component...which is my concern for the next point

you can define those hooks in your component using a static function, I'll provide you an example!

What this getMergedField is used for ? Merge configurations + build fields ?

only merge configurations

Why did you add wrapper: undefined ?

in getMergedField when wrapper is undefined we set the default.

I would like to specify the expressionProperties in my custom field and not on the parent...the goal is to re-use this field and not force all developers to duplicate the expressionProperties. In this way, they can just specify the type of the field and the key, then the field will manage all dynamic stuff ;).

it can be done using FormlyExtension

see https://stackblitz.com/edit/dynamic-field-v5-bgq6dz
for expressionProperties I'll look into it later.

Ok thanks, I understand now how it it working.
But I am still not enthusiast with the implementation. It is a bit "cheating" with the component pattern ;). You put code in the component place but it doesn't belong to the component. It's still a global configuration that we have to create...
I agree with the last comment of @FritzHerbers in #1109, It is a really good idea to have a formly lifecycle but it should be specify in the same way as angular specify his licycle like OnInit, AfterViewInit, etc..I can understand it is not possible for the moment :s
Anyway, thank a lot for you work and your reactivity

seems possible :), just reopened  #1109. stay tuned!

I am really enthusiast about the formly lifecycle.
Related to this issue, I still have some difficulties to use the expression properties even on the parent..

in order to take account of expression properties the field should be part of the main field so instead I suggest to follow the same way that you've done for select-requester-field rely on fieldGroup.

Ok I'll wait your new lifecyle :(
Now, even if I use the static prePopulate, I have an issue with the observable configuration like the translate for example. It's injected as a service so not possible to use it in the static method onPopulate...

done in beta.5 see https://stackblitz.com/edit/dynamic-field-v5-bgq6dz


System.Typeload Exception: Could not load type TechTalk.Specflow.Infrastructre.IBindingInstanceResolver from assemble TechTalk.Specflow Version <IP_ADDRESS>

I was having trouble using Specflow with my project that uses Autofac. After searching and trying your proposed solution I had this issue. I downloaded you sample to test but it has same issue

Issue is because latest version of SpecFlow <IP_ADDRESS> doesnt make use of the said library. So either this plugin needs to be updated or anyone (who is using it - like me) will have to downgrade to <IP_ADDRESS>

I am running into the same issue, is this currently being addressed?


Time since last upgrade is wrong

For some reasons my Model 3 has not updated since 23 January 2021. (already opened a case with Tesla...)
Today it is March 1st and the teslaMate updates dashboard shows "time since last update: 2 weeks"

Expected behavior

How to reproduce it (as minimally and precisely as possible):

Relevant entries from the logs

paste logs here

Screenshots

Data

Environment

TeslaMate version:
Type of installation: Docker or Manual
OS TeslaMate is installed on:
User OS & Browser:
Others:

Can you get the latest updates that were stored in the database and post them here?
$ docker-compose exec database psql teslamate teslamate -c \
      "COPY (  select * from updates where start_date > '2020-10-01') TO STDOUT WITH CSV HEADER" > updates.csv

Hi, i run the script:
id,start_date,end_date,version,car_id
1,2020-10-13 19:33:30.863,2020-10-13 19:33:30.863,2020.36.11 ccacdb181f16,1
2,2020-10-13 20:06:52.418,2020-10-13 20:20:49.019,2020.40.4 ffa5212dba76,1
3,2020-10-22 20:27:08.888,2020-10-22 20:38:41.395,2020.40.8 9744d0c0d399,1
4,2020-11-03 07:31:17.072,2020-11-03 07:48:48.24,2020.40.9 507cb67a95c8,1
5,2020-11-09 07:32:06.38,2020-11-09 08:06:37.457,2<IP_ADDRESS> 955dc1dd145e,1
6,2020-11-22 16:27:33.555,2020-11-22 16:27:33.555,2020.44.15 19675f719e84,1
7,2020-12-02 21:06:51.867,2020-12-02 21:16:27.218,2020.44.25 0ba7ea36671e,1
8,2020-12-11 08:15:23.33,2020-12-11 08:28:36.381,2020.48.10 f8900cddd03a,1
9,2020-12-22 07:40:56.35,2020-12-22 08:01:32.304,2<IP_ADDRESS> 3095698c8a55,1
10,2020-12-28 06:25:27.332,2020-12-28 06:50:29.932,2020.48.26 e3178ea250ba,1
11,2021-01-07 07:12:27.513,2021-01-07 07:31:56.987,2020.48.30 040912887bad,1
12,2021-01-23 22:14:23.188,2021-01-23 22:35:43.092,2<IP_ADDRESS> 842a95863251,1

Is this just a confusing column title? I think it should be changed to "Since Previous Update" ("Last" means the most recent one, but the column shows the duration between updates).

Yup, that is correct. It refers to the duration between updates, so in your case the duration between "2020.48.30" and "2<IP_ADDRESS>".


Fixes exception with map saving

This exception would occur due to the map saving system (F5) taking the current name of the server and saving over the current server's map. This would cause an accessdenied exception due to the map being in use while the server is running presumably.
This fixes that problem and adds a timestamp added when saving maps using the debug verb (to prevent any overwriting of maps)

There was another exception too, that I accidentally fixed and thought it was related to the first one fixed by this PR so I removed it, but it comes from not having the folder maps are saved into. This properly builds the directory so the exception is fixed.


Provide type definition for TypeScript

Could you provide type definition for TypeScript?
My idea
Add index.d.ts.
import middy from '@middy/core'

type Type = 'string' | 'int' | 'float' | 'bool'
type Config = string | [string] | [string, Type] | [string, Type, string]

export interface Options {
  names: Record<string, Config>
  cache?: boolean
  cacheExpiryInMillis?: number
  setToContext?: boolean
}

declare const env: middy.Middleware<Options, any, any>

export default env

Then, add types field to pacakge.json.
{
  "types": "index.d.ts"
}

See also

https://www.typescriptlang.org/docs/handbook/declaration-files/introduction.html
https://github.com/middyjs/middy/blob/master/packages/db-manager/index.d.ts

I don't think context can be typed for middy 2.x because it is hard-coded as LambdaContext, only the request event can be typed.
I think this requires a minor update to add an option to distribute into a new event property, for example event.env, instead.

Please feel free to raise a PR.


--version flag added, fixes issue #581

Description
This PR fixes #583
Imports sys in argument_parser.py
Added --version flag

Closing this, as this feature has been added here - https://github.com/sparckles/robyn/pull/619


Label sector in show all bookmark panel

This issue is bound with issue #39
Bookmark panel has 2 sectors, the first one is labels.
Actually there is a link near the string Labels that do nothing, only change the text of link from cancel to edit and vice versa

Moving to #39, has to be fixed as part of that scope.


Use Clang on Windows

Node Version: All
Platform: Win 10
Compiler: Clang (clang-cli or clang)
Module: All

How can we use Clang on Windows to build native modules? I want to use a single open-source compiler for all the platforms. Writing compiler flags for each compiler is not efficient, and they also may have different performance.
This might be related to:

https://github.com/nodejs/node-v8/issues/163
https://github.com/nodejs/node/issues/19630

you could just not use node-gyp and do something manual in your package.json "build". node-gyp (& gyp) is designed to make it flexible across platforms but if you just want to enforce clang on your users then make your own build script that does everything and bypass node-gyp entirely.
I'm not sure gyp has full clang for windows support, does it @nodejs/gyp? it'd just be a bunch of hacks on hacks to make that work for an addon wouldn't it?

For now, it's kind of buggy, not production ready. Maybe after when we resolve https://github.com/nodejs/node/pull/35433.

For now, it's kind of buggy, not production ready. Maybe after when we resolve https://github.com/nodejs/node/pull/35433.

@rvagg could you elaborate on "you could just not use node-gyp and do something manual in your package.json"
I can use napi.h in my c++ files and use clang to generate a dll file on windows but I'm confused on what a .node module file is. Is there a way I can convert a dll to one?

@TrevorDev I can't give you an example off the top of my head, but there are plenty of Node.js addons that don't use node-gyp to build. cmake.js I think is doing their own thing. I've seen plain Makefile builds. The end goal is to build a library that can be dynamically loaded by node and .node is the convention for naming that binary libary. But how you build it is up to you. It just happens that node-gyp has integration in npm, but npm will also run scripts on install so just insert your build process in there. On windows, it's really just a .dll with a different name. I don't know the specifics of the build process but I think you can set an environment variable V to 1 and build someone's addon (via npm install or node-gyp rebuild or whatever) then gyp will show you a verbose compile output and you can see what it's doing. I assume this also works on Windows. You could learn from this process to make your own build script(s) that do what you want. Experiment and you may find a pattern that others can learn from!

Woot got it working 🚀, thanks for the info.
For anyone else, this command worked for me:
clang++ test.cpp -Inode_modules\node-addon-api\ -IC:\Users\username.cmake-js\node-x64\v10.16.3\include\node -o test.node -shared C:\Users\username.cmake-js\node-x64\v10.16.3\win-x64\node.lib

@TrevorDev I'm glad that you made it work.
I have been working on a new project called cmake-ts. It provides a unified interface for cross-compiling prebuilds for other operating systems (including Windows). So, stay tuned for that (it is still unstable).
https://github.com/EmbeddedEnterprises/cmake-ts/


[QHC-43] [BUG] Qblox module markers value hardcoded to 15 (RF-modules are the most affected)

Expected behavior
The marker value should be set to 0 if no output of the module is being used, 1 if only the O0 is being used, 2 if only the O1 is being used, or 2 if both O0 and O1 are being used, and after measuring, set to 0 again so we only send the actual gates. This depends on the circuit you are going to measure, and should be set at the Q1ASM program sent to the used sequencer.
Actual behavior
This marker value is hardcoded at 15 when executing the platform.initial_setup(), this opens all outputs in all modules even if you are just using one output of one module, this leads to all the other modules sending a signal at the given LO frequency, and so increasing noise or in some situations exciting other qubits.
Additional information
It is easy to reproduce, just take a module and send something in one output and read on the other one with a spectrum analyzer for example, you will notice that the given gate is executed and then there's a signal that still there even if the measurement has ended.
Source code
No response
Tracebacks
No response
System Information
Name: qililab
Version: 0.20.4
Summary: Fundamental package for fast characterization and calibration of quantum chips.
Home-page: https://github.com/qilimanjaro-tech/qililab
Author: Qilimanjaro Quantum Tech
Author-email<EMAIL_ADDRESS>License: Apache License 2.0
Location: /home/anavarro/miniconda3/envs/qililab-main/lib/python3.10/site-packages
Editable project location: /home/anavarro/qililab
Requires: h5py, lmfit, pandas, PyVISA-py, PyYAML, qblox-instruments, qcodes, qcodes-contrib-drivers, qibo, qiboconnection, qpysequence, rustworkx, tqdm, urllib3
Required-by:

Platform info:             Linux-5.15.0-86-generic-x86_64-with-glibc2.35
Python version:            3.10.11
PyVISA version:            0.5.2
QCodes version:            0.38.1
QCodes Contrib version:    0.18.0
Qblox Instrument version:  0.10.1
Qpysequence version:       0.9.3
Quantum Machines version:  1.1.3
Qibo version:              0.1.12.dev0

Existing GitHub issues

[X] I have searched existing GitHub issues to make sure the issue does not already exist.

QHC-43

This has been fixed now.


Training with new datasets

Thank you for sharing the evaluation code and pretrained models. I am interested in training and evaluating your method on a new dataset; do you plan to release the training source code in the near future?

Hi @kylelindgren , as mentioned in the README, the current training code is mixed up with some other projects. I am cleaning and summarizing the code.
I do plan to release the training code but it may take some time since it will come with other projects as well.
Anyway, for depth network training (stereo), you can refer to monodepth2 at the moment: https://github.com/nianticlabs/monodepth2
For others (optical flow training and monocular depth training), I will let you know when it is released but don't expect it will be available within these two months. Sorry about that.

Hi Huangying, thank you for the quick response and explanations. No worries about the release timing. I look forward to playing with your method when released.

@Huangying-Zhan is there any update about when the training code will be released? Thanks!

Please see anser in #35


Azure REST API and Templates browsers should use same left-nav hierarchy

Is your feature request related to a problem? Please describe.
Compare:
Azure template reference: https://docs.microsoft.com/azure/templates/
Azure REST API reference: https://docs.microsoft.com/rest/api/?view=Azure
Why is the left-nav hierarchy different? The template reference is strictly organized by resource provider. The REST API reference is organized by...well, I don't know what it is, frankly. Networking is split up with DNS and Traffic Manager separate, but a bunch of other top-level resource types grouped under 'virtual networks' (which is itself a resource type)
Describe the solution you'd like
Use the resource-provider hierarchy (as used on the template reference) on the REST API reference.
Describe alternatives you've considered
Nothing else makes any sense.
Additional context
Thanks for listening!

@mikekinsman could you help address this with the content partners?

@jtuliani whoa very late reply here.  I missed the ping back in July and I didn't come back around to it.  My apologies for the huge delay.   The accurate, but incomplete answer to your question, is that the two TOCs are maintained and built separately, which is why they deviate.  We automatically generate the REST content, which makes it much faster, but introduces some constraints on how the content is structured.  Going forward I think @rloutlaw and @dend might be able comment on the future of reference content structure from Azure.  Thanks for your feedback on how consistency for reference content across Azure is important for your usage of content.


add tests to test_append_if_schema_identical

addresses #103
Adds the following checks to the appended dataframe

has four rows
has the correct names in the correct order
has the correct cities in the correct order
has an equivalent schema to the target dataframe

side note - the dataframe append works with on source_df which has id, city, name and target_df which has id, name, city. Is this expected behavior? I was originally going to check if the result dataframe had the same schema as the source dataframe, but that failed because of the different column order.

that a SchemaMismatchError is raised when unidentical_df (with an additional string column) is appended to the source df

===
Let me know if I should split these into separate tests or if there others I should account for.

the dataframe append works with on source_df which has id, city, name and target_df which has id, name, city. Is this expected behavior?

Yep, this behavior seems correct.
LGTM


DOC: How to write my own short scripts within a fork to test some code?

Issue with current documentation:
I want to contribute to one of the libs and started a fork. Here are the steps I took:
I am trying to add a new feature but first need to experiment with it. I am unsure on how to get started writing some short scripts to use the libs.

I went into libs/experimental, libs/core, libs/community libs/langchain and ran poetry install in all of them.

I start an environment from libs/langchain with poetry shell

I created a file inside of it, made some short code:

from langchain.embeddings.sentence_transformer import SentenceTransformerEmbeddings
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import Chroma

# load the document and split it into chunks
loader = PyPDFLoader("./llm_in_a_flash_apple.pdf")
documents = loader.load_and_split()

# create the open-source embedding function
embedding_function = SentenceTransformerEmbeddings(model_name="all-MiniLM-L6-v2")

# load it into Chroma
db = Chroma.from_documents(documents, embedding_function)

# query it
query = "What did the president say about Ketanji Brown Jackson"
docs = db.similarity_search(query)

# print results
print(docs[0].page_content)

and got the following error:
Traceback (most recent call last):
  File "/Users/polo/langchain/libs/langchain/test.py", line 9, in <module>
    loader = PyPDFLoader("./llm_in_a_flash_apple.pdf")
  File "/Users/polo/langchain/libs/community/langchain_community/document_loaders/pdf.py", line 154, in __init__
    raise ImportError(
ImportError: pypdf package not found, please install it with `pip install pypdf`

This is my first time in a Python project like this. I am unsure how to get started using all the different packages while in a fork of the repository. If anyone can guide me I would love to make a PR on this, it is quite daunting for beginners to get around and start contributing!
Idea or request for content:
No response

I have the same problem here.
The pdf file is exist in the directory, but it is impossible to load it. I continuously get the error below.
ValueError                                Traceback (most recent call last)
Cell In[7], line 4
1 from langchain.document_loaders import PyPDFLoader
3 loaders = [
----> 4     PyPDFLoader("DSM5.pdf")
5 ]
6 pages = []
7 for loader in loaders:
File /opt/conda/lib/python3.10/site-packages/langchain_community/document_loaders/pdf.py:182, in PyPDFLoader.init(self, file_path, password, headers, extract_images)
178 except ImportError:
179     raise ImportError(
180         "pypdf package not found, please install it with " "pip install pypdf"
181     )
--> 182 super().init(file_path, headers=headers)
183 self.parser = PyPDFParser(password=password, extract_images=extract_images)
File /opt/conda/lib/python3.10/site-packages/langchain_community/document_loaders/pdf.py:116, in BasePDFLoader.init(self, file_path, headers)
114         self.file_path = str(temp_pdf)
115 elif not os.path.isfile(self.file_path):
--> 116     raise ValueError("File path %s is not a valid file or url" % self.file_path)
ValueError: File path DSM5.pdf is not a valid file or url
Any ideas?


More detail

You do not cover signing or any of the other tasks involved. I would strongly suggest covering every step/task needed, especially signing the APK since that's what I've been stuck on the most trying to set this up for our company. You should also cover setting up pre-reqs such as installing Android SDK Levels and setting the ANDROID_HOME environment. I've been working on this for 2 days and still can't produce an APK that works.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: c45a95ce-73c8-861e-f92d-a26f1667aa60
Version Independent ID: ecb82128-cdf4-3834-62c1-2cf61989ab5e
Content: Build your Xamarin app
Content Source: docs/pipelines/apps/mobile/xamarin.md
Product: devops
GitHub Login: @andyjlewis
Microsoft Alias: alewis

Thanks for creating this issue, Jason.  I'd like us to document what you suggested ASAP.  I'm sorry for the wasted time and frustration you went through.  If you haven't exhausted all of your available time, could you please email me at<EMAIL_ADDRESS> We can help you get things working and then use our weak spots to improve the product and guide what we put in the documentation, including what you suggested above.

This is great for Xamain.ios and Xamarin.Android -  BUT, what about Xamarin.MAC?

Keep-alive

Sorry, I had forgot to reply to this. I'll email you this week to help with
the documentation.
Thanks,
Jason Willett
On Sun, Mar 10, 2019, 2:43 PM David Staheli<EMAIL_ADDRESS>wrote:

Keep-alive
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/vsts-docs/issues/1656#issuecomment-471332268,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEz4PZ4Ym9xXvT-1wUSta5Bya1YtVbljks5vVVJpgaJpZM4WbtIo
.


Marriage abroad civil partnership amend

PT story: https://www.pivotaltracker.com/n/projects/1270592/stories/109439008
This PR adds a missing word in civil partnerships outcome for non-CNI countries. It was originally included as part of PR #2194 but was moved out into its own PR as it changes more than just USA outcomes.
Expected change

URL on gov.uk
Updates second sentence from "eligible enter" to "eligible to enter"

Before

After

Note: This PR was previously merged as part of PR #2198 but then reverted to rebase it, so Jenkins can merge it onto the release branch as part of the build.

I've just rebased this with the master branch and forced pushed to this PR's branch.


getmail6: 6.18.10 -> 6.18.11

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for getmail6 is: "A program for retrieving mail"
meta.homepage for getmail6 is: "https://getmail6.org"
meta.changelog for getmail6 is: "https://github.com/getmail6/getmail6/blob/refs/tags/v6.18.11/docs/CHANGELOG"
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 6.18.11 with grep in /nix/store/sk8s49066pjkcgpjd3bsw1a9674gs551-getmail6-6.18.11
found 6.18.11 in filename of file in /nix/store/sk8s49066pjkcgpjd3bsw1a9674gs551-getmail6-6.18.11

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
getmail6

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/sk8s49066pjkcgpjd3bsw1a9674gs551-getmail6-6.18.11 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A getmail6 https://github.com/r-ryantm/nixpkgs/archive/f65ede2be86476c8f1a6858096edab16d6e1e4e1.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/sk8s49066pjkcgpjd3bsw1a9674gs551-getmail6-6.18.11
ls -la /nix/store/sk8s49066pjkcgpjd3bsw1a9674gs551-getmail6-6.18.11/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

getmail6

Maintainer pings
cc @wahjava @dotlambda for testing.

Result of nixpkgs-review pr 208669 run on x86_64-linux 1

1 package built:

getmail6


Feature Request: Programmatically add custom info to summary

Is your feature request related to a problem? Please describe.
Some Plex clients either do not show audience and/or critic ratings, or make it difficult to access from the movie page without clicking on further links.
Describe the solution you'd like
To programmatically add custom info to the summary, from other metadata fields.
For example, adding the IMDB and RT scores as a prefix to the existing summary (Alien - 1979):
IMDB: 8.4; RT: 98%. After a space merchant vessel receives an unknown transmission as a distress call, one of the crew is attacked by a mysterious life form and they soon realize that its life cycle has merely begun.
Describe alternatives you've considered
It could also be useful to extend this to add other info into the summary, for example play count or date added.
Additional context
This used to be possible with the old OMDB agent, before it was removed.

Just to expand upon this idea, you could theoretically add labels to the item too with IMDB: 84 rt: 98 etc, and maybe PMM could have a custom logic for let's say "rt.gte: 30" which would scan the labels with the "RT: x" format and make the collection of ones where it is greater than or equal to 30

Could this be addressed in a similar way to as mentioned in #699 (e.g. using a more general metadata_mapper), in combination with a template that allows access to other metadata fields?

This is not gonna happen. i already have the mass rating operations and if thats done then it will work plex_search/smart_filter

Just to expand upon this idea, you could theoretically add labels to the item too with IMDB: 84 rt: 98 etc, and maybe PMM could have a custom logic for let's say "rt.gte: 30" which would scan the labels with the "RT: x" format and make the collection of ones where it is greater than or equal to 30

This isnt really mapping the metadata itll be its own library operation


[BUG] Job cancelling fails at scale

Describe the bug
A couple people have reported that attempting to cancel a scan when the library reaches a certain size fails. The description for what happens aligns between both reports:

Attempt to cancel
The thread seems to stay alive and updates are still coming through
Attempting to cancel again results in a 500 because the JobManager already dropped the Worker

Killing the server fixes the issue
Expected behavior
Cancelling a job, regardless of how large the library is, should immediately kill the thread and stop execution

Closing in favor of https://github.com/stumpapp/stump/issues/102