NullPointerException when going back online

With agora_rtc_engine: 1.0.5
And following init
await AgoraRtcEngine.enableVideo();
await AgoraRtcEngine.disableAudio();
await AgoraRtcEngine.enableWebSdkInteroperability(true);
AgoraRtcEngine.setParameters(
'{"che.video.lowBitRateStreamParameter":{"width":320,"height":180,"frameRate":15,"bitRate":140}}');
await AgoraRtcEngine.joinChannel(null, widget.channelName, "something", 0);
i stumbled upon the following issue

the user creates his channel
once joined the user goes offline
then the user goes online
Nullpointer exception appears in the logs
it doesnt seem to affect the streaming session.

I/flutter (20301): [NetworkConnectivity] network status updated to ConnectivityResult.wifi
I/flutter (27793): [CallPage] onConnectionStateChanged with state = 3 & reason = 1 (MY Own log)
E/RtcEngine(27793): onEvent: java.lang.NullPointerException: Attempt to invoke virtual method 'java.lang.Object java.util.HashMap.put(java.lang.Object, java.lang.Object)' on a null object reference
more details in attachement
stackTrace.txt

it crash?

@LichKing-2234  i think it does not crash as he mentioned stream is not affected.

yes no crash

you may ignore this log before we find it's cause


Extracting the lit value of the image

Is it possible to extract the light values of the image by for example sampling the opposite way? which could give output for the map of the light data. in my mind, process is there, just reversing it, either by inversing the model or the sampling process.
which will give us more freedom to manipulate existing image's lit values and manually fix or finetune it then reuse it in latent_bg?

I don't know about reverse sampling such values, but we can get the difference and maybe utilize that, even a normal map can be created this way, as lllyasviel demonstrated in the original demo. I have now made nodes and added example workflow to do that:

that's really cool, but what I wanted is feed its original lighting as is and the image wouldn't change after the sampling.
main purpose is inpainting and storing the original light state so it could be partially forced over inpaints, light value usually don't match up very well on inpaints.
I tried to feed the image as latent_bg, it works somewhat, but it changes the image and multiplies the lights and shadows or giving extra color so I tried to fix it by contrast/brigness and saturation nodes. it needs some magical numbers to not the change the image.


[Accessibility]A11y_VSCode_SelectEncoding_Screenreader: NVDA does not read the placeholder present on the screen.

Environment Details:
VSCode Version : 1.24.1
OS Version : Win10
Additional Details:
NVDA 2018.1.1,JAWS18.0.5038,MAS1.3.1
Repro Steps:
1)Launch VSCode.
2) Navigate to Status Bar and select " Select Encoding" button.
3)Check with NVDA reading.
Actual:
NVDA does not read the placeholder present on the screen.
Expected:
NVDA should read the placeholder which is present on the screen.
Recommendations:
Refer below link which is repository of bug fixes code snippets:
https://microsoft.sharepoint.com/teams/msenable/mas/pages/browse-fixes.aspx

User Impact:
User will not able to know that there is placeholder present  on the screen.
MAS Reference:
https://microsoft.sharepoint.com/:w:/r/teams/msenable/_layouts/15/WopiFrame.aspx?sourcedoc={8492c4eb-c179-40ae-8777-cd044ed725a2}

Attachment for Reference:
[Accessibility]A11y_VSCode_SelectCoding_Screenreade.pptx
Does this issue occur when all extensions are disabled?: Yes

Dup of https://github.com/Microsoft/vscode/issues/51824


Pre-processing in stage1. EOFError: No data left in file.

Hello, this is a great job.  You mentioned: "At the first training time, we do stereo rectify and determine the disparity offline, the processed data will be saved at render_data/rectified_local. This process takes several hours and can extremely speed up the following training scheme. " I didn't skip this pre-processing and set data_root in stage1.yaml to the path of unzipped folder render_data. I have completed 77% of the pre-processing(5245/6816), but the following error occurred at this point:
Traceback (most recent call last):   File "/mnt/GPS-Gaussian/train_stage1.py", line 174, in <module>     trainer = Trainer(cfg)   File "/mnt/GPS-Gaussian/train_stage1.py", line 28, in __init__     self.train_set = StereoHumanDataset(self.cfg.dataset, phase='train')   File "/mnt/GPS-Gaussian/lib/human_loader.py", line 129, in __init__     self.save_local_stereo_data()   File "/mnt/GPS-Gaussian/lib/human_loader.py", line 134, in save_local_stereo_data     view0_data = self.load_single_view(sample_name, self.opt.source_id[0], hr_img=False,   File "/mnt/GPS-Gaussian/lib/human_loader.py", line 198, in load_single_view     intr, extr = np.load(intr_name), np.load(extr_name)   File "/home/pai/envs/gps_gaussian/lib/python3.10/site-packages/numpy/lib/npyio.py", line 436, in load     raise EOFError("No data left in file") EOFError: No data left in file
What is the reason for this? If you can help me solve this problem, I would greatly appreciate it.

Hi,
According to the log, I suggest checking the .npy data of sample 5245 to see whether it was damaged during the zip and unzip process. Also, you can render the training data from THuman2.0 scan using the provided scripts instead of downloading the rendered training data.


fix tag value if statement

Fixed the first if statement in getting tag value

See #659

@scwong93 thanks!
Would be extremely valuable for us, right now and going forward, if we got a test or two in our test suite that reproduces a bug & verifies these changes fixes it.

Closing this for now as author has not responded to the request for tests.


Event hubs stop receiving data

Expected Behavior
We can read from an event hub partition and continue receiving data forever, or until an error is returned.
Actual Behavior
We stop receiving data after some time. This happens particularly frequently under high RPS. Here's the code we have. We've used a timer and a reset in order to kill and then retry receiving data if we don't get events for a given amount of time. Here's a simplified version of the code we're using:
const readerIdleReset = time.Second * 30

func (e *EventHubReader) readFrom(partition string) (listener *eventhub.ListenerHandle, err error) {
	resetTimer := time.NewTimer(readerIdleReset)

	receive := func(_ context.Context, event *eventhub.Event) error {
		message := &rpc.QueueEvent{}
		if err := message.Unmarshal(event.Data); err != nil {
			// error handling
		}

		resetTimer.Reset(readerIdleReset)

		e.data <- message

		return nil
	}

	listener, err = e.hub.Receive(context.Background(), partition, receive,
		eventhub.ReceiveWithLatestOffset())
	if err != nil {
		return nil, err
	}

	go func() {
		select {
		case <-resetTimer.C:
			ctx, cancel := context.WithTimeout(context.Background(), e.timeout)
			_ = listener.Close()
			cancel()
		case <-listener.Done():
		}
	}()

	return listener, err
}

We detect timeouts by receiving from listener.Done() and seeing if the resulting error was context.Canceled
One suspicious we had was e.data <- message taking a long time, but we added telemetry here that logged any sends which took more than a second, and they were never tripped.
Environment

OS: Linux/CentOS
Go version: 1.12
Version of Library: v2.0.0

go.sum:
github.com/Azure/azure-amqp-common-go/v2 v2.0.0 h1:98xtMbghfioKloSBZgkIwH/SINcDYtxXBbUZoqCePiI=
github.com/Azure/azure-amqp-common-go/v2 v2.0.0/go.mod h1:YDoDY50iQ2OabOP0WUQoNR7vpDjRlB13vIZVrvUoJLo=
github.com/Azure/azure-event-hubs-go/v2 v2.0.0 h1:ZzYGZOBxlB5Y2ZL0aGypSOYHTAWd3hKoWlLZH0CLuIY=
github.com/Azure/azure-event-hubs-go/v2 v2.0.0/go.mod h1:HqOqkORXit1YOUS7CWSVkS+r+2l6yG85YPbOs+01rYU=
github.com/Azure/azure-pipeline-go v0.1.8/go.mod h1:XA1kFWRVhSK+KNFiOhfv83Fv8L9achrP7OxIzeTn1Yg=
github.com/Azure/azure-pipeline-go v0.1.9/go.mod h1:XA1kFWRVhSK+KNFiOhfv83Fv8L9achrP7OxIzeTn1Yg=
github.com/Azure/azure-sdk-for-go v29.0.0+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
github.com/Azure/azure-sdk-for-go v30.0.0+incompatible h1:6o1Yzl7wTBYg+xw0pY4qnalaPmEQolubEEdepo1/kmI=
github.com/Azure/azure-sdk-for-go v30.0.0+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
github.com/Azure/azure-storage-blob-go v0.6.0/go.mod h1:oGfmITT1V6x//CswqY2gtAHND+xIP64/qL7a5QJix0Y=

Can you build your code with -tags debug and execute the process with DEBUG_LEVEL=3. This should output protocol debugging info to stdout. From that, we should be able to diagnose what is going on.
@gavinfish we should probably review the logs together to help diagnose the issue.

Dropped a build on a box with those settings. I'll send logs to you via email once we hit a timeout. Thanks!

Sent. Interestingly, amqp logs that it still gets data even up to the point where we timeout (meaning, the function hasn't been called for 30 seconds).
00:04:14.868640 RX(Session): Transfer{Handle: 0, DeliveryID: <nil>, DeliveryTag: "<nil>", MessageFormat: <nil>, Settled: false, More: false, ReceiverSettleMode: <nil>, State: <nil>, Resume: false, Aborted: false, Batchable: false, Payload [size]: 368}
00:04:14.868647 RX: Transfer{Handle: 0, DeliveryID: <nil>, DeliveryTag: "<nil>", MessageFormat: <nil>, Settled: false, More: false, ReceiverSettleMode: <nil>, State: <nil>, Resume: false, Aborted: false, Batchable: false, Payload [size]: 368}
00:04:14.868664 RX(Session): Transfer{Handle: 0, DeliveryID: 55955, DeliveryTag: "", MessageFormat: 0, Settled: true, More: true, ReceiverSettleMode: <nil>, State: <nil>, Resume: false, Aborted: false, Batchable: true, Payload [size]: 450}
00:04:14.868671 RX: Transfer{Handle: 0, DeliveryID: 55955, DeliveryTag: "", MessageFormat: 0, Settled: true, More: true, ReceiverSettleMode: <nil>, State: <nil>, Resume: false, Aborted: false, Batchable: true, Payload [size]: 450}
00:04:14.868691 RX(Session): Transfer{Handle: 0, DeliveryID: <nil>, DeliveryTag: "<nil>", MessageFormat: <nil>, Settled: false, More: false, ReceiverSettleMode: <nil>, State: <nil>, Resume: false, Aborted: false, Batchable: false, Payload [size]: 363}
00:04:14.868698 RX: Transfer{Handle: 0, DeliveryID: <nil>, DeliveryTag: "<nil>", MessageFormat: <nil>, Settled: false, More: false, ReceiverSettleMode: <nil>, State: <nil>, Resume: false, Aborted: false, Batchable: false, Payload [size]: 363}
Aug 01 00:04:15 constellation-pulsar1.dal09.mixer.com pulsar[21332]: 00:04:15.151254 TX(Session): Detach{Handle: 0, Closed: true, Error: *Error(nil)}
Aug 01 00:04:15 constellation-pulsar1.dal09.mixer.com pulsar[21332]: time="2019-08-01T00:04:15Z" level=warning msg="event hub read timed out" error="event hub read timed out" partition=4

So this seems like it might be something internal to the client or our usage of it. Trying to wrap everything inside the receive() function into a spawned goroutine, and then immediately returning nil, didn't seem to do much, so I don't think it's that method backing up.

I think there is no need to introduce timer.Reset() to make the diagnosis more complex. It should be a potential temporary solution to mitigate your situation but won't be much help for the debug.
Is it possible to provide clean code to be able to reproduce the error stably?

@devigned  any more information could we get from the amqp log?

It looks like I also observed similar issue. After couple of hours of consuming messages from 32 partitions (~ 104 messages / second on each partition) my consumer stopped receiving messages one one of them. Actually, it didn't stop completely, it was still receiving something like 0.5 message / minute. That situation last for around 25 minutes after which consumer suddenly started to consume messages from lagging partition and was able to catch up other partitions.
Situation like this happens for me about 2 times per 24 hours.
I'm running my consumer as single Go application consuming all 32 partitions on Intel(R) Xeon(R) Platinum 8168 CPU @ 2.70GHz (8 cores machine). I'm still using version 1.3.1 of package, I'm planning to upgrade soon.

We have the same problem. Is there any news or progress in this bug?

I think I'm seeing the same problem. We have a hub with 2 partitions and about 1 day's backlog of data. I have a small test program that's trying to read out the entire backlog from the beginning. It reads a portion of the backlog data quite quickly, then slows to a crawl. I've never seen it read all the data
At the beginning the app reports the partitions as follows
partition 0
seq 704004 -> 756295
last enqueued offset<PHONE_NUMBER>60 at 2021-09-22 13:14:20.561 +0000 UTC

partition 1
seq 703687 -> 756296
last enqueued offset<PHONE_NUMBER>84 at 2021-09-22 13:14:20.544 +0000 UTC

I wait until the app stops using any significant CPU, then stop it. It reports the latest sequence number it has seen on each partition as follows.
PID 0 last sequence 721902 - offset<PHONE_NUMBER>84 at 2021-09-22 02:25:55.689 +0000 UTC
PID 1 last sequence 739886 - offset<PHONE_NUMBER>08 at 2021-09-22 07:16:20.531 +0000 UTC

If I wait much much longer (hours) then the received sequence numbers do appear to go up, but very slowly and I've not waited long enough to see them reach the initial reported last enqueued sequence numbers.
The code I'm using to read from the hub is here: https://gist.github.com/philpearl/62cb8a514f82787df65e2fb14c2c2a9e. I can't share the EventHub I'm reading from, but let me know if that would be useful and I may be able to arrange something.

If I update the program to use a file persister and restart the program once it has slowed to a crawl, then it reads quickly from the saved position when it restarts. Eventually it grinds to a halt again until I restart it.

My problem seems to be that I accidentally used an old version of the event hubs library - you really need to make sure you use the v3 version (at least), not the v1 version that's the default. Make sure you use github.com/Azure/azure-event-hubs-go/v3 (or check if there's a later version)

Closing this issue in favor of #240.


How to keep yAxis label in same in two different devices ?

Current Behavior
in iPhone5:

in iPhone6:

I want to make yAxis label Keep in three decimal. Is it made by the device-width ? How to Fix it ? Or can I customize the yAxis label value ?
Environment

Chart.js version: 2.6.0
Browser name and version: Chrome 59.0.3071.86

@Sakuyakun you can use http://www.chartjs.org/docs/latest/axes/cartesian/linear.html#step-size

@etimberg thanks, i miss this part


OtioView should indicate the presence of empty tracks

While debugging #650, otioview has blank lines but they don't read.  Better to draw the track widet with EMPTY TRACK or something on it so that its clear that there are tracks there, they simply contain no data.

@ssteinbach I'd like to give this a try.
I think showing an empty track should be good enough because it seems clear that an empty track is present even of we don't put anything there.
I think modifying this part by removing the check on list(t) should do the trick.
        if isinstance(self.composition, otio.schema.Stack):
            video_tracks = [
                t for t in self.composition
                if t.kind == otio.schema.TrackKind.Video and list(t)
            ]
            audio_tracks = [
                t for t in self.composition
                if t.kind == otio.schema.TrackKind.Audio and list(t)
            ]

It gives me something like this (one empty video track and one empty audio track):

@ssteinbach does this seem right?

Any thoughts on this?

It would be more clear with a label of some kind. Instead of "Empty Track" maybe we could add the names of the tracks on the left? That would also help people distinguish the audio and video tracks from each other.


[1571] Remove rspec-json_matchers

Context
It's not necessary for checking types and attributes on jsonapi hashes. jsonapi-rspec has better matchers which we already bundle.
Changes proposed in this pull request
Replace calls to the library matchers and remove the library.
Guidance to review
SHIP IT
Checklist

[x] Make sure all information from the Trello card is in here
[x] Attach to Trello card
[x] Rebased master
[x] Cleaned commit history
[x] Tested by running locally

 Add validations to course update action.


Updates

Waiting for https://github.com/conda-forge/pyiron_contrib-feedstock/pull/11

Will change it back to 500. Also changed a few explanations.

Seems to fail in another notebook should I merge it?

Seems to fail in another notebook should I merge it?

No, only merge pull requests which pass the tests.


added option to "--force" renew and

fixed the LE Problem with expired DST certificate
by forcing the full certifate chain as update

Hi @alxwolf,
Very cool.
Just tried it out.
Interestingly, it returned the same error I reported in my update on #2.
chmod: /mnt/data/unifi-os/unifi-core/config/ca.cer: No such file or directory
UPADATE: I just saw your comment about it being an "extraneous line of code which I now removed."


Fix integration tests again

a previous PR https://github.com/Alluxio/alluxio/pull/18313 restored the integration tests but caused the unit tests to also run with the integration tests.
the issue was correctly identified as an issue with the maven project list but the underlying problem was that the project name was not correctly set; dora/tests is not a module. it was previously a module until dora/tests/testcontainers was introduced, thus separating testcontainer tests vs integration tests.
this fix updates the previous project list with the correct module name

alluxio-bot, merge this please


http 报错  http  * Serving Flask app 'channel.http.http_channel'  * Debug mode: off Permission denied

Load config success
[INFO][2023-03-23 07:46:15][log.py:34] - [INIT] Start up: chatgpt on http

Serving Flask app 'channel.http.http_channel'
Debug mode: off
Permission denied

换个端口

同样的问题。更改端口无效


percent-pos property is 'null' before quitting

Bug Description
When listening to 'statuschange' event and adding a 'percent-pos' property to observe, when quitting MPV, the percent-pos is set to 'null' on MPV > 0.29
How To Reproduce
let position = 0;

mpv.start().then(() => mpv.load(<myfile>)).then( () => mpv.observeProperty('percent-pos', 50));

mpv.on('statuschange', states => {
    position = states['percent-pos'];
    console.log(position); // Will display 0.01% up to 99.99%, no problem here
});

mpv.on('quit', () => console.log(position)) // Will display 'null' instead of the last percent-pos known, as it used to on 0.29

Software Versions

Node-Mpv: branch 2, commit 6fc6037 (latest at writing time)
MPV: 0.32.0 stable, windows build taken from the official recommended repository

Additional  context
Might not be a node-mpv issue but a MPV issue? Trying to update from 0.29 has caused that problem mainly, afaik.

The question is how would the fix look like? Output 100.0 or always save the previous value when the value is null? I'm not 100% sure if that wouldn't eventually cause some error at a different place and doesn't quite seem like an elegant fix.

Currently, I save the last percent-pos elsewhere and update it if it's non-null only, so after exiting I have the last known percent-pos transmitted by mpv: https://github.com/vankasteelj/frak/commit/39e1e8bcdcff14fa27657615ef12db84a6185bc8
It's inelegant, but it works so far.

I don't know if mpv introduced a new state to avoid that "null" percent-pos in their latest updates, if they decided to void that value, or what.
But I know that node-mpv gives back correct "states" values after exiting, but not for the added observed-property percent-pos. It could be that observeProperty needs to be udpated in your code?
The thing is, I don't know where that "null" comes from, if it's from mpv or your code.

I believe it has to do with how events are handled since MPV 0.31, that states in the release notes something like "deprecate timed events", because as of >0.30 atm pause/resume events are fired twice.

Interessting. I will have to read into that. It's always a pain to deal with different MPV versions, because you never know which one people have installed. If you use the Verison from Ubuntu oder Debian Distributions it's ancient...

Has this issue ever occured again? Just asking to see if it's worth investing time in.

Has this issue ever occured again? Just asking to see if it's worth investing time in.

Still using the workaround, so I'm guessing yeah, nothing changed on my side. But I havent tested without the workaround

Still using the workaround, so I'm guessing yeah, nothing changed on my side. But I havent tested without the workaround

I used the following code to reproduce the issue (slightly adapted to the breaking changes I made with version 2)
mpv.start().then(() => mpv.load(<somefile>)).then( () => mpv.observeProperty('percent-pos', 50));

mpv.on('status', state => {
    if (state['property'] == 'percent-pos') {
        position = state['value'];
        console.log(position); // Will display 0.01% up to 99.99%, no problem here
    }
});

mpv.on('quit', () => console.log(position))

It does output 0, but it's undefined once before the song quits. I'm still unsure if this is an issue or not, if it causes problems or not. But I might have an idea I could publish as a PR. I'll notify you if I got one.


Update sbt to 1.7.3

Updates org.scala-sbt:sbt from 1.7.2 to 1.7.3.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Files still referring to the old version number
The following files still refer to the old version number (1.7.2).
You might want to review and update them manually.
ui/yarn.lock

Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-sbt", artifactId = "sbt" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "@monthly" },
  dependency = { groupId = "org.scala-sbt", artifactId = "sbt" }
}]

labels: library-update, early-semver-patch, semver-spec-patch, old-version-remains, commit-count:1

Not upgrading, see https://github.com/playframework/playframework/issues/11522
sbt 1.8 will need a new Play release.


Feature/aoaf 273 add oauth2

Added oauth2 to apis

CI: Security warnings found!

CI: Security warnings found!

CI: Security warnings found!

CI: Security warnings found!

CI: Security warnings found!


Exception acquiring database connection from an interrupted thread

OrientDB Version: 2.2.15
Java Version: 8
OS: Windows
Expected behavior
Calling com.orientechnologies.orient.core.db.OPartitionedDatabasePool.acquire() returns normally when the pool is not exhausted.
Actual behavior
If the current thread is in the interrupted status (flag not cleared), it's not possible to acquire a connection to the database, even though the pool is not exhausted.  Calling com.orientechnologies.orient.core.db.OPartitionedDatabasePool.acquire() throws an exception:
com.orientechnologies.common.concur.lock.OInterruptedException: Acquiring of new connection was interrupted
    at com.orientechnologies.orient.core.db.OPartitionedDatabasePool.acquire(OPartitionedDatabasePool.java:198)
...
Caused by: java.lang.InterruptedException
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
    at java.util.concurrent.Semaphore.acquire(Semaphore.java:312)
    at com.orientechnologies.orient.core.db.OPartitionedDatabasePool.acquire(OPartitionedDatabasePool.java:196)

https://github.com/orientechnologies/orientdb/blob/2.2.15/core/src/main/java/com/orientechnologies/orient/core/db/OPartitionedDatabasePool.java#L196
Steps to reproduce
Interrupt the thread that will call OPartitionedDatabasePool.acquire(), then call it.
Is it intentional that OPartitionedDatabasePool can't be used if the current thread is in the interrupted state?

Hi @dukeyin , I do not think that it is a bug. I mean what other means will developer have to stop thread if pool is exhausted and no connection is acquired ?

The pool is not exhausted, which I've added explicitly to the report.
The report is that OrientDB does not check the interrupted status of the current thread before using java.util.concurrent.Semaphore.  That is why I asked "Is it intentional that OPartitionedDatabasePool can't be used if the current thread is in the interrupted state?"  If it is not intentional, it requires code changes.
OrientDB current code looks like this:
try {
  if (connectionsCounter != null)
    connectionsCounter.acquire(); // this will fail for reasons unrelated to the connection pool
} catch (InterruptedException ie) {
  throw OException.wrapException(new OInterruptedException("Acquiring of new connection was interrupted"), ie);
}

I'd expect it should look something like this, with checking and restoring of the interrupted flag:
boolean interrupted = Thread.interrupted(); // clears the interrupted flag
try {
  if (connectionsCounter != null)
    connectionsCounter.acquire(); // this no longer fails for reasons unrelated to the connection pool
} catch (InterruptedException ie) {
  throw OException.wrapException(new OInterruptedException("Acquiring of new connection was interrupted"), ie);
} finally {
  if (interrupted) {
    Thread.currentThread().interrupt(); // restores the interrupted flag
  }
}

@dukeyin now I see your point and agree with you I will push fix soon.

@dukeyin sorry it was evening, now I am looked at your code and can not agree with the proposal. I do not think that we should clear the interrupted flag in a pool before we start to wait for other connections.  If user wants to interrupt thread I do not think that should be any reason to force thread to wait for indefinite amount of time.

Close issue because of no reply.


Did somebody see a splash message at startup like this? nothing is wrong but the splash,a little annoying

will disappear quickly, I must record the screen and play it slowly to cut this screen.
MacOS, tmux 3.2 , most recent conf version.

Hello @FinallyFinancialFreedom 👋
This is currently  a tmux limitation. The next version of tmux behaves slightly differently and this won't happen anymore.

thank you , should i update tmux to 3.3-rc?

Your choice.
As you noticed, this <... not ready> "flash" is harmless and happens only briefly when starting tmux with Oh my tmux!


Range Error

I downloaded the latest version of generate as well as generate-package, generate-license and a few others, but when I run package and license they both error.
Package error: https://gist.github.com/connorhartley/a15cd6392e77d5fed34a9bb87ad6364b
License error: https://gist.github.com/connorhartley/5ecb0d5d53b5acee201fac02646a7f8a
Version details:
Windows 10
node v6.5.0
npm v3.10.6
generate v0.11.2

Hmm, at first I thought I knew the cause, but after reviewing the errors I'm not sure. I'll look into it. @doowb any thoughts?
thanks for reporting.
(on the off-chance that it's related to what I was initially thinking... the following is what my response would have been)
There is a known bug causing this, it's a pretty easy fix that in reality will take 5 min, but I want to shift some code around to prevent similar errors from happening.
Here is what's happening and what I'm doing about it.

There are just a handful of fields that we try to pre-populate  for you, like license - if available (using user environment, package.json, etc).
these pre-populated fields should only be used as "hints" when you run generate. However...
I introduced a regression in a recent release. To explain, there are two places that come to mind where license is used differently. In package.json the license field expects a valid SPDX license as its value, but in the readme templates (in the generators I created - this is where my lapse of judgement caused the error) the license template variable is used to populate the license "statement". This value is overwriting the value from prompts. That shouldn't happen. Again, it was human error (mine).

Sorry about this. I'll get it fixed and update this when it's done.


cilium: Rework neighbor handling

See commits.

test-runtime

/test

/test

/test

/test

would also make sense to remove mentions the neighbor from comments

Fyi, https://github.com/vishvananda/netlink/pull/707 just got merged, will drop the remap to cilium/netlink then.

/test-1.16-netnext

/test

Travis hitting docker rate limits:
# Pre-pull FROM docker images due to Buildkit sometimes failing to pull them.
grep "^FROM " Dockerfile | cut -d ' ' -f2 | xargs -n1 docker pull
3.7.9-alpine3.11: Pulling from library/python
toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
make[1]: *** [Makefile:18: builder-image] Error 123
make[1]: Leaving directory '/home/travis/gopath/src/github.com/cilium/cilium/Documentation'
make: *** [Makefile:564: postcheck] Error 2
make: *** Waiting for unfinished jobs....
contrib/scripts/check-log-newlines.sh

The only lingering question I have is if CI has the correct set of kernel images to test the various configurations with regards to neigh entries.

The only lingering question I have is if CI has the correct set of kernel images to test the various configurations with regards to neigh entries.

The bpf-next one got fast-forwarded finally today due to open merge window, so the managed neighbour work is in there now, but at the moment it's still blocked by the <8byte stack spill breakage fix. We can upgrade the CI box once that is resolved, hopefully this week. Been testing with net-next locally to confirm that managed neighbor entries have been inserted.

/test

The only lingering question I have is if CI has the correct set of kernel images to test the various configurations with regards to neigh entries.

The bpf-next one got fast-forwarded finally today due to open merge window, so the managed neighbour work is in there now, but at the moment it's still blocked by the <8byte stack spill breakage fix. We can upgrade the CI box once that is resolved, hopefully this week. Been testing with net-next locally to confirm that managed neighbor entries have been inserted.

FWIW, he revert of the <8byte stack spill breakage patch still applies cleanly in our CI kernel build on latest bpf-next (at least it did yesterday in cilium/packer-ci-build#293). So I think once we rebuild our CI VM images and update them in this repo we should have coverage wrt. the neigh entries.

/test

/test

/test

CI: All green except test-runtime which timed out for known reason (hitting 6min base timeout GOTEST_BASE). The test is passing locally, just need to reduce neighbor base reachable time. The other TODO left aside from this is rebase the vendor stuff that got merged into master which also seem to have updated vishvananda.

/test

test-runtime


Drag and drop outside favorite bot list to remove it

To remove a bot from favorite bot list, user can drag and drop its logo outside the list.

Wow, it works! Wonderful!
A little pity is that the dragged up icon will move back to favbar before disappearing. Is it possible to no moving back?

From my side, I don't see dragged icon that will move back to favbar before disappearing. Can share screen recording?

https://github.com/sunner/ChatALL/assets/255413/15e38a9b-ef5e-4189-9b97-069458c0adc8
Look like it works fine on Windows but weird on Mac

@sunner that's really weird. I have updated the changes without using the sortablejs built-in plugin. Can you try again to see if the issue persists?

No change on macOS. But it is acceptable. Thank you @tanchekwei . Another great PR!


Support bitbank ACCESS-TIME-WINDOW authentication method

Refer: https://bitbank.cc/blog/articles/406396361
This pull request adds support for the ACCESS-TIME-WINDOW authentication method in the bitbank API. It includes changes to the bitbank function to handle the ACCESS-TIME-WINDOW header in both GET and POST requests. The bitbank_get_with_window test case
has been added to ensure the correct behavior of the new functionality.

The header can also be set to ACCESS-TIME-WINDOW. pybotters will honor it:
async def get_wallet_balance(client: pybotters.Client):
    result = await client.fetch(
        "GET", "/v1/user/assets", headers={"ACCESS-TIME-WINDOW": "3000"}
    )
    print(result.text[:1000])


Add "no tree" mode

As we already parse the elements in a linear way, providing also the linear output, rather than tree mode, makes it easy to find elements.

Done in v0.3.0


Counterparty chains don't have ICS23 0.10.0 support.

We discovered a critical issue with ICS23 support on counterparty chains that will disrupt our ability to connect to them unless we work with other ecosystem participants.
ICS23 provides a generic mechanism for non-inclusion proofs: a non-inclusion proof for key k is a pair of inclusion proofs for keys k1, k2, a check that k1 and k2 are adjacent in the tree, and a check that k1 < k < k2. The problem is that this mechanism only works on ordered trees like the IAVL tree where the key ordering corresponds to the tree structure. On a sparse merkle tree like our JMT, this isn't the case -- sparsity is achieved by using the hash H(k) of the key, so keys are randomly ordered.
To fix this, we upstreamed a prehash_key_before_comparison field into the ICS23 spec. This provides a backwards-compatible extension supporting trees like the JMT: if the field is absent, it's false, and noninclusion proofs are verified as before; if it's present, it's true, and the noninclusion proof verifier checks that H(k1) and H(k2) are adjacent and that H(k1) < H(k) < H(k2)`.
Because all proto fields are optional and proto implementations are supposed to ignore unknown fields, this extension should also have been forwards-compatible, in the sense that a client could be created on a counterparty chain with prehash_key_before_comparison = true, with the following behavior:

if the counterparty chain is using ICS23 0.10.0 or greater, it parses the prehash_key_before_comparison field and verifies non-inclusion proofs correctly;
if the counterparty chain is using an older ICS23 version, it ignores the prehash_key_before_comparison field, and all non-inclusion proofs fail to verify.

In the latter case, packet timeouts don't work, but because the on-chain data is correct, as soon as the counterparty chain updates their ICS23 version, they would start working without any migrations to chain state.
However, what actually happens is that the Cosmos SDK rejects unknown fields: https://github.com/cosmos/cosmos-sdk/blob/a3a049bc2577632b94ff2236c2b01a250e85b16d/x/auth/tx/decoder.go#L27
To fix this, we need to urgently coordinate with the rest of the ecosystem to move them onto versions of ibc-go that use ics23 0.10.0 or greater. This is included by default in v7.1.0, but that version is much newer than many counterparty chains, so there will be longer upgrade latency. For instance, Osmosis is on v4.  Ideally, we could convince the ibc-go maintainers to release backport releases that bump the ICS23 version.

Hi, thanks for the writeup @hdevalence. For clarification, the concern with opening connections occurs when the prehash_key_before_comparison is set to true or false? In previous releases when adding new fields in a backwards compatible way, we have relied on the ability to not encode empty fields to avoid the reject unknown fields issue. The proto codec (I'm fairly certain) does this be default, but the json marshaler implemented by the SDK does not. See here, EmitDefaults needs to be set to false. The fix is actually quite simple. I opened an issue for it https://github.com/cosmos/cosmos-sdk/issues/13727, but in the meantime we have gotten around the issue by simply using a json marshaler which sets EmitDefaults to false
I'm curious if there's a fix related to the don't encode empty fields behaviour that could avoid requiring counterparties to perform software upgrades?
Assuming the above context doesn't help because you are concerned with creating a light client with the prehash_key_before_comparison key set to true before the counterparty has upgraded to v0.10, then indeed you would need counterparties to upgrade. You could potentially allow counterparties to run the light clients without timeout support in the meantime and then perform an IBC software upgrade to change counterparty proof specs to set the prehash key to true. You can see documentation on this in the IBC client breaking upgrades where changing the proof specs is supported functionality:

changing the ProofSpecs: Supported, this should be changed if the proof structure needed to verify IBC proofs is changed across the upgrade. Ex: Switching from an IAVL store, to a SimpleTree Store

This approach would not be very clean though as it would try to upgrade all counterparty clients to using the new field in ics23 v0.10 at the same moment, it is also a feature that has not been used very frequently so it may take some effort UX wise

My current understanding w.r.t ICS23 0.10.0 support:
If the counterparties (cosmos SDK) had accepted unknown fields as we expected, we would have the situation where:

The counterparties accept our proof specs and can form channels
Channels, packets, and acks, all work normally (since the inclusion proof spec works)
Timeouts will not work, since every proof verification fails without the changes we made in v0.10.0

Therefore, to enable timeouts in that scenario, the counterparty would have to:

Upgrade their ibc-go impl to use ICS23 >0.10.0 to get noninclusion proofs for the JMT

Alternatively, in the situation we find ourselves in, where unknown fields are rejected, we have this situation:

The counterparties can accept our proof specs, if we elide prehash_key_before_comparison (and allow this elision on the penumbra side (!!) )
Channels, packets, and acks work normally (with this modified proof spec)
Timeouts will not work, due to missing proof verification modification (same as above)

To enable timouts in that scenario, the counterparty has to:

Upgrade their ibc-go impl to use ICS23 >0.10.0 to get noninclusion proofs for the JMT (same as above)
Issue an IBC UpgradeClient to update the proofspec of every penumbra client

Additionally, there are security questions around allowing the prehash_key_before_comparison field to vary on the penumbra side (not doing strict validation of it). I think at minimum if we did have clients that used that approach, they should disable timeouts/anything reliant on sound noninclusion proofs.

Tracking issue for Osmosis: https://github.com/osmosis-labs/osmosis/issues/6482

Osmosis support may have landed: https://github.com/osmosis-labs/osmosis/pull/6653

Still a blocker for Osmosis: https://github.com/osmosis-labs/osmosis/issues/6482#issuecomment-1755925777

Osmosis is working as of the latest testnet release v21

Marking this as done, we have a critical mass.


Title: Issue with WalletConnect Integration in Web3 Unity Application

Comment:
I'm experiencing difficulties with the integration of WalletConnect in my Web3 Unity application. The issue arises when attempting to connect to the wallet, where the connection either fails or gets stuck in a loading state. I've followed the provided documentation and ensured all necessary dependencies are included. Any assistance or guidance on resolving this issue would be greatly appreciated.

Hello @Myomin78779,
Could you please open an issue with Bug Report template and provide more details so that I could investigate your issue?


Streamline Sync UX process / save "My clippings.txt" file location to prevent file selection popup

Is your feature request related to a problem? Please describe.
I use this great plugin to sync highlights of my kindle files, type "documents". These documents are research papers (PDFs) and web findings. For this I have to use the "My clippings.txt" method since Amazon is not uploading non-book highlights to the cloud.
It is a bit cumbersome that I need to locate the file every time I want to do a sync.
Click process:

Click on plugin Icon: Pop up appears.
Click on "Sync"
Click on "Sync now"
Locate File
Click on "Open"
Click on close / X

Describe the solution you'd like
The easiest solution to implement would probably be a default folder location where the plugin looks for the "My clippings.txt" file. This is used after step 3 without the need to locate the file. Only if the file cannot be found in the default directory then the file selection popup appears.
Describe alternatives you've considered
I think the overall user experience of the clippings sync could be further improved: The user selection to cloud / locally sync is already stored in the plugin and could be changed by users in the settings. So why are steps 2., 3., 4. and 5. needed?
Leaner approach proposal:

Click on the plugin icon: Sync is done in the background.
After sync ended, popup with metrics appears:

Thank you and keep up the good work.

I just bought a Kindle, just found this plugin, loved it, and felt exactly the same as you — I felt having to pick a file manually each time would quickly become cumbersome.
So I just wrote a PR that adds an option in the plugins' settings. You can type the path to your My Clippings file there, and the plugin will automatically use this file if it can!
I think there is room for further UX improvements too — like you suggested, automatically synchronising whenever the Kindle is connected to the computer would be great!

@vii33 thank you for raising this issue and the effort you've put in to share your idea.
I rarely ever use the My Clippings method myself so hence haven't experienced the inefficiency in the UX that you spoke about.
Happy to make the changes you proposed as they make sense. But can I clarify one thing - how would the plugin know the "default" location for the clippings file?
One option that could work is requiring the user to select a file location on first sync. This "last used" location is then considered as the default going forward.
How does that sound?

Hi @hadynz,
thank you for replying and considering this. In my opinion, it should be a setting, like the PR from @fzslm suggested. Otherwise you cannot edit the location if it changes at a later time.
You could also put in the settings a toggle to select either the cloud or the file sync as the default option. With this, you could save two UI screens, basically getting both sync methods to the minimal user flow (screen above).


randomPartition looks a bit complicated and slow

At present,
randomPartition :: Int -> [a] -> IO [[a]]
randomPartition n xs = shuffleM (take (length xs) $ cycle [1..n])
  <&> M.elems . M.fromListWith (++) . (`zip` map pure xs)

Micro-optimizations: use takeList :: [a] -> [b] -> [b] instead of take . length. Use IntMap instead of Map.
But I suspect a full shuffle is overkill for what you're doing here. Does it actually matter that the order of elements in buckets is random?  If you only care about which elements are grouped with which others, I think you can probably do something simpler:
data Bin a = Bin {size :: !Int, elements :: [a]}

Create an MVector of empty bins.
For each element in the list to partition, choose a random index into the vector and insert the list element into the bin there. If the bin is then "full", swap it off to the right (the first one to fill is swapped to the last spot, the second one to fill is swapped to the second to last spot, etc.).
Continue until all list elements are gone.

Then you'll end up with a vector full of random buckets.

This can be done, but the performance impact is not important as we're partitioning a small list, i.e. up to configK elements (16 by default).


Buffer wrapping OwnedBuffer is problematic for the BCL

I started looking more at the Buffer / OwnedBuffer in corefxlab, and I’m concerned about the design.
Specifically, for the BCL we want to add APIs in various places that work with buffers, in order to:

Support passing in arbitrary pointers, e.g. Stream.Write
Support passing in array subsets, e.g. int.Parse

etc. We could add individual overloads to these methods for native pointers, for offsets+counts, but part of the goal of this whole effort has been to unify all of that.
My hope had been that we'd be able to add an individual new overload in many of these situations, taking a single type that unified all of this and that didn't pull in other dependencies.  I thought that was Buffer<T>.  But in looking at it today, creating a Buffer<T> from a pointer or an array allocates an OwnedBuffer<T>, which is bad.  For example, I'd want someone to be able to do a stackalloc and then call fileStream.Read(new Buffer<byte>(ptr, len)) to read into that memory, but with the current design, that allocates.  And even if we pooled OwnedBuffers under the covers, it's still allocating and/or potentially a source of contention on a pool.  And folks shouldn't need to be maintain their own pool of OwnedBuffes to avoid such things.
In my mind, there's the fast Span<T>, and then Buffer<T> simply be the heap-able version of Span<T>, essentially an ArraySegment<T> that can also work with pointers in addition to arrays.  The BCL could operate in terms of those two simple exchange types and not need to be concerned in these APIs with types like OwnedBuffer<T>; it would be up to callers to use them as they needed in their system:
public struct Span<T>
{
    private ByReference<T> _ptr;
    private int _length;
    ...
}
public struct Buffer<T>
{
    private T[] _array;
    private IntPtr _ptr;
    private int _offset;
    private int _count;
    ...
}

I understand that one of the impacts on the existing design is control over the buffers to help with pooling, to help avoiding tearing, etc.  I think we're paying too much for that and such issues should be left to types that sit entirely above Span<T> and Buffer<T> rather than being inherently intertwined.  Developers using Span<T> and Buffer<T> are responsible for doing so correctly; if you put a Buffer<T> on the heap and have a race condition that ends up tearing it, that's a bug in your program that you need to fix, not something that every developer who doesn't have such bugs should be forced to pay for in usability and runtime cost.  A higher-level set of types can be classes, be concerned with pooling, etc.
I realize we've probably been around and around on many of these designs, that various issues are called out in https://github.com/dotnet/corefxlab/blob/master/docs/specs/memory.md, etc., but I think we need to revisit this from the perspective of the minimal set of core types that we'd use throughout the BCL, with no unnecessary integration with an entire buffer pooling system, and then have such a system that's built on top, that's replaceable if someone wants to use a different one, etc.

@stephentoub
Do I understand it correctly that your plan to use Buffer and Span is like this: take Buffers on the API edges, use Spans in the synchronous parts of the implementations, and  when switching to async make copies for  the Buffers wrapping pointers?

cc @KrzysztofCwalina, @joshfree, @shiftylogic

Do I understand it correctly

Based on my current understanding, I think it would make sense for guaranteed always synchronous APIs to take Span<T> and for possibly asynchronous APIs to take Buffer<T>.  But the APIs wouldn't make any copies: it would be the responsibility of the caller to ensure that the supplied Buffer<T> is valid for the lifetime that's necessary.  Higher-level types could help with that, but the API should not and would not make a copy, nor would Buffer<T> itself do any kind of reference counting, etc.

Perhaps we should settle on a mental model for Span and Buffer that aligns with the BCL's desire for an abstract slice, and with Pipeline's heavy use of pooling.
The way I see it, both Span and Buffer are types that don't own their contents, where Span has a minimal lifetime and Buffer has a maximal lifetime. The way we're managing the lifetime for Span is through its by-ref constraints. The way we're managing the lifetime for Buffer is through reference counting against an explicitly typed owner.
I feel like Buffer as it stands makes a good effort to encourage memory safety, and I would be interested to see what falls back on the user if it becomes their responsibility to manage the lifetime of buffers taken from arbitrary memory, without the OwnedBuffer to do it for them.
What do you think @stephentoub? Please correct me if I'm wrong.

This actually wouldn't change anything with pipeline's usage much as life time the OwnedBuffer<byte> is managed by the pipe itself. For other scenarios though, the caller has to manage the lifetime of the buffer which IMO isn't that big a deal. The bigger issue is that there is no first class way to pass around a life time aware slice of OwnedBuffer<T>. That's basically what Buffer<T> is today. It would make it possible to do something like this:
OwnedBuffer<byte> ownedBuffer  = pool.Lease(1024);
ownedBuffer.AddReference();
await Stream.WriteAsync(ownedBuffer.Buffer);
ownedBuffer.Release();

public class CustomStream : Stream
{
    private List<Buffer<byte>> _owned = new List<Buffer<byte>>();
    public override Task WriteAsync(Buffer<byte> data)
    {
         // Take ownership of the buffer
         data.AddReference();
         // Add it to a list
         _owned.Add(data);
    }

    // Imagine some background thread flushing the bytes on some interval
   public void OnTimerCallback()
   {
        foreach(var buffer in _owned)
        {
             WriteToSomethingElse(buffer);
              // Now I'm done with it
              buffer.Release();
        }
   }
}

The goal here to have an exchange type that enables building systems that can be buffer lifetime aware. Without the slice type, it would require passing both the OwnedBuffer<T> and the Buffer<T> Maybe the answer is the have something like:
Life time aware types

class OwnedBuffer<T> - A reference counted aware buffer
struct OwnedBufferSpan<T> - A struct that represents a slice of OwnedBuffer<T> (libraries can help manage the lifetime e.g. upping the reference count when there's a pending async operation so that native memory can't be cleaned up via OwnedBuffer during that call)

Non lifetime aware

struct Buffer<T> - A heapable span of memory (up to the caller to manage lifetime, libraries cannot handle lifetime on behalf of the caller)

Stack only

struct Span<T> - Stack only pointer like type

It would be possible to go from:

OwnedBuffer<T> -> Span<T>
OwnedBuffer<T> -> Buffer<T>
OwnedBufferSpan<T> -> Span<T>
OwnedBufferSpan<T> -> Buffer<T>
Buffer<T> -> Span<T>

Thoughts?

This is along the lines of what I had in mind, at least in terms of the separation of concerns:

The workhorse here is Buffer<T>.  It's the primary type that represents the memory, it's the most generic thing that can be passed around anywhere stack or heap, etc.  Most APIs in a general purpose framework would work in terms of Buffer<T>.
We then have Span<T> as a performance optimization, which we pay for with its stack-only-ness.  Some APIs in a general purpose framework may take Span<T> rather than Buffer<T>, if the developer of it is sure that won't limit any potential future implementation that would require it to be on the heap.  Since Span<T> can't be converted to anything else (other than a raw pointer) but everything can be converted to it, it's the universal receiver and thus if an API can be written in terms of it, it should be.
Lifetime management can then be layered on top in separate types, e.g. your suggested OwnedBuffer<T> and OwnedBufferSpan<T>, even in a separate library.

That said, is there a reason the "owned"ness needs to be tied to Buffer<T>?  Rather than OwnedBuffer<T>, why not a more general-purpose type like:
public class RefCount<T>
{
    private T _value;
    private int _refCount;
    ...
    public RefCount(T value) { ... }
    public void AddReference() { ... }
    public void Release() { ... }
    public ref T Value => ref _value;
    ...
}

such that it could be used as a RefCount<Buffer<T>> but could also be used to reference count any other type (we already have custom versions of this floating around, e.g. SafeHandle)?  And if it's critical to enable the slicing scenario on this with a struct that could share in the reference counting of its parent, you could also have:
public struct ChildRefCount<T>
{
    private readonly RefCount<T>_owner;
    private T _value;
    ...
    public ChildRefCount(RefCount<T> owner, T childValue) { ... }
    public void AddReference() => _owner.AddReference();
    public void Release() => _owner.Release();
    public ref T Value => ref _value;
}

which could be used as a ChildRefCount<Buffer<T>>.  Assuming Buffer<T> is sliceable, then you can still do things like var slice = new ChildRefCount<T>(refCount, refCount.Value.Slice(...));, and if that's too onerous, helpers could be provided as extension methods to operate on RefCount<Buffer<T>> and ChildRefCount<Buffer<T>>, as they are today in BufferExtensions.
There may be some holes here we could patch, but I like this direction in general more because:

it keeps Buffer<T> and Span<T> as the two core pieces that general libraries can be written in terms of, without any interference from additional functionality like reference counting
it provides general-purpose reference counting in the form of RefCount<T> and ChildRefCount<T>, that can be used with Buffer<T> but can be used with other types as well
it enables a higher-level pooling library to pool RefCount<Buffer<T>>s, hand out RefCount<Buffer<T>>s and ChildRefCount<Buffer<T>>s, without requiring that such instantations be used in implementations and signatures everywhere.
etc.

@stephentoub, @davidfowl  I like this proposal. I think it tidies up the API nicely.  I guess the proof will be in how the rewritten code looks like.

I like the direction this is moving too, and the idea of a general purpose RefCount type. It would make the intent for APIs and fields that require a lifetime longer than transient clearer. So the idea would be APIs that need longer lifetimes can require RefCount<Buffer<byte>> at their edges. The danger would be people thinking they can just wrap a Buffer they don't 'own' in RefCount which isn't necessarily safe since the real owner of the memory that contains that buffer might not be participating in ref counting. For managed data though it should be fine, since a live reference back to the original object would keep it from being collected, right?
So if we've got APIs at the point where memory comes in to .NET that encourage safe abstractions then being management-scheme agnostic down the line could be ok.

Sorry for getting late to the thread. I think this is a good issue that we need to resolve. One way of resolving it is being discussed above (dependency free Buffer).
Another that we talked about in the past is to change OwnedBuffer into an interface that could be implemented by T[] and/or System.String. It seems like it would solve the problem @stephentoub outlined at the beginning of this issue, but it would also prevent tearing and result in less types/concepts in the system.
Thoughts?

It seems like it would solve the problem @stephentoub outlined at the beginning of this issue

Only partially.  You would still need to allocate to pass data into an API if you didn't have one of these T[] or Strings, e.g. if you had a pointer (such as some stackalloc'd memory, or data handed to you from a native lib like libcurl), if you slice'd an array, etc.

If you have a pointer to a stack allocated memory, you should probably not wrap it in Buffer. It would be a bug farm, as Buffer has different lifetime than stack pointer. For stack pointers, Span should be used.
You would not need to allocate anything for sliced arrays. Buffer stores index and count.

If you have a pointer to a stack allocated memory, you should probably not wrap it in Buffer. It would be a bug farm, as Buffer has different lifetime than stack pointer.

a) A stack-based pointer was just one example.  A pointer provided by some other native component would apply equally.
b) Even in the stackalloc case, that's up to the developer... if they choose to pass it around, they need to understand what they're doing.
c) We need to define BCL APIs that take the appropriate type.  We can't, for example, define a Stream.Read(Span) method, because many streams that are implicitly async-only in implementation will implement Read by delegating to ReadAsync and blocking, which means such a Stream.Read method would need to take a Buffer, not a Span.

Re a) I agree. It would be a downside. But I think one object allocation per native buffer might be acceptable.
Re b) I don't think it's "up to the developer". If they want to pass the buffer into their own API, they can just make the API take Span. If it's not their API (e.g. Stream.Read), then they should not pass a stack allocated pointer wrapped in Buffer; they have no way to ensure that the code will work, and because the API takes Buffer and not Span, it clearly communicates that it might need to store the buffer for later, and so it probably won't work with stack allocated buffers.
Re c) I think we decided Span for sync APIs and Buffer for async APIs. I think it's a good clarification to say that virtual sync APIs that might need to do async in their implementation need to take Buffer too.

But I think one object allocation per native buffer might be acceptable.

This seems to be assuming that the managed developer is creating and caching native buffers. What about the case where native code is creating/storing its own buffers and calling back to managed code with some pointer of data for the managed code to handle? For example, libcurl calls back to a registered handler every tone it had header or body data to be processed. It just calls the registered function with a pointer and a length. I'd like to use Buffer, but I don't want to allocate an object on every callback.

it clearly communicates that it might need to store the buffer for later

I disagree. See my Stream.Read example. It's a synchronous method. It doesn't store the argument for later. Stackalloc'd space would be fine. But it can't take a Span.

I think the way Span interacts with async is unfortunate but necessary. So if I think of Buffer as just a generalised Span that I would reach for first, then building safe abstractions over that for managing a Buffers lifetime if it needs to be practically longer than Spans sounds like a clear way forward.
If you pass a raw Buffer to a method then I think a reasonable expectation is that it must live at least until that method returns (so possibly after an await). If a method needs something longer then it should require an owned reference.
That's basically just an implicit version of what @davidfowl proposed.
So with that approach, @stephentoub's Stream.Read example would require Buffer, with the expectation that implementors won't hold that reference after the method completes. Which is fine, because they block on the async calls, and anyone else calling ReadAsync with borrowed memory would need to do the same before returning or require owned memory in their signature.

I think it will be very difficult to ensure that async methods "don't hold to Buffer after they complete". It might be doable for simple async methods, but the moment your async method calls some other async methods, which in turn call yet other, and so on, the code will get very complicated and opaque to the point where providing such guarantee will be impractical.
This is why I like the simpler rule better: span for sync, Buffer for async. When you pass buffer, you need to ensure it will not AV, by either using GCed memory or OwnedMemory that can be "poisoned".
cc: @jkotas, @mjp41

When you pass buffer, you need to ensure it will not AV, by either using GCed memory or OwnedMemory that can be "poisoned".

Side question: What is AV?

@ahsonkhan  Access Violation, normally caused by accessing freed memory.
@KrzysztofCwalina , @stephentoub is there a coding pattern that could work for when to use Buffer versus RefCount. For example, if you pass a Buffer when the method returns it should have kept a reference. If you pass a Buffer to an Async method, it can't keep it after awaiting it? If it lasts longer, then you need should take a refcount?

I think it will be very difficult to ensure that async methods "don't hold to Buffer after they complete".

@KrzysztofCwalina You're totally right, it's just a recommended pattern for the callee to know if they require a buffer with a minimal or maximal lifetime, like @mjp41 suggests. Applied recursively I think it would be reasonably safe, but a bit crummy because it means you have to trust this opaque stack to play by the rules when there isn't a strong incentive for individual APIs to do so besides an inability to give a Buffer to a method that requires an OwnedBuffer or RefCount<Buffer>. I think other proposals here are much better, but I'm assuming the always-ref-countable API is untenable in the BCL.

We enabled Memory<T> to wrap arrays directly.

@KrzysztofCwalina, is there an issue somewhere tracking adding an OwnedMemory<T>-derived type that let's me easily create one from a pointer and length? e.g.
namespace System.Buffers
{
    public sealed class NativeOwnedMemory<T> : OwnedMemory<T>
    {
        public NativeOwnedMemory(void* pointer, int length);
        ...
    }
}

so that I can still used Memory<T> with a pointer and length?

opend issue #1924


pip with ensure => latest can spuriously skip the install

The pip resource checks whether the latest version is installed using essentially pip search pkg_name  | grep -i INSTALLED.*latest. This can give false positives if you have one or more other related packages already installed, because pip search matches package names and descriptions. It doesn't seem to have an option for matching the package name exactly.
I observed this in practice when installing multiple Google API packages, for example:
  $pip_packages = [
    'google-cloud-core',
    'google-cloud-dataflow',
    'google-cloud-storage',
    'google-api-python-client',
  ]

  python::pip { $pip_packages:
    ensure       => latest,
    virtualenv   => $virtual_env_dir,
    owner        => $::user,
  }

And google-api-python-client spuriously matched a bunch of installed packages:
Debug: Executing with ...: 'pip search  google-api-python-client | grep -i INSTALLED | grep -i latest'
Debug: /Stage[main]/Python::Pip[google-api-python-client]/Exec[pip_install_google-api-python-client]/unless:   INSTALLED: 0.0.2 (latest)
Debug: /Stage[main]/Python::Pip[google-api-python-client]/Exec[pip_install_google-api-python-client]/unless:   INSTALLED: 0.5.0 (latest)
Debug: /Stage[main]/Python::Pip[google-api-python-client]/Exec[pip_install_google-api-python-client]/unless:   INSTALLED: 0.7.5 (latest)
Debug: /Stage[main]/Python::Pip[google-api-python-client]/Exec[pip_install_google-api-python-client]/unless:   INSTALLED: 0.22.1 (latest)
Debug: /Stage[main]/Python::Pip[google-api-python-client]/Exec[pip_install_google-api-python-client]/unless:   INSTALLED: 2.6.0 (latest)
Debug: /Stage[main]/Python::Pip[google-api-python-client]/Exec[pip_install_google-api-python-client]/unless:   INSTALLED: 1.6.0 (latest)

possible fix would be to modify the grep? like so