Retention time configurable per series (metric, rule, ...).

Hello,
I'm evaluating prometheus as our telemetry platform and I'm looking to see if there's a way to set up graphite-like retention.
Let's assume I have a retention period of 15d in prometheus and I define aggregation rules that collapse the samples to 1h aggregates. Is there a way to keep this new metric around for more than 15 days?
If this is not possible, could you provide some insight on how you approach historical data in your systems?
Thank you

This is not something Prometheus supports directly at the moment and for the foreseeable future. The focus right now is on operational monitoring, i.e. the "here and now".
You can get something like this by using a tiered system. The first-level Prometheus would scrape all the targets and compute the rules. A second-level Prometheus would federate from it, only fetching the result of these rules.
It can do so at a lower resolution, but keep in mind that if you set the scrape_interval to more than 5 minutes your time series will no longer be treated as contiguous. It can also keep them for longer. Theoretically this is only limited by disk space, however again, very long retention is not a focus so YMMV.
Additionally, the second-level Prometheus could use the (experimental) remote storage facilities to push these time series to OpenTSDB or InfluxDB as they are federated in. To query these you will need to use their own query mechanisms, there is no read-back support at the moment.

The "5min-problem" is handled by #398. The planned grouping of rules will allow individual evaluation intervals for groups. So something like a "1 hour aggregate" can be configured in a meaningful way.
The piece missing is retention time per series, which I will rename this bug into and make it a feature request. We discussed it several times. It's not a high priority right now, but certainly something we would consider.

A per job retention period is what I need for my use-case.
I pull 4 metric from my solar panel every 30 second, and want to store them forever (so I can for example go 6 months back and see the production at that momemt) but I don't need that for all the other metric (like Prometheus metric).

Prometheus is not intended for indefinite storage, you want #10.

Prometheus is not intended for indefinite storage, you want #10.

I see #10 make sense if you have a lot of time series, but OpenTSDB seems kind of overkill just to store 4 time series forever. Isn't it just a question of allowing people to set retention period to forever? or do you think people will "abuse" that?

We make design decisions that presume that Promtheus data is ephemeral, and can be lost/blown away with no impact.

Coming here from google groups discussion about the same topic
I think we could use some per-series retention period for recording rules and metrics it is based upon.
We have 3k hosts, which are reporting country they served requests from, we aggregate this values in recording rule, and basically never need raw metrics. But they are using RAM, storage etc.

I plan to tackle this today. So essentially it would mean this, regularly calling the delete API and in the background cleaning up the tombstones. Where should this live is the question.
My inclination is that we could leverage the delete API itself and then add a tombstone cleanup API, and add functionality to promtool to call the APIs regularly with the right matchers.
Else, I would need to manipulate the blocks on disk with a separate tool which I must say, I'm not inclined to do.
/cc @brian-brazil @fabxc @juliusv @grobie

One alternative is to make it part of the tsdb tool and "mount" the tsdb
tool under "promtool tsdb", which has other nice benefits.
That would make the functionality usable outside of the Prometheus context.
Prometheus users would need to run 2 extra commands for disable/enable
compaction. Or just wrap those around it when calling via promtool.
On Wed, Nov 22, 2017 at 6:29 AM Goutham Veeramachaneni <
<EMAIL_ADDRESS>wrote:

I plan to tackle this today. So essentially it would mean this, regularly
calling the delete API and in the background cleaning up the tombstones.
Where should this live is the question.
My inclination is that we could leverage the delete API itself and then
add a tombstone cleanup API, and add functionality to promtool to call the
APIs regularly with the right matchers.
Else, I would need to manipulate the blocks on disk with a separate tool
which I must say, I'm not inclined to do.
/cc @brian-brazil https://github.com/brian-brazil @fabxc
https://github.com/fabxc @juliusv https://github.com/juliusv @grobie
https://github.com/grobie
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/prometheus/prometheus/issues/1381#issuecomment-346255896,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEuA8tV_IIlR7d8IDAAyISIhpKG06IHaks5s478rgaJpZM4HXqa7
.

My concern there is the edge-cases, what if the request to restart compacting fails? While the tsdb tool makes perfect sense on static data, I think it would be cleaner if we could make it an API on top of tsdb.DB that the applications built on top can leverage.
For the tsdb tool case, if we know we are acting on static data then we can instantiate a DB and work with that. We can have two options for promtool delete live and delete static though I highly doubt anybody will be working with static dirs.

I wouldn't object to delete and force cleanup functionality being added to promtool.
I have a general concern that users looking for this tend to be over-optimising and misunderstanding how Prometheus is intended to be used, such as the original post of this issue. I'd also have performance concerns with all this cleanup going on.

don't think anything can be done on the tsdb side for this so removed the local storage label.
Doesn't seem there is a big demand for such a use case and since the issue is so old maybe should close it and revisit if it comes up again or if @taviLaies is still interested in this.

A few of us had discussions around this at KubeCon and find dynamic retention valuable for both Prometheus and Thanos. Generally the approach we were discussing is to include the tool within the Prometheus code as part of compaction, and allow users to define retention with matchers. Design doc will be coming soon, but I am happy to hear any major concerns around compaction time processing sooner than later so I can include them.

Compaction is currently an entirely internal process that's not exposed to
users, and in particular does not affect query semantics.
I'd prefer to ensure that we'd expose the delete API via promtool, and let
users work themselves from there.
On Fri 22 Nov 2019, 16:25 Chris Marchbanks<EMAIL_ADDRESS>wrote:

A few of us had discussions around this at KubeCon and find dynamic
retention valuable for both Prometheus and Thanos. Generally the approach
we were discussing is to include the tool within the Prometheus code as
part of compaction, and allow users to define retention with matchers.
Design doc will be coming soon, but I am happy to hear any major concerns
around compaction time processing sooner than later so I can include them.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/prometheus/prometheus/issues/1381?email_source=notifications&email_token=ABWJG5RNVF2DGM4KKJ4LVYTQU72YLA5CNFSM4B26U252YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE5645Y#issuecomment-557575799,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABWJG5TUETHIMM6H36RYWNTQU72YLANCNFSM4B26U25Q
.

I think in this case it's justified to expose this to users – "I want to keep some metrics longer than others" is such a common use case that I don't think we should relegate it to "write your own shell scripts". The impact on query semantics doesn't have to be explicitly bound to compaction – it can simply be "samples will disappear within X hours after they have reached their retention period".

There's a few unrelated things being tied together there. One thing we do know is that users tend to be over-aggressive in their settings, which then causes them significant performance impact. This is why we don't currently have a feature in this area, the last person to investigate it found it to not work out in practice.

is such a common use case that I don't think we should relegate it to "write your own shell scripts".

It'd be a single curl/promtool invocation, so it's not something that even really classifies as a shell script.

It would still need to be executed regularly to fulfill the need. So it
needs to be scheduled, monitored, updated.
When would the corresponding space be freed?
On Mon, Nov 25, 2019 at 11:30 AM Brian Brazil<EMAIL_ADDRESS>wrote:

There's a few unrelated things being tied together there. One thing we do
know is that users tend to be over-aggressive in their settings, which then
causes them significant performance impact. This is why we don't currently
have a feature in this area, the last person to investigate it found it to
not work out in practice.
is such a common use case that I don't think we should relegate it to
"write your own shell scripts".
It'd be a single curl/promtool invocation, so it's not something that even
really classifies as a shell script.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/prometheus/prometheus/issues/1381?email_source=notifications&email_token=AABAEBTBGXUAUXYKI7XPUETQVOZNXA5CNFSM4B26U252YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFCCRIY#issuecomment-558114979,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AABAEBUUG7YFKLHJPDI57K3QVOZNXANCNFSM4B26U25Q
.

It would still need to be executed regularly to fulfill the need. So it
needs to be scheduled, monitored, updated.

Cron covers that largely, plus existing disk space alerting.

When would the corresponding space be freed?

Typically it'd be automatically within 2 hours. Unless they trigger it manually (which is where performance problems tend to come in, this gets triggered far too often).

How close can one get to an ideal scenario where a user is not made to worry about what to retain for how long, but instead the system adapts to a storage quota?  It could track actual query usage of metrics and their time windows, so it can predict metrics / times that are likely to be unneeded, and prefer them for disposal.

That's not possible, all it takes is one overly broad query and everything gets retained. If you want to try to build something like that it would be best done outside Prometheus.
If you just want to have an overall byte limit, we already have a feature like that.

I think that ease of use is desirable and worth it for this feature. Otherwise users have to allow the admin APIs, protect them, and standup whatever cron type job they need to use, even if the calls are made easier with promtool. I will add an alternatives section to try to do a more detailed analysis in the doc.
I plan to have compaction continue to be an internal detail of when samples are deleted. Compaction is a convenient time to do the work, and it is already mentioned in the storage documentation that it may take 2 hours for data to be removed.

the last person to investigate it found it to not work out in practice.

Any links on this that you know of? I would love to learn from them.

I think that ease of use is desirable and worth it for this feature.

I question that. Many users seem to be holding on to how things work with other less efficient systems, and where things like Thanos are in play doing it there would make more sense.

it is already mentioned in the storage documentation that it may take 2 hours for data to be removed.

Removed from disk is not the same as no longer accessible to queries, which happens by the time the delete call is complete. Any changes from that would be quite messy semantically, and is a reason not to go deleting recent data regularly.

Any links on this that you know of?

That's @gouthamve posts above back in 2017, which I think he only wrote the results of in IRC. The summary was that forcing compaction every 5 minutes is a very bad idea, so he gave up.

That's only at the bounds of full retention, and IMHO we should keep the 1.x behaviour of having a consistent time for that. It's not the last few hours of data with typical retention times.

I have started a design doc for this work here: https://docs.google.com/document/d/1Dvn7GjUtjFlBnxCD8UWI2Q0EiCcBEx_j9eodfUkE8vg/edit?usp=sharing
All comments are appreciated!

Is there any progress on this issue? I had similar problem. I want to monitor total errors count on  networks switches, but on some of them there isn't snmp oid for total errors. So i should get different types of errors(CRC, Aligment, Jabber etc.) and calculate sum of them. But i want to keep only total errors, not others.

No progress to report, there are still many unresolved comments in the design doc I put forward, and I have not had the time or energy required to get consensus. There is some work related to this in Thanos that has been proposed for Google Summer of Code (https://github.com/thanos-io/thanos/issues/903).
If you only need to delete certain well known series, calling the delete series api on a regular schedule is an option.

Sorry, I was unclear. The storage documentation already says that blocks will not get cleaned up for up to two hours after the have exceeded the retention setting.

FWIW I can live with that.  Ceph already behaves sort of this way when deleting RBD volumes.
In my situation, there are metrics that aren't likely to be useful past, say, 30 days like network stats.  Others could have value going back for months, eg. certain Ceph capacity, performance, etc. metrics.  Ideally I'd love to be able to downsample older metrics - maybe I only need to keep one per day.
Use-case:  feeding Grafana dashboards and ad-hoc queries.  The federation idea is clever and effective, but would complicate the heck out of queries.  I would need to duplicate dashboards across the 2 (or more) datasources which doesn't make for the best user experience, and is prone to divergence.

Has this been looked into any further by the development team? Or have any users found any work arounds? This would help me a lot with my dashboards.

My workaround is to deploy a VictoriaMetrics next to the prometheus. Then configure Victoria to scrape prometheus but filter which metrics to scrape, have different retention, and loose granularity.
Command flags:
      - -retentionPeriod=120 # 120 months
      - -dedup.minScrapeInterval=15m

promscrape.config
scrape_configs:
  - job_name: prometheus
    honor_labels: true
    metrics_path: '/federate'
    params:
      'match[]':
        - '{__name__="metric1"}'
        - '{__name__="metric2"}'
    static_configs:
      - targets:
        - 'prometheus:9090'

Has this been looked into any further by the development team? Or have any users found any work arounds? This would help me a lot with my dashboards.

We decided to put in place clean up policy (Prometheus REST API) - default is 12 weeks, but a part is cleaned up after 6 weeks. We considered this cheaper to maintain than several instances.

Has this been looked into any further by the development team?

There is a topic for a prometheus dev summit to discuss this issue. I am hopeful that we will discuss it either this month or in January, but I cannot say for sure. After the discussion, we will be able to provide a more complete answer as to how we would like this in (or not in) Prometheus.

Or have any users found any work arounds?

The current ways to do this are either with federation to a second Prometheus instance, or having an external process call the admin delete API.

This would be a really great feature for prometheus. The use case of this feature is not only for long term metrics (which some people argued in the comments that is not the prometheus intend). There are lots of expensive metrics which we want to be able to have them only for a single day, but the rest of the metrics for 15 days. So now we have to operate 2 prometheus instances. Apart from operation, some of our queries needs matching operators to filter the metrics based on the series in the other instance, so it also makes it harder. I know there is thanos option for global query of multiple prometheis, but it is overkill to use it only for not being able to have some metrics for a shorter time.

It would be nice to revisit this 🤗
There are big wins if we have something like this: Prioritizing, Aggregations, satisfying data for alert only and discard etc
cc @csmarchbanks wonder if it's time resurrect your proposal (:

If it doesn't get discussed at the upcoming dev summit, perhaps let's get a few interested parties together to get it moving without a dev summit? It's been on the agenda with a fair number of votes for quite awhile now so I hope it gets discussed.

Good news!
There was consensus in today's dev summit that we would like to implement dynamic retention inside of the Prometheus server. The next step is to decide how we would like to implement this feature. Right now it looks like there are two proposals in the document I linked, one for a new format that allows reducing or extending retention based on a set of matchers, and a second building on rule evaluation to delete data that is older than  age. Anyone who is interested, please provide feedback on either of those approaches (or a new one) so that implementation work can begin.

Hi, I would like to tackle this issue as my GSoC'21 project.
For now, I've read through the discussions and docs noted below.

This thread and links

#398
#10
thanos-io/thanos#903
https://groups.google.com/d/topic/prometheus-users/PhGs6luJWYs/discussion

Design Doc and comments: https://docs.google.com/document/d/1Dvn7GjUtjFlBnxCD8UWI2Q0EiCcBEx_j9eodfUkE8vg
TSDB formats: https://github.com/prometheus/prometheus/tree/main/tsdb/docs/format

Let me know if there are any other existing discussions I should read.
As a first step, I'm going to do some code readings and figure out the dependencies.
And it would be nice to learn through practice, so let me know if there are any related good first issue.
I have several questions but I'm not sure what steps I should take, so I want to decide how to proceed first.
I mean we're going to decide on the detailed specifications based on the proposal sooner or later, I want to share the technical assumptions.

@FujishigeTemma Those are great discussions to start, if you have questions about GSoC feel free to reach out to me via email or in the CNCF slack. Otherwise, part of the GSoC project will be to make sure a design is accepted and then start implementing it.

As this project was not selected in GSoC this year, do we have any other updates or progress on this?

As this project was not selected in GSoC this year, do we have any other updates or progress on this?

There is no progress.

Based on the design doc https://docs.google.com/document/d/1Dvn7GjUtjFlBnxCD8UWI2Q0EiCcBEx_j9eodfUkE8vg/edit#, for config like:
retention_configs:
- retention: 1w
  matchers:
  - {job=”node”}
- retention: 60d
  matchers:
  - slo_errors_total{job=”my-service”}
  - slo_requests_total{job=”my-service”}
- retention: 2w
  matchers:
  - {job=”my-service”}

An approach would be:

For any retention time < global retention time: in the reloadBlocks method, block.Delete() will be called to add tombstones to each block. The actual data will be deleted during compaction or clean_tombstones API.
For any retention time > global retention time: before deleting the block, rewrite the block and keep any matched series chunks for longer retention. Then delete the original block.

To achieve 2, we can extend the compactor interface with modifiers like https://github.com/prometheus/prometheus/pull/9413:
type Compactor interface {
	// Write persists a Block into a directory.
	// No Block is written when resulting Block has 0 samples, and returns empty ulid.ULID{}.
	Write(dest string, b BlockReader, mint, maxt int64, parent *BlockMeta, modifiers ...Modifier) (ulid.ULID, error)
}

// Modifier modifies the index symbols and chunk series before persisting a new block during compaction.
type Modifier interface {
	Modify(sym index.StringIter, set storage.ChunkSeriesSet, changeLog ChangeLogger) (index.StringIter, storage.ChunkSeriesSet, error)
}

We can define a retention time & matchers aware modifier to only keep the chunkSeries we want.
This approach should work, but performance is a big issue as we have to iterate through all series for each block every 1 minute.
Example code for the modifier: https://github.com/yeya24/prometheus/blob/experiment-modify/tsdb/modifiers.go#L173-L272

@yeya24  Is the implementation going to merge ? What is the stage of the design doc?


use env variable for GA log setting

Thanks @Mitso. Yeah, the second argument just supplies a default value in case the key we're looking for doesn't exist.


[carbonmark-dashboard] Research: Fix imports from the lib workspace

Tree shaking does not seem to work when Importing the barrel file from the lib workspace.
eg. Doing import { getLocales } from "@klimadao/lib/utils"; in carbonmark-data/lib/i18n seems to compile all the imports listed in the barrel file§.

@Atmosfearful


Context dependent spatial randomization

Purpose
Add context based spatial randomization to spatial analysis package.  This restricts the positional randomization to occur between pairs of contexts (catagorical data, e.g. metaclusters), as opposed to global, whole fov positional randomization.
Implementation
calculate_*_spatial_enrichment functions have an additional context_col argument for a context column in the provided cell table.  If this argument is None, global randomization is used, but otherwise, all unique context pairings are calculated, and randomizations are repeated and summed for each pairing.
Note that this makes no changes to the compute_close_cell_num_random function, and instead alters the given arguments.  This prevents requiring a new randomization function with several new arguments to accommodate context-dependence.
Remaining issues
Only calculate_cluster_spatial_enrichment has been templated.  There are no test functions at the moment.  Currently, this change is not represented in the spatial analysis notebook.

This is the general idea for adding context dependence.  Lmk if anything looks blatantly wrong.  Also, @alex-l-kong if you have some example/test data, I'm going to need that, to make sure everything's working as expected.

Got it, looks good to me

previously, the context based randomization functioned as follows:

for all cells in each context pairing, all cells of lineage (i, j) were randomly placed within the allowable context labels

now, the added randomization asymmetry warrants a revaluation of what we want from the context.  The interpretations I can see:

a cell (lineage=i, context='square') chooses |num_cells[j]| from all cells in context='square' (doesn't feel right, plus what if |'square'| < |num_cells[j]|)
a cell (lineage=i) chooses |num_cells[j]| where all cells have been randomized within their respective contexts (feels right).

Okay, neither of those is that intuitive to me. We can discuss tuesday. The original implementation of context-random should be in the matlab code too

kk, we'll talk Tuesday.  Reading over the matlab version, I'm pretty sure the original implementation is symmetric in the sense that each marker/lineage pair was randomized, not just marker/lineage j.

Looks like the diff is broken for some reason, so to see the actual difference between this branch and the target, you can use this link:
https://github.com/angelolab/ark-analysis/compare/spatial-analysis-nb-update...context-dependent-randomization

Updated diff is super concise, no complaints on my end. Maybe once you merge this into the main branch github will update? Probably the result of it not being refreshed in a while


Fixed height with fixed column's scrolling issue

Fixed height with fixed column that unable to scroll actual table (vertical). Understand fixed column is overlay in front of actual table. Is there a way we can scroll(vertical) actual table when mouse within fixed columns area.
demo sample: http://jsfiddle.net/wenyi/e3nk137y/2954/

This extension has moved to bootstrap-table, the issue has been fixed.

https://bootstrap-table.com/docs/extensions/fixed-columns/
https://examples.bootstrap-table.com/#extensions/fixed-columns.html


Updated go-restful to v3.11.0

Closing it since created another PR and getting conflicts while building image.


consider existing sharing tools

I wanted to drop a note here that the best sharing experience I've seen for a notebook deployment is CERN's, which uses CERNBox, an instance of ownCloud, an open source Dropbox clone. It allows sharing, granular permission management, browsing, etc. It's much more full-featured than hubshare is intended to be. It also requires zero integration or awareness from JupyterHub or the single-user notebook. It's all in the setup of the environment in which notebooks are run.
It may make more sense in the end to build a JupyterHub deployment that works with ownCloud / Nextcloud / etc. than to build and maintain our own sharing service that's as severely scope-limited as we are planning to make hubshare. Such an integration could also have a much smoother transition for real-time collaboration, which is orthogonal to hubshare as planned.

I strongly believe in this too.
On Wed, Feb 1, 2017 at 1:16 PM, Min RK<EMAIL_ADDRESS>wrote:

I wanted to drop a note here that the best sharing experience I've seen
for a notebook deployment is CERN's, which uses CERNBox
http://information-technology.web.cern.ch/services/CERNBox-Service, an
instance of ownCloud https://owncloud.org, an open source Dropbox
clone. It allows sharing, granular permission management, browsing, etc.
It's much more full-featured than hubshare is intended to be. It also
requires zero integration or awareness from JupyterHub or the single-user
notebook. It's all in the setup of the environment in which notebooks are
run.
It may make more sense in the end to build a JupyterHub deployment that
works with ownCloud / Nextcloud / etc. than to build and maintain our own
sharing service that's as severely scope-limited as we are planning to make
hubshare. Such an integration could also have a much smoother transition
for real-time collaboration, which is orthogonal to hubshare as planned.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/jupyterhub/hubshare/issues/14, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAB23muwHU3806fsYFfAoBLS5qosRpv9ks5rYPYRgaJpZM4L0ZXu
.

--
Yuvi Panda T
http://yuvi.in/blog

owncloud is a a php app and it would be nice to not install another stack.
We currently install a jupyter server without any user separation (just folders) because sharing isn't currently included in jupyter hub (Security is handled by a VPN). What we need is some kind of "drop it there and others can see (and maybe play with) it", so this repo looks (or looked?) very promising.

@minrk and others: Microsoft Azure has done a pretty nice job with their sharing. https://notebooks.azure.com/faq#libraries

Perhaps they would open source some of their code. Or collaborate with us here.

It may make more sense in the end to build a JupyterHub deployment that works with ownCloud / Nextcloud / etc. than to build and maintain our own sharing service that's as severely scope-limited as we are planning to make hubshare.

Our current off-the-shelf setup relies on nbviewer in docker with user notebook directories mounted from local disk plus the nbexamples extension. It was quick and easy to setup, but we definitely want to go beyond it. The key features we are looking for are:

Search across all shared notebooks (around 7000)
Reasonable previews of notebook hits (title, summary, tags, thumbnail, ...)
Rendered notebooks without having a notebook server (reports for non-notebook users)
Clone a notebook of interest to the user's notebook server
Click to share a notebook from the user's server
Some sense of the notebook's provenance and history

I realize that's not a trivial list.
I've been looking at https://github.com/airbnb/knowledge-repo and thinking about how we might build atop it. I've also been thinking about what a notebook catalog backed git with LFS might look like if we just slap a search index and UI on top. And, of course, I'm looking at what's being discussed here in hubshare to see if it might shape up to fit our use cases instead.
I don't have any strong recommendations or opinions to share yet. Just putting some info out in the open.

I've been lurking within hubshare to see where things are going while building something a bit different in commuter with @cabhishek and other folks.
I don't wish to be prescriptive on how everyone should do their own setup. My explanation on these is mostly to paint a picture of what we have in progress and where we're heading.
Background and motivation

Our Notebooks are stored on S3 (in one bucket)
People want to share notebooks (both live/real-time and statically published)
People like to share URLs
People even share URLs with themselves*
We want to iterate on this quickly

* Real-time sessions with themselves totally have to happen.
Commuter Roadmap
Our Roadmap lists out our stages of development going from:

View notebooks via URL (from S3 underneath)
Connect to kernels from a configured source (JupyterHub)
Save notebooks back
Create server side in-memory model of notebook and transient models, push to all clients

We plan on shipping during each of these stages fairly rapidly and getting feedback.
Publishing
For us, the act of sharing or publishing is simply editing. There is no distinction of publishing - you can share with other people on the same server immediately. Users want to share with URLs like:
https://notebooks.mysite.com/kyle/some-notebook.ipynb

On the current version, when you load a page you're using an implementation of the contents API on top of S3. If people don't have a notebook server up, they should still be able to view the content. This is enabling that.
All of this is leading us towards editing and running notebooks, using the realtime model we've been carving out. Long term is as close to Google docs style collaboration as possible.
As for indexing and discovery, we intend to maintain a separate service for indexing notebooks in elasticsearch that can be built (or re-built) from our existing S3 bucket.
Since we know full well that we may want to try different approaches to some of this, we aren't just publishing our opinionated server to npm - we're also pushing out individual React components (commuter is a monorepo that uses lerna underneath). We've got some UI mockups if you want to check those out too.

As an aside, we are steadfastly stabilizing nteract/nteract while exporting packages from inside it to use externally. The goal there being to use them in commuter as well as providing our transforms (renderer in JupyterLab parlance).

Thanks for you use cases!
I've been thinking about this as well, and another approach, inspired by conversations with @rgbkrk, is building on top of the contents API, rather than starting from scratch. The Jupyter Server proposal should make it easier to deploy a server that's just the contents API, and could be made to allow extending the contents API to add things like ownership, sharing, etc., but I think I can import contents from notebook 4.x without much difficulty.
Such an application could start with:

use HubAuth on top of contents service
add sharing/permissions to contents API
add download/upload zips for directories

and that should have all of the features of our current proposal, short of the hash. The mechanism for moving files between hubshare and single-user environments would be unchanged.
benefits:

Any client that can browse/edit/index/etc. the contents API can talk to HubShare to browse readable files (JupyterLab, etc.). Just like the server being a subclass, existing clients could also be subclassed to support the added features.
In the future, this contents API could be used for collaboration - talk directly to it from one (or more) single-user servers rather than only hubshare-dedicated endpoints.
Can more easily support sharing at any sub-level (single files, subdirectories, etc.)
Should inherit our CORS/etc. stuff without reimplementation, which is getting tiresome.

downsides relative to the current design:

The single hash for a share (directory) no longer makes sense unless we do something more complicated with versioned storage, which the contents API could have trouble supporting. I'm not sure if this matters.
Scope of the whole application is a little bigger, but the scope of new code is a little smaller, if anything.
Others?

I'm increasingly convinced that building on top of contents is a good idea. Enough so that I'll try to sketch out a more formal spec for how it would work, and maybe even sketch what an extended-contents server would look like, pulling from notebook 5.0.

@minrk, speaking as someone who's done a fair amount of work building things on top of the Contents API, I think building sharing on top of / as an extension to the Contents API sounds very plausible.  At the very least, re-using the file and directory model definitions for a sharing protocol strikes me as a good idea.
For what it's worth, pgcontents already has a notion of users associated with the files it stores, and I've long been of the opinion that someone could add sharing support to it in a day or two by adding a "shared_notebooks" table or something like it.  We (at Quantopian) actually use S3 as the persistent storage for shared notebooks, but that's mostly because we wanted to easily integrate with an existing non-Jupyter application (our forums), and because our sharing model is "publish this notebook to the world" rather than "share this notebook with a particular set of users".  If we ever build limited user-to-user sharing, it will almost certainly be in terms of a Contents API extension.

Thanks, @ssanderson!

After various conversations this week, I think I want to separate the 'push/pull from notebook servers' task (main goal here) from the 'public index/search/discovery' task that comes up a lot when you start using words like 'share'. I'm going to explore deploying commuter behind JupyterHub as the discovery application, and then scope-limit HubShare as 'just' a multi-user contents API that users can push to / pull from. If both commuter and HubShare are talking to the same storage, then deploying them together makes sense. In particular, this may mean no UI pages served by HubShare, only the JupyterLab extension.
It seems that for lots of deployments where there is already a shared filesystem and/or a backing/syncing store, then HubShare wouldn't be needed, only commuter. I've talked to @rgbkrk about supporting local storage for commuter, which would make this cover a pretty wide set of use cases, I think.

@minrk is there any way where I can share notebooks in Jupyterhub

We are running in an OpenStack cloud, so I wrote a Notebook ContentsManager plugin - and whilst the tree view would store files using SwiftStore, the problem was when a user want to do os.open(...) within the notebook (to access the data given for the course-work) - that goes straight to the local disk.
If one is trying to supply persistent data-storage, we've found NFS as the only viable tool for off-machine storage.
(We also tried using cinder as a block-storage medium.... not particularly successfully)
I think it's important not to lose sight of users persistent store, over and above just handling .ipynb files.

@arfon Here are some relevant materials that I ran across (CERNBox documentation is sparse currently):

http://iopscience.iop.org/article/10.1088/1742-6596/664/6/062037/pdf (Nice 2015 paper)
https://zenodo.org/record/.../cernbox_petabyte_scale_sync_and_share_platform.pdf (A much more detailed write-up on the subject - it's a thesis, apparently written by one of CERNBox core developers)
https://cernbox.github.io/reva (CERNBox middleware - not covered by the above-linked thesis - that seems to facilitate integration between ownCloud and the rest of CERNBox; it seems to me that REVA is designed and implemented nicely, with a generic revad server and ownClowd-specific ocproxy)
https://wiki.geant.org/display/OCM/Open+Cloud+Mesh (EU federated research cloud initiative, originated at ownCloud and compatible with / based on both ownCloud and NextCloud)
https://github.com/cernbox/uboxed (Single-host containerized CERNBox)
https://github.com/cernbox/kuboxed (Scalable, K8s-based CERNBox) - Yay! IMO, a good match to Z2JH.

@minrk I would suggest using NextCloud instead of ownCloud (the former is a clone of the latter) due to a more liberal software license (enterprise features are open source in NextCloud, unlike in ownCloud -- this requires some additional investigation; I suspect that a small number of ownCloud's enterprise features are not present within NextCloud's codebase, e.g., Workflows) as well as arguably more dynamic and larger open source community.

Unfortunately my level of contacts with the people at CERN who work on SWAN (what they call their JupyterHub deployment I think) is not enough to persuade them to make more public documentation and take a bigger part in the JupyterHub community.
From what I gather they used FUSE to mount owncloud directories in the docker containers that get launched by users. This makes it easy to sync (or removes the need for it)  work between your laptop, your hub home directory and other places. Not sure how/if it solves the problem of sharing between users.
Plug: Because sync'ing files between my "home directory" on a hub that runs on docker/kubernetes (can't use ssh to copy files) and my local machine is so tedious I've been investigating syncthing (binder demo https://github.com/betatim/binder-syncthing). It works (files are sync'ed two ways) but after about 60s in the web UI of syncthing an error message pops up. It is related to trying to auto update syncthing (I think), if anyone has ideas on how to disable that check I'd love to know.

@georghildebrand found a solution for syncing files between your laptop, your hub home directory and other places. Georg could you fill in your approach? I remember was based on some P2P open source solution?

For those interested, I maintain at work a JupyterHub instance and we use Nextcloud to sync the notebooks between the JupyterHub server and the users computer.

NextCloud is setup to keep users data under /opt/nextcloud/html/data/<username>
On JupyterHub server, /opt/nextcloud/html/data is mounted as NFS
We use AD to authenticate to JupyterHub
When a user login to JupyterHub, it spawns a docker container and /opt/nextcloud/html/data/<username> is mounted as a volume inside the container

This allows users to synchronise their notebooks and data between JupyterHub and their computer using NextCloud.
Note that this doesn't allow to share files between users :-(
Only files put directly in the user nextcloud directory are visible. Shared links, groups are not kept under /opt/nextcloud/html/data/<username> and can't be seen.
So I'm still looking for an easy way for users to share notebooks together.

I've used nbexamples extension. You can only share notebooks (no data). This is not compatible with JupyterLab.
I've just created a jupyterlab-gitlab extension based on jupyterlab-github to share notebooks via our internal GitLab instance.

Still not at the integration level I'd like to have.

I posted a comment on #17 but also wanted to record part of it here.
A key requirement of hubshare is treating directories as immutable, and being able to track through a directory's lineage, through a linked list of shas (both from the versioning and sharing perspective), and to have different permissions of different shas. This requirement came from our experience trying to handle homework distribution and collection for nbgrader, but is also relevant in other contexts.
The focus on directories also comes from that context, as well as the experience from binder that directories are a good "unit of reproducibility".
I think the idea that @minrk has pitched before (of content manager + permissions)  would also be very useful, but I don't yet see how to square that with the requirements of hubshare than is currently spec'd out in the REST API in this repo. Maybe they are separate things? Maybe there is a way to unify them?
From a philosophical perspective - one of the things that Jupyter has been good at is innovating on open protocols and standards, and encouraging the broader community to build around and on top of those things. I view hubshare (or more generally - protocols for document/directory sharing) as one of those things that is important enough for Jupyter to take our time and build it in a way that lives up to the vision we have established in our other standards (kernel protocol, jupyter server REST APIs, notebook document format, parts of JupyterHub, etc.).


How to run my own hooks also?

How to run hooks, that now placed in .git/hooks/old-hooks ?

My understanding is that Overcommit is intended to replace your existing hooks, allowing you to duplicate their behavior with custom hooks within the framework.

@jawshooah correct. But there is some several hooks, that hard to reimplement manually. (for example I use nice tool: https://github.com/mroth/lolcommits ) and it also put custom hook to this folder.
What about such enchantment:
Make an potion to durring installation not just move away existing scripts, but add execution of them as custom hook with :pass return. What do you think about this option? Is it possible to implement?

Well, you could add a custom hook that simply executes .git/hooks/old-hooks/pre-commit and handle the output and return code however you like.

Hey @skywinder,
As @jawshooah mentions, the easiest way is to execute your script in the hook itself. See the GerritChangeId commit-msg hook for an example.

@jawshooah already did, thanks!
@sds :+1: thanks for awesome tool.


New spam came through today

This is what I got in inbox today:
Text To Speech In 3 Clicks - Real sounding human voices!. For a free demo. Write a reply here<EMAIL_ADDRESS>

Thanks for the information.
I have added a couple of phrases from the text you provided to the blacklist and they will be available as soon as I push a fresh commit this week.

Thanks for the information.
I have added a couple of phrases from the text you provided to the blacklist and they will be available as soon as I push a fresh commit this week.

The following terms were added in commit 65f1967 to address this issue.

s!.
write a reply here

The following terms were added in commit 65f1967 to address this issue.

s!.
write a reply here


replace 2&>1 by 2>&1

Hi,
This follows the point here: https://lists.debian.org/debian-qa/2023/02/msg00052.html
Thanks!

nice, thanks!

Out in 1.6.19


ucblogo: init at 6.2.4

Description of changes
Supercedes https://github.com/NixOS/nixpkgs/pull/204748

Things done

Built on platform(s)

[x] x86_64-linux
[x] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Causes staging due to use of an alias. Proposed the fix as:

https://github.com/NixOS/nixpkgs/pull/308612


Incorrect C# client generation for polymorphic types

I've got a Swagger server hosted using Swashbuckle.AspNetCore v5.0.0-rc2 that contains a controller that has a polymorphic return type (specified using oneOf in the attached swagger.json) and NSwagStudio v12.1.0 seems to be generating incorrect non-compilable client methods using a generic type of "Anonymous" for one of the endpoints, and one of the sub-types instead of the base type for another endpoint.
This:
public System.Threading.Tasks.Task<System.Collections.Generic.List<Anonymous>> SourcesGetAsync()
should be:
public System.Threading.Tasks.Task<System.Collections.Generic.List<SourceDefinitionBaseDto>> SourcesGetAsync()
and this:
public System.Threading.Tasks.Task<ApplicationSourceDefinitionDto> SourcesGetAsync(System.Guid id)
should be:
public System.Threading.Tasks.Task<SourceDefinitionBaseDto> SourcesGetAsync(System.Guid id)
The nswag config file used is here

I see that this is likely related to #13 / #839 - is this likely to be implemented soon?

Are the correct issues linked?
Can you provide a sample swagger spec?

Ah sorry, I didn't notice those were NJsonSchema issues rather than NSwagStudio issues, I've updated the links above.

Update: This doesn't happen when using NSwag instead of Swashbuckle.AspNetCore to run the swagger endpoint so is potentially a Swashbuckle issue with generating the json schema.

Yes, maybe. But maybe its an issue in nswag/njs which is just handled correct because its one toolchain... the goal is that nswag/njs also works with other tools.


Can't see what emojis others use to react to my messages

Describe the bug, optionally with screenshots
Emoji reactions on other people's posts appear in my chat, but emoji reactions of others to my posts do not render correctly.
Example 1
Example 2
To Reproduce
Steps to reproduce the behavior:

Make post
Have someone else emoji react to it

Expected behavior
I should see emojis rendered as I would for any other post
Additional context:

Ship type = planet (comet/moon/planet/star/galaxy)
Instance type = self-hosted
Chromium-based browser on Pop!_OS

Poking around in JS console, I think it's that the people reacting to me are using MacOS and their emojis have different names than the ones on my end.

@ashelkovnykov You're on Pop!_OS -- do you have fonts-noto-color-emoji installed and configured?

@jamesacklin Yes - though I'm on version 20.10 of Pop!, so maybe there's a newer version of fonts-noto-color-emoji out that I don't have?

Let me try to repro on fresh Debian. More pressingly, can you check the network console to see if you're getting the Emoji Mart webfont?

Didn't see anything in console or network console about EmojiMart. I read online that uninstalling and reinstalling fonts-noto-color-emoji and restarting the system sometimes fixes this. I reinstalled the font, but I don't want to restart my system yet as I have my work environment setup how I like it currently. Will restart tomorrow or Sunday to check.

Did full restart, but it wasn't fixed. However, I believe I've found the problem: I'm using version 20.10 of Pop!_OS, which is limited to an older version of fonts-noto-color-emoji, which does not have Emoji 14 support.
Closing issue as won't fix.


allow selection of feeds in "show posts from my feeds"

Is your feature request related to a problem? Please describe.
not exactly a problem, i think this is a really cool experimental feature but it would be nice to have some control over it
Describe the solution you'd like
just as a rough idea i was thinking maybe it could be a selection on the "edit feeds" page, see screenshot for example
Additional context

Bump this.  My motivation for wanting this feature is that i'd like to have my feeds with low traffic integrated into my homefeed, then leave my high traffic feeds in their own container, so they don't completely overrun my homefeed.


Different python version results in different  result

when i use python3  to execute  setup.py,  result is  []   ,but when i use  python2.7  ,the result is None.
So   component detection will  visit  the  website "https://pypi.org/pypi/None/json",  and this wheel  depend many other  packages。
It  will  include  many  unexpected  packages  of  None.

Can you share a minimal setup.py to reproduce the bug?

setup.py:
import setuptools
setuptools.setup(
name="my-test-package",
version="1.0",
zip_safe=True,
)
environments: python2.7.5 , centos7.9 ,  .net6.0.401
Executed command  ： python -c "import distutils.core; setup=distutils.core.run_setup('setup.py'); print(setup.install_requires)"
result:None


[ServiceBus] add more stress tests

add docs/README to stress tests for SB + EH (follow the README in the perfstress_tests as an example)
split up existing tests into separate files
add more tests (infinite send + receive, other scenarios) - look at Go for example
test chaos/network loss for infinite send and receive.
address Richard's comments from this PR: https://github.com/Azure/azure-sdk-for-python/pull/22982

fix failing stress tests:
FAILED test_stress_queues.py::test_stress_queue_long_renew_session_send_and_receive
2/8/2022, 7:58:44 PM | stress-python-servicebus-stress-test-4--1-cmgb4 | python-sb-stress | FAILED test_stress_queues.py::test_stress_queue_check_for_dropped_messages
ERROR azure_logger_test_stress_queue_check_for_dropped_messages:app_insights_metric.py:102 Error happened when running test_stress_queue_check_for_dropped_messages: ValueError("invalid literal for int() with base 10: 'aaaaaaaaaa'"). Extra info: None
During handling above exception, another exception occurred:
azure.servicebus.exceptions.ServiceBusError: A sessionful message receiver cannot be created on an entity that does not require sessions. Ensure RequiresSession is set to true when creating a Queue or Subscription to enable sessionful behavior. TrackingId:502738cb-83a0-4150-8bc8-9ae38eb9ef93_B2, SystemTracker:t00ae01b19ced4ea2:Queue:testqueue, Timestamp:2022-02-09T03:48:29 TrackingId:3436d5c6d72943e587d3d13418f207d9_G52, SystemTracker:gateway7, Timestamp:2022-02-09T03:48:29 Error condition: ErrorCodes.NotAllowed.
`

result = stress_test.run() |
body = int(str(received_message))
`

align with Go/JS tasks

update resource group name in ARM template as per: https://github.com/Azure/azure-sdk-for-python/pull/24010#discussion_r855676209

done by @l0lawrence


NOT AN ISSUE

Sorry I didnt know how to just say thank you for this.

Thanks!! I appreciate the love.


ModuleNotFoundError: No module named 'flip_fluids_addon'

The following command is sent: "nice -n 20 ../blender/blender --background -y -noaudio --python generate.py --threads 8 -- --input_folder /home/datasda/huliwen/my_videos/videos1114411/20bc1adb/coarse --output_folder /home/datasda/huliwen/my_videos/videos1114411/20bc1adb/coarse --seed 20bc1adb --task populate --task_uniqname populate -g kelp_forest simple.gin -p          LOG_DIR='/home/datasda/huliwen/my_videos/videos1114411/20bc1adb/logs'      execute_tasks.frame_range=[1, 192] execute_tasks.camera_id=[0, 0]"
Overriden parameters: ['', "        LOG_DIR='/home/datasda/huliwen/my_videos/videos1114411/20bc1adb/logs'", '    ', 'execute_tasks.frame_range=[1, 192]', 'execute_tasks.camera_id=[0, 0]']
Traceback (most recent call last):
File "/home/huliwen/infinigen/blender/3.3/scripts/modules/addon_utils.py", line 333, in enable
mod = import(module_name)
ModuleNotFoundError: No module named 'flip_fluids_addon'
Error: Traceback (most recent call last):
File "/home/huliwen/infinigen/blender/3.3/scripts/modules/addon_utils.py", line 333, in enable
mod = import(module_name)
ModuleNotFoundError: No module named 'flip_fluids_addon'

I encountered the similar problem，my program to generate images in one command crashed, reporting the error:[19:53:17.489] [root] [WARNING] | Re-initialized 0 as trusted. Do not run infinigen on untrusted blend files.
[19:53:17.489] [root] [INFO] | Processing frames 48 through 48 inclusive
[19:53:17.530] [root] [WARNING] | Could not load addon "flip_fluids_addon"

Hello, could you each please share the full command you ran and the full logs, and your platform info?

It's working for me. Thanks!

try "bash install.sh flip_fluids"

It's working for me. Thanks!

To confirm, is this flip_fluids_error a fatal crash for you all? If so, what is your OS/platform? Installing flip_fluids is intended to be optional, and the ModuleError from failing to load flip fluids is explicitly try-catched in the code.

I have tried "try "bash install.sh flip_fluids"with the same errors on Ubuntuntu. However, the Infinigen works well with "ModuleNotFoundError: No module named 'flip_fluids_addon.'" I can get right imgs and gts. I will show you more  details soon, according to when my severs back to work.


[semantic-dom-diff] expect(el).dom.to.equal() types

When working with typescript you may notice that the types are not correct for
expect(el).dom.to.equal('
Hey
', {
  ignoreTags: ['my-custom-element']
});

e.g. the 2nd parameter is expected to be a string. Unfortunately, we currently can not change this.
For now you will need to ignore types if you want to provide extra options.
// @ts-ignore
expect(el).dom.to.equal('
Hey
', {
  ignoreTags: ['my-custom-element']
});

We probably want to introduce a new syntax in a breaking release?
expect(el).domToEqual('
Hey
', {
  ignoreTags: ['my-custom-element']
});

Generally, chai-dom-equals should ship with .d.ts

Is this still relevant @daKmoR


Add the missing negative sign in the formula.

In the following code, the formula in the comments is not consistent with the code as a negative sign has been omitted in the comments.
https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/df566e0d48e7025bba50d82269f687839d182b35/labml_nn/transformers/rope/__init__.py#L140-L141

Thank you 🙏


Option to disable the cache \ do not write to disk

I would like the option to disable the cache so that nothing is written to the disk, at least if the export mode is timeline.
I use this program to skip silence in mpv. I get chunks through the timeline export.
Now the program creates 2 temporary folders in temp and one cache.npz file.
Of course, the cache size is small, but it seems like the program can work without it, at least I didn't see the importance of the cache in the code.
For one video I run auto-editor once and will not use the cache.

Good idea. I've added the --no-cache option in master.


Update de.json

Corrected some German translations after viewing them in app V4.3.3

Sorry about the slowness on this one - there is now a conflict - which of the carmirror translations should go in?

I hope that I have proceeded correctly. I used the two previous translations and translated the new sentences at the same time.

Thanks, looks right!


Account API not properly implemented yet

Currently the account-summary method at the binder is not implemented yet. This means that

The accinfo method of the client RPC won't work
The VerifyAccount state in the state machine is a no-op

However, counting of remaining balances of users is working properly at the exits by direct access to the central PostgreSQL database. It's just that without account-summary implemented, the clients cannot see their own balances, and the connection will just mysteriously fail to work when the balance is empty.

Things that need to be done:

Connect the binder to PostgreSQL and implement account-summary
Set up an *http.Client in the client that conveniently does HTTP over the established tunnel
Implement accinfo so that it calls the tunneled client to do operations with https://binder.geph.io without going through the CDN

Done in e08c39e13778926556990af6cfcff76eb22f0cd5


chore(general): replace conversion functions with ToAlloy/ToEthers conv trait

Removes usage of all conversion functions, and uses the ToAlloy and ToEthers trait.

CI should be fixed with https://github.com/foundry-rs/foundry/pull/5858 and a subsequent rebase


feat(preset-mini): add user-valid and user-invalid pseudo selectors

I added user-valid and user-invalid pseudo selectors in preset-mini pseudo.ts. No idea if it's supposed to be this easy. Is anything else required?
Tested in playground using
<input required type="email" class="peer w-64 border border-solid user-valid:border-green-600 user-invalid:border-red-600"/>

in Chrome canary.
Fixes https://github.com/unocss/unocss/issues/3180

No idea how to fix the last test error... 👼
 FAIL  |unit| test/preset-attributify.test.ts > attributify > autocomplete extractor > without prefix
AssertionError: expected null not to be null
 ❯ test/preset-attributify.test.ts:86:23
     84|       })
     85|
     86|       expect(res).not.toBeNull()
       |                       ^
     87|
     88|       expect(res!.extracted).toMatchInlineSnapshot('"bg-blue-400"')


Where do I find /root/plot_manager/kill_nc.sh ?

Where do I find the contents of the script /root/plot_manager/kill_nc.sh ?

/root/plot_manager/receive_plot.sh is created on the fly when you run ./drive_manager.py, take a look at lines 500 - 507 of drive_manager.py.
I will add kill_nc.sh to the chianas directory! Thanks for pointing that out!

Information updated.


Make binary release

It would be great if you could create a binary release of this project and publish to maven central

OK. I open a ticket on https://issues.sonatype.org/browse/OSSRH-24621

please use
<dependency>
    <groupId>com.github.levyfan</groupId>
    <artifactId>guava-cache-redis</artifactId>
    <version>0.0.2</version>
</dependency>


Add ability to set attachment after message edition

An entry "(a)dd attachment" in the post edit choice could do the trick.

Fixed by #334.


Can´t find Iframe

So I am trying to find an Iframe with splinter and I can find the first one with,
with browser.get_iframe('iframemodal') as iframe:
iframe.do_stuff()
and instead of iframemodal I use index which is 0 and this works fine but when I press the box in that Iframe it opens another Iframe without a name or id and everytime the index order of that newly opened iframe changes so I cant use index again it has a title but I think get.iframe doesnt work with css is there a way to find the second iframe ?

Hi,
Instead of using name or id try using xpath, something like this:
 with browser.get_iframe(browser.find_by_xpath('//*[@class="ABC"]').first) as iframe:


QR code scan & face recognition not working for sm-p901 tablet

I have tried the app on my android sm-p901 tablet with API lollipop but only the barcode san was working .The same code worked on the rest of my devices .So I don't know whats the problem .I tried the actual functionalities separately on other apps and they where working .

Hi @OmarBoshra,
could you provide the console error log?
Best.


Pool transaction tab in profile shows incorrect infos

Describe the bug
In the pool transaction tab in the profile section only title (but the link it points to is broken) and time seem to be correct, data set field is empty, network is always unknown.
Screenshots

Should be fixed with #1139 although I get now a mix of assets with info and without. But might be because of older test assets interacted with my account but something to look out for when verifying if bug is gone
on https://market-git-feature-issue1042-restore-pool-section-oceanprotocol.vercel.app/profile


Allow generate install to write files to a directory

This was one of the benefits of moving the kyaml pipeline into the generator. We can reorganize the files anyway we want by applying annotations, the default layout is [{namespace}/]{kind}_{name}.yaml.


v<IP_ADDRESS> wrongly reporting some binary_sensors as active

Version of the custom_component

v<IP_ADDRESS>
Installation method (hacs / manual)
hacs
Installation method of hass (venv, docker, hassio,...)
docker
Configuration
NA

Describe the bug
Reported before in #82.
Some binary_sensors which are not reported by the VW API are shown in HA as active.

Looking at the binary_sensors added by the integration and the related extracted information with weconnect-cli:
[climatisation] Elements: 3 items
        [climatisationStatus] (last captured 2022-10-26T13:46:55+00:00)
                State: off
                Remaining Climatization Time: 0 min
        [climatisationSettings] (last captured 2022-10-26T13:46:55+00:00)
                Target Temperature in °C: 22.0 °C
                Target Temperature in °F: 72.0 °F
                Temperature unit in car: celsius
        [windowHeatingStatus] (last captured 2022-10-26T13:46:55+00:00)
                Windows: 2 items
                        front: off
                        rear: off
[charging] Elements: 5 items
        [batteryStatus] (last captured 2022-10-26T13:46:57+00:00)
                Current SoC: 65%
                Range: 267km
        [chargingStatus] (last captured 2022-10-26T13:46:57+00:00)
                State: notReadyForCharging
                Mode: manual
                Charge Power: 0.0 kW
                Charge Rate: 0.0 km/h
                Charge Type: invalid
                Charging Settings: default
        [chargingSettings] (last captured 2022-10-26T13:46:56+00:00)
                Maximum Charge Current AC: maximum
                Auto Unlock When Charged: off
                Auto Unlock When Charged AC: off
                Target SoC: 90 %
        [chargeMode]
                Preferred charge mode: manual
                Available charge modes: [manual]
        [plugStatus] (last captured 2022-10-26T14:05:23+00:00)
                Plug: disconnected, unlocked, External Power: unavailable, Led color: none
[readiness] Elements: 1 items
        [readinessStatus]
                Connection State:
                        Is online: True
                        Battery power level: comfort
                        Daily Power Budget Available: True
                Connection Warning:
                        Insufficient Battery Level Warning: False
                        Daily Power Budget Warning: False

it seems clear that not all the information reported by HA via these binary sensors is correct according to what is reported by the VW api.
Here are also some app screenshots showing that some of these binary sensors should show as inactive:

Can this please be fixed?
Debug log

I believe this issue is cause by the following by
https://github.com/mitch-dc/volkswagen_we_connect_id/blob/6e3e004720bc28c459cad0731bd0f9845ea46fc7/custom_components/volkswagen_we_connect_id/binary_sensor.py#L168
If you change this to
      return None

the issue goes away,and the erroneus sensors show as unavailable.
My reading of the code is, it tries to get the sensor value from the API.   If this is not available, it uses a previous (or default) value stored in home assistant.

My reading of the code is, it tries to get the sensor value from the API. If this is not available, it uses a previous (or default) value stored in home assistant.

The thing is that some of these sensors seem to only be reported by API when 'on', like the 'Car is Active', otherwise it should be 'off' (even if the API does not report it).

Ah, OK.  Then I suggest return “off” rather than None

Sadly, this still does not work long term.
It works on a reboot of HA, but once  the sensor is turned to On, nothing is ever called to turn it off.
I think the issue is, as the sensor is missing for the API data, the call to update the HA value (my ‘false’ fix) does not get called.
Somehow we need to trigger an update of all sensors, so they get reset if missing.

Yes, sadly I had also noticed that today.
Not sure if this logic should be implemented here or in the upstream library.

Having looked into this more, I think upstream maybe the place to fix it.
If we start HA when the EV is inactive,  the WeConnect API does not have an data for is, the the WeConnect library does not return data.   The code here creates a default sensor value (on or off, depending if you use my 'fix' above or the original code)
The EV now becomes active.
The upstream library adds isAcitve to its Dict storage entity, and sets isActive to True.
The code here sets the sensor to on.
The EV now becomes inactive.
The WeConnect API does not return any data (indicating inactive).
The WeConnect Python library no not uddate the Dict entity - so the dict has an incorrect value.
This is reported to HA, so the sensor is now incorrect.
As you say, not sure how we can fix here, so will report upstream, and see where we get to.

Upstream report: https://github.com/tillsteinbach/WeConnect-python/issues/85

I would recommend not to rely any logic on the isActive flag. It is not reliably coming from the server.

I would recommend not to rely any logic on the isActive flag. It is not reliably coming from the server.

That’s a bit odd, because running v0.0.13 of this custom integration which uses v0.45.1 of your upstream library seems quite accurate (not sure about reliability).

That is a good hint. Can we find out since when it became unreliable?

That is a good hint. Can we find out since when it became unreliable?

Somewhere between v0.45.1 and v0.48.1 of your library, I have no smaller interval

@pplucky having check upstream (https://github.com/tillsteinbach/WeConnect-python/issues/85#issuecomment-1328238676) it appears a check needs to be made in the code here.
I've attempted a fix in PR https://github.com/mitch-dc/volkswagen_we_connect_id/pull/107

Note, while the PR “improves” the situation, I believe there is a secondary logic error in the upstream library, which means the sensor will only come on once, before remaining permanently off until a HA restart.

Note, while the PR “improves” the situation, I believe there is a secondary logic error in the upstream library, which means the sensor will only come on once, before remaining permanently off until a HA restart.
I’ve re-opened the upstream issue.

I second this behaviour, which I also observed in my test system, where your latest PRs were installed.

I'm testing 0.50.1 which should fix it.

Please try the new beta, i've bumped weconnect and moved some binary sensors to normal sensors
https://github.com/mitch-dc/volkswagen_we_connect_id/releases/tag/v<IP_ADDRESS>

Running <IP_ADDRESS> now.  Looking good so far.  Need two car journeys to be sure the binary sensors are now OK…

Running <IP_ADDRESS> now. Looking good so far. Need two car journeys to be sure the binary sensors are now OK…

Same here, seems globally consistent, although I haven't driven the car too much lately.


[training] Support kubeflow recurring run and model export to consul

Support wrapping recurring run schedule time parameters and training components' parameters
Help fill experiment specific parameters for Estimator
Decouple experiment, scheduling and publish from model runners

Close in favor of new operator api server design


thoughts on splitting up the code

🔦 question?
i would love to contribute by adding some more values to what this crate parses, however i realized that in order to scale we'll probably want to split this code up into different files. figured i'd file an issue and ask what ya'll thought would be best before i imposed what i thought would be good. also i'm not sure what exactly i think the best answer is 😅
maybe per enum? or categorize the enums somehow? anyways. interested to hear what ya'll think.

I perfectly agree in splitting the code and making it extensible.
I would also like this to be a single crate that entirely takes care of npmrc
P.S. I also believe that apart from wasm-pack anyone wanna fiddle npmrc this should be capable of handling it. That is what I had envisioned when I was suggesting this.

@sendilkumarn yeah i wasn't saying that this should be separate crates, just trying to see how to split this crate into multiple files to keep it a bit more maintainable

@ashleygwilliams perfect 😄

@ashleygwilliams that'd be perfect! - yeah, I just put it all into the same file because I wasn't sure how to create (internal) modules; but all in favor of splitting it into separate files!


WIP: Improvement to Input behaviour on opening/closing composeMenu

Now composebox functions similar to FB Messenger's input box:

On tapping the menu expand button, the height of input box is immediately reduced to a fixed value.
On closing the menu, the height remains fixed until the animation is complete and then is replaced by height: 'auto'

This also serves to de-emphasize the input when the menu is open.
(Accidentally captured some background music in the screen recordings, sorry for that)
Before: https://chat.zulip.org/user_uploads/2/f9/_Deoxhl-b9RC1yemRCL9jFdE/composebox_before.mp4
After: https://chat.zulip.org/user_uploads/2/3b/rdB45oSZsbEO8VFmP93wfSLV/compose_heightChange_after.mp4
This PR attempts to fix #2572

Now that we've addressed #2572 with your simpler fix #2598, I think this PR is no longer needed; so closing.
It'd still be nice to shorten the compose box when the menu opens, to de-emphasize it; but that's a much subtler issue than the jerkiness was, so a lower priority, and this change will be complex to get right. Thanks for all your different approaches to the issue!


Error when project loads and no Tools menu item available (windows 10)

On windows when I load the sample Unity Gaussian program I get an error on load and if I proceed with the load there is no "Tools" bar item so no way to load a gaussian.

I get the same issue with both the "GaussianExample" (Unity 2022.3.7f1) and "GaussianExample-HDRP" (Unity 2022.3.8f1) projects. I have DX12 installed.
Could easily be something obvious since I'm not a Unity dev, but the error has no information attached to it so I'm not sure how to debug this better. any suggestions appreciated.

The error message seems to be very... empty? I've no idea why would that happen. Can you look at whether anything is printed in the editor log file? (in this Console window where you see the empty error, press the three-dots icon in top right, there should be an item to open editor log, attach the log here)

yeah, quite uninformative error message!
Editor.log attached
Editor.log

Hmm so Unity fails to compile all the C# code for some strange reason. The editor log is full of failed commands like:
##### CommandLine
"D:\Program Files\Unity 2022.3.7f1\Editor\Data\Tools\netcorerun\netcorerun.exe" "D:/Program Files/Unity 2022.3.7f1/Editor/Data/Tools/ScriptUpdater/ApiUpdater.MovedFromExtractor.exe" "Library\Bee\artifacts\mvdfrm\UnityEngine.TLSModule.dll_52B7D2660BF0467B.mvfrm" "D:\Program Files\Unity 2022.3.7f1\Editor\Data\Managed\UnityEngine\UnityEngine.TLSModule.dll"
##### ExitCode
1

but I've no idea why or how that would happen. My only guess, maybe this particular Unity version (2022.3.7 and .8) has a bug when it fails to do anything if the project drive (F in your case) is different than the Unity installation drive (D)? Maybe updating to more recent Unity version (2022.3.x whichever is the latest) would improve things?
I'm just guessing randomly. So far it looks like some general Unity issue that for some reason happens on your machine, and not something specific to this guassian splatting project.

ok, thanks for looking into this. I've made space on my C drive and will re-install everything on the same drive to see if that fixes the issue and then I'll report back.

You were right it was a weird unity error that was preventing any projects from compiling and providing no informative error messages.
Fixed by clearing the registgry value of "HKEY_LOCAL_MACHINE\Software\Microsoft\Command Processor\AutoRun" because who the @#$#@% know why.
(In case this is of use to anyone, solution found here: https://discussions.unity.com/t/the-project-you-are-opening-contains-compilation-errors-when-creating-new-project/901713/5)


How to use AbstractElbowLineConnectionWithMarkersFigure

Can you give an example about how to use AbstractElbowLineConnectionWithMarkersFigure?
AbstractElbowLineConnectionWithMarkersFigure contains Polyline, startMarker and endMarker, so I hope the example can draw a polyline with many points which connects two figures, Thanks a lot.

Can you explain why can AbstractElbowLineConnectionWithMarkersFigure move along with it's START_TARGET or END_TARGET?

I have now committed an example figure that implements AbstractElbowLineConnectionWithMarkersFigure into the main branch.
The name of the class is ElbowLineConectionWithMarkesFigure:
https://github.com/wrandelshofer/jhotdraw8/blob/72194870f7e37b9b086cc6abe2f42f01b44363e7/org.jhotdraw8.draw/src/main/java/org.jhotdraw8.draw/org/jhotdraw8/draw/figure/ElbowConnectionWithMarkersFigure.java
You can run the Grapher Demo application to try it out. The figure is available in the toolbar.

Can you explain why can AbstractElbowLineConnectionWithMarkersFigure move along with it's START_TARGET or END_TARGET?

You can look at the first couple of lines in the layout(RenderContext ctx) method of AbstractElbowLineConnectionWithMarkersFigure.
The algorithm is as follows:

Get the stored start and end points by calling getNonNull(START) and getNonNull(END).
If we have a START_TARGET and a START_CONNECTOR, use them to compute an initial start point by calling startConnector.getPointAndDerivativeInWorld().  The initial start position is somewhere inside the shape of the figure.
Do the same for END_TARGET and END_CONNECTOR;
Compute the intersection point at the border of the start figure by calling startConnector.chopStart() with the start figure, and the intial start and end points.
Compute the intersection point at the border of the end figure by calling endConnector.chopStart() with the end figure and the intial start and end points (given in reverse order).
Store the points that we have found by calling set(START, ...) and set(END, ...)

The code does some additional stuff with the derivative of the intersection points. The derivatives are used for layouting the elbow line.
As you can see, the computation of the location of the start/end points is performed by calling methods on Connector objects.
The Connector objects are created by the target figures. See ConnectableFigure.findConnector() methods.
https://github.com/wrandelshofer/jhotdraw8/blob/ad5bf67c859e04b8cacc14278b03e85caa50ae58/org.jhotdraw8.draw/src/main/java/org.jhotdraw8.draw/org/jhotdraw8/draw/figure/AbstractElbowLineConnectionWithMarkersFigure.java#L238-L276

Thanks a lot.
Why is the layout(RenderContext ctx) method of AbstractElbowLineConnectionWithMarkersFigure? Is the result of the doAddedToDrawing(final Drawing drawing) method of AbstractLineConnectionFigure adding AbstractLineConnectionFigure to layoutObservers of START_TARGET or END_TARGET figure?

Yes. A Figure, that observes the layout of another Figure, has to add itself as a layoutObserver on the other figure.
The layout algorithm in SimpleDrawingModel uses layoutObservers to determine in which sequence the figures must be laid out. SimpleDrawingModel builds a graph, and then sorts it topologically.
https://github.com/wrandelshofer/jhotdraw8/blob/ad5bf67c859e04b8cacc14278b03e85caa50ae58/org.jhotdraw8.draw/src/main/java/org.jhotdraw8.draw/org/jhotdraw8/draw/model/SimpleDrawingModel.java#L417-L423


Clarify error message: server -> SMTP server

In the last of the three messages is "SMTP server" used. Let's do the same for the other two messages.
I only realized that not the DB connection failed but SMTP when I looked at the source.

Thank you for the patch!   I am not sure how this escaped me, as I am usually quite pedantic when it comes to logging clear messages. :)


where is the user document

How do I use it ?
I need to test it in detail .  where is the user documentation ?

Hi @baofengzhouyu!
It's still a work in progress, and not really usable at this time. I will work on it soon.

ok, thanks for your reply.  Looking forward to the official release.


Fixed Documentation autodoc rendering

@mitkotak please resolve the conflicts and get the PR passing flake8

Sorry for the confusion. I was running flake8 without --ignore which was triggering some differences. Let me know how it looks now @alexandermichels


Modern Sharepoint sites observed to be working too slow in IE browsers

Issues been reported heavily in IE browsers than in Chrome/Firefox. As IE is used by the widely, Issue fix of the same would be much helpful.

Hey @InfosysAdmin, this is not a problem related to page transformation but rather related to modern SharePoint. IE is indeed noticeably slower compared with modern browsers (the new Edge browser from Microsoft of Chrome). I would recommend switching to the new Edge browser, this browser does have a built-in IE11 tab in case a site needs IE11 while running all the other sites at optimal performance.
If you feel that Microsoft should fix the IE11 performance then I would recommend to open a support case.

Closing this issue as it's not related to page transformation.


npm/bump

Description
Bump package dependencies.
Test Coverage

[x] unit

References

Closes #121 .

@RyanNieuwoudt npm outdated

@RyanNieuwoudt npm outdated

@davidhorak it appears that only eslint-config-standard is outdated. Upgrading it yields this error:
npm run build:js

><EMAIL_ADDRESS>build:js
> rollup -c

./src/index.ts → ./lib/wegood.esm.js, ./lib/wegood.js...
[!] (plugin eslint) Error: .eslintrc.js » eslint-config-standard:
        Environment key "es2021" is unknown

According to the changelog, support for es2021 was introduce in version 15. Do we:

Hunt for a solution?
Leave this package on version 14.x for now?
Replace standard as we have done elsewhere?

@RyanNieuwoudt npm outdated

@davidhorak it appears that only eslint-config-standard is outdated. Upgrading it yields this error:
npm run build:js

><EMAIL_ADDRESS>build:js
> rollup -c

./src/index.ts → ./lib/wegood.esm.js, ./lib/wegood.js...
[!] (plugin eslint) Error: .eslintrc.js » eslint-config-standard:
        Environment key "es2021" is unknown

According to the changelog, support for es2021 was introduce in version 15. Do we:

Hunt for a solution?
Leave this package on version 14.x for now?
Replace standard as we have done elsewhere?

Yeah ... actually I know about hat one, was not able find any quick suggestion how to fix this. So, eventually 1. for now 2..


timeout parameter doesn't work in DisqueJobCommands.getjobs method

biz.paluch.spinach.api.sync.DisqueJobCommands.getjobs(long timeout, TimeUnit timeUnit, long count, String... queues)
->
biz.paluch.spinach.impl.FutureSyncInvocationHandler.handleInvocation(Object proxy, Method method, Object[] args) {
LettuceFutures.await(command,connection.getTimeout(), connection.getTimeoutUnit());

Hi @LMig2k ,
Please raise the issue here https://github.com/mp911de/spinach/issues . Seems like you issues is specific to client library

Thanks to remind.


mqtt: add additional tests for reason codes - v2

Previous PR: #2102
Redmine ticket: https://redmine.openinfosecfoundation.org/issues/7323
Changes to previous PR:

Set minimum Suricata version for SUBACK test to 8.

One thing I'm not sure about is the forum linked in the commit message, but I'll let Victor or someone else be the judge for that.

The post is also linked in the Redmine ticket, so I could also remove it -- not sure why that would matter though :)

Merged in #2118, thanks!


Changed Files

Making it compatible with Pipeline Jobs. After merge you can use this plugin in pipelines jobs also.

Hi Gregory E. Moltchadski,
I have made the changes this plugin works for the pipeline except choice parameter, this plugin will work well with pipeline jobs as well.
Request you to merge it with main branch. So that people can leverage it.
Cheers,
Ankit


Nugets should not reference facade assemblies as framework dependencies

Referencing System.IO and other facade assemblies in the nuget metadata fails on .NET 4.5.1 installations. Remove System.IO from the metadata:

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd">
<metadata>
<id>Microsoft.AspNet.DataProtection</id>
<version>1.0.0-rc1-final</version>
…
<dependencies>
…
</dependencies>
<frameworkAssemblies>
<frameworkAssembly assemblyName="System.IO" targetFramework=".NETFramework4.5.1" />
<frameworkAssembly assemblyName="System.Security" targetFramework=".NETFramework4.5.1" />
<frameworkAssembly assemblyName="System.Xml" targetFramework=".NETFramework4.5.1" />
<frameworkAssembly assemblyName="System.Xml.Linq" targetFramework=".NETFramework4.5.1" />
<frameworkAssembly assemblyName="mscorlib" targetFramework=".NETFramework4.5.1" />
<frameworkAssembly assemblyName="System" targetFramework=".NETFramework4.5.1" />
<frameworkAssembly assemblyName="System.Core" targetFramework=".NETFramework4.5.1" />
<frameworkAssembly assemblyName="Microsoft.CSharp" targetFramework=".NETFramework4.5.1" />
</frameworkAssemblies>
</metadata>
</package>

@Tratcher  can you take care of this once the general renaming stuff has settled down?

@davidfowl said he has some ideas to make this better in @pakrym's PR above (which was closed without merging.)  Moving to 1.0.0 milestone.
@davidfowl please elaborate what you're thinking and we'll assign it to someone.

This is already fixed in the latest packages

:cool:

@davidfowl Is there a target release date for rc2?  I am unable to install package Microsoft.AspNet.WebHooks.Custom.SqlStorage 1.2.0-rc1b which depends on this DataProtection package because of this issue.  Or is there a work around?

Looks like I found my answer: https://blogs.msdn.microsoft.com/dotnet/2016/05/06/net-core-rc2-improvements-schedule-and-roadmap/


Correctly encode non-legacy mempool transactions

Motivation
If a non-legacy transaction was fetched from the mempool and then RLP-encoded for inclusion in e.g. a flashbots bundle, then the bundle would fail because the encoding only worked for legacy transactions. See https://github.com/onbjerg/ethers-flashbots/issues/11#issuecomment-907672337
Solution
RLP encode Transaction response types based on their transaction_type. I'm not super confident about this piece of code since it is a lot of duplicate logic from other files, but I'm not super well known in the codebase yet so I'm not sure what the best way is to improve it. Please let me know if you have any suggestions!

Not working yet. Please take code from here:
https://github.com/onbjerg/ethers-flashbots/issues/11#issuecomment-908130611
@onbjerg might do a PR soon.

Not working yet. Please take code from here:
onbjerg/ethers-flashbots#11 (comment)
@onbjerg might do a PR soon.

Yeah, I have some tests locally and some fixes too, but I want to add one more before pushing. Thanks for the PR on my fork, I'll take a look today :)

Just asked in the Flashbots Discord, it seems to be the return value of the called function, so I'll have to adjust that.

Should be good to go @gakonst


Adding js/lib/*.js and removing from gitignore

I've (manually) added the cookiecutter stuff for the lib subdirectory and updated gitignore to ... not ignore them.

Ok I've successfully used and installed this version of the widget with HelloWorld. 👍


Add ability to specify envs to strip selectors.

For example, removing the test selectors in a staging or other pre-production environment.

I also moved the registration logic into the setupPreprocessorRegistry hook (docs), as it seemed to make more sense there than in the included block.

Great! Only 2 smaller things on naming and docs.

changed to environments.

Wohoo, thanks @blimmer :tada:


8.11.0: h3 connections fail with "SSL routines::no cipher match" / "Out of memory"

All my HTTP/3 connections error out with "SSL routines::no cipher match" / "Out of memory" on the 8.11.0 release.
Logs:
$ curl-8.11.0 --version
curl 8.11.0 (aarch64-pc-linux-gnu) libcurl/8.11.0 OpenSSL/3.4.0 zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 c-ares/1.34.2 libidn2/2.3.7 libpsl/0.21.5 libssh2/1.11.1 nghttp2/1.64.0 nghttp3/1.6.0
Release-Date: 2024-11-06
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Largefile libz NTLM PSL SSL threadsafe TLS-SRP TrackMemory UnixSockets zstd

$ curl-8.11.0 -v --http3-only https://quic.tech:8443/ -o /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Host quic.tech:8443 was resolved.
* IPv6: 2001:19f0:5:c21:5400:1ff:fe33:3b96
* IPv4: <IP_ADDRESS>
*   Trying [2001:19f0:5:c21:5400:1ff:fe33:3b96]:8443...
* SSL: could not create a context: error:0A0000B9:SSL routines::no cipher match
* QUIC connect to 2001:19f0:5:c21:5400:1ff:fe33:3b96 port 8443 failed: Out of memory
*   Trying <IP_ADDRESS>:8443...
* SSL: could not create a context: error:0A0000B9:SSL routines::no cipher match
* QUIC connect to <IP_ADDRESS> port 8443 failed: Out of memory
* Failed to connect to quic.tech port 8443 after 1 ms: Out of memory
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* closing connection #0
curl: (27) SSL: could not create a context: error:0A0000B9:SSL routines::no cipher match

Previous release (version below) works.
$ curl --version
curl 8.10.1 (aarch64-pc-linux-gnu) libcurl/8.10.1 OpenSSL/3.3.2 zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 c-ares/1.33.1 libidn2/2.3.7 libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.63.0 nghttp3/1.5.0
Release-Date: 2024-09-18
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Largefile libz NTLM PSL SSL threadsafe TLS-SRP TrackMemory UnixSockets zstd

I tested all arm architectures and all have problems, not testing riscv/mips etc for now. x86 architectures are fine.

I deleted the release and see what causes it tomorrow.

Sorry for the mistake in my previous description, I didn't reproduce your issue.
Your issue seems to be with glibc, you can try it with the musl version.
I have discovered that all ARM versions of static-curl (including those built with glibc, musl, for arm64, armv7, and armv5 architectures) experience deadlocks on the rk3566 platform. I'm currently investigating this issue.

Thanks for the update. I retested the new release (glibc) version gives the same error.
The musl version seems to work - but I seem to get drastically fewer h3 connections using this compared with 8.10.1-glibc version (I download ~500 feeds per hour, and it would average ~300 before, now ~5).
Not sure if that is a curl regression or me trying to use musl on a glibc (Ubuntu 22.04) system?

Yes, you should use musl on a glibc system if glibc is not compatible.
There have been discussions #93 and #62 about glibc compatibility.


Save window size and position when using window gestures and desktop edges

Normally when you close a program, next time you open it, it will be shown in the same position and with same size as last time.
That doesn't work though when you use the desktop edge gestures for some reason (such as resizing the bottom to the lower edge and the top automatically goes all way up)
This is the only reason I keep this feature off. Would be great to see that being implemented. Might actually be a bug on Windows instead of a non-feature, but I believe that it could be developed faster here than waiting for a fix in the main code

@bergamin
I think this should be submitted to the Windows team via the Feedback Hub application.
Interfering with Windows when it comes to Windows Snap actions may not be a good idea since it can easily break the internal logic of Windows Snap: for example when you snap a window resizing it to the top edge of the screen, you also have the ability to restore it to its original size. If PowerToys tries to save the new windows size and position so that it will be used when the app reopens, that would break the existing functionality of Windows Snap.

@enricogior, I probably have already done that, but alright. I'll do it again...


Fixes Issue #1108: Updated *.md to *.mystnb

Description
Updated the makefile  to handle the switch to MyST format.
This PR fixes issue #1108
Changes
Changed *.md to *.mystnb in line 57 of poliastro/docs/Makefile file. This should allow the makefile to handle the switch to MyST format properly.

Thanks for addressing this @OBITORASU, merging this! :rocket:


getting-started.md: Remove note about Windows

README.md states that Windows is indeed supported, so remove this text from the tutorial

This doc branch stagnated and the Mio API has evolved significantly. Master has gotten a good amount of current documentation, so I am closing in favor of the docs on master.


Added queue size configuration for monitored item

The original values 0 is used as the default value if no queue size is configured.
Empty lines fixed automatically.
Readme documentation is updated.

We will provide a different way of configuring this in a future release. Hence we do not want to add a new API surface at this point.


Update README.md

a little typo

Thanks, but "ajax'd" was intentional. It's meant to be past participle like "fetched", but with "ajax" turned into a verb.


soc: arm: nxp_s32: s32k1: unselect CPU_HAS_xCACHE

Following changes in #64978, align CPU_HAS_xCACHE symbols with the CMSIS feature definitions in the device headers so that both have the same value.
Fixes #66147

Merging as hotfix.


support for akka 2.6.0

I have tried to use akka-http-json with akka 2.6.0 but when I try to assemble it (sbt-assembly) it says, it can't because of this version conflicts, Thanks!
[warn] Found version conflict(s) in library dependencies; some are suspected to be binary incompatible:
[warn]  * com.typesafe.akka:akka-stream_2.12:2.6.0 is selected over 2.5.25
[warn]      +- com.typesafe.akka:akka-persistence_2.12:2.6.0 ()   (depends on 2.6.0)
[warn]      +- com.typesafe.akka:akka-stream-typed_2.12:2.6.0 ()  (depends on 2.6.0)
[warn]      +- com.typesafe.akka:akka-remote_2.12:2.6.0 ()        (depends on 2.6.0)
[warn]      +- de.heikoseeberger:akka-http-circe_2.12:1.29.1      (depends on 2.5.25)

Which version of akka-http-json did you use?

the latest release 1.29.1 - de.heikoseeberger:akka-http-circe_2.12:1.29.1 (depends on 2.5.25)

Released 1.30.0

thanks @hseeberger!


Add toggle to disable humam readable for Openstack

Add ops-files to desactivate humam readabl for Openstack.
This is actually needed when using openstack as  cloud-provider in Kubernetes (ala Kubo).

@cppforlife

@shinji62 wouldn't it be better to ask the OpenStack CPI team to change then separator between the name and the id into something like "-", which would keep the human readable names and also be allowed in context of DNS?
(facing the same problem as you currently).

@jhiemer This is not only kubo related. So if people would like to desactivate this behavior. this PR will help

@shinji62 I am aware of that. But then it will be a much bigger problem with other deployments also relying on the DNS in future, or?

Agree, but again this is just to desactivate it.


How to secure instances

If users can get all instance IDs, they can use the API to manage instances that don't belong to them.
Does the framework provide a solution to this?
Or did I miss something obvious? (Most likely this)
Cheers
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a29097a2-e7ac-f8ae-c8b9-65dd9707d089
Version Independent ID: b4d2634f-9551-5851-a3f3-a134ac1cf722
Content: HTTP features in Durable Functions - Azure Functions
Content Source: articles/azure-functions/durable/durable-functions-http-features.md
Service: azure-functions
GitHub Login: @cgillum
Microsoft Alias: azfuncdf

@JordanSimba I believe you are referring to the Built-in HTTP APIs which are protected by a system key and are not really meant to be used by end consumers directly.
If you building a service where end users can see their orchestrations and should be able to manage them, then you would have to build a layer around these APIs as required with relevant access restrictions accordingly.

Thanks for your response!
I've since found an example of this and will implement some logic to remove the systemKey and present the status in a nice way!


Lesson 8 Vector Algebra

we need to create markdown for Vector Algebra
chapter-8.pdf

upto 8.4 completed


Write to Icechunk as intermediate store?

Maybe useful? Could simply roll back to state before failed stage. Also then it's Icechunk 's problem to worry about atomic writes... Idea from:
A reason to not run backup tasks is if the filesystem does not support atomic writes. Cloud object stores generally are atomic (see https://cubed-dev.github.io/cubed/user-guide/reliability.html#stragglers), but local filesystems are not.
Discussing with @applio, we thought this PR should be changed so that the default was based on the store rather than the executor. Backup tasks would be off by default, except if the store was a well-known cloud store like S3 or GCS.
Originally posted by @tomwhite in https://github.com/cubed-dev/cubed/pull/600#issuecomment-2442268954

The interesting case is HPC, where there are multiple nodes (and hence the possibility of stragglers), but an intermediate store that uses a shared filesystem that does not support atomic writes. Icechunk might be useful to provide atomicity - but perhaps there are other ways too?
Also, I'd like to move to general blob stores (not just Zarr stores) for the intermediate store, so we have more control over the chunk access pattern (to control memory usage). Work like #464 will enable this.

Using icechunk to store intermediate data might be helpful for resuming computations - you could have one icechunk store for intermediate data and the resuming code simply checks out the correct commit then restarts from that.

Using icechunk to store intermediate data might be helpful for resuming computations - each completed stage of the plan would write a new commit to one icechunk store that holds all intermediate data and the resuming code simply checks out the correct commit then restarts from that.

This would work nicely at the level of whole intermediate arrays - but not for resuming from a partially written intermediate array - but that may be OK (and is actually the current level of support for resume).
It also might be worth thinking about how this could work for incremental updates of regions of arrays.


Colours of the images in the home page header/banner are not updated to the current palette

Don't know who is the responsible for this, but those greens are outdated.
The current look and feel doesn't match with this banner.

Designers from Worldline did it in the first place. I do not know if someone want to participate, if not I will ask them to adapt it.
In the other hand, we need the texts on images translated in order to ask the designers build an image for each language.

Could you explain what do you mean exactly when you say "current palette"?

old greens vs new blues.

Images translated already have the new "palette"

Images are now following the "new colors"


feat: add omitempty in Manifest + MarshalYAML for mapSlice

We have internal tooling that uses the Manifest struct to generate substreams.yaml files.

Without the omitempty, those files have a lot of noise, so adding it.
The mapSlice type didn't had a MarshalYAML defined resulting in it getting marshaled as a slice instead of a map. When reading back a substreams.yaml generated this way into the Manifest struct, the yaml decoder fails to decode it. So, added the MarshalYAML to fix the same.

Seems like those tests are failing on the HEAD commit in the base branch itself. Please let me know if I need to do anything.

No need to wait, I don't think any test failed because of your change.


What is the criteria for a torrent to be considered "slow"?

I don't know if I should enable or disable this because I haven't seed any metrics that qbitrr seeks. For example, what if "DoNotRemoveSlow = true" starts removing torrents within my tolerable range?

The DoNotRemoveSlow  config is user configurable with the MaximumETA config

Understood, thank you for your help


Check If Picture is Present

Wthout This it will cause Unhandled Exception.

I got this bug. Fixed it in version 3.2.5, with some others changes!


Error at launch :/

Hello, thx for your sharing code.
I have an issu at launch:
D:\Bot\reaction-role-example-main\node_modules\reaction-role\dist\index.js:48
            const adapter = new bookman_1.adapters.MongoDB({
                                                   ^

TypeError: Cannot read property 'MongoDB' of undefined
    at new ReactionRole (D:\Bot\reaction-role-example-main\node_modules\reaction-role\dist\index.js:48:52)
    at Object.<anonymous> (D:\Bot\reaction-role-example-main\index.js:9:16)
    at Module._compile (internal/modules/cjs/loader.js:1063:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
    at Module.load (internal/modules/cjs/loader.js:928:32)
    at Function.Module._load (internal/modules/cjs/loader.js:769:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
    at internal/main/run_main_module.js:17:47
[nodemon] app crashed - waiting for file changes before starting...

I have filled in the fields in .env
Hve you an idea ?

Can i see your code?

My bad i only use npm i but note yarn :/
I see yarn or npm i
It work with yarn, sorry

Actually it's not your fault i just updated package and not example
i'll update it soon


feat(admin-ui): add webhooks page

Pull Request Test Coverage Report for Build<PHONE_NUMBER>9
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 9.506%

Totals

Change from base Build<PHONE_NUMBER>0:
0.0%

Covered Lines:
10358

Relevant Lines:
108966

💛  - Coveralls