proposed 1.x deprecations

The following have been proposed for deprecation in a future release of CouchDB:

_show
_list
OS daemons
Couch proxy stuff (Couch proxying other services - couch_httpd_proxy
VDUs?
Update functions?
JS engine (see #1513)
View changes (never worked right)
couch_external_*
rewrites
vhosts
@davisp: "I'd really like to remove the _replicator, _users, and other special DBs from being actual databases that support the whole database API and instead have a specific API that we define (even if we use databases under the hood)."
@chewbranca: couch_primary_sup / couch_secondary_sup

The joke about removing attachments has been removed...that's not happening :)

Hi @wohali
I wonder what are the proposed alternatives to:

VDUs?
Update functions?

Embedded into couch core Erlang functions? I'm asking because in our production environment we are using a heavily customised to our use case Erlang validate_doc_update functions and even implemented a custom logic in our fork of CouchDB to support validate_doc_read functions. So I'm interested where is VDU stuff heading to?

Don't ask me, I'm just the transcriber of the content. The proposals came from @chewbranca and @davisp.

Design documents and couch_httpd_proxy used to the onboarding feature, the reason why is very well described in Nolan's blog post.

we’ve been over this a couple of times now. This is all on the way out unless someone comes and takes responsibility for it initially and going forward. That means writing and actively maintaining Erlang code. That nobody since 2012 or so stepped up to do this is enough of an indicator that these features can go, especially with some of the partially broken already in 2.x.

That nobody since 2012 or so stepped up to do this

Oh, please. I personally tried to, and PMC reception was subzero cold. No one wants 15 PMCs directly hostile to couchapps telling him what to do, or telling ‘oh, we r sorry, we have only 8 month to new release so features list is frozen’. Taking in account ~half of PMC now consists of people only known for adding +1 under regular initiatives to amputate/deprecate/slowdown this or that, and for having no real projects with Couch, I have no doubt no one will volunteer.
Also taking in account I personally know 6 companies having custom builds of CouchDB with QS-related features and fixes, and all say they never contribute fixes/features to this repo because of PMC toxic attitude to couchappers (let me skip ML quotations).
So may be the case is not about having no QS maintainers? May be the case is PMC carefully crafting Stockholm syndrome in those who could (or even tried to) contribute?

I assume this blog post is the one being referenced? I don't really want to wade into the couchapps debate (certainly not with a blog post I wrote in 2013 😛), but FWIW many of these same features have been discussed getting deprecated in PouchDB and pouchdb-server for years. @daleharvey can probably provide a more up-to-date perspective on that.

@ermouth This is a formal code of conduct warning. You need to assume good faith in this discussion, and I don't believe you are doing so. You are just continuing to bring more anger and rancor to the community. I recognize you are angry and want to fight for something you believe in, but if you continue in this accusatory and belligerent manner, I will remove you from this discussion.
The CouchDB contributors and PMC members you mention that "just +1" have a lot of commits to the project behind the scenes - especially if you look at couchdb-fauxton (Garren, Antonio, and Alexis are very active contributors) and the Cloudant CouchDB repo (Mike Rhodes, Adam Kocoloski, Robert Newson, Ilya Khlopotov, Nick Vatamaniuc, and Paul Davis, to name just a few) from which a lot of important PRs come.  It also includes non-Cloudant people like Dave Cottlehuber and Andy Wenk, whose contributions in terms of organisation, coordination and release management are very welcome. These very people - the people who are doing the work on CouchDB and contributing regularly - are the ones who get to decide what happens on the project. That's how Apache works.
We have to make a pragmatic, practical decision about how to proceed with the resources we have, the resources we attract and the direction we're looking to go in. Couchapp functionality is no longer the focus of CouchDB, for all the reasons previously stated.
You want to contribute to couchapps still? Great - either submit a PR with your proposed changes and improvements, or stop complaining. Those are the choices I are giving you. I am personally asking you - not anyone else in this thread or any other company, and as nicely as I can - to submit your PRs in good faith and I will personally look at them in good faith.
As an side, If you know of 6 companies that refuse to contribute code here, as much effort as I could try to put in and heal wounds and mend differences, it's their choice if they contribute or not - and as our VP points out, they haven't for many years now. The AL2.0 means they are more than welcome to maintain their own fork, as long as they do not call it Couch-anything or Apache-anything.

I share the frustration of @ermouth over the fact that his work with the rewrite function was not included in the 1.7.  The requirement for active contribution from @janl has been very clear, and was met with efforts from @ermouth and @giowild in the past, the promise that was expected to follow the requirement is no longer credible.
The establishment of the couchapp mailing list led to a peak in interest, but with no proposals being accepted this came to a halt. The list no longer on the web site, another of those undead items. 1.7 and 2.0 could have been good for couch apps, but with one missing rewrite functions and the other missing couch_httpd_proxy, we already use a patched version of 1.6.2.
@natevw said it in a funny way "Backfill those rabbit holes!" since the quicky-breeding onboarding of CouchDB is what I always saw as the win-win. Getting hurt holding out for improvements in an areas which the maintainers are no longer interested is the problem that we as users must take responsibility for.
A fork is the only option left.

I dont even think a fork is needed, with PouchDB I got lucky with having the hindsight of seeing how CouchApps played out, PouchDB core got to start out as what CouchDB is moving towards, being a solid database and that makes it easier for people to write things like pouchdb-server or rxdb which provide more application level constructs

A gentle reminder from me that the opening comment on this issue is proposals for things to remove. We should use the issue to discuss which things we will actually remove, when we'd remove them, and what, if anything, would replace them.

@wohali

recognize you are angry and want to fight for something you believe in
You want to contribute to couchapps still?

There is kind of misunderstanding. We use couchapps, but not in the way you think. We have Couchbox, which completely replaces couchapp-related QS fns, able to run legacy code, and is much faster and reacher with features (but much heavier). So we do not need _list, _rewrite or _update for couchapps.
But there exist other scenarios, probably not so visible, where built-in QS stuff seem to be extremely valuable.
Update fns is the only way to send whatever data to Couch without prior reading. It‘s invaluable feature for ie nets of sensors. Couch here acts as an aggregator, and ability to run Couch on PI allows to make aggregators very lean, setup them in place, and avoid app layer completely (which is obviously important for lean devices).
Lists are also valuable for scenarios of aggregating data from net of sensors. You can perform remote data lookup on whatever basis without fetching real data, and have no serious bucket stalls, like you have with views. Mango queries seem to provide alternative mechanics for this, but they are less flexible (however much more fast).
Rewrites (only as functions surely) are just great. Ability to re-wire API remotely has a lot of applications aside of couchapps. Also this approach of providing API is easily testable without deployment. Functions are easy to play with, unlike sets of nginx or haproxy rules.
All those features just work. They might be faster, and some improvements are low hanging fruits, but they already work in acceptable way. They only need very minor repair time to time.
So I can’t accept your ”submit PRs or stop complaining“. There’s not so lot of things I personally want to improve. However, I think I can help with fixing (or at least nailing down) QS-related bugs. Hope it may help to preserve those features little bit longer.

@johs The improved rewrites not being included in 1.7 was an oversight in the rush to get it out, not an intentional omission. Sorry about that :(
@ermouth Thanks for the clarifications and specific points. Your assistance on the repro case for #1544 is greatly appreciated; more of this support going forward and the occasional PR is all I'm asking for. And while the passion is helpful, the anger is not. Thanks for responding civilly.
We've already clarified that update functions won't go away without a suitable replacement. I suspect that the JS engine replacement (see below) will still support update and VDU functions, but I don't see evidence of it yet. The question is more along the lines of: can we do better in a declarative, Mango-like fashion? There are other proposals to provide similar/useful functionality (#1532, #1533, #1498) if the operator decides to run without a JS engine (#1513), plus I suspect there are thoughts about doing Mango-based update functionality to filter fields, as well as apply database schema enforcement (type and format checking) in a declarative fashion for a NoJS mode. Could your embedded RasPi update function be written declaratively? I assume you're adding info such as incoming IP address, central timestamp, etc. which I could see being done automatically quite easily. This is a use case we can accelerate without the use of JS readily; we just need to agree on an API and how to write it down in a JSON blob.
The big problem with the query server implementation is the age of JS 1.8.5, and the relative difficulty of upgrading the JS engine without touching anything else, plus addressing the long-standing issue of poor query server performance, especially at scale. @davisp is the one working on bringing the new JS engine into CouchDB via a NIF over in https://github.com/cloudant-labs/erlang-chakracore .  From what I know of Paul's current thinking, one possibility would be a full refactor of the query server protocol/implementation and use external processes again to solve this problem, but more efficiently. This is a big task and could hold up the entire effort. Another is to just drop lists going forward, preserving the better performance provided by the NIF binding for the core JS functionality. Perhaps you can assist, if you know NIFs and C/C++ well... Alternately, what would Mango need added to it to give you sufficiently improved list functionality to fully replace JS lists?
Overall, rewrites are far less often used than either of the other two features, vhosts even less, and proxies even less than that. Cloudant doesn't support JS as rewrites, and I could see Couch going the same way for a NoJS mode at the very least - with static or templated/regex rewrites over in Mango. But I think there's some concern about having rewrites at all in what is increasingly envisioned as a pure database. I can't think of another database engine that allows you to rewrite its API.... This is where we'll have to agree to disagree on the priority of a feature, but again, I'm just a single vote.

Thanks @johs and @ermouth for mention.

There is kind of misunderstanding. We use couchapps, but not in the way you think.
All those features just work. They might be faster, and some improvements are low hanging fruits, but they already work in acceptable way. They only need very minor repair time to time.

I strongly agree with ermouth words.
I think there is a substantial lack of comprehension on what these features can already give out of the box, as they are now, and how they can be used to achieve what today is thought impossible to achieve with couchdb, if not using a proxy.
I already tried to explain (unfortunately in vain) in the past on the ML, how vhosts and rewrites can substantially be used as sorts of firewalls and routers to implement fine access control and routing(to other design docs rewrites, show,list,updates and so on) even before reading documents from database. This opens up endless possibilities for app developers. Just to make an example: the long-time asked "document-level acl" is already possible, withouth security flaws, simply by using correctly all rewrites, shows, list, views and updates features. This is in place, as example, to deliver smileupps website, cloud panel and billing!!! Other big plus are the possibility to perform hot-swap when upgrading a website from version 1.0 to 2.0 (by updating the first routing document in the chain), or possibility to work on a draft version of a website, different from what the user is watching (by simply using a vhost pointing to a different design doc/rewrites in the same database).
These are features app developers want and these are only few examples of what can ALREADY be achieved with existing couchdb features!
After that, only adding few minor enhancements, can give tremendous benefits to app developers improving overall ease of use, flexibility and security. We tried to discuss them in the ML in vain, then we ended up implementing them in our personal Couchdb-fork.
I'm referring to the possibility of pushing more details inside rewrites documents, such as acl details. What said above, becomes trivial in this way:
[
   {"to":"ABC","from":"XYZ","logged":"true"},
   {"to":"ABC","from":"XYZ","user":"joe"},
   {"to":"ABC","from":"XYZ","role":"author"}
]

or the possibility to replace special keywords in rewrites rules, such as :actualuser,:actualrole,:actualdb and :actualvhost
or the possibility to use absolute paths instead of relative paths, which greatly improves simplicity from an app developer perspective
or the possibility to use current database instead of _users database for user authentication
and much more...

It is sad to say that what is missing in CouchDB today, is not a specific feature, but the VISION and WILL of DIFFERENTIATING from other database products.
@joan said:

I can't think of another database engine that allows you to rewrite its API
and this IS AN ADVANTAGE for CouchDB!!! :-) These are exactly the kind of features where couchdb CAN and SHOULD differentiate, otherwise it will forever remain another NoSQL database, differentiating only for the fact that its API is native and not a plugin.

I would like a database meant FOR the APP DEVELOPER and not for a db or system administrator. If no app developers will ever write apps with couchdb, no db or system administrators will be ever required to mantain its databases!
IMHO app developers tends to value the following features in this order:

expressiveness, flexibility and ease of use (impacting on development time) + app-level features (authentication, authorization, website+app inside database, in-browser development, etc.)
security: no information must leak from the database (rewrites are great for this)
performance
system-level features (clustering is here :-)

In the end, FWIW this is my invaluable set of features:

_rewrites
vhosts
_show
_list
OS daemons
Couch proxy stuff (Couch proxying other services - couch_httpd_proxy
Update functions

@giowild I understand you disagree, and I appreciate your comment here, but I'm unconvinced by your argument. I especially want to thank you for being civil in your comment.
The fundamental difference of opinion here hasn't changed. The UNIX philosophy applies to CouchDB as to any other program: do one thing, and do it well. The developers and PMC of CouchDB are increasingly making the product a better database, not a better one-stop app server + database. Our key differentiation in the marketplace isn't app serving, it's replication, with the HTTP DB API/JSON interface as a close second, and scaling horizontally as the third most important feature. Apps and their related features were an afterthought, and while people like you have made it work for them, it was an experiment that didn't pay off writ large, and has become less compelling with each passing year. While we don't have live telemetry from every install that backs these claims, we have data from Cloudant customers over 6 years and from the mailing lists that do back it up.
Even on a lowly Raspberry Pi 3, you have 4 cores now; running an app server in any other language alongside CouchDB on this humblest of platforms gives you better app server functionality and performance by leaps and bounds than we can ever hope to achieve in CouchDB's core. The same goes for a proxy server like HAproxy or Nginx when it comes to proxying, vhosts and rewrites.
As to your list of features, _show and _list are arguably useful for data export in different formats from CouchDB, and update functions/VDUs are also very important (see #1554 for a possible evolution). I can see the value in them. While I know you don't want to see them go, I'm sure you'd agree that the others are not valuable in a database-only development approach, as they can be filled by an app server, a reverse proxy, or whatever other process you wish to run alongside our database.
Thanks again for the discussion. Again, I am only one vote in this deprecation discussion.

Could your embedded RasPi update function be written declaratively?

Very unlikely, devices do not post json. For now both _update and _rewrite as a fn are able to chop up requests having binary body, and then decide. I have no doubts any DSL making this kind of work will be either useless for other scenarios, or too complex if it wants to cover everything. DSLs are not a substitute for yet unknown tasks. Any DSL is domain-specific, or it’s a cumbersome mess.

The UNIX philosophy applies to CouchDB as to any other program

Indeed, however ‘do one thing’ is, looking at most of modern really user- and dev-friendly SW, bit unpopular from UX point of view. This rule was good at CLI-only times, and those times are gone. For now, standing for this rule mostly increases transaction costs: the approach of loosely stitching things together is good for scripts, but often induces heavy deployment and impedance alignment costs for continuous systems.
Also let me remind another Unix rule that still stands, the rule of diversity: Make programs flexible, allowing them to be used in ways other than those their developers intended.

The same goes for a proxy server like HAproxy or Nginx when it comes to proxying, vhosts and rewrites

I have nothing to say about vhosts and proxying, we never used them seriously as those settings are not replicatable, but as for rewrites...
What about deployment? If you have hundreds of nodes which are not intended to be members of a cluster, updating external proxy rules is a serious pain with immense number of uncomfortable (or just dangerous) subtleties. With delivering API wiring using regular data flow deployment is just one click, because it uses most powerful feature of Couch: replication. Utils like nginx-sync do not stand even near.

So far, the big place he's stuck is implementing lists

It would be nice to have more info, no one can help having no description of a problem.
Should removing or simplifying .info obj ease the task? It was done for rewrites as a fn, and this trim, aside of other effects, gives good perf improvement comparing to other QS stuff. Actually, CouchDB 1.6.1 patched with JS rewrites and called through rewrite JS function is still in most cases bit faster then naked single-node 2.x.

Alternately, what would Mango need added to it to give you sufficiently improved list functionality to fully replace JS lists?

Full replacement is impossible, or you will make new DSL just another programming language. However if narrow the scope to simplest cases and JSON-only, joins and reducers are obvious answer. If former is more or less clear and implementable, the latter just does not fit well into DSL concept except simplest cases.
So, summing up: I have nothing to say about non-replicatable features or features proven to be buggy and unreliable. But I think most features using benefits of syncing code with data flow better be kept.

We have never been closer to the core of the "couchapp" discussion than with @joan discussing differntiation in DB land:

I can't think of another database engine that allows you to rewrite its API....

.. and  @giowild discussing onboarding

If no app developers will ever write apps with couchdb, no db or system administrators will be ever required to mantain its databases!

.. and @ermouth discussing syncing code with data

I think most features using benefits of syncing code with data flow better be kept.

.. not forgetting the prototyping feature that @nolanlawson desicribed so well back in 2014:

ultimately your webapp will grow in complexity to the point where you’re forced to put a proxy server in front of CouchDB to smooth out its rougher edges, but I think you’ll still be amazed how far you can get with vanilla Couch.

CouchDB has offered truly unique CI/CD productivity on a single platform.
If the work with the https://github.com/cloudant-labs/erlang-chakracore is successful and javascript developers can still host their server-side functions in CouchDB and clientside apps as attachements maintained in couch/pouch mini-IDEs like http://ddoc.me/ I think the most interesting question is:
What is the disadvantage of preserving the experimental space of design documents?
If this opportunity space -- especially related to onboarding -- can be preserved at low cost, then it would be very unwise to remove it.

Speaking for myself, the disadvantage is that these features have poor functionality as compared to what you can do with an app server alongside or in front of CouchDB. They mislead people into thinking CouchDB is a fully-fledged application server environment. People waste time building things in this environment, discover all of the inherent limitations, and have to rewrite everything in an standalone app server. They then complain about the lack of functionality in something that has stagnated for many years.
Bringing chakracore into CouchDB is not going to allow you to npm install modules into a ddoc so you can leverage the entire JS ecosystem inside a CouchApp. The sandbox I expect to come with the new engine will continue to enforce the same limitations that exist in the current query server implementation.
To put a finer point on the standpoint of the entire development team, the documentation's single remaining reference to CouchApps has a good summary that I wrote in coordination with them:

Note: Previously, the functionality provided by CouchDB’s design documents, in combination with document attachments, was referred to as “CouchApps.” The general principle was that entire web applications could be hosted in CouchDB, without need for an additional application server.

Use of CouchDB as a combined standalone database and application server is no longer recommended. There are significant limitations to a pure CouchDB web server application stack, including but not limited to: fully-fledged fine-grained security, robust templating and scaffolding, complete developer tooling, and most importantly, a thriving ecosystem of developers, modules and frameworks to choose from.

The developers of CouchDB believe that web developers should pick “the right tool for the right job”. Use CouchDB as your database layer, in conjunction with any number of other server-side web application frameworks, such as the entire Node.JS ecosystem, Python’s Django and Flask, PHP’s Drupal, Java’s Apache Struts, and more.

I shared this in #fdb channel  in CouchDB slack, but posting here as well https://docs.google.com/document/d/1vvqgMR5U1X8yIJj7ctse_D31VVNXb9NnMIJKg8bDBl8/edit

Hi @jdmintz , thanks for joining our community. I assume you're with IBM/Cloudant? We've never had any interaction with you in the CouchDB community.
I want to note for observers of this ticket that decisions on what features are and aren't being kept from CouchDB 1.x are made by the committer community and CouchDB PMC, not by IBM/Cloudant. While we're really happy to have IBM's support in building CouchDB 4.0, the ultimate decision is reached by our community, not by any corporation.
Further, those decisions are reached through discussions on our official mailing list<EMAIL_ADDRESS>There are also formal requirements for feature deprecation set our in our project bylaws Posting plans by a single part here here is insufficient.
I would encourage you to join our developer mailing list if you are proposing deprecations over and above what's already been discussed by the community in this topic or on our list.

Apologize if it came across as decision making. That was not my intent.
I was just seeking to summarize things scattered across ML posts, RFC drafts, and earlier comments in this ticket.

I want to note for observers of this ticket that decisions on what features are and aren't being kept from CouchDB 1.x are made by the committer community and CouchDB PMC, not by IBM/Cloudant.

I want to note for observers that this statement is at least imprecise. Consider counting +1 replies under deprecation [VOTE] postings at MLs. About 3/4 of votes are from IBM/Cloudant employees.

only PMC votes are binding.

Hi @wohali ,
A summarization of outgoing features would have been nice, especially with respect to 3.0
In his August report @janl said...

Current planning includes both a CouchDB 3.0 and a CouchDB 4.0 milestone. 3.0
will include the best version of the current, mostly Erlang-based project,
with many new features contributed by various project partners (but notably
IBM).

... which made me hopeful that the "couch app" features would survive into version 3.0 together with the spider monkey update, ref https://github.com/apache/couchdb/issues/1875

@ermouth First off, this is not the place for this discussion.
Secondly, the committer base and PMC is quite diverse - here is a full list.
@johs That decision has not been taken yet, and the two decisions you mention are indeed separate.

correction, committer votes are binding for some things, PMC votes for other things.

@wohali, it’s not a discussion, I just state that +1 votes under deprecations are made mostly by PMCs who are IBM/Cloudant employees. This fact is easily accountable, which raise obvious questions to you personally, as ASF board member responsible for ethics.

@ermouth Point taken. Yes, I am well aware of this concern, and am one of the louder voices regarding "corporate capture" of projects at the Foundation level - which I think everyone on the CouchDB project knows as well.
But this is not the place to discuss this point. If you have anything further on this topic, please take it to the mailing list, not this issue - it does not belong here.

Closing this ticket in favour of the following:

#2218 What's being removed in 3.0
Beyond that, see<EMAIL_ADDRESS>


Help newcomers to learn what trigger() does

Came to this page looking for a clear description of trigger() for a tutorial.  As it reads, the section does not connect the dots for a newcomer.  This change makes a quick summary and connects to existing documentation for more details.  Same for read() and describe() since they are in the same section.

Thanks for cleaning up after me @prjemian  :)


Blank page will be displayed in options page since <IP_ADDRESS> - options.js:1 Uncaught SyntaxError: Invalid or unexpected token

Looks like build from GitHub Actions makes this issue, same build step in local makes correct.
https://github.com/kachick/depop/pull/24
The product zip is 3KiB different.
GitHub Actions: 134kb
WSL2 - Ubuntu 22.04: 137kb

Fixed by https://github.com/kachick/depop/issues/25


paml: 4.9g -> 4.9h

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools.
This update was made based on information from https://repology.org/metapackage/paml/versions.
These checks were done:

built on NixOS
ran ‘/nix/store/39cfy96430yznv6h3cg7zd8850i6kdl1-paml-4.9h/bin/chi2 -h’ got 0 exit code
ran ‘/nix/store/39cfy96430yznv6h3cg7zd8850i6kdl1-paml-4.9h/bin/chi2 --help’ got 0 exit code
ran ‘/nix/store/39cfy96430yznv6h3cg7zd8850i6kdl1-paml-4.9h/bin/chi2 help’ got 0 exit code
found 4.9h with grep in /nix/store/39cfy96430yznv6h3cg7zd8850i6kdl1-paml-4.9h
directory tree listing: https://gist.github.com/b2a37e7cd2b4b41178f0698f3aac69e2

Success on x86_64-darwin (full log)
Attempted: paml
Partial log (click to expand)
'codeml' -> '/nix/store/klmbbdqbbgyvhwsixdygnallsqib1g1w-paml-4.9h/bin/codeml'
'evolver' -> '/nix/store/klmbbdqbbgyvhwsixdygnallsqib1g1w-paml-4.9h/bin/evolver'
'mcmctree' -> '/nix/store/klmbbdqbbgyvhwsixdygnallsqib1g1w-paml-4.9h/bin/mcmctree'
'pamp' -> '/nix/store/klmbbdqbbgyvhwsixdygnallsqib1g1w-paml-4.9h/bin/pamp'
'yn00' -> '/nix/store/klmbbdqbbgyvhwsixdygnallsqib1g1w-paml-4.9h/bin/yn00'
post-installation fixup
strip is /nix/store/kdff2gim6417493yha769kh00n63lnrw-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/klmbbdqbbgyvhwsixdygnallsqib1g1w-paml-4.9h/bin
patching script interpreter paths in /nix/store/klmbbdqbbgyvhwsixdygnallsqib1g1w-paml-4.9h
/nix/store/klmbbdqbbgyvhwsixdygnallsqib1g1w-paml-4.9h

Success on x86_64-linux (full log)
Attempted: paml
Partial log (click to expand)
shrinking /nix/store/39cfy96430yznv6h3cg7zd8850i6kdl1-paml-4.9h/bin/basemlg
shrinking /nix/store/39cfy96430yznv6h3cg7zd8850i6kdl1-paml-4.9h/bin/mcmctree
shrinking /nix/store/39cfy96430yznv6h3cg7zd8850i6kdl1-paml-4.9h/bin/chi2
shrinking /nix/store/39cfy96430yznv6h3cg7zd8850i6kdl1-paml-4.9h/bin/pamp
shrinking /nix/store/39cfy96430yznv6h3cg7zd8850i6kdl1-paml-4.9h/bin/baseml
strip is /nix/store/j75dgadrff2d1fyc4fczmcgqkid2imdx-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/39cfy96430yznv6h3cg7zd8850i6kdl1-paml-4.9h/bin
patching script interpreter paths in /nix/store/39cfy96430yznv6h3cg7zd8850i6kdl1-paml-4.9h
checking for references to /build in /nix/store/39cfy96430yznv6h3cg7zd8850i6kdl1-paml-4.9h...
/nix/store/39cfy96430yznv6h3cg7zd8850i6kdl1-paml-4.9h

Success on aarch64-linux (full log)
Attempted: paml
Partial log (click to expand)
shrinking /nix/store/7vjfl7728jvai634275dqb2pjbswyy92-paml-4.9h/bin/evolver
shrinking /nix/store/7vjfl7728jvai634275dqb2pjbswyy92-paml-4.9h/bin/chi2
shrinking /nix/store/7vjfl7728jvai634275dqb2pjbswyy92-paml-4.9h/bin/basemlg
shrinking /nix/store/7vjfl7728jvai634275dqb2pjbswyy92-paml-4.9h/bin/baseml
shrinking /nix/store/7vjfl7728jvai634275dqb2pjbswyy92-paml-4.9h/bin/codeml
strip is /nix/store/gp7fylxwn18b7pl2c18ks89hsiaxyfvf-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/7vjfl7728jvai634275dqb2pjbswyy92-paml-4.9h/bin
patching script interpreter paths in /nix/store/7vjfl7728jvai634275dqb2pjbswyy92-paml-4.9h
checking for references to /build in /nix/store/7vjfl7728jvai634275dqb2pjbswyy92-paml-4.9h...
/nix/store/7vjfl7728jvai634275dqb2pjbswyy92-paml-4.9h


[Feature]: Provide ability to pass CancellationToken into Async methods

Is there a way to use cancellation tokens to cancel async operations?
It would be helpful to have most of the async methods able to be cancelled. Especially methods like
await page.GotoAsync("https://github.com/");
which would then be:
await page.GotoAsync("https://github.com/", cancellationToken);

Yea, it would be nice.
In our project on some cases we are using "dirty task cancellation" similar to this https://devblogs.microsoft.com/pfxteam/how-do-i-cancel-non-cancelable-async-operations/

Closing it, since there are no plans to support it. Browser interactions like clicks, fills are all not cancelable, so even if we would provide it, it would be a kinda fake cancel, and a certain action would still run in the background.

@mxschmitt this is true from a browser user perspective, but I think it's really not about cancelling any actions/commands but more about cancelling potential waiting process for the callbacks of the commands.
As far as I can see all commands are sent via SendMessageToServerAsync() - and for each command the callback is being awaited. Why should it not be reasonable to cancel this instead of waiting for callback to be fulfilled? For example when I want to cancel the execution (e.g: some test run, or some executable), why would I still need to wait until the await page.GotoAsync("https://github.com/") command is having some callback. I just want the execution to be stopped gracefully as fast as possible and no "hanging" execution.
I would really appreciate, if you would reconsider your decision or give us some alternative way to gracefully cancel pending commands.


error downloading Facebook video

Unable to download a video from Facebook. I verified in a  browser the video was working, and did not require the user to be signed in. Here's the URL I'm trying to get:
https://www.facebook.com/thecalvinwayman/videos/1117031128379911/
Here's the output:
:~$ youtube-dl --verbose "https://www.facebook.com/thecalvinwayman/videos/1117031128379911/"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'https://www.facebook.com/thecalvinwayman/videos/1117031128379911/']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.02.28
[debug] Python version 2.7.12 - Linux-4.4.0-59-generic-x86_64-with-Ubuntu-16.04-xenial
[debug] exe versions: ffmpeg 2.8.10-0ubuntu<IP_ADDRESS>, ffprobe 2.8.10-0ubuntu<IP_ADDRESS>
[debug] Proxy map: {}
[facebook]<PHONE_NUMBER>379911: Downloading webpage
[facebook]<PHONE_NUMBER>379911: Downloading webpage
ERROR: Cannot parse data; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/YoutubeDL.py", line 761, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/common.py", line 427, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/facebook.py", line 408, in _real_extract
    video_id, fatal_if_no_video=True)
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/facebook.py", line 325, in _extract_from_url
    raise ExtractorError('Cannot parse data')
ExtractorError: Cannot parse data; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Which is it a duplicate of, please?


Plugin not finding mix

When not starting atom from the terminal linter-elixir-credo does not find mix.
This probably since the environment variables are not the same.
Possible solution is to do like elm-format or linter-flow that lets you add a

Hi @note89
I updated the package, it now contains an option in the settings to set the path to mix.
The package now also has an option to pass along flags to credo, curtesy of @sgtpepper43 :)
Let me know if this solves your problem when running atom from a desktop shortcut.
Gerard

@note89 I'll close this for now, if you still run into any problems, let me know !

Hi i was away for awhile.
I no longer get the popup error but i also don't get any suggestions from credo.
If i start via the terminal i still works.
Maybe i should look into it, given that im the one with the issue.
I just can't find the time now thou given that i can start it from the terminal and it works just fine :)


Use test app link

Use test app url for now so that beta testers aren’t directed to login to main app, but rather directed to login to the test app. This will need to be switched to production url when getting merged to master later on.

Codecov Report

Merging #368 (a09106f) into develop (4d3a060) will decrease coverage by 0.03%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop     #368      +/-   ##
===========================================
- Coverage    60.70%   60.66%   -0.04%
===========================================
  Files           80       80
  Lines         2555     2555
  Branches       373      373
===========================================
- Hits          1551     1550       -1
  Misses         995      995
- Partials         9       10       +1

Impacted Files
Coverage Δ

app/sendgrid/templates/user_b_shared_email.py
100.00% <ø> (ø)

app/sendgrid/templates/welcome_email.py
100.00% <ø> (ø)

app/scoring/tests/data.py
96.66% <0.00%> (-3.34%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us.

@rodriguesk url should not be hardcoded at all, please create a task for you to change it back before the launch. Otherwise it could be a critical bug.

Yes, I agree. It was just fine for now so people can use the app. Non-feature technical tasks might be best for someone else to add in Jira so the details are correct. @danmash could you add it? I can include it in the next sprint we do.


Keep search field visible

I think keeping the field visible at all times is useful:

In the most common use case, it shows the name of the package being plotted. For huge projects this is helpful: https://npmgraph.js.org/?q=web-ext
It could enable this: https://github.com/npmgraph/npmgraph/issues/222

The field used to be visible, as seen in an old PR: https://github.com/npmgraph/npmgraph/pull/91

Design exploration needed. I think an always-visible field is fundamental when comparing modules. They all have it:

https://packagephobia.com/result?p=doma
<EMAIL_ADDRESS>https://npm.anvaka.com/#/view/2d/doma
https://npmtrends.com/angular-vs-react


unable to complile android app on windows

Summary
unable to compile app
Environment Information

Device Name: my pc
OS Version: windows 10
Mattermost App Version: latest available ANDROID app
Mattermost Server Version: latest available

Steps to reproduce
1follow instructions to set up dev environment attempting to compensate for lack of description for windows environment
Expected behavior
I expect the application to compile successfully
Observed behavior
failed to compile with the following errors:
2018-06-25 14:31:25,575 [ thread 40]   INFO - .project.GradleProjectResolver - Gradle project resolve error
org.gradle.tooling.BuildException: Could not run build action using Gradle distribution 'https://services.gradle.org/distributions/gradle-2.14.1-all.zip'.
at org.gradle.tooling.internal.consumer.ExceptionTransformer.transform(ExceptionTransformer.java:51)
Caused by: org.gradle.api.artifacts.ResolveException: Could not resolve all dependencies for configuration ':app:_debugCompile'.
Caused by: org.gradle.api.artifacts.UnknownConfigurationException: Configuration with name 'default' not found.
2018-06-25 14:48:14,391 [ thread 55]   WARN - ect.sync.idea.ProjectSetUpTask - Configuration with name 'default' not found.
2018-06-25 14:48:14,392 [ thread 55]   INFO - e.project.sync.GradleSyncState - Gradle sync failed: Configuration with name 'default' not found.
Possible fixes

npm WARN optional Skipping failed optional dependency /sane/fsevents:
npm WARN notsup Not compatible with your operating system or architecture<EMAIL_ADDRESS>npm WARN optional Skipping failed optional dependency /chokidar/fsevents:
npm WARN notsup Not compatible with your operating system or architecture<EMAIL_ADDRESS>npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>|| ^0.26.0 || ^0.24.0 but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm ERR! Linux 4.4.0-17134-Microsoft
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "install"
npm ERR! node v4.2.6
npm ERR! npm  v3.5.2
npm ERR! code ELIFECYCLE
npm ERR<EMAIL_ADDRESS>postinstall: make post-install
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>postinstall script 'make post-install'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the mattermost-mobile package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     make post-install
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs mattermost-mobile
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls mattermost-mobile
npm ERR! There is likely additional logging output above.
npm ERR! Please include the following file with any support request:
npm ERR!     /mnt/c/AndroidStudioProjects/mattermost-mobile/npm-debug.log

I tried again using the linux sub-system for windows and got this error
npm-debug.log

ok maybe progress; now unable to resolve dependencies:
Unable to resolve dependency for ':app@unsigned/compileClasspath': Could not resolve project :react-native-document-picker. Open FileShow Details |
-- | --
Unable to resolve dependency for ':app@unsigned/compileClasspath': Could not resolve project :react-native-keychain. Open FileShow Details |
Unable to resolve dependency for ':app@unsigned/compileClasspath': Could not resolve project :react-native-doc-viewer. Open FileShow Details |
Unable to resolve dependency for ':app@unsigned/compileClasspath': Could not resolve project :react-native-video. Open FileShow Details |
Unable to resolve dependency for ':app@unsigned/compileClasspath': Could not resolve project :react-native-navigation. Open FileShow Details |
Unable to resolve dependency for ':app@unsigned/compileClasspath': Could not resolve project :react-native-image-picker. Open FileShow Details |
Unable to resolve dependency for ':app@unsigned/compileClasspath': Could not resolve project :react-native-bottom-sheet. Open FileShow Details |
Unable to resolve dependency for ':app@unsigned/compileClasspath': Could not resolve project :react-native-device-info. Open FileShow Details |
Unable to resolve dependency for ':app@unsigned/compileClasspath': Could not resolve project :reactnativenotifications. Open FileShow Details |
Unable to resolve dependency for ':app@unsigned/compileClasspath': Could not resolve project :react-native-cookies. Open FileShow Details |
Unable to resolve dependency for ':app@unsigned/compileClasspath': Could not resolve project :react-native-linear-gradient. Open FileShow Details |
Unable to resolve dependency for ':app@unsigned/compileClasspath': Could not resolve project :react-native-vector-icons. Open FileShow Details |
Unable to resolve dependency for ':app@unsigned/compileClasspath': Could not resolve project :react-native-svg. Open FileShow Details |
Unable to resolve dependency for ':app@unsigned/compileClasspath': Could not resolve project :react-native-local-auth. Open FileShow Details |
Unable to resolve dependency for ':app@unsigned/compileClasspath': Could not resolve project :jail-monkey. Open FileShow Details |
Unable to resolve dependency for ':app@unsigned/compileClasspath': Could not resolve project :react-native-youtube. Open FileShow Details |
Unable to resolve dependency for ':app@unsigned/compileClasspath': Could not resolve project :react-native-sentry. Open FileShow Details |
Unable to resolve dependency for ':app@unsigned/compileClasspath': Could not resolve project :react-native-exception-handler. Open FileShow Details |
Unable to resolve dependency for ':app@unsigned/compileClasspath': Could not resolve project :react-native-fetch-blob. Open FileShow Details |
Unable to resolve dependency for ':app@unsignedUnitTest/compileClasspath': Could not resolve project :react-native-document-picker. Open FileShow Details |
Unable to resolve dependency for ':app@unsignedUnitTest/compileClasspath': Could not resolve project :react-native-keychain. Open FileShow Details |
Unable to resolve dependency for ':app@unsignedUnitTest/compileClasspath': Could not resolve project :react-native-doc-viewer. Open FileShow Details |
Unable to resolve dependency for ':app@unsignedUnitTest/compileClasspath': Could not resolve project :react-native-video. Open FileShow Details |
Unable to resolve dependency for ':app@unsignedUnitTest/compileClasspath': Could not resolve project :react-native-navigation. Open FileShow Details |
Unable to resolve dependency for ':app@unsignedUnitTest/compileClasspath': Could not resolve project :react-native-image-picker. Open FileShow Details |
Unable to resolve dependency for ':app@unsignedUnitTest/compileClasspath': Could not resolve project :react-native-bottom-sheet. Open FileShow Details |
Unable to resolve dependency for ':app@unsignedUnitTest/compileClasspath': Could not resolve project :react-native-device-info. Open FileShow Details |
Unable to resolve dependency for ':app@unsignedUnitTest/compileClasspath': Could not resolve project :reactnativenotifications. Open FileShow Details |
Unable to resolve dependency for ':app@unsignedUnitTest/compileClasspath': Could not resolve project :react-native-cookies. Open FileShow Details |
Unable to resolve dependency for ':app@unsignedUnitTest/compileClasspath': Could not resolve project :react-native-linear-gradient. Open FileShow Details |
Unable to resolve dependency for ':app@unsignedUnitTest/compileClasspath': Could not resolve project :react-native-vector-icons. Open FileShow Details |
Unable to resolve dependency for ':app@unsignedUnitTest/compileClasspath': Could not resolve project :react-native-svg. Open FileShow Details |
Unable to resolve dependency for ':app@unsignedUnitTest/compileClasspath': Could not resolve project :react-native-local-auth. Open FileShow Details |
Unable to resolve dependency for ':app@unsignedUnitTest/compileClasspath': Could not resolve project :jail-monkey. Open FileShow Details |
Unable to resolve dependency for ':app@unsignedUnitTest/compileClasspath': Could not resolve project :react-native-youtube. Open FileShow Details |
Unable to resolve dependency for ':app@unsignedUnitTest/compileClasspath': Could not resolve project :react-native-sentry. Open FileShow Details |
Unable to resolve dependency for ':app@unsignedUnitTest/compileClasspath': Could not resolve project :react-native-exception-handler. Open FileShow Details |
Unable to resolve dependency for ':app@unsignedUnitTest/compileClasspath': Could not resolve project :react-native-fetch-blob. Open FileShow Details |

this is not officially supported

What version of node and npm are you using? I have not tried this on Windows really so no clue but what I can say for sure is that node 10.x won’t work

Hi @andyseubert, we haven't received an update so we'll assume that the problem is fixed or is no longer valid.
If you still experience the same problem, try upgrading to the latest version.
If the issue persists, reopen this issue with the relevant information and we'd be glad to help you where we can.


Fix distance still shows Pending when preview and report header already show distance

Details

When creating a new distance request, the Distance field shows Pending even though the distance value exists. This PR fix it along with some cleanup.
Fixed Issues

$ https://github.com/Expensify/App/issues/43569
PROPOSAL: https://github.com/Expensify/App/issues/43569#issuecomment-2163457710
Tests

Same as QA Steps

[x] Verify that no errors appear in the JS console

Offline tests

Same as QA Steps
QA Steps

Prerequisite:

Have a workspace with multiple distance rates
P2P distance enabled
Enable distance rate tax tracking
Set Tax Rate and Tax reclaimable on for each rate

Open a workspace expense chat
Press the + button > Submit expense > Switch to distance tab
Enter any waypoints and next
In the confirmation page, try changing the rate value
Verify the amount and tax is updated
Submit the request
Open the request transaction thread
Verify the Distance field shows the distance value instead of Pending...
Press the Distance field and update the waypoints
Verify the distance value is updated

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

https://github.com/user-attachments/assets/56676c4a-a129-4ffb-a256-a24fad164ccc

Android: mWeb Chrome

https://github.com/user-attachments/assets/66105347-733c-4e26-b96b-50af8977bad8

iOS: Native

https://github.com/user-attachments/assets/d79dc1f9-9982-4fb6-8391-640421800acf

iOS: mWeb Safari

https://github.com/user-attachments/assets/fbf6bfd8-8c63-4442-b229-7caae327f858

MacOS: Chrome / Safari

https://github.com/user-attachments/assets/c26f0843-13a7-4a20-b1ff-29002d604957

MacOS: Desktop

https://github.com/user-attachments/assets/4fdf317b-93c8-485c-a811-cfeb4147c321

@parasharrajat yep, I noticed that too but that behavior is expected from https://github.com/Expensify/App/pull/43819. (which means it's happening on main too)

Also, how does this work when offline and there is no route? What is shown for the distance? I think it's ok if things are blank but I want to double check that it looks good, and I didn't see any offline testing videos.

The offline behavior will show Map pending image.

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/neil-marcellini in version: 9.0.17-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/marcaaron in version: 9.0.17-2 🚀

platform
result

🤖 android 🤖
failure ❌

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/marcaaron in version: 9.0.17-2 🚀

platform
result

🤖 android 🤖
failure ❌

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/marcaaron in version: 9.0.17-2 🚀

platform
result

🤖 android 🤖
failure ❌

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Where does nc come from?

There are both nc and netcat installed in /usr/local/bin.
$ ls -g -G $(which netcat)
lrwxr-xr-x 1 33 May 14  2016 /usr/local/bin/netcat -> ../Cellar/netcat/0.7.1/bin/netcat
$ ls -g -G $(which nc)
lrwxr-xr-x 1 29 May 14  2016 /usr/local/bin/nc -> ../Cellar/netcat/0.7.1/bin/nc

But when I run make in netcat-0.7.1.tar.gz, there is only the binary netcat. Does anybody know where nc comes from?

#
# Follows the local installation procedures
#
netcat_nc = $(DESTDIR)$(bindir)/nc

install-exec-hook:
        @if [ ! -L $(netcat_nc) -a ! -e $(netcat_nc) ]; then \
                echo "Creating symlink to the shorter executable name"; \
                ln -s netcat $(netcat_nc); \
        fi

in src/Makefile.am
Next time file formula-specific questions in https://github.com/Homebrew/homebrew-core/ and actually fill out the template.


Fix myvariant.info when using gn_vep.region.url

In this case the id for VariantAnnotation is region instead of hgvsg, so
instead we determine hgvsg from the VEP response.

Codecov Report

Merging #320 into master will increase coverage by 0.01%.
The diff coverage is 66.66%.

@@             Coverage Diff              @@
##             master     #320      +/-   ##
============================================
+ Coverage     21.75%   21.77%   +0.01%
  Complexity      180      180
============================================
  Files           169      169
  Lines          2211     2214       +3
  Branches        287      288       +1
============================================
+ Hits            481      482       +1
- Misses         1687     1688       +1
- Partials         43       44       +1

Impacted Files
Coverage Δ
Complexity Δ

...me_nexus/web/mixin/TranscriptConsequenceMixin.java
0% <ø> (ø)
0 <0> (ø)
:arrow_down:

...xus/service/internal/MyVariantInfoServiceImpl.java
35.13% <66.66%> (-0.16%)
5 <1> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a1bf2ad...667979f. Read the comment docs.


[Offline support] Upload new and modified features and records in background service

Blocked by #17.

@navinsivakumar, any ideas on what the API for reading/updating the local queue should look like? There's a draft of other methods for populating the db and queue in PR #60, but it doesn't define how the background sync service will use it.
One option would be to create a single object that groups a FeatureEdit and any related RecordEdits, but considering features and records will be saved separately, this may be overkill.
A simpler approach might be to just have an Edit interface that FeatureEdit and RecordEdit extend, and let expose a function Maybe<Edit> getNextEdit(). Once the edit is complete, we could call something like Completable clearEdit(Edit).
Totally spit-balling here; concrete ideas welcome!


Display kill counts

The kill count info (number of actors of each kind killed by the player and lost by him) is already recorded in high score and during the game in the faction data. For a start, display it as one of the screens just before the final high score screens. Something very simple, with a -more- prompt if it overflows, or with PgUp, PgDn as in Diary.
Later on, we will make it available during the game as part of the info ~ menu.

Done on master branch. Closing.


[SPARK-29494][SQL] Fix for ArrayOutofBoundsException while converting string to timestamp

What changes were proposed in this pull request?

Adding an additional check in stringToTimestamp to handle cases where the input has trailing characters like ':' and ' '.
Added a test to make sure this works.

Why are the changes needed?
In a couple of scenarios while converting from String to Timestamp DateTimeUtils.stringToTimestamp throws an array out of bounds exception if there is trailing spaces or ':'. The behavior of this method requires it to return None in case the format of the string is incorrect.

Does this PR introduce any user-facing change?
No

How was this patch tested?
Added a test in the DateTimeTestUtils suite to test if my fix works.

Can one of the admins verify this patch?

cc @zsxwing

Can one of the admins verify this patch?

ok to test

Test build #112185 has started for PR 26143 at commit 17edc79.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17176/
Test PASSed.

Test build #112186 has started for PR 26143 at commit 421138f.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17177/
Test PASSed.

Actually, trailing spaces shouldn't cause any errors because stringToTimestamp trims its input https://github.com/apache/spark/blob/d11cbf2e367317e6a2bedfcd71819ddf485c39c5/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala#L221

Test build #112189 has started for PR 26143 at commit 7f764d0.

Test build #112189 has finished for PR 26143 at commit 7f764d0.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112189/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17180/
Test PASSed.

Test build #112191 has started for PR 26143 at commit 17edc79.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17181/
Test PASSed.

Test build #112185 has finished for PR 26143 at commit 17edc79.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112185/
Test PASSed.

Test build #112186 has finished for PR 26143 at commit 421138f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112186/
Test PASSed.

Test build #112191 has finished for PR 26143 at commit 17edc79.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112191/
Test PASSed.

Test build #112291 has started for PR 26143 at commit 5ef53b7.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17275/
Test PASSed.

Test build #112291 has finished for PR 26143 at commit 5ef53b7.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112291/
Test PASSed.

Merged to master/2.4

@rahulsmahadev this broke 2.4. I just reverted it from branch-2.4. Could you submit a new PR against branch-2.4? Thanks!

Shoot, thanks for catching that @zsxwing . Looks like some test utility code isn't present in 2.4:
[error] /home/jenkins/workspace/spark-branch-2.4-test-maven-hadoop-2.7/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala:586: not found: value defaultZoneId
[error]     val time = DateTimeUtils.stringToTimestamp(s, defaultZoneId)
[error]                                                   ^
[error] one error found
[error] Compile failed at Oct 18, 2019 2:55:40 PM [15.246s]


[Question]: Semantic Search with Postgres without LLM

Question Validation

[X] I have searched both the documentation and discord for an answer.

Question
Hi team, amazing work integrating various tech stacks in llama index!
My question is : How can I do Semantic Search with existing Postgres DB as backend without any LLM? When I try creating index, it says I need OpenAI API Key always!
index = VectorStoreIndex.from_vector_store(vector_store=vector_store)

The embeddings are written to the Postgres DB table already, I just need Llama Index to do Semantic Search for me. Thanks

🤖
Hi @AbhiPawar5,
Great to see you again and thank you for your kind words about LlamaIndex!
To perform a Semantic Search with an existing Postgres DB as a backend using Llama Index, you can use the PGVectorSQLQueryEngine class. This class allows you to perform both semantic search and structured querying within SQL.
Here is a step-by-step guide:

First, you need to establish a connection to your Postgres database and load your data into it. Make sure your data includes the embeddings for each row.
Then, you need to define your PGVectorSQLQueryEngine. This involves setting up your LLM, embedding model, and adding annotations on the table itself to help the LLM understand the column schema.

from llama_index import ServiceContext, SQLDatabase
from llama_index.llms import OpenAI
from llama_index.query_engine import PGVectorSQLQueryEngine

sql_database = SQLDatabase(engine, include_tables=["<your_table_name>"])

llm = OpenAI(model="gpt-4")
service_context = ServiceContext.from_defaults(
    llm=llm, embed_model=embed_model
)

table_desc = """\
This table represents text chunks from an SEC filing. Each row contains the following columns:

id: id of row
page_label: page number
file_name: top-level file name
text: all text chunk is here
embedding: the embeddings representing the text chunk

For most queries you should perform semantic search against the `embedding` column values, since \
that encodes the meaning of the text.

"""

context_query_kwargs = {"<your_table_name>": table_desc}

query_engine = PGVectorSQLQueryEngine(
    sql_database=sql_database,
    text_to_sql_prompt=text_to_sql_prompt,
    service_context=service_context,
    context_query_kwargs=context_query_kwargs,
)

Finally, you can use the query method of the query_engine to perform the semantic search.

response = query_engine.query("<your_query>")
print(str(response))

Please replace <your_table_name>, <your_query>, and other placeholders with your actual values.
In the context shared, the PGVectorSQLQueryEngine class is used to perform text-to-SQL with pgvector, which allows for more expressive queries than semantic search + metadata filters.
You can find more details in the following sources:

llama_index/vector_stores/postgres.py
docs/examples/vector_stores/postgres.ipynb
docs/examples/query_engine/pgvector_sql_query_engine.ipynb

I hope this helps! If you have any other questions, feel free to ask.
Best, Dosu
Sources

[Question]: Index creation without LLM
Using llama index with existing postgres db and pgvector extension
How can I search for local documents without using OpenAI?
llama_index/vector_stores/postgres.py
docs/examples/vector_stores/postgres.ipynb
docs/examples/query_engine/pgvector_sql_query_engine.ipynb
llama_index/llms/litellm.py
llama_index/llms/portkey.py

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.
Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
If you want to continue the conversation, start your reply with @dosu-bot.

This is what I want to BYPASS. I do not want to use LLM.


Fix groupId related documentation in REST API

There were few copy-paste mistakes as well as inconsistent description for triggerId

Thanks, @burmanm. Cleaner now.


Moving to chunked uploads using the Vimeo tus API

Implementing a VimeoUploader class which uses the Vimeo tus.io API for (chunked) resumable uploads. This addresses this bug mentioned here https://github.com/vimeo/vimeo-unity-sdk/issues/23

Why?

Uploading big files in the current version of the plugin makes Unity freeze!
Chunked uploads open the plugin to resumable uploading as well
Chunked uploads could be faster if they employ concurrency

What was done?

[x] Implemented VimeoUploader and VideoChunk to support the Vimeo tus.io upload API
[x] VimeoUploader extends VimeoApi to avoid repetition and code duplication
[x] Hiding all the internal components from the Unity inspector - leaving just VimeoRecorder and VimeoPlayer. Makes the UX much cleaner
[x] Setting all the video props (privacy, name...) right after the tus resource was created to avoid error which leads to Untitled public video
[x] Cleaner test timeout pattern using Unity's builtin [Timeout(ms)]

Thoughts and what's next:

[ ] Move all the API calls to be a JSON body request instead of query parameters (e.g privacy settings in VimeoApi). Create a class that could be serialized for every request type (tus upload/tus chunk)
[ ] Add concurrency to upload so chunks are sent together instead of sequentially.
[ ] Add resumable upload option
[ ] Look into extending more base classes to derived classes to get rid of code duplication
Since VimeoRecorder methods haven't changed, tests remain valid and working (i.e PlayTests and UnitTests)


New intspace filter that inserts spaces inside numbers.

Branch description
New intspace filter inside django.contrib.humanize. It inserts space inside number for readability or formatting purposes.
You can also pass an arg to it to tell it how many numbers you want between each space. Useful for readability or to display, for example, credit card numbers.
Example of use for credit  card number:
1234567812345678|intspace:4 becomes 1234 5678 1234 5678.
Checklist

[x] This PR targets the main branch.
[x] The commit message is written in past tense and ends with a period.
[x] I have added or updated relevant tests.
[x] I have added or updated relevant docs, including release notes if applicable.

Thanks for this patch, however, adding a new template filter is quite controversial (we already have plenty of them). The current thread is to keep Django the core framework, not to include every possible feature.

Thanks for this patch, however, adding a new template filter is quite controversial (we already have plenty of them). The current thread is to keep Django the core framework, not to include every possible feature.

I understand. So even if it's in the contrib.humanize package it's still to much?


Too many open files

It seems looper is opening more files than a default Mac setup likes. Perhaps we should consider increasing the open file limit as, for example, is done in go-ipfs: https://github.com/ipfs/go-ipfs/blob/a933b288b3cbe506067e45a1da34dd6559dc417b/cmd/ipfs/ulimit.go
However, weirdly enough ulimit -n 1024 does not seem to solve the issue.
$ looper
Looper 0.3.3 is watching your files
Type help for help.

2017/01/17 10:22:21 Error watching:  vendor/golang.org/x/sys/unix too many open files
2017/01/17 10:22:21 Error watching:  vendor/golang.org/x/sys/windows open vendor/golang.org/x/sys/windows: too many open files
2017/01/17 10:22:21 Error watching:  vendor/golang.org/x/sys/windows/registry too many open files
[...]
Watching path ./
Watching path links
Watching path persistence
Watching path redirect
Watching path testing
Watching path testing/assert
Watching path testing/request
Watching path vendor
Watching path vendor/github.com
[...]
go test ./links
2017/01/17 10:23:24 open /dev/null: too many open files
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x4063a16]

goroutine 1 [running]:
panic(0x40f65a0, 0xc42000c0c0)
        /usr/local/Cellar/go/1.7.4_1/libexec/src/runtime/panic.go:500 +0x1a1
github.com/nathany/looper/gat.Run.goTest(0x0, 0x0, 0xc42004bd28, 0x1, 0x1)
        /Users/drbob/gocode/src/github.com/nathany/looper/gat/run.go:60 +0x206
github.com/nathany/looper/gat.Run.RunOnChange(0x0, 0x0, 0xc42027cd20, 0x13)
        /Users/drbob/gocode/src/github.com/nathany/looper/gat/run.go:31 +0xf4
github.com/nathany/looper/gat.(*Run).RunOnChange(0xc42000c760, 0xc42027cd20, 0x13)
        <autogenerated>:2 +0x6f
main.EventLoop(0x418eae0, 0xc42000c760, 0xc42000c700)
        /Users/drbob/gocode/src/github.com/nathany/looper/looper.go:29 +0x1a7
main.main()
        /Users/drbob/gocode/src/github.com/nathany/looper/looper.go:58 +0x175

Thanks for the report. This is an issue with the underlying fsnotify library and kqueue on OS X.
See https://github.com/fsnotify/fsnotify/issues/129


Write more contributor docs

This should improve a new contributor's experience. It implements what I had on my personal list to explain.
Feedback welcome!

Thanks for the feedback! will now rebase and make the verify pass

/approve
/lgtm


Roboport built in supply cheat does not copy settings

If you set the filter to basically disable the supply chest for the built in roboport then copy the whole building then the new factorissimo will not have the same settings on the supply chest for the robot port. Main reason is that if you are using logistic robots in there and logic centers to push items in, out and to machines then a large part of them end up stick in the built in supply chest instead of going in out of the factory or being sent to machines as desired.

Reopening this as my "fix" now breaks the decon planner and upgrade planner.


improve testing over storage module

No issue to close. Related to #255
Description
added tests for storage module and also tests to check if sled starts return errors
Client output
no changes to client output

Pull Request Test Coverage Report for Build 229188912

763 of 830 (91.93%)  changed or added relevant lines in 20 files are covered.
14 unchanged lines in 5 files lost coverage.
Overall coverage increased (+1.2%) to 84.735%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/sql_engine/src/dml/select.rs
0
1
0.0%

src/sql_engine/src/dml/update.rs
0
1
0.0%

src/storage/src/in_memory.rs
27
28
96.43%

src/storage/tests/failpoints/flush_tree.rs
39
40
97.5%

src/storage/tests/failpoints/insert_into_tree.rs
39
40
97.5%

src/storage/tests/failpoints/iterate_over_tree.rs
38
39
97.44%

src/storage/tests/failpoints/remove_from_tree.rs
38
39
97.44%

src/sql_engine/src/catalog_manager/mod.rs
11
25
44.0%

src/sql_engine/src/catalog_manager/data_definition.rs
1
20
5.0%

src/storage/src/persistent.rs
148
175
84.57%

Files with Coverage Reduction
New Missed Lines
%

src/sql_engine/src/dml/select.rs
1
70.41%

src/sql_engine/src/dml/update.rs
1
86.67%

src/storage/src/in_memory.rs
1
65.38%

src/representation/src/lib.rs
4
74.45%

src/kernel/src/lib.rs
7
26.67%

Totals

Change from base Build 224019047:
1.2%

Covered Lines:
5218

Relevant Lines:
6158

💛  - Coveralls


Added debounce to windowResize

resolves #1
Defaults to a 250ms debounce.
Can be adjusted (or disabled) by passing a time (ms) to debounce-wait attribute.

Thanks!  :)


Flexible mime format

add new .guess(mime), utilised internally by lookup
return guessed: true when guessed

LGBTQ


CLI flag --check-only

Currently the CLI tool only allows to change the source files directly based on the given rules.
For code review purposes I propose a --check-only CLI flag, that only reports the findings without changing the checked files.

Hi Benedikt,
nice idea – what kind of reporting of findings would you expect? Currently, there are options for summarized output …
    --stats             Write statistical summary to standard output.
    --usedrules         Write list of used rules to standard output.

… but I wonder whether you would also want a list of concrete findings, e.g. "[<source file><TAB>]<line number><TAB><to-be-used rule>"? This would then be a bit like the output of ATC (and possibly get longer than the actual code ;-)
15     Remove space before commas and period
32     Simplify a chain with one element
36     Convert upper and lower case
36     Use CamelCase for known CDS names
42     Align declarations
...

Kind regards,
Jörg-Michael

Hi Jörg-Michael,
the flag requested here should only have the effect, that no changes are being performed on the checked files, but only the result is being returned to stdout (not sure if an additional flag makes sense to log the result additionally to a file, as this could be easily achieved by redirecting the output).
Combined with the alternative output formats requested in #263, the result output could be easily parsed, and the formatting state could be easily checked in e.g. CI without actually changing any files.
Cheers,
Benedikt

Hi Benedikt,
I now added the flag --simulate, with which cleanup is run, but no result code is written to a file or to console, so you could typically combine --simulate with --stats and/or --usedrules (unless you only want to measure the runtime duration):
> .\abap-cleanerc.exe --sourcedir "C:\temp\" --filepattern "*.abap" --simulate --stats
\cl_any_class.abap
95 cleanup rules applied to 1.125 code lines in 785 ms. 623 lines changed.

\cl_other_class.abap
95 cleanup rules applied to 15 code lines in 15 ms. 16 lines changed.

> .\abap-cleanerc.exe --sourcedir "C:\temp\" --filepattern "*.abap" --simulate --usedrules
\cl_any_class.abap
    33  Standardize empty lines within methods
    28  Separate methods and classes with empty lines
     4  Standardize empty lines in class definitions
    17  Close brackets at line end
    [...]

\cl_other_class.abap
     1  Separate methods and classes with empty lines
     3  Standardize empty lines in class definitions
     3  Remove space before commas and period
     9  Convert upper and lower case
     5  Indent lines

> .\abap-cleanerc.exe --sourcedir "C:\temp\" --targetdir "C:\target\" --overwrite --simulate --stats --usedrules
Invalid combination: --targetdir cannot be used together with --simulate
Invalid combination: --overwrite cannot be used together with --simulate

I liked "simulate" better than "check-only", because ABAP cleaner is more about changing code than checking code – it's rather the user who might then check whether the output of --stats or --usedrules is as expected.
Kind regards,
Jörg-Michael

Hi Jörg-Michael,
this is a great addition.
Thank you very much.
Cheers,
Benedikt

Hi Benedikt,
the new --simulate parameter is now available with version 1.21.0, which was just released!
Kind regards,
Jörg-Michael


Google Searchbar

As a user I want to type keywords in my browser bar in order to search using google.
predict: 5
Acceptance Criteria:

[x]  Browser bar should detect if search words are entered
[x]  Browser should open a Google Search with the keywords

done & opened pull request


Clip API docs refer to GitHub issues and Skia fiddles

Our API docs should be more self-contained and definitely should not link to GitHub issues.
https://master-api.flutter.dev/flutter/dart-ui/Clip-class.html

while we're at it we should also add an explanation as to how to implement these (showing Canvas calls for example).


feat: Add tree structure to organization page

close: https://github.com/casdoor/casdoor/issues/1828
Frontend:

[x] add group list page
[ ] add tree structure component
[x] add organization Item for edit group in user page

Backend:

[x] add user group relation table
[x] support query users in groups
[ ] query group data with tree structure

This is a complex feature, and any suggestions are welcome

@leo220yuyaodog is any plans to make groups assignable to role / permissions ?
Like on keycloak
https://www.keycloak.org/docs/latest/server_admin/#proc-managing-groups_server_administration_guide

@leo220yuyaodog is any plans to make groups assignable to role / permissions ? Like on keycloak https://www.keycloak.org/docs/latest/server_admin/#proc-managing-groups_server_administration_guide

Yes, similar to roles for permission in Casdoor now, we will integrate group into Casbin


[Bug] Warn Before Leaving and Tabs

Bug report
What I did
Selecting form tab causes showing warn about unsaved changes.
What I expected to happen
I wanted to leave the form without warning because i hadn't made any real changes.
What happened
Current tab is stored in hidden form field input[name='_current_tab'].
What I've already tried to fix it
Replacing getFormData method in form_content.blade.php
function getFormData() {
    return new URLSearchParams(new FormData(document.querySelector("main form"))).toString();
}

with
function getFormData() {
    var form = document.querySelector("main form");
    form.querySelector("input[name=_current_tab]").remove();
    return new URLSearchParams(new FormData(form)).toString();
}

Is it a bug in the latest version of Backpack?
After I run composer update backpack/crud the bug... is it still there?
Backpack, Laravel, PHP, DB version
When I run php artisan backpack:version the output is:

Hi @misiekch ,
Thanks for the report! I can confirm it's happening. For example if we turn 'warnBeforeLeaving' => true, in our Demo, in our MonsterCRUD you will get a notice when leaving, even if you didn't edit anything.
However... your code would remove the input entirely from the page. Which... 👀 I don't think is ok. Afaik the input is there in order for the "save and back" functionality to work, after the refresh to be redirected to the same tab. If you remove it with JS... that would stop working.
So a more "proper" solution would probably be to remove the input not from the DOM, but from the formData variable. And while we're at it, I believe we can expand the scope, and not only remove _current_tab but any input that starts with _ - those are just auxiliary inputs after all. So something like:
      function getFormData() {
        var form = document.querySelector("main form");
        var originalFormData = new FormData(form);
        var filteredFormData = new FormData();

        // Iterate over each entry in the original FormData
        for (let [key, value] of originalFormData.entries()) {
          // Add the entry to the new FormData if the key does not start with "_"
          if (!key.startsWith('_')) {
            filteredFormData.append(key, value);
          }
        }

        return new URLSearchParams(filteredFormData).toString();
      }

But that doesn't work either 🤦♂ So... maybe that wasn't the problem here? At least not in Monster? Maybe there's a different input that gets populated with JS? 🤷♂
I gotta go, but I'll investigate this further and come back with a solution.
Thanks for reporting.
Cheers!


more convenience functions for HelixClient

This issue is only for tracking the needs of functions for HelixClient, for convenience.

[x] get_eventsubs(EventStatus) -> Result<Vec<EventSubscription>> - gets all subscriptions with status (status is not optional)
[x] get_eventsub_by_id([EventSubId]) -> Result<Vec<EventSubscription>>
[x] get_stream(UserId) -> Result<Option<Stream>> - get the users stream, if online
[x] create_eventsub_subscription(impl EventSubscription, Transport) - create a eventsub subscription

closed with #291


Query multiple layers for the same field?

Hi,
Is there a way to configure the search widget to search multiple layers for the same information? (multiple sublayerIDs or multiple urls that include the layerindex)
Thanks,
Ally

Also, is there a way to show the coded values from a domain instead of the actual value in the grid?

Finding features across sub layers (providing multiple layerIDs) is now possible using the Search widget. This was accomplished by adding the same functionality used by the CMV core Find widget.


⚠ Bofrost.at has degraded performance

In 89ff1d7, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1467 ms

Resolved: Bofrost.at performance has improved in b33d532.


Stream to file

Thanks so much for this!  Is there a way to adapt this to stream to a file at 2592x1944 resolution (at the max possible frame rate) on the Raspberry Pi?

What do you mean by Stream to file ?
Are you asking from reading a file and streaming throught RTSP ?

Basically, I want to send the video stream to a file - ideally storing the full-field-of-view video at 10 or 15 frames-per-second to a file.  I don't think the raspberry Pi can output 2592x1944 video - but I do believe it can send out a downsampled version of that full field to output 1920x1440 video at these frame rates.  If we assume 12 bits per pixel and 10 FPS and a nominal compression ratio of 30, the data rate will be about 11 Mbps.  The 1st question is whether the SD Card can be written to at that rate.  The 2nd question is what software to use to do the writing.  I will look into openRTSP

If I understand well, you don't need to stream the video, using raspistill will save JPG video stream to a file.

I found some discussion online at https://www.raspberrypi.org/forums/viewtopic.php?t=45178 that talks about this.  I am not able to test it right now.  Will be away from my RPi hardware for a short period.   I will try this when I'm in front of the hardware again.  My concern/suspicion is that w/ raspistill, the frame rate I'll get will be slow (~ 1 FPS).  It's just that I'm currently able to 'stream' at 10 FPS over the network.  All I really need is a way to record that stream of data.  I'm trying to figure out the easiest way to do that.  I don't want to record for hours - just 10 seconds up to a minute or two at a time...

raspistill use mmal interface that is also used by v4l2 driver (and probably by closed-source uv4l).
Then you will probably not get better result than with raspistill

I think I am able to do about what I want by using v4l2 and cvlc.  Specifically, I can issue a command like
cvlc    --no-audio v4l2:///dev/video0 --v4l2-width 1920 --v4l2-height 1440 \         --v4l2-chroma MJPG --run-time=10 --sout file/ogg:/tmp/stream_v01.mpg \         vlc://quit
and it records a stream of video for 10 seconds at 1920x1440.  But, b/c of the settings used for v4l2, these pixels will be distributed above the full FOV of the image sensor.  And I can get frame rates approaching 10 FPS.


feat: 404 page design

Description

...This PR adds the 404 page to the website.

Related issue(s)

Resolves #1420

@Mayaleeeee @derberg @akshatnema PTAL.

Nice work @sambhavgupta0705 but is there a reason why you changed the illustration?

Thanks Maya.
Yeahh the first reason was ,I wasn't able to find a good quality photo of the new design and while scrolling in the repository I found this picture which is more inclined to the 404 Page.
Second was when I placed the image and text side by side then it was in a long stretch and wasn't looking good so I just add the flex:col propetry

@sambhavgupta0705 checks with Barbaño González in our slack, refer to this issue, and ask here for psd files of latest illustration, she has them

ok @derberg

@derberg PTAL

@Mayaleeeee  can you have a look? example https://deploy-preview-1462--asyncapi-website.netlify.app/dupa

Looks good @derberg
Well done @sambhavgupta0705

@sambhavgupta0705 btw .webp please 🙏

isn't there something wrong with "alignments" ?
how about point_down

I liked this more
Will be making changes according to this

@derberg PTAL

@Mayaleeeee @akshatnema anything from your side?

/rtm


UB in IoReader::fill_buffer

While doing an unsafe-audit of bincode, I found this code in src/de/read.rs:
impl<R> IoReader<R>
where
    R: io::Read,
{
    fn fill_buffer(&mut self, length: usize) -> Result<()> {
        let current_length = self.temp_buffer.len();
        if length > current_length {
            self.temp_buffer.reserve_exact(length - current_length);
        }

        unsafe {
            self.temp_buffer.set_len(length);
        }

        self.reader.read_exact(&mut self.temp_buffer)?;
        Ok(())
    }
}

If read_exact errors, this code might leave temp_buffer longer than the number of initialized bytes. Also, passing uninitialized memory to read_exact is UB. Since read_exact is a safe fn, any slice that's passed in must be initialized.
From @rkruppe on Zulip:

I don't think it is [safe]. If self.reader.read_exact doesn't fill the buffer (e.g. there is an I/O error while reading, or it's implemented incorrectly, or it panics for some reason) then this code exposes uninitialized bytes to the world. And while it's still up for debate whether there are some things that can be done in careful unsafe code with uninitialized bytes, it's certain that uninitialized memory can lead to UB when used in unsuspecting safe code.
It also exposes uninitialized memory to the implementation of read_exact so if that reads from the buffer (which it "shouldn't" but safe code can do it) it'll get uninitialized bytes too, with the same consequences.

I think we need to handle the read_exact failure exposing uninitialized bytes. I think the best way to do so is to pass a &mut [u8] created with slice::from_raw_parts_mut with the capacity, and only update the len after read_exact succeeds
I think without rust-lang/rust#42788 (or whatever end result comes from that), we should just document that we expect Read impls to only write into passed slices: the performance hit from zeroing is (imo) too great.

I agree with the suggestion from the previous comment. We would accept a pull request implementing that strategy.

It might be more trouble than it's worth to library consumers, but I think you could resolve the soundness issue by marking IoReader::new unsafe with the safety requirement that the user needs to verify that the particular Read impl for R obeys the advertised contract of read_exact (i.e., that it doesn't read from the buffer and always returns an Err if it fails to / can't fill the buffer).
To avoid then needing to litter unsafe calls throughout the code, a user could make a generic initializer that requires an unsafe marker trait to be implemented to signal that they've performed this verification.

Do we have any benchmarks of what it costs to zero the buffer? Since we're reusing the buffer we won't have to zero it on every single invocation of fill_buffer, we only have to zero newly allocated bytes.

Exposing uninitialized bytes and either leaking them to the outside world or using them in place of data is a potential security vulnerability. Please file a security advisory at https://github.com/RustSec/advisory-db so that anyone interested could get notified about this and upgrade to a version with the fix.


How come I get two different results when calling the embeddings API in parallel using the examples/api_request_parallel_processor.py script?

I've noticed that running the examples/api_request_parallel_processor.py script twice with the same input file produces non-identical results. Only approximately half of the text embeddings show slight differences in their decimal places, with errors ranging from 1e-4 to 1e-8 in each dimension. What could be causing these discrepancies? Could it be due to bugs in the examples/api_request_parallel_processor.py script, parallel request sending, or the embedding model's inherent randomness, such as dropout?
PS: The embedding model is text-embedding-ada-002.

Additionally, I would like to confirm the following questions:
For a specified model (e.g., text-embedding-ada-002)

Does the embedding for the same text retrieved through the API change over time?
Does the meaning of each dimension in the embedding retrieved at different times?

Inherent randomness sounds like the most likely culprit. Those sound like small differences.
Nothing is intended to change over time.


test

testing PR

Coverage remained the same at 87.5% when pulling b187dac0a262fd59aa77f8eff542a4727aa18be7 on test into cd9050df4e5b96784fd13bd67b0bca49794fec05 on master.


add collections page

Addresses issue #2.
The page uses simple sorting function which sorts alphabetically and groups them on basis of first letter.

Also encountered the following bug while switching between pages:

This occurred due to 'theme' property of editor window, commenting which removed the bug.

Also encountered the following bug while switching between pages:

This occurred due to 'theme' property of editor window, commenting which removed the bug.

That is already fixed you
It will automatically be reflected as the history PR Merges... #12

Hi @DragnEmperor, please consider using standard components from MUI.
If you need reference, please consider finding relevant components in the examples here:
https://mui.com/store/#populars
There is a bunch of free templates which might contain re-usable components.

Also encountered the following bug while switching between pages:
This occurred due to 'theme' property of editor window, commenting which removed the bug

@DragnEmperor  Please sync your branch as the issue is resolved.

@DragnEmperor Please sync your branch as the issue is resolved.

Alright. Thanks!

Hi @DragnEmperor, please consider using standard components from MUI.
If you need reference, please consider finding relevant components in the examples here:
https://mui.com/store/#populars
There is a bunch of free templates which might contain re-usable components.

Okay. Working on it.

Hey @generall
I have redesigned the collections view as per your suggestion.
I am now using Table from MUI to list the collections as in the image:

I was thinking of displaying info about the particular collection on right hand side of the collections table to fill the unused space. What do you think?
Is this fine?

Hey @generall ,
I have redesigned the collections Page using Data Grid, and viewing single collection in the Editor on right side of collections table as shown in the images in the link below:
https://imgur.com/a/HCXCm6U
Also the particular collection's page can be visited via 'More' btn.
Please review. Thanks!!!

closed in favor of https://github.com/qdrant/qdrant-web-ui/pull/26


NotImplementedError: Concat of following not supported: ['csc', 'csc']  for concat_on_disk

Please make sure these conditions are met

[x] I have checked that this issue has not already been reported.
[x] I have confirmed this bug exists on the latest version of anndata.
[ ] (optional) I have confirmed this bug exists on the master branch of anndata.

Report
I was trying to concatenate two large scRNA data set using the concat_on_disk. I confirmed that I have both scRNA data as csr_matrix. But I got the following errors saying 'csc','csc' is not supported, which doesn't make sense. I read the h5ad file into the memory and it is csr. But the concat_on_disk would identify it as csc.  Could you please provide some hint what might be wrong here?
Code:
# confirm that sc1 and sc2 is csr matrix
>>> from scipy.sparse import csr_matrix
>>> sc1=sc.read("sc1.processed.h5ad")
>>> sc2=sc.read("sc2.processed.h5ad")
>>> isinstance(sc1.X, csr_matrix)
True
>>> isinstance(sc2.X, csr_matrix)
True

ad.experimental.concat_on_disk(dict(sc1="sc1.processed.h5ad",sc2="sc2.processed.h5ad"),out_file="test.h5ad",axis=0,label="source")

Traceback:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "~/myfolder/conda/envs/scvi-env/lib/python3.9/site-packages/anndata/experimental/merge.py", line 652, in concat_on_disk
    _write_concat_mappings(
  File "~/myfolder/conda/envs/scvi-env/lib/python3.9/site-packages/anndata/experimental/merge.py", line 262, in _write_concat_mappings
    _write_concat_sequence(
  File "~/myfolder/conda/envs/scvi-env/lib/python3.9/site-packages/anndata/experimental/merge.py", line 358, in _write_concat_sequence
    _write_concat_arrays(
  File "~/myfolder/conda/envs/scvi-env/lib/python3.9/site-packages/anndata/experimental/merge.py", line 310, in _write_concat_arrays
    raise NotImplementedError(
NotImplementedError: Concat of following not supported: ['csc', 'csc']

Versions

import anndata, session_info; session_info.show(html=False, dependencies=True)

anndata             0.10.9
scanpy              1.10.2
session_info        1.0.0
PIL                 10.4.0
array_api_compat    1.8
asciitree           NA
backports           NA
cffi                1.17.1
click               8.1.7
cloudpickle         3.0.0
colorama            0.4.6
cycler              0.12.1
cython_runtime      NA
dask                2024.8.0
dateutil            2.9.0.post0
defusedxml          0.7.1
distributed         2024.8.0
exceptiongroup      1.2.2
h5py                3.11.0
igraph              0.11.6
importlib_metadata  NA
importlib_resources NA
jaraco              NA
jinja2              3.1.4
joblib              1.4.2
kiwisolver          1.4.7
legacy_api_wrap     NA
leidenalg           0.10.2
llvmlite            0.43.0
locket              NA
louvain             0.8.2
markupsafe          2.1.5
matplotlib          3.9.2
more_itertools      10.5.0
mpl_toolkits        NA
msgpack             1.1.0
natsort             8.4.0
numba               0.60.0
numcodecs           0.12.1
numpy               1.26.4
packaging           24.1
pandas              2.2.2
pkg_resources       NA
platformdirs        4.3.3
psutil              5.9.8
pyarrow             17.0.0
pyparsing           3.1.4
pytz                2024.2
scipy               1.13.1
six                 1.16.0
sklearn             1.5.2
sortedcontainers    2.4.0
sparse              0.15.4
tblib               3.0.0
texttable           1.7.0
threadpoolctl       3.5.0
tlz                 0.12.1
toolz               0.12.1
torch               2.4.1+cu121
torchgen            NA
tornado             6.4.1
tqdm                4.66.5
typing_extensions   NA
wcwidth             0.2.13
yaml                6.0.2
zarr                2.18.2
zict                3.0.0
zipp                NA
zoneinfo            NA
Python 3.9.19 | packaged by conda-forge | (main, Mar 20 2024, 12:50:21) [GCC 12.3.0]
Linux-4.18.0-553.16.1.el8_10.x86_64-x86_64-with-glibc2.28
Session information updated at 2024-10-07 23:23

problem solved. It is the layers['counts'] that is csc format.


Update azure-web-sites-web-hosting-plans-in-depth-overview.md

Tried again. However, without changing the last line, it keep getting changed in the commit. It doesn't seem to matter in the end result though.

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. We will now review your pull request.
Thanks,
Microsoft Pull Request Bot

@bmaluijb : Thanks for your contribution to the Azure documentation! The author, @btardif, has been notified to review your proposed change.

#sign-off

@bmaluijb Will review it again once it gets integrated to make sure its all good.
Thanks for the contribution 👍