NixOS/auto-upgrade: add gzip to service path

Motivation for this change
Backport of #67587 to nixos-19.09
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[x] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Hmm, I'd think if they used fetchGit the service would need to git as well.


Remove GroupedDropdownField::validate() (regression)

Regression caused by 41ea83b,
which introduced field validation in the parent class without
also adapting this subclass.
Fixing the immediate issue (can't save this field type),
as a first step to the actual fix (implementing validate).
See silverstripe/silverstripe-translatable#179

Ah crap, sorry about this one. We should write a few more tests for the these fields :laughing:

I've merged and added #3923 to ensure this @todo doesn't get lost in the aether.


can't find acme-viewer

I run this case:  /p/cscratch/acme/mccoy20/test_2017-09-14-Qi . ncclimo and AMWG were completed. I am trying to find the plots on the web. In ./output/workflow.log I can see
'host_url': 'https://acme-viewer.llnl.gov/mccoy20/20170628.POP_MG2.A_WCYCL1850.ne30
_g16.edison/amwg/0041-0045',
but when I try to open it in a browser, the file is 'Not Found'.
I had run this case last Tuesday, restarted it today after updating, to see if the  acme-diags would get calculated, so far it says 'JobStatus.PENDING', same for coupled_diags.

It looks like your image host directory wasnt created. Im not sure why this would happen but not give you an error.

Fixed in the new nightly build.


Documente shared application services

Adds and fleshes out jsdocs for the requests and services utils
Follow up to #554
Developer's Certificate of Origin 1.1
   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

just force pushed my changes after a rebase and included a small amend for Kevin's comment.

Thanks for the updates @lresende  this LGTM


Rename and Migrate ngrok2 to Caskroom

There was already a Cask in the Caskroom, so I just added a rename from ngrok2 to ngrok and the migration to Caskroom.

Related to this: https://github.com/caskroom/homebrew-cask/issues/23938

Closing in favour of https://github.com/Homebrew/homebrew-binary/pull/359 because the rename won't work quite how you expect here, unfortunately. Thanks anyway for the PR, though!


Screen Marker Warning Behavior

When "Client Always on Top" is active, the screen marker warning gets hidden behind the client. Once the warning is open, you cannot move the client out of the way to close the warning. Only way I've fixed this is to just hit Enter to confirm the warning.
Ideally the warning should be forced to show in front of the client no matter what so it can always be clicked.

Duplicate #2690


Add option to disable truecaller listener once done

TRUECALLER.on(TRUECALLER_EVENT.TrueProfileResponse, profile => { //callback executed })
Once the Truecaller listener is added using the above code, in the componentDidMount, there are no methods available to remove this event listener. There after in the entire app session until exiting, whenever the app state goes to background and returns, this callback function gets called.
I guess there should be some method like TRUECALLER.removeListener(TRUECALLER_EVENT.TrueProfileResponse) to remove this listener, so we can invoke that in componentWillUnmount or whenever the verification with truecaller is done.

@HishamMubarak  did you find some way to fix this ?

@HishamMubarak did you find some way to fix this ?

Nope, haven't  yet figured out a way to solve this. Ignored it for now. Will need to add a method to remove the listener and submit and PR

Hi,
Any update on the remove listener PR?


Fixes #4

Not sure about possible side effects, so feel free to reject. =)

You're right, that method is the correct one to call here. It would seem that the process is faulty only when doing row inserts - at least I could not get the bug appear while updating an existing item. But it seems that we still need to fix inserting complex data types (tags) that use multiple Row instances. I'm adding a fix for both of these.


Doesn't Trade

Idk, but is it supposed to print out an output every time it trades? IDK, but it isn't putting out any output

not trades also !!


fix: remove forget password

yes but i thought it looked like a bug since the link did nothing... hm we can close this for now


Add docker.Fetch fuzzer

Adds a fuzzer that targets the request and open APIs in remotes/docker.Fetch.
This fuzzer can run continuously through OSS-fuzz, and I will add it on the OSS-fuzz side once this PR has been merged.

Try putting this in contrib/fuzz

@containerd/committers Can someone take a look and merge?

/ok-to-test

@AdamKorcz is it possible to put the setup-script in containerd contrib side

Yes. The build script has been added here. Once merged, I will instruct OSS-fuzz to run that script.

/retest

/retest

@AdamKorcz I re-run the cases. The /retest won't help here :(

@AdamKorcz I re-run the cases. The /retest won't help here :(

Ah, thanks for the heads up!

CI is unhappy again :( ....

@fuweid Error looks unrelated to this PR. Shall we run tests again?

@AdamKorcz Yes. But would you mind to squash the commits into one, please? Thanks!

would you mind to squash the commits into one, please?

Done


Courses summary spike [WIP]

Context
A spike implementation of https://manage-courses-prototype.herokuapp.com/history/status-logic-before-transition

In https://manage-courses-prototype.herokuapp.com/quiz:
We also have "Opens on [date]" for applications.

looks about right to me, from data point of view

Looks good to me. It would be nice to clearly define the v1 and v2 serialisers but the logic is good and very clear 😍

Looks good to me. It would be nice to clearly define the v1 and v2 serialisers but the logic is good and very clear 😍

We could move the existing v1 serialisers into a UCAS namespace (since the representation is quite UCAS-specific). That way we could reclaim the default serializer names.

We could move the existing v1 serialisers into a UCAS namespace (since the representation is quite UCAS-specific). That way we could reclaim the default serializer names.

Great idea @benilovj. I'm onboard with that

 Create api/v2 for manage courses frontend

In https://manage-courses-prototype.herokuapp.com/quiz:
We also have "Opens on [date]" for applications.

Covered in this commit
https://github.com/DFE-Digital/manage-courses-backend/pull/93/commits/7b82d80ea6b501b7b3c5dd14502503310b94d972
Basically Applications is a ui concern of which backend needs to provide enough to determine what to exactly show ie Open|Closed|Date|Vacancy

Closing, will reopen with better branch name and cleaner PR.

Reopened as #118.


TreeWalker

TreeWalker implementation for native,
The wpt/dom/traversal/TreeWalker unittest seems to be broken, it passes all other test.
I hope this helps.

Thanks for the PR. A few things:
The codebase style needs to be honored, so

only strict equality (===) except for == null to match null or undefined
always end lines with semi-colons
only double-quotes for strings
always 2 space indents
trailing commas
Typescript enum instead of abstract classes
spaces after control flow block keywords i.e. if (...) and while (...) instead of if(...), etc

I really need to setup deno fmt or smth already :sweat:
There are also parts that touch unrelated parts of the codebase; please submit a separate PR for those

I did you the honor.

There are also parts that touch unrelated parts of the codebase; please submit a separate PR for those

Any idea how to outsource part of the commits to a separate branch?
Also, how did you run the wpt tests? I had to comment out the argument check since deno doesnt allow arguments when testing.

Alright, thanks. Everything mostly looks fine, but I'll check it in more detail later today or tomorrow.

Any idea how to outsource part of the commits to a separate branch?

Also, how did you run the wpt tests? I had to comment out the argument check since deno doesnt allow arguments when testing.

It's in the "Running tests" section of the readme:

Then append -- --wpt to the test command before running it, e.g. for WASM
deno test --allow-read wasm.test.ts -- --wpt

Btw, to revert a full file you can use
git checkout 813ce27 ./path/to/file.ts

@chances unfortunately, no. The contributor themselves has to give me access to modify the PR. I'll probably close this and implement it myself at some point.

@b-fuze Access granted


Allow search by any header in email, add search to Queue & Held

What point of X-Postal-MsgID if you can't search by it? 😊
I think postal can easy search by any header[key=value] but it not support this 😢

I've always hoped some search improvements were possible but haven't had an opportunity to see if the database would or would not help in its current state.


Minor content updates to Home page, case studies, and partners

Update case studies and homepage
Adds additional case studies, swaps video and banner on home page, and
changes the link under ebay case study to Read more from Read the story."
Fixes misspelling on Partners page, changing "There partners to "These partners"

This change is

I signed it!


Tracks doubled on v1.2 beta update

I had a user report to me that they had all their tracks duplicate when updating to v1.2 Beta 1.

This could have maybe been related to me not setting the APS Environment to "production" before pushing to TestFlight?


Error! After Installation of AspNetCore and dotnet core the IIS Handler Mappings - aspNetCore is not installed

Error! After Installation of AspNetCore and dotnet core the IIS Handler Mappings - aspNetCore is not installed
Windows 10 Pro
AspNetCore.2.0.5.RuntimePackageStore_x64
dotnet-runtime-2.0.5-win-x64
I also tried AspNetCore.2.0.6.RuntimePackageStore_x64
dotnet-runtime-2.0.6-win-x64

Worked on another Windows 10 computer, but had the same issue. I installed, repaired, uninstalled, rebooted.
There were other similar posts on this back in December 2017. The issue was the installer did not correctly register the aspNetCore dll.
Thank you
Dennis Aubrey

When it works correctly. See above.

Note: The following is NOT installed on the Computer.
Microsoft .NET Core 1.0.4 Runtime (x64)
Microsoft .NET Core 1.0.4 & 1.1.1 - Windows Server Hosting
Microsoft .NET Core 1.1.1 - Runtime (x64)
Microsoft .NET Core Runtime (x64)

Note: Important. This Windows 10 Computer works.. See installed Packages.

Where to download
AspNetCore v1.1.1 and AspNetCore.Hosting.WindowsServices v1.1.2

https://docs.microsoft.com/en-us/dynamics-nav/troubleshooting-http-error-500.21-internal-server-error-aspnetcore-handler

If it is not installed, download and install Microsoft .NET Core - Windows Server Hosting bundle.

Installed

Web site is now working.

Hi @dennisaubrey - I'm not sure I quite follow the posts above. Do you think there's a bug in the installer? Or somewhere? If so, can you provide more details? Thanks!

Hi Thank you,
After installing the AspNetCore and Donet - My Web API on IIS would not run and displayed an general Error.
It was not clear that I must install "Server Hosting Install" for AspNetCore on IIS.
Through debugging, on two computers, I came to the conclusion that the only difference between the two Windows 10 Computers, one worked and the other did not,  is one had the Server Hosting and the other did not. It was also not clear on where to download the Server Hosting Install from reading the Windows docs on installation requirements.
I installed the Server Hosting Install and everything is now working.
"Error! After Installation of AspNetCore and dotnet core the IIS Handler Mappings - aspNetCore is not installed"
The Error Messaged displayed while running the My Web Site with AspNetCore - should have displayed that the AspNetCore Hosting  was not installed on IIS. I did install AspNetCore.

@dennisaubrey thanks for the details.
@shirhatti - is there something we can do to make our installation instructions clearer in this space?

Note: Error Message on My Web Site. Before I started.

Closing this issue as I've created a docs bug to track what @dennisaubrey complained about


How to typecheck composite typescript projects?

I tried this and it seems like project references are ignored. I guess tsc is not run in --build mode?
https://www.typescriptlang.org/docs/handbook/project-references.html

Seems like tsc -p--noEmit is run here. In order to support typescript composite projects (project references) it must be possible to specify --build flag. This also means it needs to be possible to remove --noEmit flag becuase this is not allowed in combination with --build. Which also means that it will produce output but that is not a problem for me at least.

Just found #65 so closing in favor of that.


Run build on all branches

When we were using TravisCI this was the behavior, then when we switched to GH we dropped it. Was this intentional? I noticed this while updating the supplementary material of the histolab paper

When we were using TravisCI this was the behavior, then when we switched to GH we dropped it. Was this intentional? I noticed this while updating the supplementary material of the histolab paper

It's intentional cause we do not create ad-hoc branched where we wanna run the CI, the only way to create branch is via a PR and on the PR we have the rule on the workflow file

Alright, I will close this PR


Upload is broken in Edge

If you click the upload button in Edge, you can never close the file explorer. If you click cancel, or the close x or select a file, the file explorer is forced open again. I couldn't find a way out besides closing the browser from the task manager. This can be reproduced from the demo https://aberezkin.github.io/ng2-image-upload/#/demo

Works fine for me on Microsoft Edge 40.15063.0.0
Please post your version.
Can anyone else reproduce this?

I'm running 38.14393.1066.0
A colleague of mine on the same version had the same problem.

I can't find a download link for legacy Edge (if there even is one), if you can find one I'll gladly try to reproduce the problem on your version, if not I recommend you try Edge 40 and see if the issue remains.


module is packaged with .pem files

Not sure if this is the right place, but I am using browserify in a chrome extension that I think uses this module somewhere down the chain of dependencies. The presence of .pem files (even though they are just tests) is causing problems, because chrome does not want to pack an extension that has .pem files (in case you accidentally distribute your real keys).
This is largely chrome's fault for being so restrictive and providing no mechanism that I know of to override the .pem file detection, but it would be very helpful if you could somehow get rid of the .pem files or rename them to something else. Thanks!

@zhaizhai ha, that is quite interesting!
@dominictarr thoughts? Haha.
@calvinmetcalf I guess we could just white list in package.json.
"files": [
  "browser.js",
  ...
]

However,   maybe renaming the files will be just as effective and maintains a status quo?

Hmm, if you bundle this with browserify then you won't need to keep node_modules at all.
Am I missing something?

We can just put *. pem in a .npmignore
On Sat, Jun 6, 2015, 5:32 AM Dominic Tarr<EMAIL_ADDRESS>wrote:

Hmm, if you bundle this with browserify then you won't need to keep
node_modules at all.
Am I missing something?
—
Reply to this email directly or view it on GitHub
https://github.com/crypto-browserify/publicEncrypt/issues/7#issuecomment-109557649
.

first we should understand why this is causing @zhaizhai's problem

@dominictarr Good point, you're right that I don't need to package node_modules in this case. So never mind, it doesn't matter for me. Thanks for responding so quickly!

:+1:

@zhaizhai how were you packaging node_modules in the first place?

I know this is a little late, but I was hitting this problem today. I am developing a chrome extension that depends on browserify. Chrome throws the following errors when trying to unpack the extension:
This extension includes the key file '/app/node_modules/browserify/node_modules/crypto-browserify/node_modules/public-encrypt/test/test_key.pem'. You probably don't want to do that.
This extension includes the key file '/app/node_modules/browserify/node_modules/crypto-browserify/node_modules/public-encrypt/test/test_rsa_privkey.pem'. You probably don't want to do that.
This extension includes the key file 'app/node_modules/browserify/node_modules/crypto-browserify/node_modules/public-encrypt/test/test_rsa_pubkey.pem'. You probably don't want to do that.
I removed the files manually and all was well. However, the next time I run npm install I will hit the same issue. I was thinking maybe adding the test directory to .npmignore would do the trick.

@RobbyChapman if you are using browserify,  why is the node_modules in the application directory at all?

@dcousens I'm not sure I understand the question. If you are building a chrome extension the entire root directory is the application. Moving node modules wouldn't solve that.

@RobbyChapman the contents of node_modules especially node_modules/browserify is a development dependency that should not actually be packed into the chrome extension, though as you are the 2nd one to make this mistake it probably wouldn't hurt to put the .pem files into the npm ignore

@dcousens  I see what you are saying. I rearrange the app based on your suggestion and all is well. I do agree with the .pem fix though. Thanks for looking into it!

@RobbyChapman we could move it,  but this sounds like a great way to help people realise you shouldn't pack node_modules, especially since there is no reason to just ignore the PEM files otherwise.
If we want to not include the tests folder,  that is another issue.

@dcousens I'm developing a Chrome extension. I know I shouldn't pack node_modules, but when I'm developing the app I usually have manifest.json, /src and node_modules all on top level, and load the extension from the project root. When I'm publishing I only copy over the built files and manifest.json into a publish folder.
I'll send another PR that just npm-ignores the test/ folder. Hope you can publish a new version. Thanks!


im a new lua coder so these answers might be obvious, pardon me

im making my ui and i have 2 questions. How do i make it so that when i click a button, it opens the link?
eg. local MyButton = LeftGroupBox:AddButton({
Text = 'Join Discord Server',
Func = function()
if setclipboard then
setclipboard("")
end
end,
DoubleClick = false
Question 2 is how do i make a button that has a function of closing the gui/script entirely? ik these answers might be obvious but im very new to lua coding/scripting

please help

you cant with current executors, but you can try using Vynixius's Inviter utility:

local Inviter = loadstring(game:HttpGet("https://raw.githubusercontent.com/RegularVynixu/Utilities/main/Discord%20Inviter/Source.lua"))()

-- Creates an interactable prompt
Inviter.Prompt({
    name = "your server name",
    invite = "your invite",
})

----- ALTERNATIVELY -----
Inviter.Join("your invite")

Check Example.lua, its not our fault you didnt look at the end where it shows how to unload.

and last but not least, dont skid (i know your skidding from poopdoors modified and bobhub)


get_account_history may terminate early due to using first op as sentinel value

Calling get_account_history on the account which owns get_object 1.11.0 may cause the iteration to stop early because of a confusion between two cases:

Case 1:  There is no next operation_history_object, we have reached the end of the list
Case 2:  The next operation_history_object exists and has ID zero

The simple fix for new chains is to create a dummy sentinel operation_history_object with ID zero.  However, this fix will not work on already-deployed chains because it will change history ID's.
A deployable fix would append 1.11.0 to its result when:

The account for which the operation history is being requested is the account for 1.11.0
Appending it would not exceed the request limits

This issue was moved to bitshares/bitshares-core#168


provide ppc64le release binaries

Title: provide ppc64le release binaries
Description:
I'm hoping to figure out a way we can get binaries for ppc64le built and hosted by the envoy community. I've been building binaries myself for the past year or so, to be consumed by Power users of istio. Now that it seems that all the bazel + luajit issue are ironed out, it would be great if we could agree on a way to have this be automated.
We could provide the community with a public VM via either OSU or Unicamp (in Brazil). If someone is familiar with using Circle with an outside system (and if one could be another architecture), this could also be hooked into your build process.
I'm also open to other ideas.

@clnperez we would be happy to post GitHub events to an external build hook if you want to run builds on master pushes.

Not stale, we just gained access to the test VM for this issue today so we should be able to fulfill this issue soon.
So far we have tests and builds passing on 18.04 with clang/gcc. The only remaining issue is getting a working bazel-0.22.0 binary.

We have had successful builds in our internal system since March 13th. Now we just need to determine how to get remote builds working for Circle on the external build machine we have setup.

Good news, i just received access to the Oregon State Power9 Jenkins service. This IBM partner provides a similar service for Tensorflow & Docker.
I'm tweaking the job now and will let it run for a few days before requesting the build hook integration.

Ok I have master building successfully on ppc64le machines with Ubuntu 16.04. I still have to test with newer versions of gcc/clang but so far so good. I'm also working with the OSU lab to integrate the docker custom build environment plugin. This should replicate the custom build container with all deps preinstalled process that we use with CircleCI.
You can view the pipeline here.


Update mono-mdk to <IP_ADDRESS>

After making all changes to the cask:

[x] brew cask audit --download {{cask_file}} is error-free.
[x] brew cask style --fix {{cask_file}} left no offenses.
[x] The commit message includes the cask’s name and version.

Actually the version now appears to be stable now and fully noted as part of the path. I'm going to update this with checksum


Check unavailable from async in interface file

We need to check the feature availability of _unavailableFromAsync
before it gets picked up in the swift interface file. This updates the
compiler to provide the necessary wrappings for that check.
This should fix the issue where older compilers complain when they import interfaces from the new compiler that have _unavailableFromAsync.

@swift-ci please test


Refactor push update

This PR refactor the push update logic to make it more clear and easier to change. Replace #10426

Codecov Report

Merging #13381 into master will decrease coverage by 0.04%.
The diff coverage is 61.84%.

@@            Coverage Diff             @@
##           master   #13381      +/-   ##
==========================================
- Coverage   42.14%   42.10%   -0.05%
==========================================
  Files         691      691
  Lines       75966    75967       +1
==========================================
- Hits        32019    31984      -35
- Misses      38703    38734      +31
- Partials     5244     5249       +5

Impacted Files
Coverage Δ

services/repository/push.go
44.64% <44.00%> (-13.39%)
:arrow_down:

modules/notification/action/action.go
60.49% <70.58%> (+2.23%)
:arrow_up:

modules/indexer/stats/db.go
52.17% <0.00%> (-8.70%)
:arrow_down:

modules/repository/push.go
47.61% <0.00%> (-4.77%)
:arrow_down:

modules/charset/charset.go
68.53% <0.00%> (-4.50%)
:arrow_down:

services/pull/temp_repo.go
26.59% <0.00%> (-3.20%)
:arrow_down:

services/pull/check.go
48.90% <0.00%> (-2.92%)
:arrow_down:

models/unit.go
46.57% <0.00%> (-2.74%)
:arrow_down:

modules/repository/commits.go
84.09% <0.00%> (-2.28%)
:arrow_down:

services/pull/patch.go
53.97% <0.00%> (-1.71%)
:arrow_down:

... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f070249...b08add8. Read the comment docs.

@lafriks done.


Added support for extracting Node information - to be used with Docke…

…r swarm

Great Peter!
Released it as version 0.6.0.


[ETL] Incremental ETL Pipeline

Fixes #1001
@kdetry ... after our conversation I took the time to organize this draft such that you have a good foundation to take it in the right direction. I'm concerned that although "your solution seems simpler" (to copy the table and then do onEventHandlers), it will lead to a less efficient and burdensome pipeline...
Like I have mentioned, my goal is to lean further into DuckDB/SQL such that all logic is done in SQL, through more atomic operations, such where everything is only processed once.
I have provided 2 examples in here.
[Example 1 - Required ETL/SQL]
In here, I have simplified things such that we only need predictions and payouts.
Trueval events do not contain the UserID, so it will not do anything useful until the slot table arrives.
This is the solution I have provided the following DoD, although it's a pseudo implementation.
The overall architecture/approach however remains the same.
pdr_backend/lake/required_etl.py
pdr_backend/lake/required_sql_bronze_predictions.py
pdr_backend/lake/required_sql_payouts.py
pdr_backend/lake/required_sql_predictions.py

Changes proposed in this pseudo-code:

[x] 1 event processed can now generate multiple rows on multiple tables
[x] _temp tables hold new events
[x] _update tables hold update events
[x] _update_raw tables is designed to hold N update events per ID
[x] because we're not analyzing update events, reduce multiple update events into 1

update logic generates 5 prediction events for slot ID
reduce step groups by ID, and turns 5 rows into a single (1) update record, before joining with millions

[x] merge -> drop -> merge records in such a way that updates production tables

[Example 2 - Broader ETL/SQL]
The other SQL queries, were the ones I originally wrote and I have now paused.
pdr_backend/lake/sql_etl_bronze_predictions.py
pdr_backend/lake/sql_etl_payouts.py
pdr_backend/lake/sql_etl_predictions.py
pdr_backend/lake/sql_etl_truevals.py

These are much more extensive, and show you how the system grows to also include updates to the slots table, and much more, without causing additional reading/copying/processing.
I have done this to pseudo code/show/provide direction, as to how this approach continues to scale and work with the same implementation as required_etl.py
This is not required for now... only required_*.py.
After this PR completes - Continue furthering:

Renaming, and reducing table types & operations => Reduce everything to "EventsTable" Temp & Prod
TempTable -> becomes NewTable (INSERT)
UpdateTable -> becomes UpdateTable (UPDATE)

NewTable + UpdateTable -> become EventsTable (INSERT + UPDATE Records - 1 per row)
-> These can then be collapsed/reduced with a group operation.
-> These can then be merged 1:1 w/ the prod table
-> In the future, we may want to store the EventsTable for faster development workflows around the Lake/ETL
Perhaps introduce:
-> eventName (predictoorSlot, predictoorPrediction, predictoorTrueval)
-> eventType (new vs. update)

Most of my queries need to use table.fullname rather than table.table_name and it's not intuitive. I.E. The most obvious interface name should likely be the one I use the most. This also matches up to the base/class oop nature of the code base.

[x] Rename NamedTable/Table.table_name -> base_table_name.
[x] Rename NamedTable/Table.table_name -> table_name

DoD:

[x] Re-write the queries such that we're processing each contract/subgraph event only once (subscribe, slot, prediction, trueval, payout) into New + Update events.
[x] Re-write queries such that every Contract Event, yields the right New + Update records
[x] Re-write ETL such that New + Update records are going into their respective tables (TempTable + UpdateTable)
[x] Re-write ETL such that Update records are reduced into a single row (I.E. N SlotUpdate records -> 1 SlotUpdate record) before being joined historically
[x] Re-write ETL such that only 1:1 Update:Prod record is executed, reducing historical join complexity.
[x] Re-write ETL such that as new events show up, existing records are updated.
[x] Clean up logs and output so we can follow what's happening from inside the CLI. Right now there are a lot of poor logging (like every query being sent to subgraph) that is just spam and adds no value since things have evolved quite a bit.
[x] Disabled prediction subgraph data from being used in the ETL pipeline, such that enforce that the ETL is correctly enriching historical data as we would expect
[X] Started to abstract conftest and  test functionality to more easily build a multi-stage test
[x] Export large set of pdr_predictions and pdr_payouts
[x] Setup mock/fixture so we can easily retrieve larger data samples for processing
[x] Create test_first_step_workflow() to verify that the first work updates with itself, doesn't look for historical records, and all incoming null payouts/values w/ matching payout events, are updated successfully
[x] Implement a ppss clamp in ETL so checkpoint calculations are clamped to ppss.st_timestamp, ppss.fin_timestamp
[x] Create test_incremental_workflow() to verify the fisrt-batch updates successfully, and that multi-stage scenarios are working
[x] Debug and fix remaining issues with incremental pipeline
[x] Fix start/end conditional checks that are causing 1-2 rows to not be populated/processed correctly
[x] Rewrite swap strategy so that each table is processed atomically
[x] Swap old ETL for incremental ETL and remove any old functionality that is no longer used
[x] make sure everything is wired up to the new system
[x] start running the pipeline via cli and make sure nothing breaks...
[x] fix duplicate issue where by simply incrementing the pipeline, we would see a duplicate record
[x] Cleanup table types, table object, and interface
[x] Drop any tests that no longer apply
[x] Update ETL tests that are valid to use new fixture
[x] fix duplicate issue where you are incrementing the pipeline, and then sett a new fin_ts < last_ts of the last record and a bunch of duplicates are created.
[x] add lake validation for checking if all valid payouts from raw that should be in bronze_predictions, are in bronze_predictions

Prepare Mege

[x] merge latest from main
[ ] review code
[ ] simplify logic, address DRY, and cleanup code wherever appropriate before merging
[ ] merge PR

Configured ETL for hard-mode such that there is no data coming from subgraph and we need to rebuild all state through ETL

Some of the predictions should remain null (because we don't have those payouts yet)
Some of the payouts cannot find a prediction (because it happened in the past, so it can't be joined)

Cleaned CLI output to not spam (i.e. we don't need to log every query, and every result)... It's now compact, easier to read, and I'm able to understand full e2e ETL.

Generated CSV files w/ extended sample data so we can more easily configure a thorough ETL test, that includes incremental validations. This test data is configured to be as empty as possible, such that we can verify that all state is being built correctly on-the-fly.

Starting to built out an incremental test w/ a much larger dataset, such that i can more easily replicate and resolve issues. As an example my current configuration has 2058 predictions, but I only see 2057 in my bronze_predictions table.

Updates regarding ETL sample data and coverage:

New ETL fixture _sample_etl
Sample data and fixture has roughly 2000 predictions and payouts ~= 1mb
datetime ranges (from,to) ("2024-05-05_00:00", "2024-05-05_02:00")
e2e tests are now setup and running

Test 1 => 1 ETL step
Step 1 - ("2024-05-05_00:00", "2024-05-05_02:00")
Test 2 => 3 ETL steps
Step 1 - ("2024-05-05_00:00", "2024-05-05_00:40")
Step 2 - ("2024-05-05_00:40", "2024-05-05_01:20")
Step 3 - ("2024-05-05_01:20", "2024-05-05_02:00")
Next:

I'm now debugging Step 1 -> Step 2 and fixing issues in the incremental step

I have now completed the 2 final features that were required and the latest is pushed to the PR

[x] Fix start/end conditional checks that are causing 1-2 rows to not be populated/processed correctly
[x] Rewrite swap strategy so that each table is processed atomically

I'm now going to:

[ ] start removing old functionality
[ ] make sure everything is wired up to the new system
[ ] start running the pipeline via cli and make sure nothing breaks

There are a lot of changes in this PR and a lot of them are outside the scope and makes reviewing very hard to follow.
Could you only pick the changes related to this PR?

ETL completes successfully and results in the describe command are looking good, but testing or inspecting the updates on live tables it's currently difficult, maybe something could be added to the describe command to show the number or rows containing null values or something else that could make inspecting easier.
Another improvement to validate updates could be checking if there is a predictions with payout but still containing null values. In other words all predictions that have a payouts associated should not have null values

I'm getting some issues here. This is how I'm testing:

running etl update to get the latest events
get latest payout event id and check if predictions with the same id has null values
i expect predictions that have payout event to not have null values, but they do both in db and CSVs

but testing or inspecting the updates on live tables it's currently difficult, maybe something could be added to the describe command to show the number or rows containing null values or something else that could make inspecting easier.

Another improvement to validate updates could be checking if there is a predictions with payout but still containing null values. In other words all predictions that have a payouts associated should not have null values

these two comments, and
i expect predictions that have payout event to not have null values, but they do both in db and CSVs
Looks to mainly appear on 1h feeds

led me and norbert to talk, i have added a validation that makes sure that from all predictions that are there, all valid matches have been updated and have payout != null.
although these vaidations might be redundant with tests, they are very useful operationally this is something that I have to keep telling myself.

i have added a validation that makes sure that from all predictions that are there, all valid matches have been updated and have payout != null.

Was this change pushed? I only see validation on each payout having a matching prediction, bot not on predictions being updated

"I only see validation on each payout having a matching prediction, bot not on predictions being updated"
@KatunaNorbert

The tests provide deep coverage over thousands of records showing that predictions are updated
The validations show that all payouts that came in, have been matched up with bronze predictions and there are no NULL values. This means that they have been updated correctly.

very difficult to review for me as well. Can we cherry-pick? It currently shows 440 files changed, which is because of licenseheader updates in main.

@calina-c @KatunaNorbert you can change the selected commit from the dropdown when reviewing the PR, such that you're only looking at the changes I made

if cherry-picking is the way you want to move forward, then so be it


[develop] Sending 60 reliable RPCs per second breaks the system but does not in Master

Version Number and Operating System(s):
Windows 10 64-bit
Unity 2018.4 / 2019.2 https://github.com/BeardedManStudios/ForgeNetworkingRemastered/commit/b69d15bf34598e2f498c70bf735cc11ec0aa6543
Actual behavior:
Attempting to send 1 reliable RPC per FixedUpdate causes the system to overflow and no further RPCs can be sent
Steps to reproduce:
Clone this fork: https://github.com/Cerzi/ForgeNetworkingRemastered/tree/reliableRPCexample
Build the game, Run two instances.  Note the values in the top right of the screen for the CLIENT
Every fixedupdate the client attempts to a single reliable RPC.  Around 10 or 15 are sent successfully before they stop being received.
Load CubeForgeGame scene in UnityEditor and toggle "reliable" on the RPC Tester component to off.  Re-join and note that the RPCs are successfully received.
Yes sending 60 reliable RPCs a second is probably a bad idea, but the fact that this used to work mostly but now causes catastrophic failure suggests a bug somewhere in recent refactored code.
[Optional] Discord Username:
Cerzi

@BrentFarris would you be able to have a look please?

Had the exact same problem, like @cerzi said, it's a bad idea to send reliable but it's working fine on master

I'll take a look at this issue.

@Crazy8ball would you be able to have a look at this please?


Cannot Detect Blackmagic Video Monitoring Card for AV Output

Apple Feedback Assistant ID: FB13454447
DESCRIBE THE BUG:
Since the last release of Ventura OS, Final Cut has been unable to detect any Blackmagic video cards as an AV output option.
Interestingly, AV output is successful using Adobe Premiere, DaVinci Resolve, and Blackmagic Media Express.
This has been tested with the following:

Clean install of latest Sonoma OS
Newly created partition with only required software installed for testing
Latest Final Cut
Latest Black Magic Desktop Video, as well as all prior releases up to 12.1
Blackmagic Decklink Mini Monitor 4K, UltraStudio 4K Mini, Decklink 8K Pro

TO REPRODUCE:

Use M1 or above Mac with latest OS
Use latest Final Cut
Install Blackmagic Desktop Video software & follow instructions for Mac security settings
Install Blackmagic Decklink or use Blackmagic UltraStudio 4K Mini with external monitor
Try to select AV card and enable AV output from within Final Cut

EXPECTED BEHAVIOUR:
Generally, once a user installs Blackmagic Desktop Video software and sets up appropriate preferences, the option to select a Blackmagic video output card within Final Cut is available. After enabling AV output, the user is then able to successfully monitor video from a "clean" feed on an external monitor for color grading purposes. Currently, this no longer is an option as Final Cut does not detect any Blackmagic card.

SPECS:

M1 16" MacBook Pro / M2 Ultra Mac Studio
macOS Sonoma 14.1
Final Cut Pro 10.7

Thanks @latenitefilms , I haven't. First time seeing this. I'll definitely try it tonight. Curious though, why output still works with Premiere, DaVinci, and Blackmagic Media Express. Will reply once tested!

@latenitefilms You solved it! I have been trying for nearly 2-months now to figure this out. Neither Blackmagic, nor Final Cut support could provide a solution. You're awesome, thank you so much!!!

Clearly you don't read my Upgrade Guide, haha: https://fcp.cafe/update-guide/
Great news!


Temperature at OFF instead of system at STOP

I think there is an improvement to be made when we turn off all the zones.
Currently when we turn off the last zone with Homekit it switches the whole system to STOP. I think it would make more sense to just turn off the zone (as with the previous zones) and leave the system on its current state.
I think this would be easier and more flexible.
Advantage this would allow more flexibility between Homekit and thermostats. For example, if you wanted to go through the thermostat to turn a zone back on, you would simply press the On/Off button. At the moment as the system goes into STOP mode you have to change the mode (e.g. cold) after pressing on/off and choosing the temperature.
I am at your disposal if you want more explanations.
Stephen

Hello Stephen,
Thanks for your advice. I thought about this approach but the problem is that a thermostat has system mode and temperature but not on/off is for this reason that I chose the way to do it with the all zones mode.
Additionally, if I split it, a thermostat for each device and one for each zone I need a service with on/off and temperature and I don't know which one I can use. What service type you think is good for each zone?
Regards.

Hi,
Thank you for your quick reply.
I am not sure I understand everything (I am not a specialist :-) )
However I noticed that there was a difference in treatment if we have several active zones or only one when we turn off the zone.
I give you an example.
Let's imagine that I have 2 zones ON (for example living room and bedroom). If I turn OFF the bedroom zone from Homekit => The physical thermostat of the bedroom switches OFF (also on the AirZone application) and the living room zone continues to work normally. As long as I don't turn off the living room zone from HomeKit I can turn ON the bedroom zone by pressing the ON/OFF button on the physical thermostat of the bedroom.
If I then turn OFF the living room zone (which is the last one running) the whole AirZone system goes to STOP mode (which was not the case when I turned OFF only the bedroom zone).
I think you understood but I prefer to explain it to you again in more detail.
I also looked at the link you sent me and apparently there is the possibility to set a thermostat to OFF via the Thermostat service.

I must have missed something :-)
Regards.

Hello @vemeo,
I tried to add a device for the system and on it control the mode and use the current for the zone but HomeKit doesn't have the way to implement a thermostat only with on/off.
If you want, I could add an option in the config to disable the switch off automatically when all zones are off, and with this, the user could choose if use it or not.
Regards.

Hi @fjhorrillo,
That would be great ! Thanks for taking my comment into account.
Regards

+1 to this :)
Would be nice that instead of making system to change to STOP mode, it keeps the last mode used, like it happens when using the physical  controller

Done in v0.3.6. The default value for "auto off" is true, to avoid this behavior uncheck the Auto Off option under Advance configuration on the configuration.

Done in v0.3.6. The default value for "auto off" is true, to avoid this behavior uncheck the Auto Off option under Advance configuration on the configuration.


Fix creation of new DNS_ROUND_ROBIN entries

Fixes https://github.com/istio/istio/issues/49315
Broken in https://github.com/istio/istio/pull/41831

@howardjohn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

lint_istio
47527eab7c6de0b5df38aeec2d9dc7db645508ea
link
true
/test lint

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test doc.test.profile-default

@ericvn: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test analyze-tests
/test bookinfo-build
/test gencheck
/test integ-ambient
/test integ-basic-arm64
/test integ-cni
/test integ-distroless
/test integ-ds
/test integ-helm
/test integ-ipv6
/test integ-operator-controller
/test integ-pilot
/test integ-pilot-istiodremote
/test integ-pilot-istiodremote-mc
/test integ-pilot-multicluster
/test integ-security
/test integ-security-istiodremote
/test integ-security-multicluster
/test integ-telemetry
/test integ-telemetry-discovery
/test integ-telemetry-istiodremote
/test integ-telemetry-mc
/test lint
/test release-notes
/test release-test
/test unit-tests
/test unit-tests-arm64

The following commands are available to trigger optional jobs:

/test benchmark
/test experimental-tracing
/test integ-assertion

Use /test all to run the following jobs that were automatically triggered:

analyze-tests_istio
gencheck_istio
integ-ambient_istio
integ-basic-arm64_istio
integ-cni_istio
integ-distroless_istio
integ-ds_istio
integ-helm_istio
integ-ipv6_istio
integ-operator-controller_istio
integ-pilot-istiodremote-mc_istio
integ-pilot-istiodremote_istio
integ-pilot-multicluster_istio
integ-pilot_istio
integ-security-istiodremote_istio
integ-security-multicluster_istio
integ-security_istio
integ-telemetry-discovery_istio
integ-telemetry-istiodremote_istio
integ-telemetry-mc_istio
integ-telemetry_istio
lint_istio
release-notes_istio
release-test_istio
unit-tests-arm64_istio
unit-tests_istio

In response to this:

/test doc.test.profile-default

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/test lint

@howardjohn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

lint_istio
47527eab7c6de0b5df38aeec2d9dc7db645508ea
link
true
/test lint

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

lint_istio
47527eab7c6de0b5df38aeec2d9dc7db645508ea
link
true
/test lint

integ-ds_istio
961d6b9e18f82b26b005fe8e40e711ba63812480
link
true
/test integ-ds

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retst

/retest

In response to a cherrypick label: new pull request created: #49395

In response to a cherrypick label: new pull request created: #49396

In response to a cherrypick label: new pull request created: #49397

In response to a cherrypick label: new pull request could not be created: failed to create pull request against istio/istio#release-1.20 from head istio-testing:cherry-pick-49331-to-release-1.20: status code 422 not one of [201], body: {"message":"Validation Failed","errors":[{"resource":"PullRequest","code":"custom","message":"A pull request already exists for istio-testing:cherry-pick-49331-to-release-1.20."}],"documentation_url":"https://docs.github.com/rest/pulls/pulls#create-a-pull-request"}


Buffer is not defined, Platform React JS

Steps to reproduce.

run command npm i fluentreports
then simply add following data in your file's componentDidMount if using Class Component or useEffect if using Functional Component

Sample Data
 const data = [
        { name: "Elijah", age: 18 },
        { name: "Abraham", age: 22 },
        { name: "Gavin", age: 28 },
      ];
      const rpt = new Report("grocery1.pdf")
        .data(data) // Add our Data
        .pageHeader(["My Grocery List"]) // Add a simple header
        .detail("{{count}} {{unit}} of {{item}}") // Put how we want to print out the data line.
        .render();

I already tried window.buffer and webpack handling given below

Adding plugins section with ProviderPlugin into webpack.config.js:
module.exports = {
// ...
plugins: [
// Work around for Buffer is undefined:
// https://github.com/webpack/changelog-v5/issues/10
new webpack.ProvidePlugin({
Buffer: ['buffer', 'Buffer'],
}),
new webpack.ProvidePlugin({
process: 'process/browser',
}),
]

Also add in resolve.fallback into webpack.config.js :
resolve: {
extensions: [ '.ts', '.js' ],
fallback: {
"stream": require.resolve("stream-browserify"),
"buffer": require.resolve("buffer")
}
}

Hi did you follow the instructions here https://github.com/NathanaelA/fluentreports/tree/master/generator
Specifically creating the browser version of the code base:
browserify fluentReportsBuilder.js -s fluentReports --ignore iconv-lite -o ../generator/fluentReportsBrowser.js
If you are doing server only, then Node provides "Buffer"

Hi did you follow the instructions here https://github.com/NathanaelA/fluentreports/tree/master/generator
Specifically creating the browser version of the code base: browserify fluentReportsBuilder.js -s fluentReports --ignore iconv-lite -o ../generator/fluentReportsBrowser.js
If you are doing server only, then Node provides "Buffer"

Yeah I tried it again and it works..
but its showing me the GUI .. i want to use code base functionality but i cant figure it out how to use it .. could you please help me out

The fluentReportsBrowser.js is the code based version of it for the browser.
The fluentReportsGenerator.js is used to have a report builder inside the browser.
The fluentReportsBuilder.js is used to run/generate any of the report builder reports.


Create LICENSE

Hello! I'd like to use this plugin in my place of employment and a license is required. I've added the MIT license, as that seems to be what most Flow Launcher plugins use. Feel free to choose another from choosealicense.com if you'd like.
If you'd like to make small tweaks to the MIT license, feel free to edit my PR.
Thanks!

Sure MIT looks right for me!


add page "How to use Unruggable App" for dedicated docs

Hello there,
I wanted to let you know that I have created a tutorial on how to deploy memtoken using Unruggable for "How to use Unruggable App" and submitted a pull request to the main branch, but I am not entirely sure if I chose the correct one.
I hope this helps.

Hi @0xChqrles, just make changes as you said. (it's not identical but follows the structure mentioned)


Re-enable TypeScript tests

name: "Re-enable TypeScript tests"
about: Testing is important
PR Type
Does this PR add a new feature, or fix a bug?
Internal change.
Why should this PR be included?
See https://github.com/reduxjs/redux/issues/3500#issuecomment-542363238 and following. I tried out tsd, but I did not like it as much. The expect() method syntax made it more cumbersome to assert in some scenarios. It also wasn't very well documented. I had to read the source code to figure out how to use it. So, I think sticking with typings-tester is preferable for now.
Note, I also needed to add an import in types\store.ts in order to get the global Symbol.observable definition working in the type outputs.
Checklist

[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Is there an existing issue for this PR?

No, this is an internal change.

[x] Have the files been linted and formatted?
[x] Have the docs been updated to match the changes in the PR?
[x] Have the tests been updated to match the changes in the PR?
[x] Have you run the tests locally to confirm they pass?

Bug Fixes
What is the current behavior, and the steps to reproduce the issue?
Types can break.
What is the expected behavior?
The ability to test types.
How does this PR fix the problem?
Re-enables types testing.

Ah, we had somehow missed this. Thanks for catching and fixing it!


Add badges to README

Badges should include:

conda (version)
pypi (version)
tests (passing or failing – needs #103 )

Needs input:
@musslick which badges do we want / need? Some are implemented already (see https://github.com/AutoResearch/autora)

@hollandjg
Thanks for checking. I would add

[ ] a license badge (soon to be changed to MIT license).

If it is also possible to add hyperlinks to these badges, then I would add:

[ ] a custom badge [Doc | autoresearch.github.io] with link to the documentation
[ ] a custom badge [Forum | Github Discussions] with link to Github Discussions


prompt is too long

Describe the bug
often get this error prompt is too long: 200515 tokens > 200000 maximum
i have purchased an upgraded plan after this message as well as more tokens. started new conversations multiple times and logged out and back in again.
maybe the project has become too large? or some time needs to pass by?
Link to the Bolt URL that caused the error
https://bolt.new/~/sb1-htyehy
Steps to reproduce
entering any prompt at all at this time
Expected behavior
i expected the prompt to go through as ive upgraded my account a few times.
Screen Recording / Screenshot
No response
Platform
Browser name = Chrome
Full version = <IP_ADDRESS>
Major version = 130
navigator.appName = Netscape
navigator.userAgent = Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36
performance.memory = {
  "totalJSHeapSize": 166704844,
  "usedJSHeapSize": 153159992,
  "jsHeapSizeLimit":<PHONE_NUMBER>
}
Username = singh336
Chat ID = 35d5a901f1e4

Additional context
No response

i realized the project got too big. i deleted some uneeded backups for now and its working fine


Codebox URL too picky

Codebox is found on localhost:8010/ide/ but not localhost:8010/ide.

codebox is discontinued as planemo-machine editor of choice as they stopped developing it


Reduce array size by encoding sequences.

Reduce output size about 30 times by encoding number sequences into an array!
The resulting arrays are reconstructed during execution.
The encoded array is composed from sequence of [starting number, number of incremental values in a sequence].

Coverage remained the same at 96.32% when pulling a4bd164cdebd889769f30ddae379af83a6a8dee4 on gloryknight:patch-1 into a8cfae6537193d28f63d5dfae42f20c5cae2c6f6 on jshint:master.

It is too much time consuming  to sign the Contributor License Agreement. Just take my word that I stick to the conditions.

It is too much time consuming to sign the Contributor License Agreement. Just take my word that I stick to the conditions.

That doesn't work.

Now CLA is signed

style is adjusted.


Use pyi for better support type checking of Array

Description

use stubs file to told that Array is expected to be one of ArrayLike, np.ndarray, zarr.Array or dask.array.Array
Type of change

[x] Bug-fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

References

#1713
How has this been tested?

[ ] example: the test suite for my feature covers cases x, y, and z
[ ] example: all tests pass with my change

Final checklist:

[x] My PR is the minimum possible work for the desired functionality
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] If I included new strings, I have used trans. to make them localizable.
For more information see our translations guide.

@tlambert03 see this

can you clarify a bit your intention here?  are you wanting better type checking when you provide an object to Layer(data=...)?  are you wanting better hinting when you grab layer.data?
the pyi file was a good solution for #1713, because the underlying functions were themselves dynamically generated.  Here a pyi file is less convenient, since the code is not autogenerated, and the pyi can go out of sync.  Also, the Union here isn't strictly accurate, since our array support is more of a protocol than a strict union of types.  We are aware that typing around accepted input data is vague at the moment, but this is just a generally hard problem.
the Array class that we use is mostly for internal usage in declaring our models (so that we can assert shapes of things).  it's not intended to be a general type hint for external users.  Can you provide a bit more context on exactly what you'd like to see in your IDE?

Without this change:

With this change:

So You could see that without this change PyCharm (Which is a popular IDE) mark simple code as the wrong usage which could confuse non expected user.
Add a type annotation told tools that this overload is only for pytantic and data do not need to be Array subclass instance.

I do understand the implications of annotation something with Array ... my question to you is more about where in the napari codebase you're trying to get IDE hinting.  i.e. not your "sample_fun".  See my original question:

can you clarify a bit your intention here? are you wanting better type checking when you provide an object to Layer(data=...)? are you wanting better hinting when you grab layer.data?

I prefer to have both. After think more it looks like best is to have defined two types. One for function argument and second one for function return. But there is also problem with pydantic class which may ned third class with type annotation (to have validator but not park assignment of np.ndarray as wrong one.

ok, now that I'm understanding a bit better... yes: we definitely all want better type hinting for arrays in napari (and more generally across the whole python ecosystem).  This is basically the same thing as https://github.com/napari/napari/pull/1175.
There are a number of things that make this hard.  first, as mentioned, we need to define protocols, not Unions.  This PEP (https://www.python.org/dev/peps/pep-0646/)  coming in 3.10 will also eventually help so that we can declare a bit more about the shapes of those array protocols.  Most of the time we've just fallen back to using Any, because for now we'd rather be too vague than start creating overly strict type hints that aren't really correct.
If you did want to start working on layer.data... custom_types.Array is not the place to do it.  the stubs generated for add_<image> and view_<layer> might be a way to inject a little more meaning, but we'd need to go through the code and track all the array APIs we use (i.e. .shape, .data, make a union with lists for images, etc...).  This also touches on the LayerDataProtocol that I tried to introduce in #2683.
In general, very high value, important stuff.  but non-trivial.


Added basic test for setsid

This should be a nondet test case. On lind, it'll return 1 since runs in a cage, but it will return random values (much higher than 1) in native, which will fail as a dettest.


pyav install error on Ubuntu 18

Overview
This error message on running pip3 install av. I have tried multiple methods on this issues forum and reinstalled FFmpeg to no avail, unfortunately.
ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-r5x8zuf_/av/setup.py'"'"'; __file__='"'"'/tmp/pip-install-r5x8zuf_/av/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-qw2cfq33/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.6/av Check the logs for full command output.
Expected behavior
Install to finish successfully.
Actual behavior
Too long to post here, but posted into a gist.
Build report:
Have linked to the error message here.
Investigation
Tried to completely purge previous pyav and FFmpeg and reinstalled.
Reproduction
{{ Steps to reproduce the behavior. }}
Versions

OS: Ubuntu 18.04
PyAV runtime:

- FFmpeg:

ffmpeg version 4.2.2-1ubuntu1 18.04.york0 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)

## Research

I have done the following:

- [ x] Checked the [PyAV documentation](http://docs.mikeboers.com/pyav/stable/)
- [ ] Searched on [Google](https://www.google.com/search?q=pyav+how+do+I+foo)
- [ ] Searched on [Stack Overflow](https://stackoverflow.com/search?q=pyav)
- [x ] Looked through [old GitHub issues](https://github.com/mikeboers/PyAV/issues?&q=is%3Aissue)
- [ ] Asked on [PyAV Gitter](https://gitter.im/mikeboers/PyAV)
- [ ] ... and waited 72 hours for a response.

## Additional context

{{ Add any other context about the problem here. }}

Please provider a more complete log, there is nothing usable here

@jlaine updated gist with output from pip3 install -v av. Is that what you meant by more complete log?

Sorry I hadn't seen the linked gist. AV_CODEC_CAP_HARDWARE was introduced many years ago, so I can only surmise the FFmpeg libraries you're building against are too old.
What version is stated in /usr/local/include/libavcodec/version.h ?
If it's truly FFmpeg 4.2.2 you should see:
#define LIBAVCODEC_VERSION_MAJOR  58
#define LIBAVCODEC_VERSION_MINOR  54
#define LIBAVCODEC_VERSION_MICRO 100

cat /usr/local/include/libavcodec/version.h gives me this:
#define LIBAVCODEC_VERSION_MAJOR  57
#define LIBAVCODEC_VERSION_MINOR  64
#define LIBAVCODEC_VERSION_MICRO 100

and
ffmpeg --version
ffmpeg version 4.2.2-1ubuntu1~18.04.york0 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)

So it appears to be FFmpeg 4.2.2

You seem to have an odd mix of ffmpeg versions on your computer:

the command line ffmpeg tool is indeed 4.2.2
the headers / libraries being picked in (from /usr/local) on the other hand are not. libavcodec 57 means you are actually building against FFmpeg 3.2.x which is the source of your error.

Find out why you have this old FFmpeg, remove it and that should fix your error.


Does it has streamReader of option to read and process really huge file?

Hi,
I want to read very huge file, and process data in parts, Does it has this option?

Hi @Himkar,
It seems that the module that is used under the hood doesn't support that. Take a look at this part of their docs: https://github.com/SheetJS/js-xlsx#streaming-read


[Feature] Migrate to pyproject.toml

resolves #
docs dbt-labs/docs.getdbt.com/#
Problem
dbt-snowflake currently uses setup.py and a collection of independent configs to build and test. This has been replaced by pyproject.toml as the go-forward config file for python projects. Additionally, there are configs that live in multiple files and differ from each other, leading to inconsistent usage of dev and test tools.
Solution
Config will be migrated as much as possible into pyproject.toml. There are still some projects that haven't migrated to PEP-518, namely pre-commit, flake8, and tox. These will need to live in legacy formats for now. tox and flake8 can only read ini, so they will live in setup.cfg. pre-commit can only read yaml, so it will remain in place, in .pre-commit-config.yaml.
Checklist

[x] I have read the contributing guide and understand what's expected of me
[ ] I have run this code in development and it appears to resolve the stated issue
[ ] This PR includes tests, or tests are not required/relevant for this PR
[x] This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX

I got too far ahead on this PR. This will be broken down into smaller PRs, such as #835.


single_modality pip install raise AssertionError: Unable to pre-compile ops without torch installed. Please install torch before attempting to pre-compile ops.

cd single_modality

pip install -r requirements.txt

It raised AssertionError: Unable to pre-compile ops without torch installed. Please install torch before attempting to pre-compile ops.

Thanks for your question. You can change the order in requirements.txt and install torch firstly.

Thank you for your job!
I find a solution like this:
pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113

references Previous PyTorch Versions


Rerun hack/vendor.sh to pick up gpgme changes

See https://github.com/proglottis/gpgme/pull/7 for the full discussion.

FYI https://ci.centos.org/job/atomic-rdgo-centos7/401/artifact/build-logs/build-logs/failed/skopeo/build.log

Oops, thanks for pointing this out. Filed #47 to make sure I stop doing these silly mistakes.


Design: Ability to backup different types of volume in the same namespace

https://issues.redhat.com/browse/OADP-1071

/lifecycle frozen

/remove-lifecycle frozen

/remove-lifecycle stale

Closed via https://github.com/vmware-tanzu/velero/pull/6956


Update LLM frontend docs to cover unconfigured status

Our readme here doesn't yet cover how to handle the different LLM app config options, including the new "unconfigured" status added in https://github.com/grafana/grafana-llm-app/pull/350 .
The general UX approach we'd like to model is:

activate LLM features if they're enabled (either Grafana-provided or other config)
hide LLM features from the user if they are disabled / opted-out
prompt the user that an admin can opt in to LLM-powered features in the LLM plugin if status is unconfigured.

Once our readme has a trivial example, I can start nudging different squads to use this approach (plus get some UX input on how they should do it).

Some example responses that may be useful here:
Plugin disabled:
{
  "message": "Plugin not found"
}

Plugin enabled but not configured:
{
  "details": {
    "openAI": {
      "configured": false,
      "error": "No functioning models are available",
      "models": {
        "base": {
          "error": "OpenAI not configured",
          "ok": false
        },
        "large": {
          "error": "OpenAI not configured",
          "ok": false
        }
      },
      "ok": false
    },
    "vector": {
      "enabled": false,
      "ok": false
    },
    "version": "0.10.4"
  },
  "message": "",
  "status": "OK"
}

Plugin enabled but "Disable all LLM features in Grafana" chosen (i.e. opted-out):
{
  "details": {
    "openAI": {
      "configured": true,
      "error": "LLM functionality is disabled",
      "models": null,
      "ok": false
    },
    "vector": {
      "enabled": false,
      "ok": false
    },
    "version": "0.10.4"
  },
  "message": "",
  "status": "OK"
}

The @grafana/llm has two relevant functions:

health, which more or less returns the openAI object in the above response
enabled, which I think most people are using, which calls health and checks that configured and ok are both true.

It sounds like we want to encourage people to use health and use the value of configured to prompt the user to opt-in?


Write code to access LDP/MetaDB PostgresQL database

Should transparently support both dialects, as mod-ldp does.

I can't do this until #27 is seen to be working, and that's blocked on DevOps.


DOC Announce EOF for SS4.5

Update 4.6.0 changelog to announce EOF of SS4.5

I suggest we close this PR. EOL announcements are handled in the Silverstripe Forum

Let us close this PR and never speak of it again.


Alamofire 4.0 Beta

Thanks for putting this together @esetnik! This PR has been replaced with #166. The majority of the changes are the same, but we reworked the error handling significantly. Thanks again.
Cheers. 🍻


RPC Error

Hi Craig, some reason the node-status page I'm running for my altcoin has stopped working. I believe it may have occured after the recent php update to version 5.5.9 including curl as well.
currently when i run >  curl -Ssk http://cryptostakers.com/node/stats.php  > i get the error >
173.245.xxx.xxx is not in the whitelist (however this ip changes due to the server being behind cloudflare)
so i'm not sure if the php script is trying to do rpc call from outside the server? it shouldn't be doing this however, since it wasn't doing so before.
and if i add the ip to the whitelist, and run the same curl, i get "rpc error".
If you could assist with this, please and thank you.

The stats script should be run from the server itself, not from any other source, and should be run against localhost and not the server's hostname.

root@CryptoCube:~# curl -Ssk http://localhost/node/stats.php
http://localhost/node/stats.php

404 Not Found

Not Found
The requested URL /node/stats.php was not found on this server.

Apache/2.4.7 (Ubuntu) Server at localhost Port 80

root@CryptoCube:~# curl -Ssk http://<IP_ADDRESS>/node/stats.php
http://<IP_ADDRESS>/node/stats.php

404 Not Found

Not Found
The requested URL /node/stats.php was not found on this server.

Apache/2.4.7 (Ubuntu) Server at <IP_ADDRESS> Port 80

root@CryptoCube:~#
On Mon, Jul 4, 2016 at 11:21 AM, Craig Watson<EMAIL_ADDRESS>wrote:

The stats script should be run from the server itself, not from any other
source, and should be run against localhost and not the server's hostname.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/craigwatson/bitcoind-status/issues/26#issuecomment-230312978,
or mute the thread
https://github.com/notifications/unsubscribe/ARlCrhzQFu4B5xFw4igIkHW2G6cyaePfks5qSSTvgaJpZM4JEdTQ
.

From the looks of the error, your server could be set up to use different virtual hosts based on the hostname. Is your daemon running on the same server as the webserver running the scripts?
To be clear, the only supported configuration is for the following, running Bitcoin. If you don't have this configuration, I can make a best guess at what is happening, but unfortunately no more than that:

Yes everything is on the same server, with cryptocube as the hostname. The
server itself is a VPS on a Hosting Provider...
and i have ispconfig for the cpanel, and cloudflare protection. also the
hosting provider has their own Dos protection
On Mon, Jul 4, 2016 at 12:13 PM, Craig Watson<EMAIL_ADDRESS>wrote:

From the looks of the error, your server could be set up to use different
virtual hosts based on the hostname. Is your daemon running on the same
server as the webserver running the scripts?
To be clear, the only supported configuration is for the following,
running Bitcoin. If you don't have this configuration, I can make a
best guess at what is happening, but unfortunately no more than that:
[image: setup]
https://camo.githubusercontent.com/02310654824a282315f7d865096a60dc69f05248/68747470733a2f2f646c2e64726f70626f7875736572636f6e74656e742e636f6d2f752f333238313636342f6274632e6a7067
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/craigwatson/bitcoind-status/issues/26#issuecomment-230321365,
or mute the thread
https://github.com/notifications/unsubscribe/ARlCrpojhN00nNbaVaxGs349XhZe-QWVks5qSTEQgaJpZM4JEdTQ
.

As I said, without knowing your exact server setup, complete with Apache virtual hosts, I'm unable to help any further.
My best guess is that your Apache configuration is not set to serve that DocumentRoot to requests for localhost, so I would suggest that you start there.
Your rpc_host  entry in php/config.php file should be pointing to localhost, as should your cron commands using curl. If Apache is set up correctly, then everything else should work.

https://www.howtoforge.com/perfect-server-ubuntu-14.04-apache2-php-mysql-pureftpd-bind-dovecot-ispconfig-3-p4
that's the setup i went by
On Mon, Jul 4, 2016 at 1:09 PM, Craig Watson<EMAIL_ADDRESS>wrote:

As I said, without knowing your exact server setup, complete with Apache
virtual hosts, I'm unable to help any further.
My best guess is that your Apache configuration is not set to serve that
DocumentRoot to requests for localhost, so I would suggest that you start
there.
Your rpc_host entry in php/config.php file should be pointing to localhost,
as should your cron commands using curl. If Apache is set up correctly,
then everything else should work.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/craigwatson/bitcoind-status/issues/26#issuecomment-230329114,
or mute the thread
https://github.com/notifications/unsubscribe/ARlCrg8JvoyEJ52CsiRFnIGiIduMoy6jks5qST5FgaJpZM4JEdTQ
.

ok fixed the DocumentRoot,
now the output says ->
root@CryptoCube:/etc/apache2/sites-enabled# sudo curl -Ssk http://localhost/node/stats.php

403 Forbidden

Forbidden
You don't have permission to access /node/stats.php
on this server.

Apache/2.4.7 (Ubuntu) Server at localhost Port 80

That now looks like file permission/ownership problems.

curl -Ssk http://localhost/node/stats.php
::1 is not in the whitelist
??

Try <IP_ADDRESS> instead of localhost - ::1 is IPv6.

curl -Ssk http://<IP_ADDRESS>/node/stats.php
RPC Error

should i change the config.php host to <IP_ADDRESS>?

If your daemon is listening on IPv6, then localhost should work. If not, then it'll need to be <IP_ADDRESS>.

root@CryptoCube:~# lsof -i :8332
COMMAND     PID USER   FD   TYPE     DEVICE SIZE/OFF NODE NAME
DigiCubed 23264 root   19u  IPv4<PHONE_NUMBER>      0t0  TCP *:8332 (LISTEN)
^^ The daemon

php/config.php setup ->
$config = array(
// RPC
'rpc_user'                  => 'digicube',
'rpc_pass'                  => '[ommitted]',
'rpc_host'                  => 'localhost',
'rpc_port'                  => '8332',
'rpc_ssl'                   => false,
'rpc_ssl_ca'                => null,

I'm not sure I can offer much help here - try <IP_ADDRESS> and see if it fixes the issue. Ultimately I only really support the script, anything on the RPC side is the user's responsibility.

http://cryptostakers.com/node/

LOL
i just testing with sprouts coindaemon it works.
there's something wrong with the digicube daemon, i'll have to contact the dev see what he fucked up in it.

No problem, I'll close this issue.

ty for your help today


Backend refactoring using TS-Node

In order to use full-stack TypeScript, using the NestJS framework will lead to a more clean backend architecture and type-safety from the language.

NestJS seems to be kinda overkill just for TS support. TS-Node should be a more suitable approach.

Resolved with 1cae463e18653b12ec80c44000378c189f0b5dbc


Improving error handling for all utilities

I'd like to start some discussion on error handling and how we can improve it.
Current situation
Currently, the util is simply run and returns an i32 containing an error code, which is then used as exit code. The util itself is then responsible for printing any errors for the user.
In my opinion, there are a couple of problems with this:

There is a lot of return 0; and codes for collecting and combining status codes in the utils. For example patterns like this:

let exit_code = 0;
for file in files {
    let code = foo(file);
    exit_code |= code;
}
return exit_code;

It prohibits the use of ? in the uumain functions.
By extension it discourages idiomatic handling of errors in other functions, because they also often just return i32.
All utils have their own formats for common errors. We are of course actively fixing these by checking against the GNU tests, but there is still a lot of manual work and a lot of room for (spelling) mistakes,

Proposed solution
We could move toward making a uucore::Error type like this (with just a few examples implemented):
enum Error {
    BadUsage,
    PermissionDenied(Path),
    // more common errors here
    CustomMsg { msg: String, code: i32 },
    Code(i32),
}

So the signature of uumain then becomes:
fn uumain(uumain(args: impl uucore::Args) -> Result<(), uucore::Error>

The main function generated by uucore_procs::main would then be responsible for printing the error message and returning the corresponding exit code.
That type could then also provide a method to print an error message for the utils. So this:
show_error!("'{}': Permission denied", file.display());

would become something like this:
eprintln!("{}", uucore::Error::PermissionDenied(path));
// or
uucore::Error::PermissionDenied(path).print(); // or show(), display(), etc.

Finally we could provide automatic conversion from ClapResult to our return type, to make the messages for invalid or missing arguments match GNU more closely. This would allow us to use more of clap's capabilities. For example, this is taken from a recent PR:
// clap provides .conflicts_with / .conflicts_with_all, but we want to
// manually handle conflicts so we can match the output of GNU coreutils
if (matches.is_present(options::CSH)
    || matches.is_present(options::C_SHELL)
    || matches.is_present(options::SH)
    || matches.is_present(options::BOURNE_SHELL))
    && matches.is_present(options::PRINT_DATABASE)
{
    show_usage_error!(
        "the options to output dircolors' internal database and\nto select a shell \
    return 1;
}

Since ? calls into() implicitly, the usage of clap with custom error messages would then just be:
fn uumain(uumain(args: impl uucore::Args) -> Result<(), uucore::Error> {
    let matches = App::new(...)
        .arg(...)
        .arg(...)
        .get_matches_from_safe(args)?;
    ...
}

I would love to hear what everyone's thoughts on this are. Note that the solution above is purely for getting the discussion started (I don't even think lifetimes work out in the enum) and it is quite possible that the type will need to be significantly more complex.
So my questions are:

Is this useful or is it too complex?
Do you think it's worth it to refactor everything (although maybe this can also be done incrementally)?
Are there any situations where this would not work?
Are there utilities with incompatible errors? Such as one utils returning Bad Usage with code 1 and another with code 2.

I think this is a great idea, and very useful! I think it would be useful to implement from() for the most common errors we might encounter, such as io errors, utf-8 etc.
Some quick ideas:

Such as one utils returning Bad Usage with code 1 and another with code 2.

sort is such a utility, it returns 2 for almost any error. We could allow utilities to override the conversion from an error enum variant to the error code. (I'm thinking we could have a Utility trait that every utility implements, with some default implementations)

Instead of using a CustomMsg { msg: String, code: i32 },, we could use something more like Custom(Box<dyn CustomError>), and require each CustomError object to be able to tell us the error message/code. Usage could look like this:

enum SortError {
   Disorder { line: usize },
}

impl CustomError for SortError {
  fn error_code(&self) -> i32 {
    match self {
      Disorder { .. } => 1
  }
  fn error_message(&self) -> String {
     todo!()
  }
}

Otherwise I think it encourages to create a String message the moment the error occurs. I think this pattern would also be a bit better if we want to localize error messages at some point.

This is probably a bit off-topic, but I don't particularly like crash!() and crash_if_err!(), because they immediately exit without running destructors. That's causing bugs in sort I need to fix, because I didn't think of that when implementing. I think we should use this proposed error mechanism for expected errors and just panic if there's an unespected error.

I agree completely! For some more inspiration, I'm looking at ripgrep. They do a few interesting things there:

They have a global AtomicBool to determine the final exit code. That would make the handling of non-fatal errors a lot easier. (We would adapt this to be an AtomicI32.
They indeed implement From on common errors.
Their global result type is just Result<T, Box<dyn std::error::Error>>, which is extremely flexible.

Based on this, I would propose use Result<(), Box<dyn UError>> where UError is:
trait UError : std::error::Error {
    fn error_code(&self) -> i32;
}

The Display trait would then be used to print the message, like normal Rust errors.
Then we provide a bunch of common types that implement this trait in uucore and any utility can then implement this for custom errors. The errors in uucore could then by convention include a method with_code that changes the return code (for utilities like sort).
Additionally, we have that AtomicI32 that we return if Ok(()) is returned from uumain and we could provide methods/macros to set it.

Here's a slightly different proposal: I think it would be beneficial if the common errors are enum variants, so that it is easier for utilities to override the error code and/or messages in one place.
So, I'd propose a Result<(), Box<UError>> where UError is:
enum UError {
  Common(UCommonError),
  Util(Box<dyn UUtilError>)
}

UCommonError would be a big enum with all common errors, and UUtilError a trait like the one you proposed. So what I'm proposing is basically using trait objects for utility-specific errors and an enum for common errors.
I propose the following way for utilities to override the default exit codes:

Have a method map_custom_exit_code on UError that allows to pass a function that returns the exit code for a given UCommonError variant.
uumain could look like this:

fn uumain(args: impl uucore::Args) -> Result<(), Box<UError>>{
  actual_main(args).map_custom_exit_code(exit_code_mapper)
}

fn exit_code_mapper(error: UCommonError) -> i32 {
  match error {
     _ => todo!(),
  }
}

Regardless of what we decide for I'd introduce a type alias for the result type. In this proposal that would be
type UResult<T> = Result<T, Box<UError>>;

so that you can write UResult<Type> instead of Result<Type, Box<UError>> for the return type.

UCommonError would be a big enum with all common errors

I'm currently trying to implement this (even with an almost identical name). The only difference is that I'm using the trait I proposed for both common and custom errors. I thought that would be a little bit more ergonomic. I'll put that up as a draft PR soon. My current approach is this:

A UError trait that extends Error with a code method.
A CommonError enum for the common errors.
A CommonErrorWithCode type that you can create with the with_code method on CommonError.
An ExitCode type that doesn't print anything when returned but only returns an exit code.
A set_exit_code function that sets the exit code for if Ok(()) is returned (based on ripgrep).

Have a method map_custom_exit_code on UError that allows to pass a function that returns the exit code for a given UCommonError variant.

This is an interesting idea. I can see that it would be annoying to put with_code on all your errors but rather define them in one place. What would be result after applying that map? It can't be the enum, because that does not explicitly contain the exit code. Would it map to a third enum variant CommonWithCode or maybe custom UError::Util? Also, what type is it implemented on? We can't implement it on Result, so I think it would at least be something like:
fn uumain(args: impl uucore::Args) -> Result<(), Box<UError>>{
  actual_main(args).map_err(|e| e.map_custom_exit_code(exit_code_mapper))
}

I would love to eventually make UResult a struct containing a status code and an optional message. On which we could then implement map_custom_exit_code. Unfortunately, the Try trait hasn't been stabilized yet, so we wouldn't be able to use ? on it for a long time.
With the additional enum, I'm worried that constructing errors might become too verbose. It would be something like:
Err(UError::Common(CommonError::PermissionDenied(path)))

I guess that we could make convenience methods on UError to mitigate this a bit, but then it would still be quite verbose for custom errors.

Regardless of what we decide for I'd introduce a type alias for the result type.

Agreed!

This is an interesting idea. I can see that it would be annoying to put with_code on all your errors but rather define them in one place. What would be result after applying that map? It can't be the enum, because that does not explicitly contain the exit code. Would it map to a third enum variant CommonWithCode or maybe custom UError::Util? Also, what type is it implemented on? We can't implement it on Result, so I think it would at least be something like:

The return type could be a UError. It could use the UError::Custom variant to put a struct that stores the exit code and message.
I think we can put it on Result if we use traits.

With the additional enum, I'm worried that constructing errors might become too verbose. It would be something like:

Again, traits help us here. By implementing the From trait we can replace much of the boilerplate with an into().
Here's a very simple implementation: (Note that there is not much boilerplace in map_custom_exit_code, Err(ErrorWithCodeAndMessage { code, message }.into()) is not very verbose imho)
use std::{error::Error, fmt::Display};

enum UError {
    Common(UCommonError),
    Custom(Box<dyn UCustomError>),
}

trait UCustomError: Error {
    fn error_code(&self) -> i32;
}

enum UCommonError {}

#[derive(Debug)]
struct ErrorWithCodeAndMessage {
    code: i32,
    message: String,
}

impl Display for ErrorWithCodeAndMessage {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        f.write_str(&self.message)
    }
}

impl Error for ErrorWithCodeAndMessage {}
impl UCustomError for ErrorWithCodeAndMessage {
    fn error_code(&self) -> i32 {
        self.code
    }
}

type UResult<T> = Result<T, Box<UError>>;

trait UErrorTrait {
    fn map_custom_exit_code(self, mapper: fn(&UCommonError) -> (i32, String)) -> Self;
}
impl<T> UErrorTrait for UResult<T> {
    fn map_custom_exit_code(self, mapper: fn(&UCommonError) -> (i32, String)) -> Self {
        match &self {
            Err(e) => match e.as_ref() {
                UError::Common(e) => {
                    let (code, message) = mapper(e);
                    Err(ErrorWithCodeAndMessage { code, message }.into())
                }
                _ => self,
            },
            _ => self,
        }
    }
}

impl<E: UCustomError + 'static> From<E> for Box<UError> {
    fn from(v: E) -> Self {
        Box::new(UError::Custom(Box::new(v) as Box<dyn UCustomError>))
    }
}

impl From<UCommonError> for Box<UError> {
    fn from(v: UCommonError) -> Self {
        Box::new(UError::Common(v))
    }
}

Something else: Do all utilites print their name in front of every error message?

Awesome! I didn't think of using a trait for that. I'll try to incorporate that into what I was working on and put up a draft PR tomorrow.

@miDeb How do you think we should approach wrapping errors from the standard library? For example, let's say we want to wrap the Permission denied error. The io::Result contains only an error code from the underlying OS error. Therefore, we would somehow need to provide the Path for the UError/UResult, so that we can print it later.
Additionally, we need to give it some sort of format string that specifies the "action", because different utils use different descriptions and sometimes even multiple descriptions in the same util:
touch: cannot touch 'test': Permission denied
touch: setting times of 'test': Permission denied
head: cannot open 'test' for reading: Permission denied

I can think of a few approaches:

Provide a method on UResult that takes an io::Result, a &'static str and a &Path, but this is quite verbose. Usage would be:

UResult::from_io(File::create(path), "cannot touch {}", &path);

Alternatively, we could implement From<(std::io::Result, &'static str, &Path)>?

(File::create(path), "cannot touch {}", &path).into();

Implement From<std::io::Result> and make an additional method for the message:

File.create(path).into().message("cannot touch {}", &path);

A fancy proc macro that wraps the functions that return io::Result, so you could write

uresult!(File::create(path), "cannot touch {}");

which is cool, but I imagine also quite fragile and cumbersome to create and maintain, because it would get the path from the argument to the given function.

We just create a function which returns a &'static str based on the ErrorKind of the io::Error, and leave the creation of the error up to the util. So ErrorKind::PermissionDenied just maps to "Permission Denied". At least the message is then standardized.
We wrap the functions and types we need in uucore. So maybe a create_file function or even a create function in a File (or UFile?) module. We could generate these pretty easily with macros, I think. We would have to keep adding any new function that we start using though. It also makes it harder to work with io::Result if you don't want to convert or want to convert later. It doesn't feel quite right. It would look something like:

use uucore::File;
...
File::create(path);

I'm not really happy with any of these options (not even sure if all are possible), so other approaches are definitely welcome!

Implement Fromstd::io::Result and make an additional method for the message:

We could allow giving context to any error by having a method on UResult.
The syntax could look like this:
File.create(path).into().failed_operation_message(|| format!("cannot touch {}", path))?;

It takes a closure so that it doesn't allocate a String even on success, and there has to be a macro invocation somewhere because we can't pass format strings to functions.
failed_operation_message could return a UCommonError variant like WithContext { context: String, inner: UCommonError }
Maybe that would work?
Alternatively we could leave it up to each utility to define their own custom errors for such failures.
They could use map_err() to create their own error from the io error, maybe like this:
File.create(path).map_err(|error| TouchError::Create { file_name: path, inner: error }?;


Language code for Simplified Chinese

In some places the language code used for Simplified Chinese is coded as zh but it should be zh-Hans instead, also to preserve existing URLs where possible (current website uses zh-Hans). I tried to change some instances of it in the code base and build scripts but not sure if I got it all and possibly may have messed things up a bit.
I noticed that whenever I tried to run npm test locally, it will delete many translated contributor docs for some reason, not sure if it is related though. See below for the deleted files:
        deleted:    src/content/contributor-docs/es/README.mdx
	deleted:    src/content/contributor-docs/es/contributing_to_the_p5.js_reference.mdx
	deleted:    src/content/contributor-docs/es/contributor_guidelines.mdx
	deleted:    src/content/contributor-docs/es/steward_guidelines.mdx
	deleted:    src/content/contributor-docs/es/webgl_contribution_guide.mdx
	deleted:    src/content/contributor-docs/hi/README.mdx
	deleted:    src/content/contributor-docs/hi/access.mdx
	deleted:    src/content/contributor-docs/hi/contributing_documentation.mdx
	deleted:    src/content/contributor-docs/hi/contributing_to_the_p5.js_reference.mdx
	deleted:    src/content/contributor-docs/hi/steward_guidelines.mdx
	deleted:    src/content/contributor-docs/hi/unit_testing.mdx
	deleted:    src/content/contributor-docs/hi/webgl_contribution_guide.mdx
	deleted:    src/content/contributor-docs/ko/README.mdx
	deleted:    src/content/contributor-docs/ko/contributing_documentation.mdx
	deleted:    src/content/contributor-docs/ko/contributing_to_the_p5.js_reference.mdx
	deleted:    src/content/contributor-docs/ko/contributor_guidelines.mdx
	deleted:    src/content/contributor-docs/ko/creating_libraries.mdx
	deleted:    src/content/contributor-docs/ko/friendly_error_system.mdx
	deleted:    src/content/contributor-docs/ko/inline_documentation.mdx
	deleted:    src/content/contributor-docs/ko/method.example.js
	deleted:    src/content/contributor-docs/ko/release_process.mdx
	deleted:    src/content/contributor-docs/ko/sidebar.mdx
	deleted:    src/content/contributor-docs/ko/steward_guidelines.mdx
	deleted:    src/content/contributor-docs/ko/unit_testing.mdx
	deleted:    src/content/contributor-docs/ko/webgl_contribution_guide.mdx
	deleted:    src/content/contributor-docs/ko/webgl_mode_architecture.mdx
	deleted:    src/content/contributor-docs/zh-Hans/README.mdx
	deleted:    src/content/contributor-docs/zh-Hans/access.mdx
	deleted:    src/content/contributor-docs/zh-Hans/contributing_to_the_p5.js_reference.mdx
	deleted:    src/content/contributor-docs/zh-Hans/contributor_guidelines.mdx
	deleted:    src/content/contributor-docs/zh-Hans/creating_libraries.mdx
	deleted:    src/content/contributor-docs/zh-Hans/fes_reference_dev_notes.mdx
	deleted:    src/content/contributor-docs/zh-Hans/friendly_error_system.mdx
	deleted:    src/content/contributor-docs/zh-Hans/inline_documentation.mdx
	deleted:    src/content/contributor-docs/zh-Hans/release_process.mdx
	deleted:    src/content/contributor-docs/zh-Hans/steward_guidelines.mdx
	deleted:    src/content/contributor-docs/zh-Hans/unit_testing.mdx
	deleted:    src/content/contributor-docs/zh-Hans/web_accessibility.mdx
	deleted:    src/content/contributor-docs/zh-Hans/webgl_contribution_guide.mdx
	deleted:    src/content/contributor-docs/zh-Hans/webgl_mode_architecture.mdx

is it possible to have the zh folder in the p5js repo contributor docs folder renamed to zh-Hans?

Will do that. Thanks.

verified this is works for me locally! thanks @limzykenneth!


Investigate Potential Thread Starvation

Description
During load testing, the framework exhibited potential thread starvation when the rate limiting and caching middlewares were enabled and the response size was over 70kb. After disabling these two middlewares, the application's performance improved substantially, indicating that one (or both) of these components may be holding onto threads longer than necessary during the handling of large responses.
Steps to Reproduce

Enable rate limiting and caching middlewares.
Run a Postman runner with 10,000 iterations, with each iteration involving a write, a patch, and two findAll operations in the database.
The find all requests should generate a response over 70kb.
Observe thread utilization and response times.

Expected behavior
The application should be able to handle a high volume of requests and large responses without experiencing significant latency or thread starvation.
Actual behavior
The application experienced potential thread starvation, and latency increased to minutes when the rate limiting and caching middlewares were enabled and the response size was over 70kb.
Additional context

The application is hosted on a free EC2 instance provided by AWS.
The load test involved significant database operations, including writes and fetches.

We need to investigate how these two middlewares are handling their threads, particularly when processing large responses, and ensure they're releasing them back to the pool in a timely manner. Also, we need to make sure these middlewares are thread-safe and capable of handling high concurrency loads without negatively impacting performance.

Update:
During our investigation into the performance issue we've discovered that the root cause of the problem is related to thread management. Specifically, our threads seem to be dying prematurely when the rate limiting and caching middlewares are enabled and the response size is over 70kb.
Given the fact that our application's performance improved substantially when these middlewares were disabled, we have determined that the issue is not due to inherent limitations of our server resources, but rather to how these two components are managing their threads.
We are currently in the process of conducting a deeper analysis of these middlewares to understand why they are not releasing their threads back to the pool in a timely manner. Our goal is to ensure that these middlewares are able to handle large responses effectively, without any unnecessary latency or thread starvation.
In addition, we are examining the thread-safety of these middlewares to ensure that they are capable of handling high concurrency loads without negatively impacting performance. This includes investigating any potential synchronization issues that might be causing the threads to die prematurely.

Issue Status: Partially Resolved
Update:
As we delved further into the performance issue, we discovered an interesting correlation between thread starvation and the size of request payloads. We found that the issue was more visible for requests with larger payloads due to their inherently longer processing time. This longer processing time provided a wider window for forced connection closes on the client side, leading to premature thread termination.
To address this issue, we have implemented a fix that optimizes thread management, specifically for large payloads. The fix ensures that even in cases of forced client-side connection termination, our server-side threads are properly managed and released back into the pool in a timely manner, thereby preventing starvation.
However, we want to stress that while the fix has shown promising results in our initial tests, we will continue to monitor its performance closely in a production environment to ensure its effectiveness. We also recognize that there might be other factors contributing to thread starvation, and will thus keep the issue open as we continue our investigations.
We appreciate your understanding and patience as we strive to resolve this issue fully.

Issue fully solved with last fix.


Update battery level while charging.

Using "Battery Level Widget (with percentage) (v0.14)", only while charging, it would be nice if the battery level percentage was updated.
Now, you have to press the button to see at which level the battery is.

fixed with last update afaik


Add/restore beforeCopy and afterCopy options

Q
A

Is bugfix?
❌

New feature?
✔

Breaks BC?
❌

Add FileHelper::copyFile method
Add beforeCopy and afterCopy options to FileHelper::copyDirectory and FileHelper::copyFile

Thanks.


Add performance/logging hook

Code is checkpointed.
Use new error hook
https://github.com/feathersjs/feathers-hooks/pull/110#event-826778948
Use app.hook.before & .after when done
Tests

See feathers-service-logging-stats