MMseq2 dies after write error

Hello.
MMSeq2 dies after write error, The shell output is below. Anyone knows why this happens?
Thanks,
Torsten

Create directory tmp
search kelp_database uniref50 results tmp
MMseqs Version:                        	GITDIR-NOTFOUND
Substitution matrix                    	nucl:nucleotide.out,aa:blosum62.out
Add backtrace                          	false
Alignment mode                         	2
Alignment mode                         	0
Allow wrapped scoring                  	false
E-value threshold                      	0.001
Seq. id. threshold                     	0
Min alignment length                   	0
Seq. id. mode                          	0
Alternative alignments                 	0
Coverage threshold                     	0
Coverage mode                          	0
Max sequence length                    	65535
Compositional bias                     	1
Max reject<PHONE_NUMBER>
Max accept<PHONE_NUMBER>
Include identical seq. id.             	false
Preload mode                           	0
Pseudo count a                         	1
Pseudo count b                         	1.5
Score bias                             	0
Realign hits                           	false
Realign score bias                     	-0.2
Realign max seqs<PHONE_NUMBER>
Gap open cost                          	nucl:5,aa:11
Gap extension cost                     	nucl:2,aa:1
Zdrop                                  	40
Threads                                	40
Compressed                             	0
Verbosity                              	3
Seed substitution matrix               	nucl:nucleotide.out,aa:VTML80.out
Sensitivity                            	5.7
k-mer length                           	0
k-score<PHONE_NUMBER>
Alphabet size                          	nucl:5,aa:21
Max results per query                  	300
Split database                         	0
Split mode                             	2
Split memory limit                     	0
Diagonal scoring                       	true
Exact k-mer matching                   	0
Mask residues                          	1
Mask lower case residues               	0
Minimum diagonal score                 	15
Spaced k-mers                          	1
Spaced k-mer pattern
Local temporary path
Rescore mode                           	0
Remove hits by seq. id. and coverage   	false
Sort results                           	0
Mask profile                           	1
Profile E-value threshold              	0.1
Global sequence weighting              	false
Allow deletions                        	false
Filter MSA                             	1
Maximum seq. id. threshold             	0.9
Minimum seq. id.                       	0
Minimum score per column               	-20
Minimum coverage                       	0
Select N most diverse seqs             	1000
Min codons in orf                      	30
Max codons in length                   	32734
Max orf gaps<PHONE_NUMBER>
Contig start mode                      	2
Contig end mode                        	2
Orf start mode                         	1
Forward frames                         	1,2,3
Reverse frames                         	1,2,3
Translation table                      	1
Translate orf                          	0
Use all table starts                   	false
Offset of numeric ids                  	0
Create lookup                          	0
Add orf stop                           	false
Overlap between sequences              	0
Sequence split mode                    	1
Header split mode                      	0
Chain overlapping alignments           	0
Merge query                            	1
Search type                            	0
Search iterations                      	1
Start sensitivity                      	4
Search steps                           	1
Exhaustive search mode                 	false
Filter results during exhaustive search	0
Strand selection                       	1
LCA search mode                        	false
Disk space limit                       	0
MPI runner
Force restart with latest tmp          	false
Remove temporary files                 	false
extractorfs kelp_database tmp/3499313520568641582/q_orfs_aa --min-length 30 --max-length 32734 --max-gaps<PHONE_NUMBER> --contig-start-mode 2
--contig-end-mode 2 --orf-start-mode 1 --forward-frames 1,2,3 --reverse-frames 1,2,3 --translation-table 1 --translate 1 --use-all-table-
starts 0 --id-offset 0 --create-lookup 0 --threads 40 --compressed 0 -v 3
[=================================================================] 714.62K 18s 1ms
Time for merging to q_orfs_aa_h: 0h 0m 17s 230ms
Time for merging to q_orfs_aa: 0h 0m 22s 109ms
Time for processing: 0h 1m 20s 933ms
prefilter tmp/3499313520568641582/q_orfs_aa uniref50 tmp/3499313520568641582/search/pref_0 --sub-mat nucl:nucleotide.out,aa:blosum62.out -
-seed-sub-mat nucl:nucleotide.out,aa:VTML80.out -k 0 --k-score<PHONE_NUMBER> --alph-size nucl:5,aa:21 --max-seq-len 65535 --max-seqs 300 --sp
lit 0 --split-mode 2 --split-memory-limit 0 -c 0 --cov-mode 0 --comp-bias-corr 1 --diag-score 1 --exact-kmer-matching 0 --mask 1 --mask-lo
wer-case 0 --min-ungapped-score 15 --add-self-matches 0 --spaced-kmer-mode 1 --db-load-mode 0 --pca 1 --pcb 1.5 --threads 40 --compressed
0 -v 3 -s 5.7
Query database size: 61153309 type: Aminoacid
Target split mode. Searching through 2 splits
Estimated memory consumption: 99G
Target database size: 48531432 type: Aminoacid
Process prefiltering step 1 of 2
Index table k-mer threshold: 122 at k-mer size 7
Index table: counting k-mers
[=================================================================] 24.26M 1m 44s 733ms
Index table: Masked residues: 187925951
Index table: fill
[=================================================================] 24.26M 3m 17s 822ms
Index statistics
Entries:<PHONE_NUMBER>
DB size:          45739 MB
Avg k-mer size:   4.911615
Top 10 k-mers
DFEQLPH	32892
NVPGWSP	32831
FRYAFPS	32736
RYYVLGW	32688
WRLDFLN	31763
TVDGDFS	31579
NKTDFVQ	31135
QDWVQIP	30874
LDGAYVP	30051
ETGRYNV	29832
Time for index table init: 0h 5m 17s 428ms
k-mer similarity threshold: 122
Starting prefiltering scores calculation (step 1 of 2)
Query db start 1 to 61153309
Target db start 1 to 24258060
[=================================================================] 61.15M 27h 30m 21s 285ms
2412.140792 k-mers per position
608927 DB matches per sequence
175 overflows
0 queries produce too many hits (truncated result)
196 sequences passed prefiltering per query sequence
198 median result list length
34327 sequences with 0 size result lists
Time for merging to pref_0_tmp_0: 0h 0m 44s 467ms
Time for merging to pref_0_tmp_0_tmp: 0h 12m 8s 854ms
Process prefiltering step 2 of 2
Index table k-mer threshold: 122 at k-mer size 7
Index table: counting k-mers
[=================================================================] 24.27M 1m 54s 630ms
Index table: Masked residues: 187586445
Index table: fill
[=================================================================] 24.27M 3m 32s 124ms
Index statistics
Entries:<PHONE_NUMBER>
DB size:          45742 MB
Avg k-mer size:   4.912002
Top 10 k-mers
DFEQLPH	33023
NVPGWSP	32989
QGKSPFQ	32900
FRYAFPS	32880
RYYVLGW	32788
WRLDFLN	31914
TVDGDFS	31713
NKTDFVQ	31393
QDWVQIP	31110
LDGAYVP	30048
Time for index table init: 0h 5m 43s 381ms
k-mer similarity threshold: 122
Starting prefiltering scores calculation (step 2 of 2)
Query db start 1 to 61153309
Target db start 24258061 to 48531432
[=================================================================] 61.15M 28h 53m 9s 351ms
2412.140792 k-mers per position
608911 DB matches per sequence
174 overflows
0 queries produce too many hits (truncated result)
196 sequences passed prefiltering per query sequence
198 median result list length
34543 sequences with 0 size result lists
Time for merging to pref_0_tmp_1: 0h 0m 35s 830ms
Time for merging to pref_0_tmp_1_tmp: 0h 10m 54s 136ms
Merging 2 target splits to pref_0
Preparing offsets for merging: 0h 0m 19s 175ms
[=================================================================] 61.15M 26m 22s 19ms
Time for merging to pref_0: 0h 0m 36s 177ms
Time for merging target splits: 0h 27m 38s 35ms
write error
Error: Prefilter died
Error: Search step died
-bash-4.2$

Is tmp on a network share? Could you try placing it on a local disk? Is there enough free space?

Thanks. tmp is on a local disk, but I only have 1TB availabe. That disk disk space gets full during execution, although I only want to run about 700K sequences against UniRef50. I will try --remove-tmp-files.

I was meaning to add the same additional prefiltering stage we used int the recent MMseqs2 taxonomy paper to the normal search, this would also speed-up and reduce disk use at a slight sensitivity penalty. But I haven't gotten around to that.

Thanks. This worked.


Replace testAsync to testTask everywhere in unit and integration tests

So all |> Async.AwaitTask will be removed

Closed by #203


Introducing Webpack to handle assets pipeline

Since JS ecosystem is mature enough nowadays to bundle all front-end assets (e.g. javascripts, stylesheets, images, etc). It is best to replace the sprockets assets pipeline from Rails with task bundler like Webpack.
There are many advantages by moving towards a JS run bundler:

Easy integration of new JS features from ES6 (modules, const, let, arrow functions, etc)
Easy integration of unit and integration JS test
More active community to solve problems or issues
Easy splitting of file bundlers, for instance per page if the current JS bundled (application.js) grows considerably
and many more...

This is complete. Closing since there is no UAT component


Error vietnam mobile phone

Hello,
Thank you for an awesome pkg.
Can you check for this mobile phone: +84888696413
It is valid mobile phone in Vietnam but the function return false.

@levy193 - thanks for raising, PR is welcome.
Looks like the 888 prefix is not included, which carrier is that?

Fixed by #1031.


Api getting called twice...

I have encountered a issue.. I have website whose url is passed in iframe src="", Now once i iframe is loaded what i can observe that the api is called twice.. Now the website url which i have passed in src has a api call which dynamically creates fields.. Now this api gets triggered twice. Can we some how stop it from triggering twice

Not enough info to respond to question


Update default invite modal screens

Remove the Setup email service information
Keep the illustration and the text "You haven't invited any users yet" when there are no users for that app.
We're showing the message even when there are users present, we need to remove that.

Fixed


Workloads are deployed although not part of an application

Problem
A workload is annotated with recommended K8s labels (in particular part-of) but it is not part of the KeptnApp manifest. The Lifecycle Toolkit deploys the workload anyway.
Desired result: The Lifecycle toolkit prevents the workload to be bound to a node since it is not part of the release bracket.
Replication steps

Use the main helm-chart of Keptn and annotate NATs and Mongo with default labels (link: https://github.com/keptn/keptn/tree/poc-klt-helm-chart)
Use KLT v0.4.0 to deploy
Watch mongo and nats being deploy

I think this is also an example for implicit mode. When implicit is enabled it would be added. Otherwise not.


Re-enable the full lldb test suite for non-smoke-test jobs

The lldb bots should run the full lldb test suite, not just the tests
reserved for pull-request testing.
This does not affect PR testing.
rdar://38462589

@swift-ci smoke test

@swift-ci smoke test

@swift-ci please clean smoke test

@shahmishal Do you know why there are >180 changes being tested on the bot (https://ci.swift.org/job/swift-PR-Linux-smoke-test/5361/)? I've made sure to rebase on the latest swift changes.


New assertion in Run() to resolve verification error

This assertion helps Silicon to prove the corresponding invariant.

I guess we are now running into a different issue in silicon? 🤣


feat: allowed ERC registry indexing tool to target localnet

Description:
Contract call APIs interact with the Mirror Node Web3Module, but the local network does not automatically forward these requests. As a result, the indexing tool must manually redirect contract call requests to the Web3Module endpoint.
To address this, this PR introduces the MIRROR_NODE_URL_WEB3 configuration. This addition enables the ERC registry indexing tool to seamlessly execute contract call APIs against the Web3Module, ensuring effective targeting of the local network.
Related issue(s):
Fixes #1070
Notes for reviewer:
This PR builds upon the changes introduced in PR https://github.com/hashgraph/hedera-smart-contracts/pull/1064. As a result, many updates from PR https://github.com/hashgraph/hedera-smart-contracts/pull/1064 are reflected here.
It is recommended to review PR https://github.com/hashgraph/hedera-smart-contracts/pull/1064 before proceeding with this one.
Checklist

[ ] Documented (Code comments, README, etc.)
[ ] Tested (unit, integration, etc.)

lg, but isn't this something that should be provided by local node? the forwarding to web3?
Yeah ha actually it's a good point. I think it's doable okay will create a ticket on local-node repo. Thanks for pointing it out!


Implement grey cells

This PR implements grey cells.

Need to build off this for a new feature. Manually approving.

Need to build off this for a new feature. Manually approving.


关于supernet scheduler

lib/utils/util.py 中，scheduler 的自定义函数为 lambda step: (cfg.LR - step / ITERS)，是否该改为  lambda step: (1 - step / ITERS) ？

Hi,
This scheduler is designed to help train a better supernet. In the beginning, due to random architectures, we need large learning rate to train this network. As the training process goes on, we should reduce learning rate at a steady pace. You could directly change the learning rate to 1 or use another learning rate.
Best,
Hao.


Add tests

add tests of the default values of undefined fields

Codecov Report

Merging #280 into develop will increase coverage by 1.89%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop     #280      +/-   ##
===========================================
+ Coverage    95.89%   97.79%   +1.89%
===========================================
  Files            9        9
  Lines          317      317
  Branches       102      102
===========================================
+ Hits           304      310       +6
+ Misses           8        2       -6
  Partials         5        5

Impacted Files
Coverage Δ

packages/ckb-sdk-rpc/src/resultFormatter.ts
95.23% <0%> (+3.57%)
:arrow_up:

packages/ckb-sdk-rpc/src/paramsFormatter.ts
96% <0%> (+4%)
:arrow_up:

packages/ckb-sdk-utils/src/address/index.ts
100% <0%> (+5.4%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1701bb3...2846e94. Read the comment docs.


Scope 5 Mar

[x] Disable "All Funds" for fungibles (fixing a bug)

On-ramp

[x] Make Ramp (by crypto with fiat) available for other tokens #2587 #2591

Swaps

[x] xDAI + Honeyswap #2537 #2540

UI

[x] Add "Show Seed Phrase" to Settings tab #2582 #2589
[x] Deeplink FAQ from the Settings to AlphaWallet.com/faq #2583 #2588
[x] Fix web3swift memory leaks #2584 #2586
[x] Back up seed phrase introduction screen text is clipped in iPhone 6s #2564 #2574
[x] Remove App Intro Slides #2552 #2561
[x] Add a custom graphic for empty activites #2553 #2560
[x] Update xDai token icon #2535 #2548
[x] Refreshed prompts to sign messages #2497 #2549

Analytics

[x] Log number of wallets #2578
[x] Log transaction failed #2579
[x] Log browser events #2580

Wallet Connect fixes

[x] WalletConnect using mobile linking crashes if the chain is disabled #2609 #2614
[x] WalletConnect memory leaks #2602 #2603

Performance and Internals

[x] Remove unnecessary call to name and symbol for ENS token #2613
[x] Reduce TransactionsStorage instances #2605 #2606
[x] TokenViewController is not released after popping #2595 #2601
[x] Refactor: Separate out client code that logs to analytics #2577
[x] Tweak error logging to be more granular #2576

User-friendly messages

[x] Fix: error message shown when user doesn't have enough funds for sending a transaction (send amount + gas) is not user friendly #2556
[x] Tweak (internal) logging of errors to collect them

Other fixes

[x] Fix: TokenScript attributes are not resolved #2611
[x] Sometimes crashes when disabling a chain in Settings #2597 #2600
[x] Launch swap service from token to dapp browser should always switch to correct chain #2555

Backlog

[ ] Cancel/Speed up transactions (poor thing, has been shoved around)

Build 382


{Docker} Slim down docker images

History Notes:
Save close to 400MB on docker images size.

This checklist is used to make sure that common guidelines for a pull request are followed.

[ ] The PR title and description has followed the guideline in Submitting Pull Requests.

[ ] I adhere to the Command Guidelines.

add to S166.


Recommend installing stack instead of MinGHC #75

Pinging @ndmitchell and @3noch. I'm putting this up for a PR because I really do want this to be a discussion, not just a decision on my part. It seems to me like this is a fair assessment of what we discussed on #75, but let's make sure this correctly reflects what we all believe.

I'm certainly a fan of this in principle.

My only concern is that I often want GHC 7.4, 7.6, 7.8 and 7.10 all installed. In contrast, with Stack, I tend to get various lts versions, and that doesn't really give me GHC versions. Maybe that's really a weakness in Stack, and there should be ghc-7.10.N as a tag, which is the most recent lts of GHC 7.10? If there was a good story there I'd be in favour.

Also the minghc-7.8.bat script is invaluable, I don't know how to do that with stack.

I think I can answer both of those:

stack --resolver ghc-7.8.4 does work. We only go back as far as 7.8.4 though, so you won't be able to get 7.4 out of stack.
I don't know the exact cmd invocation, but shouldn't something like stack --resolver ghc-7.8.4 exec cmd work?

FWIW, I get by when developing on Windows just fine by calling stack ghc and stack runghc.
@3noch I just pushed a commit, though I'm not thrilled with how it makes the README look. I'm OK with this or other tweaks.

I was thinking something akin to
MinGHC
Stop: You likely want to use stack instead.....
....

@snoyberg I made some edits. Feel free to change. Like I said, I'm for this change.

Any thoughts @ndmitchell?
The README is probably a nice litmus test for this change since I'm guessing most newcomers find MinGHC via haskell.org.

I've made one tweak. I also suggest we also supply a bullet point list of how to use Stack like you would minghc. Something like download binary, add to path, type stack install --ghc, run stack ghc. At the moment it's not immediately obvious how stack can do the job of minghc.

Good call. I think it'd be better to have stack host some sort of install story for Windows and just link to that.

That would be even better.

To be clear: we're talking about stack providing an NSIS-based installer for Windows in addition to the raw executable, correct? I can definitely see value in that, and it would need to just:

Copy stack to $APPDATA\local\bin\stack
Add $APPDATA\local\bin to the PATH

I'm sure there are issues we'd need to get into though about code signing, user vs global install, but something along those lines seems reasonable.

Actually I was just talking about some sort of "How to" guide. But I have also thought of doing a tiny installer for stack in the past. Perhaps now's the time for that?

The installation procedure as it is today is covered on the downloads page: https://github.com/commercialhaskell/stack/wiki/Downloads#windows

And there's already an issue about an nsis installer at: https://github.com/commercialhaskell/stack/issues/613

Cool!

I was merely talking about instructions, not an installer, on which I'm ambivalent (I prefer zip files to installers). The download says how to get stack on your PATH. With MinGHC we put a working ghc on your path, which is very useful for people experimenting and not using cabal (which is pretty much all beginners to start with). What's missing is "run stack ghc or stack ghci to invoke ghc", and then stack gives the MinGHC experience.

OK, merging. We can improve the language over time, and the installer issue is already tracked upstream with stack.


Add character encoding specification feature for search keywords (#2)

Overview
Added a feature to specify the character encoding for search keywords. This allows handling not only utf-8 and unicode but also other character encodings like shift_jis. To achieve this functionality, the "encoding.js" library has been introduced.
Areas to focus on
UI changes for character encoding specification
Introduction and configuration of the "encoding.js" library
Implementation of character encoding conversion
Example
%{s:sjis} ・・・Convert and replace text from automatically recognized character code to Shift_JIS
%{s:utf8-sjis} ・・・ Convert and replace text from specified character code to Shift_JIS

@ashutoshetw
This pull request adds a feature to specify the character encoding for search keywords. In particular, the "encoding.js" library has been introduced to support shift_jis for Japanese. Please review the code, especially focusing on the UI changes and the implementation of character encoding conversion.


Fixing #495 which add the missing domain for a user list API request

This does only apply to v3 identity API which is already hard coded in this script.
Closes #495
(Cherry picked from edc64ab)

@BjoernT can you squash these please? Other wise :+1:

@d34dh0r53 I'm not sure if I can the commit comes out of the rackertom repository. The commit should have been rebased before cherry picking right ?

@d34dh0r53 Sorry, this was from using the update branch button that GitHub exposes. I knew better. Let me try again.

+1

:+1:


Fixes Temporary CD to the Current Directory

Description
This PR fixes an exception raised when attempting to cd into the current directory. This should instead result in no changes.
This PR adds tests to verify the behaviour of this function.
Status

[X] Ready to go

Codecov Report

Merging #19 into master will decrease coverage by 3.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master      #19      +/-   ##
==========================================
- Coverage   86.21%   83.19%   -3.02%
==========================================
  Files         106      106
  Lines        4098     4101       +3
==========================================
- Hits         3533     3412     -121
- Misses        565      689     +124

Impacted Files
Coverage Δ

nonbonded/library/utilities/utilities.py
95.83% <100.00%> (+0.59%)
:arrow_up:

...nbonded/library/factories/projects/optimization.py
30.86% <0.00%> (-66.67%)
:arrow_down:

nonbonded/library/templates/forcebalance.py
57.14% <0.00%> (-42.86%)
:arrow_down:

nonbonded/library/curation/components/selection.py
61.65% <0.00%> (-28.65%)
:arrow_down:

nonbonded/cli/optimization/generate.py
75.00% <0.00%> (-25.00%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 64a1303...a28a901. Read the comment docs.


Install script should not write partially downloaded binaries

Currently the binary download is streamed to disk once a 200 response
has been recieved. When an error occurs during the download a partially
downloaded binary is left on disk. Subsequent installs see the binary
and bail out of re-downloading it. Worse yet those subsequent installs
move the binary into the global cache so even removing node_modules
will not remove the broken binary.
With this patch the binary is only flushed to disk once it has been
fully downloaded.
Fixes #1882
Fixes #1888

Interesting the failure results in process.dlopen error message. This could be the root cause for so many of our previous issues.
module.js:598
  return process.dlopen(module, path._makeLong(filename));

What about going a bit further - writing to a temporary file and renaming only when completed?

That's certainly another alternative if we keep with the current approach of flushing to disk as data is received.
I've instead decided to go with this patch that only flushes when the download is completed.


Article on TomoChain accounts

Estimated time: 3h

Why:
Investors seems to have little understanding about how accounts works.
What:
An article explaining how accounts live on the blockchain, how tools like metamask, mew, cold wallets etc, interact with them. What are addresses, privates keys, public keys, etc.
One of the goal is to educate them about their wallet. We've seen a lot of people asking (about the swap): "Is it ok if I have my coins on MetaMask? But there's actually no value stored in MetaMask.
The only important thing for the swap is if you own the account (know the private key) or not. General awareness of how the blockchain accounts works might help them regarding this.

Assigned to: @bluebottleco
Reviewed by: @etienne-napoleone

Please find below the draft.
A few notes:

I can add more about explaining a BlockChain account if you think it’s necessary but I kept it short because I think most people likely understand this given they’re using BTC or ETH to buy Tomo.

https://medium.com/@eric585696/blockchain-accounts-82f933f0569a

Thanks @bluebottleco ! I added some comments directly on Medium!
Btw you can add it to the TomoChain publication, we will not publish it until it's ready but all the TomoChain editors can view the draft, view other editors comments and comment

Thanks @bluebottleco , the article looks good to me now.
Ready for you @alex9121


enhance: use log fields with API metadata

part of the effort for https://github.com/go-vela/community/issues/419
This change updates the server to have a more consistent, standardized set of fields when logging from API calls.
This is done by capturing all relevant values from the gin.Context{} via middleware functions:
https://github.com/go-vela/server/blob/11640888d903e0b52cf367e7a272e15986752506/api/build.go#L1024-L1028
Then, we use logrus.WithFields() to output a log message containing those fields:
https://github.com/go-vela/server/blob/11640888d903e0b52cf367e7a272e15986752506/api/build.go#L1030-L1040
The above code is called when sending a PUT request to the /api/v1/repos/:org/:repo/builds/:build endpoint.
Note
Not all endpoints will have the same level of information available to them so the formatting isn't totally consistent.
The code will make best efforts to log all relevant fields that are available in the content of the request.
Most, if not all, of these endpoints should output a user field in the log messages.
A slightly smaller amount should have an org and repo field in the log message as well.
Based on the endpoint hit, some other fields that are logged are build (number), service (number) or step (number).
Log Samples
Here is a snippet of what the logs look like, while running a build, before this change:
{"level":"debug","msg":"Reading repo JordanBrockopp/vela-play","time":"2021-12-17T15:43:26Z"}
{"level":"trace","msg":"getting repo JordanBrockopp/vela-play from the database","time":"2021-12-17T15:43:26Z"}
{"level":"debug","msg":"Reading build JordanBrockopp/vela-play/1","time":"2021-12-17T15:43:26Z"}
{"level":"trace","msg":"getting build JordanBrockopp/vela-play/1 from the database","time":"2021-12-17T15:43:26Z"}
{"level":"debug","msg":"Verifying user vela-worker has 'write' permissions for repo JordanBrockopp/vela-play","time":"2021-12-17T15:43:26Z"}
{"level":"info","msg":"Updating build JordanBrockopp/vela-play/1","time":"2021-12-17T15:43:26Z"}
{"level":"trace","msg":"updating build 1 in the database","time":"2021-12-17T15:43:26Z"}
{"level":"trace","msg":"getting build JordanBrockopp/vela-play/1 from the database","time":"2021-12-17T15:43:26Z"}
{"api-version":"v0.11.0","body":{"author":"JordanBrockopp","base_ref":"","branch":"main","clone":"https://git.example.com/JordanBrockopp/vela-play.git","commit":"b015d468dcc3606b57e7fc26a7d9ffe3aca441e2","created":1639755805,"deploy":"","deploy_payload":{},"distribution":"linux","email":"jordan_brockopp@example.com","enqueued":1639755806,"error":"","event":"push","finished":0,"head_ref":"","host":"worker","id":1,"link":"http://localhost:8888/JordanBrockopp/vela-play/1","message":"Create .vela.yml","number":1,"parent":1,"ref":"refs/heads/main","repo_id":1,"runtime":"docker","sender":"JordanBrockopp","source":"https://git.example.com/JordanBrockopp/vela-play/commit/b015d468dcc3606b57e7fc26a7d9ffe3aca441e2","started":1639755806,"status":"running","title":"push received from https://git.example.com/JordanBrockopp/vela-play"},"build":1,"fields.time":"2021-12-17T15:43:26Z","ip":"<IP_ADDRESS>","latency":8922218,"level":"info","method":"PUT","msg":"","path":"/api/v1/repos/JordanBrockopp/vela-play/builds/1","repo":"JordanBrockopp/vela-play","status":200,"time":"2021-12-17T15:43:26Z","user":"vela-worker","user-agent":"vela-sdk-go/0.11.0"}

Here is a snippet of what the logs look like, while running a build, after this change:
{"level":"debug","msg":"reading repo JordanBrockopp/vela-play","org":"JordanBrockopp","repo":"vela-play","time":"2021-12-17T15:48:19Z","user":"vela-worker"}
{"level":"trace","msg":"getting repo JordanBrockopp/vela-play from the database","time":"2021-12-17T15:48:19Z"}
{"build":1,"level":"debug","msg":"reading build JordanBrockopp/vela-play/1","org":"JordanBrockopp","repo":"vela-play","time":"2021-12-17T15:48:19Z","user":"vela-worker"}
{"level":"trace","msg":"getting build JordanBrockopp/vela-play/1 from the database","time":"2021-12-17T15:48:19Z"}
{"level":"debug","msg":"verifying user vela-worker has 'write' permissions for repo JordanBrockopp/vela-play","org":"JordanBrockopp","repo":"vela-play","time":"2021-12-17T15:48:19Z","user":"vela-worker"}
{"build":1,"level":"info","msg":"updating build JordanBrockopp/vela-play/1","org":"JordanBrockopp","repo":"vela-play","time":"2021-12-17T15:48:19Z","user":"vela-worker"}
{"level":"trace","msg":"updating build 1 in the database","time":"2021-12-17T15:48:19Z"}
{"level":"trace","msg":"getting build JordanBrockopp/vela-play/1 from the database","time":"2021-12-17T15:48:19Z"}
{"body":{"author":"JordanBrockopp","base_ref":"","branch":"main","clone":"https://git.example.com/JordanBrockopp/vela-play.git","commit":"b015d468dcc3606b57e7fc26a7d9ffe3aca441e2","created":1639756099,"deploy":"","deploy_payload":{},"distribution":"linux","email":"jordan_brockopp@example.com","enqueued":1639756099,"error":"","event":"push","finished":0,"head_ref":"","host":"worker","id":1,"link":"http://localhost:8888/JordanBrockopp/vela-play/1","message":"Create .vela.yml","number":1,"parent":1,"ref":"refs/heads/main","repo_id":1,"runtime":"docker","sender":"JordanBrockopp","source":"https://git.example.com/JordanBrockopp/vela-play/commit/b015d468dcc3606b57e7fc26a7d9ffe3aca441e2","started":1639756099,"status":"running","title":"push received from https://git.example.com/JordanBrockopp/vela-play"},"build":1,"ip":"<IP_ADDRESS>","latency":7397964,"level":"info","method":"PUT","msg":"","org":"JordanBrockopp","path":"/api/v1/repos/JordanBrockopp/vela-play/builds/1","repo":"vela-play","status":200,"time":"2021-12-17T15:48:19Z","user":"vela-worker","user-agent":"vela-sdk-go/0.11.0","version":"v0.11.0"}

Full Log File
Here is an example of what the logs look like, while running a build, before this change:
old.txt
Here is an example of what the logs look like, while running a build, after this change:
new.txt

Is there a plan to mitigate all the This log write receives unsanitized user input from... linter errors?

@KellyMerrick
AFAICT, those errors are unrelated to my changes and probably out of scope for this 😅
I believe they are produced from the CodeQL workflow:
https://github.com/go-vela/server/blob/master/.github/workflows/codeql-analysis.yml
And appear to be present in the current codebase, before this change:
https://github.com/go-vela/server/security/code-scanning
If there is a desire to mitigate them, I think a separate story and PR should be created to address them 👍


feat: improve code and add date bounds functionality

Description

Simplify dependencies
Improve code readability (a little bit)
Add ability to specify dates for when the script should or shouldn't fire events

continued at #2


When you planning to publish book?

Hi. When you planning to publish book?

Hi @P9avel, it is scheduled for publication on 26th of May this year. Your interest in the book is much appreciated!


abi48_0_0.com.reactnativecommunity.slider.ReactSlider cannot be cast to android.view.ViewGroup

Environment
Expo version: 0.7.1
Android Emulator: Pixel_XL_API_30

react-native info output:

// react-native info

are you using the new architecture?
yes
which version of react & react-native are you using?
"react": "18.2.0",
"react-native": "0.71.8",

Description
When running my project in the emulator this error happens: abi48_0_0.com.reactnativecommunity.slider.ReactSlider cannot be cast to android.view.ViewGroup
When running it on a physical iPhone device it just work.

Reproducible Demo
<Slider             style={styles.slider}             minimumValue={0}             maximumValue={120}             value={playBackTime.min + playBackTime.sec}             minimumTrackTintColor={trackSliderColor}             maximumTrackTintColor={textColorWhite}           >            <Text>{${playBackTime.hour}:${playBackTime.min}:${playBackTime.sec}</Text></Slider>

Hello @JuanMejiaC!
Please fill the missing information regarding new architecture and repro example.

I'll figure out the problem. The problem is the Text component inside the Slider. But again it works fine on ios but not so on Adroid.

@JuanMejiaC Placing children inside the Slider component was never intentionally supported - it means that by the nature of react-native it may work and it may work not on one version or another, or depending on the platform, or changes implemented to the Slider itself.
If you would like to solve the issue then checking the implementation of children handling on Android would be the first step. If you're willing to give it a try I can assign this issue to you.
Let me know.


Simplify Webpack configuration

Currently, Webpack configs are a bit confusing and spread across too many files and even different directories.
It would be nice to have just four files:

webpack-base.js
webpack-dev.js
webpack-prod.js
webpack-server.js

https://github.com/creativecommons/cccatalog-frontend/issues/485#issue-491818528

Hey! I am aware of webpack and what it does. But I am a little confused about how to start with this issue. Could you please help me to start off?

This might end up being a rewrite of the webpack config, because this is now related to #548, so it's better to work on that one while simplifying the current webpack configuration, which is currently a bit spread out in different files. If possible, it would be a lot better if we could end up with only: webpack.base.js, webpack.dev.js and ``webpack.prod.js`

If you suggest I can start working on #548 and while doing that I can take care of this.
Thank you.
Dhruvi.

Is this issue still open?

I would like to work on this.


Hey Cousin!

I was searching for a way to read bookshelf books on my kindle scribe and came across your repo. Made my day!! Based on your message in 2018, I am guessing you haven’t found a work-around.
Anyways, hope you are doing well.

Hey! Nope, I abandoned this after that 2018 note, sorry! Definitely didn't expect to run into you on GitHub of all places! 😄 Hope you're doing well too.


Add apiextensions s/v1beta1/v1/ to scheme

We were adding version v1beta1 of apiextensions to the scheme for the operator and manager. This is deprecated, and blows up if we try to run on OCP 4.9 where CRD v1beta1 has been removed. Cut over to version v1.

/lgtm

/hold
for testing

/lgtm

/hold cancel
works.


Misc fixes

See commits :) CC @ferjm

I've just realised i fixed the same clippy warnings in both PRs :D


Run time Error for Simulator Project hidden testcases

Summary
Facing run time error without error messages in Code Crunch for discrete event simulator project.
Description
Hello! I have been debugging my code pretty smoothly with the codecrunch test cases despite them being hidden, but today I got to a new hidden test case (test case 5_16) and it gave me a run time error (without telling me what it is, like the screenshot below)... Usually if it is an exception, it will be in the form of "Incorrect Output" and the error message will be shown (in the other sc below). Does anyone know how I should go about solving this? Thank you!!
Screenshots (if any):
current problem:

usual exceptions:

i think i found the problem!
used the 5_16 answer file in another issue and it says I have 5000 lines of incorrect output (i dont even know how that's possible) but that may have been the cause (maybe an overflow or smth)...

I'm not sure what's the issue, but this is the input for testcase 5_16: 2 3 2 10 1000 7.0 1.0 1.0 1.0 0.5 . Hope it helps!

I'm not sure what's the issue, but this is the input for testcase 5_16: 2 3 2 10 1000 7.0 1.0 1.0 1.0 0.5 . Hope it helps!

thank you so much! that helps my debugging a lot :o)


Automatically generate OpenAPI compliant documentation

Hi, do you have any plans on adding a new feature that adds support for generating OpenAPI compliant documentation out of the api configuration? It would be really nice to have!

Hi! Yes this is something I'd love the package to do at some point... but time is currently my enemy! I think the refactoring mentioned in #411 needs to be done first, because I don't think it would be sensible attempting to do the OpenAPI stuff now when I'm about to refactor the internals of the package.

I just saw #411 yes it makes sense for that to be done first. okay thank you for the response, and maintaining the project! awesome job 😁

Hi ! Any updates for generating a documentation for the API ? =D

No updates yet, it's dependent on implementing the new version of the package. I'm planning a sprint on that probably in November as I'm expecting to have a week or so between paid work.

The new package is now available - laravel-json-api/laravel see https://laraveljsonapi.io
I'd love to add Open API documentation generation.... but time is still a problem for me. If there are any volunteers who would like to step forward to take this on as a project, then that would be superb - just let me know.

I'm willing to have a go at this over the weekend

It won't be added to this package, as this package (cloudcreativity/laravel-json-api) is legacy. Still an intent to add it to the new package (laravel-json-api/laravel) but realistically someone needs to step up to contribute it as it's difficult for me to prioritise at the moment.

There is a WIP package working on a tool for this: https://github.com/swisnl/openapi-spec-generator

This can be accomplished/addressed with https://scribe.knuckles.wtf/laravel/documenting/custom-endpoints. It's not quick and easy but then none of the packages that help with API docs using docblocks and such are.
This one however allows you to create create YAML files (they can Camel files) and it will respect (not overwrite) and integrate those during doc generation/regeneration. It supports OPENAPI generation/creation and even has a nice JS based 'Try It' feature should you choose to enable it.
Those YAML files can be used to pretty easily document things like JSON:API endpoints or anything that does not follow the standard Laravel conventions for route/controllers/requests, and so on that most of the doc generators try to parse when generating docs.
Basically instead of the typical docblocks, you create one of these yaml files for each JSON:API 'group' of endpoints you want to document along with the details for each endpoint in that group. Then upon doc generation/regeneration, those get parsed instead of routes and/or docblocks to provide the docs and examples for the JSON:API endpoints. You can also use the other methods (routes/docblocks) for non-json:api endpoints in conjunction with your custom yaml files (in fact doc generation creates or appends to them to propagate them into the docs when you generate anyway).
I/we are not associated with Laravel Scribe at all and this is not a plug for it. It just happens to be what we landed on after a ton of trial and error trying to solve the exact issue of API documentation for the JSON:API or any other non-typical Laravel integrations.
Once you get familiar with the Scribe package and the foundation set up with everything the way you like it, we actually find much faster and easier then docblocks anyway.  There is a YAML file for each 'group'.


Fix ref directive title

Hey there 👋! I've noticed someone was too fast and forgot to rename the example title of "ref directive" 😀

Great launch event today by the way!

@googlebot I signed it!


How do I copy the content of the grid？

The contents of the Grid are not selected for replication. How should I manipulate the contents of a single td like the text on the HTML ?

Hi @Mingscript
you can copy content from grid use ALT key with seleced method (mouse or keyboard).
I hope that i help you.

do vitmalina died ?

The contents of the Grid are not selected for replication.

As @pidlo mentioned, you can hold the ALT key to select text. There's also a new column feature to add a "copy to clipboard" icon to all cells of a column.

How should I manipulate the contents of a single td

Use inline editing mode of the grid. Example: http://w2ui.com/web/demos/#!grid/grid-21
Closing issue as resolved.


GootLoaderAutoJsDecode passing directory as parameter?

Hello,
First of all, thanks for sharing this tool!
I would like to ask if there any way to execute the "GootLoaderAutoJsDecode.py" but instead of passing as parameter a particular file like "evil.js" giving a directory with multiple files like "/path/to/evilsamples" to try bulk deobfuscation?
Something like that:

Thanks in advance.
Regards.

That might be something I can add later on. For now you can replace line 485-end with this and it will print them all to console.
import os
directory = os.fsencode(args.jsFilePath)

for fileName in os.listdir(directory):
    filePath = os.path.join(directory, fileName)

    goot3detected = False

    gootDecode(filePath)

    if goot3detected:
        gootDecode('GootLoader3Stage2.js_')

Might not be perfect since DecodedJsPayload.js_, FileAndTaskData.txt, and GootLoader3Stage2.js_ will get constantly over written, but you'll at least get the C2.

That might be something I can add later on. For now you can replace line 485-end with this and it will print them all to console.
import os
directory = os.fsencode(args.jsFilePath)

for fileName in os.listdir(directory):
    filePath = os.path.join(directory, fileName)

    goot3detected = False

    gootDecode(filePath)

    if goot3detected:
        gootDecode('GootLoader3Stage2.js_')

Might not be perfect since DecodedJsPayload.js_, FileAndTaskData.txt, and GootLoader3Stage2.js_ will get constantly over written, but you'll at least get the C2.

Thanks for the feedback! I will be testing that :)


run labeler on this repo

lpass-cli is not working for Eliot or myself

lastpass-cli had a cert rotation a couple of weeks ago. uninstalling and reinstalling should fix issues.

When I ran this in terminal: lpass show hubot_github_token --notes, it gave me an error that it could not find specified accounts. However, on the lastpass application itself it seems I have access?

Nevermind, figured out that it was a "hide passwords" setting causing issues

Done!


Update deploy-vercel.md

adds an extra tip around setting up env vars in vercel, this issue took a while for me to resolve.

Thanks @bryce-larson


Why is [MemoryPackOrder] required?

I need to serialize with CircularReference option enabled. Now MemoryPack requires me to add [MemoryPackOrder] to all properties. I'm just wondering why this is necessary, and if there could be an option to omit these attributes?
The only reason I can think of is for version compatibility (so you can deserialize data that was serialized with a different "version" of the model, with different properties). In my case, this is not an issue as the serialization and deserialization will always be "in sync" with the same model. Could it be possible to add an opt-in to omit the order attributes, with the "risk" that deserialization will fail if the version was not in sync?
Alternatively, would it be possible to set the order in code via reflection, just like you can use the DynamicUnionFormatter for base classes to avoid Union attribute everywhere? For example - on initialize I can probably get a list of properties that have to be serialized, and then I can just register them all with an incrementing order number. When I add or remove properties later, the order will be different the next runtime, but I don't care because I guarantee that both sides are in sync anyway.
Not a big deal, just would be a nice addition for large models with many properties already defined. It's a chore to add all the orders and even bigger chore to later add more properties; it's easy to lose track of which numbers are already in use...

Ok, previously VersionTolerant/CircularReference automatically set as StructLayout.Explicit.
v1.9.11 supports explicit SerializeLayout.Sequential.
[MemoryPackable(GenerateType.CircularReference, SerializeLayout.Sequential)]
public partial class Foo


possible invalid share group values by setting external coupling nodes.

If I post the following configuration to the inputs of a scenario, the share group "coal_gas_ccus" is balanced on posting. However, this posts disables "share_of_energy_steel_energy_production_coal_gas" and "share_of_energy_steel_chemical_feedstock_coal_gas" and thus results in an invalid total share of zero percent for the share group. The hourly curves seem to be evaluating fine, so I'm not sure if this is intended behaviour or if something is going wrong here and an error message would be appropriate.
inputs : {'external_coupling_energy_distribution_coal_gas_chemical_feedstock_share': 0.0,
'external_coupling_energy_distribution_coal_gas_energy_production_share': 0.0,
'share_of_energy_steel_chemical_feedstock_coal_gas': 0.0,
'share_of_energy_steel_energy_production_coal_gas': 100.0}

To check for these type of errors before posting, it might be nice to be able to check if a parameter that is attempted to being set can be disabled by another parameter that is also being set. In the current inputs json I can only see the disabled_by parameter of parameters that are already disabled.

These input shouldn't be in the same group. To set the shares of the coal gas distribution node using external coupling, only the external coupling sliders should be used. The original sliders, share_of_energy_steel_chemical_feedstock_coal_gas and share_of_energy_steel_energy_production_coal_gas are then rightfully disabled.
Both of the external coupling inputs should therefore have their own group.

This issue is now fixed on master i.e., our beta server, and will be on production with the coming Deploy in the beginning of May.


Adding License info in README.md

Adding License section and related info in README.md of the project
e.g.

Created PR128 for the same
@elandau please review the issue and fix


MNT: Add collect_asset_docs to Mockflyer

Codecov Report

Merging #660 into master will decrease coverage by 0.18%.
The diff coverage is 38.09%.

@@            Coverage Diff             @@
##           master     #660      +/-   ##
==========================================
- Coverage   90.73%   90.54%   -0.19%
==========================================
  Files          60       60
  Lines        9236     9257      +21
==========================================
+ Hits         8380     8382       +2
- Misses        856      875      +19

Impacted Files
Coverage Δ

ophyd/sim.py
76.61% <38.09%> (-1.43%)
:arrow_down:

ophyd/ophydobj.py
96.93% <0%> (-1.85%)
:arrow_down:

ophyd/signal.py
95.6% <0%> (-0.36%)
:arrow_down:

ophyd/tests/test_signal.py
99.65% <0%> (-0.35%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 39811f6...194a6dc. Read the comment docs.


Arduino Due

Hi,
If I want to use an Arduino Due to communicate with a sensor that uses the SDI12 protocol, how should I connect them instead?
Thanks.

Hola @s-hicks2, con que arduino dices que te funciona sdi-12, con arduino ¿Due especificamente?

I'd assume the CS655 should work with the library.  I don't have one and don't know anything about it.
The Arduino Due, however, is not supported.  From what I've read the Due is not designed to be a low-power board and thus is not a good choice for long-term in-situ monitoring with environmental sensors.  It also doesn't seem to be a very popular board.  And I don't have one.  Those things, combined with the fact that the Due uses a different interrupt system than any of the boards that are supported, mean that Due is unlikely to ever be supported.

Hola Sara,
Gracias por responder
Matías
El mié., 22 de ago. de 2018 a la(s) 00:03, Sara Damiano (
<EMAIL_ADDRESS>escribió:

I'd assume the CS655 should work with the library. I don't have one and
don't know anything about it.
The Arduino Due, however, is not supported. From what I've read the Due is
not designed to be a low-power board and thus is not a good choice for
long-term in-situ monitoring with environmental sensors. It also doesn't
seem to be a very popular board. And I don't have one. Those things,
combined with the fact that the Due uses a different interrupt system than
any of the boards that are supported, mean that Due is unlikely to ever be
supported.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/EnviroDIY/Arduino-SDI-12/issues/9#issuecomment-414893314,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AG8HLIsVW4YsluRZMt8H7LG7TUxYp-Khks5uTMoRgaJpZM4CSrmv
.


Updated SecretManager from v1beta1 to v1

SecretManager is in GA since Q1 2020.

@gwhitehawk Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

@gwhitehawk Thank you for signing the Contributor License Agreement!

Thank you @gwhitehawk ! I just merged in #2638 so will close this one.


Bug: The route PropType doesn't support object routes

Using React Router 2.3.0. It appears that the route PropType doesn't support object routes.
Consider this:
import RouterPropTypes from 'react-router';

export default class MyComponent extends Component {
	static propTypes = {
		route: RouterPropTypes.route.isRequired
	};

	static contextTypes = {
		router: PropTypes.object.isRequired
	};

	render () {
		return <span />
	}
}

Will throw with:
Warning: Failed propType: MyComponent: prop type `route` is invalid; it must be a function, usually from React.PropTypes. Check the render method of `RouterContext`.
The route is defined as:
const routes = {
	childRoutes: [{
		getComponent: Another,
		path: "another"
	}],
	getComponent: (location, cb) => require(MyComponent)),
	path: "my_route"
}

And if I console log the this.props.route I see:
console.log(this.props.route);

{
	__id__: 1,
	getComponent: function MyComponent(location, cb),
	path: "my_route"
}

Looking at the source code for React Router, I see the proptype is defined as object or array, so I'm not sure where this error message is coming from.

import RouterPropTypes from 'react-router'; is not valid; there's no default export from react-router
The route prop type used as an internal prop type for validation; it doesn't refer to the route prop injected into a route component
These prop types are all deprecated anyway because of (2)


Copy vs Deepcopy in SAC

I'm using a previous version of SAC (with separate Q and V value function) but noticed something strange.
there is this line in twin_sac.py:
       self.target_vf = vf.copy()
Changing it to a deepcopy results in very different training curves (see curves attached):
        from copy import deepcopy
        self.target_vf = deepcopy(vf)

vf.copy() is defined as such:
  def copy(self):
       copy = Serializable.clone(self)
       ptu.copy_model_params_from_to(self, copy)
       return copy

It looks like it should do essentially the same thing as deepcopy, so what's causing the difference..

Yeah, that's really odd. Can you check if deepcopy copies the weight values as well?

@richardrl  Did you ever get around to seeing what's going on? Also, which curve is which?

I have not figured out why this is happening yet. The .copy() is the orange curve that actually trains. This is just on picknplace with the fetch robotics task.

Did you check if deepcopy copies the weights over (as reference or as value)?


Day 14: Add recipe ID route

To support dedicated recipe routes on the client side, I added a new endpoint to fetch a recipe by ID using spoonacular. Thankfully, the response can be built the same way as the random recipe endpoint, so that's less I need to configure for the web app. And it uses about the same quota from spoonacular.
I'm also glad I separated the server logic into several helper methods as that allowed me to reuse those methods and keep the code clean. 👍

Lesson learned: never trust user input!


Document why we don't require a repr for an enum to be FromZeroes

When deriving FromZeroes on an enum, and in the safety requirements in the doc comment for that trait, we don't require the enum to have an explicit repr attribute. The justification is that, regardless of repr, an enum's discriminant is well-defined according to the reference. We should document this justification in the FromZeroes docs like we do for not requiring a repr for that trait (or for FromBytes).
Is this actually sound?
Though I was initially convinced by this argument, I'm now second-guessing myself since, even if the discriminant's semantic value is well-defined without an explicit repr, it's less clear that the layout of an enum (ie, how its bits are used to represent that semantic value) is well-defined.
If it turns out that this is actually unsound, then of course we should change the behavior before releasing rather than documenting a justification for it.
cc @djkoloski

This is no longer the behavior of our derive.


Built-in Function: round

Built-in function for rounding numbers to given number of decimal places.
Two arguments: number, and number of decimal places.

Commit: https://github.com/johneastman/boomerang/commit/6ee26ade354e998fd248ffb2d94556c59cca6af4


Some renaissance perf targets do not work on Windows ARM

From Slack conversation with Joe Braley:
On Windows ARM64 / aarch64_windows, we are seeing some errors for perf suite of tests. The error we are receiving is hadoop\bin\hadoop.dll: Can't load AMD 64-bit .dll on a ARM 64-bit platform  for the following perf tests:
renaissance-als_0
renaissance-chi-square_0
renaissance-dec-tree_0
renaissance-gauss-mix_0
renaissance-log-regression_0
renaissance-movie-lens_0
We pull renaissance jar from https://github.com/adoptium/aqa-tests/blob/master/perf/renaissance/build.xml#L37
and the jar contains the hadoop.dll (https://github.com/renaissance-benchmarks/renaissance/blob/master/benchmarks/apache-spark/src/main/resources/hadoop.dll) which sounds like is not "Windows ARM" ready... in which case, may need to exclude those perf targets for that platform until that is addressed (upstream in the renaissance repo).

auto exclude test renaissance-als plat=aarch64_windows

auto exclude test renaissance-chi-square plat=aarch64_windows

auto exclude test renaissance-dec-tree plat=aarch64_windows

auto exclude test renaissance-gauss-mix plat=aarch64_windows

auto exclude test renaissance-log-regression plat=aarch64_windows

auto exclude test renaissance-movie-lens plat=aarch64_windows


📈 Instrument to track CPU/Mem usage & requests

@david-martin @aidenkeating Would you like to review? I haven't included the forked version of fh-mbaas-api. Should I?
Also, is this something that is expected to get merged back, or not? In any case, I don't think I have the ability to merge since I'm not in this org.

@grdryn Top notch! 👍  I haven't tested it though, but I am confident you have
I don't think it would hurt to include a forked version of fh-mbaas-api. It might give you insights into how sync factors in to raincatcher performance or something like that. If it doesn't, you can always remove it after. Although, probably not best to include it if you'll be merging these changes.
There are a lot of metrics forks and most have other changes, this has only the metrics changes aidenkeating/fh-mbaas-api#add_method_time_monitoring.

@grdryn Since this is using the instrumented fork of fh-mbaas-api, lets not merge back.
If you want to cherry-pick the changes for metrics only, I see no reason that couldn't be merged back given its behind a boolean flag

:+1: on this being merged since not enabled by default. Might be good to add docs for users if it would be useful for them.
Else it would still be nice to push this to a branch in the original repo instead of keeping it in a personal fork.
@nialldonnellyfh might be interested as well.

Yeah, the first two commits are probably mergeable, maybe not the 3rd one as @david-martin said. If @paolobueno and @nialldonnellyfh wish, I can open a separate PR for just those ones, and we can have that merged back?
I don't think I can add a branch to the origin repo for the rest, since I'm not a member of the feedhenry-raincatcher GitHub org.

pushed up the merged branch to upstream at RAINCATCH-566

@paolobueno ok, thanks. I'll close this PR in that case.


Set-up SASS codesniffer

@nacho87 has a gulp module to use.

Added.
@mitogh what do we need to do to make it run in Travis when pushing to github?

What rules are we following for this?

I think we need to define the rules before.

It's just using the defaults provided by the Gulp SASS Lint module for now. Agreed we'll need to update them as we work, same for the JS/Angular rules.

@adamf321 @mitogh i add some basic rules for test and works ok


[TypeError: this.pineconeClient.upsert is not a function]

Getting this.pineconeClient.upsert is not a function error when trying to read texts.
  const pineconeClient = new PineconeClient();

  await pineconeClient.init({
    environment: "environment",
    apiKey: "apiKey"
  });

  const db = await PineconeStore.fromTexts(
    texts.map((text) => text.pageContent),
    [],
    embeddings,
    {
      pineconeClient,
    }
  );

Packages versions
    "@pinecone-database/pinecone": "^0.0.10",
    "langchain": "^0.0.29",

Hi @munkhorgil What should be passed to pineconeClient is the pinecone index, ie. the return value of calling in your snippet pineconeClient.Index("my-index"). Will open a PR to clarify the argument name


Document multi component support

hi there! how is it going? I'm one of the very interested parties, both personally and as cabal maintainer. May I help perchance?

@Mikolaj Thanks for the ping!
@bgamari Are you satisfied with the changes?
@coot Sorry for the latency >_>

@coot One conflict to solve and I'll merge the PR :)

@Kleidukos thanks


Add external-dns with route53 support

[!warning]
This is a public repository, ensure not to disclose:

[x] personal data beyond what is necessary for interacting with this pull request, nor
[x] business confidential information, such as customer names.

What kind of PR is this?
Required: Mark one of the following that is applicable:

[x] kind/feature
[ ] kind/improvement
[ ] kind/deprecation
[ ] kind/documentation
[ ] kind/clean-up
[ ] kind/bug
[ ] kind/other

Optional: Mark one or more of the following that are applicable:

[!important]
Breaking changes should be marked kind/admin-change or kind/dev-change depending on type
Critical security fixes should be marked with kind/security

[ ] kind/admin-change
[ ] kind/dev-change
[ ] kind/security
[ ] kind/adr

Release notes
Adds external-dns as a service to manage dns records present in aws route53.

What does this PR do / why do we need this PR?
This PR adds external-dns with route53 support.
External-dns can mange dns records from kubernetes objects, such as service, ingress and DNSEndpoints.
If enabled, it will only manage dns records it owns, referenced by txtOwnerId.
So, it will not touch/take ownership of already created dns records.

Fixes #280

Information to reviewers
I will put this PR as draft, as I still need to look at the following tasks.
But it other then that, it works as intended.

[x] Look at adding appropriate network polices
[x] Look at adding migration script to allow external-dns to take over management of already existing dns records.

Checklist

[x] Proper commit message prefix on all commits

Change checks:

[ ] The change is transparent
[ ] The change is disruptive
[ ] The change requires no migration steps
[x] The change requires migration steps
[ ] The change upgrades CRDs

Metrics checks:

[ ] The metrics are still exposed and present in Grafana after the change
[ ] The metrics names didn't change (Grafana dashboards and Prometheus alerts are not affected)
[ ] The metrics names did change (Grafana dashboards and Prometheus alerts were fixed)

Logs checks:

[ ] The logs do not show any errors after the change

Pod Security Policy checks:

[x] Any changed pod is covered by Pod Security Admission
[ ] Any changed pod is covered by Gatekeeper Pod Security Policies
[x] The change does not cause any pods to be blocked by Pod Security Admission or Policies

Network Policy checks:

[x] Any changed pod is covered by Network Policies
[x] The change does not cause any dropped packets in the NetworkPolicy Dashboard

Audit checks:

[ ] The change does not cause any unnecessary Kubernetes audit events
[ ] The change requires changes to Kubernetes audit policy

Falco checks:

[ ] The change does not cause any alerts to be generated by Falco

Bug checks:

[ ] The bug fix is covered by regression tests

Config checks:

[ ] The schema was updated

Should also the main README section for DNS be updated to reflect this change?

I can take a look at it.

Is it possible to create a new environment and let external-dns handle the creation of all the necessary DNS records during deployment? Im thinking that this might require some dependencies in the Helmfile stack.

Im not really sure what you mean here :thinking: I have not tested if external-dns is installed at the correct moment for new clusters. But it should work for new and existing clusters, if the migration script it used.

Is it possible to create a new environment and let external-dns handle the creation of all the necessary DNS records during deployment? Im thinking that this might require some dependencies in the Helmfile stack.

Im not really sure what you mean here 🤔 I have not tested if external-dns is installed at the correct moment for new clusters. But it should work for new and existing clusters, if the migration script it used.

I was referring to the need field in the helmfile stack, so that if you have externalDNS enabled it should be, as an example, a need for dex to ensure that the domain record is created before it is deployed. Would that make sense?

Is it possible to create a new environment and let external-dns handle the creation of all the necessary DNS records during deployment? Im thinking that this might require some dependencies in the Helmfile stack.

Im not really sure what you mean here 🤔 I have not tested if external-dns is installed at the correct moment for new clusters. But it should work for new and existing clusters, if the migration script it used.

I was referring to the need field in the helmfile stack, so that if you have externalDNS enabled it should be, as an example, a need for dex to ensure that the domain record is created before it is deployed. Would that make sense?

Ah, yeah now i understand. Im not sure how the need hierarchy is planned. If we say dex needs external-dns, then it should also need ingress and cert-manager. But I can not see we have done those types of needs. @aarnq maybe have some input about this?

Is it possible to create a new environment and let external-dns handle the creation of all the necessary DNS records during deployment? Im thinking that this might require some dependencies in the Helmfile stack.

Im not really sure what you mean here 🤔 I have not tested if external-dns is installed at the correct moment for new clusters. But it should work for new and existing clusters, if the migration script it used.

I was referring to the need field in the helmfile stack, so that if you have externalDNS enabled it should be, as an example, a need for dex to ensure that the domain record is created before it is deployed. Would that make sense?

Ah, yeah now i understand. Im not sure how the need hierarchy is planned. If we say dex needs external-dns, then it should also need ingress and cert-manager. But I can not see we have done those types of needs. @aarnq maybe have some input about this?

@anders-elastisys I had a talk with André about needs, and it is meant to be so lean as possible. So you can deploy a service without needing a lot of other components. So I will not add more needs in this PR. :slightly_smiling_face:

Is it possible to create a new environment and let external-dns handle the creation of all the necessary DNS records during deployment? Im thinking that this might require some dependencies in the Helmfile stack.

Im not really sure what you mean here 🤔 I have not tested if external-dns is installed at the correct moment for new clusters. But it should work for new and existing clusters, if the migration script it used.

I was referring to the need field in the helmfile stack, so that if you have externalDNS enabled it should be, as an example, a need for dex to ensure that the domain record is created before it is deployed. Would that make sense?

Ah, yeah now i understand. Im not sure how the need hierarchy is planned. If we say dex needs external-dns, then it should also need ingress and cert-manager. But I can not see we have done those types of needs. @aarnq maybe have some input about this?

@anders-elastisys I had a talk with André about needs, and it is meant to be so lean as possible. So you can deploy a service without needing a lot of other components. So I will not add more needs in this PR. 🙂

Context in the last part of usage.
Also please update the local cluster support matrix if applicable :smile:

@anders-elastisys @aarnq I have now done the changes asked for and added some more comments. PTAL :slightly_smiling_face:


Ability to delete user

This is the front-end corresponding to the following pull request : https://github.com/oaeproject/Hilary/pull/1416 .

This change is

Check the comments. My overall notes are:

let's protect the delete archive user on the backend
let's make sure the archive user is created with confirmed email

both of these are backend related --'

Reviewed 4 of 4 files at r1.
Review status: all files reviewed, 7 unresolved discussions (waiting on @ClaireLozano)

node_modules/oae-admin/manageuser/manageuser.html, line 244 at r1 (raw file):

        </button>
    {/if}
    {if !userProfile.isUserArchive}

let's not forget that just because we are removing this interface, we still need to prevent this from happening on the API side of things... I don't care much about the other actions, but this one is important

shared/oae/bundles/email/default.properties, line 15 at r1 (raw file):

ACTIVITY_EMAIL_SUMMARY_IMMEDIATE_SINGLE_ACTOR_SINGLE_ACTIVITY = ${actorLink} has been active recently
ACTIVITY_EMAIL_SUMMARY_WEEKLY = Here's what happened in the last week
EMAIL_DELETED_RESOURCE_BODY = This/these resource(s) will be deleted :

why space before :?

shared/oae/bundles/email/default.properties, line 16 at r1 (raw file):

ACTIVITY_EMAIL_SUMMARY_WEEKLY = Here's what happened in the last week
EMAIL_DELETED_RESOURCE_BODY = This/these resource(s) will be deleted :
EMAIL_DELETED_RESOURCE_BODY_EXPLANATION = For now you can continue to access to this/these resource(s) but in ${nbMonth} month(s), the resource will definitively be removed from OAE. If you want to become manager of one of this resource(s), please contact the admin at the following email address : ${archiveEmail}.

...for now you may continue...
...removed from the system.
...manager of this/these resource(s),
again, why the space before :?

shared/oae/bundles/email/default.properties, line 17 at r1 (raw file):

EMAIL_DELETED_RESOURCE_BODY = This/these resource(s) will be deleted :
EMAIL_DELETED_RESOURCE_BODY_EXPLANATION = For now you can continue to access to this/these resource(s) but in ${nbMonth} month(s), the resource will definitively be removed from OAE. If you want to become manager of one of this resource(s), please contact the admin at the following email address : ${archiveEmail}.
EMAIL_LEFT_RESOURCE_BODY = This user has left the resource(s) :

here : as : well : )

shared/oae/bundles/email/default.properties, line 18 at r1 (raw file):

EMAIL_DELETED_RESOURCE_BODY_EXPLANATION = For now you can continue to access to this/these resource(s) but in ${nbMonth} month(s), the resource will definitively be removed from OAE. If you want to become manager of one of this resource(s), please contact the admin at the following email address : ${archiveEmail}.
EMAIL_LEFT_RESOURCE_BODY = This user has left the resource(s) :
EMAIL_SHARING_RESOURCE_BODY_EXPLANATION = ${userDeletedName} has deleted his account and you're actually sharing resource(s) together.

remove the actually.. maybe put it in the past too: USER has deleted his account and you have been sharing resources together.

shared/oae/bundles/email/default.properties, line 23 at r1 (raw file):

EMAIL_VERIFY_SUBJECT = Please verify your email address
EMAIL_VERIFY_VERIFY_MY_EMAIL = Verify my email
USER_REMOVED_SUBJECT = A user has left oae

has left the system.

shared/oae/bundles/email/fr_FR.properties, line 16 at r1 (raw file):

ACTIVITY_EMAIL_SUMMARY_IMMEDIATE_SINGLE_ACTOR_SINGLE_ACTIVITY=${actorLink} a \u00e9t\u00e9 actif/active r\u00e9cemment
ACTIVITY_EMAIL_SUMMARY_WEEKLY=Voici ce qui s'est pass\u00e9 la semaine derni\u00e8re
EMAIL_DELETED_RESOURCE_BODY=Ces ressources seront supprim\u00e9es :

same here and all the ones below : ok? : )

Comments from Reviewable

Closing.
See https://github.com/oaeproject/3akai-ux/pull/4224 for more information


pageing not working

Uncaught (in promise) TypeError: setting getter-only property "/dashboard"

show me error related code


⚠ ClamAV has degraded performance

In a8d4313, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 155 ms

Resolved: ClamAV performance has improved in ba14eb7.


Define the "standard" logging for bin/qq

Update the ad-hoc output I scripted initially by adding:

QQ::Formatters::MessageOutput class
QQ::Formatters::PipelineOutput class
QQ::Formatters::ExecutorOutput class

These should be the presentation layers, and support formatting these objects in different ways (colorizing, levels of verbosity/compactness.
Make the standard output from the tool (no arguments) look fairly compact, but still produce sufficient information to work from - something like this?
$ qq
## Rubocop: Passed
## Standardrb: Failed
  - foo/bar.rb:1-5   Foo/BarBazBam
  - foo/bar.rb:4      Bim/Bam
## Rspec: Passed

Passed: [Rubocop, Rspec], Failed: [Standardrb]

Importantly, the ExecutorOutput class shouldn't be handling the PipelineOutput strings - the Executor should receive a stream and options, and write the presentation onto the stream as appropriate. This allows the concurrent executor to write each outcome out (roughly) as it happens.

I think this has essentially been completed along the way. We could iterate on the format, but what we're printing already seems .. sufficient, for the most part.


chore(merge-back): 1.109.0

See CHANGELOG

AWS CodeBuild CI Report

CodeBuild project: codebuildgithubautobuildPro-fkVQbXRiQi6A
Commit ID: fc897001af49016f4758a18790f6ddae10fcb783
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


/preview and /crop with max_size returns different result

Problem description
I am using two titiler endpoint /crop and /preview with max_size parameter=1024
When I use /preview, I get result wih 1024 HEIGHT and computed width.
When I use /crop with whole raster extent, I get result with 1024 WIDTH and computed height
I can not use width and height parameters. I need max_size parameter.
Expected Output
In both cases I need to the same result with height=1024 and with computed width.
Is this bug or not?
Thank you for any help.

@fabry24 I'm not sure to understand.
can you provide the urls you're using?
If there is a bug it will be on rio-tiler side. you can easily confirm with

from rio_tiler.io import Reader

with Reader(your_file) as src:
    preview = src.preview(max_size=1024, width=None, height=None)
    crop = src.part(src.bounds, max_size=1024, width=None, height=None)

    print(preview.data.shape)
    print(crop.data.shape)

oooooh, I did bad analyse of my problem. results from titiler enpoints are OK, width and height computed from max_size are good. this issue can be closed.
thanks.


Speaker @NodeJS Event Africa, Nigeria

A call out to awesome Noders and CC_Members to volunteer for a remote talk... Thanks, @MylesBorins  for accepting to participate!!

Is there a time set? If the talk is done remotely as long as it does not conflict with other things on my scheduled I would participate.

Happy to help.

Thank you all for participating... the event starts by 10 am WAT - 4pm WAT

What day ?  Given the timezone differences 3PM would work best for me.
Are you looking for specific topics and how long are the talks going to be ?

10th March, 15mins, 20mins tops... More talks on collaboration and contribution. Getting started with NodeJS...

Awesome Myles!!!!

@mhdawson would like to have a flier too?

This is awesome I can't miss this

I would be glad to give a talk on behalf of the Node.js Foundation. I am trying to figure out my schedule that day so let me know if you'd like that and if so what time slot you'd like for me to do so. Great job pulling together this event!

@codeekage Do I have to travel down to PH?

@flickz  oh yes... lol!

@mrhinkle  Thanks a million

I could do sometime between 3-4 PM WAT, which I think translates to between 9-10AM for me.
Sounds like you want talks on getting started with Node.js.  I could put together a few slides on my recommendations of how to get started and then a quick overview of a few of the working groups and strategic initiatives as  good places to look at starting.   Node sure what @MylesBorins was planning to cover.

Alternatively if you want technical talks I could give one on N-API.

@codeekage I'd be happy to present as well, if you still need slots 👍

please, @mhdawson @bnb @mrhinkle @dshaw drop your twitter handles.
Thanks  a million

My twitter handle is @mhdawson1

My twitter handle is @bitandbang

Drum roll....... for NodeJS Event, the order of event!!!!!!

NodeJS Event order of event

Opennig speech by NodeSchoolPhc Organizers - 10:30am - 10:45am

Welcome video by NodeJS Foundation TSC Director Myles Borins - 10:45am - 10:50am

Setup and preparation for Workshop proper - 10:50 - 11:00

First mentoring section in JS by GDG Gloria - 11:00 am - 12:00pm

Second mentoring section in Node by NodeJS Africa Precious Dyke - 12:00 pm - 1:00pm

Third mentoring section in Node by NodeSchoolSF and NodersJS champion Fernando Larrañaga - 1pm - 2:30pm

Networking, Games and Lite Refreshment - 2:30 am - 3:00pm

TSC and Strategic Initiative by NodeJS Foundation TSC Chair Michael Dawson - 3:00 am - 3:20 pm

Collaboration and Open Source by NodeJS Africa and Tech Ph Precious Chunkundah - 3:20 pm - 3:40 pm

About NodeJS Foundation by Chair NodeJS Foundation Tereny - 3:40pm - 4:00pm

Closing speech by NodeJS Africa Hassan Sani - 4:00pm - 4:10pm

Photos.

All-time is UTC(WAT)

@bnb @MylesBorins @mhdawson @mrhinkle here is a zoom link to Port Harcourt NodeJs Event https://zoom.us/j/171884193

Thanks to @inidaname  @mhdawson @bnb for making the event a successful!


[progress-circle] show percentage in the middle of the circle

Bug, feature request, or proposal:
Feature request
What is the expected behavior?
Show the value inside the circle
What is the current behavior?
Does not show the percentage value
What are the steps to reproduce?
http://plnkr.co/edit/v0B4uU
What is the use-case or motivation for changing an existing behavior?
http://i.stack.imgur.com/DzNnN.png
Which versions of Angular, Material, OS, browsers are affected?
Angular 2+
Material 2.0.0-alpha.9
Is there anything else we should know?
No

Maybe just allow to print anything inside the circle, with ng-content only.

OK. @neuquino I manage to do a workaround with css. Just position: absolute; the section you want to show inside the spinner. Both must be inside a single div, and only them to be. Well, I have not tested out of my project, but I guess this should work, as the middle of the spinner is transparent.

plunkr is broken

Is anyone still working on this?

Closing this since there are no plans to add this feature.


Local image option

It would be interesting to have an option to choose if you want the images locally in the preferences.
Maybe something similar to what this plugin does would help:
https://github.com/niekcandaele/obsidian-local-images

Actually, I thought about that when creating this plugin. I just got time today and added the feature. 2d2e339a37913314f888591d659e58e71b409eb9
Now there's an option in the settings tab and you can choose the insert mode which can be remote or locally.
Remote means the image will not be downloaded and just use the unsplash image URL.
Locally means the image will be downloaded in the attachment folder, then insert it into the note.

Great!, thank you very much


⚠ SCOD has degraded performance

In 1b7337e, SCOD ($URL_SCOD) experienced degraded performance:

HTTP code: 200
Response time: 7414 ms

Resolved: SCOD performance has improved in 761867b after 567 days, 10 hours, 57 minutes.


PHPLIB-1183: Initial performance tests

PHPLIB-1183
This PR adds initial performance tests. For the time being, this tests the performance of MongoDB\BSON\Document and MongoDB\BSON\PackedArray, and runs comparative benchmarks as defined in the BSON benchmark spec.
The performance tests for the BSON classes could theoretically be moved to the PHPC repo, but would require setting up an entirely new composer project. For the time being the tests can live here, but they should be moved to PHPC when we start regression testing so that any performance regressions can be caught directly while doing the work.

But there is no automatic comparison with previous build ATM?

Not yet - there are a number of variables to control before making that viable, most notably ensuring that unrelated changes (e.g. newer server versions, operating systems, etc.) don't affect results. We'll also have to define metrics and add assertions to then make the build fail.


Add CLI option to exclude specific schema/custom tests

Please make sure to fill out either the issue template or the feature template and delete the other one!
Feature
Add CLI option to exclude specific schema/custom tests
Feature description
Currently dbt does not allow using tags with tests. Suggest adding functionality to tag tests like model to allow using exclude in dbt test.
Also recommend modifying existing dbt compile output behaviour to generate names (and scripts) of tests in folder target so users can access those names and use exclude on them more easily.
Who will this benefit?
Some tests take significant time to complete and adding tags functionality would give users more flexibility with test scheduling.

Thanks for making this issue @jmak123! Check out this semi-related discussion: https://discourse.getdbt.com/t/pre-proposal-column-tags/433/4

Fixed in #2039 - added tags on columns (which push down to all tests for a given column) and tests themselves.


update drone

Coverage increased (+0.2%) to 66.57% when pulling c85ad361cb99e857d904c16f4e0c892a08162a7d on wy65701436:drone-test into e07c6e470e98e514afa141589b816146c970588f on vmware:drone.


Different results depending on how the input is "chunked"

https://github.com/image-rs/fdeflate/pull/24 demonstrates that for some inputs fdeflate::Decompressor may produce a different result (e.g. Ok(_) vs Err(_)) depending on how the input is chunked (e.g. a single read that passes the whole input may produce Ok(_) but calling read for each byte of input may produce Err(_)).  AFAIU this difference is unexpected and needs to be fixed within the fdeflate crate.

I've updated https://github.com/image-rs/fdeflate/pull/24 with a minimized test case - only 207 bytes of input.

Do you know what the specific error being returned is? What you're describing is a bug in fdeflate, but the specific error code should give a good hint about what's going wrong

Still looking, but what I got so far is (the "state=..."

Ok, I think I see see the issue. Before returning InvalidDistanceCode, we need to check whether there's enough bits in the buffer and break out of the loop if not

RE: https://github.com/image-rs/fdeflate/issues/25#issuecomment-2353813427

Ok, I think I see see the issue. Before returning InvalidDistanceCode, we need to check whether there's enough bits in the buffer and break out of the loop if not

Thank you very much for taking a look - I really appreciate it!  (I still struggle to understand and follow how fdeflate works.  Hopefully I'll learn as I work with this source code more.)  Could you please put together a commit with a fix (either combining it with https://github.com/image-rs/fdeflate/pull/24 or putting together a separate PR - either way is probably fine as long as we include the new test case from my PR).
Otherwise, I may try to understand the code (and the possible fix) tomorrow or Wednesday, but currently I am only able to dedicate a few hours a day to my png / fdeflate project...

RE: https://github.com/image-rs/fdeflate/issues/25#issuecomment-2353710545
Quick correction/clarification.  I previously said:

left: Err(BadZlibHeader)

This is artificial - I've inserted if output_index > 3 { return Err(DecompressionError::BadZlibHeader); } into fn read to truncate the log output from the "whole" case (since the "byte-by-byte" case returns InvalidDistanceCode at output_index = 3).
In the unmodified code, the "whole" testcase returns Ok(_).

I can put together a fix. That part of the code is quite subtle

I've refreshed https://github.com/image-rs/fdeflate/pull/24 with a new failing input.
This is the original output:
---- decompress::tests::test_input_chunking_sensitivity_example2 stdout ----
WHOLE ======================
state=ZlibHeader, output_index=0, nbits=0
state=BlockHeader, output_index=0, nbits=40
state=CodeLengthCodes, output_index=0, nbits=39
state=CodeLengths, output_index=0, nbits=57
state=CompressedData, output_index=0, nbits=58
[src/decompress.rs:778:17] dist = 3
[src/decompress.rs:779:17] output_index = 0
[src/decompress.rs:780:17] self.nbits = 58
[src/decompress.rs:781:17] total_bits = 19
BYTE-BY-BYTE ======================
state=ZlibHeader, output_index=0, nbits=0
state=ZlibHeader, output_index=0, nbits=8
state=BlockHeader, output_index=0, nbits=0
state=BlockHeader, output_index=0, nbits=0
state=BlockHeader, output_index=0, nbits=8
state=BlockHeader, output_index=0, nbits=16
state=CodeLengthCodes, output_index=0, nbits=7
state=CodeLengthCodes, output_index=0, nbits=7
state=CodeLengthCodes, output_index=0, nbits=15
state=CodeLengthCodes, output_index=0, nbits=23
state=CodeLengthCodes, output_index=0, nbits=31
state=CodeLengthCodes, output_index=0, nbits=39
state=CodeLengthCodes, output_index=0, nbits=47
state=CodeLengths, output_index=0, nbits=1
state=CodeLengths, output_index=0, nbits=1
state=CodeLengths, output_index=0, nbits=6
state=CodeLengths, output_index=0, nbits=3
state=CodeLengths, output_index=0, nbits=5
state=CodeLengths, output_index=0, nbits=5
state=CodeLengths, output_index=0, nbits=2
state=CodeLengths, output_index=0, nbits=6
state=CodeLengths, output_index=0, nbits=8
state=CodeLengths, output_index=0, nbits=5
state=CodeLengths, output_index=0, nbits=2
state=CodeLengths, output_index=0, nbits=8
state=CodeLengths, output_index=0, nbits=5
state=CodeLengths, output_index=0, nbits=2
state=CodeLengths, output_index=0, nbits=6
state=CodeLengths, output_index=0, nbits=6
state=CodeLengths, output_index=0, nbits=3
state=CodeLengths, output_index=0, nbits=4
state=CodeLengths, output_index=0, nbits=6
state=CodeLengths, output_index=0, nbits=6
state=CodeLengths, output_index=0, nbits=6
state=CodeLengths, output_index=0, nbits=5
state=CompressedData, output_index=0, nbits=2
state=CompressedData, output_index=0, nbits=2
state=CompressedData, output_index=0, nbits=10
state=BlockHeader, output_index=0, nbits=18
state=BlockHeader, output_index=0, nbits=18
state=BlockHeader, output_index=0, nbits=26
state=BlockHeader, output_index=0, nbits=34
state=UncompressedData, output_index=0, nbits=0
state=Checksum, output_index=0, nbits=0
state=Checksum, output_index=0, nbits=0
state=Checksum, output_index=0, nbits=8
state=Checksum, output_index=0, nbits=16
state=Checksum, output_index=0, nbits=24
ASSERTING SAME RESULT ======================
thread 'decompress::tests::test_input_chunking_sensitivity_example2' panicked at src/decompress.rs:1295:9:
assertion `left == right` failed
  left: Err(ProdError(DistanceTooFarBack))
 right: Ok([])

And this is what I get when I replace a line in fn read_compressed with peak_bits which asserts that we have that many self.nbits in self.buffer (see here):
BYTE-BY-BYTE ======================
...
state=CodeLengths, output_index=0, nbits=5
state=CompressedData, output_index=0, nbits=2
thread 'decompress::tests::test_input_chunking_sensitivity_example2' panicked at src/decompress.rs:209:9:
assertion failed: nbits <= 56 && nbits <= self.nbits

One thought that I had was that maybe buffer + nbits could maybe be encapsulated into a separate struct (let's call it BitBuffer which would have accessors/methods that ensure that "uninitialized" bits are not read.  One motivation was that I wanted to debug_assert! before dist_table[...bits & 0x1ff...] that we have all the bits, but this is difficult because self.nbits is no longer relevant because of earlier bits >>= litlen_code_bits and bits >>= length_extra_bits.  OTOH, if bits was a BitBuffer with its own >>= implementation then it could maintain nbits of bits separately from self.nbits / self.buffer.  OTOH some of this is only for debug_assert! so maybe this is too heavy-weight here?  I dunno...

I think that maybe fn read_compressed is pre-maturely advancing to the State::BlockHeader state.  For example, we may break because we don't have enough bits for the distance code - in this situation it seems incorrect (but this is what I am observing) to go to the State::BlockHeader state.
At least, what I have in https://github.com/image-rs/fdeflate/pull/24/commits/18d403d74266cb6385e4b7a68e07f29aa8d797e3 and https://github.com/image-rs/fdeflate/pull/24/commits/3bfe707ad975eb4ed5622efc3713b31fc2038d22 works ("works" = the new test passes).

I'll switch to something else today, but it would help me resume tomorrow if you could please provide feedback on:

Desirability of introducing a BitBuffer struct sketched in https://github.com/image-rs/fdeflate/issues/25#issuecomment-2356628449 (with >>=, fill(input), peek(n), etc)
Desirability of changing break to return Ok(output_index) for cases where we used to break out of the loop because of too-little-nbits

I don't like the code duplication return Ok(output_index) in a few places.  But I don't quite see how to do better.

Overall fix direction (I feel that I am starting to understand this code better, but I am still not quite confident in what I am doing... :-( ).

In this new failing input both versions are producing the wrong result. The input is actually corrupt because the first block header doesn't define any end-of-block symbol which zlib considers to be a invalid.
Since build_tables is missing a check for this case, it ends up incorrectly initializing eof_mask to all zeros. Which has the effect of treating any sequence of bits as an end-of-block, provided we don't notice some other corruption first. The "whole" version notices the corruption first while the "byte-by-byte" hallucinates an eof first, leading to the different output.

Desirability of introducing a BitBuffer struct sketched in https://github.com/image-rs/fdeflate/issues/25#issuecomment-2356628449 (with >>=, fill(input), peek(n), etc)

I don't think this would work. There's a couple different things are going on here...

We intentionally construct the decoding tables so that it is safe to index them with uninitialized bits. For instance, if there's a 8-bit symbol 1010_1010 we'll actually insert entries for it all indexes xxxx_1010_1010 so we'll get the right entry regardless what other four bits are used in the lookup.

The read_compressed method needs to be able to make forward progress when there are as few as 34-bits available, because that's how many bits are required to encode a 1-bit literal, followed by a 1-bit end-of-block symbol, followed by a 32-bit checksum. At the same time, the maximum length-distance pair takes 48-bits: 15-bit length + 13 extra-bits + 15-bit distance + 5 extra-bits.

In the fast-paths we don't want to check nbits multiple times in the same loop iteration. Those branches would be easy to predict (in normal operation nbits is almost always >=56), but they'd still add a couple instructions to the hot part of the loop where every cycle matters. Because of (2), the check can only be once we know precisely how many bits are needed.

Net result is that I don't think meaningful additional checks / debug asserts could be added because they would conflict with how the logic currently works.

Desirability of changing break to return Ok(output_index) for cases where we used to break out of the loop because of too-little-nbits

The main point of this would be to skip the end-of-block check in cases where we know that either (a) the next symbol cannot be an end-of-block symbol, or (b) if the next symbol is an end-of-block symbol, then read will be called again because we haven't been provided the full deflate stream yet.
I think that is a valid optimization, but the arguments involved are quite subtle, so we'd have to think through them for each break.

Thank you very much for the thorough reply.  This is very helpful.  I appreciate your time.

In this new failing input both versions are producing the wrong result. The input is actually corrupt because the first block header doesn't define any end-of-block symbol which zlib considers to be invalid.
Since build_tables is missing a check for this case, it ends up incorrectly initializing eof_mask to all zeros. Which has the effect of treating any sequence of bits as an end-of-block, provided we don't notice some other corruption first. The "whole" version notices the corruption first while the "byte-by-byte" hallucinates an eof first, leading to the different output.

Oh, I see.  That makes sense.
I should have looked closed at why exactly we move to the next block (because self.peak_bits(15) as u16 & self.compression.eof_mask == self.compression.eof_code is always true for empty eof_mask) instead of guessing that this is because the final checks shouldn't happen when breaking because of insufficient nbits.
I've tried putting a fix for this at https://github.com/image-rs/fdeflate/pull/28

Desirability of introducing a BitBuffer struct sketched in #25 (comment) (with >>=, fill(input), peek(n), etc)

I don't think this would work. There's a couple different things are going on here...

We intentionally construct the decoding tables so that it is safe to index them with uninitialized bits. For instance, if there's a 8-bit symbol 1010_1010 we'll actually insert entries for it all indexes xxxx_1010_1010 so we'll get the right entry regardless what other four bits are used in the lookup.

I see.
I note that we still check self.nbits < litlen_code_bits a bit further down, so 1) uninitialized bits may be used in a table lookup, but 2) (because of the self.nbits < litlen_code_bits check) uninitialized bits are never relied on for producing decompressed output.

The read_compressed method needs to be able to make forward progress when there are as few as 34-bits available, because that's how many bits are required to encode a 1-bit literal, followed by a 1-bit end-of-block symbol, followed by a 32-bit checksum. At the same time, the maximum length-distance pair takes 48-bits: 15-bit length + 13 extra-bits + 15-bit distance + 5 extra-bits.
In the fast-paths we don't want to check nbits multiple times in the same loop iteration. Those branches would be easy to predict (in normal operation nbits is almost always >=56), but they'd still add a couple instructions to the hot part of the loop where every cycle matters. Because of (2), the check can only be once we know precisely how many bits are needed.

Net result is that I don't think meaningful additional checks / debug asserts could be added because they would conflict with how the logic currently works.

Desirability of changing break to return Ok(output_index) for cases where we used to break out of the loop because of too-little-nbits

The main point of this would be to skip the end-of-block check in cases where we know that either (a) the next symbol cannot be an end-of-block symbol, or (b) if the next symbol is an end-of-block symbol, then read will be called again because we haven't been provided the full deflate stream yet.
I think that is a valid optimization, but the arguments involved are quite subtle, so we'd have to think through them for each break.

Ack.  And it seems that return Ok(output_index) is not needed for correctness, because (as you point out) we wouldn't have advanced to the next block if the eof_mask was right.

With #26 and #27 both merged, I'm going to go ahead and close this. Let's create new issues if any more chunking bugs are found. I do want to land the tests/fuzzing PRs as well, but might take me a bit to review