Balances integration test

P2P message exchange in bee is incentivized within some protocols (retrieval and pushsync). The first step is to have accounting between peers in order to trigger a settlement between each of them. Currently every node holds balances with every node that it exchanged incentivized messages. Debug API contains balances endpoint which provide information that can be used to validate the accounting.
This Beekeeper test should validate the accounting on the high level, checking only the symmetry of balances between all peer pairs. This test does not validate a specific retrieval or pushsync message accounting, or any other protocol message exchange that may be incentivized. Such test may exist, and can be added later, but it would be highly coupled with other protocols. For every incentivized protocol, a separate balances test can be made.
Scenario
The test should be performed on a clean or a cluster that already has some state. In both cases balances should be symmetric.
Options:

number of nodes in the cluster
file size
number of upload/download iterations to perform

Steps

Upload a file with random data to a randomly selected node in the isolated cluster (trigger pushsync protocol).
Validate the balances.
Download a file from a randomly selected node (trigger retrieval protocol).
Validate the balances.
Go to 1. unless the configured number of uploads is reached.

Balance validation
Balance validation should be done in a loop until it stabilizes as protocols may be still exchanging messages after the upload (pushsync). It is good to validate that the balances are being dynamically updated, instead to use other mechanisms, like tags, to wait for pushsync to finish.

Get balances from every node in the network.
Validate that there are balances and that they are not equal to 0. If not, wait for a short time and go to 1. until a retry count limit is reached.
Compare that every node pair has a symmetric balance (zero sum). If not, wait for a short time and go to 1. until a retry count limit is reached.
Compare if all balances are the same as in the previous iteration (they have stabilized) if it is not the first iteration. If they are, exit the loop.
Store balances for the stability check in step 3. for the next iteration.
Wait for some time (2-3 seconds), go to 1.

Beekeeper v0.2.20 can be used for testing balances: beekeeper check balances
Executing it with options --dry-run will just validate balances without uploading/downloading a a file.
Right now it seems like there is an issue with cluster balances. After uploading one file to the cluster, balances never reached symmetric state. Here is check printout from the cluster in the namespace svetomir:
./dist/beekeeper check balances --dry-run Node ef3e7035fca1abe1dfd2a4156b950f914357a4ee84af39f0ced9f12dc126f690 has balance 3870 with peer 0a1641d655548ca9c53d5099c83014f74077a5971bad3b0e279c022623015d33 Peer 0a1641d655548ca9c53d5099c83014f74077a5971bad3b0e279c022623015d33 has balance -3780 with node ef3e7035fca1abe1dfd2a4156b950f914357a4ee84af39f0ced9f12dc126f690 Difference: 90 Node 0a1641d655548ca9c53d5099c83014f74077a5971bad3b0e279c022623015d33 has balance -5590 with peer 1751cd48bc65c84ddae5f6ad661f1e6edf26a0fb8fc4366c0730e49e454ed343 Peer 1751cd48bc65c84ddae5f6ad661f1e6edf26a0fb8fc4366c0730e49e454ed343 has balance 6150 with node 0a1641d655548ca9c53d5099c83014f74077a5971bad3b0e279c022623015d33 Difference: 560 Node 0a1641d655548ca9c53d5099c83014f74077a5971bad3b0e279c022623015d33 has balance -9630 with peer c19d8bcb3402ef4d3a2dcdea9dfd010d9093ca0dc7217a532da56837bdd9e9ab Peer c19d8bcb3402ef4d3a2dcdea9dfd010d9093ca0dc7217a532da56837bdd9e9ab has balance 10890 with node 0a1641d655548ca9c53d5099c83014f74077a5971bad3b0e279c022623015d33 Difference: 1260 Node 0a1641d655548ca9c53d5099c83014f74077a5971bad3b0e279c022623015d33 has balance -3330 with peer e0aa5b66c2352b4697bfa75e0f17fe690f37a2e5bef3d77718e2c33fb2c0c04a Peer e0aa5b66c2352b4697bfa75e0f17fe690f37a2e5bef3d77718e2c33fb2c0c04a has balance 3470 with node 0a1641d655548ca9c53d5099c83014f74077a5971bad3b0e279c022623015d33 Difference: 140 Node 0a1641d655548ca9c53d5099c83014f74077a5971bad3b0e279c022623015d33 has balance -3780 with peer ef3e7035fca1abe1dfd2a4156b950f914357a4ee84af39f0ced9f12dc126f690 Peer ef3e7035fca1abe1dfd2a4156b950f914357a4ee84af39f0ced9f12dc126f690 has balance 3870 with node 0a1641d655548ca9c53d5099c83014f74077a5971bad3b0e279c022623015d33 Difference: 90 Node 1751cd48bc65c84ddae5f6ad661f1e6edf26a0fb8fc4366c0730e49e454ed343 has balance 6150 with peer 0a1641d655548ca9c53d5099c83014f74077a5971bad3b0e279c022623015d33 Peer 0a1641d655548ca9c53d5099c83014f74077a5971bad3b0e279c022623015d33 has balance -5590 with node 1751cd48bc65c84ddae5f6ad661f1e6edf26a0fb8fc4366c0730e49e454ed343 Difference: 560 Node c19d8bcb3402ef4d3a2dcdea9dfd010d9093ca0dc7217a532da56837bdd9e9ab has balance 10890 with peer 0a1641d655548ca9c53d5099c83014f74077a5971bad3b0e279c022623015d33 Peer 0a1641d655548ca9c53d5099c83014f74077a5971bad3b0e279c022623015d33 has balance -9630 with node c19d8bcb3402ef4d3a2dcdea9dfd010d9093ca0dc7217a532da56837bdd9e9ab Difference: 1260 Node e0aa5b66c2352b4697bfa75e0f17fe690f37a2e5bef3d77718e2c33fb2c0c04a has balance 3470 with peer 0a1641d655548ca9c53d5099c83014f74077a5971bad3b0e279c022623015d33 Peer 0a1641d655548ca9c53d5099c83014f74077a5971bad3b0e279c022623015d33 has balance -3330 with node e0aa5b66c2352b4697bfa75e0f17fe690f37a2e5bef3d77718e2c33fb2c0c04a Difference: 140 Error: invalid balances

Thanks @svetomir. I assumed that balances should be symmetric between peer pairs on upload or download. In this case only one upload happened, pushsync pushes the chunks and peers balances should be changed symmetrically from the uploader to the peers that is pushed to. In this case the uploader is 0a1641d655548ca9c53d5099c83014f74077a5971bad3b0e279c022623015d33 and all other peers should have a balance with it, as they do, but not symmetrical. Is there something else that we are missing @ralph-pichler?

@svetomir hm, that does not look good. Seems the difference is about 1-3 chunks worth. Not sure what could cause this, will try to replicate and investigate.

ok, I tried this with setting the push sync prices to 0 and it's valid balances, so at least retrieval works and imbalance comes from push sync.

disabling the delivery callback fixes this, so there seems to be some weird effect between the handler not found error from pss and accounting.

Merged


Documentation webpage doesn't work

Hi, dear partners.
Today I visit the documentation webpage and currently doesn't work. I verified with other devices and still down.

Sorry. There's not issue, the problem is my network.


Compile failed

when I try to compile https://compile.jsgo.io/dave/html2vecty
I find no buildable Go source files in goroot/src/syscall/js

Hi, this is fixed now.


training 12b model seems to require more memory than expected

Describe the bug
Hi, I was trying to finetune  pythia-12b model via the following code in DeepSpeed-Chat 's step1 code.
main.py is from DeepSpeed/DeepSpeedExamples/applications/DeepSpeed-Chat/training/step1_supervised_finetuning/main.py
deepspeed main.py \
   --sft_only_data_path {my_dataset} \
   --data_split 10,0,0 \
   --model_name_or_path EleutherAI/pythia-12b-deduped \
   --per_device_train_batch_size 1 \
   --per_device_eval_batch_size 1 \
   --max_seq_len 512 \
   --learning_rate 9.65e-6 \
   --weight_decay 0. \
   --num_train_epochs 16  \
   --gradient_accumulation_steps 8 \
   --lr_scheduler_type cosine \
   --num_warmup_steps 0 \
   --seed 1234 \
   --lora_dim 64 \
   --gradient_checkpointing \
   --zero_stage 3 \
   --deepspeed \
   --output_dir $OUTPUT_PATH \
   &> $OUTPUT_PATH/training.log

and according to zero3's estimation, the model finetuning should only take resources like
Some weights of the model checkpoint at EleutherAI/pythia-12b-deduped were not used when initializing GPTNeoXModel: ['embed_out.weight']
Estimated memory needed for params, optim states and gradients for a:
HW: Setup with 1 node, 8 GPUs per node.
SW: Model with 11586M total params, 259M largest layer params.
  per CPU  |  per GPU |   Options
  291.35GB |   0.97GB | offload_param=cpu , offload_optimizer=cpu , zero_init=1
  517.96GB |   0.97GB | offload_param=cpu , offload_optimizer=cpu , zero_init=0
  258.98GB |   3.66GB | offload_param=none, offload_optimizer=cpu , zero_init=1
  517.96GB |   3.66GB | offload_param=none, offload_optimizer=cpu , zero_init=0
   11.60GB |  25.25GB | offload_param=none, offload_optimizer=none, zero_init=1
  517.96GB |  25.25GB | offload_param=none, offload_optimizer=none, zero_init=0

since I have 720 cpu RAM and 8* 32GB V100 GPU totally, this spec looks sufficient enough to run even with such small batch size (only 1 now)
but I still got OOM error, and memory occupied rate up to almost 100% (30GB~31GB/32GB) for each gpu. Any idea why it consume so much memory?
other alternative I try to deal with OOM: change --gradient_checkpointing into --only_optimize_lora:
but this resulted in index error, another bug I guess.
the error message of this is quite long, I'll paste it in the additional context.
To Reproduce
Steps to reproduce the behavior:
just run the code with the above  environment setting and code.
Expected behavior
Should train succesfully.
ds_report output
--------------------------------------------------
DeepSpeed C++/CUDA extension op report
--------------------------------------------------
NOTE: Ops not installed will be just-in-time (JIT) compiled at
      runtime if needed. Op compatibility means that your system
      meet the required dependencies to JIT install the op.
--------------------------------------------------
JIT compiled ops requires ninja
ninja .................. [OKAY]
--------------------------------------------------
op name ................ installed .. compatible
--------------------------------------------------
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  async_io: please install the libaio-dev package with apt
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
async_io ............... [NO] ....... [NO]
cpu_adagrad ............ [NO] ....... [OKAY]
cpu_adam ............... [NO] ....... [OKAY]
fused_adam ............. [NO] ....... [OKAY]
fused_lamb ............. [NO] ....... [OKAY]
quantizer .............. [NO] ....... [OKAY]
random_ltd ............. [NO] ....... [OKAY]
 [WARNING]  using untested triton version (2.0.0), only 1.0.0 is known to be compatible
sparse_attn ............ [NO] ....... [NO]
spatial_inference ...... [NO] ....... [OKAY]
transformer ............ [NO] ....... [OKAY]
stochastic_transformer . [NO] ....... [OKAY]
transformer_inference .. [NO] ....... [OKAY]
utils .................. [NO] ....... [OKAY]
--------------------------------------------------
DeepSpeed general environment info:
torch install path ............... ['/usr/local/lib/python3.8/dist-packages/torch']
torch version .................... 1.14.0a0+44dac51
deepspeed install path ........... ['/home/twsfphn198/.local/lib/python3.8/site-packages/deepspeed']
deepspeed info ................... 0.9.1, unknown, unknown
torch cuda version ............... 12.0
torch hip version ................ None
nvcc version ..................... 12.0
deepspeed wheel compiled w. ...... torch 1.14, cuda 12.0

Screenshots
If applicable, add screenshots to help explain your problem.
System info (please complete the following information):

OS: "Ubuntu" VERSION="20.04.5 LTS (Focal Fossa)"
GPU count and types :one machine  with 8 32GB V100
Python version: python 3.8.10

Launcher context
Are you launching your experiment with the deepspeed launcher, MPI, or something else?
yes, with deepspeed
Docker context
Are you using a specific docker image that you can share? No.
Additional context
index error I try to use --only_optimize_lora instead of --gradient_checkpointing
the message is long, so I paste the latter part of it and remove the repeated message from other subthreads, if you need the whole message, please tell me!
[2023-04-26 15:22:51,673] [INFO] [utils.py:785:see_memory_usage] Stage 3 initialize beginning
[2023-04-26 15:22:51,673] [INFO] [utils.py:786:see_memory_usage] MA 3.86 GB         Max_MA 4.83 GB         CA 11.34 GB         Max_CA 11 GB
[2023-04-26 15:22:51,674] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory:  used = 110.99 GB, percent = 14.7%
[2023-04-26 15:22:51,676] [INFO] [stage3.py:113:__init__] Reduce bucket size 500,000,000
[2023-04-26 15:22:51,676] [INFO] [stage3.py:114:__init__] Prefetch bucket size 30000000
Using /home/twsfphn198/.cache/torch_extensions/py38_cu120 as PyTorch extensions root...
ninja: no work to do.
Loading extension module utils...
Time to load utils op: 0.5340464115142822 seconds
Traceback (most recent call last):
  File "main.py", line 345, in <module>
    main()
  File "main.py", line 290, in main
    model, optimizer, _, lr_scheduler = deepspeed.initialize(
  File "/home/twsfphn198/.local/lib/python3.8/site-packages/deepspeed/__init__.py", line 165, in initialize
    engine = DeepSpeedEngine(args=args,
  File "/home/twsfphn198/.local/lib/python3.8/site-packages/deepspeed/runtime/engine.py", line 308, in __init__
Loading extension module utils...
    self._configure_optimizer(optimizer, model_parameters)
  File "/home/twsfphn198/.local/lib/python3.8/site-packages/deepspeed/runtime/engine.py", line 1167, in _configure_optimizer
    self.optimizer = self._configure_zero_optimizer(basic_optimizer)
  File "/home/twsfphn198/.local/lib/python3.8/site-packages/deepspeed/runtime/engine.py", line 1445, in _configure_zero_optimizer
Time to load utils op: 0.10356998443603516 seconds
Traceback (most recent call last):
      File "main.py", line 345, in <module>
optimizer = DeepSpeedZeroOptimizer_Stage3(
  File "/home/twsfphn198/.local/lib/python3.8/site-packages/deepspeed/runtime/zero/stage3.py", line 133, in __init__
    self.dtype = self.optimizer.param_groups[0]['params'][0].dtype
IndexError: list index out of range
    main()
  File "main.py", line 290, in main
    model, optimizer, _, lr_scheduler = deepspeed.initialize(
  File "/home/twsfphn198/.local/lib/python3.8/site-packages/deepspeed/__init__.py", line 165, in initialize
    engine = DeepSpeedEngine(args=args,
  File "/home/twsfphn198/.local/lib/python3.8/site-packages/deepspeed/runtime/engine.py", line 308, in __init__
    self._configure_optimizer(optimizer, model_parameters)
  File "/home/twsfphn198/.local/lib/python3.8/site-packages/deepspeed/runtime/engine.py", line 1167, in _configure_optimizer
    self.optimizer = self._configure_zero_optimizer(basic_optimizer)
  File "/home/twsfphn198/.local/lib/python3.8/site-packages/deepspeed/runtime/engine.py", line 1445, in _configure_zero_optimizer
    optimizer = DeepSpeedZeroOptimizer_Stage3(
  File "/home/twsfphn198/.local/lib/python3.8/site-packages/deepspeed/runtime/zero/stage3.py", line 133, in __init__
    self.dtype = self.optimizer.param_groups[0]['params'][0].dtype
IndexError: list index out of range

[2023-04-26 15:22:59,098] [INFO] [launch.py:428:sigkill_handler] Killing subprocess 9668
[2023-04-26 15:22:59,102] [INFO] [launch.py:428:sigkill_handler] Killing subprocess 9669
[2023-04-26 15:22:59,104] [INFO] [launch.py:428:sigkill_handler] Killing subprocess 9670
[2023-04-26 15:22:59,318] [INFO] [launch.py:428:sigkill_handler] Killing subprocess 9671
[2023-04-26 15:22:59,320] [INFO] [launch.py:428:sigkill_handler] Killing subprocess 9673
[2023-04-26 15:22:59,321] [INFO] [launch.py:428:sigkill_handler] Killing subprocess 9675
[2023-04-26 15:22:59,323] [INFO] [launch.py:428:sigkill_handler] Killing subprocess 9677
[2023-04-26 15:22:59,323] [INFO] [launch.py:428:sigkill_handler] Killing subprocess 9679
[2023-04-26 15:22:59,617] [ERROR] [launch.py:434:sigkill_handler] ['/usr/bin/python', '-u', 'main.py', '--local_rank=7', '--sft_only_data_path', 'appier/martechQA', 'sharegpt', '--data_split', '2,4,4', '--model_name_or_path', 'EleutherAI/pythia-12b-deduped', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '1', '--max_seq_len', '512', '--learning_rate', '1e-4', '--weight_decay', '0.', '--num_train_epochs', '16', '--gradient_accumulation_steps', '1', '--lr_scheduler_type', 'cosine', '--num_warmup_steps', '0', '--seed', '1234', '--zero_stage', '3', '--only_optimize_lora', '--lora_dim', '128', '--lora_module_name', 'decoder.layers.', '--deepspeed', '--output_dir', '/home/twsfphn198/DeepSpeedExamples/applications/DeepSpeed-Chat/output/actor-models/pythia-12b-deduped'] exits with return code = 1

hi @mrwyattii any idea about the index error using lora? or what should be the best practice for training 12b models? Since I also failed running the example run_13b.sh successfully as well?

May be I'm wrong, but I think the real resourse requests is much bigger than said in doc.
I use a 4*2080ti 22g node to train 1.3B. The vram of gpus are almost full

And this is even under the batchsize with 4 , while in the example, it was set to 8.
I also tried in colab with an A100 40, with even per_device_train_batch_size1 and gradient_checkpointing, it threw oom

But in the doc, the 1.3B training can be done with a single 48G a6000 within 2 hours, is that possible or I just mess up some important settings?


Fix compilation of tests

Apparently, nobody tried to run them for a while...

To prevent this kind of an oversight from happening again, I wrote a small script that checks the staged changes before the commit is made. If you guys are interested in using this, you can copy the script to core/.git/hooks/pre-commit.
#!/bin/sh

RED='\033[0;31m'
NC='\033[0m' # No Color
BOLD=$(tput bold)

# change to the repository root
cd $(git rev-parse --show-toplevel)

# make sure the hook only checks the code that is being commited
git stash --keep-index --include-untracked

echo "Executing pre-commit hook..."

cargo test --all
ret=$?

if [ $ret -ne 0 ]; then
	printf "\n\n\n-------------------------------------------\n"
	printf "${RED}${BOLD}One or more unit tests failed${NC}\n"
	git stash pop &> /dev/null
	exit 1
fi

cargo-fmt -- --check
ret=$?

if [ $ret -ne 0 ]; then
	printf "\n\n\n-------------------------------------------\n"
	printf "${RED}${BOLD}Repository contains unformatted code${NC}\n"
	git stash pop &> /dev/null
	exit 1
fi

git stash pop &> /dev/null
exit 0


Subrecipient: change data type of zip and zip+4 to String

Zip and Zip+4 (cols M, N) should be changed to type String (from Numeric) so that leading zeros will always be preserved.

Include UEI and TIN fields

The following updates were made to v20220815
Subrecipient
- Zip, Zip+4 and TIN (cols M, N & D) metadata has been changed to type String (from Numeric)
- Additionally, Zip, Zip+4, UEI and TIN (cols M, N, C & D) are all now formatted to Text so that leading zeros will always be preserved


Absolute paths restrictions for Storage Resize Image

[READ] Step 1: Are you in the right place?
Issues filed here should be about a feature request for a specific extension in this repository. To file a feature request that affects multiple extensions or the Firebase Extensions platform, please reach out to
Firebase support directly.
[REQUIRED] Step 2: Extension name
This feature request is for extension: storage-resize-images
What feature would you like to see?
It would be great to have the possibility to define a list of absolute paths inside the bucket in which listen for new image uploads to be resized.
The absolute paths to be provided in the configuration option could be a comma-separated list, like this:
/users/avatars,/posts/pictures,/design/resources/icons
This will mean that any image uploaded inside these paths, or inside any folder at any depth inside these will be resized.
And of course, any image outside of these paths will not be resized.
How would you use it?
This will allow us to limit the images to be resized by the extension. For instance, the user's avatars could be resized, but what about if we don't want resized images for posts pictures?
This feature could be very helpful to save much more space in the storage and not waste the resource of Firebase.

@yulioaj290 This is exactly the feature I'm looking for in my current project. Hope your pull request goes through! 👍

@yulioaj290 thanks for the PR!
@jhuleatt what do you think in terms of costs? Would this actually reduce costs for the users? Also, is this going to be backward compatible?

Having an option to include a list of paths (otherwise all paths would be included if the list is empty) and an option to exclude paths would be very useful. This would give fine grain control.
Example:
Include: /images --> this would include /images and all sub-folders
Exclude: /images/full-size --> exclude this specific sub-folder

2 concerns:

Even if the extension ignores parts of the bucket when it runs, it will still trigger for everything uploaded to the bucket. So, this change will reduce some cost and unnecessary resizes, but it won't totally get rid of extra cost because of the times it triggers unnecessarily.
We need to make sure this is backwards compatible (if the config is left blank, the whole bucket should still be converted)

I think (1) isn't a blocker, and (2) is something we can take care of in the implementation, so let's go ahead with this.

@wdm954 I think I can add another config option for the Excluded Absolute Paths, in this PR.
The thing related to the size restriction per path is more sophisticated, I vote to do that in a later PR.


Nicks/usernames with spaces are not sent properly to IRC

If Slack side has a bot user with spaces in name (e.g. "Travis CI") then IRC PRIVMSG's will be incorrect and fail parsing in IRC clients.
For example currently matterircd sends
Travis CI!Travis CI@Travis CI PRIVMSG #dev :Build done
whereas it should be
Travis_CI!Travis_CI@Travis_CI PRIVMSG #dev :Build done
One trivial fix would be to replace (at least) space in sorcix/irc
diff --git a/vendor/github.com/sorcix/irc/message.go b/vendor/github.com/sorcix/irc/message.go
index 088938d..9d34b27 100644
--- a/vendor/github.com/sorcix/irc/message.go
+++ b/vendor/github.com/sorcix/irc/message.go
@@ -123,14 +123,14 @@ func (p *Prefix) IsServer() bool {

 // writeTo is an utility function to write the prefix to the bytes.Buffer in Message.String().
 func (p *Prefix) writeTo(buffer *bytes.Buffer) {
-       buffer.WriteString(p.Name)
+       buffer.WriteString(strings.Replace(p.Name, " ", "_", -1))
        if len(p.User) > 0 {
                buffer.WriteByte(prefixUser)
-               buffer.WriteString(p.User)
+               buffer.WriteString(strings.Replace(p.User, " ", "_", -1))
        }
        if len(p.Host) > 0 {
                buffer.WriteByte(prefixHost)
-               buffer.WriteString(p.Host)
+               buffer.WriteString(strings.Replace(p.Host, " ", "_", -1))
        }
        return
 }

https://tools.ietf.org/html/rfc2812#section-2.3.1 contains list of allowed characters but at least for me it's enought to convert space into underscore

Thanks for reporting, fixed this another way in master.
Please reopen if still an issues


Vendor package github.com/bugsnag/panicwrap to v1.1.0 to support ARM64

Done with command govendor fetch<EMAIL_ADDRESS>This PR is essential to fix the build errors detected in PR #3960.
Signed-off-by: Dieter Reuter<EMAIL_ADDRESS>

Hmm... it needs to be done through Godep unfortunately (I always have troubles with godep)


Missing pinned and quick access

Description
Sometimes when I open the files app, the pinned folders to the left and the quick access folders are missing.

Steps To Reproduce

Open the app
the folders are missing.

Requirements
Fix the issue if possible.
Files Version
<IP_ADDRESS>
Windows Version
10.0.22631.3593
Log File
debug.log

Thanks for the report, Do you have Files settings > Advanced > Open Files on Windows startup enabled?

I just open the files app randomly, sometimes works nice, sometimes like you see.

I have Open Files on Windows startup enabled, and I am also receiving this issue.

Hiow about Files settings > Advanced > Leave app running in the background when the window is closed

this open is left on
On 6/4/2024 4:07:21 PM, Josh65-2201 @.> wrote:
How about Files settings > Advanced > Leave app running in the background when the window is closed
—
Reply to this email directly, view it on GitHub [https://github.com/files-community/Files/issues/15540#issuecomment-2147492180], or unsubscribe [https://github.com/notifications/unsubscribe-auth/AQDJFZVBBMGM4NHNC7R44ETZFW3YTAVCNFSM6AAAAABIWLUQTCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBXGQ4TEMJYGA].
You are receiving this because you authored the thread.Message ID: @.>
[7aa114c4-47b1-44ab-9ef1-b3366490e4aa]

Ok, so far, very good. I reopened it like 3 times, and it behaves normally. I will update it in 2 days or so to make sure.

Great. We will continue tracking this in #15410. You can check there or click subscribe in the right sidebar for updates.


chore:  Move tailwindcss to devDependency & peerDependency

Description
Moves tailwindcss to in ./package.json from dependencies to:

devDependencies - for developing & testing the plugin
and to peerDependencies, so the projects which use your awesome plugin can install tailwindcss on their own, with a certain version.

The installation instructions in the ./README.md also got updated.
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update

How Has This Been Tested?
There is no need for testing this, AFAIK.
Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] Any dependent changes have been merged and published in downstream modules
[x] I have checked my code and corrected any misspellings

Thank you for contributing @xeho91 👍


Feature request: provide more information when an @required parameter is null

It's common for @required named parameters to be non-null.
class Foo {
  Foo({ @required this.bar }) : assert(bar != null);
  final String bar;
}

void main() {
  Foo foo = Foo();
}

Currently, upon "hot restart", this idiomatic Flutter class fails with:
E/flutter (22230): [ERROR:flutter/lib/ui/ui_dart_state.cc(148)] Unhandled Exception: 'package:test_issue/main.dart': Failed assertion: line 4 pos 40: 'bar != null': is not true.

It would be somewhat more informative if it failed with a message like:
E/flutter (22230): 'package:test_issue/main.dart': required Foo parameter bar must not be null, line 4 pos 40

For Flutter classes it might also be helpful if a link to the API doc for the required parameter was provided.
This issue was originally reported here: https://github.com/flutter/flutter/issues/29046#issuecomment-480941903

Wouldn't you use the second field in assert to do this?
: assert(bar != null, 'Required parameter bar must not be null')

Something to consider if we introduce an actual keyword to replace @required.

Cf. https://github.com/dart-lang/language/issues/156#issuecomment-451523811, such a keyword could very well be introduced, and it would be implied (maybe without the keyword, maybe the keyword would be mandatory) that a named parameter with a non-null type and no default is required. It wouldn't make sense to require a named parameter with a default value to be passed, so it should most likely be an error to have that.
I think there wouldn't be an obvious justification for the converse rule: A named parameter whose type is nullable could be required and it could be optional, with and without a default value, all combinations will work and might make sense in the given application context (so we might as well allow developers to make the choice using an explicit required).
With that, this could be considered as yet another voice in favor of an explicit required modifier on parameters in function types and function declarations, that is, part of the discussion in dart-lang/language#156.

@matanlurey you're right, we could generate a reasonable message (even a link to the API doc) by hand. There are loads of these, so it would be nice if it the reasonable message was generated by default.

The issue here is @required is not a language feature, so the VM would not be parsing it and using it to fill in details of the error like you'd prefer. This could change if it becomes a keyword/language feature in the future.
@HansMuller: On a related note, I think your own team requested a lint to enforce adding messages?
https://dart-lang.github.io/linter/lints/prefer_asserts_with_message.html

@matanlurey: yes we did request such a lint. Once that's landed we'll be asking for a lint that checks if the messages are actually helpful :-).


Access dataview functions from dataviewjs

How can I access convenience functions of dataview like the newly introduced date(<link>) from dataviewjs?
I am currently limited to use the dataviewjs workaround for filtering out all open tasks.
I now want to find all open tasks that are (over-)due based on a connected date link like [[2021-05-08]].
So something like this:
dv.taskList(dv.pages('""').file.tasks.where(t => !t.completed).map(t => date(t.text)));

The .map(t => date(t.text)) does not work (I also don't get an error, just a blank screen, btw)

Also look at the little triangles at the left. You can expand things to inspect them:

And whenever you suffer from information overload, just press Ctrl+L in the console—this will clear it out. Happy debugging!

Thank you for your feedback!!
dv.call('date', <arguments>) would indeed already help a lot. dv.date(...) would of course be really nice!
I would be fine with closing the issue as soon as there is a workaround :)
Awesome work with the quick and frequent releases!

Will the workaround be in the next release? @blacksmithgu


[swift-3.0-preview-2-branch] build-script fixes

Explanation: unbreak the 'build-swift-stdlib' option.  Argparse was trying to helpfully match it with 'build-swift-stdlib-unittest-extra', but this wasn't our intention.
Scope of the issue: build-script invocations that use build-swift-stdlib.
Risk: low.
Reviewed by: nobody.
Testing: manual.

Before merging this pull request to apple/swift repository:

[ ] Test pull request on Swift continuous integration.

Triggering Swift CI
The swift-ci is triggered by writing a comment on this PR addressed to the GitHub user @swift-ci. Different tests will run depending on the specific comment that you use. The currently available comments are:
Smoke Testing
    Platform     | Comment
    ------------ | -------------
    All supported platforms     | @swift-ci Please smoke test
    All supported platforms     | @swift-ci Please smoke test and merge
    OS X platform               | @swift-ci Please smoke test OS X platform
    Linux platform              | @swift-ci Please smoke test Linux platform

Validation Testing
    Platform     | Comment
    ------------ | -------------
    All supported platforms     | @swift-ci Please test
    All supported platforms     | @swift-ci Please test and merge
    OS X platform               | @swift-ci Please test OS X platform
    OS X platform               | @swift-ci Please benchmark
    Linux platform              | @swift-ci Please test Linux platform

Lint Testing
    Language     | Comment
    ------------ | -------------
    Python       | @swift-ci Please Python lint

Note: Only members of the Apple organization can trigger swift-ci.

@swift-ci Please test

Oops, sorry, looks like this was introduced by my changes in https://github.com/apple/swift/pull/3027. Thanks for the fix, @gribozavr! 👍

LGTM

@ddunbar Thank you!


Allow for config to be a Map instead of a string

Added a condition to check if the item is a dict, when fetching the config. This allows for config to be a Map in DynamoDB, which is easier to edit.
@Brett55 you seem to be the main contributor. Could you review and merge if acceptable?

Can one of the admins verify this patch?

Latest commits change to boto3 and boolean values were causing errors.


Stop Caching Video

add stop download cache for HLS video, example, if download 5 sec of video and stop, similar to precache. Sorry for my english, i speak spanish :)

This plugin is only to play videos from cache when they are fully available. So HLS video's are not the targeted type for this plugin.


Don't send update_type in content payload

update_type is supposed to be sent as an argument to the publish action.
Part of: https://github.com/alphagov/govuk-content-schemas/pull/259

Note that update_type is still valid in the content item. The publishing-api publish will use this value if the update_type is nil in the publish call.
This PR is still useful however, because it makes explicit that Whitehall sets the update_type in the publish call.

:+1: when green

Test fail for unrelated reasons, investigating.


The results in the mnist example are wrong/misleading

Describe the bug
The results in your colab ipython notebook are misleading: https://colab.research.google.com/github/Rishit-dagli/Gradient-Centralization-TensorFlow/blob/main/examples/gctf_mnist.ipynb
In this example, the model is first trained with a normal Adam optimizer:
model.compile(optimizer = tf.keras.optimizers.Adam(),
              loss = 'sparse_categorical_crossentropy',
              metrics = ['accuracy'])

history_no_gctf = model.fit(training_images, training_labels, epochs=5, callbacks = [time_callback_no_gctf])

And afterwards the same model is recompiled with the gctf.optimizers.adam(). However, recompiling a keras model does not reset the weights. This means that in the first fit call the model is trained and then in the second fit call with the new optimizer the same model is used and of course then the results are better.
This can be fixed, by recreating the model for the second run, by just adding these few lines:
import gctf #import gctf

time_callback_gctf = TimeHistory()

# Model architecture
model = tf.keras.models.Sequential([
                                    tf.keras.layers.Flatten(),
                                    tf.keras.layers.Dense(512, activation=tf.nn.relu),
                                    tf.keras.layers.Dense(256, activation=tf.nn.relu),
                                    tf.keras.layers.Dense(64, activation=tf.nn.relu),
                                    tf.keras.layers.Dense(512, activation=tf.nn.relu),
                                    tf.keras.layers.Dense(256, activation=tf.nn.relu),
                                    tf.keras.layers.Dense(64, activation=tf.nn.relu),
                                    tf.keras.layers.Dense(10, activation=tf.nn.softmax)])

model.compile(optimizer = gctf.optimizers.adam(),
              loss = 'sparse_categorical_crossentropy',
              metrics=['accuracy'])

history_gctf = model.fit(training_images, training_labels, epochs=5, callbacks=[time_callback_gctf])

However, then the results are not better than without gctf:
Type                   Execution time    Accuracy      Loss
-------------------  ----------------  ----------  --------
Model without gctf:           24.7659    0.88825   0.305801
Model with gctf               24.7881    0.889567  0.30812

Could you please clarify what happens here. I tried this gctf.optimizers.adam() optimizer in my own research and it didn't change the results at all and now after seeing it doesn't work in the example which was constructed here. Makes me question the results of this paper.
To Reproduce
Execute the colab file given in the repository: https://colab.research.google.com/github/Rishit-dagli/Gradient-Centralization-TensorFlow/blob/main/examples/gctf_mnist.ipynb
Expected behavior
The right comparison would be if both models start from a random initialization, not that the second model can start with the already pre-trained weights.
Looking forward to a fast a swift explanation.
Best,
Max

Addition
This is also true for the horses-v-humans example. The crucial bug here is that model.compile does not reset any learned weights:
https://stackoverflow.com/a/47996024

Hi,
I tried reproducing with the code edits you asked for with TensorFlow Enterprise 2.5 with an NVIDIA Tesla V100 on a GCP AI Platform Instance using an M71 VM Image and as you can see I am not able to reproduce these. Find the gist here. Following this, I am closing this issue.
Also, you mentioned using this in your research, the authors did mention using this only for limited kinds of problems.


Avoids SetUserData() {     PlayFabClientAPI.UpdateUserData(new UpdateUserDataRequest() {         Data = new Dictionary<string, string>() {             {"Ancestor", "Arthur"},             {"Successor", "Fred"}         }     },     result => Debug.Log("Successfully updated user data"),     error => {         Debug.Log("Got error setting user data Ancestor to Arthur");         Debug.Log(error.GenerateErrorReport());     }); }

Do not know what

You want us to change 'void' to 'Avoids'? That wouldn't be correct. Closing this.


Partial tiles with some images

Due to differences in the keys used to get textures when we get to the three.js layer (caused by a rounding error, the difference of 1 pixel) the last row of tiled meshes is missing textures and rendered visible = False.
The fix is to make sure our tiling helper method uses the same rounding method as OpenSeadragon.

I am testing a fix.

OpenSeadragon uses Math.floor instead of Math.ceil.

Fix is on QA for testing.


Add sync test for automerge-repo-network-broadcastchannel

This also bumps the tests Node version to 18, as BroadcastChannel isn't available in the global scope in anything lower. This is the current LTS version.

lgtm


Add Memory chart for Availability Zones

Add Memory chart for Availability Zones. I deleted this chart in #12609 but now i figure out hat it was wrong fix. For some providers, which don't support memory CU collection, chart will all values equal zero.
Screenshots
Before:

After:

Links [Optional]
https://github.com/ManageIQ/manageiq/pull/12609
Steps for Testing/QA [Optional]

@miq-bot remove_label wip

It seems that we now need more proper fix for https://bugzilla.redhat.com/show_bug.cgi?id=1393574

@PanSpagetka please attach bz for proper euwe & fine backport.

Euwe backport details:
$ git log -1
commit 0e433d922e59825736b7509ca0ba95f1ad3b0b2a
Author: Šimon Lukašík<EMAIL_ADDRESS>Date:   Tue May 2 12:56:13 2017 +0200

    Merge pull request #14938 from PanSpagetka/add-memory-chart-for-availiability-zones

    Add Memory chart for Availability Zones
    (cherry picked from commit e096670f7077e1203fdbefde2322b6e08acd6fa8)

This PR reverts fix for https://bugzilla.redhat.com/show_bug.cgi?id=1393574 (Fine) and https://bugzilla.redhat.com/show_bug.cgi?id=1397155 (Euwe).

Fine backport details:
$ git log -1
commit 5a83bb8f3847384449284babc2635f7e5938d8a5
Author: Šimon Lukašík<EMAIL_ADDRESS>Date:   Tue May 2 12:56:13 2017 +0200

    Merge pull request #14938 from PanSpagetka/add-memory-chart-for-availiability-zones

    Add Memory chart for Availability Zones
    (cherry picked from commit e096670f7077e1203fdbefde2322b6e08acd6fa8)


Feat: Workflow to greet when new issue/pull request opens

Description
Currently, this project does not have any action to greet to contributors. Github action is the great tool to automate the manual stuff for your project. I would like to create Github Action workflow to greet to the new PR and issue creator.
Record

[x] I agree to follow Contributing guidelines and Code of conduct of this project.
[x] I am a GSSoC'23 Contributor.

Please assign me this issue under GSSoC'23
Thanks

Hey @harmeetsingh11  I am Already Created Isuue on that: https://github.com/Gaurav-Verma07/sampark/issues/82


Add support for more FocalPlaneResolutionUnit values

Hello! :hand:
It looks like get_mm_per_unit was missing a few enum values for units. This causes errors with certain camera vendors (found this from a Parrot Sequoia dataset). The enum values are taken (per  comment indications) directly from https://www.sno.phy.queensu.ca/~phil/exiftool/TagNames/EXIF.html (FocalPlaneResolutionUnit).
I've introduced the cm_in_mm and um_in_mm variables to be consistent with the inch_in_mm conversion.
:clinking_glasses:

We had this discussion before. The problem is that these are outside of the Exif specification, only inch and cm have been specified. Cameras which use other values (units) than those specified violate the Exif specification. If you still want to use images from such cameras with OpenSfM, you are basically left with either recomputing and updating the affected values or you run a privately patched version of OpenSfM on your machine(s).

@YanNoun Please revert this PR. These units are NOT standard complient. This PR will eventually pollute databases with bad data. Please check with @paulinus.

Thanks for the warning, @GITNE.
On the one hand, I understand the concern about supporting unofficial standards, as we discussed here https://github.com/mapillary/OpenSfM/commit/a074feab193776569ff23ce0ff5cab31535e34ff#r31973708
On the other hand, while not official, the mm and um units seem to be used by manufacturers currently and it is good that OpenSfM's supports it. We do not run this particular exif extraction code for the images in the Mapillary database and we are moving to rely the less and less on the exif focal length data. So this change will not affect Mapillary.
So, all in all, I think we are good with the change that will make more cameras work.


Labelled Image to mesh function.

Currently isn't clear how to get a set of meshes from a labelled image. This should be made simple possibly with a menu item.

This has been added to the interface and the javascript console with some parameters. The method itself needs to be improved, but it is a good start.


Add breakpoint on Swift throw

LLDB command is b swift_willThrow

This might require a change to vscode-lldb

https://github.com/vadimcn/codelldb/pull/881

CodeLLDB PR is now merged. Just waiting for a release now


modify wrapResolvers to properly handle Subscription

this change was very similar to the modification made to wrapResolvers to handle a resolver map that contains an internal enum remapping. A subscription resolver map declaration looks like this:
const resolvers = {
  Subscription: {
    postAdded: {
      subscribe: () => { ... },
      // optionally
      resolve: (payload) => { ... }
    }
  }
}

which meant that we were trying to bind postAdded which is an object and not a function.

@tlivings - I updated the changelog to accurately reflect what is currently released. According to npmjs our @latest is 1.2.4. The changes listed in 1.3.0 in the changelog have not been released so all of those items and what is added this PR are now listed under unreleased.


access the search value from within the onChange callback

Hi, nice plugin thanks.
I needed to access the search value from within the onChange callback, that's because my users can type things that don't match my list yet, and i need to take action based on what they type (basically add what they typed to my server list).
So i updated my autocomplete.js version so that i could do this:
    <script type="module">
      import Autocomplete from "{{ asset('vendors/bs5-autocomplete/autocomplete.js') }}";

      Autocomplete.init('#autocomplete', {
        server: '/some/route',
        serverParams: {
          action: 'autocompleteList',
        },
        onChange: function(item, searchValue, inst){
            console.log("change", searchValue);
        },
        onSelectItem: function(item, inst){
            console.log("select", item);
        },
      });
    </script>

In autocomplete.js around line 521 i changed that:
  onchange(e) {
    const search = this._searchInput.value;
    const item = Object.values(this._items).find((item) => item.label === search);
    this._config.onChange(item, search, this);
  }

If it's a good idea, I would love to see it (or something equivalent) in an official release.
Cheers.

Right, i missed that. Thank you, exactly what i needed :)


1-1. Introduction to Linux

Topics covered: Command line basics, git, user management, systemd, firewalls

Resources:

https://github.com/alex-pakalniskis/TechnicalTeacherRoleChat/blob/main/January2023_RoleChat.md
https://github.com/alex-pakalniskis/TechnicalTeacherRoleChat/blob/main/February2023_RoleChat.md


Include auto-categorizing and sorting after install || update gems in Gemfile

At my current project we have about 350 gems. It's not so much but it's how it is..
Basically when somebody add new gem it's added to bottom of Gemfile. Without categorization and alphabetic order it's painfully hard to read for new people.
Could you please somehow integrate with https://github.com/nikolalsvk/ordinare.
It would be really awesome to have pre-installed ability not to think about all this stuff.
I would really appreciate any callback for ordinare after running bundle install || update

This would be most suited to a Bundler plugin that can update the Gemfile after a bundle install/update

Hi! Thanks for the suggestion on ways to improve Bundler. This would be most suited to a Bundler plugin that can update the Gemfile after a bundle install/update.

Yeah, I don't think this belongs in bundler either. You can also use rubocop which has the style rule you're requesting, and more: http://www.rubocop.org/en/stable/cops_bundler/


webcam not loading, displays error: Webcamjs error: Could not access webcam: NotallowedError: Permission denied

Describe the bug
Webcam does not load. It throws up error: "Webcam.js error, could not access webcam: Notallowed error"
A clear and concise description of what the bug is, better if you can attach a log.
We imported your project, changed asw url and linked it to our getusermedia example page, which is a working example in all modern browsers. Upon initialization of app, the error pops up:  "Webcam.js error, could not access webcam: Notallowed error"
Expected behavior
The webcam should load correctly without the aforesaid error.
Screenshots

Device (please complete the following information):

Device name/variant: [e.g. Samsung Galaxy, J2]
Samsung Galaxy S10

Build [e.g. arm, x86]
x86

Android Version [e.g. API 27 or Oreo]
29, not sure

Additional context
Add any other context about the problem here.

We need WebRTC feature to achieve that - this link might help you.

If i'm right, Process Camera Input is part of this project. How's that different from getusermedia api ?
And if you do not mind, could you please inject the code given in the reference link in the project's MainActivity.Java file. Since i'm not actually a pro in android and java language, its quite time consuming and difficult to actually pick a code and insert it into the appropriate line of the project.

I already got the request to do so but it's not just a matter of copying and pasting the code from there to here. SWV is getting more complex with each new request, so every single change brings a new issue and currently I'm kind of busy procrastinating, I'm sorry but it's a slow process and I don't know when I'm going to touch this subject.
Meanwhile, you're free to experiment and if you did it before us, do let us know how you did it. It might help a lot of people here. :)

I understand and appreciate.  I shall try the code given in the reference link. Not trying to impose on you, but atleast give me a pointer as where i should inject this code.
Based on what i presume, its probably in 894 line of Mainactivity.java

is the webcam issue resolved ?

Its resolved. Here is the solution for all of those who are struggling with the current issue. Thanks to the aws android sample webview, i was able to take out its codes and implement in the smart webview. So here it is
Under MainActivity.Java file / asw_view.SetwebChromeClient. Post this code
@Nullable
			public Bitmap getDefaultVideoPoster() {
				return Bitmap.createBitmap(10, 10, Bitmap.Config.ARGB_8888);
			}
			@Override
			public void onPermissionRequest(@NotNull final PermissionRequest request) {
								final String[] requestedResources = request.getResources();
				for (String r : requestedResources) {
					if (r.equals(PermissionRequest.RESOURCE_VIDEO_CAPTURE)) {
						request.grant(new String[]{PermissionRequest.RESOURCE_VIDEO_CAPTURE});
						break;
					}
					if (r.equals(PermissionRequest.RESOURCE_AUDIO_CAPTURE)) {
						request.grant(new String[] { PermissionRequest.RESOURCE_AUDIO_CAPTURE});
						break;
					}
				}
			}

Add the line under websettings
webSettings.setMediaPlaybackRequiresUserGesture(false);
And make sure record_audio and camera permissions are granted in android manifest xml. Any issues on this, any of you can message me. Thanks.

Its resolved. Here is the solution for all of those who are struggling with the current issue. Thanks to the aws android sample webview, i was able to take out its codes and implement in the smart webview. So here it is
Under MainActivity.Java file / asw_view.SetwebChromeClient. Post this code
@Nullable
			public Bitmap getDefaultVideoPoster() {
				return Bitmap.createBitmap(10, 10, Bitmap.Config.ARGB_8888);
			}
			@Override
			public void onPermissionRequest(@NotNull final PermissionRequest request) {
								final String[] requestedResources = request.getResources();
				for (String r : requestedResources) {
					if (r.equals(PermissionRequest.RESOURCE_VIDEO_CAPTURE)) {
						request.grant(new String[]{PermissionRequest.RESOURCE_VIDEO_CAPTURE});
						break;
					}
					if (r.equals(PermissionRequest.RESOURCE_AUDIO_CAPTURE)) {
						request.grant(new String[] { PermissionRequest.RESOURCE_AUDIO_CAPTURE});
						break;
					}
				}
			}

Add the line under websettings webSettings.setMediaPlaybackRequiresUserGesture(false);
And make sure record_audio and camera permissions are granted in android manifest xml. Any issues on this, any of you can message me. Thanks.

Its very helpful thanks you brother

Its resolved. Here is the solution for all of those who are struggling with the current issue. Thanks to the aws android sample webview, i was able to take out its codes and implement in the smart webview. So here it is
Under MainActivity.Java file / asw_view.SetwebChromeClient. Post this code
@Nullable
			public Bitmap getDefaultVideoPoster() {
				return Bitmap.createBitmap(10, 10, Bitmap.Config.ARGB_8888);
			}
			@Override
			public void onPermissionRequest(@NotNull final PermissionRequest request) {
								final String[] requestedResources = request.getResources();
				for (String r : requestedResources) {
					if (r.equals(PermissionRequest.RESOURCE_VIDEO_CAPTURE)) {
						request.grant(new String[]{PermissionRequest.RESOURCE_VIDEO_CAPTURE});
						break;
					}
					if (r.equals(PermissionRequest.RESOURCE_AUDIO_CAPTURE)) {
						request.grant(new String[] { PermissionRequest.RESOURCE_AUDIO_CAPTURE});
						break;
					}
				}
			}

Add the line under websettings webSettings.setMediaPlaybackRequiresUserGesture(false);
And make sure record_audio and camera permissions are granted in android manifest xml. Any issues on this, any of you can message me. Thanks.

I tried to do as you instructed, but I'm still getting the "NotAllowedError: Permission denied" error.
Is there a solution to this problem?

Hello There,
I am facing same issue.
I used the web view for my flutter app,The camera working properly in my website but getting this error in Android and iOS.
Also can't able to MainActivity.java file so please help that how can solve it ?


use automated build container image

I rule and have completed some work on Case Manager that's ready for review!

use docker hub automated build

This pull request makes the following changes:

create an automated build on docker hub
point the docker-compose.yml file to use the built container
update readme to deprecate build step

It relates to the following issue #s:

Fixes #491

You still need to create an automated build that points to this repo colinxfleming/dcaf_case_management so any updates will be automatically included in the built container: https://www.brianchristner.io/how-to-automate-docker-builds-end-to-end/ (just the first part "Create an Automated Build Docker Repository")
Then change the container name in docker-compose.yml and voila!

This is sick, thanks @mchelen ! Had no idea docker could do this, and that absolutely rules.
I'm going to let this marinate until after the meeting on Tuesday, and then we can get this going.

@mchelen I've set some stuff up on my end at docker-hub! When you get a sec, want to try pulling down from colinxfleming/dcaf_case_management so we can make sure it works somewhere other than my machine?

@mchelen bumping on this! Want to confirm it works for you before I hit the button.

A thought, you might want to create a codefordc user on Dockerhub to use rather than tie it to your personal accounts.

I was able to get @colinxfleming image running with some slight tweaks to the docker-compose file, nothing shocking. I'm not an expert on the app yet so I'm hesitant to say YEAH it's working but everything appears to be working fine.

@DarthHater I think if it passes the majority of the test suite I feel good about that. (We have a few scrub integration tests that randomly choke.)
The organization user for this is a good call -- let me see what I can spin up in a hurry for dcaf. I've got a bunch of infra set up on that sort of level.
Want to spin your edits to the compose file into a new PR and I'll merge it up?

#720 looks good, @colinxfleming if you want to create an automated build on Docker hub then we can point the docker-compose.yml there and make sure everything is working nicely.

@colinxfleming my edits were just to test yours, nothing amazing. If you setup a more generic user I assume you or @mchelen can point the docker-compose file there?


AWS tasks don't stop if the previous step fails

Context
The whole code assumes all of the previous steps are completed successfully, and not aborting the execution of the job if any of the steps above fail.
Ex: the task below couldn't download the git repo but still tries to cd into the folder and rename it, try to build it, etc. until all of the tasks are completed (or failed) in order.
...
 (1/2) Installing unzip (6.0-r8)
(2/2) Installing zip (3.0-r9)
Executing busybox-1.32.1-r6.trigger
OK: 6 MiB in 16 packages
/bin/sh: cd: line 4: can't cd to Library: No such file or directory
zip error: Nothing to do! (try: zip -q -r build-2-iOS-6yTb.zip . -i build/iOS/*)
mv: can't rename 'build-2-iOS-6yTb.zip': No such file or directory
End of task logs
Task has finished successfully
Cleanup complete
Starting part 4/4 (upload build to s3)
Creating worker cluster...
Creating cleanup cluster...
Worker cluster created successfully (skipping wait for cleanup cluster to be ready)
Task is starting on worker cluster
Task is running on worker cluster
Task status is STOPPED
...

Suggested solution
Abort the mission if something goes wrong.

@frostebite could you have a look at this issue?
I believe this has been solved since right?

Should be resolved in latest.


Error when commiting to GitLab - Missing flush-pkt

When I try to commit (or create a new branch etc.) to GitLab (public cloud version), I get this error.
Missing flush-pkt

Although the commit is created in GitLab. After refreshing the repo in abapGit the state is clean.
Maybe it has something to do with recently improved error handling #4778.
https://github.com/abapGit/abapGit/blob/946180abda363d85fdfa1b3d4795ba17f6d45ba4/src/git/zcl_abapgit_git_transport.clas.abap#L219

According to specs 00000000 is required but apparently gitlab uses only 0000. Oh well, we can just tweak the if a bit.

there are git capability negotiations in the beginning of the protocol
https://git-scm.com/docs/protocol-capabilities
this might be related to using or not using side-band

Same error with Bitbucket as they also use 0000. I prepare a fix.


Change Table Output to Reflect 1 Cell Line or Many Samples (by Tissue)

Change the data.frame that gets generated dependent on number of samples

[x] Columns for 1 sample: "Value"
[x] Columns for many samples: Also, just put "Median" and "Average"

Updated!


Figure out if we need docs on events

According to the 0.12.0 release notes, we now emit an "auction end" event.
I think we might need to capture docs on what events are emitted by Prebid.js in one place.  Maybe this exists, but I couldn't find it with a quick search.
Scope of this ticket is: find out if we need this, and if so, define what "this" is, and then do it.

  pbjs.onEvent('auctionEnd', function() {
    //auction done code here.
  });
});```

Here is on onEvent & offEvent PR: https://github.com/prebid/Prebid.js/pull/204

Addressed by #245


Determine if upsert is an insert or update in PostgreSQL

Environment
Knex version: ^0.95.9
Database + version: PostgreSQL 13.6
OS: Ubuntu
Bug
How to implement PostgreSQL xmax check to determine if upsert was an insert or update?
Using the following query:
this.db.insert(product).into('product').onConflict(['product_id']).merge().returning('*')
How can I tell if it is a fresh insert or an updated row?
There is a solution to this by using xmax as seen here:
RETURNING (xmax = 0) AS inserted
How can I put this into returning(....) without errors?

.returning(knex.raw('(xmax = 0) as inserted')) is a bit hacky but looks like it would work.


tree-sitter: Move dynamic libraries for grammar parsers into lib/

Description of changes
The previous version of the code was creating a tree-sitter "grammars" derivation with all parser's .so file inside the top-level folder. This is unusual.
Additionally, this prevents other packages from smartly referencing the parsers. For example, I would expect
  nix-build -E 'with import ./default.nix {};
    pkgs.emacs29.pkgs.withPackages (_:
      [(pkgs.tree-sitter.withPlugins (p: pkgs.tree-sitter.allGrammars))])'

to get an emacs-packages-deps derivation with a lib/ folder and all parser's .so files there. This would make the
  (add-to-list 'treesit-extra-load-path "$out/lib/")

line in build-support/emacs/wrapper.nix work.
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 230547 run on x86_64-linux 1

/cc @clhodapp (author of https://github.com/NixOS/nixpkgs/pull/219559 which introduced basic tree-sitter support for Emacs).

@clhodapp do you have a working derivation to get tree-sitter .so files inside the emacs' lib/ folder?

I do, actually.
For a while, I was using the admin/notes/tree-sitter/build-module/build.sh script from the emacs source tree to build them myself from the raw upstream source in the various tree-sitter-XXX repos.  For that, I was using code like this: https://gist.github.com/clhodapp/4d9431c409d1bee2ab84e67519ed5896#file-from-raw-source-nix
That was honestly pretty annoying because it created a LOT of rebuilding for me but it worked so I kept it for a while. However, upstream emacs eventually dropped that script from the emacs 29 development branch and when that happened, I switched to just symlinking the generated files from the existing tree-sitter grammars in nixpkgs. For that, stole from the existing code in /pkgs/development/tools/parsing/tree-sitter/default.nix from nixpkgs to create:
https://gist.github.com/clhodapp/4d9431c409d1bee2ab84e67519ed5896#file-from-nixpkgs-tree-sitter-nix

Honestly the whole build infrastructure in pkgs/development/tools/parsing/tree-sitter/grammar.nix (or even that build.sh from emacs that I was depending on at one point) are kind of bizarre to me. It seems to me that we should be generating bindings for every language we wish to support from the (JSON) grammar definition in a uniform way rather than that hoping that each project commits generated binding code in the structure that we expect.

@clhodapp I'm trying another approach in https://github.com/NixOS/nixpkgs/pull/230751. What do you think?

@ofborg build vimPlugins.nvim-treesitter.tests

I'm not sure about changing this in tree-sitter, maybe this should be in emacs instead?
On a side note, breaking changes are now restricted: https://github.com/NixOS/nixpkgs/issues/223562


Possibly misleading statement about Message ordering

The Specification has the following to say about Message ordering:

For example, reordering the result of textDocument/completion and textDocument/signatureHelp is allowed, as each of these requests usually won’t affect the output of the other. On the other hand, the server most likely should not reorder textDocument/definition and textDocument/rename requests, since executing the latter may affect the result of the former.

I understand that a completion and signature request do not affect each other, but why doesn't the same apply to a definition and rename request? This seems to suggest that a rename request orders the server to compute and apply a rename. Usually this request is followed by a didChange that actually performs the edit.

Because the rename could change the result of the definition request. So if you first execute the rename although the definition request was executed on a different document version the result might be incorrect.
Hope that explains it.

How is it possible that a rename request alters how the a request after it like definition or references should be handled?
Please correct me if I am wrong, but the rename request doesn't actually rename/modify a symbol or modify a document in any shape or form. Just like the Specification says, it only computes workspace change that can later be used to actually rename a symbol and thefore modify a document.
This is the description of the Rename  request:

The rename request is sent from the client to the server to ask the server to compute a workspace change so that the client can perform a workspace-wide rename of a symbol.

As an example, if you use the "Rename Symbol" feature in VS-Code it will send a rename request followed by a didChange.
If you start a "Rename Symbol" and then ask for a preview, VS-Code will only send the rename request.
If there is no request that modifies a document like didChange is inbetween a references and rename request, they are both performed on the same document version. Whether or not the rename or references request comes first, shouldn't matter. Therefore, reordering or executing them in parallel should not pose a problem.

If the rename request sends it result first the client might apply the changes but the server still computes the definitions on an older state hence the client might use results that are off.

If the rename request sends it result first the client might apply the changes but the server still computes the definitions on an older state hence the client might use results that are off.

Isn't this scenario achievable regardless of using message reordering. If a client send a sends a references request and then a didChange request and receives its responses in order (references first). The result has been computed before the didChange request and from the perspective of the client, the results are off because it has already modified that document in a previous request.
And also, does this only apply to the rename request? Can i reorder references, completion, semantic token, workspace symbol requests or similar if i don't change any state in between (e.g. with didChange) but not a rename request?
Just to make sure, whenever I talk about the rename request, i only mean sending a textDocument/rename without using its result. No document got modified.

Isn't this scenario achievable regardless of using message reordering. If a client send a sends a references request and then a didChange request and receives its responses in order (references first). The result has been computed before the didChange request and from the perspective of the client, the results are off because it has already modified that document in a previous request.

Yes, but the client knows about this and can act accordingly. For example by canceling the first request.
If the client doesn't use the result of the rename request then it would be fine to use the definition result.

Does the rename or references request modify any state, document or may otherwise have any observable behavior and if so what exactly?


Display reviews count on teams main page

Related: https://github.com/eloquence/lib.reviews/issues/199
Would happily get some feedback on this one, before I start doing it for the user and thing pages. :)

Thanks @Poslovitch. I've pushed a commit which should preserve functionality and simplify a bit by looking up the join relationship directly. (Doing a full join on all reviews for every team instantiation is potentially a bit expensive.) Let me know if that works for you as expected (it does for me) and I can merge.
I've also changed it from reviewsCount to reviewCount to be consistent with the review count we have for review subjects (things).

Thanks @eloquence ! It works on my end as well.
I'm glad to see there's a .count() operator in thinky. I somehow overlooked it.
Once merged, I'll get on the other suggestions of #199 :)


FALCON-1623 Implement safemode in Falcon Server

I have couple of comments for design doc. Sorry, I should have reviewed it when doc was provided long time.
1> Can you talk about behavior for safe mode when the authorization is enabled & disabled?
2> Design doc says "Super user can perform update operation only on cluster entities". Shouldn't this be set safemode ON/OFF instead of cluster updates?
3> Even the admin users/groups should be able to enable/disable safemode and not only super user
4> Doc mentions "Entity operations not allowed are : update, schedule, touch, delete, submit,
submitAndSchedule". Did you miss resume or is it allowed as its not allowed for instances.
5> In section "UI support for safemode" doesn't have the updated list of operations disallowed. Can you please update it?
Again I am sorry for the late review. Thanks!

I just started reviewing the code, not fully done with it.
Please add "safemode" in the below code in  DefaultAuthorizationProvider.authorizeResource else it will not work for admin users/groups
            if ("admin".equals(resource)) {
                if (!("version".equals(action) || "clearuser".equals(action) || "getuser".equals(action))) {
                    authorizeAdminResource(authenticatedUGI, action);
                }

@sowmyaramesh  : The above suggestion is inaccurate. The code says "allow non-admin users to access version, clearuser and getuser". So we should not add safemode to this code.

@bvellanki : Where does it say that? If you look at authorizeAdminResource it is what validates if the user has admin privileges.
I don't think current code works for admin user. Did you test it for admin user not super user case?

@sowmyaramesh Please look at the code again.
 if (isSuperUser(authenticatedUGI)) {                 return;             }
This part would let a super user perform setsafemode action. In DefaultAuthorizationProvider.authorizeAdminResource you have  if (adminUsers.contains(authenticatedUser) || isUserInAdminGroups(authenticatedUGI)) {             return;         }
This part allows admin users to perform setsafemode action. Everyone else is denied permission to perform setsafemode action

@bvellanki : My bad, missed the ! in the if statement
if ("admin".equals(resource)) {
                if (!("version".equals(action) || "clearuser".equals(action) || "getuser".equals(action))) {
                    authorizeAdminResource(authenticatedUGI, action);
                }

@sowmyaramesh  : Updated the design document in https://issues.apache.org/jira/browse/FALCON-1623

@sowmyaramesh @vrangan  I updated patch to be able to persist safemode setting between restarts of falcon server. This is done based on your feedback. Please review.

Patch looks good. One general comment: You use a file to persist whether Falcon server is in safe mode or not. For long term use, looks better to have a property file owned by Falcon to persist important runtime properties (could be a separate jira) :)

+1. Thanks


Update Flutter to 3.16.5 & Change to wakelock_plus...

please merge this commit, tks. ;)

@ahujaraghav @mrinaljain

+1
See this thread for more info: https://github.com/creativecreatorormaybenot/wakelock/issues/220

+1
Please merge.

+1


Azure Functions: Assembly with same name is already loaded System.Management.Automation.dll

Steps to reproduce
Importing the module version 3.0.10 fails with errors.
ERROR: Assembly with same name is already loaded

Exception             :
    Type       : System.IO.FileLoadException
    Message    : Assembly with same name is already loaded
    TargetSite :
        Name          : Bind_LoadAssemblies
        DeclaringType : initialsessionstate
        MemberType    : Method
        Module        : System.Management.Automation.dll
    StackTrace :
   at System.Management.Automation.Runspaces.InitialSessionState.Bind_LoadAssemblies(ExecutionContext context)
   at System.Management.Automation.Runspaces.InitialSessionState.Bind(ExecutionContext context, Boolean updateOnly, PSModuleInfo module, Boolean noClobber, Boolean local, Boolean setLocation)
   at Microsoft.PowerShell.Commands.ModuleCmdletBase.LoadModuleManifest(String moduleManifestPath, ExternalScriptInfo manifestScriptInfo, Hashtable data, Hashtable localizedData, ManifestProcessingFlags manifestProcessingFlags, Version minimumVersion, Version maximumVersion, Version requiredVersion, Nullable`1 requiredModuleGuid, ImportModuleOptions& options, Boolean& containedErrors)
    Source     : System.Management.Automation
    HResult    : -2146232799
CategoryInfo          : InvalidOperation: (:) [Import-Module], FileLoadException
FullyQualifiedErrorId : FormatXmlUpdateException,Microsoft.PowerShell.Commands.ImportModuleCommand
InvocationInfo        :
    MyCommand        : Import-Module
    ScriptLineNumber : 68
    OffsetInLine     : 1
    HistoryId        : 1
    ScriptName       : D:\home\site\wwwroot\profile.ps1
    Line             : Import-Module PartnerCenter 4>$null

    PositionMessage  : At D:\home\site\wwwroot\profile.ps1:68 char:1
                       + Import-Module PartnerCenter 4>$null
                       + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    PSScriptRoot     : D:\home\site\wwwroot
    PSCommandPath    : D:\home\site\wwwroot\profile.ps1
    InvocationName   : Import-Module
    CommandOrigin    : Internal
ScriptStackTrace      : at <ScriptBlock>, D:\home\site\wwwroot\profile.ps1: line 68

Expected behavior
Module should load.
Actual behavior
Module fails with reported error. It started happening on the 28th of October.
Diagnostic logs
Environment
I did not change anything when the issue started appearing.
Azure functions runtime version 3
PS 7.0
requirements.psd1
@{
    'Az' = '4.7'
    'PartnerCenter' = '3.*'
}

I have tried using Az 5.0 as well but this did not change anything.
I am not sure if it is related to Az Powershell changes, as I did not see anything too specific about this. I could be wrong.
profile.ps1 imports module AzTable before importing Partner Center.
I will update if I find anything that could be used.

I have been having very similar issues since around the same time.
None of my Azure functions that are using the partner centre PowerShell module are working due to whatever issue this is.
I have tested code on my work Azure account as well as my personal one with the same results.
Are your Az modules importing? I'm also getting this error at the moment:
The Function app may be missing a module containing the Connect-AzAccount command definition
In my troubleshooting, I've found the PartnerCenter module will only import when the system-assigned managed identity is disabled, when the managed identity is enabled the I get errors stating that the module could not be loaded.
As part of troubleshooting, I've been using the following run.ps1 code:
using namespace System.Net

param($Request, $TriggerMetadata)

Write-Host "PowerShell HTTP trigger function processed a request."

Get-PartnerCustomer

Push-OutputBinding -Name Response -Value ([HttpResponseContext]@{
    StatusCode = [HttpStatusCode]::OK
})

With the Requirements.psd:
@{

    'Az' = '5.*'
    'PartnerCenter' = '3.*'

}

Other than that its the default HTTP request template.
With system managed identity disabled, I get the error:
[Error] ERROR: Run Connect-PartnerCenter to login.
With system managed identity enabled, I get the error:
[Error] ERROR: The 'Get-PartnerCustomer' command was found in the module 'PartnerCenter', but the module could not be loaded. For more information, run 'Import-Module PartnerCenter'.
Running  'Import-Module PartnerCenter' doesn't change anything.
I also am getting the following errors with the managed identity enabled:
[Error] ERROR: The term 'Disable-AzContextAutosave' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

[Error] ERROR: The term 'Connect-AzAccount' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

Thanks @AusXTech as Az.Account is required for MSI the conflict seems to come from there, my function is successfully signing in using the MSI and getting Key Vault access. I'll try with the specific version of Az.Accounts and go from there.

Same issue here. I have tried even in PowerShell 7.1
Import-Module -Name Az.Accounts -RequiredVersion 2.1.2
Import-Module -Name Az.Resources -RequiredVersion 3.0.0
Import-Module -Name Az.ManagementPartner -RequiredVersion 0.7.2
Import-Module -Name Az.Storage -RequiredVersion 3.0.0
Import-Module -Name Az.Monitor -RequiredVersion 2.2.0
Import-Module -Name PartnerCenter -RequiredVersion 3.0.10
Import-Module: Assembly with same name is already loaded
Exception             :
    Type       : System.IO.FileLoadException
    Message    : Assembly with same name is already loaded
    TargetSite :
        Name          : Bind_LoadAssemblies
        DeclaringType : initialsessionstate
        MemberType    : Method
        Module        : System.Management.Automation.dll
    StackTrace :
   at System.Management.Automation.Runspaces.InitialSessionState.Bind_LoadAssemblies(ExecutionContext context)
   at System.Management.Automation.Runspaces.InitialSessionState.Bind(ExecutionContext context, Boolean updateOnly, PSModuleInfo module, Boolean noClobber, Boolean local, Boolean setLocation)
   at Microsoft.PowerShell.Commands.ModuleCmdletBase.LoadModuleManifest(String moduleManifestPath, ExternalScriptInfo manifestScriptInfo, Hashtable data, Hashtable localizedData, ManifestProcessingFlags manif
estProcessingFlags, Version minimumVersion, Version maximumVersion, Version requiredVersion, Nullable`1 requiredModuleGuid, ImportModuleOptions& options, Boolean& containedErrors)
    Source     : System.Management.Automation
    HResult    : -2146232799
CategoryInfo          : InvalidOperation: (:) [Import-Module], FileLoadException
FullyQualifiedErrorId : FormatXmlUpdateException,Microsoft.PowerShell.Commands.ImportModuleCommand
InvocationInfo        :
    MyCommand        : Import-Module
    ScriptLineNumber : 1
    OffsetInLine     : 1
    HistoryId        : 13
    Line             : Import-Module -Name PartnerCenter -RequiredVersion 3.0.10
    PositionMessage  : At line:1 char:1
                       + Import-Module -Name PartnerCenter -RequiredVersion 3.0.10
                       + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    InvocationName   : Import-Module
    CommandOrigin    : Internal
ScriptStackTrace      : at <ScriptBlock>, <No file>: line 1

@slavizh I haven't been able to make it work with Az.Accounts 2.1.2 despite the fix mentioned here https://github.com/Azure/azure-functions-powershell-worker/issues/552
Trying today with 1.9.5, I might import it in the modules folder

@penicaudm  I am not even using functions but downgrading to 1.9.5 is not an option as actually I am trying to upgrade our scripts to Az 5.0.0 so if this is a problem I would just not upgrade at this time.

This works fine if I try to import first the partner center module but it is not something I want to do as import it only when I have to use it.

Ok, same here, partner center module broken in Azure functions, hours wasted. Any news on this issue?
I get:
2020-11-10T11:59:41.924336985Z: [INFO] 2020-11-10T11:59:41.924684085Z: [INFO]        Exception             : 2020-11-10T11:59:41.924706485Z: [INFO]            Type           : System.Management.Automation.CommandNotFoundException 2020-11-10T11:59:41.924711985Z: [INFO]            ErrorRecord    : 2020-11-10T11:59:41.924716285Z: [INFO]                Exception             : 2020-11-10T11:59:41.925030985Z: [INFO]                    Type    : System.Management.Automation.ParentContainsErrorRecordException 2020-11-10T11:59:41.925043685Z: [INFO]                    Message : The 'New-PartnerAccessToken' command was found in the module 'PartnerCenter', but the module could not be loaded. For more information, run 'Import-Module PartnerCenter'. 2020-11-10T11:59:41.925048685Z: [INFO]                    HResult : -2146233087 2020-11-10T11:59:41.925302185Z: [INFO]                TargetObject          : New-PartnerAccessToken 2020-11-10T11:59:41.925314785Z: [INFO]                CategoryInfo          : ObjectNotFound: (New-PartnerAccessToken:String) [], ParentContainsErrorRecordException 2020-11-10T11:59:41.925319785Z: [INFO]                FullyQualifiedErrorId : CouldNotAutoloadMatchingModule 2020-11-10T11:59:41.925517085Z: [INFO]                InvocationInfo        : 2020-11-10T11:59:41.925529385Z: [INFO]                    ScriptLineNumber : 99 2020-11-10T11:59:41.925533985Z: [INFO]                    OffsetInLine     : 24 2020-11-10T11:59:41.925538285Z: [INFO]                    HistoryId        : 1 2020-11-10T11:59:41.925542485Z: [INFO]                    ScriptName       : /home/site/wwwroot/TimerTrigger1/run.ps1

I've not been able to fix it yet, even with Az.Accounts 1.9.5. I'll let you know if I manage to find a common version all the modules can load (1.9.5 of Az.Accounts gave me errors with AzTable, Az.Storage and Az.Resources...)

@penicaudm Ok, thanks for the update, this is serious, hopefully someone on the Microsoft team can have a look soon.

Just tested in PS 7.1 and issue is still present.

The PowerShell Az team reported that the problems I have - the one specific to PS 7.1  and other that is not specific to PS version should be fixed by the Partner Center module team. Statement is located here: https://github.com/Azure/azure-powershell/issues/13423

You can now load Partner Center 3.0.10 before loading Az.Accounts 2.2.1 but when you try to use New-PartnerAccessToken you will get error:
AADSTS9002313: Invalid request. Request is malformed or
     | invalid. Trace ID: 56dff39f-e523-456e-b2b2-46a073b78800
     | Correlation ID: 8ca604e3-ba2c-401a-8c7a-bcda1ed0fdef
     | Timestamp: 2020-11-18 14:01:44Z

Any news to this issue? Anyone been able to find a combination that actually works?


Admin can log in to Farmer Dashboard

Description
Any admin account can be logged in to the farmer dashboard, run through the tutorial, but will encounter a server error upon finishing it and re-directing the account to dashboard.
Reproduction steps

Log in an admin account on a farmer dashboard
Proceed through the tutorial until the last slide
Click 'Handa Na!'

Expected behavior
The admin can not log in through the farmer dashboard.
Relevant log output
No response
Affected browsers (if applicable)
No response

The back-end should catch this and prevent this from happening.


v4.9.3 not exist anymore?

https://github.com/sass/node-sass/releases/download/v4.9.3/win32-x64-67_binding.node
Got 404.
Why?

npm update node-sass


chore: update workflows

:tada: This PR is included in version 5.0.3 :tada:
The release is available on:

v5.0.3
GitHub release

Your semantic-release bot :package::rocket:


Back to WPT master branch

Description
Test against the WPT master branch after the upstream PR was merged

WPT report for this branch: https://wpt-master-wpt-results--anchor-position-wpt.netlify.app

@jgerigmeyer I compared these results with the results from main  and it looks like the results are the same, except that there's two new tests:

anchor-default-basics.html 6 of 40 passing
anchor-name-in-shadow-002.html Already passing 🎉

Hmm, it looks like we're having issues installing Playwright system deps.  Removing the apt-spy2 trick fixes it 🤷♂
I think we can remove it since it should be temporary anyways and considering it can potentially point us to mirrors that cause conflicts


Improve API client UX

Right now the usage is a multi-step process including at least:

get the providers registry
filter registry for desired providers
configure resulting providers with auth_token etc., assuming you know how to generate all the different types of tokens
finally, use a ProviderClient to get_status_changes() or get_trips()

It is also somewhat confusing to have a ProviderClient that "manages" or queries multiple Providers at once. The code is probably over-complicated because of this as well...
This could all be simplified with some specific improvements to this class:

[x] allow the providers keyword arg in both __init__ and the get_* methods to accept a string or array of string provider names. Internally handle filtering the registry (ProviderClient already can query and obtain the registry itself).

[x] allow for a config parameter in __init__ and the get_* methods to pass in provider configuration, vs. having to configure them ahead of time.

[x] consider removing support for "managing" multiple providers at once - one Provider, one ProviderClient.

another use case: i would like to use the API client modules but am not interested in the db loading. from that perspective it may be logical to split the api/db purposes into separate packages.
i'll be spending more time with this project in the coming weeks. hope i can contribute.

consider removing support for "managing" multiple providers at once - one Provider, one ProviderClient

+1

another use case: i would like to use the API client modules but am not interested in the db loading. from that perspective it may be logical to split the api/db purposes into separate packages

I'm not sure I see the benefit here. The only dependency related to mds.db that isn't also used elsewhere is sqlalchemy; there isn't much to be gained in that regard from separating api and db.
Of course you don't have to import mds.db! Is there another concern that I'm not thinking of?

True. I'm mostly thinking of the pip install experience. My install happens to be choking on geopandas, but I haven't spent any time on it yet (and I'd be surprised if that's hard to resolve). Definitely less of a concern if the dependencies are shared.

as an aside, looks like geopandas dependency can easily be dropped. will open new issue/pr.

It's only used here: https://github.com/CityofSantaMonica/mds-provider/blob/dc5b7e0b2d74a89fbcbd2b923548f7378e567eb3/mds/json.py#L70
To convert a shapely.geometry.Shape into a GeoJSON object. Probably extreme overkill to use an entire library just for that 😆 Happy to accept a PR to remove that dependency.

The work I'm doing in #46 and #45 is moving towards "one provider -> one ProviderClient". This seems to be independent of the two other items listed in this issue (provider names & provider config).

Thanks to #40 and #71 the above changes will be made, including the move to a single provider per ProviderClient.
Initialize with a provider name or identifier directly:
client_A = ProviderClient("ProviderA")
client_B = ProviderClient("da80482e-1118-48e8-bccc-38492960bd17")

# optionally pass in configuration data
client_C = ProviderClient("ProviderC", config={ "token": "private" })

Use a provider name or identifier at request time:
client = ProviderClient()

trips_A = client.get_trips("ProviderA")
trips_B = client.get_trips("da80482e-1118-48e8-bccc-38492960bd17")

# optionally pass in configuration data
trips_C = client.get_trips("ProviderC", config={ "token": "private" })


Add configs for readonly and writeonly proxies.

Implements #69.

This change is

ping @buchgr

@buchgr is OOO until Feb 25

ping @buchgr

ping @buchgr

We could really use this feature.
@uri-canva would you reopen this if @buchgr would accept it?
How about it @buchgr 🙏?

We're not using this patch anymore ourselves, as it didn't fit our purpose as well as I thought, you're welcome to base a new PR on it though.

Since there are a few interrelated issues/PRs on this topic, I want to make sure that I understand the use case...
Is this the scenario:

There's a central CI system with its own cache. This is trustworthy because the CI setup is locked down.
There is a developer-local cache, which is more easily poisoned.
The link between the central and developer-local cache is slow enough that you don't want synchronous lookups between them.
You want the central cache to asynchronously populate the developer-local cache, but not to read from it.
(Not sure if this was requested, but do you potentially want multiple developer-local caches to work like this, with a single central cache?)

I’ll let @uri-canva answer for their situation, but for us it is much simpler, we only really need the writes_disabled setting:

Central CI cache. Written to by CI, since it’s trusted to not poison it.
Local developer cache on each machine. Used since disk_cache doesn’t clean up, and implementing naive cleanup in a non-AC aware way breaks Builds without the Bytes.
Local developer cache points it’s proxy to CI cache with writes_disabled to get remote cache benefits without possibility of poisoning it.

Disregard the note around disable_reads above, we have it set up differently now because it wasn't working that well anyway. We were using disable_writes exactly the same way as @brentleyjones: to use bazel-remote as a local disk cache, given the discussion in https://github.com/bazelbuild/bazel/issues/5139.

Essentially, if disk_cache could set a size limit and clean up in an LRU way, we wouldn’t need this solution at all. This allows bazel-remote to fill that purpose instead.

We use it to fall back to reading from a remote cache too, which Bazel doesn't do.


Question about a Regex

What's the point of the [a-z0-9\-_]* component of the [a-z0-9\-_]*(bot|crawler|archiver|transcoder|spider|uptime|validator|fetcher) pattern? It's a * pattern, so since it's entirely optional, shouldn't it just be excluded?

is not a wildcard in regex. It means "some or none" of the characters between the brackets. HTH!

(Meaning that other characters may appear begore the word bot or crawler, etc...)

Yes, I understand. My point is that since it's some or none, it's optional, so 5bot and bot would match all the same even if the pattern was simply (bot|crawler|archiver|transcoder|spider|uptime|validator|fetcher).

@dliebner You got it. Its so we can return the whole bot name
These show the differences...
https://regex101.com/r/uJab18/1
https://regex101.com/r/uJab18/2


Add hyperlink to element text, on  tags with a href value

This addresses issue #2.
I welcome any feedback.

That's exactly what I had in mind!


Add column to person for census DOB groups

Description
With the addition of Schools product is interested in use age to track what portion of tests is happening to students vs staff.
Proposed solution
Create new enum column in the no PHI person table that uses the age() function and maps the persons DOB to 5 year buckets. (<5, 5-9, 10-14,..., >85)
Additional context
https://www.census.gov/data/tables/2019/demo/age-and-sex/2019-age-sex-composition.html

Merged + deployed to test


FIX-#4450: Ensure Modin successfully initializes when Ray cluster has no resources

Signed-off-by: Rehan Durrani<EMAIL_ADDRESS>
What do these changes do?
This patch ensures that Modin can initialize successfully even when the ray cluster has no resources (because it is autoscaling from 0).

[x] commit message follows format outlined here
[x] passes flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py
[x] passes black --check modin/ asv_bench/benchmarks scripts/doc_checker.py
[x] signed commit with git commit -s
[x] Resolves #4450 ?
[ ] tests added and passing
[ ] module layout described at docs/development/architecture.rst is up-to-date
[x] added (Issue Number: PR title (PR Number)) and github username to release notes for next major release

While this patch does fix the issue, I think it would be helpful to have a discussion around how we want Modin to behave in such scenarios (i.e. whether it should just use some number, say 16, for the number of partitions until the user specifies how many partitions they want, whether it should ask the user to set the number of partitions manually, and not set num partitions, or whether it should launch ray tasks to cause the autoscaler to spin up workers, and then get the number of CPUs).

I double checked dask, and it uses CpuCount.get which should work even if its an autoscaling cluster!

While this patch does fix the issue, I think it would be helpful to have a discussion around how we want Modin to behave in such scenarios (i.e. whether it should just use some number, say 16, for the number of partitions until the user specifies how many partitions they want, whether it should ask the user to set the number of partitions manually, and not set num partitions, or whether it should launch ray tasks to cause the autoscaler to spin up workers, and then get the number of CPUs).

Is there another (more complicated but better) solution, which is to wait until ray tasks launch, then get the number of CPUs, update the number of partitions, and propagate the change? I think you mentioned in the Discuss forum some challenges with this approach since we don't currently repartition when NumPartitions is updated. However, could this be a longer term approach?
In the nearer term, I think the current approach of showing a warning and defaulting to 4 (or some other number) is best for this edge case.

Another option, in contrast to surfacing a warning and setting a default, is to raise an Error with an error message that gives instructions to bypass the error with a config variable to ignore the error, or to set the default NPartitions manually.
The reason this might be better is that users in batch mode won't see a warning, but they will see an error (because the script will fail), and we don't want users to have mysterious behavior with Modin without knowing about it.


Revert #2230

bors r+
🤖

cc @kjeremy
At least, we tricked @killercup to improve error handing by using anyhow and contexts :)

Haha, and I haven't even looked what autoconf is even supposed to do

Haha im all for subterfuge and trickery. Was this causing issues or just incorrect?


AssertionError: unknown FUN_DECL

The function
fun wordcount lines =
  let
    fun split0 [] word words = word :: words
      | split0 (#" " :: s) word words = split0 s "" (word :: words)
      | split0 (c :: s) word words = split0 s (word ^ (String_str c)) words
    fun split s = List_rev (split0 (String_explode s) "" [])
  in
    from line in lines,
        word in split(line)
    group word compute count of 1 as count
  end;

gives error
Exception in thread "main" java.lang.AssertionError: unknown FUN_DECL; fun split0 ([]) word words = word :: words | split0 (  :: s) word words = split0 s "" (word :: words) | split0 (c :: s) word words = split0 s (word ^ String_str c) words
	at net.hydromatic.morel.compile.Compiler.compileDecl(Compiler.java:327)
	at net.hydromatic.morel.compile.Compiler.compileLet(Compiler.java:301)

Also note that in the error message, split0 (  :: s) word words should be split0 (#" "  :: s) word words.

Simpler case:
fun foo arg =
  let
    fun baz x = x
  in
    baz arg
  end

Fixed in b988f3c62fe20efe0ba1b20661fbebd4c829b36e.


bug: [LoadingController] - Loading modal doesn't hide if dismiss is called when page is in the background

Bug Report
Ionic version:
[x] 4.x
Current behavior:
If you create and present a loading controller modal, it'll show without problem, but if you dismiss it when the tab or page is in the background (for example, looking at other tab in Chrome), the dismiss function won't work.
Expected behavior:
dismiss function should hide the loading modal all the times.
Steps to reproduce:

Create a loading controller modal
Present it
Set to hide after a couple of seconds (or with another function), needs to be automatically.
Quickly, change browser or tab so the ionic app became an inactive one.
Return after your dismiss conditions have been met
Check that the loading modal is still there

Ionic info:
Ionic:

   Ionic CLI                     : 5.2.1 (/Users/eduardoroth/.npm-global/lib/node_modules/ionic)
   Ionic Framework               : @ionic/angular 4.5.0
   @angular-devkit/build-angular : 0.800.4
   @angular-devkit/schematics    : 8.0.4
   @angular/cli                  : 8.0.4
   @ionic/angular-toolkit        : 2.0.0

Utility:

   cordova-res : 0.5.1
   native-run  : 0.2.7

System:

   NodeJS : v10.16.0 (/usr/local/bin/node)
   npm    : 6.9.2
   OS     : macOS Mojave

I'm having a similar issue, but when dismiss is called before the page is rendered because the HTTP calls finish too fast. Most prominent in test environments and low usage periods.

Thanks for the issue. Is this still an issue with the latest version of Ionic Framework? If so, please provide a repo with the code required to reproduce this issue.