[Bug] multiple deletion of best checkpoint while using dist_train.sh

Prerequisite

[X] I have searched Issues and Discussions but cannot get the expected help.
[X] I have read the FAQ documentation but cannot get the expected help.
[X] The bug has not been fixed in the latest version (master) or latest version (3.x).

Task
I have modified the scripts/configs, or I'm working on my own tasks/models/datasets.
Branch
3.x branch https://github.com/open-mmlab/mmdetection/tree/3.x
Environment
System environment:
sys.platform: linux
Python: 3.8.13 (default, Oct 21 2022, 23:50:54) [GCC 11.2.0]
CUDA available: True
numpy_random_seed: 429034773
GPU 0,1,2,3,4,5,6,7: NVIDIA A40
CUDA_HOME: /usr/local/cuda
NVCC: Cuda compilation tools, release 11.5, V11.5.119
GCC: gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10)
PyTorch: 1.10.0
PyTorch compiling details: PyTorch built with:

GCC 7.3

C++ Version: 201402

Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications

Intel(R) MKL-DNN v2.2.3 (Git Hash 7336ca9f055cf1bfa13efb658fe15dc9b41f0740)

OpenMP 201511 (a.k.a. OpenMP 4.5)

LAPACK is enabled (usually provided by MKL)

NNPACK is enabled

CPU capability usage: AVX512

CUDA Runtime 11.3

NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37

CuDNN 8.2

Magma 2.5.2

Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.2.0, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.10.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,
TorchVision: 0.11.0
OpenCV: 4.6.0
MMEngine: 0.3.0

Runtime environment:
cudnn_benchmark: False
mp_cfg: {'mp_start_method': 'fork', 'opencv_num_threads': 0}
dist_cfg: {'backend': 'nccl'}
seed: None
Distributed launcher: pytorch
Distributed training: True
GPU number: 8
Reproduces the problem - code sample
default_hooks = dict(
    logger=dict(type='LoggerHook', interval=interval),
    checkpoint=dict(
        interval=interval,
        max_keep_ckpts=3,
        save_best=['coco/bbox_mAP'],
          # only keep latest 3 checkpoints
    ))

Reproduces the problem - command or script
bash tools/dist_train.sh configs/rtmdet/rtmdet_x_8xb16_300e_panicle.py 8 --cfg-options load_from='p_weights/rtmdet_x_8xb32-300e_coco_20220715_230555-cc79b9ae.pth'

Reproduces the problem - error message
Traceback (most recent call last):
  File "tools/train.py", line 120, in <module>
    main()
  File "tools/train.py", line 116, in main
    runner.train()
  File "/data/home/rlchen/app/miniconda3/envs/mmdet/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1661, in train
    model = self.train_loop.run()  # type: ignore
  File "/data/home/rlchen/app/miniconda3/envs/mmdet/lib/python3.8/site-packages/mmengine/runner/loops.py", line 96, in run
    self.runner.val_loop.run()
  File "/data/home/rlchen/app/miniconda3/envs/mmdet/lib/python3.8/site-packages/mmengine/runner/loops.py", line 351, in run
    self.runner.call_hook('after_val_epoch', metrics=metrics)
  File "/data/home/rlchen/app/miniconda3/envs/mmdet/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1723, in call_hook
    getattr(hook, fn_name)(self, **kwargs)
  File "/data/home/rlchen/app/miniconda3/envs/mmdet/lib/python3.8/site-packages/mmengine/hooks/checkpoint_hook.py", line 297, in after_val_epoch
    self._save_best_checkpoint(runner, metrics)
  File "/data/home/rlchen/app/miniconda3/envs/mmdet/lib/python3.8/site-packages/mmengine/hooks/checkpoint_hook.py", line 412, in _save_best_checkpoint
    self.file_client.remove(best_ckpt_path)
  File "/data/home/rlchen/app/miniconda3/envs/mmdet/lib/python3.8/site-packages/mmengine/fileio/file_client.py", line 346, in remove
    self.client.remove(filepath)
  File "/data/home/rlchen/app/miniconda3/envs/mmdet/lib/python3.8/site-packages/mmengine/fileio/backends/local_backend.py", line 416, in remove
    raise FileNotFoundError(f'filepath {filepath} does not exist')
FileNotFoundError: filepath /data/home/rlchen/ProJect/mmdetection/work_dirs/rtmdet_x_8xb16_300e_panicle/best_coco/bbox_mAP_epoch_1.pth does not exist
11/05 14:56:10 - mmengine - INFO - The best checkpoint with 0.0020 coco/bbox_mAP at 7 epoch is saved to best_coco/bbox_mAP_epoch_7.pth.

Additional information
No response

https://github.com/open-mmlab/mmengine/pull/682 has been merged. Please update mmengine to the latest.


"....moc: not found" when run npm i ic-mops -g

>npm i ic-mops -g
npm WARN deprecated<EMAIL_ADDRESS>Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.

added 456 packages in 10s

122 packages are looking for funding
  run `npm fund` for details
CLI 0.41.1
API 1.2
Installing<EMAIL_ADDRESS>[          ]
Installing<EMAIL_ADDRESS>[:         ]
Installing<EMAIL_ADDRESS>[::        ]
Installing<EMAIL_ADDRESS>[:::       ]
Installing<EMAIL_ADDRESS>[::::      ]
Installing<EMAIL_ADDRESS>[:::::     ]
Installing<EMAIL_ADDRESS>[::::::    ]
Installing<EMAIL_ADDRESS>[:::::::   ]
Installing<EMAIL_ADDRESS>[::::::::  ]
Installing<EMAIL_ADDRESS>[::::::::: ]
Installing<EMAIL_ADDRESS>[::::::::::]
Installing https://github.com/aviate-labs/uuid.mo#v0.2.0 [          ]
Installing https://github.com/aviate-labs/uuid.mo#v0.2.0 [::::::::::]
Dependency https://github.com/dfinity/motoko-base#51758e16085f212752cd2442683a79
368f9fe3fa [          ]
Dependency https://github.com/dfinity/motoko-base#51758e16085f212752cd2442683a79
368f9fe3fa [:         ]
Dependency https://github.com/dfinity/motoko-base#51758e16085f212752cd2442683a79
368f9fe3fa [::::::::::]
Dependency https://github.com/aviate-labs/encoding.mo#v0.3.1 [          ]
Dependency https://github.com/aviate-labs/encoding.mo#v0.3.1 [::::::::::]
Dependency https://github.com/dfinity/motoko-base#51758e16085f212752cd2442683a79
368f9fe3fa (local cache)
Dependency https://github.com/aviate-labs/array.mo#v0.1.1 [          ]
Dependency https://github.com/aviate-labs/array.mo#v0.1.1 [::::::::::]
Dependency https://github.com/aviate-labs/io.mo#v0.3.0 [          ]
Dependency https://github.com/aviate-labs/io.mo#v0.3.0 [::::::::::]
Installing https://github.com/aviate-labs/array.mo#v0.1.1 (local cache)
Installing https://github.com/aviate-labs/crypto.mo#v0.2.0 [          ]
  signal: null,
  output: [
    null,
    Buffer(0) [Uint8Array] [],
    Buffer(71) [Uint8Array] [
       47,  98, 105, 110,  47, 115, 104,  58,  32,  49,  58,
       32,  47, 104, 111, 109, 101,  47, 1[14](https://github.com/modclub-app/decide_id/actions/runs/8492731296/job/23266042339?pr=19#step:9:15), 117, 110, 110,
      101, 114,  47,  46,  99,  97,  99, 104, 101,  47, 100,
      102, 105, 110, 105, 116, 121,  47, 118, 101, 114, 1[15](https://github.com/modclub-app/decide_id/actions/runs/8492731296/job/23266042339?pr=19#step:9:16),
      105, 111, 110, 115,  47,  48,  46,  49,  52,  46,  51,
       47, 109, 111,  99,  58,  32, 110, 111, 1[16](https://github.com/modclub-app/decide_id/actions/runs/8492731296/job/23266042339?pr=19#step:9:17),  32, 102,
      111, 1[17](https://github.com/modclub-app/decide_id/actions/runs/8492731296/job/23266042339?pr=19#step:9:18), 110, 100,  10
    ]
  ],
  pid: 1985,
  stdout: Buffer(0) [Uint8Array] [],
  stderr: Buffer(71) [Uint8Array] [
     47,  98, 105, 110,  47, 115, 104,  58,  32,  49,  58,
     32,  47, 104, 111, 109, 101,  47, 114, 117, 110, 110,
    101, 114,  47,  46,  99,  97,  99, 104, 101,  47, 100,
    102, 105, 110, 105, 116, 121,  47, 1[18](https://github.com/modclub-app/decide_id/actions/runs/8492731296/job/23266042339?pr=19#step:9:19), 101, 114, 115,
    105, 111, 110, 115,  47,  48,  46,  49,  52,  46,  51,
     47, 109, 111,  99,  58,  32, 110, 111, 116,  32, 102,
    111, 117, 110, 100,  10
  ]
}

Node.js v18.17.0```

Can you share mops.toml contents?

clear the dfx cache with
dfx cache install


Add Mk 2/3 versions

Only mk1 supported currently.

To avoid spoilers, I will leave this out, possibly add something in the future as a toggle to "show spoilers" but for now mk1 miner will show and E can be pressed to toggle mk2 etc...


Synchronize requirements and setup.py to make it work correctly as de…

…pendency.

I verified that it works as dependency by creating new virtualenv and then installing it by 'pip install .' and then running tests in different directory (but with active virtualenv).

Travis can't refresh pull request after amending the commit, so I'm closing it and I'll open another one


Ordering by a computed column

Hi,
Found my way here via https://github.com/dotnet/efcore/issues/20967#issuecomment-983857486 and frankly your project is brilliant and has solved our pagination woes in a repeatable and consistent way.
I have one corner case that I was hoping to discuss how it might be solved (I can attempt a PR for it if necessary). We would like to be able to order by an EF Core 'computed column'. Something like:
	[NotMapped]
	public short status_order { get; set; }

			modelBuilder.Entity<OurTable>()
				.Property(x => x.status_order)
				.HasComputedColumnSql(@"
					CASE status
						WHEN 0 THEN 0
						WHEN 3 THEN 1
						WHEN 2 THEN 2
						WHEN 1 THEN 3
						WHEN 99 THEN 99
						ELSE -1
					END
				");

			var keysetContext = ourTable
				.KeysetPaginate(
					b => b.Ascending(p => p.status_order),
					KeysetPaginationDirection.Forward,
					reference
				);

                        var items = await paginationContext
			    .Query
			    .Take(pageSize)
			    .ToListAsync(cancellationToken);

Right now this builds a SQL query with an ORDER BY clause that directly refers to the computed (not actually existing) column:
...
ORDER BY e.status_order

Which obviously does not work.
So initially by query is two-fold:

Is this even possible at all right now with EF Core?
If yes, can you give me some pointers as to how you might go about attacking this so I can have a go myself?

Hi. Glad it was of help.
So, for the query to work, the generated sql shouldn't be qualifying it with e.. This library only dynamically creates the comparison expressions and feeds them back to EF, it's not involved in translating anything, so the problem here is from EF itself most likely.
Does this work if you don't use KeysetPaginate at all (and just use an OrderBy directly) or does the same thing happen? If the same happens, that means maybe it's an EF bug? It seems to me this should work as is from the code you've shown.

You're right, dropping MR.EntityFrameworkCore.KeysetPagination out completely and doing an OrderBy through EF Core generates the 'bad' SQL against the non-existent column - thanks for the pointer!


revisions during RFC phase

This PR is for ongoing revisions from the forum thread: https://forum.makerdao.com/t/mip15-dark-spell-mechanism/2578

LGTM


URL for GET request does not include version

I have used the following code to generate the swagger documentation:
add_swagger_documentation(api_version: "v1", add_version: true, mount_path: "/api/v1/swagger_doc", hide_documentation_path: true, hide_format: true)
However the GET request URL in the documentation is just /api/<model_name>. The v1 is missing in the GET request URL.
How do I fix this?

I'll have a look on it

did it, and the spec api_swagger_v2_global_configuration says, it is ok

Unfortunately, if you look at the documentation created here: https://trial-promoter-app-praveenangyan.c9users.io/api_documentation#!/clinical_trials/getApiClinicalTrials,
it still looks like the get and the post requests are not using /api/v1 in the generated documentation schema.

I've fixed this using routes that route from /api/resource_name to /api/v1/resource_name, so I am good for now.

Not sure if I'm doing something wrong too, but for future users, I solved this by downgrading to 0.10.5

fixed here: https://github.com/ruby-grape/grape-swagger/pull/405


use native crypto when available

fixes #52, mostly based on my code from native-crypto

also #18

since async is only used in browsers had to remove it from the coverage tests

and we're green, one thing though that concerns me is that now environments like mobile safari while be drastically slower compared to environments that have subtle crypto and in all likelihood going to freeze the app, so thinking about ways to make it non blocking are probably a good idea as a next step.

anyway but that would be a whole different thing

Going to close and prefer just https://github.com/crypto-browserify/pbkdf2/pull/54 - as much to review tbh


:racehorse: Adjust benchmarks that are too quick

Some of our benchmarks complete too quickly and this tends to lead to
inconsistent results. In general it seems results are pretty consistent
if each iteration takes at least 50ms to complete and ideally 100ms.
Having at least 100 iterations also helps since we report on the 95th
percentile and this gives a bit of room for a few outliers.
Adjusting our benchmarks to try and fit with these numbers. In some
tests I've just increased the amount of data we deal with and in others
we run the same operation multiple times (depending on what works best
for the operation being tested).
Verified that this does not add any significant overhead for the single
iteration of each test that we run during build.cmd (~40sec for EF7 tests
and ~80sec for EF6 tests on my laptop - which is generally pretty slow).

:shipit:


Normalization method with Seurat Object.

Hi,

Thank you for creating such a great tool!
I am trying to use it with a Seurat object containing 130,000 cells, and I have a question about normalization for using this tool.

I noticed that in previous versions, a different normalization method was used instead of log normalization.
(https://github.com/Teichlab/cellphonedb/issues/279).
In your biorxiv, I saw that for cellphonedb V2.0 with Seurat object, you divided the expression of each gene in a cell by the total expression of the cell and multiplied by 10,000 for normalization.
However, in your 0_prepare_your_data_from_Seurat.ipynb script, I saw that log-normalization was performed on Seurat object.

In the previous versions, I thought that you recommended a different normalization method than log normalization, but it seems like the latest version is using log normalization.
I wanted to ask if it is okay to use log normalization for the latest versions.

Even though it is an old issue, I saw that when a user used log normalization, only 18 outputs were generated, whereas using raw count, 1800 results were obtained.
(https://github.com/Teichlab/cellphonedb/issues/12).
I think it shows that normalization is an important issue.

I tried to use a version of cellphoneDB that includes DEG analysis, but it wasn't the latest version.
When I used log normalization, I only got four results, which was too few.
I wondered if this was due to log normalization, but the latest version of cellphoneDB seems to be agnostic to any type of normalization.

As seen in this issue, methods 1 and 2 depend on mean values, so normalization is required, but it is stated that it is just a preferred normalization
(https://github.com/ventolab/CellphoneDB/issues/84).

Does this mean that any normalization, including log-normalization, is acceptable?

2. Or is there a recommended normalization method that you can recommend?
I want to find a way to improve my output because it is too low.
Should I use the method used in cellphoneDB V2.0? (count/sum(count) * 10000)

Also, since there are 130,000 cells, the data is too large to run the tool in txt file format.
Therefore, I am trying to convert it to h5ad format.
Can I use sceasy for this? (https://github.com/Teichlab/cellphonedb/issues/321)
I used the r tool, saveH5seurat, to convert it because I am not familiar with Scanpy.
Can the conversion process affect the results? Do you still recommend using sceasy to convert Seurat to h5ad?

Sorry for the long question.
I want to use your tool, but I have many questions because it doesn't work as I expected, and I've been searching for solutions.

Thank you so much for creating such a great tool!

 Best regards,
Ji-Hye Choi.

Hi.
You can use any normalization method. If make use of the subsampling strategy you must indicate whether your data is log transformed or not subsampling-log.
I would suggest you to read your data employing seurat and save it as a h5ad file. Both methods should convert the objects while keeping the exact values.
Kind regards


copy OTTableReader before copying/pickling lazily loaded table

Fixes https://github.com/fonttools/fonttools/issues/2965
In otTables.py we have a custom implementation of ClipList.postRead where we build a clips dictionary of ClipBox records keyed by glyph name, ensuring each value is a distinct object (they may be shared for ranges of glyph IDs in the binary table):
https://github.com/fonttools/fonttools/blob/f73220816264fc383b8a75f2146e8d69e455d398/Lib/fontTools/ttLib/tables/otTables.py#L1325-L1332
We call copy.copy(rec.ClipBox). If the TTFont was loaded lazily like the subsetter does by default (unless one passes --no-lazy), then the ClipBox record is only decompiled when any of its attribute is accessed (thanks to our magic BaseTable.__getattr__).
In python <= 3.10 we didn't need to do anything special and calling copy.copy on a lazy object would automatically decompile it (I don't know exactly how, something again about our magic __getattr__).
Whereas since python3.11, calling copy.copy(rec.ClipBox) does not automatically decompile our lazy objects.
So we should define a custom __getstate__ as explained in the python docs to control how an object state gets copied/pickled.
In the case of lazy objects we could either call ensureDecompiled before returing self.__dict__, or better keep the object in lazy mode make a shallow copy of its OTTableReader. This way each distinct copy owns a different reader and can be un-lazified independently instead of stepping on each other's toes.
I hope that makes sense. It does fix the bug @RoelN found out in #2956, now I need to figure out how to test this..

Makes sense. Thanks for figuring this out!

ensureDecompiled

actually calling ensureDecompiled in __getstate__ (or in __copy__) would be weird in that it would modify the state of self in a method that is supposed to "get" something (as opposed to change the object), so I think this makes more sense

Maybe it's because of this change in python 3.11:
https://docs.python.org/3/library/pickle.html#object.__getstate__

Changed in version 3.11: Added the default implementation of the getstate() method in the object class.

Since BaseTable didn't define its own __getstate__, and since in python < 3.11 neither did the super class object, then the custom BaseTable.__getattr__ method would be triggered whenever copy or pickle modules attempt to get __getstate__ through getattr() built-in, and the custom __getattr__ would automatically decompile our lazy objects.
Whereas with 3.11, since object parent class does implement a default __getstate__, the __getattr__ is not triggered and the lazy object remained lazy. But then each shallow copy were still sharing the same reader, which is single-use only, so decompiling one instance would destroy the other one.

after merging #2970 into this PR branch that includes the fix, the tests that were passing on 3.10 but failing on 3.11 now pass on both


fix: decode as utf-8 only text files

closes https://github.com/frouriojs/create-frourio-app/issues/180


file_cmds: init at 264.1.1

Motivation for this change
This will add the "file_cmds" derivation to apple-source-releases.
Things done

[ ] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] Linux

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Fits CONTRIBUTING.md.

what does file_cmds do?

file_cmds provides a lot of the same commands provided by GNU coreutils. Currently the commands are:

chflags (not in coreutils)
chmod
chown
cksum
compress
cp
dd
df
du
gzip
install
ipcrm (not in coreutils)
ln
ls
mkdir
mkfifo
mknod
mv
pathchk
pax (not in coreutils)
rm
rmdir
stat
touch

Some of the ones that are in coreutils have differences in options from their coreutils equivalent so they may still be useful.


Adding changes to support k3d-managed/local registries

Description

Added support to publish bicep recipes to local registries
Fixes

Type of change

This pull request fixes a bug in Radius and has an approved issue (issue link required).
This pull request adds or changes features of Radius and has an approved issue (issue link required).
This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).

https://github.com/radius-project/radius/issues/6648
Auto-generated summary

🤖 Generated by Copilot at 7fe4372
Summary
🧪🚀🔗

Added support for local registries without HTTPS for bicep publish and read commands. Modified pkg/cli/cmd/bicep/publish and pkg/rp/util/registry files.

Sing, O Muse, of the bicep publish command,
That lets the skilled developers of the land
Send their azure templates to local hosts
Without the need for HTTPS boasts.

Walkthrough

Allow publishing and reading bicep files to and from a local registry without HTTPS by setting the PlainHTTP field of the remote.Repository struct to true for local hosts (link, link, link, link)

Use the IsLocalhost function from the util package in pkg/rp/util/registry.go to check if a registry string is a local host (link, link, link)
Add a test case for the prepareDestination function in pkg/cli/cmd/bicep/publish/publish_test.go to verify the PlainHTTP field setting for a local registry (link, link)

I'd like to pause for a sec and have a chat about this. We should be really careful with making assumptions when security is involved.
http (in contrast with https) makes it really easy for requests to be hijacked by a malicious server. We need to tread carefully.
@vishwahiremat - can you log a bug for this first (or link the correct one)? The issue you linked isn't related to this.

@vishwahiremat discussed offline and we're going to make this opt-in for users since it weakens security. Stay tuned for a different solution 👍

@vishwahiremat discussed offline and we're going to make this opt-in for users since it weakens security. Stay tuned for a different solution 👍

The oras CLI treats localhost as a special case: https://github.com/oras-project/oras/blob/d9b2c5c0f0d69ab9fccb41e70240fb935bf8ab9c/cmd/oras/internal/option/remote.go#L322. Would running locally be a safe place?

@vishwahiremat discussed offline and we're going to make this opt-in for users since it weakens security. Stay tuned for a different solution 👍

The oras CLI treats localhost as a special case: https://github.com/oras-project/oras/blob/d9b2c5c0f0d69ab9fccb41e70240fb935bf8ab9c/cmd/oras/internal/option/remote.go#L322. Would running locally be a safe place?

We're going to have the same problem on pull inside the Recipes engine, where the registry URL will not be a localhost URL. The user is going to have to think about it anyway.

/ok-to-test

/ok-to-test


Update logits_process.py

This change enable hpu graph by moving:
"import habana_frameworks.torch.core as htcore" outside of the StaticWraper class
What does this PR do?

Fixes # (issue)
Before submitting

[ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[ ] Did you make sure to update the documentation with your changes?
[ ] Did you write any new necessary tests?

Closing this PR as it is a duplicate of #330.


congquyen99 - change css framework.

Change css framework from bootstrap to tailwindcss in Web and GloryAdmin project for easier styling.

well done.


About pretrained weight file loading errors that occur when visualizing with attnmap.py

When I was debugging attnmap.py, I got an error when I ran the following code
vssm.load_state_dict(AttnMamba.convert_state_dict_from_mmdet(torch.load(open(f"/media/olmman/sharkE/Ubuntu/VMamba/pretrained/mask_rcnn_vssm_fpn_coco_small_epoch_11.pth", "rb"), map_location="cpu")["state_dict"]), strict=False)

The error is as follows：
RuntimeError
shape '[96, 96]' is invalid for input of size 18432
File "/media/olmman/sharkE/Ubuntu/VMamba/classification/models/vmamba.py", line 50, in _load_from_state_dict
state_dict[prefix + "weight"] = state_dict[prefix + "weight"].view(self.weight.shape)
File "/media/olmman/sharkE/Ubuntu/VMamba/analyze/attn_visualize.py", line 110, in main_vssm
vssm.load_state_dict(AttnMamba.convert_state_dict_from_mmdet(torch.load(open(f"/media/olmman/sharkE/Ubuntu/VMamba/pretrained/mask_rcnn_vssm_fpn_coco_small_epoch_11.pth", "rb"), map_location="cpu")["state_dict"]), strict=False)
File "/media/olmman/sharkE/Ubuntu/VMamba/analyze/attn_visualize.py", line 278, in
main_vssm()
RuntimeError: shape '[96, 96]' is invalid for input of size 18432
I found that the prefix variable here represents'layers.0.blocks.0.op.in_proj weight 'in dimension [192, 96], and the model here has weight dimension [96, 96]
This doesn't match. Is it a problem with the pre-trained weights or the model settings?
In addition, I also refer to the configuration file of the pre-trained weights and change the
vssm: nn.Module = VSSM(
        depths=[2, 2, 8, 2],
        dims=[96, 192, 384, 768],
        ssm_d_state=1,
        ssm_ratio=1.0,
        ssm_dt_rank="auto",
        ssm_conv=3,
        ssm_conv_bias=False,
        forward_type="v05_noz",
        mlp_ratio=4.0,
        norm_layer="ln2d",
        downsample_version="v3",
        patchembed_version="v2",
    ).cuda().eval()

of setting VSSM in the original code to
vssm: nn.Module = VSSM(
        depths=[2, 2, 15, 2],
        dims=96, # [96, 192, 384, 768]
        ssm_d_state=1,
        ssm_ratio=1.0,
        ssm_dt_rank="auto",
        ssm_conv=3,
        ssm_conv_bias=False,
        forward_type="v05_noz", # v3_noz
        mlp_ratio=4.0,
        norm_layer="ln2d",
        downsample_version="v3",
        patchembed_version="v2",
        drop_path_rate=0.3
    ).cuda().eval()

but there is still the same problem.

The hyperparameter of the small scale is
def vmamba_small_s2l15(channel_first=True):
    return VSSM(
        depths=[2, 2, 15, 2], dims=96, drop_path_rate=0.3,
        patch_size=4, in_chans=3, num_classes=1000,
        ssm_d_state=1, ssm_ratio=2.0, ssm_dt_rank="auto", ssm_act_layer="silu",
        ssm_conv=3, ssm_conv_bias=False, ssm_drop_rate=0.0,
        ssm_init="v0", forward_type="v05_noz",
        mlp_ratio=4.0, mlp_act_layer="gelu", mlp_drop_rate=0.0, gmlp=False,
        patch_norm=True, norm_layer=("ln2d" if channel_first else "ln"),
        downsample_version="v3", patchembed_version="v2",
        use_checkpoint=False, posembed=False, imgsize=224,
    )

That may be the key to error weight loading.


Anonymise clients

Allow anonymisation for archived clients to comply with GDPR

@evanschris This might need a bit of frontend ❤

Coverage increased (+0.1%) to 91.466% when pulling 808438690af67cfa0737a34c880de25bcb4f0f11 on anonymise-clients into 201f22204de964aa7d4d3479d488c7fe7ebd036f on staging.


Update /etc/os-release based on stack.toml content

Summary
This change fixes #82 by reading the necessary adjustments for /etc/os-release from stack.toml. It is based on two mandatory fields name (pretty name of the stack) and homepage (already included). The additionally introduced fields support-url and bug-report-url are defaulted off from homepage. This replicates the capabilities currently implemented as part of the run-image Dockerfile and a separate os-release file. Afterwards the stacks should be updated accordingly.
Use Cases

Dynamically update /etc/os-release with data coming from stack.toml.

Checklist

[X] I have viewed, signed, and submitted the Contributor License Agreement.
[X] I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)
[X] I have added an integration test, if necessary.
[X] I have reviewed the styleguide for guidance on my code quality.
[X] I'm happy with the commit history on this PR (I have rebased/squashed as needed).

CC @phil9909

As far as the semver label, I was thinking minor. However, I could see an argument for major given that we are requiring new fields in stack.toml. Thoughts @paketo-buildpacks/stacks-maintainers @ryanmoran ?

This is a great refactoring. I've just got one comment about how the new fields should be handled in the definition file. Its not clear to me if we should mark them as required, or just not update them in the layer creator if they are not set in the definition. I'm open to thoughts on this.

No strong opinion if they should both be required as well. We wanted to make the update on the tiny stack as minimal as possible. We'd also be fine to just copy only the data given from stack.toml (name, homepage, support-url or bug-report-url) into /etc/os-release.
After adding the check, if homepage contains github.com, it does not need to be mandatory any more.

As far as the semver label, I was thinking minor. However, I could see an argument for major given that we are requiring new fields in stack.toml. Thoughts @paketo-buildpacks/stacks-maintainers @ryanmoran ?

The following just came to my mind: We could make name and homepage optional and in the internal/ihop/creator.go only call c.osReleaseLayerCreator.Create, if the necessary values are present. That way it would be completely backwards compatible.

I have one outstanding question about whether we want to fail if the homepage doesn't contain github.com or silently pass.
Other than that - looks good.

We even thought about removing that github.com logic entirely and leave it up to the stack owners to provide this information.
Once stacks are updated, e.g. remove the old /etc/os-release overwrite, fields like homepage, bug-report-url and support-url can be made mandatory.
WDYT?

I'm happy with this, and with the backwards-compatible changes we can label this: minor.
Thanks again for the PR @modulo11 @phil9909 @pbusko !

Merged - thank you everyone for your contributions and for your ongoing effort keeping this branch up to date with main while we worked through the feedback.


TypeError: Cannot read property 'target' of undefined

This happens at line 707 of raven.js for me while testing things.
browser = Chrome Mobile 42.0, level = error, logger = javascript, os = Android 4.4.2
Not exactly sure why it happens, maybe someone here has an idea?

Can confirm it also happens on Chrome Mobile 45.0 and 46.0.

I think, but are not sure, that error within my javascript is not created by an event such as a click, but just by code that is executed on page load. The tricky part is that I cannot reproduce any error in chrome on linux. I have seen the Sentry event now for 60 times already, so it is reproducable on the mobile (Android) device(s). Will continue investigate this...

What version of Raven.js? The latest master (3.0.4) doesn't have a target near line 707. Were you loading Raven.js from the CDN (e.g. cdn.ravenjs.com)?

It has evt.target in the dist of this repo?

Any news on this? I had already 174k of these errors in 5 months time..

@gabn88 – it's hard for me to make progress on this without a page to reproduce the error.
I suppose I can just bail out if there's no target on evt, but I'm not comfortable making that change without seeing the error that triggered it first (i.e. how did evt become undefined in a basic click/keypress handler?).

It is not a basic click/keypress handler. There are some timers running on the webapp that trigger events when the time is reached. So practically speaking, there is no target when the event triggers.
But I think there are many cases where an event is not triggered by a click/keypress/touchstart, etc...
Sometimes even a reload of the page can trigger an event, so that it just executes on page load.

Same error here guys! there are no errors being triggered on the page without sentry, however this same error on raven pops up when activating sentry.

Your failing example looks like a classic case of not binding this. This will do what you want:
setTimeout(this.onOverlayClick.bind(this), 100);

as will this:
var self = this;
setTimeout(function () { self.onOverlayClick(); }, 100);

as does your example with the arrow function thanks to how arrow functions handle this binding.
Just doing
setTimeout(this.onOverlayClick, 100);

will not bind this, and then things get weird from there. I'm not exactly sure, though, why the eventual outcome is this error specifically; @zanona do you have a minimum reproducing HTML page that we can play with it on?

Thanks @LewisJEllis, that’s very insightful.
There are two things that are confusing me a bit.

I am already binding all the class methods to its instance as this on the framework side;
No errors are throw without sentry and the functionality works as expected?

In the end raven error is not blocking any functionality since the methods still works just as fine, but seeing those errors throw in there was a bit strange.
I will to see if I can reproduce that in isolation and provide you with an example anytime soon.

@LewisJEllis if I understand correctly JS6 fat arrow => binds receiver to this which should be the case.

@sija - yes, I indicated that in my previous comment/linked to the MDN page on that behavior - that's why zanona's snippet in the screenshot doesn't trigger the problem
@zanona - interesting; at best this is a confusing scenario for users to debug, and at worst it's a bug in raven-js. It might be a case of "error elsewhere ends up looking like error in raven-js" (like in #756 for example), but if everything works as expected without raven-js then I can't be sure of that. Either way there's something to improve, so please do let me know if you can provide a repro (even if it's not necessarily small/simple).

Thansk @LewisJEllis will do.
I have event tried to wrap the error in a try…catch but it was getting anything from that and the error still being displayed on raven, so that is indeed strange.
I will hopefully be able to provide a reproduction soon. Thanks

@LewisJEllis true! I missed that part 😐


v2

This is a major update for the bot that overhauls most of it and makes it a lot better for moderation as well.
I worked weeks on this, I'm pretty sure I covered all the bases and made sure the code is fine and there are no bugs.
CHANGES:

/pingmods have been deprecated in support of /contactmods, which requires you to define a topic and further details, and is a ticket system, which allows users to put further details and screenshots in the tickets, instead of moderator DMs, compared to a simple ping before.
/purge exists now for moderators to quickly purge a certain amount of messages.
/closethread exists now for moderators to quickly shutdown a thread, meant for ticket closure once it's done and other threads.
Thread Owner functions now check if a message is pinned or unpinned before doing their respective actions
Logging is now a thing

Logs edited messages with full diff support (using the difflib library), pinned message checks, and suppressed embed checks
Logs deleted messages
Logs purged messages

Edited and deleted logs, if the text content of the message is over 250 characters, it'll send the message as a .diff and .txt respectively

Logs voice chat actions such as people joining, leaving, and screen sharing state

Makes it so we can actually moderate the voice chats somewhat

Logs users joining and leaving the server
Logs user bans, unbans, kicks, invite creation, and invite deletion.

Mod ticket system (The Great Elaboration)

Tickets, when made, will be entitled by the users nickname and their ID (the ID is for searching the ticket if it exists)
A ticket can be created by three actions, /contactmods, right click user > apps > Report User, and right click message > apps > Report Message
If no ticket is open by the ticket opener, then it'll make a new ticket, and the first report will ping mods into the ticket. And if for some reason moderators haven't closed the ticket once it's resolved or isn't resolved, and not auto closed by the bot, then the new report will be sent in the old ticket.
The bot checks ticket inactivity every 30 seconds, if the bot finds an open ticket that's been inactive for 24 hours, it'll auto close the ticket.

REQUIREMENTS:
Before this Pull Request can be merged, the bot now needs two new channels to be made.

A log channel (ex. #logs): This channel would only be accessible to moderators and higher. Preferably for no moderator abuse, make sure moderators can't speak or delete messages in the channel.
A contact mods channel (ex. #contact-mods): This channel would be public to the server, except people can't speak in that channel, nor react. The channel will have a message, most likely from you, GarboMuffin, or could be me, that explains how to use /contactmods, report user, and report message, and a general small Q/A when it comes to contacting mods.

RECOMMENDATIONS:

Rename the use /pingmods role for the bot to use /contactmods

PEAK ‼

Okay, turns out I forgot to add two last minute changes to the bot, small code errors, and I'm not going to be home for a bit, so I gotta do the changes before the merge. I'll try to see if I can do it while I'm away for now.

Alright, the PR is ready for merge

PEAK 💯 can't wait
On Fri, Sep 6, 2024 at 12:20 PM Cubester @.***> wrote:

Alright, the PR is ready for merge
—
Reply to this email directly, view it on GitHub
https://github.com/GarboMuffin/turbo-robot-open-source/pull/1#issuecomment-2334403368,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BE272PSKI5QX7SBBPOQR5B3ZVHI4PAVCNFSM6AAAAABNXS4SIGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZUGQYDGMZWHA
.
You are receiving this because you commented.Message ID:
@.***>

Mistake on my part for using GitHub's conflict resolver, I'll have to manually fix this


two rules don't exist in JavaScript

https://github.com/ruschaaf/extended-embedded-languages/blob/c128e894612034954006ef0e221641ddfcd3a43c/syntaxes/javascript.embedded.json#L45
https://github.com/ruschaaf/extended-embedded-languages/blob/c128e894612034954006ef0e221641ddfcd3a43c/syntaxes/javascript.embedded.json#L48

I'm seeing those rules in https://github.com/microsoft/vscode/blob/main/extensions/javascript/syntaxes/JavaScript.tmLanguage.json, but yeah the rules aren't getting used. I'll remove them for now

Just pushed a fix, will be in next release, thanks


Decrease mbedtls_aes_context_t size

Decrease the size of the aes context, on the expense of
heap size(at most 544 bytes),and adding to stack 88 bytes

cc @kl-cruz, @frkv

@andresag01 I went over most of your suggestions and changed accordingly

I will park the review until this PR is updated.

Closing this PR as not relevant anymore


Fix Windows path bug in GetCommitInfos

Fixes https://github.com/go-gitea/gitea/issues/1933. Also rename variable to avoid collision with path package, and update stale comment.

LGTM

LGTM


Missing syntax: ALTER TABLE ... MODIFY [ COLUMN ] ...

https://mariadb.com/kb/en/alter-table/#modify-column

See also https://github.com/DerekStride/tree-sitter-sql/issues/82#issuecomment-1416227381
Should be combined with #82.


Language agnostic procedure for test token creation

Can you describe the language agnostic procedure to create the test token?
The example EncryptWithAES shows how to use the .Net framework to create the test token, however I don't really know what's going on behind those library methods. I would like to replicate it's functionality using any other languaje.
Is there any Media Services API v3 method to generate this token?
Help will be very much appreciated,

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d49b5477-e862-d90b-07fc-18d9f17c1dd3
Version Independent ID: 35f47cc0-0d57-3241-b79d-7f742b168e38
Content: Encrypt video with AES-128 - Azure Media Services
Content Source: articles/media-services/latest/protect-with-aes128.md
Service: media-services
GitHub Login: @Juliako
Microsoft Alias: juliako

Hi @BernardoMB ,thanks for the feedback! We are currently investigating and will update you shortly.

Hi @BernardoMB, what language would you like to use? have you tried using the node.js examples here: https://github.com/Azure-Samples/media-services-v3-node-tutorials/tree/master/AMSv3Samples

Since we have not heard back from you we will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly reopen the issue and continue the discussion.


Custom Folders for Collection Items

Rather than having all SO created under the same named folder, I'd like to have each collection use it's own folder.
That way I can name the SO items without prefix and they are grouped in different folders.
For the moment I've added to top of ScriptableObjectCollection
        public string itemFolderName = "Items";

and line 134 change to
            string parentFolderPath = Path.Combine(assetPath, itemFolderName);
which allows me to set custom folder for custom Collections
    public class PlayerGameEventCollection : AtomEventCollection<PlayerGameEvent>
    {
        PlayerGameEventCollection() { itemFolderName = "Player"; }
    }

Not sure whether this will break things  - presume SO are found via reference from creation?
I found one more hardcoded "Items" in the ScriptableObjectCollectionUtility which seems to have only one reference from CreateCollectionWizard.cs - so just used via Wizard and can be changed after.

Yeah this falls a bit more into personal preference, for me every collection has his own folder, so:
Consumables (Folder)

ConsumablesCollection (ScriptableObject)
Items (Folder)|

ConsumableItem1
ConsumableItem 2

I think keeping all the collections on the same level can be quite messy and adding a folder name can be a solution, but honestly its not my prefered way

Sure - I just changed so I can edit but the default would still remain 'Items'.  Minor! As things build up I'll probably go with your way :)


Allow for -C (list entries by columns) flag - GNU coreutils 8.28

My old alias:
alias l='ls -CF'

set from some config or another throws an error when alias ls='lsd' is set.
error: Found argument '-C' which wasn't expected, or isn't valid in this context

I don't really know the difference between the outputs using this flag, but it'd be nice if lsd could handle the flag gracefully.
This is part of ls's api in GNU coreutils 8.28.

Some reference for -C
manpage: -C     list entries by columns
it is the default behavior for ls in stdout. for example:
stdout without -C:
λ ls
CHANGELOG.md CODEOWNERS   Cargo.lock   Cargo.toml   LICENSE      README.md    build.rs     ci           go.mod       go.sum       src          target       tests

stdout with -C:
λ ls -C
CHANGELOG.md CODEOWNERS   Cargo.lock   Cargo.toml   LICENSE      README.md    build.rs     ci           go.mod       go.sum       src          target       tests

it is NOT the default one beside stdout, pipeline for example:
pipeline without -C:
λ ls | cat
CHANGELOG.md
CODEOWNERS
Cargo.lock
Cargo.toml
LICENSE
README.md
build.rs
ci
go.mod
go.sum
src
target
tests

pipeline with -C
λ ls -C | cat
CHANGELOG.md    Cargo.toml  build.rs    go.sum      tests
CODEOWNERS  LICENSE     ci      src
Cargo.lock  README.md   go.mod      target

Related: https://github.com/Peltoche/lsd/issues/526


error running bundler

Note that this is on a system where I do not have admin access; bundle and jekyll were installed using the command gem install --user-install jekyll bundler. I originally wanted to run jekyll new avi-site, but I got a similar error report to what's below. So, I ran jekyll new avi-site --skip-bundle. Then (after cd-ing to ./avi-site) I tried to run bundle exec jekyll serve, and got the error message
Could not find gem 'minima (~> 2.0)' in any of the gem sources listed in your Gemfile.
Run `bundle install` to install missing gems.

So, I ran bundle install and got the following error report (modified as specified within it):
Error Report
Questions
Please fill out answers to these questions, it'll help us figure out
why things are going wrong.

What did you do?
I ran the command [HOME]/.gem/ruby/2.4.0/bin/bundle install

What did you expect to happen?
I expected Bundler to install Minima (I think?).

What happened instead?
Instead, what happened was I got this error report.

Have you tried any solutions posted on similar issues in our issue tracker, stack overflow, or google?
I tried searching various places for "Errno::EROFS: Read-only file system @ rb_sysopen" and "Errno::EROFS: Read-only file system", but nothing came up that seemed useful.

Have you read our issues document, https://github.com/bundler/bundler/blob/master/doc/contributing/ISSUES.md?
Yes.

Backtrace
Errno::EROFS: Read-only file system @ rb_sysopen - /pkgs/ruby-2.4.2/lib/ruby/gems/2.4.0/bundler.lock
  [HOME]/.gem/ruby/2.4.0/gems/bundler-1.16.0/lib/bundler/process_lock.rb:9:in `initialize'
  [HOME]/.gem/ruby/2.4.0/gems/bundler-1.16.0/lib/bundler/process_lock.rb:9:in `open'
  [HOME]/.gem/ruby/2.4.0/gems/bundler-1.16.0/lib/bundler/process_lock.rb:9:in `lock'
  [HOME]/.gem/ruby/2.4.0/gems/bundler-1.16.0/lib/bundler/installer.rb:72:in `run'
  [HOME]/.gem/ruby/2.4.0/gems/bundler-1.16.0/lib/bundler/installer.rb:25:in `install'
  [HOME]/.gem/ruby/2.4.0/gems/bundler-1.16.0/lib/bundler/cli/install.rb:65:in `run'
  [HOME]/.gem/ruby/2.4.0/gems/bundler-1.16.0/lib/bundler/cli.rb:224:in `block in install'
  [HOME]/.gem/ruby/2.4.0/gems/bundler-1.16.0/lib/bundler/settings.rb:136:in `temporary'
  [HOME]/.gem/ruby/2.4.0/gems/bundler-1.16.0/lib/bundler/cli.rb:223:in `install'
  [HOME]/.gem/ruby/2.4.0/gems/bundler-1.16.0/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
  [HOME]/.gem/ruby/2.4.0/gems/bundler-1.16.0/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
  [HOME]/.gem/ruby/2.4.0/gems/bundler-1.16.0/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
  [HOME]/.gem/ruby/2.4.0/gems/bundler-1.16.0/lib/bundler/cli.rb:27:in `dispatch'
  [HOME]/.gem/ruby/2.4.0/gems/bundler-1.16.0/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
  [HOME]/.gem/ruby/2.4.0/gems/bundler-1.16.0/lib/bundler/cli.rb:18:in `start'
  [HOME]/.gem/ruby/2.4.0/gems/bundler-1.16.0/exe/bundle:30:in `block in <top (required)>'
  [HOME]/.gem/ruby/2.4.0/gems/bundler-1.16.0/lib/bundler/friendly_errors.rb:122:in `with_friendly_errors'
  [HOME]/.gem/ruby/2.4.0/gems/bundler-1.16.0/exe/bundle:22:in `<top (required)>'
  [HOME]/.gem/ruby/2.4.0/bin/bundle:23:in `load'
  [HOME]/.gem/ruby/2.4.0/bin/bundle:23:in `<main>'

Environment
Bundler       1.16.0
  Platforms   ruby, x86_64-linux
Ruby          2.4.2p198 (2017-09-14 revision 59899) [x86_64-linux]
  Full Path   /pkgs/ruby-2.4.2/bin/ruby
  Config Dir  /pkgs/ruby-2.4.2/etc
RubyGems      2.6.13
  Gem Home    /pkgs/ruby-2.4.2/lib/ruby/gems/2.4.0
  Gem Path    /pkgs/ruby-2.4.2/lib/ruby/gems/2.4.0:[HOME]/.gem/ruby/2.4.0
  User Path   [HOME]/.gem/ruby/2.4.0
  Bin Dir     /pkgs/ruby-2.4.2/bin
OpenSSL
  Compiled    OpenSSL 1.0.2k  26 Jan 2017
  Loaded      OpenSSL 1.0.2k-fips  26 Jan 2017
  Cert File   /etc/pki/tls/cert.pem
  Cert Dir    /etc/pki/tls/certs
Tools
  Git         <IP_ADDRESS>
  RVM         not installed
  rbenv       not installed
  chruby      not installed

Bundler Build Metadata
Built At          2017-10-31
Git SHA           10f20fa33
Released Version  true

Gemfile
Gemfile
source "https://rubygems.org"

# Hello! This is where you manage which Jekyll version is used to run.
# When you want to use a different version, change it below, save the
# file and run `bundle install`. Run Jekyll with `bundle exec`, like so:
#
#     bundle exec jekyll serve
#
# This will help ensure the proper Jekyll version is running.
# Happy Jekylling!
gem "jekyll", "~> 3.6.2"

# This is the default theme for new Jekyll sites. You may change this to anything you like.
gem "minima", "~> 2.0"

# If you want to use GitHub Pages, remove the "gem "jekyll"" above and
# uncomment the line below. To upgrade, run `bundle update github-pages`.
# gem "github-pages", group: :jekyll_plugins

# If you have any plugins, put them here!
group :jekyll_plugins do
  gem "jekyll-feed", "~> 0.6"
end

# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby]

Gemfile.lock
<No [HOME]/WWW-local/avi-site/Gemfile.lock found>

Thanks for the report! Bundler is trying to write to /pkgs/ruby-2.4.2/lib/ruby/gems/2.4.0/bundler.lock but the filesystem/directory appears to be read only.
You will need to make the directory writable for Bundler, or specify a path that Bundler is able to write to. You can read more about specifying a path in bundle install on bundler.io

ping @foxjwill were you able to resolve your problem?

@colby-swandale Sorry about the late response. This is for my office computer, and I was at home for the weekend. A question about your response: Should I use the path [HOME]/.gem/ruby/2.4.0 (i.e. the User Path from the Environment section of the Error Report)? In other words, should I run bundle install --path=[HOME]/.gem/ruby/2.4.0?

You can use whatever path you like, though I'd recommend using a project-specific path, i.e. --path=.bundle

This seems to have worked.

Thanks for getting back to us, i'm closing this issue.


individual css for each project cost

right now, the "costs" indicator for a tile project turns red if any of the costs cannot be fulfilled. Ideally, we should instead only turn the costs that cannot be afforded red. This probably means making them separate html paragraphs.

I can do this


test: pkg/certificate/providers/certmanager/certificate_manager.go certificate request failure

In pkg/certificate/providers/certmanager/certificate_manager.go, the unit test coverage is low for the certificaterFromCertificateRequest() func. See highlighted lines below.

Scenarios not covered:

if given certificate request is nil
if certificate cannot be decoded from cr

ref #1489

Consolidated in #3435


[FLINK-16000] Move "Project Build Setup" to "Getting Started" in documentation

This is pretty much the first thing you have to do when programming with Flink, so it belongs in the "Getting Started" section, this is outlined in FLIP-42.

I agree with both of Konstantin's suggestions.
Maybe we could add a sentence that this is the setup for developing a Flink application.
You can still play with other parts without that, like launch a cluster, use SQL CLI, especially with the Docker playgrounds.

@knaufk I suggest to remove the icons from "Code Walkthroughs" and "Docker Playgrounds". None of the other sections except of the top-level sections have icons so that makes it more consistent. And I would then move "Project Setup" to last position.
@StephanEwen I will add some explanation in the Overview section.

@knaufk & @StephanEwen I did the changes, I also re-ordered them to make it internally consistent (see commit message). PTAL.

Thanks for the reviews! Merged.


test: enable coverage when running unit tests

What problem does this PR solve?
pingcap/parser doesn't have a coverage report of its own.
What is changed and how it works?
Enabled coverage report from test.sh.
Created a codecov project at https://codecov.io/gh/pingcap/parser.
Update coverage to codecov after make test succeed.
Check List
Tests

Unit test

Code changes
Side effects
Related changes

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@6454aba). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #324   +/-   ##
=========================================
  Coverage          ?   53.44%
=========================================
  Files             ?       31
  Lines             ?     6511
  Branches          ?        0
=========================================
  Hits              ?     3480
  Misses            ?     2692
  Partials          ?      339

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6454aba...b533f7d. Read the comment docs.

Note: EE team still needs to enable the CodeCov GitHub integration on this project.

PTAL @lamxTyler @tiancaiamao

Do we need to add a badge in readme?


RFP - regedix

Checklist

[x] The package I am requesting does not already exist on https://community.chocolatey.org/packages;
[x] There is no open issue for this package;
[x] The issue title starts with 'RFP - ';
[x] The download URL is public and not locked behind a paywall / login;

Package Details
Software project URL : https://regedix.webrox.fr/
Direct download URL for the software / installer : https://regedix.webrox.fr/download.php?v=<IP_ADDRESS>&file=<IP_ADDRESS>/Regedix_installer-2.0.0.exe
Software summary / short description: Windows Registry editor. Breadcrumb navigation, favorites, dark theme...

Package added: https://community.chocolatey.org/packages/regedix

Package approved


Finish implementation of dynamic reader

Thanks for this! It seems once again I had one type slightly wrong :slightly_smiling_face: . In contrast to your comment here:

I’m not convinced this gives the same sort of dynamic behaviour as effectful though. Your version is dynamic in the reader implementation. The effectful code has dynamic scoping of the effect itself.

As far as I can tell, this achieves the same level of dynamism as effectful/mtl. Let me try to sum up my position:
Static Reader
The static versions (Effectful.Reader.Static, transformers, Reader.Static) do the same thing: they provide a single interpretation for Reader's API. In particular, local provides a way to locally modify the environment, so while the environment is "dynamic", in a sense, the Reader operations are not. Everything is statically determined. In other words, there is only one way to implement runReader:
runReader
  :: HasCallStack
  => r -- ^ The initial environment.
  -> Eff (Reader r : es) a
  -> Eff es a
runReader r = evalStaticRep (Reader r)

Dynamic Reader
On the other hand, Effectful.Reader.Dynamic and mtl provide dynamism in the Reader effect itself. That is, we can provide custom interpretations:
runReader
  :: HasCallStack
  => r -- ^ The initial environment.
  -> Eff (Reader r : es) a
  -> Eff es a
runReader r = reinterpret (R.runReader r) $ \env -> \case
  Ask       -> R.ask
  Local f m -> localSeqUnlift env $ \unlift -> R.local f (unlift m)

We could, for instance, choose to apply f twice in local:
Local f m -> localSeqUnlift env $ \unlift -> R.local (f . f) (unlift m)

This is the same level of dynamic behavior allowed with typeclasses, and you have achieved in this PR:
diff --git a/src/Reader/Dynamic.hs b/src/Reader/Dynamic.hs
index 494ae03..78de7ef 100644
--- a/src/Reader/Dynamic.hs
+++ b/src/Reader/Dynamic.hs
@@ -65,7 +65,7 @@ runReader env k =
     MkReader
       { askImpl = pure env,
         localImpl = \modEnv onRdr ->
-          runReader (modEnv env) (useImplUnder . onRdr)
+          -- apply modEnv twice
+          runReader (modEnv $ modEnv env) (useImplUnder . onRdr)
       }

More Dynamism?
Can you elaborate on what you mean by:

The effectful code has dynamic scoping of the effect itself.

Maybe an example in effectful that cannot be implemented here?
Which effectful effect gets chosen will depend on which runReader interpreter is used, but it's the same for bluefin, right?
I can see how effectful might seem weirder, since the interpreter is chosen at the "outer" scope, whereas actual usage is in the inner scope. But it feels the same to me, just replacing bluefin's explicit argument passing with implicit typeclass dictionary passing.
Crucially, it seems like the expressiveness is the same? But maybe I'm missing something.

Can you elaborate on what you mean by:

The effectful code has dynamic scoping of the effect itself.

Crucially, it seems like the expressiveness is the same? But maybe I'm missing something.

Yeah, I think I was just wrong about this. Seems it all goes through fine in Bluefin: https://discourse.haskell.org/t/what-is-a-higher-order-effect/10744/17?u=tomjaguarpaw
Thanks for explaining!


chore: bump sbt-ci-release from 1.6.0 to 1.6.1

About this PR
📦 Updates com.github.sbt:sbt-ci-release from 1.6.0 to 1.6.1
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

🔍 Files still referring to the old version number
The following files still refer to the old version number (1.6.0).
You might want to review and update them manually.
website/package-lock.json

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.github.sbt", artifactId = "sbt-ci-release" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "com.github.sbt", artifactId = "sbt-ci-release" }
}]

labels: sbt-plugin-update, early-semver-patch, semver-spec-patch, old-version-remains, commit-count:1

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 67.90%. Comparing base (ae2e609) to head (343cb38).
Report is 1 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #287   +/-   ##
=======================================
  Coverage   67.90%   67.90%
=======================================
  Files          79       79
  Lines        1087     1087
  Branches      126      126
=======================================
  Hits          738      738
  Misses        349      349


Deprecate then eliminate max/min

moment().min and moment().max were a good idea for helper functions, but combined with the mutability that most moment() methods have it turned out disasterously confusing.
So we're deprecating those two in favor of moment.min and moment.max, which are non-mutating and work on any number of objects:
// if a, b, c, d are moments ordered in time
moment.min(a, b);  // a
moment.min(a, b, c, d);  // a
moment.min([a, b, c, d]);  // a
moment.max(a, b);  // b
moment.max(a, b, c, d);  // d
moment.max([a, b, c, d]);  // d

moment().min(start).max(end) was useful for clamping.
How about adding moment().clamp(start, end)?

That seems easy enough to add from userspace. Untested, but:
moment.fn.clamp = function(start, end){
   return moment.max(start, moment.min(end, moment());
}

Since I suspect this is relatively rare use case and it's easy to do from userspace, I'd prefer to keep out of the lib in the name of limiting the surface area.


slidesToScroll > 1 while centerMode = true - bug

Is this a bug? If not, could we make it a feature request? ... see also link to fiddle at bottom. Thanks in advance, btw the plugin rocks!
Steps to recreate: (already setup in fiddle)
When centerMode = true, slidesToScroll is only sliding 1 regardless of what is set.
Expected: if centerMode = true and slidesToScroll = 3, nav items should scroll 3 slides on center.
FIDDLE HERE...
http://jsfiddle.net/befreestudios/hk5uqb5w/

I think #541 is related.

Closing as a dupe of 659


SecurityContext.getAccessToken()  returning Null

Hi ,
We are trying to upgrade to the new security Library and followed the steps from the below Migration guide
https://github.com/SAP/cloud-security-xsuaa-integration/blob/master/java-security/Migration_SAPJavaBuildpackProjects.md
I have attached Pom.xml and web.xml of my project ..
We are trying to access the application using Client credentials flow . Please note that there are no scopes in the client credential JWT .
we have added the below dependency in the web.xml
<security-constraint>
    <web-resource-collection>
        <web-resource-name>All SAP Cloud Platform users</web-resource-name>
        <url-pattern>/*</url-pattern>
    </web-resource-collection>
   <!--  <auth-constraint>
        <role-name>ODPManage</role-name>
        <role-name>APIManage</role-name>
        <role-name>APIFullAccess</role-name>
        <role-name>ODPAPIAccess</role-name>
    </auth-constraint> -->
</security-constraint>

if we add the  auth-constraint  with the scopes , the client credential call does not reach the application layer . On removing the
auth-constraint the call reaches our application but  we can see that SecurityContext.getAccessToken()  returns null .
I can see in the guide that it is a known issue . Is it fixed in the latest releases or can you suggest what needs to be done to get the token value .
pom.txt
web.txt

hi @KaminiSingh,
what do you exactly mean by "call does not reach application layer"?  It is not authenticated or you do not see any incoming requests at all?
If the incoming request Jwt doesn't have any roles, but you have defined them in the application security configuration, then I would expect that the request doesn't get through.
Can you please increase the logging level and try to see what exactly happens with the incoming requests, why and where it fails.
In case your application provides no scopes, consider the documentation here.
Thanks and regards,
Liga Ozolina

Hi @liga-oz
Attached are the logs when I specify the roles in application security configuration . I see the request gets Authenticated , but the request does not comes to the application .
Even if I remove the constraint I see that AccessToken is returned as null .
while trying to invoke the method com.sap.cloud.security.token.AccessToken.getTokenValue() of a null object loaded from field com.sap.gateway.core.service.authorization.TokenValidation.reqToken of an object loaded from local variable 'this'" }
We are trying to use  SAP Java Container Security API  using the buildpack  first time .
log.txt

Hi @KaminiSingh
so, I've checked your logs...
I guess you followed the guide @liga-oz has provided to you, as you asked for support for Servlet Security when you do not have any scopes?
Have you set environment variable DISABLE_LOCAL_SCOPE_AS_AUTHORITIES = true in the mean while?
Has the Nullpointer Exception disappeared  for the endpoint that supports a client credential token without scopes?
So, in order to support you, we need to understand your current configuration, the endpoints your are calling and the full log.

Please note, if you've disabled local scope checks you would have to provide (within the same cloud foundry application) the application id for the scope checks, e.g.:
@ServletSecurity(@HttpConstraint(rolesAllowed = { "xfsrt-application-sandbox!t9563.APIManage" }))

or
<role-name>xfsrt-application-sandbox!t9563.APIManage</role-name>

BUT, as the application id is different in all your Cloud Foundry landscapes you would have to take it from the system environment variable dynamically. Not sure whether and how this works? In case you have to support both use cases as part of the same app (the client-credentials token without scopes and the user token with scopes) i would not disable the local scope checks.

Best regards,
Nena

Hi @KaminiSingh
Right now the SAP Java Buildpack supports only scope checks against one single xs-appId for local scope checks. That means in case of multiple xsuaa bindings, xsuaa(application) is preferred over xsuaa(broker). That means the "xsappname" from xsuaa(application)  is used for local scope checks.
Limitations:

There is no option to select another xsuaa instance as primary one as part of the SAP Java Buildpack
Another idea would be whether wildcards can be used as part of the role check? This might be an idea for SAP Java Buildpack enhancement.
Not sure whether you could read the respective "xsappname" from the complex VCAP_SERVICES environment variable (see "Stackoverflow: Referencing Environment Variables in web.xml"), so that you can implement it properly with global scope checks.

Closing this, as we follow up as part of this ticket (SAP internal).


Can't Login

I can't login

Maybe due to the supabase free tier email limit :(
I can give you a hard-coded account for using


SARAALERT-826: (Bugfix) Updating the UI eligibility icon logic to check for closed records.

Description
Jira Ticket: SARAALERT-826
How to Replicate

Click on a record that is eligible for notifications.
Move the record to the closed line list by changing the Monitoring Status to "Not Monitoring."
Notice that the eligibility icon has not changed.

Solution
Added a check to the logic that supports that eligibility icon to check the monitoring status.
Important Changes
app/models/patient.rb

Added check to see if the record is actively being monitored.

Testing
Replicate the above steps and notice the icon now updates appropriately.

Looking at this, I think there's a fair number of questions for what message(s) should show in a bunch of situations (ex: if a dependent has notifications paused, the icon will only show one message and it will be about them being a dependent), as well as if we can show multiple messages/reasons in certain situations. Update this ticket to include also going through and verifying the logic as it's out of scope of this one I think: https://tracker.codev.mitre.org/browse/SARAALERT-732


bib: resolve container content from the external dnf again

This commit goes back to the approach we had in PR#565 to resolve using
the solver from outside of the container. The reason to back is that
some containers (like f41) do not have the python dnf available so
they break with the "run dnfjson inside" approach (that was meant
as a quick fix only anyway).
Huge kudos to Achilleas for most of the work here.
Closes: https://github.com/osbuild/bootc-image-builder/issues/700
It would be ideal if this could be tested end-to-end on a rhel9 system building a rhel9 iso installer.

Smoke test failed with "Bad gateway" -_-
Restarted

LGTM. The nested test is a bit out there but I like it!

Yeah, not super happy about it, ideas (very) welcome how to make it cleaner!

Linter is unhappy though.

Fixed now. There was also another issue that the new unit tests in testingfarm uncovered so yay for tests.


Textcomplete doesn't work in AngularJS UI Bootstrap accordion...

Hi, I really like your textcomplete plugin as I need this for multi-autocomplete suggestions in my textarea element. However, I require this plugin to be inside my AngularJS app's Accordion field which is part of the UI Bootstrap library. For some reason, textcomplete doesn't show the matches when inside the
 element, however it works outside of it.
Here is a link to Bootstrap UI's accordion control: https://angular-ui.github.io/bootstrap/#/accordion
Can you please help me with this?
Thanks.

Yes, Me to. Also not work with ReactJS.
Thanks.


Only just saw this message.
Good suggestion!
I'll add it when i can get around to it otherwise i'm happy for someone else to do a PR with the changes


Repeated introduction & removal of 'index="0"' in .tscn files causes version control clutter

Godot version:
3.1
OS/device including version:
macOS 10.14.4
Issue description:
When scenes are saved, Godot keeps introducing and removing an index="0" attribute to/from the .tscn files. Not a critical bug, but a constant nuisance for version control. The issue seems to be related to derived scenes (scenes that use an instance of another scene as their root, which is something I use often and works really well except for this nuisance).
See the following lines from version control diff:
-[node name="SomeNode" instance=ExtResource( 1 )]
+[node name="SomeNode" index="0" instance=ExtResource( 1 )]

later for the same file:
-[node name="SomeNode" index="0" instance=ExtResource( 1 )]
+[node name="SomeNode" instance=ExtResource( 1 )]

Steps to reproduce:

Create a scene and save it (any root node will do)
Create another scene that has an instance of the first scene as its root node
Make any change to the derived scene and save all; observe that the derived scene's .tscn file has an index="0" attribute now.
Make any change to the first scene and save all; observe that the derived scene's .tscn file does not have an index="0" any longer.
Repeat steps 3 & 4.

In Steps 3 & 4, make sure you either use "Save All Scenes" (via menu or keyboard) or simply Run the project each time to ensure all files are saved in one step (i.e. don't simply hit Cmd/Ctrl+S to save each scene one by one).
Minimal reproduction project:
IndexIssue.zip
This project already has Step 1-2 from above done. Simply open "DerivedScene.tscn" in a text editor and repeat Steps 3 and 4 from above to observe the issue.
Related issues:
I thought some work was done to remove resource indices from the .tscn files in v3.1, which would have solved this issue as a side affect, but I can't find that post/entry and Godot 3.1 still has this issue.

Found a related issue, #17562, that's marked as fixed, which seems to cover a wider range of causes. The remaining issue that I describe in this issue entry is for inherited scenes only, which is not fixed by #17562 apparently.

Aha, there it is! So I wasn't crazy after all. Good job narrowing that down.

OMG, I found the cause XDDD
https://github.com/godotengine/godot/blob/36164168b5525bae3aead5d00063c6a7770bafd7/scene/resources/packed_scene.cpp#L397-L403
get_index() will normally return 0, unless the node is out of SceneTree. And it is out of the tree in the editor if you are on another tab. Heh.


chore: fix build:watch scripts

Motivation
Currently build:watch scripts (added in https://github.com/ChainSafe/lodestar/pull/1142) are broken as they reference outdated commands removed in https://github.com/ChainSafe/lodestar/pull/2933.
PR to drop lerna (https://github.com/ChainSafe/lodestar/pull/4258) suggests to remove those scripts but I find them quite handy for faster feedback cycles and generally observing build errors. Running yarn build:ifchanged works but is not ideal in my opinion.
Description

removes build:types:watch as build:types no longer exists
rename build:lib:watch to build:watch and update reference to correct build command
add missing build:watch script to packages

Running this from root directory causes MaxListenersExceededWarning warnings but might not be worth further investigate how to prevent this as it does not impact functionality
Max listeners warning is fixed by using lerna exec instead of lerna run (see stackoverflow).

:tada: This PR is included in v1.9.0 :tada:


Update Mk2Rebalance.netkan

This now has significant support for Mk2Expansion.

@CensoredUsername would you also like to add “suggests” or “recommends” mk2 expansion?

It is perfectly usable and intended to work with stock, just contains some code to also support Mk2Expansion due to popular request. ecommends definitely isn't needed, and from what I read about the meaning of the different descriptions, suggests would also not be correct


Exported variable resets to default after collapsing the structure

Tested versions
4.3 Stable
System information
Godot v4.3.stable - Windows 10.0.19045 - Vulkan (Forward+) - dedicated NVIDIA GeForce GTX 1070 (NVIDIA; <IP_ADDRESS>70) - Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz (8 Threads)
Issue description
One of two exported variables in a scene will not save anything when changed in the editor.
Steps to reproduce
I have a scene with two exported variables. One in an Array and I can add things to it just fine. The other, I've tried several different things, but it always resets when I collapse it. For instance if I have it as @export var cats = Array[Dictionary]. In the editor, I add a Dictionary element to the 'cats' exported variable. I give the key a String value 'test'. I then add an Array as the value of the test key in the cats array. I add a couple int values into that "test" array. Now I should have the structure below
cats [
    {"test":[0,1,2,3]}
]

I hit ctrl+s to save. I click the button for the 'cats' exported variable to collapse the structure.
Click the 'cats' exported variable button again to look at the structure, and it's all gone. Nothing saved. It's not in the scene file.
I've tried changing variable names, I've tried reloading the project, etc. It always disappaers.
Minimal reproduction project (MRP)
test.zip

You need press the Add Key/Value Pair button to apply changes in Dictionary.

@DCTewi From a quick test, it appears you might be correct on the issue.
If this is truly the problem, this should be changed. "Add New" and "Save" are different concepts. It's not clear that this must be done to save what is shown.

Just want to add that I was confused the first time I encountered this and I have seen questions about it on Discord. The UI here is pretty unintuitive. "+ Add Key/Value Pair" feels like you are trying to add additional data to the dictionary after what you just entered but instead it is commiting the data you entered into the dictionary. I think it would be a good idea to update the button label.


Invalid sha1sum in hunter.cmake for yaml-cpp

For some reason the SHA1 hash of the release-0.5.3-hunter-p2.tar.gz package on github has changed from ee1fe3713409de48b203f6b631c1284190d95671 to da0f41b63be987b468e3a45c1d449ba18cc502d3. Please either restore the original .tar.gz or update the sha1sum in hunter.cmake

See details here:

https://github.com/ruslo/hunter/issues/1032#issuecomment-329382772

As a solution if you have archive with sources locally (in ~/.hunter/_Base/Downloads) please upload it and we will change URL without changing SHA1. Otherwise add new SHA1, but note that this will affect binary cache and we have to re-upload libraries.

Oops, where should I upload the "original" matching the SHA1? (sorry lame me)

Where should I upload the "original" matching the SHA1?

Anywhere, I will download it, check and upload as asset to https://github.com/hunter-packages/yaml-cpp/releases/tag/release-0.5.3-hunter-p2

https://filebin.ca/3b1pACirHU6P/release-0.5.3-hunter-p2.tar.gz
Thanks in advance!

Fixed: https://github.com/ruslo/hunter/releases/tag/v0.19.108


add Rupiah Token (IDRT)

Name : Rupiah Token
Symbol : IDRT
Decimals : 2
Description : Rupiah Token is a fiat stable coin to represent Indonesian Rupiah by one-to-one peg.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

[ ] Filename in the lowercase format
[x] Image format: png
[x] Image size: 256 by 256 px

Lowercase file name

Please, lowercase file name for next upload, for now, added lowercased image file name here: https://github.com/TrustWallet/tokens/commit/c495ccb65cf6dc42cfe8e9e346d7b3d0ba5332c1

Thank you, @kolya182!


[WIP][FLINK-15918]

What is the purpose of the change
(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring them on each deployment (during recovery).)
Brief change log
(for example:)

The TaskInfo is stored in the blob store on job creation time as a persistent artifact
Deployments RPC transmits only the blob storage reference
TaskManagers retrieve the TaskInfo from the blob cache

Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end deployment with large payloads (100MB)
Extended integration test for recovery after master (JobManager) failure
Added test that validates that TaskInfo is transferred only once across recoveries
Manually verified the change by running a 4 node cluser with 2 JobManagers and 4 TaskManagers, a stateful streaming program, and killing one JobManager and two TaskManagers during the execution, verifying that recovery happens correctly.

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

I've started a separate build on travis-ci.com. You can track the progress here: https://travis-ci.com/flink-ci/flink/builds/147751241

Test failure rate is acceptable.

2020-02-06 19:39:58,390 INFO  org.apache.flink.runtime.taskmanager.Task                     - Ensuring all FileSystem streams are closed for task Source: Custom Source (1/1) (e809b769c817e73fa009ec5055e883e2) [CANCELED]
2020-02-06 19:39:58,394 WARN  org.apache.flink.streaming.runtime.tasks.StreamTask           - Error while canceling task.
org.apache.flink.runtime.execution.CancelTaskException: Consumed partition PipelinedSubpartitionView(index: 0) of ResultPartition 20cd8853199f84e8a26339a17fe731e9@e809b769c817e73fa009ec5055e883e2 has been released.
	at org.apache.flink.runtime.io.network.partition.consumer.LocalInputChannel.getNextBuffer(LocalInputChannel.java:190)
	at org.apache.flink.runtime.io.network.partition.consumer.SingleInputGate.waitAndGetNextData(SingleInputGate.java:509)
	at org.apache.flink.runtime.io.network.partition.consumer.SingleInputGate.getNextBufferOrEvent(SingleInputGate.java:487)
	at org.apache.flink.runtime.io.network.partition.consumer.SingleInputGate.pollNext(SingleInputGate.java:475)
	at org.apache.flink.runtime.taskmanager.InputGateWithMetrics.pollNext(InputGateWithMetrics.java:75)
	at org.apache.flink.streaming.runtime.io.CheckpointedInputGate.pollNext(CheckpointedInputGate.java:125)
	at org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput.emitNext(StreamTaskNetworkInput.java:133)
	at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:69)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:311)
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:187)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:487)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:470)
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:707)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:532)
	at java.lang.Thread.run(Thread.java:748)
2020-02-06 19:39:58,409 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutor            - Un-registering task and sending final execution state CANCELED to JobManager for task Source: Custom Source (1/1) e809b769c817e73fa009ec5055e883e2.
2020-02-06 19:39:58,425 INFO  org.apache.flink.runtime.taskmanager.Task                     - Map -> Sink: Unnamed (1/1) (e2897fbac05b906ad242f5dd8e21fbbe) switched from CANCELING to CANCELED.
2020-02-06 19:39:58,425 INFO  org.apache.flink.runtime.taskmanager.Task                     - Freeing task resources for Map -> Sink: Unnamed (1/1) (e2897fbac05b906ad242f5dd8e21fbbe).
2020-02-06 19:39:58,426 INFO  org.apache.flink.runtime.taskmanager.Task                     - Ensuring all FileSystem streams are closed for task Map -> Sink: Unnamed (1/1) (e2897fbac05b906ad242f5dd8e21fbbe) [CANCELED]
2020-02-06 19:39:58,427 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutor            - Un-registering task and sending final execution state CANCELED to JobManager for task Map -> Sink: Unnamed (1/1) e2897fbac05b906ad242f5dd8e21fbbe.
2020-02-06 19:39:58,451 INFO  org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl  - Free slot TaskSlot(index:0, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=384.000mb (402653174 bytes), taskOffHeapMemory=0 bytes, managedMemory=512.000mb (536870920 bytes), networkMemory=128.000mb (134217730 bytes)}, allocationId: 0fd30c7ec2d4e14753fe9e4803721b73, jobId: b3cc3c987e7aa8f99f5814b05d0057fe).
2020-02-06 19:39:58,452 INFO  org.apache.flink.runtime.taskexecutor.JobLeaderService        - Remove job b3cc3c987e7aa8f99f5814b05d0057fe from job leader monitoring.
2020-02-06 19:39:58,452 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutor            - Close JobManager connection for job b3cc3c987e7aa8f99f5814b05d0057fe.
2020-02-06 19:39:58,455 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutor            - Close JobManager connection for job b3cc3c987e7aa8f99f5814b05d0057fe.
2020-02-06 19:39:58,455 INFO  org.apache.flink.runtime.taskexecutor.JobLeaderService        - Cannot reconnect to job b3cc3c987e7aa8f99f5814b05d0057fe because it is not registered.
Checking for non-empty .out files...
No non-empty .out files.

[FAIL] 'State Migration end-to-end test from 1.6' failed after 0 minutes and 17 seconds! Test exited with exit code 0 but the logs contained errors, exceptions or non-empty .out files

Shouldn't be related?

https://api.travis-ci.org/v3/job/646941244/log.txt

Looks this is not an issue. Just that the upstream task was canceled earlier and triggered a CancelTaskException of the downstream task which is still processing data on the same TM.

I was just about to post that this is not a problem. See https://issues.apache.org/jira/browse/FLINK-15751?focusedCommentId=17024366&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17024366

https://travis-ci.com/flink-ci/flink/builds/147754663 seems to pass as well.
Could we check whether we have a test ensuring that when we are in RESTARTING and calling cancel on the ExecutionGraph that we stop all tasks and eventually go into the state CANCELED?

@tillrohrmann  https://github.com/apache/flink/pull/11032/files#diff-71e4371a8ee17b6ce39a7022298ee228R627


[#31475] Filter out test stream using test from spark suite

New TestStream using test was added, and not filtered out from the spark runs, causing failures.
Part of resolving #31475

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

R: @damondouglas


Make the decoder open file from the sound directory by default

I'd suggest prepending soundPath with "sound/" when opening the file so NewSoundDuration could be used as a drop in replacement for SoundDuration (e.g. NewSoundDuration("test.wav") instead of NewSoundDuration("sound/test.wav"))

I think it's preferable to leave it as is so sound files from any directory can be accessed with NewSoundDuration, sounds downloaded to the data directory for example. If your use case doesn't require that then you're free to make the change for yourself.


Confused About Loadstone & LoadstoneNightly

I'm just wondering what the difference is between Loadstone and LoadstoneNightly?
Idk if I'm just blind or dum but I can't see any differences to these in the readme (other than LoadstoneNightly not asking for support at the bottom).

Nightly is effectively a beta. It has a couple small tweaks, as well as the ability to toggle some experimental features

Thanks! I was just wondering.


Mlperf/4.1 grain

Use a custom grain wheel (Need to rebuild dependency image, see changes in requirements.txt and setup.sh)
Add grain support for per_device_batch_size<1 while eval_per_device_batch_size>1
Run on v5p-128 using xpk (per_device_batch_size=0.5, eval_per_device_batch_size=1), using the script in this PR. command: bash MaxText/configs/v5e/mlperf-grain-tpu.sh PER_DEVICE_BATCH_SIZE=0.5 GRAIN_WORKER_COUNT=1 ICI_TENSOR=16 PLATFORM=gke
Results: https://cloudlogging.app.goo.gl/5BpmD9iCEm7uVGNx5
The padding batch for eval doesn't work well. Suggest to first test with eval off (eval_interval=-1), then try eval on. Eval may not terminate but hang on some slice sizes. I'll continue working on that.

The padding batch issue for eval should be fixed now. To use this feature, set eval_steps to the just enough to finish the eval data (the data iter will generate empty examples until eval_steps met). For mlperf eval data, set eval_steps = math.ceil(5700/global_batch_size) and check if the output total_weights==11590004. Add more eval_steps if needed for total_weights to reach that value.


Pending openshift-migration Backups, Restores, PVBs and PVRs should be removed at migration start

Is your feature request related to a problem? Please describe.
Failed migrations can leave behind Velero resources in pending state:

Backups
Restores
PodVolumeBackups (PVBs)
PodVolumeRestores (PVRs)

Sometimes, these leftover Velero resources will result in Velero and Restic controllers being "locked up" and unable to respond to new requested actions from a newer migration attempt.
Describe the solution you'd like

Remove any pending Backups, Restores, PVBs, PVRs in the openshift-migration namespace at the beginning of each migration attempt
Possibly run a DeleteBackupRequest to remove pending backup if needed
If anything was removed in (1.) or (2.), bounce the corresponding appropriate Velero or Restic controller pods to unlock them
Wait for Velero / Restic controller to re-gain readiness before continuing

Describe alternatives you've considered
A more elegant solution to the above would be to implement a "cancel" feature on all of the above mentioned Velero kinds so that we could gracefully stop pending tasks. @sseago are you aware of the work being done on this upstream?
cc @alaypatel07 @dymurray @pranavgaikwad

Picking this up today.

@djwhatle Won't backups/restores that are queued up from other running migrations also be 'Pending'? I think we'd only want to delete pending ones if they belong to Failed (or at least Completed) MigMigrations. Also, since we may eventually be sharing velero with OADP use, a non-MTC pending migration probably shouldn't be touched by MTC if it's waiting on a migration or something (rather than pending because velero is hung somehow).

@sseago

Won't backups/restores that are queued up from other running migrations also be 'Pending'?
Yes. I don't think it should be permissible to run multiple migrations at once though. This came up in the last scrum of scrums, wrote this up: https://github.com/konveyor/mig-controller/issues/733

Also, since we may eventually be sharing velero with OADP use, a non-MTC pending migration probably shouldn't be touched by MTC if it's waiting on a migration or something (rather than pending because velero is hung somehow).

This makes sense to me as something we should look out for. I can filter the CRs we delete to only ones with Mig* correlation labels attached. Will update linked PR.

@djwhatle So the "prevent 2 migmigrations" plan feels like a workaround. If the credentials problem in velero is solved, for example, then two users ought to both be able to click "migrate". The fact that velero works serially shouldn't prevent migration 2 from starting backup after migration1 is done with the backup phase. I understand why, in the short term, we may want to prevent that, but I don't think we should assume that this workaround is permanent when we write things like cleanup code. I think we still probably only want to delete velero CRs if they belong to completed/failed/canceled migmigrations. Ignore them if we can't identify the migmigration or they don't have one.

@sseago I modified the linked PR to meet the spec we discussed earlier today. Please take another look when you have a moment.


fix: BotState cache hash skipped properties

#minor
Description
This PR fixes an issue where the predefined skipped properties were removed from the bot's state cache object, instead of removing them only from the cache hash calculation because of the original issue (https://github.com/microsoft/botbuilder-js/issues/4325).
Specific Changes

Updated skipProperties method to not modify the parameter value, and instead generate a copy, so later the hash calculation can use.
Updated the unit tests for BeginSkill, showing the properties are no longer deleted from the cache.
Added skillClient to the skipped properties.

Testing
The following image shows the current error, and the bot working from the original issue.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.4%) to 84.155%

Totals

Change from base Build<PHONE_NUMBER>:
-0.4%

Covered Lines:
20326

Relevant Lines:
22872

💛  - Coveralls


Support type promotion with object patterns

I'm wondering if using an object pattern could promote the type in the same way an if statement does. For example, this code promotes the variable foo:
void main() {
  Object? foo = 123;
  if (foo is int) {
    print(foo / 10);
  } else {
    print(foo);
  }
}

But in Dart 3, a switch statement with an object pattern doesn't promote foo:
void main() {
  Object? foo = 123;
  switch(foo) {
    case int:
      print(foo / 10);
    default:
      print(foo);
  }
}

The operator '/' isn't defined for the type 'Object'.

case int matches the constant Type value for int.
What you want to use is a variable pattern:
void main_() {
  Object? foo = 123;
  switch(foo) {
    case final int foo:
      print(foo / 10);
    default:
      print(foo);
  }
}

Another case for https://github.com/dart-lang/language/issues/2911.
And as @blaugold points out, it does promote when the pattern actually checks the type.

Oops, I didn't realize I was matching the type int and not the value. This code works also:
void main() {
  Object? foo = 123;
  switch(foo) {
    case int():
      print(foo / 10);
    default:
      print(foo);
  }
}

It's a very well known foot-gun in the new switch syntax.
I personally blame type literals for the problem, not switches (https://github.com/dart-lang/language/issues/2393 :wink:).
We plan to have a lint reminding you to do something else when you write case Type: (https://dart-lang.github.io/linter/lints/type_literal_in_constant_pattern.html).


Agent doesn't automatically fetch updated tags

From @aeschli on November 17, 2015 13:12
Tags are supposed to be immutable. Situations do occur, though, in which we must update tags. The build agent never seems to fetch the updated tags. We had to manually connect to the agent and run git fetch --tags.
@joaomoreno
Copied from original issue: Microsoft/vso-agent#152

From @bryanmacfarlane on April 29, 2016 11:20
Can you log this issue in the new agent?  We'll make sure the new one handles this
https://github.com/Microsoft/vsts-agent

@aeschli  this is the git command we run on fetch in new agent:
// git fetch --tags --prune --progress origin [+refs/pull/:refs/remote/pull/]

@aeschli can you help us confirm whether this is still an issue to you?

We are still in the old agent, just close this, since we probably got a workaround for it.


Generate queries for different SQL dialects

Previously, SQL queries were constructed using string concatenation. With this approach it was difficult to support different SQL dialects. To deal with the problem, it was decided to use Querydsl library, which helps with SQL queries generation.
Currently, an SQL dialect for query generation is determined by AbstractQuery basing on Connection metadata, so an additional configuration doesn't required.
All the supported dialects are listed in the related section of Querydsl documentation.
Other changes:

Naming of tables was changed to avoid issues with repeated table names. See DbTableNameFactory for the details.
Query factories were removed and now tables construct queries directly.
SelectQuery and WriteQuery interfaces were introduced.
Hierarchy of storage queries was changed.
A significant part for working with connections was moved to AbstractQuery, it includes closing of connections and transactions handling (from now, all queries are transactional).
Package structure was changed to hide a non-public API.
kind column was added into AggregateEventRecordTable to distinguish events and snapshots.
The issue #36 was fixed, because Querydsl constructs queries using column names instead of indexes.
The version was increased to 0.9.78-SNAPSHOT.

@armiol PTAL.

@armiol PTAL.

@armiol PTAL.

@armiol PTAL.

@armiol PTAL.

@armiol PTAL.

@dmytro-grankin LGTM.


Black formatter

Hello, @dcrosta!
I understand, it's may be too late for this PR, but I've found this library and I made a little project with it. I found this repo a little bit outdated, so I've decided to contribute.
I added black support in tox's env 'style', as you mentioned in issue https://github.com/dcrosta/flask-pymongo/issues/128
You said you want to keep your flake8 imports plugin ordering, but black
currently support the way it is already.
flake8 had only one conflict with current black: W503 rule, so I added it to ignore.
p.s. this is my first PR, so if you have something to fix up here, let me know!
p.p.s. If you don't mind, may be I can do next issue with mypy?
p.p.p.s. pymongo's current version 4.1.1, may be we can update flask_pymongo to support this version as well?

Finaly had time to fix up CI issue. Turns out it was dependencies for flask < 2.0 version. Added them to tox.ini


bower_components

Is the "bower_components" folder taken out intentionally

Yes.  As you can dobower iand download the needed front-end frameworks and libs.  This is to save space on github and in the project it self.  It also should install the bower files when you do npm i


build: rename opaqueImageView component ff

Description
Rename opaqueImageView component feature flag to be released only in latest app version (8.1.0 +)

This PR resolves []
QA Testing

Acceptance Criteria

make sure that the new image component (on staging it has a purple watermark NewImg) renders normally and no weird behavior is encountered. This covers many surfaces so exploratory testing might be suitable for this. Most of the artwork grids use the new image.

Setup Instructions

go to admin menu and enable the feature flag Enable New Image Component.

PR Checklist

[x] I tested my changes on iOS / Android.
[ ] I added screenshots or videos to illustrate my changes.
[ ] I added Tests and Stories for my changes.
[ ] I added an app state migration.
[x] I hid my changes behind a feature flag.
[x] I have prefixed changes that need to be tested during a release QA with [NEEDS EXTERNAL QA] on the changelog.

To the reviewers 👀

[ ] I would like at least one of the reviewers to run this PR on the simulator or device.

Changelog updates
Changelog updates

Cross-platform user-facing changes

iOS user-facing changes

Android user-facing changes

Dev changes

[NEEDS EXTERNAL QA] rename opaqueImageView component ff - gkartalis

Need help with something? Have a look at our docs, or get in touch with us.

will fill out the qa test cases / setup in a bit 🙈 in b2b meetings rn

@MounirDhahri @brainbicycle is the QA ACs and setup instructions that I added enough for applause?

Yes, they're enough imo


Taskbar blur doesn't auto hide when bar grows to full with

Describe the bug
Once the taskbar reaches full width and auto hide is enabled,  the blur stays visible once the bar hides.
This is true if the bar size is dynamic or set to screen width
Steps to reproduce

Enable blur behind the menu bar
Set taskbar to auto hide
Either
3a. Set taskbar to full width
3b. Open a bunch of stuff
Enjoy the blur where the task bar slides away from

Expected behavior
Blur should hide with taskbar
System information:

Windows version: 2004 OS Build (19041.508)
Cairo version: 0.3.7086.0 - Pre-release
32-bit or 64-bit: 64-bit
In Cairo Settings > Advanced, is Cairo set as shell?: No

Screenshots
If applicable, add screenshots to help explain your problem.

Oops. Thanks for the report, we'll get this fixed.

@Plloi This will be resolved in the next release; if you'd like you can install this build to get the fix early.

This is resolved in the latest release.


Verify that export account works, complete what's missing if necessary.

Does export account actually work, I don't see any code navigating to exportAccountPage.dart.
Specifically I would expect in accountManagePage.dart to see more than just an enter password dialog...

I think you are right. The exportAccountPage is obsolete, and has been replaced with the exportResultPage. (I am not happy with the naming though).
I don't get what you mean with the password dialog. We enter the password and then we navigate to the exportResultPage, what is the issue with that?

Of course, export account works!
Account detail page besides share account there's a "..." Where you can export. And it does work as expected

Tested with v1.60 that is works