How to expand ALFWORLD to new Instruction datasets?

Hi, authors! thanks for your great work!
ALFWORLD provides a way for many embodied agents to train in text-level. And in recent, I want to create new instruction dataset different from ALFRED but using the same scene setting. And I want to know if the instruction changes, can the ALFWORLD still provide the text description of the world?
If ALFWORLD can not provide the text description of world, can I have a way to achieve the similar effect like ALFWORLD?
Thanks so much if you can take time to reply!

Hello, thanks for reaching out. Which dataset do you want to adapt ALFWorld to? If you have access to a smililar set of meta-information as we have in ALFRED (for example, PDDL that describes the world dynamics like this), then in principle you should be able to adapt our code to create a "text version" of your dataset. By "adapt" I mean you may have to re-implement many things using our code as a reference.

Thanks for your reply!
Actually, the dataset is created by ourself and it includes some actions unmentioned in ALFRED like 'break' or 'dirty' but supported by AI2THOR v5.0. I want to know if these changes make the adaptation of ALFWORLD difficult to achieve.
And I also know if there is a way to only use visual environment AI2THOR provides and turn it into text description like ALFWORLD has. I think it may be a feasible plan if difficult to adapt.
Thanks so much if you can take time to reply!

Hi @shengyin1224, the text descriptions are generated from the PDDL files, not the AI2THOR's images. So, in order to use ALFWorld, you would need to update the alfred.pddl to include your new actions: dirty and break, then update the ALFWorld's alfred.twl2 grammar file accordingly. Once that's done, you will need to update the problem files (found in the downloaded data, e.g. ~/.cache/alfworld/json_2.1.1/train/look_at_obj_in_light-BasketBall-None-DeskLamp-314/trial_T20190907_142504_035718/initial_state.pddl) to 1) add the relevant predicates required by your new actions and 2) the goal for that game.

Thank you very much @MarcCote! I am also curious that if the version of AI2THOR influence the modification because ALFWORLD use version 2.1.0 but the newest version is 5.0.0.

Hello, thank you for your excellent work!
I have a question related to this issue. Since I'm not very familiar with PDDL or game development, I would like to know where to start if I only want to modify the goal condition. I want to keep the same scenes and initial state but change the goal condition.
Thank you once again!

Hi @shengyin1224 , I do believe it will change the corresponding .pddl file. @MohitShridhar would know more about this. Also, the codebase won't work with a more recent version of AI2THOR, it will just hang (at least that's what people reported to me).

ok，thanks very much!


querying returns a different table name, GSI in dynamoDB

I am using dynamoid gem 3.6.0. I have a model and have GSI on one of its fields.When I query that table to fetch records of that field, it seems to use incorrect table name.
Please help!!

Could you please provide an example to reproduce the issue?

it seems to use incorrect table name.

Could you please clarify how you did determine that the table name is wrong?

When I query to fetch some record, looking at the dynamodb in logs it looks
for incorrect table and throws non existent table error
On Mon, 23 Nov, 2020, 11:18 pm Andrew Konchin<EMAIL_ADDRESS>wrote:

Could you please provide an example to reproduce the issue?
it seems to use incorrect table name.
Could you please clarify how you did determine that the table name is
wrong?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Dynamoid/dynamoid/issues/474#issuecomment-732321205,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARUIPZKFFSQ4IKPMWJLI5D3SRKN63ANCNFSM4T7UUZ5Q
.

Could you provide some code to reproduce the issue? We need at least the model class (field and table declarations) and how to query a table to reproduce the error.

Hey, I run into the same issue, I was able to reproduce it with the code below:
require 'dynamoid'

class LoggableEvent
  include ::Dynamoid::Document

  table name: :loggable_events

  field :event_object, :serialized
end

LoggableEvent.table
 => nil
LoggableEvent.table_name
 => "dynamoid_app_development_loggableevents"
# while the expected should be "dynamoid_app_development_loggable_events"

It's quite an expected behavior as far as table method is called twice:
# first time
  table name: :loggable_events

# second time
LoggableEvent.table
# => nil

The second call overrides the table configuration and specified name loggable_events.

I guess this makes sense. That said, this might be a hint why in some edge cases there might be an incorrect table name used when querying DynamoDB.


Nil value causes crash at startup

I get this error when I launch the server :
init.lua:117: attempt to index field 'defs' (a nil value)

This test server is created for a long time (only on singleplayer), and I used moreblocks since its debut.

Did you do any changes in the original moreblocks? And do you use its the latest versions (it's very recommended)? Also, I've made a fix in the mod allowing to show all registered microblocks, not only an upper half of them.

Yes, I was used a 3-year-old version :grin: Thanks for the fix anyway :)

Yes, I was used a 3-year-old version :grin: Thanks for the fix anyway :)


Update: Charges.Clients and Charges.Clients.Registration to latest v 6.0.0

Description
This PR bumps below packages to v. 6.0.0

Energinet.DataHub.Charges.Clients
Energinet.DataHub.Charges.Clients.Registration

This PR also removes the BFF's ChargeController method GetAsync(). Which is no longer used by frontend, nor supported by backend.
References

https://github.com/Energinet-DataHub/geh-charges/issues/1799

Codecov Report

Merging #1134 (8661aad) into main (2201b2e) will increase coverage by 7.89%.
The diff coverage is 26.66%.

@@            Coverage Diff             @@
##             main    #1134      +/-   ##
==========================================
+ Coverage   68.40%   76.29%   +7.89%
==========================================
  Files          18       30      +12
  Lines         269      481     +212
  Branches       25        0      -25
==========================================
+ Hits          184      367     +183
- Misses         75       94      +19
- Partials       10       20      +10

Impacted Files
Coverage Δ

...ce/DataHub.WebApi/Controllers/ChargesController.cs
33.33% <0.00%> (ø)

...Api/Controllers/MarketParticipantControllerBase.cs
0.00% <0.00%> (ø)

.../Integration/Controllers/ChargesControllerTests.cs
100.00% <100.00%> (ø)

...stration/JwtTokenSecurityRegistrationExtensions.cs
46.66% <0.00%> (-53.34%)
:arrow_down:

...gistration/HealthEndpointRegistrationExtensions.cs
36.84% <0.00%> (-42.11%)
:arrow_down:

...WebApi/RequireNonNullablePropertiesSchemaFilter.cs
80.00% <0.00%> (-8.00%)
:arrow_down:

apps/dh/api-dh/source/DataHub.WebApi/Startup.cs
93.90% <0.00%> (ø)

...e/DataHub.WebApi.Tests/Fixtures/WebHostTestBase.cs
0.00% <0.00%> (ø)

...egration/Controllers/ChargeLinksControllerTests.cs
100.00% <0.00%> (ø)

...ration/Controllers/MeteringPointControllerTests.cs
100.00% <0.00%> (ø)

... and 10 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


feat!: Add OIDC AuthClass provider

Description
Created during hackathon
Definition of Done Checklist

Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
Please make sure all these things are done and tick the boxes

# Author
- [ ] Changes are OpenShift compatible
- [ ] CRD changes approved
- [ ] Integration tests passed (for non trivial changes)

# Reviewer
- [ ] Code contains useful comments
- [ ] (Integration-)Test cases added
- [ ] Documentation added or updated
- [ ] Changelog updated
- [ ] Cargo.toml only contains references to git tags (not specific commits or branches)

# Acceptance
- [ ] Feature Tracker has been updated
- [ ] Proper release label has been added

I did a review of the code others have written and the current state LGTM.
As I have written parts of the code as well someone else needs to review as well :)


Comment about the security section and Poseidon

In the security section of the repository, a comment is made about the Poseidon hash function based on an analysis in BBLP22, saying that it only offers 95 bits of security in Plonky2. However, the corresponding analysis in this paper discusses a slightly different case. In particular, Section 4.3 describes a root-finding approach which was relevant in the case of the original challenges, where t=3 and finding the solution to a univariate equation was thus sufficient in order to find a preimage. However, the Poseidon instance in Plonky2 uses a 64-bit prime field, and in a univariate approach this technique would allow to find a 64-bit part of a preimage with around 95 bits of work. In other words, even exhaustive search would be more efficient.
To summarize, the approach discussed in BBLP22, while correct, does not apply to the Poseidon instance used in Plonky2 (and does not break any full-round Poseidon instance in general). As far as we know, the Poseidon instance used in Plonky2 retains the full 128 bits of the claimed security level.

Sorry I missed this earlier, thanks for clarifying this for us! I was wondering why the 95 bit figure didn't seem consistent with the scripts from the Poseidon authors. (If those estimates had been broken, I imagine that would have been made very clear and the scripts might be updated.) Good to know.


wenetruntime with paraformer gives a TypeError

Describe the bug
I have successfully trained a Paraformer model, exported it into the JIT format and I want to use wenetruntime library to test decoding.
To Reproduce
Steps to reproduce the behavior:
My code:
import sys
import torch
import wenetruntime as wenet

wav_file = '/home/yehor/ext-ml-disk/asr/test_file.wav'
decoder = wenet.Decoder(model_dir='models/paraformer_model')
ans = decoder.decode_wav(wav_file)
print(ans)

Expected behavior
A transcription.
Error
Traceback (most recent call last):
  File "recognize_wenet.py", line 8, in <module>
    ans = decoder.decode_wav(wav_file)
  File "/home/yehor/Tools/anaconda3/envs/my/lib/python3.8/site-packages/wenetruntime/decoder.py", line 114, in decode_wav
    return self.decode(wav, True)
  File "/home/yehor/Tools/anaconda3/envs/my/lib/python3.8/site-packages/wenetruntime/decoder.py", line 96, in decode
    _wenet.wenet_decode(self.d, pcm, len(pcm), finish)
RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript, serialized code (most recent call last):
  File "code/__torch__/wenet/paraformer/paraformer.py", line 131, in forward_attention_decoder
    r_hyps2 = torch.cat([_24, r_hyps1], 1)
    decoder = self.decoder
    _25 = (decoder).forward(encoder_out0, encoder_mask, hyps, hyps_lens, r_hyps2, reverse_weight, )
           ~~~~~~~~~~~~~~~~ <--- HERE
    decoder_out, r_decoder_out, _26, = _25
    decoder_out0 = _18(decoder_out, -1, 3, None, )
  File "code/__torch__/wenet/transformer/decoder.py", line 38, in forward
    _40 = getattr(decoders, "4")
    _5 = getattr(decoders, "5")
    _6 = (_00).forward(x, tgt_mask1, memory, memory_mask, None, )
          ~~~~~~~~~~~~ <--- HERE
    x0, tgt_mask2, memory0, memory_mask0, = _6
    _7 = (_10).forward(x0, tgt_mask2, memory0, memory_mask0, None, )
  File "code/__torch__/wenet/transformer/decoder_layer.py", line 25, in forward
    if normalize_before:
      norm1 = self.norm1
      tgt0 = (norm1).forward(tgt, )
              ~~~~~~~~~~~~~~ <--- HERE
    else:
      tgt0 = tgt
  File "code/__torch__/torch/nn/modules/normalization.py", line 16, in forward
    weight = self.weight
    bias = self.bias
    _1 = _0(input, [256], weight, bias, 1.0000000000000001e-05, )
         ~~ <--- HERE
    return _1
  File "code/__torch__/torch/nn/functional.py", line 86, in layer_norm
    bias: Optional[Tensor]=None,
    eps: float=1.0000000000000001e-05) -> Tensor:
  _8 = torch.layer_norm(input, normalized_shape, weight, bias, eps)
       ~~~~~~~~~~~~~~~~ <--- HERE
  return _8
def silu(input: Tensor,

Traceback of TorchScript, original code (most recent call last):
  File "/home/yehor/ext-ml-disk/github/wenet-2.2.1/wenet/transformer/asr_model.py", line 951, in forward_attention_decoder
        #   >>>         [sos, 2, eos, eos]])

        decoder_out, r_decoder_out, _ = self.decoder(
                                        ~~~~~~~~~~~~ <--- HERE
            encoder_out, encoder_mask, hyps, hyps_lens, r_hyps,
            reverse_weight)  # (num_hyps, max_hyps_len, vocab_size)
  File "/home/yehor/ext-ml-disk/github/wenet-2.2.1/wenet/transformer/decoder.py", line 136, in forward
        x, _ = self.embed(tgt)
        for layer in self.decoders:
            x, tgt_mask, memory, memory_mask = layer(x, tgt_mask, memory,
                                               ~~~~~ <--- HERE
                                                     memory_mask)
        if self.normalize_before:
  File "/home/yehor/ext-ml-disk/github/wenet-2.2.1/wenet/transformer/decoder_layer.py", line 92, in forward
        residual = tgt
        if self.normalize_before:
            tgt = self.norm1(tgt)
                  ~~~~~~~~~~ <--- HERE

        if cache is None:
  File "/home/yehor/Tools/anaconda3/envs/my/lib/python3.8/site-packages/torch/nn/modules/normalization.py", line 190, in forward
    def forward(self, input: Tensor) -> Tensor:
        return F.layer_norm(
               ~~~~~~~~~~~~ <--- HERE
            input, self.normalized_shape, self.weight, self.bias, self.eps)
  File "/home/yehor/Tools/anaconda3/envs/my/lib/python3.8/site-packages/torch/nn/functional.py", line 2515, in layer_norm
            layer_norm, (input, weight, bias), input, normalized_shape, weight=weight, bias=bias, eps=eps
        )
    return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)
           ~~~~~~~~~~~~~~~~ <--- HERE
RuntimeError: Given normalized_shape=[256], expected input with shape [*, 256], but got input of size[10, 23]

Exception ignored in: <function Decoder.__del__ at 0x7f6aaa715c10>
Traceback (most recent call last):
  File "/home/yehor/Tools/anaconda3/envs/my/lib/python3.8/site-packages/wenetruntime/decoder.py", line 56, in __del__
TypeError: 'NoneType' object is not callable

Versions:

Python 3.8
wenetruntime (latest)
torch 1.13.0+cu117

same question

hello, any update on it?

We haven't  tested it paraformer in runtime yet. I'm afraid we don't have bandwidth to fix the problem recently.


FIx noops

Splitting noop logic out of https://github.com/microsoft/FluidFramework/pull/11262/files
Through code review, I've noticed that we send content="null" for non-immediate noops, but service expects content === null to differentiate immediate vs. non-immediate noops. In other words, non-immediate noops are considered to be immediate noops by service and they are never coalesced! I've noticed it only because I worked (not part of this PR) to move content strigification out of loader layer into runtime.
While we can fix it on service side (and leave client as is), I really do not like current design and special types based on type of content. So splitting noops into "accept" (formerly - immediate noops) and noops (non-immediate). Once this change propagates through the system, service can

coalesce ALL noops.
service may choose to do some special treatment of accept ops, though I do not see any reason for that. For example, today service will drop it if such op does not change MSN. Given they are used only for acceptance of code proposals (which are very rare), I'd suggest simplifying service logic and not have any handling for it.

Closing in favor of PR #11790 - for some reason this PR does not "see" latest commits in the branch (it's same branch!)


[ALLUXIO-2509] Improve comment in FileLocationOptions

https://alluxio.atlassian.net/browse/ALLUXIO-2509

Can one of the admins verify this patch?

ok to test

Jenkins, add to whitelist

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/13080/
Test PASSed.

LGTM. Thanks for your contribution.


Ensure that :relation_id renders errors for belongs_to :relation

These kinds of errors might not be surfacing correctly.

Fixed in https://github.com/comfy/comfy-bootstrap-form/commit/4ebc21f0e7e3071c4bb81f61d9d0a4ec6c723383


Wrong link to manuscript in README.md

Link in readme has to be paper/paper.md if I'm not mistaken, find it in this line https://github.com/BRAIN-TO/GIRFReco.jl/blob/9f5b4b98625a5aea1e87d77ca5505ed42ea13c10/README.md?plain=1#L13C62-L13C62

Hi @felixhorger, thanks for pointing out this issue. I have fixed it in PR4.


To Send Command to Device using Https

I am new to NodeJS, In the documentation an available option to send command is available,
'cmd':'other_command'
this.send_comand = function(cmd,data){ var msg = [this.device.uid,cmd,data]; this.device.send(this.format_data(msg)); }
But how to trigger this from http request to a specific connected device? Please do help. Thanks in advance.

var http = require('http');
http.createServer(function(req, res){
	server.send_to('device_id', 'command');
	res.end();
}).listen(3000);

how you set var server? for "server".send_to('device_id', 'command')

var server is already created when we use the library


How to test demo with IP camera (RTSP)

Hi Team,
Thanks for sharing...
Please help me in below 2 points.

As per guidelines I can able to run demo from local machine, I placed some breakpoints but while running demo break points are not hit - directly executed. how can I run this demo with breakpoints to execute step by step in IDE: PyCham.

Demo application is working with Web Camera, how can I test with IP Camera in this demo. where I need to made required changes.

Thanks
Ramachandra Reddy

Which code are you running from the repository in PyCharm? The in-browser demo does not use Python, so PyCharm mightn't be the best tool for this. You'd be best to consult the manual/software vendor for support.

As I mentioned in #44, the demo is built to make use of browser APIs for working with webcams. It might not be practical to try and retrofit IP camera support onto the demo, since this isn't a common browser use-case and there aren't first-party APIs from browsers available to do it.

You might want to take a look at https://github.com/aws-samples/amazon-rekognition-video-analyzer as an example of using Rekognition with an IP camera.

Hi ctd,
Please suggest best tool for this demo to run locally with breakpoint execution.
Thanks in advance.

I'd suggest using the web developer tools included with your web browser. I know the web developer tools included with Firefox and Chrome are quite good and can be used to add breakpoints, inspect execution state, etc..

I'm closing this issue as it's been inactive for several days. If you require any further assistance using the demo please post a comment to reopen it.


Removed openjdk dependency.

@juju-solutions/bigdata

LGTM +1 I have also seen this passing in LXC.


LoRA in Benchmark Settings

Do you use LoRA for the step3 RLHF benchmarks in https://github.com/microsoft/DeepSpeedExamples/blob/master/applications/DeepSpeed-Chat/training/step3_rlhf_finetuning/BenckmarkSetting.md ? Or are you finetuning all model parameters?

@xiaoxiawu-microsoft any info on this? sorry for the bump, want to give a fair comparison :)


Register object/stream for dispose in ObjectResult and FileStreamResult

Issue - #1331
Related PR: https://github.com/aspnet/HttpAbstractions/pull/196

Need to add a test to ensure that if the object implements IDisposable that it indeed gets called.

Updated.

Updated.

:shipit:

Updated.

:watch:

Updated.

:shipit:

@Eilon, addressed your comments. Merging this for now. Will do a followup PR if you have any more comments.


[logging] Centralize configuration for trace flags

All TraceFlags are now configured in src/core/lib/debug/trace_flags.yaml. The format is:
my_flag:
  default: false                   # the default value; default=false
  description: Some Description
  debug_only: false                # debug_only flags only work in debug builds; default=false
  internal: false                  # internal flags will not show up in documentation; default=false

To regenerate the trace flag source code, run tools/codegen/core/gen_trace_flags.py (requires mako). This script is also run when sanity checking.
This PR also adds two new features:
Glob-based flag configuration
Trace flag configuration now supports ? (single wildcard character) and * (one or more wildcard characters).  For example, using GRPC_TRACE='event_engine*' will enable all flags that match that glob. It expands to:

event_engine
event_engine_client_channel_resolver
event_engine_dns
event_engine_endpoint
event_engine_endpoint_data
event_engine_poller

A cleaner trace-logging macro in abseil logging format
If your goal is only to add log statements when the fault_injection_filter trace flag is enabled, you can use the macro:
GRPC_TRACE_LOG(fault_injection, INFO) << "Filtered:" << 42;

When the trace flag is enabled, the the log will show something like this:
I0000 00:00:1715733657.430042      16 file.cc:174] Filtered:42

Note: just like with the gpr_log to abseil logging conversion, the pre-existing trace logging usages can be replaced with the new tracing macro across multiple PRs.

@drfloob : Please can you let me know approximately when this PR will be submitted. I will avoid editing the files in the PR for some time to avoid more merge conflicts for both of us.
In the meantime I can finish all other files that need gpr to absl migration.

@drfloob : Please can you let me know approximately when this PR will be submitted. I will avoid editing the files in the PR for some time to avoid more merge conflicts for both of us. In the meantime I can finish all other files that need gpr to absl migration.

@tanvi-jagtap Thanks. I'll try to wrap it up this week, I have a bit of codegen work to do.


add sharks::Share + Cargo.lock update

Added sharks library to enclave. I fixed the error by running
cargo update -p sgx_tstd --precise ed9e7cce4fd40efd7a256d5c4be1c5f00778a5bb within the enclave workspace :)
You don't need to merge if you want to do it yourself, but since I did it to check anyway, I created this PR

I did it in in the branch for issue #8. So no need to merge it. Thanks.

This was a PR to branch 8/keyvault_store_load, not develop. But since you did already, I'm closing this PR


Quality requirements

Maybe we can go in-depth in some quality requirements already?

TODO

[ ] Mindmap


ARROW-6130: [Release] Use 0.15.0 as the next release

See discussion on the mailing list:
[Discuss] Do a 0.15.0 release before 1.0.0?
<EMAIL_ADDRESS>

@ursabot crossbow package -g linux

These commits except 66362f8 are generated by the following command line:
PREPARE_DEFAULT=0 PREPARE_VERSION_POST_TAG=1 PREPARE_DEB_PACKAGE_NAMES=1 dev/release/00-prepare.sh 0.14.1 0.15.0

Ah, we need to update versions in java/**/pom.xml too.

Codecov Report

Merging #5007 into master will increase coverage by 0.43%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #5007      +/-   ##
==========================================
+ Coverage   87.57%   88.01%   +0.43%
==========================================
  Files        1005      915      -90
  Lines      143457   134627    -8830
  Branches     1418     1418
==========================================
- Hits       125630   118487    -7143
+ Misses      17465    16130    -1335
+ Partials      362       10     -352

Impacted Files
Coverage Δ

cpp/src/arrow/csv/column-builder.cc
95.29% <0%> (-1.77%)
:arrow_down:

cpp/src/plasma/thirdparty/ae/ae.c
70.75% <0%> (-0.95%)
:arrow_down:

rust/parquet/src/encodings/encoding.rs
92.4% <0%> (-0.18%)
:arrow_down:

go/arrow/math/uint64_amd64.go

go/arrow/memory/memory_avx2_amd64.go

go/arrow/ipc/file_reader.go

go/arrow/array/builder.go

go/arrow/type_traits_boolean.go

go/arrow/math/int64_avx2_amd64.go

go/arrow/math/float64_avx2_amd64.go

... and 86 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6f98807...6833dd7. Read the comment docs.

Yes: https://github.com/apache/arrow/pull/5007#issuecomment-518039650

Can this be merged?

Yes.


encodeSpecialChars ignore attribute feature

I ran into a problem recently with encodeSpecialChars, attributes and ExactTarget. I have a Spanish language email that needs to be encoded, so encodeSpecialChars is a big help. Unfortunately for me, encodeSpecialChars encodes the entirety of special characters. This behavior is problematic because it includes ampersands in href attribute values. I didn't think it would be a problem until I ran a few emails that contained multiple &s and anchor links referencing a section within a page. When this email is sent, ExactTarget replaces each url with it's url forwarding tracking magic and some parameters get lost in the sauce. This does not happen when ampersands are not encoded in the href attribute.
I'm thinking that it might make sense to include an ignore attributes or ignore specific attributes option for encodeSpecialChars, because I don't necessarily need to not encode all ampersands. I know this is partly an ExactTarget problem and might be a pretty extreme edge case, but it has become a pretty consistent issue for me. I'm probably going to fork the repo and try to fix it soon. It just depends on how many Spanish emails I have to code. Just wanted to make you aware.

Underneath we use https://github.com/hughsk/special-html. I think the best thing would be to convert the & back to their decoded form using something like grunt-replace and using the pattern:
{
  match: /&/g,
  replacement: '&'
}


test: fix check split bug in pd

cherry-pick #7841 to release-3.1

Signed-off-by: Liqi Geng<EMAIL_ADDRESS>What problem does this PR solve?
Problem Summary:
Fix check split bug in pd.
What is changed and how it works?
What's Changed:
It's possible that a left region has an old epoch so I remove the assert in check_split.

r1  => [0, 10)  r2 => [10, 20)
r1 merge to r2
r2 => [0, 20)
r2 split
r3 => [0, 10) r2 => [10, 20)
When r1 reports to PD again, the check_split will panic.

Move test_node_merge_write_data_to_source_region_after_merging test to failpoint test.
This is a mistake.
Related changes

Need to cherry-pick to the release branch

Check List
Tests

No code

Side effects
No.
Release note

No release note

/run-all-tests

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

3.1 is not maintained.


Can't undo conversion to CPUParticles

Godot version:
3.1.1
Issue description:
Particles and Particles2D have an option to convert to CPU particles. However, you can't undo this action and the only way to go back is to revert scene.
Hint for junior contributors:
#28349 introduced changes to editor's replace_node() that make this easy to fix. Just see how _convert_to_mesh_2d_node() got undo support in that PR and do similar thing for particles.

I would like to work on this.
It seems like I have 4 options where to implement this:

As a helper function in cpu_particles_2d_editor_plugin.cpp
As a helper function in particles_2d_editor_plugin.cpp
As a member function in particles_2d.cpp
As a member function in cpu_particles_2d.cpp

Which one is most suitable?

You don't need to create any new function. Just add UndoRedo calls to the existing one.


Add ability to save incidents that do NOT have discrete locations

What issue are you seeing or what feature needs to be added?
Examples:

E. 61st St. Street between S. Greenwood Ave. to S. Woodlawn Ave.
1100 E. 57th St. to 1526 E. 55th St.
5400 South Shore to 5700 S. Maryland Ave.

This is such a small subset of locations, roughly 0.4% of the total. It isn't worth the effort required to address it.


Fix SIGSEGV in tbuffered_server

discussion here: https://gitter.im/jaegertracing/Lobby?at=5a31aea1c65707ba2b4bfe1d

formatting issue (run make fmt), and please amend the commit to be signed (see CONTRIBUTING.md)

Coverage remained the same at 100.0% when pulling 9dbf2c8da933c9b208bb5ca31298cc1752abb2e7 on manannayak:master into 6cb46f5aa7ce39160b9df8d5efa14f7a00c5de41 on jaegertracing:master.

thanks!


Fix the network metrics for clusters with containerd

How to categorize this PR?

/area monitoring
/kind bug
What this PR does / why we need it:
Fix the network metrics for clusters with containerd
The network metrics
# HELP container_network_receive_bytes_total Cumulative count of bytes received
# HELP container_network_transmit_bytes_total Cumulative count of bytes transmitted

are exposed by the /metrics/cadvisor endpoint of the kubelet. They can be accessed via:
kubectl proxy
curl -s http://localhost:8001/api/v1/nodes/<node>/proxy/metrics/cadvisor

It turns out that the exposed metrics differ depending on the underlying container runtime: docker or containerd.
The prometheus scrape job "cadvisor" uses metric relabeling rules to keep only the "relevant" metrics; the cadvisor that is embedded in the kubelet exposes many metrics, most of which were considered to be not so valuable some time ago.
The network metrics are not container specific but rather specific to the pod level: all the containers of a pod share the same network namespace (by definition) and hence the network metrics show the network traffic together for all the containers of a pod. Consequently, the container label is not so meaningful for the network metrics.
It turns out that the cadvisor that is compiled into the kubelet sets a different value for the container label depending on the underlying container runtime: it is the dummy value POD for docker and an empty string for containerd.
Unfortunately the existing metric relabeling rules depended on the value of the container label and in the end they resulted in dropping the network metrics for containerd based clusters.
This PR fixes this issue: the network metrics are properly kept in Prometheus also for containerd based clusters.
This PR also includes tightly related improvements for docker based clusters: previously, all the network metrics were kept for all the pods. Unfortunately the network metrics are not meaningful for pods that use the host network, because they contain the host network's network metrics which is not specific to the pod and also have a significant cardinality: they contain the bridge network devices for each regular pod. Now these metrics are not kept in Prometheus any more. In larger clusters with many pods, they contributed significantly to the overall cardinality of the data that is stored in Prometheus.
Furthermore this PR fixes the node level network metrics (query) as well: it did not show any data for containerd based clusters and showed bogus data for docker based clusters.
Which issue(s) this PR fixes:
Fixes #2800
Special notes for your reviewer:
Please check the commit messages for more details.
/cc @rickardsjp @wyb1
Release note:

Fix the network metrics for clusters with containerd.
The "Kubernetes Pods" dashboard's "Network I/O" panel showed no data for clusters with containerd. Now it correctly shows the network metrics (sent and received bytes/s) for pods that are not in the host network namespace, also for clusters with containerd. For pods in the host network namespace no network metrics are shown because by definition the host network namespace's network stats include all the pods and system services and hence are not meaningful in the context of a specific pod. This explanation is as also included on the dashboard to avoid confusion due to missing data.
The "Node Details" dashboard's "Network I/O Pressure" panel showed incorrect readings for clusters with docker and no data for clusters with containerd. Both aspects are fixed.

Tide is unable to merge this PR because of the changed GitHub Actions in the base branch (ref https://github.com/gardener/ci-infra/pull/383).
Merging manually because it would need to be rebased otherwise.
Good thing, that we are getting rid of the GitHub Actions now.


Add Rapid Transit Database and Equity Dashboards

These seem to be good resources. I think these are open(ish) data licenses or require citation, but I can't find details on that. I will merge for now.


Object is black on some Android devices

Description
The relevant THREE github issue on this case: https://github.com/mrdoob/three.js/issues/14137. It was opened more than year ago and there is still no solution. Model-view player based on the same THREE.js library, but it does not work on a lot of popular devices. For example, many Xiaomi phones (even flagman ones) can't work with it. Tested on Redmi 4A (Adreno 308), Mi A1 (Adreno 506), Mi6 (Adreno 540). More devices with black textures issue can be found by the link above.
Errors in the console:

WEBGL_depth-texture extension not supported
OES_texture_float_linear extension not supported
EXT_shader_texture_lod extension not supported.

Screenshot of our own test , but can also be produced on the main page with astronaut.

Browser Affected

[x] Chrome
[x] Firefox

I guess all available for Android platform browsers are affected.
OS

[x] Android

Additional information

The problem is in MeshStandartMaterial (https://threejs.org/docs/#api/en/materials/MeshStandardMaterial). We use difuse, normal, metallic and roughness maps with this shader and get black texture problem in our own project with THREE.js too. I think the astronaut in model-viewer inherits the same issue.
From the documenation about MeshStandartMaterial we can get that it's about "physically based rendering (PBR) has recently become the standard in many 3D applications, such as Unity, Unreal and 3D Studio Max.". Ok, I'm Unity developer, so let's do the test.

Open Unity 2019.1
Create project with LightWeightPipeline
Choose shader for material object similiar to THREE's MeshStandartMaterial. I'll attach the screenshots from Unity and THREE editor. Shaders parametres are very similiar.
Make a build
That's it, rendering is fine!

So, I supppose the phones are OK with PBR and can handle diffuse/normal/metallic/rougness shader. THREE's developers think that there is nothing they can do.
I hope google engineers can handle this issue together with THREE developers, because Web AR really depends on it.

@AytoMaximo thanks for reporting. I only glanced at the upstream Three.js issue, but it sounds like they decided that this is a WebGL / vendor-specific issue?
Are you seeing this problem on all devices, or on a specific Android device?

@cdata I've tested on Xiaomi devices, all of them have this issue.

Redmi 4A (Adreno 308)
Mi A1 (Adreno 506)
Mi6 (Adreno 540).

Also, you can get a big list of devices from the https://github.com/mrdoob/three.js/issues/14137 thread:

LG Aristo (LG-M210) (by me)
LG Fiesta (LG L63BL) (by me)
Motorolla Moto G3
Samsung Galaxy J3 Mission (SM-J327VPP)
Huaweï MediaPad T3 10 (AGS-L09) - Androïd 7 / EMUI 5.1 / Snapdragon 425 / 2Go RAM / 800x1280
Samsung SM-T555 (Galaxy A)
Acer Predator 8 (GT-810 rolex), Android 7.1
LG K20 Plus (lv517), Android 7.0
Galaxy Tab A 9.7 (gt510wifi), Android 7.1
Moto G4 Play (harpia), Android 7.1
Galaxy Tab E 9.6 (gtelwifiue), Android 7.1
Galaxy J5(2016) (j5xnlte), Android 7.1
Galaxy Tab A 9.7 (gt510wifi), Android 7.1
HM NOTE 1LTETD (dior), Android 4.4](#12679 (comment))

They can't decide that it is a WebGL issue 'cas from the first post we see the following:
"This doesn't appear to be a problem with all WebGL texturing, though -- https://fltr.world/ (simple texture demo implemented with three.js shaders) works fine, as does https://webglsamples.org/aquarium/aquarium.html"(c).
The first link is unreachable, however the aquarium is really fine.
Also, that's why it is not vendor-specific issue, because aquarium is fine, and my example with Unity is fine.
I also posted the same issue on Chromium bug tracker, they asked me produce a minimal test case that shows that there is a problem with handling of a valid shader on this phone. And suggested these links: https://github.com/KhronosGroup/WebGL , in particular the tests under https://github.com/KhronosGroup/WebGL/tree/master/sdk/tests/conformance/glsl/misc , can be used as a template.
But I'm not shader developer and don't understand what is going on there :(
It seems like guys really need some help, because they struggle this issue for a year or longer with no results. Moreover, I think the list of devices is very big and it's not over yet, so developing Google AR view with THREE.js library inherits a serious bug.

I was able to fix rendering on my "Moto G4" by passing precision: 'lowp' here: https://github.com/GoogleWebComponents/model-viewer/blob/6eb788f5417ca34155c282199cb92f63cae3507a/src/three-components/Renderer.ts#L75
This is suggested as an workaround in this thread:
https://github.com/mrdoob/three.js/issues/14137
Of course hardcoding this value doesn't seem like a good solution to the problem.

I'm not sure if my previous post adds any value to the discussion as @AytoMaximo is aware of this workaround as an active participant of the pointed (in my comment) threejs issue thread. @AytoMaximo should I delete the above workaround suggestion?

Sorry for the late answer, had plenty of work to do :)
Unfortunately, now it's black everywhere. On my Windows PC (Chrome) and iOS device too. The same test link: https://image.faradise.ru/arcore.html
Did we mess up something after the update? We turned on automatic updates for this mini-project.

@AytoMaximo We have a similar report in #736
@elalish I'm tentatively creating a v0.6.1 milestone in case we discover a rendering regression

@cdata I've noticed a very funny thing: my Xiaomi Redmi 4A is not showing model in the scene anymore (before it was just black). The situation is the same in our test project and in model-viewer.glitch.me

@AytoMaximo Thanks for the info! I've looked at your model and I think I know why it's black: It has an entirely black AO (ambient occlusion) map:

I also noticed something strange with the shadow when I turned that on:

This leads me to believe something else is not quite right with this model - maybe triangle orientation? How was this model created?
One final question, @donmccurdy this model renders fine in your viewer, which makes me think one or the other of us has a bug. Are we somehow being more strict with our glTF parsing?

Is there a link to the model?
My viewer applies some direct light, which isn't affected by AO. If model-viewer is only using indirect lights, that (combined with a black AO map) may be why it's dark.

Ah yes, that explains it! I used the test model link three posts up; you can pull the model out of that.

@AytoMaximo Until I manage to get a local repro of your vanished model case, would you be so kind as to expand that shader error and copy the results here for me? That should give me a clear idea of what's going wrong.

@AytoMaximo Thanks for the info! I've looked at your model and I think I know why it's black: It has an entirely black AO (ambient occlusion) map:

As I know, we are using specific texture maps only for android, but our standard map set (for WebGL and iOS) is: diffuse, normal, metallic and roughness. I first faced black texture problem while developing our own THREE.js player. So, I'm sure I was using the map set mentioned above.
Here the config for our test case:
<model-viewer ar src="https://237529.selcdn.ru//awespace/product/6190/8786.glb" alt="3d model Faradise" auto-rotate camera-controls background-color="#ffffff" ios-src="https://237529.selcdn.ru/faradise/product/6190/8786.usdz" ></model-viewer>
As you can see, there is .USDZ file included for iOS, where the model is also black. Also, iOS .USDZ has metallic+roughnesss maps, not AO. That's for sure.
I'll check the information about .glb contents with the guy who developed our model converter. Personally, I was very surprised to know about the usage of AO map. I was sure, there is a united metallic/roughness map for android apps.

Also, the models for Android include united metallic/roughness map and the black AO map in the Red channel.

I think this is the problem. Why a black AO map? That means all environmental lighting is turned off (which is all we use now, for consistency of rendering).

@AytoMaximo Until I manage to get a local repro of your vanished model case, would you be so kind as to expand that shader error and copy the results here for me? That should give me a clear idea of what's going wrong.

Yes, sure :)

I think this is one of the problems. Why a black AO map? That means all environmental lighting is turned off (which is all we use now, for consistency of rendering).

I'm not android developer, so I can't properly answer this question, but  it was the only to make it work on android devices just like other platforms. We couldn't get the proper result in Android studio without this fix. Also, as I know, our android developer was also discussing this issue with Google.

We have no code to render a USDZ; all we do is pass it to QuickLook on iOS (hence the short transition into AR). If you're not in AR, then on iOS you're still looking at a glTF. If the AR one is black, then QuickLook doesn't like the file either.
I did just notice that you have a warning about not supporting texture_float_linear, which I thought we did not require. I'm going to look into that.

If you're not in AR, then on iOS you're still looking at a glTF

Oh, i see. In AR the model is fine indeed. So, seems like we need to upgrade our glb model for the model-viewer.

I did just notice that you have a warning about not supporting texture_float_linear, which I thought we did not require.

This is one of the three errors (there is a list in my first post) inherited from the THREE.js library

Okay, I've verified that all three of those WebGL extension warnings are okay; three.js looks for each of them, regardless of whether they will be used. We intentionally use none of these so that rendering works without their presence. If you decide to include an AO map without data, ensure it is white, since it is multiplicative (1 has no effect).
Regarding your shader error, normally it outputs the the line on which the shader compiler had a problem. I'm not sure why it's not doing that here. I may need to get a local repro to debug. Does that only happen on the Xiaomi Redmi 4A?

Does that only happen on the Xiaomi Redmi 4A?

It's the only android device I have access to :(
I'll ask around, but it's gonna take some time

Xiaomi Redmi Note 7 Pro (Snapdragon 660) - OK
Honor 20 (Kirin 980) - OK
Mi A1 (Adreno 506) - OK
Mi6 (Adreno 540) - OK
Redmi 3S (Adreno 505) - OK
Xiaomi Redmi 4A (Adreno 308) - FAIL
Sony Xperia Z (Qualcomm APQ8064) - FAIL
It looks like old devices have some problems. Maybe we need some other reports about Adreno 3xx or 4xx. Just to be sure when the problem starts.

Great, so it seems the 0.6.0 update did fix at least the 506 and 540 models. If you can change the AO map to white and check that fixes the rendering, that would be helpful. If the only issue left is on old phones and can be fixed by switching to mediump (as stated above, is that still true?), then I would consider making that change here as long as we don't see major moire patterns from it.

If you can change the AO map to white and check that fixes the rendering, that would be helpful.

So, we changed AO to white and the chair become colorful. However, it's too bright :)

I feel strange about the THREE.js material you are  using.
I've imported .glb model to THREE.js editor. It's also black, but! because of no directional light was added to the scene. If I add a light, everything will be fine. Changing AO color or disabling it do nothing.

Also, if I add black AO map to your astronaut, it'll be OK too:

So, the question is: what kind of THREE.js material are you using in model-viewer? For example, MeshBasicMaterial does not need light to be visible (and it's OK if you use only diffuse map). But if you have a set of diffuse, normal, metallic and roughness, you need a MeshStandardMaterial, and this guy need lights in the scene.

So, that's why we didn't expect the AO map affects anything.

Thanks for the detailed analysis! Three.js does not apply AO to direct lighting, since this is a more basic lighting type. It is (and always has been) applied to environmental lighting, but you may not have noticed the difference since the render was dominated by the point lights. We recently removed the directional light from , so all light is now environmental, which gives us a more consistent render because the two lighting types use different approximations. In fact the directional light was hiding a number of our own errors in environmental lighting which have since been corrected. In this case it was also hiding an AO error in the model.

As for brightness, please take a look at our exposure attribute, which provides the easiest way to adjust this.

So, I guess we need to find proper shade of gray color for AO map to make the model look not too bright :)
Seems like old devices the only painful thing that remains...

Yeah I wouldn't recommend using AO as a method of controlling overall brightness – if it's going to be a uniform color, you may as well omit AO entirely or use solid white. Anything else is taking you off a physically-plausible rendering path — nothing ever really gets zero indirect light — and lighting adjustments from there on out will likely be hacks.
Environmental lighting is higher quality than punctual/analytical lighting like directional lights. Changing the IBL or changing exposure should be enough to get realistic results. One could debate whether AO should affect IBL, but that's a thorny question and not the source of the problem here. :)

What about the problem with shadows? Is it on our side too?

Thanks, that's very helpful! I was just cleaning up the shadow code, so I'll be sure to check that it works properly even at 100x scale in 0.7.

Thanks, that's very helpful! I was just cleaning up the shadow code, so I'll be sure to check that it works properly even at 100x scale in 0.7.

Sounds great! We have some problems with the scale while converting models to GLB. So, if model-viewer can handle such cases, it will be very helpful))

I plan to, but keep in mind, you definitely need proper scale for AR.

I plan to, but keep in mind, you definitely need proper scale for AR.

Yes, sure)

I'm getting this "black faces error" also on pure Three JS project.
In my project, the model is partially black. I'm using FBX exported from Maya.
After checking everything from faces normals, to texture colors and UV smooth values, I decided to make a test recreating the material directly in Three JS. To my surprise, everything starts to render properly.
So, for those are facing this black faces issue in ThreeJS project, here's a workaround
FBXloader.load( 'model.fbx', function ( object ) { 	 	object.traverse( function ( child ) { 			if ( child.isMesh ) {             	var newMat = new THREE.MeshStandardMaterial({             		map: child.material.map,             		metalnessMap: child.material.metalnessMap,             		roughnessMap: child.material.roughnessMap,             		normalMap: child.material.normalMap,             		aoMap: child.material.aoMap,             	});             	child.material = newMat;             }         }); 	});
Unfortunately for the model viewer there isn't a workaround since it is some kind of a black box around the base Three JS

@Deankovitch What did you use to convert the FBX to glTF? That's what you'll want for model-viewer, and if it's generating black faces, please file a bug on that glTF exporter. Also make sure to check it with the Khronos glTF validator.


Cannot read property 'id' of null

via url : http://<IP_ADDRESS>:31337/api/user/test5/following
TypeError: Cannot read property 'id' of null
    at /mnt/data/home/mdupont/experiments/pump.io/lib/finishers.js:61:42
    at Array.map (native)
    at addFollowed (/mnt/data/home/mdupont/experiments/pump.io/lib/finishers.js:60:23)
    at /mnt/data/home/mdupont/experiments/pump.io/lib/streams.js:707:9
    at Function.<anonymous> (/mnt/data/home/mdupont/experiments/pump.io/lib/streams.js:257:17)
    at next (/mnt/data/home/mdupont/experiments/pump.io/node_modules/step/lib/step.js:51:23)
    at Function.<anonymous> (/mnt/data/home/mdupont/experiments/pump.io/node_modules/databank/lib/databankobject.js:245:21)
    at next (/mnt/data/home/mdupont/experiments/pump.io/node_modules/step/lib/step.js:51:23)
    at Function.<anonymous> (/mnt/data/home/mdupont/experiments/pump.io/node_modules/databank/lib/databankobject.js:225:21)
    at next (/mnt/data/home/mdupont/experiments/pump.io/node_modules/step/lib/step.js:51:23)

Code here:
https://github.com/e14n/pump.io/blob/1029ac71b94dec2b51f6b2aa461b5b2a514e585a/lib/finishers.js#L61
 edgeIDs = objects.map(function(object) {
        return Edge.id(profile.id, object.id);
    }

I think the problem is that I am following my account on another server but my hostname is <IP_ADDRESS> and the dialback url in the database dump shows:
memcache  dump :
dialbackrequest:https://identi.ca/api/client/register/<IP_ADDRESS>/q_PV7kyiVxI/1445339845000
dialbackrequest:https://identi.ca/api/client/register/<IP_ADDRESS>/iHVrzZQdjEI/1445337641000
dialbackrequest:https://identi.ca/api/client/register/<IP_ADDRESS>/O_xSA5eNAmA/1445337641000

The problem that I found on this problem is the people list is null :
On this line :
https://github.com/e14n/pump.io/blob/1029ac71b94dec2b51f6b2aa461b5b2a514e585a/lib/streams.js#L242
The ids are [''] a list of one empty to be read.

The user data is :
{ preferredUsername: 'test5',
  _user: true,
  url: 'http://<IP_ADDRESS>:31337/test5',
  displayName: 'test5',
  _uuid: 'Yn8ltVVFRyG-i4kZlDciqA',
  id: 'http://<IP_ADDRESS>:31337/api/user/test5/profile',
  links:
   { self: { href: 'http://<IP_ADDRESS>:31337/api/user/test5/profile' },
     'activity-inbox': { href: 'http://<IP_ADDRESS>:31337/api/user/test5/inbox' },
     'activity-outbox': { href: 'http://<IP_ADDRESS>:31337/api/user/test5/feed' } },
  objectType: 'person',
  _created: '2015-10-19T02:08:26Z',
  followers: { url: 'http://<IP_ADDRESS>:31337/api/user/test5/followers' },
  following: { url: 'http://<IP_ADDRESS>:31337/api/user/test5/following' },
  favorites: { url: 'http://<IP_ADDRESS>:31337/api/user/test5/favorites' },
  lists: { url: 'http://<IP_ADDRESS>:31337/api/user/test5/lists/person' },
  _user_confirmed: true,
  updated: '2015-10-19T02:08:26Z' }

Memcache  read  type:streamsegments id:user:test5:following
which returns this
memcached got:"user:test5:following:stream:NBzXqTz6SB28ShaKOZblJA"
The database dump contains this row:
streamsegment:user:test5:following:stream:NBzXqTz6SB28ShaKOZblJA
streamsegmentcount:user:test5:following:stream:NBzXqTz6SB28ShaKOZblJA

Then it reads :
memcached read  type:streamsegment id:user:test5:following:stream:NBzXqTz6SB28ShaKOZblJA
and gets the user id :
memcached got<EMAIL_ADDRESS>So it seems that we are just missing an internal id for this user.

In the database dump it seems that this is the culprit :
streamsegmentcount:user:test5:following:stream:NBzXqTz6SB28ShaKOZblJA
stream:user:test5:following
_databank_index:person:url:undefined
_databank_index:person:_uuid:undefined
stream:user:test5:lists:person
stream:user:test5:favorites

we can see the uuid is undefined and this is not being checked.

Nice work. If/When I have time. I'll take a look at it, should be nice to have also have a test for this.

The test is pretty simple, Setup a user on a local server with no incoming connection. Then try and follow someone on identi.ca, the communication will fail and go into a tailspin.

@h4ck3rm1k3 @profOnno was referring to a Vows spec in the automated test suite, not a manual test.

Same problem here. I have lost "my following list".
I am using the server www.mipump.es and the client Dianara. When Dianara tries to receive the following list this error appears: Error HTTP: Internal error (500) - TypeError: Cannot read property 'id' of null : https://mipump.es ... ount=200&offset=0
When I try to see my following list on the Web UI of mipump.es, I can see only the first 40 contacts: it loads 20 first, if I go down, it loads 20 more, and if I try to load more, web UI gives me an error: "0: error" an it stop loading anymore. This happens because the web ui loads the contacts in blocks of 20 contacts. So I think that that third block of 20 contacts contains one contact that is damaged and it gives the error when retrieving it.
Dianara can't retrieve any contact because it retrieves the contacts in blocks of 200.
P.S. @jankusanagi, developer of dianara, told me to comment here.
Regards,
Aqa-Ib


Added the scorecard github action and its badge

Fixes: #33009

PR to add the Scorecard GitHub Action and its badge in the README file.

Hi @harshitasao, thanks for your PR! We had a similar request a few months ago to add the OpenSSF badge, and there were some unresolved points from that discussion which meant the change didn't make it through.
I have taken a look at the current results being reported by the badge, and the specific issues we identified in the previous PR still seem to occur. As I mentioned in my response to the previous PR, at minimum I think we would like to be able to stop checks that aren't working correctly from contributing to the score. Is there any way to do this?

Converting to draft to get this off the maintainer dashboard, looks like there have been a few interactions.

Hello @ferozsalam , there is currently no way to stop checks that do not work properly from contributing to the final score. Hence, I'm closing this now and apologize for opening a duplicate issue.


Update manage-stale-devices.md

PS - Fixed select to AccountEnabled (vs Enabled which doesn't exist)
PS - Include DeviceOSType and DeviceOSVersion in export as this information is generally useful
PS - Updated time based example to pull devices that haven't logged on in X days ago vs fixed date
PS - Added NoTypeInformation flag to export as this is generally more useful

@johnmeilleur : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Looks good! Thanks for the contribution! #sign-off


rename #[async_worker] to #[ockam::worker]

This export would change
https://github.com/ockam-network/ockam/blob/develop/implementations/rust/ockam/ockam/src/lib.rs#L40

to:
... as worker;

we'll also have to find and replace all use of #[async_worker] with #[ockam::worker]
remove imports of  async_worker, #[ockam::worker] will work if ockam is a dependency.

Looks like #[async_worker] is mostly used in ockam_examples. Does that sound correct? Also used somewhat in documentation and ockam_transport_tcp. Just making sure I am on the right path

Yes that's right.

I am just thinking should the name be kept as it is or just the original name async_trait as async_worker (or proposed worker) is just an alias to async_trait, which may not be obvious to the reader of the code simply looking at that code. And as such that annotation is really only required because of it being async_trait.  It's perhaps still a good idea to bring it under ockam:: namespace explicitly.
Also, related nit:

Inside the ockam_channel, original async_trait is used, so for consistency, it should also use ockam::* whatever is decided to be the right choice of name.
Also the ockam_core trait should also use the same name for consistency! It is using async_trait.

There may be other places, where I might have missed it, so perhaps a good idea to define it consistently at one place and use the same name everywhere.

@gabhijit thank you, completely agree on consistency 👍
The reason to rename to ockam::worker is because we suspect we may be able hide more worker definition boilerplate behind it. The associated types, for example, are potential candidates cc: @spacekookie @jared-s
https://github.com/ockam-network/ockam/blob/9a6b2940e814b297376a30b04886dcf19aabd270/examples/rust/get_started/src/echoer.rs#L1-L16

@gabhijit @mrinalwadhwa In order to use ockam::Worker macro everywhere, it should be moved to ockam_core (or even separate dedicated crate), because many ockam crates need to implement Worker, but they do not depend on ockam crate

@SanjoDeundiak (Note: it's a worker and not Worker), Worker is a trait and worker is just alias to async_trait proc-macro. Yes, it can come from ockam_core::  (and can be re-exported from ockam) as long as it is consistently used everywhere.
As such for the boiler plate part, one will have to write another proc-macro ockam::worker say that internally wraps this async_trait and handles associated types. But that appears to be a bit of an overkill at the moment, since the user (implementer of the Worker trait for their own type), will have to know that those are associated types and will have to deal with them anyways.

All good points @gabhijit @SanjoDeundiak.
To elaborate on what I'm imagining ... Line 10 has all the information for line 7 and 8 and my hope is we can eliminate line 7 and 8
https://github.com/ockam-network/ockam/blob/9a6b2940e814b297376a30b04886dcf19aabd270/examples/rust/get_started/src/echoer.rs#L1-L16

For the time being I don't think it matters too much where we create the async_trait to worker alias. Although considering that we have lots of crates that create workers and might not be able to depend on the ockam crate (because of circular dependencies), it makes sense to go with @SanjoDeundiak's suggestion and move that alias to ockam_core.
Ultimately we'll need to have a separate crate, similar to how the ockam::node attribute is implemented. Or we rename that crate to ockam_attributes and put both node and worker there.


[WIP] Create catalog pods as pullIFNotPresent, use a pullalways probe pod to check for new SHAs

Description of the change:
Instead of just creating catalog pods as pull always (becomes a problem in temporarily disconnected clusters), create themn as pull if not present.
when creating a new pod to check for updates, create it as pullalways, see if it gets a new SHA, if so, delete it and recreate it as a pullifnotpresent pod but using the new SHA.
Motivation for the change:
Clusters w/ only intermittent registry connection experience pod startup failures when
catalog pods are restarted while the registry is inaccessible.  This change
reduces the period of time during which registry access is required.
The downside is that when there is an update to the catalog image for a given catalogsource, we will do 2 pod creations/startups instead of just one. (And, if the 2 pods land on different nodes, we'll also have to pull the image twice).
Reviewer Checklist

[ ] Implementation matches the proposed design, or proposal is updated to match implementation
[ ] Sufficient unit test coverage
[ ] Sufficient end-to-end test coverage
[ ] Docs updated or added to /doc
[ ] Commit messages sensible and descriptive
[ ] Tests marked as [FLAKE] are truly flaky
[ ] Tests that remove the [FLAKE] tag are no longer flaky

notes to self:  there's some logic in EnsureRegistryServer that tries really hard to ensure the existing catalog pod has an image field value that exactly matches the image field value from the catalogsource object, which makes it hard to create a pod based on the SHA instead.
Everything else about this PR seems to work as expected, but then that reconcile logic comes in and stomps the SHA based pod back to being whatever value is in the catalogSource.

well this "works" w/ the caveat that for an initial catalog creation:

a pod is created as pullalways, referencing the catalogsource image pullspec
the update pod gets created as pullalways referencing the pullspec
if the update pod does not pull a new SHA, the update pod is thrown away and the pod from (1) is kept, meaning we still have a pod using pullalways and a tag

if the update pod does pull a new SHA, this code correctly deletes the update pod and creates a new "update" pod using pullifnotpresent with the newly determined SHA, which then gets promoted and the original catalog pod is deleted.
So until the first SHA update happens, the catalog pod is still using pullalways and a tag.
But the more important thing i realized while doing this is it doesn't really help at all because these pods are not deployment managed, so if the catalog pod gets deleted/drained and recreated on a new node, we're back to having to do an initial pullalways of a tag.
So to really solve this problem, we need to:

create the pullalways updated pod using the pullspec from the CatalogSource spec
determine if the update pod gets a new SHA
place that SHA on the CatalogSource status for future reference
whenever creating a catalog pod, create it based on the SHA that is on the CatalogSource status, using pullifnotpresent.

i'm going to close this since per https://github.com/operator-framework/operator-lifecycle-manager/pull/2659#issuecomment-1072655734 i think it's a dead end, but i think the overall direction is worth considering for how catalog pods are created/managed, to shift them to a SHA-based, pullifnotpresent configuration.


Use Cases & Benefits

I only see one sentence in the current article which even hints at when and why I might want to use this - "Azure AD replaces the need to configure and use client authentication certificates."
Please consider adding more info on what use cases this best serves, and what the benefits of using AzureAD auth are over existing methods.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: acbd8029-6391-7ae1-a098-9c348be1dd98
Version Independent ID: e202fe2f-876c-44a5-5e03-d769ef31f72c
Content: Install the client with Azure AD - Configuration Manager
Content Source: sccm/core/clients/deploy/deploy-clients-cmg-azure.md
Product: configuration-manager
GitHub Login: @aczechowski
Microsoft Alias: aaroncz

Thanks @NeighborGeek for the feedback. It probably just needs a link over to the Azure AD section of this article: https://docs.microsoft.com/sccm/core/plan-design/security/plan-for-security#bkmk_planazuread, which provides this information.

Unless I'm missing something, that link talks about what SCCM features use
AzureAD integration, which I don't think is quite what I was looking for.
I wanted to know, when would I need to configure an SCCM client
specifically to use AzureAD authentication, as opposed to certificate
auth?
When I enable co-management for example, it works for all of my existing
SCCM clients, I don't have to switch them over to AzureAD auth.  Unless
that's being done automatically behind the scenes?
On Thu, Feb 21, 2019 at 12:35 PM Aaron Czechowski<EMAIL_ADDRESS>wrote:

Thanks @NeighborGeek https://github.com/NeighborGeek for the feedback.
It probably just needs a link over to the Azure AD section of this article:
https://docs.microsoft.com/sccm/core/plan-design/security/plan-for-security#bkmk_planazuread,
which provides this information.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/SCCMdocs/issues/1259#issuecomment-466114301,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADPo5q9x0o7S0dVJxALWLSxldSOEvrIvks5vPubxgaJpZM4bH4TA
.

Setting up Azure AD may be easier for some customers than setting up PKI. If you're already using certificate authentication, then there are not many client-side benefits. There are features that require you onboard the site to Azure AD, but don't necessarily require the clients to be Azure AD-joined. I'm working on this article right now, so I'll add some language around this topic, and then close this issue. Thanks for the feedback!

The changes should publish live tomorrow (Wednesday, 6 March) around 11am Pacific time.


[RIP-68] RocketMQ ACL 2.0

Status

Current State: Draft
Authors: ShuangxiDing<EMAIL_ADDRESS>Shepherds: lollipopjin<EMAIL_ADDRESS>Mailing List discussion<EMAIL_ADDRESS>Pull Request: #PR_NUMB
Released: <relased_version>

Background & Motivation
What do we need to do

Will we add a new module? --No.
Will we add new APIs? --Yes.
Will we add a new feature? --Yes.

Why should we do that
Are there any problems with our current project?
Currently, the RocketMQ ACL design has the following limits:

Non-standard IP whitelist control: As usual, IP whitelist is used for restricting users to access from specific IP or IP ranges. However, in the current RocketMQ ACL, the IP whitelist is used to bypass authentication, which is inconsistent with the common way. This inconsistency may lead to security issues as it allows untrusted IP addresses to bypass access control.
Lack of a scalable interface definition: Currently, ACL configuration information is stored in YAML file pattern. However, in some cloud scenarios, there is a desire to achieve ACL configuration information extension through database storage or microservice provisioning, which is not very convenient in the current design. This limits the flexibility and scalability of ACL configuration, and may not be suitable for specific application scenarios. At the same time, the access control for control-related interfaces is relatively insufficient, resulting in the risk of cluster data leakage.
Ineffective separation of users and permissions: Best practice is to separate the logic of creating users and setting permissions, clearly defining the responsibilities of each. However, in the current design, users and permissions are coupled in a single file, which can lead to the potential leakage of user passwords and subsequently, security issues.

当前RocketMQ ACL的设计存在以下一些限制：

IP白名单控制非标准化。通常情况下，IP白名单用于限制用户只能从特定IP或IP段进行访问。然而，当前RocketMQ ACL中的IP白名单被用于跳过鉴权验证，与常见的IP白名单控制方式不一致。这种不一致性可能导致安全性问题，放过不受信任的IP地址绕过访问控制。
ACL配置缺乏可扩展的接口定义。目前，ACL配置信息采用文件形式存储，但在某些云上场景中，希望能够通过数据库存储或微服务提供来实现ACL配置信息的扩展，但当前设计并不十分方便。这限制了ACL配置的灵活性和可扩展性，并且可能不适用于特定应用场景。同时对于管控相关接口的访问控制也相对不足，导致集群数据存在泄露的风险。
用户和权限未被有效地分离。最佳实践是将创建用户和设置权限的逻辑进行分离，明确划分两者的职责。然而，当前设计中用户和权限之间耦合在一个文件中，这会导致用户的密码可能会被泄露，从而引发安全问题。

What can we benefit from proposed changes?

Standardized IP whitelist control: The enhanced ACL design provides a more standardized IP whitelist control mechanism. It effectively restricts user requests to specific IP sources and blocks access from untrusted IP addresses.
Scalable ACL configuration and authentication mechanism: The improved design allows for easy extension and implementation of ACL-related logic. Users can conveniently customize and expand ACL configurations to meet their specific requirements. Additionally, the ACL design includes access control for control-related interfaces, enhancing the overall security of the system.
Effective separation of user and permission management: The optimized design successfully achieves a clear separation between user authentication and permission management, establishing explicit responsibilities and boundaries for each. This enhancement significantly improves the security of the system. Additionally, user passwords are securely encrypted, effectively reducing the risk of password leaks.

更加标准化的IP白名单控制：优化后的ACL设计提供了更加标准化的IP白名单控制机制。它有效地限制了用户只能从特定的IP来源进行请求，并阻止了不受信的IP地址的访问。
可扩展的ACL配置和鉴权逻辑：改进后的设计允许用户便捷地扩展和实现ACL相关逻辑。用户可以方便地自定义和扩展ACL配置，以满足特定需求。此外，ACL设计还包括对管控接口的ACL访问控制，提升了系统的整体安全性。
用户权限有效分离：优化后的设计实现了用户认证和权限管理的有效分离，明确划分了它们的职责和边界，提升了系统的安全性。同时，用户密码采用安全加密，有效降低了密码泄露的风险。

Goals

What problem is this proposal designed to solve?

To address the aforementioned issues, the key objectives of RocketMQ ACL 2.0 optimization are as follows:

Provide a relatively standardized and universal model to support user authentication for different operations on all resources in RocketMQ. This ensures consistency and comprehensiveness in permission control throughout the RocketMQ system.
Offer a set of standardized interface abstractions for user and permission management, as well as authentication logic in RocketMQ, to facilitate extension and implementation. This enables users to flexibly customize ACL configurations based on their specific needs and seamlessly integrate with other systems.
Implement separation of users and permissions, while supporting encryption of user passwords, to prevent security issues caused by password leakage. By separating user and permission management logic and employing appropriate encryption measures, user password security is ensured.
Provide a certain level of compatibility with ACL 1.0 to facilitate a relatively seamless upgrade from ACL 1.0 to the 2.0 mode. This minimizes inconvenience during the upgrade process and preserves users' existing ACL configurations and permission settings.

为了解决以上问题，RocketMQ ACL 2.0的主要目标如下进行优化：

提供一个相对标准且通用的模型，以支持RocketMQ中所有资源的不同操作的用户鉴权。这将确保权限控制在整个RocketMQ系统中具有统一性和完备性。
为RocketMQ的用户、权限的管理，以及鉴权逻辑提供一套标准的接口抽象，以方便进行扩展和实现。这将使用户可以根据自身需求灵活地定制ACL配置，并且能够轻松地与其他系统进行集成。
实现用户和权限的分离，同时支持用户密码的加密，以避免密码泄露引发安全问题。通过将用户和权限的管理逻辑分离，并采用适当的加密措施，确保用户的密码安全性。
提供与ACL 1.0的一定程度上兼容的功能，以便用户能够相对轻松地从ACL 1.0升级到2.0模式。这将减少升级过程中的不便，并保护用户现有的ACL配置和权限设置。

To what degree should we solve the problem?

Introduce a new user authentication and permission verification system (2.0) to replace the existing ACL 1.0 version.
Authorization scope covers message sending, message consumption, and all admin-related interfaces.
Retain ACL 1.0's authorization logic and configuration, with version switching supported through configuration, allowing for a seamless migration from 1.0 to 2.0.

新增一套全新的用户认证和权限验证体系（2.0），以替代现有的ACL 1.0版本
鉴权范围覆盖消息发送、消息消费，以及所有admin相关的接口
保留ACL 1.0的鉴权逻辑及配置，通过配置支持版本切换，支持从1.0迁移到2.0版本

Non-Goals

What problem is this proposal NOT designed to solve?

To ensure the security within the RocketMQ cluster, we plan to introduce a permission control mechanism in scenarios such as Broker registration with NameServer and primary-backup replication between Brokers. This is to prevent data leaks and other security issues. This feature will be supported in subsequent iterations.

During the Broker registration process with NameServer, we will control access permissions for relevant interfaces to ensure that only authorized Brokers can successfully register. This effectively prevents unauthorized Brokers from accessing the cluster, thus protecting the entire cluster's security.
During primary-backup replication between Brokers, we will also implement permission control for relevant interfaces. Only authorized backup Brokers can obtain data replication permissions from the primary Broker, ensuring that data replication only occurs within authorized scope and preventing the risk of unauthorized data leaks.

In order to enhance user management experience, the RocketMQ open-source console plans to introduce permission control functionality to achieve fine-grained resource management. This functionality mainly includes two aspects:

Console login authentication: During the console login process, users need to provide a username and password for identity verification. As the Broker already supports permission control for management-related APIs, the console also needs to carry the corresponding user information when accessing the Broker. This ensures that only authorized users can successfully access the resources. In this iteration, we provide a solution that allows users to add a username and password with full access control permissions in the console's configuration to access the Broker. User login permission management will be further improved in subsequent versions.
Console user and permission management: In order to configure users and permissions in a more friendly and convenient manner, we plan to provide a visual interface for managing users and permissions, instead of relying on command-line operations. This will greatly enhance the user experience and simplify the operation process. This feature will be implemented in subsequent iterations.

为了确保RocketMQ集群内部的安全性，在诸如Broker向NameServer注册、Broker之间主备复制等场景中，我们计划引入权限控制机制，以防止数据泄露等安全问题的出现。该功能将在后续迭代中予以支持

在Broker向NameServer注册的过程中，我们将对相关接口进行访问权限的控制，确保只有授权的Broker才能成功注册。这样可以有效防止未经授权的Broker接入集群，从而保护整个集群的安全。
在Broker之间进行主备复制时，我们也会对相关接口进行权限控制。只有经过授权的备份Broker才能获取主Broker的数据复制权限，确保数据只在授权范围内进行复制，防止未经授权的复制操作导致数据泄露的风险。

为了提升用户管理体验，RocketMQ开源控制台计划引入权限控制功能，以实现对资源进行精细化管理。该功能主要包括两个方面：

控制台登录鉴权：控制台登录过程中，用户需要提供用户名和密码进行身份验证。由于Broker已经支持对管控相关API的权限控制，因此控制台访问Broker时也需要携带相应的用户信息。这样可以确保只有经过授权的用户才能成功访问资源。在本次迭代中，我们提供了一种解决方案，允许用户在控制台的配置中添加一个具有全部访问控制权限的用户名和密码，以实现对Broker的访问。用户登录权限的管理将在后续版本中进行完善。
控制台用户及权限管理：为了更加友好和便捷地配置用户和权限，我们计划提供一个可视化的界面来管理用户和权限，而不再依赖于命令行操作。这将大大提升用户的使用体验，并简化操作流程。该功能将在后续的迭代中实现。

Changes
Concepts
RocketMQ ACL 2.0 primarily adopts the Attribute-Based Access Control (ABAC) model in its permission system. The main concepts are as follows:
关于RocketMQ ACL 2.0主要采用权限体系中的ABAC模型，主要的概念如下：

定义
名称
描述

Principal
主体
The principal typically refers to the entity that attempts to access a resource, which can be a user, application, service, or other entity. 主体通常是指试图访问资源的实体，可以是用户、应用程序、服务或其他实体。

Resource
资源
A resource is an object that needs to be subject to access control, such as a cluster, topic, group, and so on. 资源是需要受到访问控制的对象，如集群、Topic、Group等。

Action
动作
An action represents the operation that a subject attempts to perform, such as read, write, delete, execute, and so on. Different actions may require different permissions and policies for control. 动作表示主体试图执行的操作，如读取、写入、删除、执行等。不同的动作可能需要不同的权限和策略来控制。

Environment
环境
An environment represents the contextual environment in which an access request occurs, such as time, location, network status, and so on. 环境表示访问请求发生的上下文环境，例如时间、地点、网络状态等

Attributes
属性
Attributes are characteristics or information about subjects, resources, and context, used to describe various aspects of entities. 属性是关于主体、资源和上下文的特征或信息，用于描述实体的各个方面

Policy
策略
Authorization policies define authorization rules and conditions that determine whether a subject is allowed to perform specific actions on a resource. 授权策略定义了授权规则和条件，用于决定主体是否有权对访问资源执行特定的动作

Decision
决策
A decision is the outcome of an access request, where the system determines whether to grant or deny access based on the defined access policies and attribute information. 决策是访问请求的结果，基于定义的访问策略和属性信息，系统决定是否授予访问权限。通常是允许（Grant）或拒绝（Deny）。

Evaluation Engine
评估引擎
An evaluation engine is a component used to execute ABAC rules and policies. It makes access control decisions based on the attributes of the subject, attributes of the resource, and the required action. The evaluation engine typically includes permission validation and decision making. 评估引擎是用于执行ABAC规则和策略的组件，它根据主体的属性、资源的属性以及所需的动作来做出访问控制决策。评估引擎通常包括了许可的验证和决策。

Architecture
Original

In the original architecture of RocketMQ, the Broker is responsible for managing the storage of users and permissions, as well as implementing the logic for authentication and authorization. Whether it is handling interfaces such as message sending and receiving, cluster management, or data querying, all must go through ACL's authentication and authorization logic to ensure the legitimacy of all requests.
在RocketMQ的原始架构中，Broker负责管理用户和权限的存储，并实现认证和授权的逻辑。无论是处理消息的发送和接收、集群管理还是数据查询等接口，都必须经过ACL的认证和授权逻辑，以确保所有请求的合法性。
With Proxy

In the RocketMQ architecture with Proxy, the sending and consumption of messages are delegated to the Proxy component, hence authentication and authorization are carried out within the Proxy component. However, the cluster management-related interfaces are still handled by the Broker, requiring authentication and authorization from the Broker. As for the storage of user and permission information, due to the fact that only the Broker is a stateful application, it is still stored on the Broker, and the Broker provides corresponding interfaces for the Proxy to query and perform subsequent authentication logic.
在具备Proxy的RocketMQ架构中，消息的发送和消费由Proxy组件进行代理，因此在Proxy组件中进行认证和授权。然而，集群管理的相关接口仍由Broker负责处理，因此这些接口需要经过Broker的认证和授权。关于用户和权限的存储方面，由于只有Broker是有状态应用，因此仍将其存储在Broker上，并由Broker提供相关接口供Proxy查询，以执行后续的鉴权逻辑。
Domain Model

Date Type
Principal

字段
名称
描述

User
用户

Resource

字段
名称
描述

Cluster
集群

Tenant
租户

Namespace
命名空间

Topic
主题

Group
消费者组

User
用户

ACL
权限

Action

字段
名称
描述

PUB
Produce messages 生产消息

SUB
Consume messages 消费消息

Create
Create resource 创建资源

Update
Update resource 更新资源

Delete
Delete resource 删除资源

Describe
Describe resource detail 描述资源详情

List
Batch list resources 批量查询资源

Interface Design/Change

First, all APIs related to RocketMQ ACL 1.0 will be removed.
Then, all APIs related to RocketMQ ACL 2.0 will be added.

首先，会去除所有关于RocketMQ ACL 1.0相关的API
然后，会新增所有关于RocketMQ ACL 2.0相关的API

The following are the API definitions for RocketMQ ACL 2.0:
以下是RocketMQ ACL 2.0的API定义：
User management

Interface definition
Interface name
Interface description

CreateUser
Create user api 创建用户

DeleteUser
Delete user api 删除用户
Deleting a user will also delete their permissions. 删除用户同时会删除权限

UpdateUser
Update user api 更新用户
Primary update the user's password.

主要更新用户密码

DescribeUser
Describe user detail api 查询用户详情
This interface will return the user's password. 该接口会返回用户密码

ListUser
Batch list user api 查询用户列表
This interface does not return the user's password. 该接口不返回用户密码

Permission management

Interface definition
Interface name
Interface description

CreateAcl
Create acl 创建权限

DeleteAcl
Delete acl 删除权限
It is possible to delete individual permission policies under an account or delete the entire permission policy. 可删除账号下单个权限策略，也可以删除整个权限策略

UpdateAcl
Update acl 更新权限
Individual permission policies under an account can be updated, as well as the entire permission policy. 可更新账号下单个权限策略，也可以更新整个权限策略

DescribeAcl
Describe acl detail 查询权限

ListAcl
Batch list acl 查询权限列表

Implementation
Mapping of API, Resources, and Actions
为了方便获取每个API请求对应的操作和资源信息进行鉴权，需要进行以下改造：
To facilitate the authorization process by obtaining information about the operation and resource corresponding to each API request, the following modifications need to be made:
Remoting API
For Remoting-related interfaces, as there are a large number of interfaces and each interface has a different resource name definition, to facilitate obtaining resource definitions, annotations can be used to label each API's RequestHeader, indicating which fields are resources and their IDs. This way, it will be easier to retrieve relevant information during the authorization process. The specific steps are as follows:
对于Remoting相关接口，由于接口数量较多且每个接口定义的资源名称不同，为了方便获取资源定义，可以通过注解的方式为每个API的RequestHeader进行打标，标识字段中哪些是资源以及资源的ID。这样可以在鉴权过程中更方便地获取相关信息。具体如下：

Considering the frequent invocation and high performance requirements of the message sending and receiving interfaces, a hard-coded implementation approach is still used to ensure performance.
考虑到消息收发相关接口调用频繁且对性能要求较高，仍然采用硬编码的方式实现消息收发接口，以保证性能。
gRPC API
For gRPC-related interfaces, it is difficult to implement them using abstract classes and annotations because the protocol is defined and generated through proto files. Currently, it mainly involves PUB and SUB types of interfaces, so it can be implemented through hardcoding. Please refer to:
对于gRPC相关的接口，由于协议是通过proto文件进行定义和生成的，难以通过抽象类和注解的方式实现。目前主要涉及PUB和SUB类型的接口，因此可以先通过硬编码的方式实现，具体参考：
org.apache.rocketmq.acl.plain.PlainAccessResource#parse(com.google.protobuf.GeneratedMessageV3, org.apache.rocketmq.acl.common.AuthenticationHeader)
Storage model
The storage of ACL is primarily kept locally in the Broker, and there are two main options: file storage and RocksDB-based storage. In this case, RocksDB is used for storage, while file storage is provided as an example for reference purposes.
关于ACL的存储主要在Broker本地进行，有两种主要方案可供选择，一种是文件存储，另一种是基于RocksDB的存储。本次存储采用RocksDB，而文件存储仅作为示例供参考。
4.2.1 File storage
User file storage
[{
  "username": "rocketmq",
  "password": "xxxxxx"
}]

Permissions file storage
[{
  "principal": "User:rocketmq",
  "policies":[{
    "policyId": 1,
    "resources":["Topic:topic-*","Group:group-*"],
    "actions":["PUB","SUB"],
    "environment":{
      "sourceIps": ["<IP_ADDRESS>/24"]
    },
    "decision": "Grant"
  }]
}]

RocksDB-based storage
In RocksDB, the data storage method is similar to file storage, but it uses the principal as the primary key for storage. Compared to file storage, using RocksDB may be simpler in terms of management.
在RocksDB中，存储的数据方式与文件存储相似，但是使用主体(principal)作为主键进行存储。相对于文件存储，使用RocksDB的方式可能在管理层面更加简单。
ACL evaluation

Compatibility, Deprecation, and Migration Plan
Are backward and forward compatibility taken into consideration?
Yes. In order to facilitate user migration, relevant compatibility handling needs to be done. Here are several modifications for compatibility:

Add the corresponding version number in the configuration item, such as rocketmq.acl.version. By default, the value is 2.0, but users can change it to 1.0 to read the existing files and execute the original authentication logic.
For ACL 1.0, the existing authentication logic should be kept separate from the newly added 2.0 authentication logic to avoid mutual impacts on subsequent code modifications.

是的。为了方便用户进行迁移，需要进行相关的兼容性处理。以下是针对兼容的几点改造：

在配置项中添加相应的版本号，例如rocketmq.acl.version，默认情况下该值为2.0，但用户可以将其改为1.0，以读取原有的文件并执行原有的鉴权逻辑。
针对ACL 1.0，仍然保留现有的鉴权逻辑，和新增的2.0鉴权逻辑分离，避免后续代码修改的相互影响。

Are there deprecated APIs?
Yes, the APIs related to ACL 1.0 will be removed. If you want to modify the permissions of ACL 1.0, you can achieve it by directly modifying the file.
是的，ACL1.0相关的API会被移除。如果你想要修改ACL1.0的权限，你可以通过直接修改文件的方式来实现。
How do we do migration?

Add the configuration rocketmq.acl.version=1.0 to the Broker's configuration file, then upgrade to a RocketMQ version that supports ACL 2.0.
Use the Admin tool to migrate the permission configurations from ACL 1.0 to ACL 2.0, ensuring data integrity. Note that the global IP whitelist is no longer supported. Please confirm whether you need to use it and implement it through alternative means.
Modify the Broker configuration by changing rocketmq.acl.version to 2.0, then restart the Broker to confirm the correctness of the authentication.

在Broker的配置文件中添加 rocketmq.acl.version=1.0 配置，然后升级到支持ACL 2.0的RocketMQ版本。
使用Admin工具将ACL 1.0的权限配置迁移到ACL 2.0中，确保数据完整性。请注意，全局IP白名单不再支持，请确认是否需要使用，并通过其他方式实现。
修改Broker配置，将 rocketmq.acl.version 改为2.0，并重启Broker，确认鉴权是否正确。

Implementation Outline
The feature is currently under development, and I will complete it as soon as possible and submit it to the community.
该功能正在研发中，我会尽快完成并提交到社区。

Regarding the ACL storage component, i have a little question, especially as the Controller and Dledger modes are increasingly becoming the top choices. Why not place ACL information in Namesrv? This could potentially simplify Broker scaling, and proxy will get the metadata(include acl) from namesrv.

@echooymxq
First, the NameServer is a stateless application that allows for flexible scaling and dynamic upscaling or downscaling without bearing the responsibilities related to data persistence storage.
Second, at the architectural level, the NameServer does not handle metadata capabilities; instead, it serves the functions of service registration and routing discovery.
Finally, as a stateful application, the Broker has the responsibility and capability for metadata storage at the data storage layer.

@dingshuangxi888 The namesrv not  stateless actually. such as in controller mode, maybe the controller mode is special case. but there's some kvConfig.json about orderConf, and rocketmq-mqtt also rely on it.

Interesting, looking forward to contribute!

@dingshuangxi888 can you assign somthing to me to contribute on this feature?

when this feature release?

If you use the proxy cluster mode and enable authentication, each request is authenticated twice (proxy and broker)？@dingshuangxi888

If you use the proxy cluster mode and enable authentication, each request is authenticated twice (proxy and broker)？@dingshuangxi888

The configuration for the proxy and the broker is different; only the proxy will be enabled if it's in local mode.

Are there any plans to reverse this ability into RocketMQ 4.x?


Bug Report: everytime i finish a song, the game crashes because of the replays feature, i play on windows 10 64 bits, an i have tried in various versions of the game, and is always the same problem

Please check for duplicates, similar issues or closed issues with the same problem, as well performing simple troubleshooting steps (such as deleting the export folder if it's an issue with compiling, reinstalling Kade Engine) before submitting an issue.
If you are playing a downloaded version of the game, what operating system are you using?
Windows (x86), Windows (x86_64), Linux, or macOS? Specify below.
What version of the game are you using? Look in the bottom left corner of the main menu.
Have you identified any steps to reproduce the bug? If so, please describe them below. Use images if possible.
Please describe your issue. Provide extensive detail and images if possible.

this is what happens everytime


Doesn't always work.

Sometimes it works, sometimes it doesn't. its very picky on which images it wants to work on.

Sorry, due to the sunsetting of Powercord, this repo will no longer be maintained.


Short URL is based on localhost host.

On production it returns something like localhost:3000/shortener.php/zdjmmwiy and writes it to DB so it's impossible to get the right redirect.

Thanks for this issue @GSokol. Will be waiting your patch. Thanks :)

As a heads up, I think the issue is here: https://github.com/ch3nkula/url-shortener/blob/master/php/shorten.php#L8. By removing "dev" as default argument to constructor, this should do the job. :)

https://github.com/ch3nkula/url-shortener/pull/27 FIX it


Move optional dependencies to extras

IMPORTANT: this PR requires a migration path
We should think about how and when we will expose users to this change.
Motivation

Fixes #443

Checklist
Before you create the PR, please make sure you have:

[ ] Picked a concise, informative and complete title
[x] Clearly explained the motivation behind the PR
[x] Linked to any existing issues that your PR will be solving
[ ] Included tests for any new functionality
[ ] Checked that all unit tests pass

Workflow
Please avoid rebasing and force-pushing to the branch of the PR once a review is in progress.
Rebasing can make your commits look a bit cleaner, but it also makes life more difficult from the reviewer, because they are no longer able to distinguish between code that has already been reviewed, and unreviewed code.

I'm not sure about this but I think the imports in the code should now be in try except ? Like if someone installs the smart_open[gcs] the smart_open.s3 will throw an import error

Test coverage keeps failing in 3.8. @Tyrannas technically not yet because this doesn't not install the AWS deps when you do smart_open[gsc] as they are still in the base requirements.

We can still add it just so it's not forgotten though.

We can still add it just so it's not forgotten though.

I was thinking something like:
try:
    import boto3
except ImportError:
    logging.getLogger('smart_open').info("Boto3 dependency is not installed, smart_open s3 will therefore not be working.")

Using info instead of warning to avoid regular loggers to be spammed since logging level is by default warning.
What do you think?

@Tyrannas I think we already do this:
https://github.com/RaRe-Technologies/smart_open/blob/29092ba823ba61028598dbe94324b750f19244a1/smart_open/transport.py#L43
The next step is to get the backends to automatically tell the users which packages they need, e.g. "to enable S3 support, run pip install smart_open[s3]".  Not sure what the best way to do that is, but it's probably out of scope for this PR.

@Tyrannas I think we already do this:
https://github.com/RaRe-Technologies/smart_open/blob/29092ba823ba61028598dbe94324b750f19244a1/smart_open/transport.py#L43
The next step is to get the backends to automatically tell the users which packages they need, e.g. "to enable S3 support, run pip install smart_open[s3]". Not sure what the best way to do that is, but it's probably out of scope for this PR.

@mpenkov yeah but for example you are importing .s3 in smart_open init file and .s3 imports boto3, so this will lead to an Import Error

We can just use deprecation warnings. It's built into python.
https://docs.python.org/3.6/library/warnings.html

Merging. Thank you for the hard work @Amertz08 .
I've made additional issues #463 and #464 to continue the discussion @Tyrannas .


Suggestion: reduce readme's top buttons

IMHO readme's top is overloaded with buttons. I understand desire to show "awesomeness" of project, but this looks like tons of animated smiles :). You already has webassembly builds, that's much better to make impression. Also, some button's links go to image. That's useless.
Buttons are usually expected to contain some super-important info. For example:

CI tests status
Coverage %
Version number
Link to gitter chat (to keep tracker more clean from simple question)

I do not insist, only share personal opinion/experience. Feel free to close if it's not useful for you.

Some updates:

I changed the display frame to a more realistic one.
Reduced a the shadow

I'd like to keep the center alignment. I asked my designer colleague who also thinks it really looks better.
@puzrin
I suggest forking lvgl and update the Readme as you think. This way we can compare them.

No problem. See https://github.com/puzrin/lvgl. A did a bit more polish. Use any part you wish if you like it.

Thank you! To be honest it looks better than I thought! :sweat_smile:
I still need to think about it.

... Looks like collection of buzzwords :).

That was the goal :) Make it as informative and impressive as possible.

Header + animated image already contain this info in more descriptive form.

Having a description is crucial because it will be displayed in a Google search:

Hm...
Search "lvgl":

Search "embedded gui":

Updated readme:
https://github.com/puzrin/lvgl
Moved text below image, try to visually hide without remove.

+links order +header
I suggest use anything you prefer and close this. Issue is too primitive to spending more time for it.

I suggest use anything you prefer and close this. Issue is too primitive to spending more time for it.

You are right. The Reame was really too crowded. I think it's okay now. From now it's only a matter of taste.
Thank your for the suggestions!


new config and Dockerfile for 0.12.1

this looks good! I'd be in favor of moving this branch as a PR on master and closing the 0.10beta branch since it is no longer very useful.

closing in favor of #11


Add some information about class, pseudostate etc to make it comprehensive. Also, the glitch project is slightly different from the examples used

Welcome! Please use this template for reporting documentation issues on https://developers.google.com/web/.
For questions about Chrome DevTools, checkout the
google-chrome-devtools tag on StackOverflow.
For questions about Chrome, check the google-chrome
tag. If you think you've found a Chrome bug, check out our
feedback page for more details on how to file it.
Page Affected: https://developers.google.com/web/TODO
What needs to be done?
Please provide description of what's needs to be done here. Please be sure to include the link to the original page
and any supporting links/data.

[ ] Appropriate type-something label has been added.
[ ] Appropriate section-something label has been added.

What page are you referring to?


free_command_buffers()

I don't see DeviceV1_0::free_command_buffers()
Is that just an oversight?

@mikedilger Yes, there are actually a bunch of missing functions at the moment. You can still use the "raw" C function with some_device.fp_v1_0().free_command_buffers(..); as a temporary workaround. I'll add it this week.
I mostly only wrap functions that I currently use in my project and I never free command buffers at all, I only free them by destroying the pool, or I just reset them.

Fixed by https://github.com/MaikKlein/ash/commit/463e862a45b56f7a3e2221d6ad62532ddb8ecd21


@gr2m/hapi-to-express

Workaround until https://github.com/jakepruitt/hapi-to-express/pull/6 is released. See also: https://github.com/jakepruitt/hapi-to-express/issues/5

this is already reviewed via jakepruitt/hapi-to-express#5

okay I of course forgot to update the actual require() in the code :( I’m sorry, should have been patient after all ugh


Folders for each channel

Would it be possible to create an option where, for each video, it creates a new folder for each channel or chooses an existing folder with the name of the channel, and it creates the note there? This could be in addition to or an alternative to #4.

It would be an addition to #4 in the next release.

Please, update to 1.1.0. Your request should be resolved


Provide support for using AND logic for triggers

Proposal
Allows end-users to scale out/in when the queue depth is high enough, and it is within a given time window, for example
Use-Case
Provide support for using AND logic for triggers so that it scales when all triggers meet the criteria
Anything else?
No response

I think we can close this because it's not doable without formulas (and they already have its own issue) because it's the HPA Controller who does it, KEDA only exposes the raw metrics.
WDYT?

I don't really agree as the formula should be transparent to the end-user.
If I have two triggers, I should simply be able to say scale when both meet the criteria and not worry about writing formulas.
(That does not mean formulas are bad, but they serve a different audience)

#3330 is related but not 100% the same, #2440 is what this feature should build on but not solve my needs.
And then there is also https://github.com/kedacore/keda/issues/2614 but that's more advanced and for later on imo

@tomkerkhove Close this?
It can be implemented using a formula applied to a cron trigger and a queue trigger, right?

I belive we could close this, wdyt @tomkerkhove ?

I'd personally say we need to have explicit docs on doing AND operations. It feels like these formulas are the new silver bullet; but I personally argue that this is too technical and the UX for this should be simpler.
If somebody wants to do AND; we need to provide a knob and we'll build the formula for them.

This is why I believe we need a simpler and more integrated way to do AND: https://github.com/kedacore/keda/discussions/6086


dataset download problem

Hi! I have the connection problem with google drive. Please upload your rna dataset to huggingface or other cloud disk that is accessible in mainland

Hello, I wanted to let you know that I have uploaded the dataset to Baidu Netdisk and included the link in the README file. Please check the README file to access the dataset. Thank you.


fix(release-2.9): remove using undefined flags in magefile

What problem does this PR solve?:
Incorrect Magefile changes were merged when cherry-picking https://github.com/mesosphere/konvoy-image-builder/pull/1133 to release-2.9 branch.
This PR removes use of the unused flags.
Which issue(s) does this PR fix?:

https://jira.d2iq.com/browse/D2IQ-NUMBER

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

The RHEL 7.9 and RHEL 8.4 are deprecated. Their mirrors does not have needed packages to build the VM.
We will ignore the e2e failures in RHEL 8.4 and RHEL 7.9 for releasing v2.9.9.
We need to backport https://github.com/mesosphere/konvoy-image-builder/pull/1072 later after v2.9.9 is released.


add: resticprofile-bin

Configuration profiles manager for restic backup.

This is fully tested and ready for merging.

Oops, I did it twice 😁

Fix the merge conflicts then I can merge.


Cannot pass parameters to Invoke-WebRequest from New-RsRestSession

We van no longer deploy reports using New-RsRestSession cmdlet to a non-SSL internal URL.  We get the error:
 The cmdlet cannot protect plain text secrets sent over unencrypted connections. To suppress this warning and send plain text secrets over unencrypted networks, reissue the command specifying the AllowUnencryptedAuthentication parameter.
We would like to set the AllowUnencryptedAuthentication parameter, but we are not aware of any way to send this to the internal commands that the cmdlet calls.
This used to work in previous versions (without any flag setting), but it seems that since newer versions require newer Powershell, this is blocking us from deploying.

We have the same issue.  Please respond, if at all possible.

Created a fork that will work with non-SSL Report Servers:
https://github.com/schwarrior/HttpEnabledSsrsPsTools
The inherited cmdlets were modified to pass a new security gate added to PowerShell 6.0.
Use at your own risk.
Invoking web (or intranet) endpoints that don't support HTTPS requires the inclusion of a new confirmation flag.
In this fork, all instances of Invoke-WebRequest and Invoke-RestMethod within module functions now include the -AllowUnencryptedAuthentication flag.
Replaced 78 results in 27 files.
Ability to use a HTTP only Reporting Server portal restored.
Updated module name and .net namespace name to match fork.
Removed scripts that would deploy to public pckg repositories.
Replaced any install scripting referencing github or any public pckg repo. Replaced with local install commands.
Added disclaimer to top of ReadMe.md regarding the risk of exposing login info.


CreateNewProduct

Hello, I'm not sure if this is a bug, or i'm doing something completely wrong.
I am using a button to:
        private async void button1_Click(object sender, EventArgs e)         {             client.Initialize("http://www.MySite.com", "Consumer", "Secret");             client.AuthenticateAdmin("User", "Pass");             await AddProducts();         }
Which runs:
`private async Task AddProducts()
{
        Debug.WriteLine("Working");

        string mySku = "FPP 923829832";

        Product prod = new Product
        {
            name = "Name",
            description = "Raider",
            short_description = "supershort",
            sku = mySku,
            price = 12.34,
            type_id = "simple",
            attribute_set_id = 4,
            weight = 1,
            status = ProductStatus.Enabled,
            visibility = ProductVisibility.CatalogSearch,
            tax_class_id = 0
        };

        var response = client.CreateNewProduct(prod).Result;

        if (response.HasErrors)
        {
            label2.Text = "Error : " + response.ErrorString;
        }
        else
        {
            var newProduct = client.GetProductBySku(mySku).Result;
            label2.Text = "Success : " + newProduct.Result.entity_id.ToString();
        }
        Debug.WriteLine("DONE Working");
    }`

It seems to be hanging up on:
            var response = client.CreateNewProduct(prod).Result;
According to fiddler, the response is:
{"messages":{"error":[{"code":400,"message":"The request data is invalid."}]}}
I am using Magento Community <IP_ADDRESS> if that helps anything.
Thanks.

Are you running this in an asp.net website?