da: celestia-openrpc - use blob commitment

Overview
This PR updates the FrameRef to use a blob commitment using the BlobModule API from celestia-openrpc.
While here also added a unit test for FrameRef serialization / deserialization.
Checklist

[ ] New and updated code has appropriate documentation
[ ] New and updated code has new and/or updated testing
[ ] Required CI checks are passing
[ ] Visual proof for any user facing features like CLI or documentation updates
[ ] Linked issues closed with keywords

We can use BlobModule API Submit(blobs []*blob.Blob)uint64 so that we don't have to hardcode the gas and fee.

are there any changes to existing UX? here or here. i don't think so

are there any changes to existing UX? here or here. i don't think so

Nope. We did remove hardcoded gas and fee but that wasn't documented before anyway.

utACK, just add an invalid test case to the test so we can be even more confident in the encoding and decoding.

Done 👍🏽


Honour project Maven repos in Lambda build.

Agreed; this unblocks me in the short term, and I've created https://github.com/StediInc/cdk-clj/issues/36 to tackle the long-term fix.


Fix metadata's leaked goroutines.

Signed-off-by: mikechengwei<EMAIL_ADDRESS>Fix: https://github.com/apache/skywalking/issues/9459

Codecov Report

Merging #233 (6f0d2d5) into main (2434e59) will increase coverage by 1.12%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #233      +/-   ##
==========================================
+ Coverage   44.04%   45.17%   +1.12%
==========================================
  Files          88       87       -1
  Lines        9092     8872     -220
==========================================
+ Hits         4005     4008       +3
+ Misses       4706     4485     -221
+ Partials      381      379       -2

Impacted Files
Coverage Δ

banyand/measure/encode.go
25.00% <0.00%> (-2.91%)
:arrow_down:

banyand/tsdb/block.go
60.18% <0.00%> (-1.36%)
:arrow_down:

pkg/index/lsm/lsm.go
78.26% <0.00%> (-0.91%)
:arrow_down:

pkg/encoding/int.go
87.91% <0.00%> (-0.17%)
:arrow_down:

pkg/encoding/xor.go
57.57% <0.00%> (ø)

banyand/stream/metadata.go
53.54% <0.00%> (ø)

banyand/tsdb/series_seek.go
0.00% <0.00%> (ø)

banyand/measure/measure_query.go
0.00% <0.00%> (ø)

banyand/tsdb/series_seek_sort.go
0.00% <0.00%> (ø)

pkg/bit/reader.go

... and 8 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


wrong documentation for collections dependencies?

Issue Type

Bug report

Molecule and Ansible details
ansible --version && molecule --version
ansible [core 2.14.1]
  config file = /Users/trw/.ansible.cfg
  configured module search path = ['/Users/trw/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/trw/Library/Python/3.9/lib/python/site-packages/ansible
  ansible collection location = /Users/trw/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/trw/Library/Python/3.9/bin/ansible
  python version = 3.9.6 (default, Oct 18 2022, 12:41:40) [Clang 14.0.0 (clang-14<IP_ADDRESS>)] (/Library/Developer/CommandLineTools/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True
molecule 4.0.4 using python 3.9
    ansible:2.14.1
    delegated:4.0.4 from molecule
    docker:2.1.0 from molecule_docker requiring collections: community.docker>=3.0.2 ansible.posix>=1.4.0

Molecule installation method (one of):

pip

Ansible installation method (one of):

pip

Detail any linters or test runners used:
Desired Behavior
Molecule documentation should mention, that the collections.yml is parsed during dependency management.
And the output of galaxy dependency should change to
INFO     Running default > dependency
WARNING  Skipping, missing the requirements file.
WARNING  Skipping, missing the collections file.

Actual Behaviour
I'd like add a collection to my molecule test, because there are filters in the collection, I need to use. See https://stackoverflow.com/questions/75765614/ansible-molecule-ignores-collection-requirements/75775152#75775152 for description. The role has no other dependencies then this collection.
I've added the collection as described here - https://molecule.readthedocs.io/en/latest/configuration/#dependency
According to that, I cannot use requirements.yml, because it must contains role only.
# molecule.yml
---
dependency:
  name: galaxy
  options:
    requirements-file: collections.yml
...

# collections.yml
---
collections:
  - name: https://github.com/acoby/ansible-collection.git
    type: git
    version: main

But when I run molecule test, I'm getting twice the warning and the test failed because of the missing filters from the collection.
INFO     Running default > dependency
WARNING  Skipping, missing the requirements file.
WARNING  Skipping, missing the requirements file.

I didnt really understand why this happens. Then I removed the options key again
dependency:
  name: galaxy
...

and then it works.
INFO     Running default > dependency
WARNING  Skipping, missing the requirements file.
Cloning into '/homedir/.ansible/tmp/ansible-local-83810gxw1uelq/tmpkm0riwyv/ansible-collectionczw7j2r4'...
...

So, there is something wrong and not documented.

@trickert76
Would you help verify whether it is addressed in #3916?

yepp, thats fine für role-file. But I'm not sure, if its okay with collections.yml in requirements-file. This isnt mentioned in #3916. What is the default search path für requirements-file? Thats not clear. Because I've set requirements-file: collections.yml which is under /molecule/default/collections.yml.

This isnt mentioned in https://github.com/ansible-community/molecule/pull/3916. What is the default search path für requirements-file?

By default, molecule will find role-file and requirements-file from <role-name> if options is not defined.

Does it make sense?


feat(spanner): merge CMEK changes into preview branch

This change is

ci/test-abi/bigtable_client.expected.abi.dump.gz, line  at r1 (raw file):
Previously, devbww (Bradley White) wrote…
Reverted the others.  Thanks.  I understand things are complicated, but it would be nice if update-abi didn't change things it didn't have to.

Created #5470


Parquet column encryption support for Delta

Recently, parquet added support for columnar/modular encryption in version parquet-mr 1.12 (IBM, GitHub), meaning that only the footer file and certain columns of the parquet file are encrypted, minimising encrypt/decrypt overhead while keeping sensitive columns safe. Since Delta is built on top of parquet files and this encryption method became available in Spark 2.3.0 (which the latest Delta version supports), I was wondering if Delta already supports this due to being built on the parquet format or if there is a way to make it work. For my application that would be the perfect combination! (incrementally adding rows while keeping some columns encrypted to keep PII safe).
df.write.option("parquet.encryption.footer.key", "k1").option("parquet.encryption.column.keys", "k2:DeviceName,image_id").parquet(STORE_PATH
This is how it works with spark dataframes in Databricks (which is what I am using). I tried to do something very similar for Delta as follows:
df.write.option("parquet.encryption.footer.key", "k1").option("parquet.encryption.column.keys", "k2:DeviceName,image_id").format("delta").mode("overwrite").save(STORE_PATH_DELTA)
This did work but the Delta table was able to be opened from a cluster not having the encryption keys, meaning the encryption did not actually work. I also tried to use DeltaTable.convertToDelta() on the encrypted parquet files (both in scala spark and PySpark) and this also did not work as the Delta table was not encrypted.
Any help with getting it to work with Delta would be highly appreciated, but learning that this is no (yet) supported and that there is no workaround would also be helpful since I can then look at different options. Thanks in advance!

Delta doesn't support encryption even if parquet supports. A Delta table contains data files and transaction logs. The transaction logs are in json format, we will also create parquet checkpoint to optimize the transaction log reading. But the json files will not be encrypted even if parquet files are encrypted.
Will enabling the encryption in the storage system layer be a sufficient workaround for you?

Thank you for the very fast response. I understand how the way Delta works makes it difficult/impossible to support this type of encryption (and was already fearing this would be the case, but nothing ventured nothing gained :) ).
I will indeed look at such other solutions, though this would've been a beautiful one considering EU regulation (Schrems 2.0 etc.). Again thank you for the response, I'll close the issue now that my questions have been answered!

@zsxwing can we encrypt the data files alone

Can we reopen this request?

hi all, Could we consider revisiting this topic for further discussion? I have some ideas in mind:
Is it possible to implement Parquet's built-in encryption feature for the data files?
Perhaps we could apply the identical footer key encryption configuration to secure the entire metadata file (the JSON file).
If anyone has suggestions or guidance, especially regarding specific sections of the code base we should concentrate on, it would be greatly beneficial as we explore this option.


[prism] Skip python tests that require an expansion service at this time.

Related to #28187.
Skip the copied kafka, SQL, and external transform tests that require an expansion Service stood up.
We should eventually have a post commit for these, but in the short term, filter them out to enable a pre-commit suite to execute.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

R: @jrmccluskey
I'm pretty confident the tests I'm skipping are not what's causing the checks to fail, though I am re-running them.

The line failing is here:
https://github.com/apache/beam/blob/7af43ee4a4a6fb348e57ea066ed6261c453dcc35/sdks/java/container/common.gradle#L118
I'm inclined to say that is not an associated issue

LGTM, only nit is that it may be better to comment out the test content to restore later unless you intend to write new tests later

Oh i originally copied those from the Flink runner tests, so nothing of value has been lost.
Thanks!


Make pdf reader flexible

Features:
add default orientation to pdf files.
fix PDFControls size change, when user zoom in.
Make PDFControls fixed, that the user can access it faster
Fix typo errors in previous pull request.

@cyntler Every thing fixed :)

@salemaljebaly @facularrauri
Merged in the another PR.
README: https://github.com/cyntler/react-doc-viewer/tree/main#config -> pdfVerticalScrollByDefault.


Different mask values when same model is loaded again

I trained a nomic-bert from scratch. When I am loading the model for mask filling inference, the model return different values for mask each time the model is loaded.
The code to do mask filling is -
`from transformers import AutoModelForMaskedLM, AutoConfig, AutoTokenizer, pipeline
from model_latest.modeling_hf_nomic_bert import NomicBertForPreTraining
tokenizer = AutoTokenizer.from_pretrained('some-tokenizer') #
model_path = 'custom-model'
config = AutoConfig.from_pretrained(model_path, trust_remote_code=True) # the config needs to be passed in
model = NomicBertForPreTraining.from_pretrained(model_path,config=config, trust_remote_code=True)
To use this model directly for masked language modeling
classifier = pipeline('fill-mask', model=model, tokenizer=tokenizer,device="cpu")
print(classifier("I [mask] playing football."))`
Is the model loading causing some random initialization? Any fix for this?

I'm not super familiar with the pipeline code but a few things I'd check if the model in eval mode/dropout turned off when using the pipeline. Otherwise can dive into what's going on. It shouldn't be using any random init for weights but you can double check this by loading two models and seeing if they have the same weights

even with custom inference code, the same behaviour is seen -
`def mask_filling_inference(sentence, model_path=None, top_k=5):
config = AutoConfig.from_pretrained(model_path, trust_remote_code=True) # the config needs to be passed in
model = AutoModelForMaskedLM.from_pretrained(model_path,config=config,trust_remote_code=True)

# Tokenize input
tokenized_input = tokenizer.tokenize(sentence)
masked_index = tokenized_input.index('<mask>')  # assuming only one [MASK] token

# Convert tokens to tensor
indexed_tokens = tokenizer.convert_tokens_to_ids(tokenized_input)
tokens_tensor = torch.tensor([indexed_tokens])

 # Create attention mask
attention_mask = torch.ones_like(tokens_tensor)
attention_mask[tokens_tensor == tokenizer.mask_token_id] = 0

# Predict
with torch.no_grad():
    outputs = model(tokens_tensor, attention_mask=attention_mask)
    predictions = outputs[0]

# Get top predictions for the masked token
_, predicted_indexes = torch.topk(predictions[0, masked_index], top_k)
predicted_tokens = tokenizer.convert_ids_to_tokens(predicted_indexes.tolist())

return predicted_tokens

Example usage
sentence = "I want to  a car."
predicted_tokens = mask_filling_inference(sentence,model_path)
print("Top predictions for the masked token:", predicted_tokens)`

I think you need to set model.eval otherwise any non-deterministic layers (like dropout) will cause different answers
import torch
from transformers import AutoConfig, AutoModelForMaskedLM, AutoTokenizer

model_path = "nomic-ai/nomic-bert-2048"
config = AutoConfig.from_pretrained(model_path, trust_remote_code=True) # the config needs to be passed in
model = AutoModelForMaskedLM.from_pretrained(model_path,config=config,trust_remote_code=True)
model.eval()

tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)

sentence = "quick brown fox jumps over the lazy [MASK]"

# Tokenize input
tokenized_input = tokenizer.tokenize(sentence)
masked_index = tokenized_input.index('[MASK]')  # assuming only one [MASK] token

# Convert tokens to tensor
indexed_tokens = tokenizer.convert_tokens_to_ids(tokenized_input)
tokens_tensor = torch.tensor([indexed_tokens])

 # Create attention mask
attention_mask = torch.ones_like(tokens_tensor)
attention_mask[tokens_tensor == tokenizer.mask_token_id] = 0

# Predict
with torch.no_grad():
    outputs = model(tokens_tensor, attention_mask=attention_mask)
    predictions = outputs[0]

# Get top predictions for the masked token
_, predicted_indexes = torch.topk(predictions[0, masked_index], 5)
predicted_tokens = tokenizer.convert_ids_to_tokens(predicted_indexes.tolist())
print(predicted_tokens)


AdminLTE 2 - 11 Error 200's /vendor/.......

Running latest version of Laravel-AdminLTE (using AdminLTE 2 in config)
Latest version of Laravel 6
The pages look ok but in the console there are 11 Error 200's every time a page is loaded.
[Wed Nov 13 15:56:37 2019] <IP_ADDRESS>:52026 [200]: /vendor/bootstrap/dist/css/bootstrap.min.css [Wed Nov 13 15:56:37 2019] <IP_ADDRESS>:52029 [200]: /vendor/font-awesome/css/all.min.css [Wed Nov 13 15:56:37 2019] <IP_ADDRESS>:52030 [200]: /vendor/Ionicons/css/ionicons.min.css [Wed Nov 13 15:56:37 2019] <IP_ADDRESS>:52031 [200]: /vendor/adminlte/dist/css/AdminLTE.min.css [Wed Nov 13 15:56:37 2019] <IP_ADDRESS>:52032 [200]: /vendor/adminlte/dist/css/skins/skin-purple.min.css [Wed Nov 13 15:56:37 2019] <IP_ADDRESS>:52033 [200]: /vendor/jquery/dist/jquery.min.js [Wed Nov 13 15:56:37 2019] <IP_ADDRESS>:52034 [200]: /vendor/jquery/dist/jquery.slimscroll.min.js [Wed Nov 13 15:56:37 2019] <IP_ADDRESS>:52035 [200]: /vendor/bootstrap/dist/js/bootstrap.min.js [Wed Nov 13 15:56:37 2019] <IP_ADDRESS>:52036 [200]: /vendor/adminlte/dist/js/adminlte.min.js [Wed Nov 13 15:56:37 2019] <IP_ADDRESS>:52037 [200]: /vendor/font-awesome/webfonts/fa-solid-900.woff2 [Wed Nov 13 15:56:37 2019] <IP_ADDRESS>:52038 [200]: /vendor/font-awesome/webfonts/fa-regular-400.woff2
Have done vendor:publish and select Laravel-AdminLTE and that doesn't resolve it.
The files do not exist in the installation so the error is correct. But I'm not sure what is suppose to be generating those files as it doesn't seem to be working.
I'm fairly new to Laravel and this is the first time using Laravel-AdminLTE.

The status 200 is not a error status, it's says that everything is ok.

Oh. lol. ok my bad then. Everything is fine then :D


[3.x] Bugfix - Fix tween.tell() returning 0 at end of tween

Currently, in godot 3.x, tween.tell() returns 0.0 on the final iteration of the tween. This means that on that iteration, there is actually no way for the client to know how much time is left on the tween.
eg)
func _ready():
tween.interpolate_property(self, "tweened_property", 0.0, 1.0, 1.0, Tween.TRANS_LINEAR, Tween.EASE_IN, 1.0)
	tween.interpolate_property(self, "tweened_property", 0.0, 1.0, 3.0, Tween.TRANS_LINEAR, Tween.EASE_IN, 0.5)
	tween.start()

func _process(delta: float) -> void:
	if tween.is_active():
		print("tween.tell %f tween.runtime %f" % [tween.tell(), tween.get_runtime()])

will output
tween.tell 1.983497 tween.runtime 2.000000
tween.tell 1.999786 tween.runtime 2.000000
tween.tell 0 tween.runtime 2.000000

This implies that tell() returning 0 could either mean the tween hasnt ran its first iteration yet or its ran its final iteration. tell() also never returns the actual final runtime. This is a bug because it means tween.tell() requires very careful and special handling by the user to actually determine what tell() == 0 means.
This PR fixes this by simply not setting elapsed to 0 when the tween is finished. Elapsed is clamped to the final runtime of the tween earlier in this same function.
This has been tested and there are no bugs introduced into repeating tweens or non-repeating tweens or delays, as all the internal code for those scenarios uses 'tween.finished' to determine the state of the tween.
Tween is different on master and doesn't have this bug, so there is no corresponding PR to master.

Thanks!


Added OpenTelemetry Metrics API bridge and instrumentation

What does this PR do?
Adds an API Bridge for the OpenTelemetry Metrics API, which delegates to a shaded, agent-provided SDK.
Checklist

[x] This is an enhancement of existing features, or a new feature in existing plugins

[ ] I have updated CHANGELOG.asciidoc
[x] I have added tests that prove my fix is effective or that my feature works
[ ] Added an API method or config option? Document in which version this will be introduced
[ ] I have made corresponding changes to the documentation

:broken_heart: Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-02-08T15:08:57.339+0000

Duration: 19 min 22 sec

Test stats :test_tube:

Test
Results

Failed
0

Passed
1346

Skipped
3

Total
1349

Steps errors
Expand to view the steps failures

Shell Script

Took 2 min 3 sec . View more details here
Description: #!/bin/bash                      set -euxo pipefail                      ./mvnw compile javadoc:javadoc

:green_heart: Flaky test report
Tests succeeded.
:robot: GitHub comments
Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

run benchmark tests : Run the benchmark tests.

run jdk compatibility tests : Run the JDK Compatibility tests.

run integration tests : Run the Agent Integration tests.

run end-to-end tests : Run the APM-ITs.

run windows tests : Build & tests on windows.

run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)


StackOverflowError when use adapter.setDisplayHeadersAtStartUp(true) or adapter.showAllHeaders();

hi@davideas ,so sorry to disturb you,i try to make a sticky grid header,it can show items well when without adapter.setDisplayHeadersAtStartUp(true) or adapter.showAllHeaders(), but if i set one, it crashed,  Caused by: java.lang.StackOverflowError: stack size 8MB
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6335)
here is my code, wish you'r help
public class TestActivity2 extends BaseActivity {
private static final String TAG = "TestActivity2";
HashMap<String, HeaderItem> headerMap = new HashMap<>();

private List<GridItem> dataList = new ArrayList<>();

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_test2);

    RecyclerView rcv = (RecyclerView) findViewById(R.id.rcv);
    rcv.setLayoutManager(new StaggeredGridLayoutManager(4, StaggeredGridLayoutManager.VERTICAL));

    getLocalData();

    FlexibleAdapter adapter = new FlexibleAdapter(dataList);
    rcv.setAdapter(adapter);
    rcv.addItemDecoration(new DividerItemDecoration(this, R.drawable.horizontal_line));
    adapter.setDisplayHeadersAtStartUp(true);
    adapter.showAllHeaders();
    adapter.enableStickyHeaders();
}

private void getLocalData() {
    AlbumUtil albumUtil = new AlbumUtil(this);
    ArrayList<AlbumFile> list = albumUtil.getLocalAlbumFiles();
    Collections.sort(list);

    for (AlbumFile albumFile : list) {
        if (headerMap.containsKey(albumFile.dayDate)) {
            HeaderItem headerItem = headerMap.get(albumFile.dayDate);
            GridItem gridItem = new GridItem(albumFile, headerItem);

            dataList.add(gridItem);
        } else {
            HeaderItem headerItem = new HeaderItem(albumFile.dayDate, false);
            headerMap.put(albumFile.dayDate, headerItem);

            GridItem gridItem = new GridItem(albumFile, headerItem);

            dataList.add(gridItem);
        }
    }
}

class HeaderItem extends AbstractFlexibleItem<HeaderItem.HeaderViewHolder> implements IHeader<HeaderItem.HeaderViewHolder> {
    String title;
    boolean isShowSelect;

    public HeaderItem(String title, boolean isShowSelect) {
        this.title = title;
        this.isShowSelect = isShowSelect;
    }

    public String getTitle() {
        return title;
    }

    public void setTitle(String title) {
        this.title = title;
    }

    public boolean isShowSelect() {
        return isShowSelect;
    }

    public void setShowSelect(boolean showSelect) {
        isShowSelect = showSelect;
    }

    @Override
    public boolean equals(Object inObject) {
        if (inObject instanceof HeaderItem) {
            HeaderItem inItem = (HeaderItem) inObject;
            return this.getTitle().equals(inItem.getTitle());
        }
        return false;
    }

    @Override
    public int getLayoutRes() {
        return R.layout.album_sticky_date_header;
    }

    @Override
    public HeaderViewHolder createViewHolder(FlexibleAdapter adapter, LayoutInflater inflater, ViewGroup parent) {
        return new HeaderViewHolder(inflater.inflate(getLayoutRes(), parent, false), adapter);
    }

    @Override
    public void bindViewHolder(FlexibleAdapter adapter, HeaderViewHolder holder, int position, List payloads) {
        if (payloads.size() > 0) {

        } else {
            holder.dateTV.setText(title);
        }
    }

    class HeaderViewHolder extends FlexibleViewHolder {
        TextView dateTV;
        TextView selectTV;

        public HeaderViewHolder(View view, FlexibleAdapter adapter) {
            super(view, adapter);
            dateTV = (TextView) view.findViewById(R.id.date_tv);
            selectTV = (TextView) view.findViewById(R.id.select_all_tv);
            selectTV.setOnClickListener(new View.OnClickListener() {
                @Override
                public void onClick(View v) {
                    v.setSelected(!v.isSelected());
                    showShortMessage(v.isSelected() ? "selectAll" : "unselectAll");
                }
            });
        }
    }
}

private class GridItem extends AbstractFlexibleItem<GridItem.GridViewHolder> implements ISectionable<GridItem.GridViewHolder, HeaderItem> {

    private AlbumFile albumFile;
    private HeaderItem headerItem;

    public GridItem(AlbumFile albumFile, HeaderItem headerItem) {
        this.albumFile = albumFile;
        this.headerItem = headerItem;
    }

    public AlbumFile getAlbumFile() {
        return albumFile;
    }

    public void setAlbumFile(AlbumFile albumFile) {
        this.albumFile = albumFile;
    }

    @Override
    public boolean equals(Object inObject) {
        if (inObject instanceof GridItem) {
            GridItem inItem = (GridItem) inObject;
            return getAlbumFile() == inItem.getAlbumFile();
        }
        return false;
    }

    @Override
    public int getLayoutRes() {
        return R.layout.album_grid_item;
    }

    @Override
    public GridViewHolder createViewHolder(FlexibleAdapter adapter, LayoutInflater inflater, ViewGroup parent) {
        return new GridViewHolder(inflater.inflate(getLayoutRes(), parent, false), adapter);
    }

    @Override
    public void bindViewHolder(FlexibleAdapter adapter, GridViewHolder holder, int position, List payloads) {
        AlbumItem albumItem = (AlbumItem) holder.itemView;
        albumItem.setAlbumFile(getAlbumFile());
    }

    @Override
    public HeaderItem getHeader() {
        return headerItem;
    }

    @Override
    public void setHeader(HeaderItem header) {
        this.headerItem = header;
    }

    public class GridViewHolder extends FlexibleViewHolder {

        public GridViewHolder(View view, FlexibleAdapter adapter) {
            super(view, adapter);

        }
    }
}

}

Hi @54binge, you don't have to call showAllHeaders() because that method is already called in setDisplayHeadersAtStartUp().
Also for the HeaderViewHolder, you need to pass true to the super constructor, to avoid some crashing issues when you will scroll.

Hi @davideas ,thanks for your reply.
In fact, the application crashed immediately when setDisplayHeadersAtStartUp() is called,there is nothing to do with the scrolling.
And I can't find where to pass "true" in the HeaderViewHolder constructor.
looking forward to your next reply!

@54binge, indeed scrolling is a different thing, but if you don't find the new parameter means that you are using an old version, please use beta7, and you actually need it.
(I've updated a little the Wiki for beta7).
Regarding the main problem, I think it's the double call, but I need to check better, in the meantime, please remove showAllHeaders().

Another important thing, you have to provide a copy of dataList. At the moment that is necessary.

@54binge, I've tried your code, I removed the creation of dataList because I don't have the objects. But I used my own list and everything works fine, just the header is duplicated because duplicated is the call...
Can you please enable logs and paste them here, and also can you try to run it debug mode?

@davideas
it works fine use handler,i need research it more.
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_test2);
    RecyclerView rcv = (RecyclerView) findViewById(R.id.rcv);
    rcv.setLayoutManager(new StaggeredGridLayoutManager(4, StaggeredGridLayoutManager.VERTICAL));

    getLocalData();

    final FlexibleAdapter adapter = new FlexibleAdapter(dataList);
    rcv.setAdapter(adapter);
    rcv.addItemDecoration(new DividerItemDecoration(this, R.drawable.horizontal_line));
    baseHandler.post(new Runnable() {
        @Override
        public void run() {
            adapter.showAllHeaders();
        }
    });

    adapter.enableStickyHeaders();

}

Well, it is a workaround, it would be interesting to see your logs, maybe there's a problem somewhere else.

@davideas
there is logs,i don't use handler it still crash
06-25 14:13:45.283 7711-7711/com.example E/JavaBinder: !!! FAILED BINDER TRANSACTION !!!
06-25 14:13:45.296 7711-7711/com.example E/AndroidRuntime: Error reporting WTF
android.os.TransactionTooLargeException
at android.os.BinderProxy.transactNative(Native Method)
at android.os.BinderProxy.transact(Binder.java:504)
at android.app.ActivityManagerProxy.handleApplicationWtf(ActivityManagerNative.java:4310)
at com.android.internal.os.RuntimeInit.wtf(RuntimeInit.java:344)
at android.util.Log$1.onTerribleFailure(Log.java:104)
at android.util.Log.wtf(Log.java:294)
at android.util.Log.wtf(Log.java:275)
at com.mob.tools.log.MobUncaughtExceptionHandler.uncaughtException(Unknown Source)
at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:693)
at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:690)
06-25 14:13:45.511 7711-7733/com.example I/art: Background partial concurrent mark sweep GC freed 82701(6MB) AllocSpace objects, 16(46MB) LOS objects, 5% free, 74MB/78MB, paused 1.591ms total 213.745ms
06-25 14:13:45.511 7711-7711/com.example I/art: WaitForGcToComplete blocked for 53.259ms for cause Alloc
06-25 14:13:45.694 7711-7728/com.example W/art: Suspending all threads took: 48.071ms
06-25 14:13:45.704 7711-7733/com.example W/art: Suspending all threads took: 9.962ms
06-25 14:13:45.722 7711-7733/com.example I/art: Background partial concurrent mark sweep GC freed 59992(4MB) AllocSpace objects, 2(4MB) LOS objects, 4% free, 85MB/89MB, paused 11.469ms total 177.634ms
06-25 14:13:45.988 7711-7733/com.example I/art: Background partial concurrent mark sweep GC freed 73002(5MB) AllocSpace objects, 2(15MB) LOS objects, 3% free, 96MB/100MB, paused 1.371ms total 163.060ms
06-25 14:13:45.988 7711-7711/com.example I/art: WaitForGcToComplete blocked for 37.299ms for cause Alloc
06-25 14:13:46.103 7711-7711/com.example E/AndroidRuntime: Original WTF:
android.util.Log$TerribleFailure: stack size 8MB
at android.util.Log.wtf(Log.java:291)
at android.util.Log.wtf(Log.java:275)
at com.mob.tools.log.MobUncaughtExceptionHandler.uncaughtException(Unknown Source)
at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:693)
at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:690)
Caused by: java.lang.StackOverflowError: stack size 8MB
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6335)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.resetResolvedLayoutDirection(ViewGroup.java:6336)
at android.view.ViewGroup.
06-25 14:13:46.255 7711-7728/com.example W/art: Suspending all threads took: 108.457ms
06-25 14:13:46.265 7711-7733/com.example W/art: Suspending all threads took: 10.055ms
06-25 14:13:46.285 7711-7733/com.example I/art: Background partial concurrent mark sweep GC freed 16605(1330KB) AllocSpace objects, 15(66MB) LOS objects, 7% free, 49MB/53MB, paused 11.360ms total 168.989ms
06-25 14:13:46.555 7711-7733/com.example I/art: Background partial concurrent mark sweep GC freed 78396(5MB) AllocSpace objects, 3(10MB) LOS objects, 6% free, 58MB/62MB, paused 1.562ms total 160.541ms
06-25 14:13:46.555 7711-7732/com.example I/art: WaitForGcToComplete blocked for 111.069ms for cause HeapTrim
06-25 14:13:46.660 7711-7728/com.example W/art: Suspending all threads took: 12.462ms
06-25 14:13:46.670 7711-7733/com.example W/art: Suspending all threads took: 9.850ms
06-25 14:13:46.679 7711-7733/com.example I/art: Background sticky concurrent mark sweep GC freed 56102(4MB) AllocSpace objects, 0(0B) LOS objects, 0% free, 72MB/72MB, paused 11.124ms total 38.032ms
06-25 14:13:46.855 7711-7711/com.example E/JavaBinder: !!! FAILED BINDER TRANSACTION !!!
06-25 14:13:46.859 7711-7733/com.example I/art: Background partial concurrent mark sweep GC freed 66782(5MB) AllocSpace objects, 2(13MB) LOS objects, 4% free, 84MB/88MB, paused 1.981ms total 169.145ms
06-25 14:13:47.163 7711-7733/com.example I/art: Background partial concurrent mark sweep GC freed 149207(9MB) AllocSpace objects, 5(31MB) LOS objects, 5% free, 70MB/74MB, paused 5.763ms total 195.674ms
06-25 14:13:47.383 7711-7733/com.example I/art: Background partial concurrent mark sweep GC freed 147382(9MB) AllocSpace objects, 2(5MB) LOS objects, 4% free, 79MB/83MB, paused 1.361ms total 182.026ms
06-25 14:13:47.630 7711-7807/com.example I/Process: Sending signal. PID: 7711 SIG: 9

@54binge, ok I've seen the logs and deleted the message, not useful.
I need that you enable the logs of the Adapter and send that to me. Also put some logs in the various methods of your Activity so I can know in which point the execution is.
Thanks.

@davideas
when i see the logs,i also think it not useful,even if i call FlexibleAdapter.enableLogs(true);
just now, i put the method setAdapter() after method adapter.setDisplayHeadersAtStartUp(true),
all become normal and works very well.
thank you for these days!

Cool, so the problem is the order of the set?
I wonder why the demo App doesn't crash too, can you put here the logs of when it crash?
Because I've rewriting the Wiki pages just now and I need to be sure that the call setDisplayHeadersAtStartUp() can be after or must be before setting the Adapter to the RV.
Thanks.

I pasted all logs when it crashed,when it became normal,i tried the order of method setDisplayHeadersAtStartUp(true),in fact,it is not the reason,i think it is better to  call it after setAdapter(),maybe it is the time to restart android studio or reboot my computer :)

I understood that could be something related with the setContentView and view hierarchy even before the Adapter since you don't see any logs from it when you enableLogs().


Why ground truth(gt_sel) is participate here?

https://github.com/xxj96/SQLNet/blob/master/sqlnet/model/modules/aggregator_predict.py#L41
What if test/predict mode which don't have the ground truth?
Thank you @xxj96

Hi, the ground truth of selection is used so that the attention of aggregator is conditioned on the selected column. I haven't realized the test mode which doesn't require the ground truth yet because the breakdown accuracy on aggregator requires the ground truth of selection. So even in test mode, the model will use the ground truth of selection. It will not affect the overall accuracy, but yes, I do need to realize the mode which first predicts the selected column and then uses it to predict aggregator.

Thank you


[Bug]: Send contact list not scrollable

Describe the bug
We should ensure the send modal contact list is always scrollable, especially when use user has ±>10 contacts. Currently, on first attempt scrolling is not possible. Closing and reopening the modal sometimes resolves this. We should make scrolling work consistently.
Reproduce

Go to '...'
Click on '....'
Scroll down to '....'
See error

Screenshots / Recording
No response
Operating system
iOS
Bitkit version
No response
Log output
No response

Not able to reproduce this so far. If anyone can share a screen recording that would probably help.
Here's a seed with 10 contacts:
pilot siren inch visa mosquito ahead orchard offer nurse marine rifle nut
https://github.com/synonymdev/bitkit/assets/8538369/47bb4b08-0d10-49e7-afa7-838ff84b593f


Update with 0.2.3 release

Endians
Sponges
Core version bump
Core xof
Hash version bump
Hash sha3
MACs version bump
MACs hmac-sha3
MACs kmac

Blocked until https://github.com/KotlinCrypto/MACs/issues/19 and https://github.com/KotlinCrypto/MACs/issues/20 are complete an 0.2.2 gets pushed out

db0c8e77e442aa7174398a11749bc6a49b98e7fe


examples/points_over_time.py is broken after #3730

🐛 Bug
git bisect points to #3730 as the commit that broke the example points_over_time. Something is off about the face_color_mode and how it's set/reset. It doesn't seem to stick, so as the layer gets updated the face colors do not change. @andy-sweet , any quick idea on where to look for the issue?
To Reproduce
Run the example and move the dims slider; nothing changes, whereas it should update point colors.

Expected behavior

Environment

Please copy and paste the information at napari info option in help menubar here:

Any other relevant information:

Additional context

I knew that the silence after that change was too good to be true :D As noted in the description, PR #3730 breaks a couple of behaviors.
The important one here is that we cannot rely on in-place mutation of Layer.properties, which in this example we do with pts_layer.properties['value'] = np.asarray(values_table[:, column]). I think it will work sometimes (in the case that Series.to_numpy() does not copy so that the returned numpy array has the same base as the dataframe memory block, but maybe not in this case.
The fix that seems to work for me is to use pts_layer.features['value'] = np.asarray(values_table[:, column]) instead, which seems good to me.
As a part of fixing this issue, we should probably take a look at the other relevant examples to check they are not broken either.
@brisvag : I'm happy to assign myself and fix this issue, probably by the end of today. But if you feel like doing it, please assign yourself and I can review.

This issue also makes me want to prioritize deprecation of properties, but as we found out, those PRs can get complex. I can probably write a PR to deprecate the properties getter/setter, maybe the init parameter, but not the state field, so I will probably try to do that this week.

pts_layer.properties['value'] = np.asarray(values_table[:, column])

The fix that seems to work for me is to use pts_layer.features['value'] = np.asarray(values_table[:, column]) instead, which seems good to me.

I see; yeah this is a bit annoying, I feel like inplace editing is prevalent and important. We definitely need proper deprecation out, which is gonna be tricky indeed :)
I'll let you take care of this for now, since you're more familiar with the quirks, but please let me know if you'd like help!


File system

Add a representation of a filesystem to ogit, as preparatio of running FileSystem on Graph
File -> locatedIn Directory
Directory -> locatedIn Directory
File -> corresponds Attachment

Frank Otto voiced some concerns about the implementation of the file-system in graph. Will close until this clears up


Remediate deallocated VMs

We have a cluster where the subscription was suspended and the VMs automatically deallocated.  Now the subscription is good again.
We need to detect this and power the VMs on.  Another step like the pull secret fix on upgrade.

Notes: need to check subscription is not suspended, then list vms with power status, then power on deallocated vms, then wait for API server with timeout if power status changed.

And unit test.

Notes:

We probably not only need this in admin update (similar step as pull secret fix, etc), but also when RP receives a suscription update where subscription changes from state suspended to registered. Needs to be handled similarly to Deleted subscription state (ref), but instead of deleting clusters - rescuing them
We might have issues with certs and probably something else. Need to look into it. For example see this blog post

See the steps to recover the cluster from expired certs. TLDR: it requires access to master hosts.

Reduced the priority to medium.
We've got https://github.com/Azure/ARO-RP/pull/706 merged. It will help us to remediate clusters with stopped VMs where certs are still valid.


Update 2160p CFs (Radarr/Sonarr) and SDR CF (Radarr)

Pull request
Purpose
Change 2160p CFs to oneliners, update SDR with negated HDR formats
Approach
2160p now properly only matches HDR formats, and SDR correctly excludes HDR formats
Open Questions and Pre-Merge TODOs
Check all boxes as they are completed

[x] Use github checklists. When solved, check the box and explain the answer.

Learning
If you're adding a new Custom Format make sure you follow the Radarr/Sonarr Custom Format (JSON) Guidelines.
Requirements
Check all boxes as they are completed

[x] These changes meet the standards for contributing.
[x] I have read the code of conduct.

Nope - I think we only have the SDR CF for Radarr to fix an SQP issue.


Make sure that the security role can see the team groups used by RBAC

Describe the issue
We wish to have security automation coverage over the RBAC connected policies that are assigned when RBAC users are provisioned to SEA items.
Which Sprint Priority is this issue related to?
Goal 2

Note: 'Milestone' is a ZenHub term that we use synonomously with 'Sprint'.

The 'Milestone' overview/description (created in ZenHub) should clearly list Sprint Priorities/Goals. This section should indicate which of the priorities/goals this issue is related to.
Additional context

KeyCloak is the current RBAC mechanism
An IDIR or GitHub user attempts a login at KeyCloak and fails (but loads the user meta into KeyCloaks records)
A Cloud Pathfinder Team member (future automation) connect that user meta link with the team accounts that have been created with automation

Definition of done

Meet with Ryan B and confirm his CloudCustodian security role access cannot list the team groups (confirm the issue)
Make sure security role can see the groups (eg. yrt234ty-developer -> yrt234ty-dev) (fix the issue)

Place this ticket top priority for the tech team so that Ryan is not blocked at start of sprint

unclear on exactly what is needed for this ticket, but will get clarity from Ryan B

@sheaphillips Appreciate the clarification today. Is there any current mechanism outside of keycloak to see the groups being assigned?

@bluemel-gov for tenant team users and workload accounts, the only way users are assigned to groups is within KeyCloak.  This is currently done manually by our team on behalf of tenants  but eventually we hope to delegate to teams to self-serve.  It may also be relevant to note that in this context "group" is a KeyCloak group which has no relation to an AWS IAM Group, which your controls are likely referencing.


Changing staging alt domain to be under notification.cdssandbox.xyz

Summary | Résumé
Modifying staging alt domain to be underneath the notification.cdssandbox.xyz domain. This is required because the ACM validation needs to be able to create route53 records on the alt domain, which I have not mapped in staging (it is shared with other apps). This will not have any impact on prod, since we will not be migrating the DNS.
Test instructions | Instructions pour tester la modification

Terragrunt plan/apply should work
ACM Validation should work in staging

seems simple enough! Will this make staging.alpha.notification.cdssandbox.xyz exist? or do we have to do something else?

Yes! Because we now own notification.cdssandbox.xyz, terraform can (and will) automatically add/update the DNS records.


Fix: Introspector in iOS 14 not working for ScrollView (#55)

Looks like in iOS 14 the sibling view is the ScrollView, as opposed to a ViewHost containing a UIScrollView. Added a check to see if the sibling itself is a UIScrollView.
The iOS SwiftUI ScrollView types is as follows:
<SwiftUI.HostingScrollView: 0x7fd3ae835200; baseClass = UIScrollView; frame = (84.5 0; 151.5 568); anchorPoint = (0, 0); clipsToBounds = YES; gestureRecognizers = <NSArray: 0x600001de89f0>; layer = <CALayer: 0x60000134e060>; contentOffset: {0, -20}; contentSize: {151.5, 1746.5}; adjustedContentInset: {20, 0, 0, 0}>
Tested using a simple example app:

Been using this fix in production on a ScrollView and List. It works in both iOS 13 and 14.

Been using this fix in production on a ScrollView and List. It works in both iOS 13 and 14.

Are you able to change the sectionFooterHeight for a List/TableVIew? The code gets called but the height never changes until the list is updated (such as adding or deleting something in the list).

I haven't tried (though I can in a bit). I am using it successfully to hide
list separators and disable scrolling in Lists and ScrollViews.
On Sun, Sep 27, 2020 at 11:21 AM Gian Frangiamore<EMAIL_ADDRESS>wrote:

Been using this fix in production on a ScrollView and List. It works in
both iOS 13 and 14.
Are you able to change the sectionFooterHeight for a List/TableVIew? The
code gets called but the height never changes until the list is updated
(such as adding or deleting something in the list).
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/siteline/SwiftUI-Introspect/pull/56#issuecomment-699648663,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJVI7LR3MKUQO7AG7GEDR7TSH5J7TANCNFSM4RZC4DAQ
.

I haven't tried (though I can in a bit). I am using it successfully to hide list separators and disable scrolling in Lists and ScrollViews.

Does this work for you?
struct ContentView: View {
    var body: some View {
        List {
            ForEach(1..<10) { num in
                Text("\(num)")
            }
        }
        .introspectTableView(customize: { tableView in
            tableView.separatorStyle = .none
        })
        .listStyle(InsetGroupedListStyle())

    }
}

I've checked out the specific commit. The code above doesn't remove the separator inset. Am I doing something wrong?

Looks like in iOS 14 SwiftUI is handling the separator on its own and not respecting tableView.separatorStyle. In your code snippet tableView.separatorStyle is actually already .none before it gets assigned (meaning SwiftUI sets it to .none and simply handles the separators itself). A better library for your use case might be:
https://github.com/SchmidtyApps/SwiftUIListSeparator

Looks like in iOS 14 SwiftUI is handling the separator on its own and not respecting tableView.separatorStyle. In your code snippet tableView.separatorStyle is actually already .none before it gets assigned (meaning SwiftUI sets it to .none and simply handles the separators itself). A better library for your use case might be:
https://github.com/SchmidtyApps/SwiftUIListSeparator

I was actually just using the line separator as an example.
What I wanted to use this library for was to reduce the height of the section footer/header.
I've noticed that if you call tableView.reloadData() after changing the height, then it works, but it takes a second to kick in, so when you launch the app the height is set to whatever the default is, and then you see the rows suddenly jump and height becomes smaller.
Have you got any suggestions?

Hi, this fix works for my use cases... Waiting for this to be pulled to master...

Somehow this doesn't work for me on device/simulator. But works in SwiftUI Previews 🤷♂🙈
Device iOS 14.0.1
Simulator iOS 14.1 (Preview should be the same 0_o)

@lucastimeless Thanks a ton for looking into this! I adapted your code and made 2 different methods instead. It's now fixed in faster, and we can close both this PR and #55 . See 6ddd059
Thanks again and sorry for the delay!


fix lsq

fix scale calc in lsq

close this PR. The bug fixed in https://github.com/megvii-research/Sparsebit/pull/20


Alternative promise steps handling and simplify some string operations

Some sql written using ` symbol to simplify.
Some promises are now handled differently to handle errors.

It looks like app.js in tsugi-node-sample crashes with this version of the library code.  You can do a quick functionality test of app.js using tsugi-python-test - it hits the server and sends good and bad launches.

Never mind - it was not your fault.  Your error checking was catching an SQL error that my code ignored.  So I just committed a fix to fix the SQL and it happily re-passes the functionality test.


[Python 3.8] Invalid syntax for Python 3.8

Description of the bug
In deepeval\test_run\cache.py, there is an invalid type hint for Python 3.8, see : https://github.com/confident-ai/deepeval/blob/main/deepeval/test_run/cache.py#L65
Indeed, generic type hints in standard collections is a feature available starting from Python 3.9, which makes the package unsuitable for Python 3.8 projects.
How to fix the bug
Replace Optional[dict[str, CachedTestCase]] with Optional[Dict[str, CachedTestCase]]

Just saw that the package does not officially support Python 3.8, although apart from the bug I described, it seems to work fine on Python 3.8.

@PaulWassermann  Thanks! Just cut a new release for this.  Come join our discord btw: https://discord.com/invite/a3K9c8GRGt for any other suggestions!


Update to node20

Thanks, just released it: https://github.com/WyriHaximus/github-action-composer-php-versions-in-range/releases/tag/v1.13.0


Icons in maven central repo

Hi! I got the same trouble described as this link
Icons and tags on mvnrepository.com
And noticed that u have solved this problem. So could u explain how to fix it in detail?
Thanks!

+1

Sorry, I just noticed this issue now. I wonder if you problem has been solved? I don’t know how to solve it, but I did the following things:

Create an organization and set the organization’s avatar to the logo you want to display in mvnrepository.com;
Create a repository  in this organization and publish it to maven central;
Send an email to<EMAIL_ADDRESS> and tell them to set a logo for your maven group and give them a link to the logo.
Hope it helps you!

Hello @iofairy,
How long did it take for you to receive a response from<EMAIL_ADDRESS>Also @boxbox-0427 and @EthanWang779,
Were you able to resolve the issue ? If so, did you use a different method than what @iofairy mentioned?
Thank you.


Preloading of the MTCNN model is very slow.

Hello,
I'm using VueJS. When you open a Modal, these codes are working, which I created from your sample code. The value of "console.time" is quite high.
Code:
console.time("fullFaceDescriptions");
vm.yuz_degisken_objesi.fullFaceDescriptions = (await faceapi.allFacesMtcnn(vm.yuz_degisken_objesi.videoEl, vm.yuz_degisken_objesi.mtcnnParams))
    .map(fd => fd.forSize(vm.yuz_degisken_objesi.hw.width, vm.yuz_degisken_objesi.hw.height));
console.timeEnd("fullFaceDescriptions");
console.log("*");

fullFaceDescriptions: 5500.3720703125ms
*
fullFaceDescriptions:<PHONE_NUMBER>59375ms
*
fullFaceDescriptions:<PHONE_NUMBER>28125ms
*
fullFaceDescriptions:<PHONE_NUMBER>71875ms
*
fullFaceDescriptions:<PHONE_NUMBER>6875ms
*
fullFaceDescriptions:<PHONE_NUMBER>4375ms
*
How can I make it more stable?

More stable or faster? Faster inference time of the mtcnn detector can be achieved by increasing the minFaceSize parameter for example, which will skip the detection of smaller faces.

Preloading? What can be done for this?
fullFaceDescriptions: 5500.3720703125ms

Nothing really that I know of, seems like the first time a tensor of a certain shape is uploaded to the GPU there is an initial delay, which they refer to as "warmup" in the mobilenet example. The MTCNN uploads tensors of N + 2 different shapes (N in stage 1, can be adjusted by the maxNumScales forward parameter and 1 each in stages 2 and 3). This unfortunately causes the first forward pass of MTCNN to be much longer on certain machines.
I am not sure where this inital delay comes from (maybe due to allocating textures of certain sizes on the GPU, just a wild guess, I am not that familar with WebGL for GPGPU). If it concerns you, maybe you could ask at tfjs for help.
PS: I would also be interested in the answer to that.

Here's a comment from a tfjs-example:

Warmup the model. This isn't necessary, but makes the first prediction faster.

This warmup works on computeFaceDescriptor, but not works on mtcnn or allFacesMtcnn on my project. Maybe you can make a predict first after the model is loaded.

I've been fighting with this for a week or so now... I'm currently looking into web workers as a way to do an initial prediction without tying up the main UI thread so that my actual predictions later on are lightning-fast. I'll let you know if I make any progress.

Alas... no such luck. I think I'd have to understand tfjs much better than I do to convert it in such a way to get it to work in a web worker. I have gotten it so far as to attempt to load the models, but the registration of the backend failed. Some Googling tells me that webgl isn't supported in Web Workers, but cpu backend also fails to register.

@justadudewhohacks Awesome! I look forward to this.

@justadudewhohacks hi,
its take 0.5 second to load image? how do i faster this process?

async function createBbtFaceMatcher() {
await faceDetectionNet.loadFromDisk(path.join(__dirname, '../weights'))
await faceapi.nets.faceLandmark68Net.loadFromDisk(path.join(__dirname, '../weights'))
await faceapi.nets.faceRecognitionNet.loadFromDisk(path.join(__dirname, '../weights'))
const labeledFaceDescriptors = await Promise.all(classes.map(
async className => {
let descriptors: any = [];
let uri = getFaceImageUri(className, 1);
const img = await canvas.loadImage(uri);
if (img) {
let descriptor = await faceapi.computeFaceDescriptor(img);
if (descriptor) {
descriptors.push(descriptor);
}
}
return new faceapi.LabeledFaceDescriptors(
className,
descriptors
)
}
))
return new faceapi.FaceMatcher(await labeledFaceDescriptors)
}


Unable to use s3.read_parquet_table() to read from Glue tables whose name is the prefix of another table.

Describe the bug
When using the function s3.read_parquet_table(), Data Wrangler will first lookup the table URI and use this as the path argument for s3.read_parquet(). Relevant code snippet:
res: Dict[str, Any] = client_glue.get_table(**args)
try:
    path: str = res["Table"]["StorageDescriptor"]["Location"]

The issue with this piece of code is that the storage descriptor returned by Glue does not necessarily contain a slash ("/") as suffix, meaning that when it is passed to s3.read_parquet(), it will expand everything including paths matching the table name. The issue becomes apparent when you have tables matching this format in Glue:
database.prefix
database.prefix_suffix

Attempting to read from database.prefix will also match files under database.prefix_suffix, resulting in an error of the form: InvalidArgumentValue: Object s3://<bucket>/<database>/<prefix_suffix>/<file>.parquet is not under the root path (s3://<bucket>/<database>/<prefix>/).
A simple fix could simply be to add the "/" suffix to the path when it's not present:
path: str = res["Table"]["StorageDescriptor"]["Location"]
path: str = path if path.endswith("/") else f"{path}/"

To Reproduce
Tested with Python 3.7.9 and Wranger 2.6.0, installed via pip.
Steps to reproduce the behavior:

Create a Glue database "test_database" with two tables "test_table" and "test_table_extra".
Write some data to these tables so they are populated.
Use Data Wrangler to read data from "test_table", e.g.

import awswrangler as wr
df = wr.s3.read_parquet_table(database='test_database', table='test_table')

You will get an error indicating files belonging to test_table_extra are not under the root path of test_table.

Covered in 2.7.0 release


az pipelines run --parameters is throwing unrecognized arguments error

Describe the bug
running az pipelines run with --paramters option throws unrecognized arguments: --parameters error.
To Reproduce
az pipelines run --id 3 --parameters platform=ios --output table
result in
unrecognized arguments: --parameters platform=ios
Expected behavior
It should run the pipeline of given id with parameter platform set to ios
Environment summary

Install Method: homebrew
CLI version: (result of az --version)
azure-cli                         2.34.1

core                              2.34.1
telemetry                          1.0.6

Extensions:
azure-devops                      0.22.0
interactive                        0.4.5

Dependencies:
msal                              1.16.0
azure-mgmt-resource               20.0.0

Python location '/opt/homebrew/Cellar/azure-cli/2.34.1/libexec/bin/python'
Extensions directory '/Users/*****/.azure/cliextensions'

Python (Darwin) 3.10.2 (main, Feb  2 2022, 05:51:25) [Clang 13.0.0 (clang-13<IP_ADDRESS>)]

OS version: macOS 12.2.1
Shell Type: zsh, fish
Additional context

Document website https://docs.microsoft.com/en-us/cli/azure/pipelines?view=azure-cli-latest#az-pipelines-run list this option but it simply doesn't work, at least on macOS version.

route to CXP team

Adding Service Attention label for the service team to look into this.

The --parameters option is implemented in azure-devops 0.23.0.
Please upgrade your azure-devops extension.
https://github.com/Azure/azure-devops-cli-extension/releases/tag/20220131.1

Pipeline client 6.0 added to support template parameters for pipelines runs (https://github.com/Azure/azure-devops-cli-extension/pull/1211)

@kaito-ms
I've been using ubuntu version (via docker) meanwhile.
It looks like it worked with ubuntu version because it installed v0.25.0 when my local version was still at v0.22.0.
Now that I updated devops extension to the latest (v0.25.0), --parameters works now.
Thanks.
I do see that table output is somehow broken with the new version for the pipeline.
I'm guessing I should create an issue on devops extension repo.


Postgres init script fails on MacOS: Unix-domain socket path is too long (maximum 103 bytes)

Hi all, I'm back :)
After the last MacOS Update (14.3) I'm facing again the "Unix-domain socket path is too long" issue on my Mac.
The issue happens here:
https://github.com/cachix/devenv/blob/ad0ae333b210e31237e1fc4a7ddab71a01785add/src/modules/services/postgres.nix#L81-L125
Essentially the postgres service starts the DB (only during the first setup) using a socket server, ignoring all the user's configurations.
Indeed the path is set to:
PGHOST=$(mktemp -d "$DEVENV_STATE/pg-init-XXXXXX")

In my case, it means:
[postgres	] 2024-01-24 09:59:14.557 GMT [41621] LOG:  Unix-domain socket path "/Users/bigluck/b/<11 chars dir name>/<32 chars dir name>/service/.devenv/state/pg-init-fv97zJ/.s.PGSQL.35431" is too long (maximum 103 bytes)

I've already moved the project root into /Users/bigluck/b/, but:

I can not change my username :D
I can not move the service to a different location

This is the current workaround I have in place:
{ pkgs, lib, ... }:

# See full reference at https://devenv.sh/reference/options/
{
  dotenv.disableHint = true;

  # https://devenv.sh/reference/options/#entershell
  enterShell = "
    mkdir -p /tmp/my-project-name
  ";

  # https://devenv.sh/packages/
  packages = [ ... ];

  # https://devenv.sh/languages/
  languages.python = { ... };

  # https://devenv.sh/reference/options/#processimplementation
  process.implementation = "process-compose";

  # https://devenv.sh/services/
  services.postgres = {
    enable = true;
    package = pkgs.postgresql_15;
    listen_addresses = "<IP_ADDRESS>";
    port = 35431;
    initialDatabases = [{ name = "MY_DB_NAME"; } ];
    initialScript = ''
      CREATE USER MY_DB_USER WITH PASSWORD 'MY_PASSWORD';

      GRANT ALL PRIVILEGES ON DATABASE ${toString bpln_service.db_name} TO postgres;
      ALTER DATABASE MY_DB_NAME OWNER TO MY_DB_USER;
    '';
  };

  # https://devenv.sh/processes/
  processes = {
    postgres = {
      process-compose = {
        environment = [
          "DEVENV_STATE=/tmp/my-project-name"
        ];
      };
    };
  };
}

Essentially I'm overwriting the DEVENV_STATE ENV only for the postgres service; it works, but it's a workaround.

Most likely a duplicate of #540

Outsider here, but I've come across this issue as well - think for organisation, we should close this issue, and move resolution discussions to #540.
A solution to me is to use the OS-native tmp for sockets but keep PGDATA in the usual 'state' directory.
(I've tried disabling unix sockets, but it's always forced as enabled)

Following @shymega suggestion, I’m closing this issue in favor of #540


Please update XTERM to the latest

The build script does not update xterm.
You have here 2 choices:
ora you update xterm and build it every time or you include xterm as a subgit :D

It's updated manually, done on master.


Migrate to CircleCI 2.0

CircleCI 1.0 will be sunset August 31. Instead of opening 13 separate issues, I thought this might be an OK place to track the work. List generated with https://github.com/CircleCI-Public/find-circle-yml/.
Core repositories

[ ] https://github.com/openzipkin/brave/blob/master/circle.yml
[ ] https://github.com/openzipkin/zipkin-api/blob/master/circle.yml
[ ] https://github.com/openzipkin/zipkin-go-opentracing/blob/master/circle.yml
[ ] https://github.com/openzipkin/sleuth-webmvc-example/blob/master/circle.yml
[ ] https://github.com/openzipkin/b3-propagation/blob/master/circle.yml
[ ] https://github.com/openzipkin/zipkin-go/blob/master/circle.yml
[ ] https://github.com/openzipkin/zipkin-aws/blob/master/circle.yml
[ ] https://github.com/openzipkin/zipkin-gcp/blob/master/circle.yml
[ ] https://github.com/openzipkin/brave-karaf/blob/master/circle.yml
[ ] https://github.com/openzipkin/zipkin-azure/blob/master/circle.yml
[ ] https://github.com/openzipkin/brave-cassandra/blob/master/circle.yml

Attic

https://github.com/openzipkin-attic/zipkin-ui/blob/master/circle.yml (archived)
https://github.com/openzipkin-attic/zipkin-sparkstreaming/blob/master/circle.yml (archived)

Contrib
No relevant repos under openzipkin-contrib

thanks for opening this!

It uhh... seems like we're done? I'll be optimistic and close this.


chore(deps): update common-dev-assets digest to 282b3d6

This PR contains the following updates:

Package
Update
Change

common-dev-assets
digest
d8a1f23 -> 282b3d6

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, click this checkbox.

This PR has been generated by Renovate Bot.

:tada: This PR is included in version 1.0.1 :tada:
The release is available on:

GitHub release
v1.0.1

Your semantic-release bot :package::rocket:


Blocks cannot be right-clicked on

Expected Behavior
You can right click on blocks to get various actions.
Screenshot from Scratch

Actual Behavior
You cannot right click on blocks, and instead get an error in the console.

Steps to Reproduce

Drag out any block
Right click on it

Operating System and Browser
Windows 11 on MS Edge 128

The right click on the block is working as expected.


[TEST] Add cross-channel upgrade test feature

backport of https://github.com/apache/camel-k/commit/d0f64e066332c1bd4535bcffb6b09bd9f98ce11c

@squakez can you review?


Multiple file view at the same time

Hello!
Great editor, thank you for all the hard work!
Question - is it possible to open multiple files side by side? Like here in notepad:

Very often it is a really necessary option for comforting copying from one file to another.
Thank you in advance, George.

Hi George,
Thanks for the feedback.
To implement your suggestion as it's presented by the screenshot, I had to plan it from the very beginning, alas.
Though rarely, this mode is necessary indeed. In such cases I just open another editor for a parallel file and place it aside by alited, optionally making it "staying topmost".
When I need to compare a file's versions, I just call Differences (of alited's tools). This is customized to kdiff3 by default, still I prefer meld.
Nonetheless, I could make a "File / Detach" menu item - when a currently edited file is detached to another editor window placed aside by the current editing frame. This parallel window would be above the main window of alited and serve as an "errand boy" only, i.e. no alited's functionality except editing.

Hello, the last proposal from your message sounds great, often the second window is needed just for comfort copying of code and/or just look at it.
Thank you in advance!

Hello!
Just tested the new feature - it works well, thank you!
The only question - how to enable syntax highlighting in this window?

It's only a draft, skeleton sort of. Of course, there will be syntax highlighting and saved geometries of first 8 windows. Also, the feature will be available in the context menus of tab bar, file viewer and Projects/Files.

Great, looking forward to final version!

I think this is done, though without docs yet.

All works great, thank you very much!

Hello, found an exception here - if I detach new file and that file has not been saved, the detach doesn't work

And the other one - while using WSL with detach, it doesn't allow me to edit main window files, because cursor is always on focus in detached window

George,
I've changed something in this feature, hopefully, to better side.
As for your remarks:

for the new file, alited will offer to save it first, then detach it; imo the detachment of "no name" text looks weird;
the currently edited text will be refocused after detachments.

Got it, thank you!


The documentation tab must not exist in the France connect plugin manager

Steps to Reproduce the Problem

As an administrator go to the plugin manager
Click on France Connect Plugin

Actual Behavior
Seen :The documentation tab appears as you can see on the attached screenshot

Resolved, see :<PHONE_NUMBER>ed3bb0915ef39164e716d0bfc86f2c

Validated in FranceConnect Plugin 1.9 (#40)


Fix detrend transforms to handle integer timestamp

Before submitting (must do checklist)

[x] Did you read the contribution guide?
[x] Did you update the docs? We use Numpy format for all the methods and classes.
[x] Did you write any new necessary tests?
[x] Did you update the CHANGELOG?

Proposed Changes
Look #155.
Closing issues
Closes #155.

Error during creation of Pull request, I used wrong branches.


Adding porch-pirate scanner

Adding porch-pirate scanner which will show exposed postman collections
Added vim
Updated crud utils

This needs to be refactored according to the new extended assets concept.


Select filters closing immediately on Firefox mobile (Android)

Describe the bug
On Android / Firefox Mobile (latest versions), select filters (i.e. those on the expense page) are randomly closing themselves immediately after opening. I'm not reproducing on desktop.
https://user-images.githubusercontent.com/1556356/189304541-9528ec8b-9344-4502-9a01-f6715024b0ba.mp4
Context
Could be related to https://github.com/opencollective/opencollective/issues/5599

I'm using the latest Firefox (currently 105.1.0), Android 13. It sometimes appears only after playing with the filters by toggling them quickly.

Could be the same as https://github.com/JedWatson/react-select/issues/3886.


Output of installed software is swallowed - How to handle?

I added cask "fastlane" to my Brewfile.
brew bundle ran as expected and installed it.
But it didn't work, as the manual steps that are normally output if needed, weren't done:
...
==> Couldn't detect shell config file (bash - ~/.bash_profile)
==> Please add the following line to your bash profile:
==>
==> export PATH="$HOME/.fastlane/bin:$PATH"
...

Similar situation with brew "nvm". Normally it outputs these instructions:
Virtuals-Mac:~ foo$ brew reinstall nvm
==> Reinstalling nvm
==> Downloading https://github.com/creationix/nvm/archive/v0.33.6.tar.gz
Already downloaded: /Users/foo/Library/Caches/Homebrew/nvm-0.33.6.tar.gz
==> Caveats
Please note that upstream has asked us to make explicit managing
nvm via Homebrew is unsupported by them and you should check any
problems against the standard nvm install method prior to reporting.

You should create NVM's working directory if it doesn't exist:

  mkdir ~/.nvm

Add the following to ~/.bash_profile or your desired shell
configuration file:

  export NVM_DIR="$HOME/.nvm"
  . "/usr/local/opt/nvm/nvm.sh"

You can set $NVM_DIR to any location, but leaving it unchanged from
/usr/local/opt/nvm will destroy any nvm-installed Node installations
upon upgrade/reinstall.

Type `nvm help` for further information.

Bash completion has been installed to:
  /usr/local/etc/bash_completion.d
==> Summary
🍺  /usr/local/Cellar/nvm/0.33.6: 7 files, 134.9KB, built in 4 seconds
Virtuals-Mac:~ foo$

How should something like this be handled?
homebrew-bundle swallowing all output is certainly normally useful, but what about cases like this?

homebrew-bundle swallowing all output is certainly normally useful, but what about cases like this?

This is either-or, unfortunatley.. brew bundle --verbose may print what you desire and if it doesn't: we'd welcome a PR to do so.

--verbose does output more, will check if this includes these.
Specific reason this is not documented or can I submit a PR?
Might there be a way for brew bundle to "collect" these "Caveats" outputs and echo them when all installation is finished?

It's documented in brew bundle --help.

Might there be a way for brew bundle to "collect" these "Caveats" outputs and echo them when all installation is finished?

Nah, that's going to be really messy.

It's documented in brew bundle --help.

Which itself is not documented in the README ;)

Nah, that's going to be really messy.

Thought so. I'll try to write down the "manual steps" somehow.
(Sorry for putting your software through this noob treatment by the way - usually not a Mac user, which is why homebrew-bundle could also be so super useful to get a consistent setup for iOS development)

Which itself is not documented in the README ;)

I think that's pretty redundant given pretty much all Unix software supports --help.

(Sorry for putting your software through this noob treatment by the way - usually not a Mac user, which is why homebrew-bundle could also be so super useful to get a consistent setup for iOS development)

No worries, it's always good to get people with a different take to try this stuff out. It may seem like I'm discounting your problems but it's something I'll remember if more people have similar problems.


subgrid icons are not displayed if metadata

Apparently, the component or characters that are added into the first column of a subgrid row (parentRowCollapsedComponent and parentRowExpandedComponent, default or custom) are not displayed when a metadata object (from columnMetadata) exists for this first column.
This is a problem if we want to customize the displayed name of this column, for example.
Thanks!

+1. I was able to reproduce this error too

+1. I just started using Griddle and this issue popped up for me.  Any work arounds available?

Another big +1.
Before finding this bug report, I had wasted several hours tweaking my code, unable to understand why the subgrid icon wasn't appearing in the table.  Simply removing my columnMetadata props object "fixed" it.   Only now my column headings are showing the underlying field names, which is just ugly.
This needs a fix or workaround urgently.

For the moment, I've worked around this by using the CSS ::before selector to add some plus and minus characters, like so:
.griddle-container .parent-row td:first-child::before {
content: "+ ";
}
.griddle-container .parent-row.expanded td:first-child::before {
content: "- ";
}

+1. This is still happening on version 0.3.0


Automation on Mail App on Windows

Hello. I would like to do some automation on the Windows built-in Mail application. I would like to input some text into "SearchBox", "SubjectTextBox" and "Body" respecetively (all of them are auto_id) . However, when I successfully connect to the mail app and go to the write email page, I cannot access the "Body" part only with an error "RuntimeWarning: The window has not been focused due to COMError: (-2146233079, None, (None, None, None, 0, None))
'COMError: {}'.format(exc), RuntimeWarning)".
The following is some of my code.
new_app.connect(title_re=".Inbox.")
dlg = new_app.top_window()
message = dlg.child_window(auto_id = "Body", control_type = "Edit")
message.wait("exists ready active")
message.set_focus()
message.type_keys(MESSAGE, with_spaces = True)

try setting backend='uia' when you connect:
new_app.connect(title_re=".Inbox.", backend='uia')

I am also seeing this for Edge. Same problem?
  File "C:\Users\spull\Projects\OpenAdapt\openadapt\window\_windows.py", line 174, in get_properties
    properties = element.get_properties()
  File "C:\Users\spull\Projects\OpenAdapt\lib\site-packages\pywinauto\base_wrapper.py", line 577, in get_properties
    props[propname] = getattr(self, propname)()
  File "C:\Users\spull\Projects\OpenAdapt\lib\site-packages\pywinauto\controls\uia_controls.py", line 224, in texts
    super(ComboBoxWrapper, self).collapse()
  File "C:\Users\spull\Projects\OpenAdapt\lib\site-packages\pywinauto\controls\uiawrapper.py", line 539, in collapse
    self.iface_expand_collapse.Collapse()
_ctypes.COMError: (-2146233079, None, (None, None, None, 0, None))


Improve status  of resources in AssetGroup

@derberg commented on Tue Jun 18 2019
Description

Status of the ClusterAssetGroup and AssetGroup should change whenever the resource is updated and underlying Asset cr is processed again
Check if the same is not the case with Asset resources

Reasons
At the moment status of resources is not properly updated. If you for example remove one source, or do some other changes and at the end you know that Asset resource is processed, the status should switch to Pending and then again to Ready or Failed.

I think it is OK now, so if any change is still needed, please reopen this issue.


Add Media Strategy content

Repurposes the top level "News" link to make a new page for Jupyter Media Strategy
Adds a sibling page with blog submission guidelines.

I agree with @ivanov that we should have a link to the blog (with title "Blog") in the nav. A link called "News" should be to a page full of news about Jupyter, not a page about media strategy. The latter might be better titled "Social", but we should have a list of our accounts more prominently displayed — at present, they're at the bottom of the media strategy page.

FYI the linkcheck and lighthouse were both fixed in master branch by #749

Thanks @ivanov and @JasonWeill, I believe I've addressed on all your concerns, except for how and when to ask for a direct social media post or boost, which we're going to add at our next meeting.

Kicking CI.

Hmm, it looks like netlify isn't getting triggered because we changed branch names.

netlify isn't getting triggered because we changed branch names.

@blink1073 is this a problem that we need to address before merging this?

is this a problem that we need to address before merging this?

I don't know who set this up, @Carreau?

I don't know who set this up...

I'm not even sure how to see that it's failed somehow - where do you see an impact of netlify not getting triggered (or getting triggered, heh, for that matter)

where do you see an impact of netlify not getting triggered

It used to show up as one of the PR checks, e.g. in https://github.com/jupyter/jupyter.github.io/pull/750

I'm guessing this will just resolve itself on the next PR. Looks like it's set up as a webhook right now

I don't know who set this up, @Carreau?

I can't seem to login to netlify anymore:

Authentication Error
Authenticating failed due to the following error: undefined method `developement?' for Netlify::Env:Module

I will retry later.

I've updated the submission guidelines, including a section for requesting a Mastodon post.

I can login to netlify again and seem to have access to the deployment for this repo, I've switched the default branch from master to main, hopefully that will fix things. I can't add team members without moving to a paying plan, but if you let me know I guess I can destroy the deploymeent and we can create a netlify under a Jupyter Org.

I've switched the default branch from master to main, hopefully that will fix things. I can't add team members without moving to a paying plan, but if you let me know I guess I can destroy the deploymeent and we can create a netlify under a Jupyter Org.

It seems to be working now, thanks!  No worries, worst case we can create a new account if we decide we need to change something.

Overall feedback: 9 top level navigation tabs is too many. I think 5 or 6 would be more intuitive/simple.
However, I also feel we are past due in publishing the JMS submission guidelines and don't want to block publishing on this new tab. Leaving this comment here for consideration for future iterations of this website.
Governance could be a paragraph and a link in the About Us page and we could find a way to combine Get Involved, News, and Social. That alone would take us down to 6 nav bar items which should increase ease of use for the website overall.


Bluetooth: Fix various deadlock issues when running low on buffers

The first and third patches are for fixing potential deadlocks when running low on buffers. The Number of Completed Packets patch (the first one) affects pretty much all HCI drivers, including the native controller one. The last patch only affects the H:4 HCI driver. The middle patch (second one) is to make it valid to use some of the upper bits of buf->flags as extended user data.
Fixes #16864

Added DNM label until it's verified that this actually fixes the issue and nobody is unhappy with the added HCI driver API for allocating event buffers.

@jukkar I've added a net_buf patch here to make it officially ok to use some bits of buf->flags as extended user data, could you take a look?
@carlescufi @joerchan since you use the H:4 driver on the nRF91 it'd be good if you also check at least the changes I'm doing to that driver.

Through my tests, the latest patches work well on my system.

Add support to iterate the RX queue and discard (discardable) packets
in case the RX buffer pool is empty. This will eliminate potential
deadlock scenarios.

"Make less likely", perhaps. "Eliminate", unlikely.

@carlescufi @Vudentz @joerchan this PR has been left hanging around - it has an informal approval from the person who reported the original issue, as well as one formal approval, however nothing from actual Bluetooth maintainers yet.

I've read through the original issue and then through the code. I see that the new patch for number of completed packets addresses the issue with unblocking the TX thread that might be waiting on those. But what is the need for BT_H4_DISCARD_RX_WAIT? What is the deadlock that we are solving there, given that, if there are discardable packets, those will be processed and will free items in the rx queue?

what is the need for BT_H4_DISCARD_RX_WAIT? What is the deadlock that we are solving there, given that, if there are discardable packets, those will be processed and will free items in the rx queue?

@carlescufi IIRC I explained this on Slack already, but I'll repeat here:
There may be multiple pending HCI packets that the controller wants to send to the host. At the same time the host might be stuck waiting for something like a Number of Completed Packets event. If that event is the next one in the queue for the controller to send to the host then all is fine (as guaranteed by the other patch in this PR), but if there are other, non-discardable, packets before it then we have to be able to receive those packets in order to eventually receive the Number of Completed Packets event. By digging into the host RX queue we're able to free up buffers to receive the pending packets from the controller.

what is the need for BT_H4_DISCARD_RX_WAIT?

@carlescufi seems I failed to address this specific question: the purpose of this initial wait is that digging into the RX queue is expensive. So we give the other threads in the system a chance to process and free up buffers before commencing our own expensive free-up operation.

what is the need for BT_H4_DISCARD_RX_WAIT? What is the deadlock that we are solving there, given that, if there are discardable packets, those will be processed and will free items in the rx queue?

@carlescufi IIRC I explained this on Slack already, but I'll repeat here:
There may be multiple pending HCI packets that the controller wants to send to the host. At the same time the host might be stuck waiting for something like a Number of Completed Packets event. If that event is the next one in the queue for the controller to send to the host then all is fine (as guaranteed by the other patch in this PR), but if there are other, non-discardable, packets before it then we have to be able to receive those packets in order to eventually receive the Number of Completed Packets event. By digging into the host RX queue we're able to free up buffers to receive the pending packets from the controller.

Right, but where is the actual deadlock? why is a pending Number of Completed Packets, preventing us from processing other, unrelated events? Since the memory pools are separate for events and data anyway, that's the bit I don't understand. Sorry if you already explained this and I missed it.

Right, but where is the actual deadlock? why is a pending Number of Completed Packets, preventing us from processing other, unrelated events? Since the memory pools are separate for events and data anyway, that's the bit I don't understand. Sorry if you already explained this and I missed it.

@carlescufi
That event releases the TX thread to send more packets to the controller. Until then TX buffers will sit around waiting in the TX queue. If some thread tries to send more data they'll be blocked on trying to allocate a TX buffer.

Right, but where is the actual deadlock? why is a pending Number of Completed Packets, preventing us from processing other, unrelated events? Since the memory pools are separate for events and data anyway, that's the bit I don't understand. Sorry if you already explained this and I missed it.

@carlescufi
That event releases the TX thread to send more packets to the controller. Until then TX buffers will sit around waiting in the TX queue. If some thread tries to send more data they'll be blocked on trying to allocate a TX buffer.

Yes, but how is that a deadlock? The currently allocated event buffers will be processed and freed up by the RX thread regardless of what is happening to the TX thread?

Yes, but how is that a deadlock? The currently allocated event buffers will be processed and freed up by the RX thread regardless of what is happening to the TX thread?

They don't have to be processed and freed up by the RX thread. E.g. both in Mesh and L2CAP cases they may be passed to the system workqueue for processing. net_buf's in general have the whole ref/unref feature and the ability to be passed through FIFOs to other contexts. So there's no guarantee that once you've done a bt_recv() call that the buffer will have gotten freed up.
Disclaimer: In all honesty I'm not 100% clear on what the precise deadlock is in #16864 or the various theoretical ways things can deadlock, but I'm fairly confident that we have to try to avoid the H:4 driver from halting the flow of packets from the controller so that we eventually get to process any packets that may unblock threads that are waiting for them.

. So there's no guarantee that once you've done a bt_recv() call that the buffer will have gotten freed up.

Thanks for explaining. I agree that this is safe and reasonable.

@carlescufi Mesh stack sends two ACL packets in Rx threads synchronously. This action will block Rx threads before numCompleted is received. Because of the continuous reporting of advertising data packets, the Rx buffers is exhausted. Therefore, Tx threads cannot continue to report numCompleted events, resulting in deadlocks.
This is the call stack for Rx threads:
#0 __swap (key=0, key@entry=16965509) at /home/ubuntu/zephyr/arch/arm/core/swap.c:68
#1 z_swap_irqlock (key=16965509) at /home/ubuntu/zephyr/kernel/include/kswap.h:128
#2 z_swap (key=..., lock=0x2000346c <k_sys_work_q>) at /home/ubuntu/zephyr/kernel/include/kswap.h:145
#3 z_pend_curr (lock=lock@entry=0x2000346c <k_sys_work_q>, key=<EMAIL_ADDRESS>wait_q=wait_q@entry=0x0, timeout=timeout@entry=-1) at /home/ubuntu/zephyr/kernel/sched.c:448
#4 z_impl_k_sem_take (sem=sem@entry=0x0, timeout=timeout@entry=-1) at /home/ubuntu/zephyr/kernel/sem.c:160
#5 k_sem_take (timeout=-1, sem=0x0) at /home/ubuntu/zephyr/samples/bluetooth/mesh_test_platform/build/xxxx/zephyr/include/generated/syscalls/kernel.h:103
#6 bt_att_send (conn=0x0, conn@entry=0x20000ae4 , buf=buf@entry=0x200083a8 <net_buf_acl_tx_pool+48>, cb=cb@entry=0x0) at /home/ubuntu/zephyr/subsys/bluetooth/host/att.c:2222
#7 gatt_notify (conn=conn@entry=0x20000ae4 , handle=, data=0x0, data@entry=0x20004980 <rx_thread_stack+3120>, len=len@entry=28, cb=cb@entry=0x0) at /home/ubuntu/zephyr/subsys/bluetooth/host/gatt.c:1167
#8 bt_gatt_notify_cb (conn=conn@entry=0x20000ae4 , attr=0x20009320 <proxy_attrs+80>, attr@entry=0x2000930c <proxy_attrs+60>, data=0x20004980 <rx_thread_stack+3120>, len=, func=func@entry=0x0) at /home/ubuntu/zephyr/subsys/bluetooth/host/gatt.c:1379
#9 bt_gatt_notify (len=, data=, attr=0x2000930c <proxy_attrs+60>, conn=0x20000ae4 ) at /home/ubuntu/zephyr/include/bluetooth/gatt.h:759
#10 proxy_send (conn=conn@entry=0x20000ae4 , data=, len=) at /home/ubuntu/zephyr/subsys/bluetooth/host/mesh/proxy.c:893
#11 proxy_segment_and_send (conn=conn@entry=0x20000ae4 , type=type@entry=0 '\000', msg=msg@entry=0x20004974 <rx_thread_stack+3108>) at /home/ubuntu/zephyr/subsys/bluetooth/host/mesh/proxy.c:918
#12 bt_mesh_proxy_send (conn=0x20000ae4 , type=type@entry=0 '\000', msg=msg@entry=0x20004974 <rx_thread_stack+3108>) at /home/ubuntu/zephyr/subsys/bluetooth/host/mesh/proxy.c:955
#13 bt_mesh_proxy_relay (buf=buf@entry=0x200086b4 <net_buf_adv_buf_pool+56>, dst=) at /home/ubuntu/zephyr/subsys/bluetooth/host/mesh/proxy.c:878
#14 bt_mesh_net_send (tx=tx@entry=0x20004a64 <rx_thread_stack+3348>, buf=buf@entry=0x200086ac <net_buf_adv_buf_pool+48>, cb=0x1030210 <seg_sent_cb>, cb_data=cb_data@entry=0x200001d8 <seg_tx>) at /home/ubuntu/zephyr/subsys/bluetooth/host/mesh/net.c:885
#15 send_seg (net_tx=net_tx@entry=0x20004a64 <rx_thread_stack+3348>, sdu=sdu@entry=0x20004a90 <rx_thread_stack+3392>, cb=cb@entry=0x0, cb_data=cb_data@entry=0x0) at /home/ubuntu/zephyr/subsys/bluetooth/host/mesh/transport.c:411
#16 bt_mesh_trans_send (tx=tx@entry=0x20004a64 <rx_thread_stack+3348>, msg=msg@entry=0x20004a90 <rx_thread_stack+3392>, cb=cb@entry=0x0, cb_data=cb_data@entry=0x0) at /home/ubuntu/zephyr/subsys/bluetooth/host/mesh/transport.c:507
#17 model_send (model=model@entry=0x20008a6c <root_models>, tx=tx@entry=0x20004a64 <rx_thread_stack+3348>, implicit_bind=implicit_bind@entry=false, msg=msg@entry=0x20004a90 <rx_thread_stack+3392>, cb=cb@entry=0x0, cb_data=cb_data@entry=0x0) at /home/ubuntu/zephyr/subsys/bluetooth/host/mesh/access.c:638
#18 bt_mesh_model_send (model=model@entry=0x20008a6c <root_models>, ctx=ctx@entry=0x20004be8 <rx_thread_stack+3736>, msg=msg@entry=0x20004a90 <rx_thread_stack+3392>, cb=cb@entry=0x0, cb_data=cb_data@entry=0x0) at /home/ubuntu/zephyr/subsys/bluetooth/host/mesh/access.c:654
#19 dev_comp_data_get (model=0x20008a6c <root_models>, ctx=0x20004be8 <rx_thread_stack+3736>, buf=) at /home/ubuntu/zephyr/subsys/bluetooth/host/mesh/cfg_srv.c:198
#20 bt_mesh_model_recv (rx=rx@entry=0x20004be4 <rx_thread_stack+3732>, buf=buf@entry=0x20004b28 <rx_thread_stack+3544>) at /home/ubuntu/zephyr/subsys/bluetooth/host/mesh/access.c:579
#21 sdu_recv (rx=rx@entry=0x20004be4 <rx_thread_stack+3732>, seq=1, hdr=, aszmic=aszmic@entry=0 '\000', buf=buf@entry=0x20004bfc <rx_thread_stack+3756>) at /home/ubuntu/zephyr/subsys/bluetooth/host/mesh/transport.c:627
#22 trans_unseg (buf=buf@entry=0x20004bfc <rx_thread_stack+3756>, rx=rx@entry=0x20004be4 <rx_thread_stack+3732>, seq_auth=seq_auth@entry=0x20004bc0 <rx_thread_stack+3696>) at /home/ubuntu/zephyr/subsys/bluetooth/host/mesh/transport.c:898
#23 bt_mesh_trans_recv (buf=buf@entry=0x20004bfc <rx_thread_stack+3756>, rx=rx@entry=0x20004be4 <rx_thread_stack+3732>) at /home/ubuntu/zephyr/subsys/bluetooth/host/mesh/transport.c:1400
#24 bt_mesh_net_recv (data=data@entry=0x20009218 <clients+24>, rssi=rssi@entry=0 '\000', net_if=net_if@entry=BT_MESH_NET_IF_PROXY) at /home/ubuntu/zephyr/subsys/bluetooth/host/mesh/net.c:1324
#25 proxy_complete_pdu (client=client@entry=0x20009200 ) at /home/ubuntu/zephyr/subsys/bluetooth/host/mesh/proxy.c:405
#26 proxy_recv (conn=, attr=, buf=0x20008261 <net_buf_data_hci_rx_pool+109>, len=, offset=0, flags=2 '\002') at /home/ubuntu/zephyr/subsys/bluetooth/host/mesh/proxy.c:467
#27 write_cb (attr=0x200092f8 <proxy_attrs+40>, user_data=0x20004c98 <rx_thread_stack+3912>) at /home/ubuntu/zephyr/subsys/bluetooth/host/att.c:1225
#28 bt_gatt_foreach_attr (start_handle=start_handle@entry=25, end_handle=end_handle@entry=25, func=func@entry=0x10287f5 <write_cb>, user_data=user_data@entry=0x20004c98 <rx_thread_stack+3912>) at /home/ubuntu/zephyr/subsys/bluetooth/host/gatt.c:935
#29 att_write_rsp (conn=conn@entry=0x20000ae4 , req=req@entry=0 '\000', rsp=rsp@entry=0 '\000', handle=, offset=offset@entry=0, value=0x20008261 <net_buf_data_hci_rx_pool+109>, len=22 '\026') at /home/ubuntu/zephyr/subsys/bluetooth/host/att.c:1268
#30 att_write_cmd (att=, buf=0x200081ac <net_buf_hci_rx_pool+24>) at /home/ubuntu/zephyr/subsys/bluetooth/host/att.c:1500
#31 bt_att_recv (chan=0x20000c18 <bt_req_pool>, buf=0x200081ac <net_buf_hci_rx_pool+24>) at /home/ubuntu/zephyr/subsys/bluetooth/host/att.c:1947
#32 l2cap_chan_recv (chan=, buf=buf@entry=0x200081ac <net_buf_hci_rx_pool+24>) at /home/ubuntu/zephyr/subsys/bluetooth/host/l2cap.c:1573
#33 bt_l2cap_recv (conn=conn@entry=0x20000ae4 , buf=buf@entry=0x200081ac <net_buf_hci_rx_pool+24>) at /home/ubuntu/zephyr/subsys/bluetooth/host/l2cap.c:1606
#34 bt_conn_recv (conn=conn@entry=0x20000ae4 , buf=buf@entry=0x200081ac <net_buf_hci_rx_pool+24>, flags=flags@entry=2 '\002') at /home/ubuntu/zephyr/subsys/bluetooth/host/conn.c:1149
#35 hci_acl (buf=buf@entry=0x200081ac <net_buf_hci_rx_pool+24>) at /home/ubuntu/zephyr/subsys/bluetooth/host/hci_core.c:551
#36 hci_rx_thread () at /home/ubuntu/zephyr/subsys/bluetooth/host/hci_core.c:4683
#37 z_thread_entry (entry=0x1005b11 <hci_rx_thread>, p1=, p2=, p3=) at /home/ubuntu/zephyr/lib/os/thread_entry.c:29
#38 0xaaaaaaaa in ?? ())

hci_rx_thread () at /home/ubuntu/zephyr/subsys/bluetooth/host/hci_core.c:4683

@xiaoliang314 this is something I don't understand. In #16864 you said you're using the H4 driver, which enables CONFIG_BT_RECV_IS_RX_THREAD. This option in turn disables hci_rx_thread in hci_core.c, so I don't understand how you have that in your backtrace?

@jhedberg Sorry, I may not have told you more details. On our chip, we transplanted the Bluetooth controller. The HCI driver of our controller reference the H4 driver. But the actual H4 interface is not used.

The above comment brings up an important point though: if it's the bt_recv() call that's blocking there's still an issue since it's in the same thread that we try to free up buffers, i.e. the freeing would need to be moved to some independent thread. Since the problem seems to be fairly complex I'm thinking I'll move the less controversial patches to a separate PR that could be merged first (i.e. the new one-buffer pool and the net_buf flags usage)

Closing this PR - I'll open a new one for the num_completed pool and some patches to a different approach I'm planning to take. This also means that no change is needed to the net_buf API (the flags stuff) for now.


Add a warning about infinite Tween loops

I was actually aware of this in the original rewrite, but I assumed it isn't a common issue. But after running into it accidentally multiple times, it's probably worth mentioning in the docs. I don't think there is a way to prevent this issue.

Thanks!