Converting graph information from dgl.Graph back into Structure

Hi,
I am looking to extract the gradient with respect to atom positions from the band gap prediction model "MEGNet-MP-2019.4.1-BandGap-mfi".
By going through the wrapper into MEGNet.predict_structure function I can extract the gradients with respect to the edge attributes. Is there a way to convert this information into gradient with respect to atom positions in the structure (i.e. g.ndata["pos"] in the code below).
Below I provide the unpacked method to access the gradient within the predict_structure() function.
I can do the same within the forward method, but I am unclear which representation of the graph will provide me with the information I require for translation back to the structure.
Thank you for your help.
Code to reproduce:
import matgl
import torch

from mp_api.client import MPRester
from pymatgen.io.ase import AseAtomsAdaptor
from matgl.ext.pymatgen import Structure2Graph
from matgl.graph.compute import compute_pair_vector_and_distance

if __name__ == '__main__':
    # loading arbitrary structure from Materials Project
    with MPRester() as mpr:
        structure = mpr.get_structure_by_material_id("mp-1840", final=True)
    atoms_2 = AseAtomsAdaptor.get_atoms(structure)

    # selecting a band gap method
    state_feats = torch.tensor([3])

    band_gap_model_wrapper = matgl.load_model("MEGNet-MP-2019.4.1-BandGap-mfi")

    # unpacking forward method of MEGNet.predict_structure to access gradient information
    graph_converter = Structure2Graph(
        element_types=band_gap_model_wrapper.model.element_types,
        cutoff=band_gap_model_wrapper.model.cutoff,
    )
    g, state_feats_default = graph_converter.get_graph(structure)
    if state_feats is None:
        state_feats = torch.tensor(state_feats_default)
    bond_vec, bond_dist = compute_pair_vector_and_distance(g)
    g.edata["edge_attr"] = band_gap_model_wrapper.model.bond_expansion(bond_dist)

    # adding requires_grad argument edges to access gradients
    g.edata["edge_attr"].requires_grad_()

    model_output = band_gap_model_wrapper.model(g, g.edata["edge_attr"], g.ndata["node_type"], state_feats)

    gradient_wrt_edge_attr = torch.autograd.grad(
        model_output, g.edata["edge_attr"], create_graph=True, retain_graph=True,
    )

    model_output = model_output.detach()
    model_output_converted = band_gap_model_wrapper.transformer.inverse_transform(model_output)

    # asserting that unpacked method gives the same answer as the wrapper method provided
    band_gap_value = band_gap_model_wrapper.predict_structure(
        structure=structure,
        state_feats=state_feats
    )

    assert model_output_converted == band_gap_value

Hi, thank you for your quick reply!
I'm not sure if I understood you correctly, but my problem isn't coming from matching a prediction to a graph. I would like to use the gradient from the model (which is with respect to the graph) to inform a change in the structure itself. Is that possible?

Perhaps the title of my issue is misleading, let me elaborate on the problem briefly.
Currently the process to make the prediction is:

Pass structure in
Convert structure to graph
Make a prediction on the graph

What I would like to do is after step 3:
4. Extract gradient from model,
5. Convert gradient from the embedding representation into the gradient on each atom position.
I think this is similar to what is done to extract the forces when computing energy, except I would like to do it when making a prediction using "MEGNet-MP-2019.4.1-BandGap-mfi".

That's the same thing. In the PES models, both energy and forces (the gradient of the energies) are explicitly handled as part of the prediction. So you do have the actual structure gradient in this case.
But for property prediction models, you are predicting the band gap. You have the gradient of the band gap wrt to the graph. While you can "move" the graph in the direction of the gradient, that is no easy way to translate the modified graph back to a structure.
Note that the gradient of band gap wrt to graph is not a force per se. It is not like the PES model where the gradient is an actual force on each atom that you can use to move the atoms using a Verlet algorithm.

Hi @mwolinska. Thank you for your questions. @shyuep is right that we only return the gradient of bandgap with respect to graph attributes include nodes, edges and states for property models. If you really want to get the gradient of bandgap with respect to atomic positions. I would suggest you explicitly calculating the gradient of bond expansion (edge attributes) with respect to atomic position and then perform matrix multiplication with the gradient of band gap with respect to bond expansion (edge attributes).

Hi @kenko911, thank you for your suggestion! Do you think computing the gradient of bond expansion with respect to atomic position from your implementation would be fairly straightforward?

Based on the script you provided, I am confident that you can figure it out by your own ;). I will close this issue if no further question. Thanks.


radicle-org-node gets stuck, ignoring traffic, spinning 2 cpu cores

netstat shows unread packets accumulating
top shows 200% CPU usage
the last line in logs was:
Aug 19 19:18:34.619 INFO ureq::unit: sending request POST https://api.thegraph.com/subgraphs/name/radicle-dev/radicle-orgs
this is at commit 0a4a3954684daea7693d385a2f902f34f4ecf766
release build
Linux 4.18.0-305.3.1.el8.x86_64
CentOS Linux release 8.4.2105
The CPU usage may be unrelated as it does that at times it is responding to network traffic.
@kim thinks it may be due to https://github.com/radicle-dev/radicle-link/issues/734

Oh so this is a radicle org node. Would capturing logs not suggest that #5, radicle-dev/radicle-link#734 and this one are related?

Yeah, they are all the same issue AFAICT.

I'd like to upgrade the seed node to the latest radicle-link, as well as the org node, and see if this is still an issue :thinking:

I think it stopped leaking memory around the same time it began using a lot of CPU so I think #5 is fixed.
There's not much network traffic so I don't think it relates to https://github.com/radicle-dev/radicle-link/issues/734 either.

For the hangs (probably unrelated to CPU usage) In at least 3 of 3 cases, the last line in logs is:
INFO ureq::unit: sending request POST https://api.thegraph.com/subgraphs/name/radicle-dev/radicle-orgs
Maybe that request isn't spawning its own thread?

Dropping traffic to api.thegraph.com didn't cause hangs so maybe not.  Maybe that's just the part that wasn't hung so is why it kept showing up in logs when nothing else was.

For 30 minutes while the network receive queue was full, "INFO ureq::unit: sending request POST https://api.thegraph.com/subgraphs/name/radicle-dev/radicle-orgs" was the only line in the logs.

So what you're seeing now is:

Normal memory consumption
High CPU usage
Normal network usage

Is that right?

Closing this issue since we deprecated the org-node some time ago.


Add Getting started section to Readme

README Update
Please add a getting started section to the README so folks know how to start with this project.

Added Getting Started section in README file


Question: What does the upload function do exactly?

Upload what and where exactly? I'm sorry I'm a lil confused but I'm not even sure what's exactly executing that function. Is there a button I'm missing that's supposed to upload something?

When you drag and drop/paste file to the editor will fire the upload event


Apply Yaml pre-commit formatter

Folks have different YAML formatters in our code editors so we need a pre-commit hook to do the formatting for us!

Doesn't seem like there are any yaml formatting pre-commit hooks that are amenable to our current yaml formatting. I found three options:

yamlfix: Opinionated formatted that can't be configured.
yamlfmt: Formatterr than can be configured but I couldn't figure out options to recreate our current formatting.

@zaneselvans what yaml formatter are you using?

Not totally sure what the conflict is in the pre-config settings because the whole file was re-written.

https://prettier.io/ is also an option. It offers a pre-commit and a vscode extension but it can't be configured like the red hat extension. The Red Hat extension doesn't offer a pre-commit or command-line tool.

Ooo I like prettier:

It retains our - formatting for the settings files.
It's by far the most established YAML formatter I've found (42k stars).
Has a popular VSCode extension.

Widespread support seems like a strong reason to use it.
It seems like it's putting years on individual lines sometimes but not others and I'm not sure why.
I'm fine with whatever you choose! The most important thing is we can have something standardized and well supported.

Changes:

Add prettier pre-commit hook for consistent YAML formatting.


[api-docs] Fix details about thread members

A new commit was made into the api-docs repo. This issue was created automatically. Need to review if we made any change to the library for this commit.
https://github.com/discord/discord-api-docs/commit/90e08a09749175dbe8bd37fcf0f152843956f363

Done


EXPLAIN tweaking: non-objects

We need a way to track, say,

Go Rum
Total T

These are not official items in the game. They are in as of the latest release, but I need to search aggressively for them.

These two items are fixed, but I want to make sure nothing else skipped.
Conceptville now holds items til we know about them.

I've picked up stuff from XYZZY, etc., as I've gone through the code. So I think this is linked to #48 and they will both fall at once.


resubmit - (multi-account with TCert attributes) workaround

Re-submit for PR #1915 (previous merge failed due to un-detected go vet error and inclusion of a supposedly removed file)
Demonstrate a work-around option to support multi-account per user (each user with multiple account IDs) with the TCert attributes
Description
The current tcert-attribute based design allows user to select which attribute to include in the TCert. However, the attributes are pre-defined. So for any given attribute (e.g. account IDs), user can only choose to either include or not include in his TCert.
This strategy works if attribute values are static. However, account IDs are expected to be dynamic overtime in many use cases (for obfuscation purposes .... e.g. bitcoin addresses are dynamically created from pub keys generated/managed by wallets).
This commit demonstrate a work-around option to support multi-account option with what's there in the fabric today
Motivation and Context
Current attribute samples does not provide a blueprint on supporting dynamically supporting multi-account per user. This is a sample code that offers a work around to the issue described in #1844
Fixes #1844
How Has This Been Tested?
Example code does not alter the behavior of the fabric. The example code is tested with Unit test which is also part of this commit
Checklist:

[x] I have added a Signed-off-by
[x] Either no new documentation is required by this change, OR I added new documentation
[x] Either no new tests are required by this change, OR I added new tests
[x] I have run goimports, go vet, and golint. I have cleaned up all valid errors and warnings in code I have added or modified. These tools may generate false positives.  The goal is clean, consistent, and readable code.

Signed-off-by: Frank Lu<EMAIL_ADDRESS>

Confirmed that this resolved the previous issues. Thanks @frankyclu. LGTM.


Enable shapes layer (right click) > convert to labels

🚀 Feature

Enable shapes layer (right click) > convert to labels so that this function is accessible from GUI directly.
Pitch

Currently it is accessible through API to generate labels layer from shapes layer through napari.layers.Shapes.to_labels, but the option to access from the shapes layer (right click) > convert to labels is grayed out.
The additional input that might be required is the labels layer data shape. One possible implementation is after clicking on “convert to labels”, add a dialog box to select “match dimension to [available image layer]”, which might be the most common usage.
Alternatives

Continue to only support via API. Though it would be nice to have a tooltip when hovering on the grayed out option that shows the corresponding API.
Additional context
In general, the right click menu from different layers shows the same contents but grayed out with different parts.
This could be confusing as the grayed out options may be due to:
(1) Non-applicable actions
(2) Not sufficient selection/condition (i.e. Link Layers is only available if more than one layer is selected)
Ideally (1) should be removed so that users do not have to guess if those actions could be activated at all.

This would be great!

added in #3976


Add experience component

fix: #24
Added a Experiance component which takse data from the API

You have mispelled "Experience".

@randmon Thanks to notice that, I'll Updated that.

From build log

Keep both the Testimonials component and the experience component.

@thtauhid Updated

Please follow the image's instruction

@thtauhid Updated the changes that you requested for.

@thtauhid Updated the path of the Experience component and added responsiveness to that Experience section

Can you fix the type Experiance. It should be Experience
Marge the base branch into your feature branch and resolve any conflicts.

@thtauhid I am making another PR with all that changes.


The Twilio icon is blurry

We need to find a proper sharp icon and see which option looks best
Option 1

Option 2

Need to check suggestion from Nikhil about using this logo:
https://www.twilio.com/brand/elements/logos

Select option 1, this is downloaded as svg in the link  and convert to png.

Select option 1, this is downloaded as svg in the link  and convert to png.


Add firmwareUpdate capability to matter and zb profiles

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Please ignore the failure in the "Check for duplicate profiles" step below. This is due to an issue with the duplicate profile script not handling profiles without categories and not properly handling preferences. It is an issue with the script and not with this PR.
@greens do you know if there was a specific reason why firmwareUpdate was excluded from all matter drivers?

@ctowns I think we just didn't know it should have been included. As far as I knew, firmwareUpdate was just for Zigbee and no one told us otherwise.

Is it true that going forward this capability should be included in every matter profile without exceptions?

I believe so. The hubs running 0.47.x will emit firmwareUpdate capability with the current software version post device-onboarding for all matter devices. Additionally, if they undergo an update whether through our hub or any other fabric, the availableVersion attribute and update statuses will be reflected in the capability throughout the process.

feel free to merge @samadDotDev


feat(icons): added clock-check icon

related to https://github.com/lucide-icons/lucide/issues/2400
What is the purpose of this pull request?

[x] New Icon

Description
Added new clock-check icon.
Icon use case

These icons could be used for reflecting input validation, and to indicate delays and such for public transport
Alternative icon designs

Icon Design Checklist
Concept

[x] I have provided valid use cases for each icon.
[x] I have not added any a brand or logo icon.
[x] I have not used any hate symbols.
[x] I have not included any religious or political imagery.

Author, credits & license

[ ] The icons are solely my own creation.
[x] The icons were originally created in # by @colebemis
[x] I've based them on the following Lucide icons: clock
[ ] I've based them on the following design:

Naming

[x] I've read and followed the naming conventions
[x] I've named icons by what they are rather than their use case.
[x] I've provided meta JSON files in icons/[iconName].json.

Design

[x] I've read and followed the icon design guidelines
[x] I've made sure that the icons look sharp on low DPI displays.
[x] I've made sure that the icons look consistent with the icon set in size, optical volume and density.
[x] I've made sure that the icons are visually centered.
[x] I've correctly optimized all icons to three points of precision.

Before Submitting

[x] I've read the Contribution Guidelines.
[ ] I've checked if there was an existing PR that solves the same issue.

Great icon. Also the uses for this.
I think the check should be a bit bigger. On 100% scale it disappears a bit in the rounded shape of the clock, To me on the first view it didn't pop as a check icon.

Great icon. Also the uses for this.
I think the check should be a bit bigger. On 100% scale it disappears a bit in the rounded shape of the clock, To me on the first view it didn't pop as a check icon.

This is the standard check icon, we use everywhere.

Great icon. Also the uses for this.
I think the check should be a bit bigger. On 100% scale it disappears a bit in the rounded shape of the clock, To me on the first view it didn't pop as a check icon.

This is the standard check icon, we use everywhere.

Maybe we could give the new lucide/12 checkmark a spin?
Open lucide studio

Great icon. Also the uses for this.
I think the check should be a bit bigger. On 100% scale it disappears a bit in the rounded shape of the clock, To me on the first view it didn't pop as a check icon.

This is the standard check icon, we use everywhere.

Maybe we could give the new lucide/12 checkmark a spin?
Open lucide studio

Actually, I'd also be in favour of swapping some of our existing checkmarks with this, e.g.:

Out of these, copy-check is the only one that uses remains unchanged, and I think, square-check and circle-check look much more balanced and more consistent with copy-check.

I agree ✅ for changing those.

@karsa-mistmere Thanks for reply, this way better to the eye!

The larger checkmark indeed seems to look a lot more balanced


[test]update 2048 image

What this PR does / why we need it:
update 2048 image
Special notes for your reviewer:
Does this PR introduce a user-facing change?

Codecov Report

Merging #402 (adc2da1) into main (b13e61a) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #402   +/-   ##
=======================================
  Coverage   26.14%   26.14%
=======================================
  Files          19       19
  Lines         964      964
=======================================
  Hits          252      252
  Misses        682      682
  Partials       30       30

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


how to tranlate the "stale bot" into chinese?

hello,l have a question that how to tranlate the "stale bot" into chinese?

Good Morning.
Thanks for your help.
I have another question about What does  the "Lint" mean in the "explanation-for-github-workflow"?

About "what is Lint":

Linting is the process of running a program that will analyse code for potential errors.
https://stackoverflow.com/questions/8503559/what-is-linting
https://www.perforce.com/blog/qac/what-lint-code-and-why-linting-important

For example, when u submit a PR, the github workflow find out that your code style does not meet the project specification and give you an error, that's linting
About "how to translate it":
I don't know either......maybe use the english name ?

ok,thank you for your help!


numeric era dates: https://github.com/metanorma/metanorma-plateau/iss…

…ues/138
Metanorma PR checklist

[ ] Breaking changes (list related PRs)
[ ] Documentation update required (create task for this)
[ ] External dependency introduced (documentation update need)
[ ] Gem with native library introduced

Moving update to Plateau


Update CMakeLists.txt to 2.6.0

What does this Pull Request accomplish?
Bumps the version in CMake file
Why should this Pull Request be merged?
To reflect that the next grpc-device release will be 2.6.0
What testing has been done?
None

We should bump the version in AzDo first, then we can bump here. Otherwise, if this goes into main as is it'll create another 2.5 adhoc export.

Wouldn't doing it in this order mean that, for a small moment, ni-central will be pulling in a version of grpc-device that doesn't yet have builds?

We should bump the version in AzDo first, then we can bump here. Otherwise, if this goes into main as is it'll create another 2.5 adhoc export.

Wouldn't doing it in this order mean that, for a small moment, ni-central will be pulling in a version of grpc-device that doesn't yet have builds?

I don't think so, ni-central is only notified of builds from grpc-device. Changes on the ni-central side won't trigger a new adhoc build (see here in the pipeline file).

AzDO changes have been merged
https://ni.visualstudio.com/DevCentral/_git/ni-central/pullrequest/681880


[AgriFood] Review for client updates to support API version 2021-07-31-preview

Hi, requesting a review for changes in clients to support API version 2021-07-31-preview for AgriFood service
Link to API review - https://apiview.dev/Assemblies/Review/788e5820948d475cb5a7ae75de1057c7
Link to pull request - https://github.com/Azure/azure-sdk-for-js/pull/23189

Label prediction was below confidence level 0.6 for Model:ServiceLabels: 'Docs:0.13307431,API Management:0.123134606,IoT:0.033713784'
Label prediction was below confidence level 0.6 for Model:CategoryLabels: 'Client:0.5393933,Mgmt:0.44897026,Service:0.009109637'

Label prediction was below confidence level 0.6 for Model:ServiceLabels: 'Docs:0.13307431,API Management:0.123134606,IoT:0.033713784'
Label prediction was below confidence level 0.6 for Model:CategoryLabels: 'Client:0.5393933,Mgmt:0.44897026,Service:0.009109637'


Respond according to network connectivity or Hub upgrade issues

While looking at the peer logs collecting in the Hub I noticed a lot of noise due to network connectivity issues. Some of these are resulting from Hub deployment. Please see https://github.com/optdyn/hub/issues/2617 for more details on deployment based network issue handling. Others especially for local peers on laptops result from networking being down.
This is unnecessary noise that hides real issues. SS needs to be aware of when network connectivity is down so it does not blow exceptions for reasons of deployment or a laptop running offline. The offline problem can be solved with a REST endpoint where the Tray can report network connectivity issues to the management host. This way exceptions that occur are ignored or their level is dropped from ERROR to INFO without the need for a stack trace.

We might just reduce error log from stacktrace to error message

Fixed

This is NOT a fix. You're dangerously silencing the logs. We might actually need to see the stack trace if the network issue is a valid one. The key is to differentiate when it is a valid error and when it is not. If you have not done this we must reopen this issue.
I checked git. This is not a fix. Reopening...

@Lezh1k seems we need your participation

I restored the stacktraces except the one that happens on heartbeat. if HB fails then whatever reason is, SS will have the same cause - Connection Exception


About training loss,I used my own dataset whose format is the same as what crowdAI and ,however after several epoches ，The loss value becomes negativeral.can you help me ?Thank you very much！

however ,the loss Keep going down，Finally converged

Besides，i think the result is based on semantic segmentation, adding boundary loss.
In the original text:Exponential Weighted Boundary Loss (EWC): Similar to [23], which introduces a third building boundary class,we add a weighted loss function [27] that heavily penalizes wrong predictions close to and on building edges. Our exponential boundary loss generates a weight map w(x) that forces the model to separate neighboring objects into distinct entities
however,in my opinin,Semantic segmentation plus boundary loss is more like instance segmentation。
I don’t know if I have the right idea,

Hey, sorry it took so long to get back to you. If you're using a beta that is less than 1.0, then it is likely the combined loss will converge to be negative. By nature of the F-Beta measure, a beta less than 1 will cause the numerator to exceed the denominator so doing 1 - FBeta will result in a negative number.


fixed layout issues on bounty page

Before vs. After Comparison

Before
After

Checklist before requesting a review

[x] I have performed a self-review of my code
[x] I assure there is no similar/duplicate pull request regarding same issue

thankyou sir


Add spotify

I used OAuthSwift for Spotify, so I added it as an example.

thanks!

fixed link in readme 6b90b3b23ac2ef7bbba84313c673b269f2d364e7


Directory created even if internalLogLevel set to off?

I noticed that the directory _c:\temp_ of value internalLogFile="c:\temp\nlog-internal.log" is created even when internalLogLevel is set to "Off":
internalLogLevel="Off"
internalLogFile="C:\temp\nlog-internal.log"

Is this on purpose or should the directory _c:\temp_ only be created when the internalLogLevel is not set to "Off"? It's a minor issue but now I have an empty temp directory and I didn't know what caused it at first..
Tested with NLog version: 4.2.3.

That a good one! We will check this and fix it in 4.3.

Thanks for your acknowledgement!
I'm afraid that I'm not able to pick this up shortly, but it's good to know where to start ;)

Hi, I want to help you fixing this bug, I'm new to github and open source in general.
Do you need me to write a test for this case? Or is there something more for me to do.

@eduardorascon That would be great! We will help you to the process.
We really need unit tests for bugs. I would recommend to write a test before the bug is fixed, (TDD style), so we are sure the bug is solved.
In this case you can alter the current unit test to see the bug.
Steps

Create a new branch of master in your fork
Change the CreateDirectoriesIfNeededTests](
https://github.com/NLog/NLog/blob/18a7e20d2de44ae9aad0783ba64c5e859504845b/tests/NLog.UnitTests/Common/InternalLoggerTests.cs#L397) unit test with this as a start (please note, we use xUnit1)

  [Theory]
        //todo InlineData for all loglevels

        [InlineData("trace", true)]

        public void CreateDirectoriesIfNeededTests(string rawLoglevel, bool shouldCreateDirectory)
        {

            var loglevel = LogLevel.FromString(rawLoglevel);

//todo change unit test so it uses loglevel  and shouldCreateDirectory
            string expected =
                    "Warn WWW" + Environment.NewLine +
                    "Error EEE" + Environment.NewLine +
                    "Fatal FFF" + Environment.NewLine +
                    "Trace TTT" + Environment.NewLine +
                    "Debug DDD" + Environment.NewLine +
                    "Info III" + Environment.NewLine;

fix the bug
send PR

Hi @304NotModified
I think I got the fix done, buuut there's something strange. AssertFileContents is failing beacuse sometimes the message its not written to log file.
Here's the code in InternalLogger.cs that exits before writing to the log file:
if (level < LogLevel) { return; }

To expanda little bit more.
Seems to me that this happens beacuse LogLevel is set to "Trace" before any changes were made. Now, with the new changes LogLevel is set using rawLoglevel.
LogLevel.FromString(rawLoglevel);

I think the unit test is testing too much (more then the name suggests). Maybe I a better idea to copy the unit test and create a new one? (no  [InlineData("trace", true)] etc needed). What do you think?
If so, please do create a new branch (from master), so the current commits won't pollute the history.

I think AssertFileContents should be removed from this test because we should only be testing if the directory its created or not.
Also, there is a WriteToFileTests method in the InternalLoggerTest class already and this method also calls the AssertFileContents method.

That's also fine to me :)

Nice one, that was quick! Thanks @eduardorascon  @304NotModified


Investigate possibility to provide Irdeto error information

As a support team member I would like to be able to identify reasons why playback of DRM-protected content failed.
Hints

Irdeto provides reasons why a call to their services fail, in the response body. At the moment we only return the reason associated with the HTTP error code in SRG Data Protection. Would it be possible to extract the body info and to return it, also for Pillarbox in the end?
We can use www.tunnelbear.com to simulate a proxy or location abroad.

Acceptance criteria

Feasibility of error information extraction has been investigated.
Implementation tasks have been created if needed.

Tasks

[x] Investigate.

We can extract the payload and provide additional information when Irdeto returns an error. Messages are not localized, though, so maybe we should add it as additional information to the localized error message.

Error extraction should be used as a last resort. There is a better alternative that should be discussed first.

A quick and dirty implementation is provided on the associated branch. This task is done and will only lead to a better implementation should the discussion mentioned above not deliver the optimal result we can expect.


Add support for access token authentication to an Azure Postgres Flexible Server - Postgres scaler

Proposal
The Postgres scaler only supports basic authentication through username and password, but it is now possible to authenticate to a Postgres server hosted in the cloud by using "access token".
The purpose of this issue is to add support to enable access token authentication within the Postgres scaler by extending it, possibly with additional parameters.
Use-Case
In Azure, a managed identity does not have a password associated to it, but can obtain short-lived credentials (called access tokens) to connect to certain resources. It is possible for an Azure managed identity to connect to an Azure Postgres Flexible server.
Is this a feature you are interested in implementing yourself?
Yes
Anything else?
I created this PR: https://github.com/kedacore/keda/pull/5589. I tested it in my own Azure subscription to see if it works with my own use case (related to Apache Airflow) and it worked.

The doc of this new feature has been updated via this PR in keda-docs.
Thank you very much for your help on this @JorTurFer, much appreciated!

You're welcome :)
Thanks for this nice improvement! We will cut a release during the week, so you can expect it soon :)


flanky crashes

There is a flanky crashes when trying to create pull requests like this
hub pull-request -b staging

Report:
Crash report - runtime error: invalid memory address or nil pointer dereference
Error (runtime.errorString): runtime error: invalid memory address or nil pointer dereference
Stack:
goroutine 1 [running]:
panic(0x4c4000, 0xc8200100f0)
	/usr/local/Cellar/go/1.6.3/libexec/src/runtime/panic.go:443 +0x4e9
github.com/github/hub/github.(*Client).CreatePullRequest(0xc820173930, 0xc82000b9c0, 0xc8203541b0, 0x0, 0xa40028, 0xc82040a930)
	/var/folders/03/jqjv4qms2b18_hwt2dcgsghc0000gp/T/go/src/github.com/github/hub/github/client.go:107 +0x2b7
github.com/github/hub/commands.pullRequest(0x7e8880, 0xc820090090)
	/var/folders/03/jqjv4qms2b18_hwt2dcgsghc0000gp/T/go/src/github.com/github/hub/commands/pull_request.go:214 +0x1c42
github.com/github/hub/commands.(*Command).Call(0x7e8880, 0xc820090090, 0x0, 0x0)
	/var/folders/03/jqjv4qms2b18_hwt2dcgsghc0000gp/T/go/src/github.com/github/hub/commands/commands.go:47 +0x188
github.com/github/hub/commands.(*Runner).Call(0xc820011560, 0x7e8880, 0xc820090090, 0x0, 0x0, 0x0)
	/var/folders/03/jqjv4qms2b18_hwt2dcgsghc0000gp/T/go/src/github.com/github/hub/commands/runner.go:93 +0x5a
github.com/github/hub/commands.(*Runner).Execute(0xc820011560, 0x0, 0x0, 0x0)
	/var/folders/03/jqjv4qms2b18_hwt2dcgsghc0000gp/T/go/src/github.com/github/hub/commands/runner.go:85 +0x367
main.main()
	/private/tmp/hub-20160814-33207-3ih85c/hub-2.2.5/main.go:12 +0x40

Runtime:
GOOS: darwin
GOARCH: amd64

Version:
git version 2.10.0
hub version 2.2.5

Thanks for reporting! This was fixed in later versions of hub. Please upgrade!


python312Packages.pyeapi: 1.0.2 -> 1.0.4

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python312Packages.pyeapi is: Client for Arista eAPI
meta.homepage for python312Packages.pyeapi is: https://github.com/arista-eosplus/pyeapi
meta.changelog for python312Packages.pyeapi is: https://github.com/arista-eosplus/pyeapi/releases/tag/v1.0.4
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.0.4 in filename of file in /nix/store/vdmrslk4akw46msagn8zswwgmy6k47ka-python3.12-pyeapi-1.0.4

Rebuild report (if merged into master) (click to expand)

9 total rebuild path(s)

9 package rebuild(s)

First fifty rebuilds by attrpath

napalm
peering-manager
python311Packages.napalm
python311Packages.napalm-hp-procurve
python311Packages.pyeapi
python312Packages.napalm
python312Packages.napalm-hp-procurve
python312Packages.pyeapi

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/vdmrslk4akw46msagn8zswwgmy6k47ka-python3.12-pyeapi-1.0.4 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python312Packages.pyeapi https://github.com/r-ryantm/nixpkgs/archive/9b8d0ec90e0c87d44b39625a8c2ef58553d76d42.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/9b8d0ec90e0c87d44b39625a8c2ef58553d76d42#python312Packages.pyeapi

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/vdmrslk4akw46msagn8zswwgmy6k47ka-python3.12-pyeapi-1.0.4
ls -la /nix/store/vdmrslk4akw46msagn8zswwgmy6k47ka-python3.12-pyeapi-1.0.4/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

15 packages built:

napalm
napalm.dist
peering-manager
python311Packages.napalm
python311Packages.napalm-hp-procurve
python311Packages.napalm-hp-procurve.dist
python311Packages.napalm.dist
python311Packages.pyeapi
python311Packages.pyeapi.dist
python312Packages.napalm
python312Packages.napalm-hp-procurve
python312Packages.napalm-hp-procurve.dist
python312Packages.napalm.dist
python312Packages.pyeapi
python312Packages.pyeapi.dist

Maintainer pings
cc @astro for testing.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 335982 run on x86_64-linux 1

15 packages built:

napalm
napalm.dist
peering-manager
python311Packages.napalm
python311Packages.napalm-hp-procurve
python311Packages.napalm-hp-procurve.dist
python311Packages.napalm.dist
python311Packages.pyeapi
python311Packages.pyeapi.dist
python312Packages.napalm
python312Packages.napalm-hp-procurve
python312Packages.napalm-hp-procurve.dist
python312Packages.napalm.dist
python312Packages.pyeapi
python312Packages.pyeapi.dist


CI build using GitHub Actions

Setting up a github actions based CI workflow for this repo seems pretty important at this point to me as I've already failed a few times to ensure fmt, clippy, etc are all happy.

If it helps, I've set up similar workflows in the imxrt-boot-gen repo. It notes clippy warnings inline, and it requires rustfmt-formatted code.
clippy reports many warnings when it checks the RAL. We may need a crate-level #![allow(clippy::all)] in the RAL to prevent the noise.

I in fact based this on the actions for imxrt-boot-gen, but its a little quirky because of the workspace. I'll have to look maybe at tokio or other workspace projects to get some ideas.

So unfortunately uses doesn't allow a working directory, and the actions-rs/clippy action would need a working directory to work.
There's also a multitude of doc test errors in ral that probably should be fixed, I'll note both as issues and move on. I think having something here that builds, runs tests, and checks formatting is a good start. Adding clippy would be a welcome addition, will note that as another issue.


[Task - Client]: Hide creating JS Objects in Workflows.

Hey team! Please add your planning poker estimate with Zenhub @ayushpahwa @nsarupr

Closing this in favour #32065


Merge branch-switch changes to main and not v0

Oops, was supposed to merge #31 to main and not v0.
The changes mentioned in #31 has already been made. I updated the primary branch, branch protection rules, and updated the other PR to point to main.

@cherewaty I was debating if I should revert the original and fix the PR and re-merge that one or just sort it out with this new PR 🤷♂


traefik fails to initialize second TLS endpoint

Do you want to request a feature or report a bug?
bug

What did you do?
I asked this on slack, but I feel like this is more likely a bug.  In looking for a workaround to #2288, I attempted to setup a second TLS enabled endpoint, running on a different port.
However, this didn't work, returning a TLS error:
(35) gnutls_handshake() failed: An unexpected TLS packet was received.
When I enabled debug and examined the Traefik output, I found that it appears that Traefik is not enabling TLS on the second endpoint.

What did you expect to see?
I expected to see a new TLS endpoint setup, using the certificates provided
What did you see instead?
The endpoint did not have TLS enabled.
Output of traefik version: (What version of Traefik are you using?)
Version:      v1.4.0-rc4
Codename:     roquefort
Go version:   go1.9
Built:        2017-10-02_02:19:32PM
OS/Arch:      linux/amd64

What is your environment & configuration (arguments, toml, provider, platform, ...)?
logLevel = "Debug"
defaultEntryPoints = ["https", "gemini"]
[entryPoints]
  [entryPoints.https]
  address = ":443"
  clientCAFiles = ["certs/platform-ca.crt"]
  [entryPoints.https.tls]
    [[entryPoints.https.tls.certificates]]
    certFile = "certs/server.crt"
    keyFile = "certs/server.key"
  [entrypoints.gemini]
  address = ":8443"
  [entryPoints.gemini.tls]
    [[entryPoints.gemini.tls.certificates]]
    certFile = "certs/gemini.crt"
    keyFile = "certs/gemini.key"

[web]
address = ":8080"

[consulCatalog]
endpoint = "<IP_ADDRESS>:8500"
exposedByDefault = false
prefix = "traefik"

If applicable, please paste the log output in debug mode (--debug switch)
INFO[2017-10-20T16:19:13Z] Preparing server gemini &{Network: Address::8443 TLS:<nil> Redirect:<nil> Auth:<nil> WhitelistSourceRange:[] Compress:false ProxyProtocol:false} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s
INFO[2017-10-20T16:19:13Z] Preparing server https &{Network: Address::443 TLS:0xc420648c60 Redirect:<nil> Auth:<nil> WhitelistSourceRange:[] Compress:false ProxyProtocol:false} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s
INFO[2017-10-20T16:19:13Z] Starting server on :8443
INFO[2017-10-20T16:19:13Z] Starting server on :443
INFO[2017-10-20T16:19:13Z] Server configuration reloaded on :8443
INFO[2017-10-20T16:19:13Z] Server configuration reloaded on :443

Note the "TLS: nil" entry for the server setup on port 8443.

Thanks for reporting 👍
Could you try with latest release (1.4.0)?

looks like the same issue still:
Version:      v1.4.0
Codename:     roquefort
Go version:   go1.9.1
Built:        2017-10-16_04:52:07PM
OS/Arch:      linux/amd64

INFO[2017-10-20T17:38:38Z] Preparing server gemini &{Network: Address::8443 TLS:<nil> Redirect:<nil> Auth:<nil> WhitelistSourceRange:[] Compress:false ProxyProtocol:<nil> ForwardedHeaders:0xc4201d2220} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s
INFO[2017-10-20T17:38:38Z] Preparing server https &{Network: Address::443 TLS:0xc4204f84e0 Redirect:<nil> Auth:<nil> WhitelistSourceRange:[] Compress:false ProxyProtocol:<nil> ForwardedHeaders:0xc4201d22c0} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s
INFO[2017-10-20T17:38:38Z] Starting server on :8443
INFO[2017-10-20T17:38:38Z] Starting server on :443
INFO[2017-10-20T17:38:38Z] Server configuration reloaded on :8443
INFO[2017-10-20T17:38:38Z] Server configuration reloaded on :443

@jaychris
I tried to reproduce the problem with both master and v1.4 branches and finally I did it!
I think it's due to a typo in your configuration file.
Indeed, you defined the HTTPS entrypoint as [entryPoints.https] and gemini entrypoint as [entrypoints.gemini]. Apparently the problem comes from the downcase p in gemini entryPoint.
Can you try to use [entryPoints.gemini] tag? TOML is not key sensitive but if a tag is defined with uppercase all others sub tags or others  tags from a list have to respect the uppercases.
Moreover, your clientCAFiles = ["certs/platform-ca.crt"] parameter has to be set in the [entryPoints.https.tls] section to work correctly.
You can try the configuration described below :
logLevel = "Debug"
defaultEntryPoints = ["https", "gemini"]
[entryPoints]
  [entryPoints.https]
  address = ":443"
  [entryPoints.https.tls]
    clientCAFiles = ["certs/platform-ca.crt"]
    [[entryPoints.https.tls.certificates]]
    certFile = "certs/server.crt"
    keyFile = "certs/server.key"
  [entryPoints.gemini]
  address = ":8443"
  [entryPoints.gemini.tls]
    [[entryPoints.gemini.tls.certificates]]
    certFile = "certs/gemini.crt"
    keyFile = "certs/gemini.key"

[web]
address = ":8080"

[consulCatalog]
endpoint = "<IP_ADDRESS>:8500"
exposedByDefault = false
prefix = "traefik"

I let you give us a feddback and close the issue if the solution is ok for you.

argh, yes, so sorry about that.  It works fine once the typo is fixed.
FWIW, the clientCAFiles line seems to work correctly under [entryPoint.https] or [entryPoint.https.tls] in the same manner.  At least, I get a challenge for a client certificate either way.

I'll close this issue, because I think the question is answered.

OK @jaychris .
Indeed, clientCAFiles can be to differents places.
Thanks for the feedback. I close the issue.


Updates to MIP40c3-SP11

Updated budget proposal with budget implementation and clarified continuity fund disposition.

looks good


StateIcon

Preparation

[x] Existing defect backlog reviewed/evaluated, prioritized, and selected
[x] Functional requirements defined: functional test script is defined/written (not coded) and signed off by Dev, Design, ADA and Product leads
[x] Non-functional requirements defined (e.g. controlled vs. uncontrolled)
[x] Existing enhancements backlog reviewed/evaluated, prioritized, and selected
[x] Usability testing plan defined or deemed unnecessary
[x] Examples needed to properly demonstrate component features are defined and signed off by Dev, Design, ADA and Product leads

Development

[x] Component implemented in/ported to TypeScript
[x] 3rd-party dependencies removed
[x] JSS converted to CSS
[x] CSS upgraded to use characteristics
[x] Selected defects resolved
[x] Component API finalized
[x] Necessary refactoring completed
[x] Selected enhancements added

Testing

[x] Functional test script is implemented in Cypress and passes
[ ] Code not covered by the functional test script is identified and reviewed by Dev & Design leads
[x] QA (v1-2 visual regression) test passes
[x] Automated Axe tests added and pass

Examples

[x] All examples present in TK1 are implemented as stories in Odyssey (ported from BitBucket)
[x] All examples needed are implemented as stories (either ported from BitBucket or created net new)

Deployment

[x] Codemods for migration created and verified

Reviews/Signoff

[ ] Dev production review
[x] Design review
[ ] Product signoff
[ ] ADA review

Documentation

[ ] Dev docs content provided
[x] TK1→Odyssey migration guide provided
[ ] Design docs provided
[ ] Consolidated documentation produced by combining dev content, migration guide, and design docs
[ ] Consolidated documentation published on Storybook
[ ] Consolidated documentation reviewed and updated according to user feedback

Figma symbols

[x] New Odyssey Figma symbol created
[x] Figma symbol references updated to use new Figma symbol

@JoanaMMoreira I c/p the definition of done checklist from confluence but feel free to edit this as you wish since we don't really have a template for these right now.

@maj-stella here's the link: https://uitk.pages.dev/?path=/story/core-stateicon--default-state-icon

An ideal DOM structure would be that  element is a top level element; therefore remove two span elements above svg and add role=“img” and aria-label=“xxx” to the  element directly.

I suggest using aria-label="success" instead of aria-label="success tick" as default value.

@maj-stella here you go: https://state-icon-aria-label.uitk.pages.dev/?path=/story/core-stateicon--all-state-icons

After a conversation with @jeanlaleuf and @pseys StateIcon should probably be renamed to StatusIcon to match the upcoming status validation work.

Further consideration - should this be "StatusIndicator" or similar, rather than StatusIcon.

Further consideration - should this be "StatusIndicator" or similar, rather than StatusIcon.

Sure, should we discuss with the team? cc @joshwooding

Documentation review

typo 'unterlying'
Can there be more detail/examples added to the content around passing Custom Props in the Props table?
Can the order of the different statuses be the same whenever they are listed? The text versions in the status row of the Props table do not match the graphical equivalents under the 'Status icons' heading.
Include information currently under 'Status icons' under the Features heading. Provide guidance as to when different icons should be used under 'Use when...' .
Add API, Props and Anatomy content as H3s under a H2 'Configuing StatusIcon'
Add Accessibility, Related components, and Help and Support sections

Further consideration - should this be "StatusIndicator" or similar, rather than StatusIcon.

Sure, should we discuss with the team? cc @joshwooding

Hey, this is fine. It came out of conversations we've had with design and the new indication pattern.

@sion-docs thanks for this, here's a draft PR for you to review please: https://github.com/jpmorganchase/uitk/pull/508
@maj-stella could you please add some wording around accessibility so we can include it in the docs?

11/10
kick-off call tomorrow 11/11
rename to statusindicator - to be confirmed tomorrow


Updated downloaders to close automatically, so auto restart doesnt need confirm.

Title. When set to auto update, it makes you press a key in console to update, which prevents auto restart

Thanks, PR master please


Yuzu support?

I understand it is not stated in the README, but I would hope to get Yuzu support too.
Here is my Yuzu folder structure in the mod load directory.
├── practice_mod
│   ├── exefs
│   │   └── subsdk1    <-- smo-practice100.nso
│   ├── romfs
│   │   ├── DebugData
│   │   │   └── Font
│   │   │       ├── nvn_font_jis1.ntx
│   │   │       ├── nvn_font_jis1_tbl.bin
│   │   │       ├── nvn_font.ntx
│   │   │       ├── nvn_font_shader.bin
│   │   │       └── nvn_font_shader_jis1.bin
│   │   └── LayoutData
│   │       └── CounterCoin.szs
│   └── romfs_ext
│       └── startlight_patch_100
│           └── 3CA12DFAAF9C82DA064D1698DF79CDA1.ips

I have also tried placing 3CA12DFAAF9C82DA064D1698DF79CDA1.ips in the exefs and romfs, with and without the startlight_patch_100 folder, but none of it works. Here is what I get from the yuzu, just ABCD on the left corner, I am not familiar with the Yuzu debugger also.

Version

System: Ubuntu 20.04
Yuzu: yuzu-mainline tag mainline-0-835, build from source
SMO Version: 1.3.0
Mod Version: 1.2.0, build from source

I am not sure if this is an issue on the Yuzu side or the mod side. As this early mod works perfectly on Yuzu. If you would point me to the right direction, I can also help to contribute to the Yuzu support for this mod.

You compiled the 1.0 version and tried to use it on 1.3, to compile for 1.3 you must set S2VER and S2VERSTR in the Makefile to 130 and 1.3.0.
The DebugData folder is needed on 1.0 only, and the LayoutData folder is needed on 1.3 only, since a different method of displaying text is used in 1.3.

Thanks @fruityloops1, I managed to get it working with the following folder structure
├── practice_mod
│   ├── exefs
│   │   ├── B424BE150A8E7D78701CBE7A439D9EBF.ips
│   │   └── subsdk1
│   └── romfs
│       └── LayoutData
│           └── CounterCoin.szs

Works like a charm now.
Do you mind me submitting a PR to include a Yuzu setup instruction in the README?

That would be a good idea.

The PR has been accepted and the description can now be found in the README, so this issue can be closed.


Revert Swift Version

Changes
@VerizonAdPlatforms/video-partner-sdk-developers: Please review.

Codecov Report

Merging #128 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #128   +/-   ##
=======================================
  Coverage   77.68%   77.68%
=======================================
  Files          90       90
  Lines        8218     8218
=======================================
  Hits         6384     6384
  Misses       1834     1834

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 46ed77b...0ea68f3. Read the comment docs.


Page loading progress bar support! 🥰🥰🥰

Page Loading, Progress Bar, Util

[x] I have searched the issues of this repository and believe that this is not a duplicate.

Summary 💡
When the page loading, the page display a progress bar at the page top like the topbar or pace-js library.
Examples 🌈

Topbar Official Site

Motivation 🔦

It allows me to have a more pleasant mood to wait for the page to load. 🥰🥰🥰

#https://github.com/mui-org/material-ui/issues/22486


Prediction on large images with thin z-axis

Hello,
I have run into issues regarding StarDist's internal requirements for image data. We are using the data to analyze cell populations in 3D whole-organ images of Drosophila midgut. The midgut is only pseudostratified, and as a result we have perhaps a bit abnormal image data in that they are large on the XY-axes, but can be only a few slices on the Z-axis.
Now, the requirements of the package on object and block size are quite restrictive due to the data, as the cells may barely fit within the image in Z-axis. We have adapted around the problem by beginning to take full thickness images of the organ. However, we do have a lot of older image data that we would have use for, and the imaging of the whole organ is quite time consuming on large scale, as you can imagine. I could change the code to ease the restrictions, but as I don't have full understanding of how the prediction functions, I would like to know if there is a way to bypass the requirements in a sensible manner? Could the package be adapted to allow for this type of image data?
Let me know if you require any additional information, and thanks in advance for any assistance!
Sincerely,
Arto Viitanen
University of Helsinki

Hi @ArtoVi ,
For training, a majority of cells should be fully included in the annotated trainings stacks - I fear there is no real way around that. Maybe you could acquire images with more z planes but reduced field of view in xy?

I don't know how your data looks like, but have you tried padding the image such that the Z axis is big enough (as a trivial workaround)?

Thanks for the replies, @maweigert and @uschmidt83

For training, a majority of cells should be fully included in the annotated trainings stacks - I fear there is no real way around that.

Sorry, I may not have been as clear as I intended. We have a working model that has been created with thicker images, and the problem occurs when applying the model on thin image-stacks. However, it seems like I am not able to reproduce the error (as in crashing the process) with the data I currently have access to, but I can't get any predictions with thinner images. I performed an example run with a representative image (XY-cropped) and a duplicate image with one z-slice removed, i.e. from 5 to 4 z-slices. I have included my output below with verbose=True.
All in all, it seems that for some reason StarDist is not able to create any candidates for the thinner stack. At the thickness of 4 z-layers it is expected that the largest cells do not neatly fit into the stack, but there should be plenty of smaller cells that I would think should still create candidates. So I would presume there has to be a reason for why none are created?
@uschmidt83

I don't know how your data looks like, but have you tried padding the image such that the Z axis is big enough (as a trivial workaround)?

I did try that some months back by adding empty slices but it caused bloating of the labels and was not viable. But now that I think about it there probably is a better way for padding the image, such as duplicating the edge layers to provide continuity in intensities.
OUTPUT:
Holidic_2018-12-19_180224-2; Model = DAPI20x ; Image dims = (4, 944, 2008)
predicting instances with prob_thresh = 0.01 and nms_thresh = 0.4
found 0 candidates
non-maximum suppression...
Non Maximum Suppression (3D) ++++
NMS: n_polys  = 0
NMS: n_rays   = 128
NMS: n_faces  = 252
NMS: thresh   = 0.400
NMS: use_bbox = 1
NMS: using OpenMP with 16 thread(s)
NMS: precompute volumes, bounding boxes, etc
precompute done
NMS: calculated anisotropy: -nan(ind) 	 -nan(ind) 	 -nan(ind)
NMS: starting suppression loop
NMS: Function calls:
NMS: ~ bbox+out:        0
NMS: ~ inner:           0
NMS: ~ kernel:          0
NMS: ~ convex:          0
NMS: ~ render:          0
NMS: Excluded intersection:
NMS: + pretest:         0
NMS: + convex:          0
NMS: Function calls timing:
NMS: / kernel:   0.00 s  (0.00 ms per call)
NMS: / convex:   0.00 s  (0.00 ms per call)
NMS: / render:   0.00 s  (0.00 ms per call)
NMS: Suppressed polyhedra:
NMS: # inner:           0 / 0  (-nan(ind) %)
NMS: # kernel:          0 / 0  (-nan(ind) %)
NMS: # render:          0 / 0  (-nan(ind) %)
NMS: # total:           0 / 0  (-nan(ind) %)
NMS: # keeping          0 / 0  polyhedra (-nan(ind) %)
NMS took 0.0060 s
keeping 0/0 polyhedra
render polygons...
warning: empty list of points (returning background-only image)
Description of 'Holidic_2018-12-19_180224-2_Ch=1.labels.tif':
Z    Y    X  Intensity  Volume  Area_Max
count  0.0  0.0  0.0        0.0     0.0       0.0
mean   NaN  NaN  NaN        NaN     NaN       NaN
std    NaN  NaN  NaN        NaN     NaN       NaN
min    NaN  NaN  NaN        NaN     NaN       NaN
25%    NaN  NaN  NaN        NaN     NaN       NaN
50%    NaN  NaN  NaN        NaN     NaN       NaN
75%    NaN  NaN  NaN        NaN     NaN       NaN
max    NaN  NaN  NaN        NaN     NaN       NaN
Holidic_2018-12-19_180224; Model = DAPI20x ; Image dims = (5, 944, 2008)
predicting instances with prob_thresh = 0.01 and nms_thresh = 0.4
found 36356 candidates
non-maximum suppression...
Non Maximum Suppression (3D) ++++
NMS: n_polys  = 36356
NMS: n_rays   = 128
NMS: n_faces  = 252
NMS: thresh   = 0.400
NMS: use_bbox = 1
NMS: using OpenMP with 16 thread(s)
NMS: precompute volumes, bounding boxes, etc
precompute done
NMS: calculated anisotropy: 6.94 	 1.00 	 1.01
NMS: starting suppression loop
|################################################# | [98 % suppressed]
NMS: Function calls:
NMS: ~ bbox+out:  5667969
NMS: ~ inner:       37360
NMS: ~ kernel:      37326
NMS: ~ convex:      32949
NMS: ~ render:      32114
NMS: Excluded intersection:
NMS: + pretest:   5630609
NMS: + convex:        835
NMS: Function calls timing:
NMS: / kernel:   22.67 s  (0.61 ms per call)
NMS: / convex:   94.72 s  (2.87 ms per call)
NMS: / render:   177.13 s  (5.52 ms per call)
NMS: Suppressed polyhedra:
NMS: # inner:          34 / 36356  (0.09 %)
NMS: # kernel:       4377 / 36356  (12.04 %)
NMS: # render:      31495 / 36356  (86.63 %)
NMS: # total:       35906 / 36356  (98.76 %)
NMS: # keeping        450 / 36356  polyhedra (1.24 %)
NMS took 21.8848 s
keeping 450/36356 polyhedra
render polygons...
+++++++++++++++ polyhedra to label +++++++++++++++
n_polys           = 450
n_rays            = 128
n_faces           = 252
nz, ny, nx        = 5 944 2008
use_overlap_label = 0
overlap_label     = 0
Description of 'Holidic_2018-12-19_180224_Ch=1.labels.tif':
Z        Y        X  Intensity    Volume  Area_Max
count  450.000  450.000  450.000    450.000   450.000   450.000
mean     6.245  138.313  431.631    428.547   391.448    43.203
std      2.450   50.080  131.428    192.670   275.302    25.886
min      3.087   14.868  143.900     75.708    18.315     3.591
25%      3.650   99.632  321.775    271.615   108.276    16.478
50%      6.283  132.960  433.391    393.953   413.712    46.317
75%      8.390  175.047  543.506    553.827   616.797    65.857
max     10.750  289.230  650.362   1038.490  1063.010    96.119

All in all, it seems that for some reason StarDist is not able to create any candidates for the thinner stack. At the thickness of 4 z-layers it is expected that the largest cells do not neatly fit into the stack, but there should be plenty of smaller cells that I would think should still create candidates. So I would presume there has to be a reason for why none are created?

I don't understand. Are cells really only 4 pixels (or even less for "smaller cells") in Z? Was the model trained with such data?

I did try that some months back by adding empty slices but it caused bloating of the labels and was not viable. But now that I think about it there probably is a better way for padding the image, such as duplicating the edge layers to provide continuity in intensities.

You should pad the image with "background" values (whatever that means for your image) to not "bloat" the segmented objects sizes.

predicting instances with prob_thresh = 0.01 and nms_thresh = 0.4
found 0 candidates

Something is clearly wrong here. There shouldn't be a need to use such a very small prob_thresh value, unless the model is badly trained or not suitable for the given image.
Can you share a/some representative training images and also a/some problematic test images?
Uwe

I don't understand. Are cells really only 4 pixels (or even less for "smaller cells") in Z? Was the model trained with such data?

Yes, the images have been taken with a spinning disc confocal and are 3.4um per Z-slice, and the model has been trained for such. The model is mainly used in finding organ-wide cell population changes in the intestine. The Drosophila intestinal wall is at max around 1.5 cells thick, and we mainly need for example cell positions and maximal XY-area, so the lower resolution on Z axis allows us to image in bulk due to the shorter imaging time.

You should pad the image with "background" values (whatever that means for your image) to not "bloat" the segmented objects sizes.

That is helpful, thank you!

There shouldn't be a need to use such a very small prob_thresh value, unless the model is badly trained or not suitable for the given image.

Ah, I had dropped it to such a low value because I was testing if the prob_thresh was causing the lack of predicted labels. In other words, the model does not normally need such a low value to function. This current model is not optimal and will be retrained in future, but I have created multiple models for similar data, even lower Z-resolution, with outstanding results.

Can you share a/some representative training images and also a/some problematic test images?

Here's one representative annotation image:
StarDist_Annotation.zip
Here's the above image with faulty prediction:
faulty_img.zip
Here's also a flattened version of one image so you get an idea what our full images look like:
flat_midgut.zip
Unfortunately, I don't have more test images to give out of hand, as the problem presented itself on a lab mate's dataset and it was a while back so I will have to go look for some.


Implement the bake button, uses a "UV camera" to cast primary rays.

Going to do a refactoring to better decouple glTF from embree.


Rewrite more Map-Reduces as Renderers to save garbage

MapX2Host and to a lesser extent MapProcess2Name have the property that they map many nodes onto fewer nodes, so looping and using the machinery in joinResults is more effective than creating many copies of the same node and merging.
Benchmarks before (c5bdebdffebfa7cc0839770a3bf70429260f1bc9):
BenchmarkTopologyList-2         	       2	 603197223 ns/op	112161732 B/op	 1253503 allocs/op
BenchmarkTopologyHosts-2        	       3	 385415232 ns/op	82957173 B/op	 1015204 allocs/op
BenchmarkTopologyContainers-2   	      10	 192766565 ns/op	33145659 B/op	  407698 allocs/op

after this PR:
BenchmarkTopologyList-2         	       3	 476705872 ns/op	101366640 B/op	 1109042 allocs/op
BenchmarkTopologyHosts-2        	       3	 373288182 ns/op	76540733 B/op	  933301 allocs/op
BenchmarkTopologyContainers-2   	      10	 183615683 ns/op	33132417 B/op	  407672 allocs/op

With more-renderers-b I am seeing a similar improvements in allocations as above, but no improvements in runtime:
master
BenchmarkTopologyList-4         	       2	 634468221 ns/op	107948256 B/op	 1056511 allocs/op
BenchmarkTopologyHosts-4        	       2	 553097064 ns/op	78231540 B/op	  812637 allocs/op
BenchmarkTopologyContainers-4   	       5	 235253550 ns/op	30075289 B/op	  302554 allocs/op

branch
BenchmarkTopologyList-4         	       2	 638986932 ns/op	97484960 B/op	  915473 allocs/op
BenchmarkTopologyHosts-4        	       2	 545031910 ns/op	71767348 B/op	  730887 allocs/op
BenchmarkTopologyContainers-4   	       5	 225194107 ns/op	30069329 B/op	  302520 allocs/op

(each line is the best result of three runs)

I've re-run the benchmarks with cpu=1, and now I do see a >5% runtime improvement in BenchmarkTopologyList.


Defining overall goals

In my opinion, in the current Readme we are focusing mostly on the activities we are performing (defining the protocols and, secondarily, providing libraries). I think it would be useful to spend a few words on the overall goals of the initiative, i.e., the motivation for performing these activities.
For this reason, I propose we keep the points currently listed as "goals" but we indicate them as "activities", and we add a short list of overall objectives before the list of activities.
What do you think?

+1

I have rephrased things a bit and separated them into Mission and Activities. Please consider this PR: https://github.com/LIonWeb-org/.github/pull/3

Let’s close this one as it has been superseded by Markus’ proposal


require for eval code

Excellent work. May I ask if there is any evaluation test code for reference?

For now, we just do asr evalution, and you may refer to asr_leaderboard from huggingface for the eval code.


External editor does not auto-update replays on MacOS

External editor does not autoupdate on MacOS when saving the generated tmp file. Likely due to fsnotify watching the file, solution may be to watch the tmp directory and monitor for when a new file is moved over the top of the monitored filename as well as write events.

Seems to be an fsnotify issue - https://github.com/fsnotify/fsnotify/issues/600


feat(cli): Add initial ORT Server CLI implementation

This commit introduces the new module cli for the ORT Server, intended to contain all upcoming implementations related to the CLI.

all upcoming implementations related to the CLI.

Interesting, I wasn't aware such a thing being planned. Can you share more about the use-case and planned commands?

The CLI will simplify the integration of the ORT Server in a CI/CD environment for our customers, especially starting an ORT run in a blocking mode. To replace our proprietary solution with the ORT Server CLI, we need the following features:

Start an ORT run
Get details about an ORT run
Download reports
Download logs


Remove unused viewType property in AppListComponent

This is an internal cleanup issue and needs no changelog entry.

Done in the meantime in https://github.com/mesosphere/marathon-ui/pull/411


On timmer resume, notify user the total amount of time for which he had paused the timmer.

Is your feature request related to a problem? Please describe.
no, just a feature request, it would be better to notify the user for which amount of time he had paused the timer, by showing a CSS Snackbar/Toast or a notification.
e.g "Timer resumed, you had paused it for 10 mins"
Describe the solution you'd like
Notify the user about for which amount of time he had paused the timmer.
Describe alternatives you've considered
Additional context

May I try working on this one?

Yes sure @CherylL26 :)


Persistent cache based on http headers

It would be nice to be able to configure persistent cache based on response http headers.  Perhaps specifying a path on disk.  Apt packages are of particular interest =)

FWIW, I think mitmproxy is the wrong tool for this, but you should be able to implement this as an addon yourself. It's likely nothing we want to have included in the core anytime soon, so I'm closing this for now.


Spelling and formatting

Formatting and spelling was changed for many lines.

@TheGiantPanda Please remove trailing whitespace changes from your commit.


RC test: Area Model Introduction 1.2.0-rc.1

@ariel-phet, @amanda-phet, area-model-introduction/1.2.0-rc.1 is ready for RC testing. Document issues in https://github.com/phetsims/area-model-common/issues and link to this issue.
Assigning to @ariel-phet for prioritization.

General RC Test

What to Test

Click every single button.
Test all possible forms of input.

Test all mouse/trackpad inputs.
Test all touchscreen inputs.

If there is sound, make sure it works.
Make sure you can't lose anything.
Play with the sim normally.
Try to break the sim.
Test all query parameters on all platforms. (See QA Book for a list of query parameters.)
Download HTML on Chrome and iOS.
Make sure the iFrame version of the simulation is working as intended on all platforms.
Make sure the XHTML version of the simulation is working as intended on all platforms.
Complete the test matrix.
Don't forget to make sure the sim works with Legends of Learning.
Check this LoL spreadsheet and notify AR if it not there.
If this is rc.1 please do a memory test.
When making an issue, check to see if it was in a previously published version

Link(s)

Simulation
iFrame
XHTML
Test Matrix
Legends of Learning Harness

FAQs for QA Members

There are multiple tests in this issue... Which test should I do first?
Test in order! Test the first thing first, the second thing second, and so on.

How should I format my issue?

We typically assign the developer who opened the issue in the QA repository.

My question isn't in here... What should I do?
You should:

Consult the QA Book.
Google it.
Ask Katie.
Ask a developer.
Google it again.
Cry.

Memory Test Results:
Start: 24.6
1 min: 32.0
2 min: 34.2
3 min: 34.9
4 min: 35.4
5 min: 36.3
6 min: 36.3
7 min: 36.6
8 min: 36.7
9 min: 36.7
10 min: 37.2

QA is done

Thanks!


Linux installation directories

On linux executables, libraries, and configuration files are usually not installed in the same location [1]. The documentation says that everything is installed under PREFIX/bin. The linux binary packages (deb, rpm) seem to install everything into /usr/bin as well. Is there a way to specify separate directories for exacutables, libraries, and  when installing TSDuck?
One of the more obvious issues with the way things are currently installed is that all tsplugin libraries appear in shell autocomplete when looking for ts tools.
[1] https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard

Hi,
I know the usages on Linux (on all Unix systems in fact). The way TSDuck is installed is probably not optimal. This is the result of many conflicting requirements:

Must be portable across Windows and various Unixen (Linux, macOS and potentially others).
Simple to troubleshoot (no need to mess with different conventions of different OS, e.g. bin vs. lib, lib vs. lib64, etc).
No hardcoded path, neither absolute nor relative. I fundamentally disagree with the pitfalls of configure scripts with PREFIX options resulting in hardcoded paths in the binaries. I wanted everything to be "moveable" anywhere.
Each executable must be useable by default anywhere using zero external configuration (no environment variable, no complicated or non-portable trick).

The obvious simplest choice was to store everything at the same place. The code retrieves the file location of the current executable (available in any OS) and uses the same directory to look for plugins and configuration files.

Is there a way to specify separate directories for executables, libraries, and when installing TSDuck?

You can install TSDuck in a local environment (/usr/local, $HOME/local or whatever). See here. But this does not separate executables and plugins.
Additionnally, you may add new plugins or relocate existing ones somewhere else using the TSPLUGINS_PATH environment variable (inspired by LD_LIBRARY_PATH on Linux), although it may open security issues. It also contradicts the (self-imposed) requirement for "useable by default anywhere using zero external configuration".

One of the more obvious issues with the way things are currently installed is that all tsplugin libraries appear in shell autocomplete when looking for ts tools.

Here, you score a point. This is the result of a mistake. The .so files are installed with execute permission. This is not necessary. I will revert the access modes to 644. They won't appear in shell autocompletion anymore.

The .so files are installed with execute permission. This is not necessary. I will revert the access modes to 644.

Done. This was a mistake with all tsplugin_*.so which were installed with 755. The big tsduck.so which contains most TSDuck code was already installed with 644.
Note that gcc creates shared objects with execute rights by default. On Linux, this is useless. But note that this article mentions that this is required on HP-UX for security reasons. This is a very good reason in my opinion: on HP-UX you can't map executable memory pages from a file without execute permission. Linux used to be lousy on security in the beginning compared to more professional OS of that time and never completely caught up, unfortunately.

Thank you for the quick fix!
Would you be open to adding make (or Makefile) options to set these directories in the future?
This feature is more for making a packages for distributions, not something you would need by default. Such packages are meant to do all the configuration so everything is ready to go anyway. I realized this issue when writing an ebuild for my own use. Though I have to say this is a minimal inconvenience at this point.
Installing files in the directories the system expects should remove the need to do any hacks like hardcoding paths. I don't know why configure would do that.

useable by default anywhere using zero external configuration

As TSDuck is perfectly happy with running from a single directory I was surprised that everything isn't built in one build directory. Currently all the build artifacts are scattered around multiple sub-directories in src. setenv.sh is inconvenient for me as it requires to use bash.

Installing files in the directories the system expects should remove the need to do any hacks like hardcoding paths. I don't know why configure would do that.

This is true for tsduck.so (common code library) but not for plugins plugin_*.so and extensions tslibext_*.so. The former is explicitly linked with all application. So the standard system-specific shared object activation applies. But the latter are dynamically looked up and activated using dlopen(). So the application needs to know where to find them. tsp does not known any list of predefined plugins. It uses plugin names from the command line and lookup the corresponding files. Similarly, the command tsp -l searches all possible plugin files. So, it needs to know where to search.
If plugins are located in the same directory on Windows, in relative path ../lib on macOS and 32-bit Linux, on ../lib64 on 64-bit Linux, etc., then this is no longer transparent to the source because the file lookup must be different.

I just realized that TSDuck installs the udev rules in the wrong place for my distribution:
 * QA Notice: udev rules should be installed in /lib/udev/rules.d:
 *
 *   /etc/udev/rules.d/80-tsduck.rules
 *

Is there a way to set the udev path? Alternatively would it be possible to have make install skip installing the udev rules?

Hi,
Which distribution are you using?
I had used /etc/udev/rules.d for TSDuck from the beginning (15 years) without thinking of checking if this was the best place. I just tried to understand the difference between /etc/udev/rules.d and /lib/udev/rules.d. All posts I read, including this one and this one, seem to agree that /lib/udev/rules.d is for system-installed non modifiable rules while /etc/udev/rules.d is made for custom rules which are manually created by the system administrator. If two files with the same name exist in both directories, the custom one in /etc/udev/rules.d is used.
As a consequence, /lib/udev/rules.d seems a better place for TSDuck udev rules. I will update this, independently of any distribution.

I have pushed the fix moving udev rules to /lib/udev/rules.d. I checked the .rpm on Fedora and the .deb on Ubuntu. The old file in /etc/udev/rules.d has disappeared and the new one appeared in /lib/udev/rules.d.

@APTX,

As TSDuck is perfectly happy with running from a single directory I was surprised that everything isn't built in one build directory. Currently all the build artifacts are scattered around multiple sub-directories in src. setenv.sh is inconvenient for me as it requires to use bash.

See #588

@APTX, @piotrserafin, all,
I plan to solve the file location issue and I would like to collect your feedback.
Requirement:

On a given platform, one binary should fit all configurations, no need to compile differently depending on the target configuration (i.e. no configure-like script).
It should continue to work as it does today because 1) this is the way it works on Windows, 2) this is the way it works in the build area.

Propose solution:

No code change:

The main big tsduck.so is in /usr/lib. It is also used by third-party applications.

Code change:

The plugins are searched in the following order:

all directories from $TSPLUGINS_PATH
same directory as main executable
Linux and macOS only:

directory ../lib/tsduck/ from main executable

default system activation for shared libraries (direct use of dlopen() on Linux)

The configuration files (tsduck.*.names and tsduck.*.xml) are searched in the following order:

all directories from $TSPLUGINS_PATH
same directory as main executable
Linux and macOS only:

directory ../../etc/tsduck/ from main executable
directory ../lib/tsduck/ from main executable
all directories from $LD_LIBRARY_PATH

all directories from $PATH

On Linux x86_64 platforms, use lib64 instead of lib.

So, instead of copying everything into /usr/bin, the package would typically install:

commands in /usr/bin
tsduck.so in /usr/lib[64]
plugins and configuration files in /usr/lib[64]/tsduck

@piotrserafin, comparing this with your PKGBUILD in AUR for Arch Linux, I see the following differences:

use /usr/lib64 instead of /usr/lib on x86_64, does Arch Linux use /usr/lib64? why not?
move tsduck.so in /usr/lib[64] (you leave it in /usr/bin)
move configuration files in /usr/lib[64]/tsduck (you leave them in /usr/bin)
move plugins to /usr/lib[64]/tsduck (you move them to /usr/lib/tsduck/plugins but I do not see any reason to create an additional level of directory)
no need to create profile script to define TSPLUGINS_PATH (but that is the original purpose)

Thanks for your feedback.

If possible, please name it libtsduck.so. -ltsduck will not find tsduck.so (you have to write it like -l:tsduck.so or use a full path, which is uncommon knowledge). The static library is already named libtsduck.a meaning that usually the static library will be found instead of the dynamic one. This is likely not what is expected. This also makes writing build scripts for 3rd party tools/plugins more complicated.
Consider putting files that are not config files (like the xml models and *.names files) or any other files that contain static data that isn't usually modified by the user under /usr/share.
I don't think it's that useful to search $PATH and $LD_LIBRARY_PATH for the config files.
lib/lib64 is not universal among linux distributions so some distributions may need to override it.
Currently I install TSDuck into /opt/tsduck so that it can all be in one directory that is also not used by anything else. I add that path to the system environment.

Agreed on most points.

Consider putting files that are not config files (like the xml models and *.names files) or any other files that contain static data that isn't usually modified by the user under /usr/share.

There is no such file in TSDuck.

lib/lib64 is not universal among linux distributions so some distributions may need to override it.

Currently, lib64 is assumed when the OS is Linux and the architecture is x86_64. I should add the condition that /usr/lib64 exists on the build system (assumed to be a similar distribution).

Consider putting files that are not config files (like the xml models and *.names files) or any other files that contain static data that isn't usually modified by the user under /usr/share.

I am unsure to understand correctly your sentence.

Do you mean that tsduck.*.xml and tsduck.*.names should be in /usr/share/tsduck.
Or do you mean that other files (there is none) should be there, meaning leaving tsduck.*.xml and tsduck.*.names in /usr/lib/tsduck?

The tsduck.*.xml and tsduck.*.names files are constant configuration files for TSDuck. Using /usr/lib/product or  /usr/share/product seems to depend on the product. Some of them, such as grub or systemd, use both without clear rationale explaining why which file is in which zone. Since /usr/lib/tsduck needs to be created for plugins anyway, I see no reason to scatter TSDuck files in too many different places.

I did mean the first one. /usr/share is intended for "architecture independent data", which are things like static data files, documentation, example files.
To use grub as an example (at least on my system): Grub puts its modules (or "plugins") into /usr/lib/grub/<platform> where <platform> are things like x86_64-efi (64-bit UEFI BIOS) or i386-pc (old non-UEFI BIOS). The plugins are built for their respective platforms. Grub puts things like themes and fonts into /usr/share/grub. Those files can be shared between all platforms.

Hi,
I made the file reorganisation. The .rpm and .deb package are conformant. You may test if you like.
By renaming tsduck.so as libtsduck.so and declaring it as a "standard" library, I discovered something I never noticed in 20 years of Linux usage. In the Red Hat family (Fedora, CentOS), all standard shared libraries have executable rights. They must have them. If they haven't, the files work but the shared library is not considered as a provision of the RPM package and this creates dependency issues. On the other hand, in the Debian family (Ubuntu, Raspbian), no shared library has executable rights. Weird....

I checked the nightly builds, but the latest build does not have the changes yet. Build 1834 should have them, but when I checked the latest one was 1832.

1834 is there now. Nightly builds are published every night at 4:10 UTC.
Generally speaking, using nightly builds is essentially useful to Windows users because it takes a long time to rebuild on Windows (typically one hour or more). For Linux or macOS users, TSDuck takes only a few minutes to rebuild, provided that you specify make -j10 for instance.

I think the best way to handle these cases is to have a "library path" setting and leave value up to whoever packages the TSDuck for a specific distribution. For example, see cmake GnuInstallDirs which lets you change the installation paths for various types of files.

the best way to handle these cases is to have a "library path" setting and leave value up to whoever packages the TSDuck for a specific distribution.

It already exists using make install USRLIBDIR=/foo/bar/usr/lib47.
I was looking for a generic reliable default. OK, let's make it simple. I will slightly improve the default behaviour and use lib64 only when /usr/lib64/libc.so* exists. That will be adequate for the Red Hat and Debian families. Then, USRLIBDIR can always be used in specific cases for specific packaging.

One more question, for Debian users.
In the Red Hat family, each package is always identified with the distro (RHEL/CentOS vs. Fedora) and version of the distro in the package file name. Example:

For RHEL/CentOS 8.x : tsduck-3.21-1819.el8.x86_64.rpm
For Fedora 32 : tsduck-3.21-1819.fc32.x86_64.rpm

In the Debian family, I don't see any such convention. Currently, in that family, I produce TSDuck binaries for Ubuntu (64-bit Intel) and Raspian (ARM 32-bit) only. So, the difference is implicit on the architecture:

Assumed to be latest Ubuntu : tsduck_3.21-1819_amd64.deb
Assumed to be latest Raspbian : tsduck_3.21-1819_armhf.deb

If I decide to produce packages for Debian 10.x and Ubuntu 20.x on the same architecture, how would you identify the various packages? Note that GitHub releases provides assets which are different by file name only. The can't be in different directories (possibly with same name).
I can still decide my own convention as follow, but is there any widely accepted convention?

tsduck_3.21-1819.ub20_amd64.deb
tsduck_3.21-1819.deb10_amd64.deb
tsduck_3.21-1819.rasp10_armhf.deb

Hello @lelegard,

@piotrserafin, comparing this with your PKGBUILD in AUR for Arch Linux, I see the following differences:
* use `/usr/lib64` instead of `/usr/lib` on x86_64, does Arch Linux use `/usr/lib64`? why not?

In Arch /usr/lib64 is only symbolic link to /usr/lib but it is worth to follow same convention will change it.

* move `tsduck.so` in `/usr/lib[64]` (you leave it in `/usr/bin`)

Ok, make sense.

* move configuration files in `/usr/lib[64]/tsduck` (you leave them in `/usr/bin`)

Ok.

* move plugins to `/usr/lib[64]/tsduck` (you move them to `/usr/lib/tsduck/plugins` but I do not see any reason to create an additional level of directory)

Same with me.

* no need to create profile script to define `TSPLUGINS_PATH` (but that is the original purpose)

Agreed.
Will adapt those changes this week.

Hi @piotrserafin
Following the exchanges above with @APTX, I modified the placement of configuration files with architecture-dependent and independent files in distinct places. The file layout is now:
/etc/security/console.perms.d/80-tsduck.perms
/lib/udev/rules.d/80-tsduck.rules
/usr/bin/tsanalyze
...
/usr/bin/tsversion
/usr/lib/libtsduck.so
/usr/lib/tsduck/tsplugin_aes.so
...
/usr/lib/tsduck/tsplugin_zap.so
/usr/share/tsduck/tsduck.channels.model.xml
...
/usr/share/tsduck/tsduck.tables.model.xml

Hello @lelegard ,
Great, PKGBUILD will simplify after those changes. This is not yet part of 3.21-1819? When you are planning new release?
Best Regards,
Piotr Serafin

I tested the new way TSDuck installs and it seems to work perfectly out of the box on gentoo.
I noticed two things:

I can not build with dektec in the ebuild, as the build script is not allowed to download anything. I don't really mind this as I do not own any such devices.
The prebuilt packages come with documentation files (like tsduck.pdf). I checked the documentation and I couldn't find anything about installing those. I could just install the files from the doc directory, if there is no install command for documentation.

I can not build with dektec in the ebuild, as the build script is not allowed to download anything. I don't really mind this as I do not own any such devices.

I cannot embed Dektec library in TSDuck source, the static library is in binary form only. Moreover, Dektec regularly releases new versions, typically each time they have new devices. So, you need to download the latest version yourself. This is automated in the make process. If you cannot download Dektec libraries, then you can't build Dektec support. This may be fine for you but is not acceptable as a generic delivery method for Gentoo.
In the DTV industry, a lot of people use Dektec devices. When working in the broadcast area, virtually everyone use Dektec devices. They are relatively inexpensive (for professional devices, compared with production modulators for instance), flexible, versatile, reliable. The perfect lab equipment. (No, I am not paid by Dektec😃). Many people use TSDuck because it directly drives Dektec devices. So, it is not possible to disable Dektec support for all users in a given distro. It can be solely for you own usage.

The prebuilt packages come with documentation files (like tsduck.pdf). I checked the documentation and I couldn't find anything about installing those. I could just install the files from the doc directory, if there is no install command for documentation.

Unlike binaries or configuration files, each distro has its own way of storing docs and license files. So, the docs are not installed in the generic make install. You have to handle that case by case. For RPM-based distros for instance, the docs are provided in a %doc directive in tsduck.spec. If there is some equivalence with Gentoo, you have to specify it in ebuild. I have never used Gentoo and I have no idea.

Is there a way to build TSDuck with dektec without TSDuck trying to download anything? The build is done in a sandbox and downloading files is not allowed. I could make a separate ebuild for the dektec library, but I'm not sure how to make TSDuck use that or how that works regarding license. I also can't really test any of it beyond making sure it builds.
I'll see what I can do with the documentation. I assumed I would have to install it "by hand", but I was just wondering if there was an install-docs or similar.

Is there a way to build TSDuck with dektec without TSDuck trying to download anything? The build is done in a sandbox and downloading files is not allowed. I could make a separate ebuild for the dektec library, but I'm not sure how to make TSDuck use that or how that works regarding license.

You may download the Dektec Linux SDK somewhere first. This is a file named LinuxSDK_v2020.05.1.tar.gz for instance. The version number is in the file name. Then, build TSDuck with make DTAPI_ORIGIN=/your/path/LinuxSDK_v2020.05.1.tar.gz.
I use that variable to rebuild TSDuck from scratch in empty places and avoid downloading the Dektec SDK again and again. You can it for the same reason.
You can use the script dektec/dtapi-config.sh to get the URL:
$ dektec/dtapi-config.sh --url
https://www.dektec.com/products/SDK/DTAPI/Downloads/LinuxSDK_v2020.05.1.tar.gz

I also can't really test any of it beyond making sure it builds.

Honestly, if it builds, it will work. And if it does not work, it won't be because of your environment. I started using Dektec device around 2003 with their very first DTA-100 ASI output device and I trust their software.

I'll see what I can do with the documentation. I assumed I would have to install it "by hand", but I was just wondering if there was an install-docs or similar.

There is no make install-docs  because, honestly, I have no idea where to install the docs in a place which would satisfy all distros...

I came across a DekTec modulator and tried using it with TSDuck.
Just building TSDuck with the DekTec modulator is not enough to use it. The SDK also contains sources for linux drivers in the form of kernel modules that need to be installed for the device can be detected by TSDuck. The license in the DekTec linux SDK appears to be 2-clause BSD, so it should be possible to make a package for it as well. Is there a way to make TSDuck use an extracted version of the DekTec SDK? If not I probably could have both packages download it.
I've tried to output to the modulator, but as TSDuck says ISDB-T is not supported I tried DVB-T instead. I couldn't use the output but I assume that it works.
I've solved the docs issue.

Although packaged by Dektec in the same zip file, the SDK and the drivers have distinct usages and should have distinct distribution channels or packages.
The SDK is statically linked with the application (Dektec provides a .o file only). So, it ends up inside libtsduck.so and is distributed everywhere, whether you actually use Dektec devices or not.
The drivers should be packaged as a DKMS module which is installed only on systems where you actually use Dektec devices. The job is already done for .rpm and .deb packaging. If you want to add another type of packaging for other distros, you may adapt this or, better, contribute to add the new packaging in the dektec-dkms project.

I've decided to follow your suggestion and have a kmod package and have the TSDuck package use the DekTec SDK by itself.
Just to clear up one thing I initially wanted to make a "DekTec SDK" package that would install the kernel modules and DTAPI files, not have TSDuck install the kernel modules.
When building TSDuck with DekTec this warning appears:
 * QA Notice: The following files contain writable and executable sections
 *  Files with such sections will not work properly (or at all!) on some
 *  architectures/operating systems.  A bug should be filed at
 *  https://bugs.gentoo.org/ to make sure the issue is fixed.
 *  For more information, see:
 *
 *    https://wiki.gentoo.org/wiki/Hardened/GNU_stack_quickstart
 *
 *  Please include the following list of files in your report:
 *  Note: Bugs should be filed for the respective maintainers
 *  of the package in question and not<EMAIL_ADDRESS> * --X --- --- usr/lib64/libtsduck.a:DTAPI64.o
 * RWX --- --- usr/lib64/libtsduck.so

The warning suggests this already, but I did make sure that the issue is with the DTAPI object file itself. I'm not sure what can be done about this, if anything.
TSDuck and its DekTec support is working despite the warning.

The DTAPI library is shipped by Dektec in binary format, "as is". If there is something that does not make the QA check happy, then we have to live with it. No other option.

Automatically closed after 820 days without update and "close pending" label set.


Unable to re-sign old commits

Error
Run: dco sign
`command': git '--git-dir=/Users/dmw/repos/cookbooks/sous-chefs/php/.git' '--work-tree=/Users/dmw/repos/cookbooks/sous-chefs/php' ls-files '--stage' ''  2>&1:fatal: empty string is not a valid pathspec. please use . instead if you meant to match all paths (Git::GitExecuteError)

Environment ChefDK
chef --version
Chef Development Kit Version: 2.4.22
chef-client version: 13.6.4
delivery version: master (73ebb72a6c42b3d2ff5370c476be800fee7e5427)
berks version: 6.3.1
kitchen version: 1.19.2
inspec version: 1.48.0

dco-1.0.1

Which folder were you in when you ran the command?

and also which version of git is installed :)

Hey,
In folder:  /Users/dmw/repos/cookbooks/sous-chefs/php/
git version 2.16.1
I ran it from another Mac and it worked. I "believe" they're setup the same, with the same weird direnv setup and their PATHS are definitely the same.
Both using the same version of dco, from the same location.
I'll check the git version on the other mac tomorrow morning.
Thanks

Git version on the machine that works:
2.14.3 (Aple Git-98)

So yes, it's  that, the newer version of git seems to break it. Reverting to 2.14.3 allows sign off.


make webpart controls usable in propertypane

Thank you for reporting an issue, suggesting an enhancement, or asking a question. We appreciate your feedback - to help the team understand your
needs please complete the below template to ensure we have the details to help. Thanks!
Please check out the documentation to see if your question is already addressed there. This will help us ensure our documentation is up to date.
Category
[ ] Enhancement
[ ] Bug
[x ] Question
so why do w have. separate repo  for property pane controls , an another for controls to be used in the body of a webpart? is it possible to encapsulate a controls that can be used in a webpart so that it could also be used in  a property pane? could save a lot of effort.
Thanks!

The reasoning is that all the property pane controls are by default React.
The controls you'll be using could be written in the language of your preference. If you, for example, use Angular, you will not be able to use the React controls directly in your solutions.
That is why we split these two projects. The property pane controls will stay React only, but for the body controls, we could in the future create new repositories to support other languages, depending on the interest.


Remove physfs from .gitmodules

The Physfs dep has been removed, but it is still listed in .gitmodules. This is confusing my third party git client very badly.

Oops, thanks!


LSP_train.py - Training stuck at 0/num_optimum_steps

Issue :  The pbar for training is stuck at 0/num_optim_steps. Could be seen in the image below.
When i clone the repository and run python demo.py everything works find until the training loop. Then the pbar is stuck at 0/num_optim_steps meaning that the training does not take place. By default python demo.py takes --data = dummy_data.

Turned from_scratch = True

After the while loop at line 273 in LSP_train.py, printed a statement if that line executes. The code skips all lines after line number 290. This is the place the skip starts

if args.no_token_id:             token_ids = None         loss, ppl = model(input_ids, position_ids, token_ids, label_ids)         logger.info("loss", loss)         logger.info("ppl", ppl)         logger.info("if this command print than line number 294 has run")
Is there something obvious that i am missing ?

Hi, I'm having the same issue. Did you perhaps figure out how to solve it? Thanks!


Update main.yml

set ubuntu1604cis_config_aide to false for pci ntp remediation and fix ntp time sync

@sjatearnin could we please get your approval on this so that we can relaunch the affected instances and continue with the Cadence audit?


Wait for twelve L1 confirmations before updating L2 data.

fixes #440
This is a significant change.  It adds the latest version of the ping-pong tests, but that does not, of itself, affect mainstream code.  However, to get our code to run reliably on Ropsten, this PR implements code to wait for 12 confirmations of a blockchain transaction before updating layer 2.
Our code won't run reliably on Ropsten without this PR.
The changes occur in two places:

In the Nf3.submitTransaction method.  The returned Promise now resolves after twelve confirmations (it used to resolve as soon as a transaction receipt was available). This uses web3js's logic, which seems to be able to cope with a chain reorg and doesn't require further input from the user (the effect of a chain-reorg is not documented, but experimentally it seems to wait for the transaction to be re-mined if a re-org occurs and then continue to wait for 12 confirmations of the re-mined transaction).
The event queue. When an event is pushed into the queue it is bundled with code that will wait until the event has been confirmed 12 times. Thus, when the event reaches the front of the queue and runs, the Promise won't resolve until the event has been seen twelve times.  Web3js doesn't directly handle a chain reorg in this circumstance, so we handle it ourselves: If, during this wait, the event is removed as part of a chain reorg, then the event is disregarded.  It will then be re-mined and re-added to the event queue, and treated as a new event.

The tests are now slower, because they wait for the 12 confirmations.  Down the road, we should revisit our original method of dynamically changing the L2 database in response to chain reorgs because this approach is more responsive.  However, this will take more time than we have available now.
To test - you can test in the usual way (./start-nightfall, npm t) but to test the ping-pong code specifically, do the following:
cd /test/ping-pong
./pong-down -v # clears any volumes and containers from a previous run
./ganache-standalone -s # starts a mining ganache instance exposed to localhost, for the test to use
./pong-nightfall -d -s # starts nightfall, using the local ganache chain, deploys contracts and uses stubs

Then, in a separate terminal, use run
./pong-apps

The readme in the test/ping-pong directory has further details, e.g. for running on Ropsten.
NB: You will need to bring these containers down before you can run conventional tests.  To do that:
./ganache-standalone -d
./pong-down -v

Yes, this is true but seems to be a fairly fundamental limitation of the 12 confirmations approach.  We discussed and can't see a viable way around it.  The situation will improve considerably after the merge because of the faster PoS confirmation.


🌐 Add Chinese translation for docs/zh/docs/project-generation.md

Thank you! @waketzheng :cat:
And thanks for the reviews @Aruelius @nick-cjyx9 :100: