Add an option to use Huggingface metrics

🚀 Feature
Support Huggingface metrics.
Motivation
Torchmetrics are great, but there are many metrics that are not publicly available. Luckily, Huggingface implemented lots of them. Can you please an easy way to add metrics from Huggingface?
Pitch
Specifying a metric from Huggingface, making sure I give it the correct arguments without needing to implement it on my own.

@yuvalkirstain I think this is a great idea! But after adding a metric to torchmetrics (link), I'll be the first to say that this might be a little tricky. There's a lot refactoring going on.

How did we want to swap metrics? I'm imagining that in the yaml, it looks like:
metrics:
  accuracy:
    path: torchmetrics.Accuracy
  recall:
    path: torchmetrics.Recall

And we can swap it by switching it to:
metrics:
  accuracy:
    path: datasets.metrics.Accuracy
  recall:
    path: datasets.metrics.Recall

If there's a custom metric, we can format it like:
metrics:
  MyCustomMetric:
    path: abs/path/to/metric.py

Where it tries to import MyCustomMetrics from abs/path/to/metric.py
Also, with a few metrics, the targets/preds order are switched or are in the process of being switched. So it might be good to generalize metric as a dataclass, just like what HuggingFace is doing, where the output is a callable dictionary. All thoughts on this are welcome!

@mathemusician with only looking at the huggingface metrics API, I think it should be relatively easy to give an adapter for that like this:
class HFMetricAdapter(torchmetrics.Metric):
    def __init__(self, hf_metric):
        super().__init__()
        self.hf_metric = hf_metric

    def update(self, *args, **kwargs):
        self.hf_metric.add_batch(*args, **kwargs)

    def compute(self):
        return self.hf_metric.compute()

That being said, I don't know much about huggingface metrics and therefore I am not sure, whether all the magic torchmetrics does in the background to properly sync and accumulate the metrics state across different GPUs (which would be required for correctness when training on multiple GPUs) will work. For that to work, huggingface would have to take care of that.
@yuvalkirstain @mathemusician
What Metrics specifically would you like to see? Maybe we can add more of them to torchmetrics for a more native experience?

@justusschock Good point. It might actually be easier to migrate as many metrics from HuggingFace, then use our hooks, rather writing workarounds for using someone else's hooks. @yuvalkirstain Any thoughts?

@mathemusician @justusschock
Huggingface metrics is very popular and might have metrics that can not be found in TorchMetrics, or implementations that slightly differ from those found in TorchMetrics. Additionally, people will probably add new metrics to Huggingface metrics. Therefore, I really like the HFMetricAdapter workaround. I find this to be an orthogonal effort to adding more metrics to TorchMetrics.

@mathemusician @justusschock
Huggingface metrics is very popular and might have metrics that can not be found in TorchMetrics, or implementations that slightly differ from those found in TorchMetrics. Additionally, people will probably add new metrics to Huggingface metrics. Therefore, I really like the HFMetricAdapter workaround. I find this to be an orthogonal effort to adding more metrics to TorchMetrics.

@yuvalkirstain Understood, as stated, with this adaptor we, however, cannot guarantee that everything works fine regarding synchronization.
And if we start implementing new torchmetrics Metrics now, I think there is a high chance that new metrics can be implemented to it quickly as well. I see a huge benefit there, because this is not tied to one ecosystem.

@justusschock Yes, I understand. Thank you so much for the help.


ENH: stats: replace ncx2 stats distribution with Boost non_central_chi_squared

Reference issue

Closes gh-11777
What does this implement/fix?

adds Boost non_central_chi_squared implementation to stats

Additional information

Hmm IIRC an overflow warning has been reported for another Boost distribution on some platforms, and another one had a divide by zero. Despite the warnings, the results were accurate. Do we know what's going on here? I don't think these are supposed to be numerically challenging cases.

Looking into it now

Other than the need to replace the pdf to fix CI and close the issue, LGTM!

This test C++ application compiles and runs fine:
#include <iostream>
#include "boost/math/distributions/non_central_chi_squared.hpp"

int main() {
    const double nc {1.0560466};
    const double df {21};
    const double x {21.36270226};
    auto ncx2 = boost::math::non_central_chi_squared_distribution<double>(df, nc);
    std::cout << boost::math::pdf(ncx2, x) << std::endl;
    return 0;
}

So I'm convinced that RuntimeWarning: overflow encountered in _ncx2_pdf warning is from numpy or the ufunc itself.  Still haven't found a good way to track it down, but I'm still digging

Using the same solution as that done in gh-14901 to suppress false positive RuntimeWarnings.  If CI is green and there are no other comments, then I think this PR is ready

: (

Fingers crossed that's the last warning to ignore

Please see mckib2/scipy#32.

So I'm convinced that RuntimeWarning: overflow encountered in _ncx2_pdf warning is from numpy or the ufunc itself.

It's possible that numpy is generating the warning but Boost is still to blame because it's setting the relevant flag (https://en.wikipedia.org/wiki/Integer_overflow#Flags).

@rgommers Not sure how to go about this -- if this is indeed an upstream Boost issue, I'd like to report it, but it seems unhelpful to say "somewhere in your stats implementation you're setting an overflow flag".  Do you know of any good ways to track this down?  Looks like there are some GCC buitins for identifying overflows, but it seems nonstandard and I think it would require changing all arithmetic operations
I did try compiling the code snippet above with -ftrapv -O0 and checking errno, but nothing is reported.  Do you know what numpy is looking at to know there was an issue?

NumPy just checks the floating point error flags.  If it reports an overflow, that means there is an inf created somewhere from finite numbers.
Now, there are a two reasons I can think of as to why:

A temporary calculation:  The inf is later followed e.g. by 1/inf which is makes the result reasonable.  If this is the case, fixing it might be tricky and involve specifically avoiding creating the inf in the first place.  (there might be other ways)
A SIMD operation is working on an incomplete load.  E.g. the SIMD register has 4 slots, but only 1 is used.  In that case, the 3 may stay undefined and trigger FPE warnings because they are doing calculations you don't even need.  This can be due to two reasons:

Manually written SIMD code, may need to ensure to avoid this.
The compiler might just spit out such code on its own.  Especially clang likes to I think (we have a few clang workarounds), and without -ftrapping-math (which newer NumPy uses), clang doesn't do a good job even on newer versions. I think.

Both sounds like "boost issue" to me, although it might be that boost has to work around compiler deficiencies to fix it.

If you know that floating point warnings should never happen and you are calling a boost function from the ufunc inner-loop (you are writing the ufunc inner-loop), you can call npy_clear_floatstatus_barrier() at the end of the ufunc-loop as a work-around.
That is not really a quite clean solution and a noticeable slowdown in some corner cases, but it may well be the pragmatic approach.

This fixes a bug, so I'll go ahead and merge. When we figure out the root cause of gh-14901, we should try to remove the warning suppressions here. Thanks @mckib2!


No Matrix Problems

Please add some relevant problems from 2-D matrix to the repo

Can you please assign me with the issue.. I can add some good 2-D matrix problems to the repo


Provide BaseModuleDispatcher with iterator for hosts

@jladdjr can you rebase this PR?  I have a feeling it may not be needed anymore, but would be good to confirm.

@jlaska - sure, will get this rebased

@jladdjr Is this still valid, or of interest?

@jlaska - don't know that we've been hurting to have this in lately. If this comes back up, can just open a new PR.


pygobject: 3.24.1 → 3.26.1

Motivation for this change
Just an update, trying if it does not fix syncthing-gtk freeze. It does not.

https://github.com/GNOME/pygobject/blob/pygobject-3-26/NEWS
https://pycairo.readthedocs.io/en/latest/changelog.html#v1-15-4

Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

The thing I do not like about this is the postPatch section in the pycairo expression. Is there a better way to set the install_base? How do other python packages solve this?

Verified by running d-feet.


Serialize amcrest snapshot commands and bump PyPI package to 1.2.4

Description:
Attempting to send a snapshot command when a previous one hasn't finished will result in warnings and/or errors. This can happen when the camera picture is clicked on in the frontend, resulting in the thread that updates the thumbnail in the background every 10 seconds to sometimes collide with the thread that updates the large picture in the foreground quickly. An automation that calls the camera.snapshot service in yet another thread can make the situation worse. Fix by adding a thread lock to serialize snapshot commands. Also bump the amcrest package to 1.2.4 which fixes error handling in the command method and improves performance by reusing requests sessions.
Related issue (if applicable): None
Pull request in home-assistant.io with documentation (if applicable): None
Example entry for configuration.yaml (if applicable):
No change to configuration.
Checklist:

[x] The code change is tested and works locally.
[x] Local tests pass with tox. Your PR cannot be merged unless tests pass
[x] There is no commented out code in this PR.

You deleted the part of the checklist that relates to the action that you forgot ...
Change looks good 👍

@amelchio

You deleted the part of the checklist that relates to the action that you forgot ...

What part/action was that? I deleted everything I though wasn't applicable.

There is (was) a whole section on new dependencies. You have to update requirements_all.txt as well for the Travis test to pass.

Ah, ok. Sorry, I missed that. Should be good now. Thx!

Hi,
there is no 1.2.4 in pypi. Just 1.2.3 and 1.2.5.
BR
Markus

Yes, I know. It was a misunderstanding between me and the maintainer.
Working on a new PR now to change to 1.2.5.
On Wed, Mar 6, 2019 at 3:42 PM Markus Jankowski<EMAIL_ADDRESS>wrote:

Hi,
there is no 1.2.4 in pypi. Just 1.2.3 and 1.2.5.
BR
Markus
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/home-assistant/home-assistant/pull/21664#issuecomment-470288527,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AiWM2XAK93IxvzN9h7Q_naPD6d-B1uClks5vUDY1gaJpZM4bdRoo
.


Add a concrete example on how to restore a cert-manager installation

The /docs/devops-tips/backup page was already describing how to backup using Velero.
However, there was no concrete example that shows how to do this.
This PR adds that example (including a workaround to properly restore the owner reference on a Certificate created by ingress-shim).
You can use Velero locally using these commands (see https://velero.io/docs/v1.12/contributions/minio/):
kubectl apply --server-side -f ./examples/minio/00-minio-deployment.yaml

./velero install \
    --provider aws \
    --plugins velero/velero-plugin-for-aws:v1.2.1 \
    --bucket velero \
    --secret-file ./credentials-velero \
    --use-volume-snapshots=false \
    --backup-location-config region=minio,s3ForcePathStyle="true",s3Url=http://minio.velero.svc:9000

/unhold


Calling out the use of backtick

David,
Thank you for the book.  I am enjoying my copy from No Starch.
Would it be useful to explain when we should be using backticks?  Specifically in Section 5 - Designing Effective Tables: Use Color intentionally?
While the code examples clearly show backticks and backticks are mentioned in the Markdown section, I did not see reference to backticks when referencing columns names vs. the single quotes for string literals.  It is possible that I misssed it.
Thank you for writing the book David.
-Michael

Good question! I assume you're referring to code like this:
select(country, `1952`, `1972`, `1992`)

The reason you need backticks is that R technically can't have variable names that start with numbers. So, as a result, the way to make those variable names work is to add back ticks before and after the numbers.
Does that help?


Fixes for stock modal and notes propagation to tracking items

I noticed that the modal was losing stock item information if not valid in the POST request (see missing stock item up top):

This PR fixes it:

Also the "Notes" field would not be propagated to the stock tracking items during a "move" action, this PR fixes this too

Thanks :)


Fix normalized config key for models architecture

Replace key of _conf with underscore to dash so that we have the same format as the TasksManager._SUPPORTED_MODEL_TYPE https://github.com/huggingface/optimum/blob/6d1ae0eb9d3660b46f1b2e705ff2f61b38baacd6/optimum/exporters/tasks.py#L281 Currently there is a mix between the two which makes this model https://huggingface.co/hf-internal-testing/tiny-random-BlenderbotSmallForCausalLM/blob/main/config.json#L27 not compatible for example

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


Empty folder using docker compose volumes mount

Describe the bug
Using docker compose volumes is not working properly. Using native docker run -v $(pwd):/srv image works good.
To Reproduce

Prepare docker-compose.yml.

version: "3.5"
services:
    app:
        image: debian:bullseye
        command: ["tail", "-f", "/dev/null"]
        volumes:
           - .:/srv

Run docker compose up.
Run docker exec -it folder-app-1 ls -la /srv

Expected behavior
See some files & folders in /srv.
Unexpected behavior
I see empty /srv folder.
Environment
MacOS: 13.4.1 (22F82)
Apple M1 ultra
❯ orb version
Version: 0.14.0 (140000)
Commit: 9a20727a93c7b42eb2e7b23cf815c2ecef334c76 (v0.14.0)

Can't reproduce:
❯ docker exec -it srvtest-app-1 ls -la /srv                                                                                ~
total 4
drwxr-xr-x 3 root root  96 Jul 11 12:30 .
drwxr-xr-x 1 root root   0 Jul 11 12:30 ..
-rw-r--r-- 1 root root 148 Jul 11 12:30 docker-compose.yml

Do you get the same result if you restart the Compose service? What does docker inspect srvtest-app-1 show?

Oh I see the problem using docker inspect folder-app-1.
I use /opt/code as symlink to my user folder. Therefor docker compose from /opt/code does not work, but starting from ~/Data/code/ works.
Should I add some settings somewhere?

I think it's same as for Docker Desktop -> https://docs.docker.com/desktop/settings/mac/#file-sharing

Got it, thank you for investigating.

Fixed for the next version.

Released in v1.4.0.


Build failure: atop (cross)

Steps To Reproduce
nix build nixpkgs#pkgsCross.aarch64-multiplatform.atop or nix build nixpkgs#pkgsCross.armv7l-hf-multiplatform.atop
Build log
Build log
Additional context
Bisected the breakage to https://github.com/NixOS/nixpkgs/pull/325704. I haven't bisected upstream yet.
Metadata

system: "x86_64-linux"
host os: Linux 6.6.59, NixOS, 24.05 (Uakari), 24.05.20241104.d063c1d
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.18.8

Notify maintainers

@7c6f434c @therishidesai

Note for maintainers: Please tag this issue in your PR.

Add a :+1: reaction to issues you find important.

I think the atop problem began in Atoptool/atop@8f79c7f8f065cd99e828d66e00ba3210d419ba8e. From the nix build log:
/run/current-system/sw/bin/bash: line 1: pkg-config: command not found

It turns out the binary is prefixed (e.g. aarch64-unknown-linux-gnu-pkg-config) when cross-compiling, so pkg-config can't be found.
But it looks like the prefixing was deliberately done in https://github.com/NixOS/nixpkgs/pull/292388. @NickCao Can you advise on the right approach here?

Upstream should use the pkg-config binary specified in the PKG_CONFIG environment variable, and fallback to pkg-config by default.

Thanks! It looks like that was already done in Atoptool/atop@eec97d53d3ed1c2cba036b90e4c7bd2409658080, but that alone didn't fix it, which confused my bisect. It looks like pkg-config has to be moved to nativeBuildInputs. https://github.com/NixOS/nixpkgs/pull/360488 fixes it.


Gateway entities: services, consumers, plugins, consumer groups

Preview Links
Updating entities that already have pages:

Services
Consumers
Plugins

And adding a new page for Consumer Groups.
To do:

~Migrate consumer groups how-to guides~
~Look for how-to guides for the other entities~
Improve offshoot plugin pages
FAQs for services, consumer groups, plugins
improve the examples

https://deploy-preview-172--kongdeveloper.netlify.app/gateway/entities/consumer-group/
https://deploy-preview-172--kongdeveloper.netlify.app/gateway/entities/consumer/
https://deploy-preview-172--kongdeveloper.netlify.app/gateway/entities/service/
https://deploy-preview-172--kongdeveloper.netlify.app/gateway/entities/plugin/
Notes and questions

{{ variable.name}} doesn't seem to work in yaml consistently
plugins page: does it still make sense to add a generic plugin config?
Notes styling: it's fine that it doesn't exist yet, but is there a format that we should follow that'll eventually get styled?
Something weird going on with arrows in mermaid: https://deploy-preview-172--kongdeveloper.netlify.app/gateway/entities/consumer-group/
How far does the "Gateway Service" term extend? Should we use it in page titles like https://deploy-preview-172--kongdeveloper.netlify.app/gateway/entities/service/?
Is it possible to generate https://deploy-preview-172--kongdeveloper.netlify.app/plugins/scopes/ now, and can we (authors) write these generators, if possible?

Checklist

[ ] Every page is page one
[ ] Tested how-to docs. If not, note why here.
[x] All pages contain metadata
[ ] Updated sources.yaml. For more info review track docs changes
[x] Any new docs link to existing docs.

{{ variable.name}} doesn't seem to work in yaml consistently
where is this happening? we need to apply the liquify filter to it, i.e. {{ variable | liquify }}

Notes styling
We have the following styles for admonitions

Something weird going on with arrows in mermaid
I fixed it on your branch :)


Schema mismatch on fetch node

According to our schema a port's default value should be string|undefined
However in the fetch component we set the default for the headers property to be an object
Not sure which is more correct, but the net result is that the Visual Editor is expecting string|undefined but in this case gets {} which then gets serialized to [object Object]

This is my bug. I defined default as "string", because that's what examples type is, but actually, JSON schema doesn't have a strong opinion here, and having default be of the same type of as its port seems more logical.
The fix here is likely to change default to be of unknown type and rely on Build API to infer the proper type. Right?

Headers should be Record<string, string> in TypeScript, or
{
  "type": "object",
  "additionalProperties": {
  }
}

The fix here is likely to change default to be of unknown type and rely on Build API to infer the proper type. Right?

Yeah, I agree the default type for a Breadboard port should be any JSON value which is:
Breadboard Type Expression:
"unknown";

TypeScript:
type JsonSerializable =
  | string
  | number
  | boolean
  | null
  | Array<JsonSerializable>
  | { [K: string]: JsonSerializable };

JSON Schema:
{}

OR identically:
{"type": ["array", "boolean", "null", "number", "object", "string"]}


WMS683 - Quarterly QA check - Minor updates

WMS 683
Put all of the info in the WMS system regarding the QA info.
This version contains minor changes to adhere to the Lintchecker.

@rpastijn can u check the conflicts please?

There are some conflicts with your PR. Please resolve these and we will review your PR.


Request for new child term(s) and definition edit of GO:0072588, Box H/ACA RNP complex

Hi,
I have been annotating the role of the box H/ACA RNP complex core proteins in the assembly of human telomerase, using information in PMID 22527283. I would like to suggest a new child of the following term, in order to better annotate this specific RNP complex:
GO:0072588 Box H/ACA RNP complex Ontology Cellular Component Definition A ribonucleoprotein complex that contains an RNA of the box H/ACA type, a subtype of the small nucleolar RNA (snoRNA) family. RNA pseudouridylation (isomerization of uridine to pseudouridine) is the major, and most likely the ancestral, function of H/ACA RNPs, although some have evolved other functions. Pseudouridylation targets include both large and small ribosomal RNAs (rRNAs), and on U2 small nuclear RNA (U2 snRNA).
Current child terms:
box H/ACA snoRNP complex GO:0031429
box H/ACA scaRNP complex GO: 0072589
The complex formed between the telomerase RNA and these same proteins is also a child of GO:0072588 Box H/ACA RNP complex, but the telomerase RNP is not a snoRNP or a scaRNP. Furthermore, both snoRNPs and scaRNPs have pseudouridylation synthase activity, but the telomerase RNP is one of several box H/ACA RNPs that does not have this catalytic activity.
Therefore I would like to suggest the following new child Cellular Component term of GO:0072588 Box H/ACA RNP complex:
box H/ACA telomerase RNP complex
Definition of new term, suggestion (or better):
A box H/ACA ribonucleoprotein complex that contains the RNA component of telomerase, the enzyme essential for the replication of chromosome termini in most eukaryotes. This ribonucleoprotein complex is a structural box H/ACA RNP, which does not have the catalytic pseudouridylation function shared by the majority of H/ACA RNPs present in the cell.
Reference: recent review PMID 25590339, in which the authors state
“All H/ACA RNPs consist of one name and function specifying H/ACA RNA and the same 4 core proteins (Fig. 2). The core proteins are essential for the stability of all H/ACA RNAs and of each other, and for catalysis of pseudouridylation. Surprisingly, it appears that for the majority of H/ACA RNPs, at least in mammalian cells, the enzymatic activity of the pseudouridine synthase is irrelevant but its function in maintaining the RNAs is paramount. These structural classes of H/ACA RNPs include the telomerase RNP, intron-encoded Alu (AluACA) RNPs and small nucleolar-long noncoding (sno-lnc) RNPs (Fig. 2)13-15.Although it is not clear on which side the orphan H/ACA RNPs will come down, structural H/ACA RNPs outnumber catalytic ones in diversity by about 4 to 1, whereas the catalytic RNPs are far more abundant in terms of copy number".
Also, I suggest adding to the tree for GO:0072588 Box H/ACA RNP complex to distinguish between these 'catalytic' and 'structural' types of box H/ACA RNP complexes as summarised in Fig. 2 of PMID 25590339. Two ways this could be achieved are suggested below (and/or/better), also see attached diagram:

Create two more Cellular Component children of GO:0072588 box H/ACA RNP complex, something like:

box H/ACA RNP complex with pseudouridylation synthase activity
box H/ACA RNP complex without pseudouridylation synthase activity

Create links between the function term GO:0009982 pseudouridine synthase activity and both of the existing terms below:

box H/ACA snoRNP complex GO:0031429
box H/ACA scaRNP complex GO: 0072589

Finally, perhaps it would be good to update the current definition of GO:0072588  box H/ACA RNP complex as follows, to address the above issue and also the fact that not all box H/ACA RNAs are snoRNAs, suggestion below (or better):
GO:0072588 Name box H/ACA RNP complex Ontology Cellular Component Definition A ribonucleoprotein complex that contains an RNA of the box H/ACA type. RNA pseudouridylation (isomerization of uridine to pseudouridine) is the major, and most likely the ancestral, function of H/ACA RNPs. Pseudouridylation targets include both large and small ribosomal RNAs (rRNAs), and small nuclear RNA (U2 snRNA). In addition to these catalytic H/ACA RNPs, a less abundant but more diverse class of structural H/ACA RNPs exists, which does not have pseudouridylation activity. These include the telomerase RNP complex.
Thanks,
Jess Buxton
GOC jbu
GOC BHF
GOC BHF_telomere
@NancyCampbell

@NancyCampbell

@NancyCampbell

@bmeldal, can you comment please?

Hi Jess and Tanya,
I agree, for the status quo situation adding the GO:NEW box H/ACA telomerase RNP complex makes sense.
As for making two subclasses for the parent term GO:0072588 Box H/ACA RNP complex: If the parent term as well as the three child terms have a sufficiently detailed defs to indicate that they can have catalytic or scaffolding functions I would hold off on creating the two extra intermediate term. If you are sure that the snoRNPs and scaRNPs are always catalytic, then they can have the capable_of link to the activity and the other two terms aren't needed at all but the reasoner will take care of the placements..
Birgit

Hi Birgit and Tanya,
Thanks for your suggestion – so to clarify, would the best solution for now be as follows:

Create one new child term of GO:0072588 Box H/ACA RNP complex:

Box H/ACA telomerase RNP complex
Definition of new term, suggestion (or better):
A box H/ACA ribonucleoprotein complex that contains the RNA component of telomerase, the enzyme essential for the replication of chromosome termini in most eukaryotes. This ribonucleoprotein complex is a structural box H/ACA RNP, which does not have the catalytic pseudouridylation function shared by the majority of H/ACA RNPs present in the cell

Update definition of GO:0072588 Box H/ACA RNP complex, suggestion (or better):
A ribonucleoprotein complex that contains an RNA of the box H/ACA type. RNA pseudouridylation (isomerization of uridine to pseudouridine) is the major, and most likely the ancestral, function of H/ACA RNPs. Pseudouridylation targets include both large and small ribosomal RNAs (rRNAs), and small nuclear RNA (U2 snRNA). In addition to these catalytic H/ACA RNPs, a less abundant but more diverse class of structural H/ACA RNPs exists, which does not have pseudouridylation activity. These include the telomerase RNP complex.

Create capable_of links between the function term GO:0009982 pseudouridine synthase activity and both of the existing child terms:

box H/ACA snoRNP complex GO:0031429
box H/ACA scaRNP complex GO:0072589

4.Create a capable_of_part_of link between the new child term
Box H/ACA telomerase RNP complex and:
The component term GO:0005697 telomerase holoenzyme complex and/or part_of  (or capable_of_part_of ?)the function term GO:0003720 telomerase activity

That sounds good so far. Couple of comments (I don't know how to reply in-line :( ):
@tberardini: should we have some reference to the proteins of the complexes in the defs? At the moment it only specifically refers to the RNA components.
Relationships:
GO:0031429 box H/ACA snoRNP complex
capable_of GO:0009982 pseudouridine synthase activity
and
GO:0072589 box H/ACA scaRNP complex
capable_of GO:0009982 pseudouridine synthase activity
are fine.
However:
GO:NEW Box H/ACA telomerase RNP complex
should be
capable_of_part_of GO:0032206 positive regulation of telomere maintenance (or a child?)
It can only be part_of GO:0005697 telomerase holoenzyme complex
if ALL its components are also components of the telomerase. If that doesn't hold, then there cannot be a link between the two above terms.
Relationships (for complexes) work as follows:
capable_of
capable_of_part_of
is_a some
part_of some
Hope that helps,
Birgit

Hi Birgit,
that's very helpful thanks - I think GO:NEW Box H/ACA telomerase RNP complex should be
capable_of_part_of GO:1904358 positive regulation of telomere maintenance via telomere lengthening (a child of GO:0032206 positive regulation of telomere maintenance)
The four core proteins that are all part of GO:0072588 Box H/ACA RNP complex are dyskerin, NOP10, NHP2 and GAR1, ref http://www.tandfonline.com/doi/full/10.4161/15476286.2014.972855#abstract

Hi again, on reflection I think this is a more specific term:
GO:NEW Box H/ACA telomerase RNP complex should be
capable_of_part_of GO:0032212 positive regulation of telomere maintenance via telomerase

Want to make sure one last time that we are not creating GO terms for specific protein complexes but rather ones for classes of complexes.  The GO term can then be used to annotate specific complexes from various organisms.

Hi Tanya,
the GO:NEW Box H/ACA telomerase RNP complex would refer to a specific type of H/ACA RNP complex, which contains a specific RNA plus the same 4 core proteins (conserved from yeast through  to mammals) present in all other H/ACA RNPs.  So it is a specific complex but yes could be used to annotate the same complex in other species - Ref fig 2 and the section entitled "H/ACA core proteins" in
http://www.tandfonline.com/doi/full/10.4161/15476286.2014.972855

Summarizing again, for action:

Create one new child term of GO:0072588 Box H/ACA RNP complex:
Box H/ACA telomerase RNP complex
A box H/ACA ribonucleoprotein complex that contains the RNA component of telomerase, the enzyme essential for the replication of chromosome termini in most eukaryotes. This ribonucleoprotein complex is a structural box H/ACA RNP, which does not have the catalytic pseudouridylation function shared by the majority of H/ACA RNPs present in the cell
Update definition of GO:0072588 Box H/ACA RNP complex:
A ribonucleoprotein complex that contains an RNA of the box H/ACA type. RNA pseudouridylation (isomerization of uridine to pseudouridine) is the major, and most likely the ancestral, function of H/ACA RNPs. Pseudouridylation targets include both large and small ribosomal RNAs (rRNAs), and small nuclear RNA (U2 snRNA). In addition to these catalytic H/ACA RNPs, a less abundant but more diverse class of structural H/ACA RNPs exists, which does not have pseudouridylation activity. These include the telomerase RNP complex.
GO:0031429 box H/ACA snoRNP complex
capable_of GO:0009982 pseudouridine synthase activity
and
GO:0072589 box H/ACA scaRNP complex
capable_of GO:0009982 pseudouridine synthase activity
4.GO:NEW Box H/ACA telomerase RNP complex
capable_of_part_of GO:0032212 positive regulation of telomere maintenance via telomerase
what about part_of GO:0005697 telomerase holoenzyme complex?
I will add in the names of the four core proteins into the parent term.  Are these: dyskerin, NOP10, NHP2 and GAR1, the names of the human proteins, yeast?  Thanks.

Hi Tanya, thanks for this - in answer to your points:

Yes the four core proteins that are present in all Box H/ACA RNP complexes are called dyskerin, NOP10, NHP2 and GAR1 in human, and Cbf5 (ortholog of dyskerin), Nop10 Nhp2 and Gar1 in yeast.

I realise I need to clarify a couple of issues that may affect the suggested new definitions of the GO:NEW Box H/ACA telomerase RNP complex and GO:0072588 Box H/ACA RNP complex, and also your suggestion to add a link between GO:NEW Box H/ACA telomerase RNP complex and part_of GO:0005697 telomerase holoenzyme complex.
Although the components of all Box H/ACA RNP complexes are evolutionarily conserved from yeast to humans, the formation of the telomerase RNP complex differs between vertebrates (where a Box H/ACA RNP complex forms with the four core proteins and the TERC RNA) and other taxa (ref Fig 1 of http://www.nature.com/nrm/journal/v7/n7/full/nrm1961.html)
Furthermore, during the assembly of the vertebrate Box H/ACA telomerase RNP complex, the proteins dyskerin, NOP10, NHP2 and NAF1 initially associate with TERC, then NAF1 is replaced with GAR1 in the Cajal body. So in answer to your other points (suggested changes to definitions in capital letters, sorry can't seem to get bold to work):

Create one new child term of GO:0072588 Box H/ACA RNP complex:
Box H/ACA telomerase RNP complex
A box H/ACA ribonucleoprotein complex that contains the RNA component of VERTEBRATE telomerase, the enzyme essential for the replication of chromosome termini in most eukaryotes. This ribonucleoprotein complex is a structural box H/ACA RNP, which does not have the catalytic pseudouridylation function shared by the majority of H/ACA RNPs present in the cell. INITIALLY THIS RNP COMPLEX CONSISTS OF THE TELOMERASE RNA ASSOCIATED WITH THE PROTEINS DYSKERIN, NOP10, NHP2 AND NAF1. THE PROTEIN NAF1 IS REPLACED WITH GAR1 DURING THE BIOGENESIS OF THE TELOMERASE HOLOENZYME.

Update definition of GO:0072588 Box H/ACA RNP complex:
A ribonucleoprotein complex that contains an RNA of the box H/ACA type AND THE FOUR CORE PROTEINS DYSKERIN, NOP10, NHP2 AND GAR1. RNA pseudouridylation (isomerization of uridine to pseudouridine) is the major, and most likely the ancestral, function of H/ACA RNPs. Pseudouridylation targets include both large and small ribosomal RNAs (rRNAs), and small nuclear RNA (U2 snRNA). In addition to these catalytic H/ACA RNPs, a less abundant but more diverse class of structural H/ACA RNPs exists, which does not have pseudouridylation activity. These include the VERTEBRATE telomerase RNP complex.

4.GO:NEW Box H/ACA telomerase RNP complex
capable_of_part_of GO:0032212 positive regulation of telomere maintenance via telomerase
what about part_of GO:0005697 telomerase holoenzyme complex?
Yes GO:NEW Box H/ACA telomerase RNP complex could also be part_of GO:0005697 telomerase holoenzyme complex if we are referring to the final version in which GAR1 is present (as stated above although NAF1 is part of the initial telomerase RNP complex, it isn't present in the telomerase holoenzyme), is changing the definition of GO:NEW Box H/ACA telomerase RNP complex as suggested above the best way to do this?
Jess

Hi again,
regarding points 1 and 4 in my previous message, about whether GO:NEW Box H/ACA telomerase RNP complex could also be part_of GO:0005697 telomerase holoenzyme complex - on reflection I think I'm overcomplicating things - I guess provided that the GO:NEW Box H/ACA telomerase RNP complex term is only used to annotate the core proteins found in the mature RNP complex (ie. dyskerin, NOP10, NHP2 and GAR1) then it will be fine to have this link to the new term, using the shorter suggested definition below:
GO:NEW Box H/ACA telomerase RNP complex
A box H/ACA ribonucleoprotein complex that contains the RNA component of VERTEBRATE telomerase, the enzyme essential for the replication of chromosome termini in most eukaryotes. This ribonucleoprotein complex is a structural box H/ACA RNP, which does not have the catalytic pseudouridylation function shared by the majority of H/ACA RNPs present in the cell.

Would one annotate NAF1 to the 'BOX H/ACA telomerase RNP complex' term?  I don't like the definition that states that the complex has ABCD at one point in time and ABCE at another.  Perhaps we need two terms, one for the complex with the NAF1  component and one with the GAR1 component.

I agree - i think the definition of the new term should just be as below, and only dyskerin, NOP10, NHP2 and GAR1 should be annotated to it. Not NAF1, as this is strictly speaking an 'assembly factor' rather than a final component of the complex.
GO:NEW Box H/ACA telomerase RNP complex
A box H/ACA ribonucleoprotein complex that contains the RNA component of VERTEBRATE telomerase, the enzyme essential for the replication of chromosome termini in most eukaryotes. This ribonucleoprotein complex is a structural box H/ACA RNP, which does not have the catalytic pseudouridylation function shared by the majority of H/ACA RNPs present in the cell.

Ok, finally done.  Thanks, everyone, for your help.

Oh, well done, everyone!
Birgit

Thanks very much!


Update dependencies and fix serde related build failure

This supersedes #372 .

Thanks for the help @Byron

Thanks for the help @Byron

Could you publish a patch-level release with that change included?
Currently v2.9.0 fails to build when running cargo install onefetch --force:
error[E0603]: module `export` is private
   --> /home/asaaki/.cargo/registry/src/github.com-1ecc6299db9ec823/onefetch-2.9.0/src/onefetch/info.rs:414:53
    |
414 |     fn serialize<S>(&self, serializer: S) -> serde::export::Result<S::Ok, S::Error>
    |                                                     ^^^^^^ private module
    |
note: the module `export` is defined here
   --> /home/asaaki/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.119/src/lib.rs:275:5
    |
275 | use self::__private as export;
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to previous error

For more information about this error, try `rustc --explain E0603`.
error: failed to compile `onefetch v2.9.0`, intermediate artifacts can be found at `/tmp/cargo-installUwsyX1`

Caused by:
  could not compile `onefetch`

To learn more, run the command again with --verbose.

Could you publish a patch-level release with that change included?
Currently v2.9.0 fails to build when running cargo install onefetch --force:
error[E0603]: module `export` is private
   --> /home/asaaki/.cargo/registry/src/github.com-1ecc6299db9ec823/onefetch-2.9.0/src/onefetch/info.rs:414:53
    |
414 |     fn serialize<S>(&self, serializer: S) -> serde::export::Result<S::Ok, S::Error>
    |                                                     ^^^^^^ private module
    |
note: the module `export` is defined here
   --> /home/asaaki/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.119/src/lib.rs:275:5
    |
275 | use self::__private as export;
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to previous error

For more information about this error, try `rustc --explain E0603`.
error: failed to compile `onefetch v2.9.0`, intermediate artifacts can be found at `/tmp/cargo-installUwsyX1`

Caused by:
  could not compile `onefetch`

To learn more, run the command again with --verbose.

@o2sh Thanks. It works now. 🥳

@o2sh Thanks. It works now. 🥳


Project import generated by Copybara.

Project import generated by Copybara.

Note, I'm just testing the copybara setup to export changes from silo


enhance the tool so that unsigned aggregates can be used

This is actually the default behaviour of ADFS but not the actual tool.

@teamktown has decided that this 'feature' is really current operational posture for ADFS. Unsigned aggregates are already handled. What is not are the trust aspects (aggregate validation) and the elaborated attribute assignment features that ADFSToolkit provides.
It is recommended that anyone testing an aggregate go and use XMLSECTOOL from the Shibboleth Consortium, create a key and sign the aggregate they want to test.
As a result of the ease to which aggregates CAN be signed and that supporting unsigned aggregates is either a rework of already existing ADFS functionality and unsigned aggregates are inherently untrustworthy, this 'feature' is going to be marked WON'T FIX because there is not a problem with requiring the aggregate to be signed for trust purposes.


Document mvp_features_only prominently

The enabling of signxet and mutableglobals in v111 has already caused confusion in cargo-contract (https://github.com/paritytech/cargo-contract/pull/888). wasmi at least doesn't seem to enable these features by default, nor does the rust llvm backend.

wasmi i think does enable them by default, but both substrate's contracts-pallet and soroban disable them


Writing Arrow IPC format panic within encode_dictionary() method due to access out of bound fields vector

It panic at this line trying to access index 0 while &field.fields is a 0 length vector.
I haven't dived too deep to figure out what should be fixed, but turns out it panics when I have a List<Utf8> as Array item (btw: schema definition is Field::new("labels", DataType::List(Box::new(Field::new("labels", DataType::Utf8, true))))
I suspect the above match array.data_type().to_physical_type() will always match on List, and when it recursively invokes encode_dictionary() method with the nested IpcField, it will panic eventually because the innermost IpcField holds an empty fields vector.
Seems like a bug? Not sure if it is similar to this one https://github.com/jorgecarleitao/arrow2/issues/830

Turns out it's a typo in my code messed up the schema...sorry for reporting this wrong issue...

Hey, thanks and no worries! I think we should still not panic and offer a better error message, so I would say we can keep this open and return a OutOfSpec error when this happens with a good description. What do you think?

Actually I noticed I can still repro this out of bound error since previously I commented out all the List fields in my schema...
Anyway I probably will try to dig deeper in the weekend to see if I can have a minimal viable repro, thanks for following up, and feel free to reopen this issue if you like. I will also keep my findings posted here. Thanks! :)


Update network detect

The code can be made shorter by using regexes instead of using string arrays to check for inclusion.

Hi @ukchukx, my sincere apologies for the delayed response. I got caught up with a lot and forgot about this project.
I think not using the regex implementation you're suggesting will make the code much easier to read for new contributors. So, I'll rather keep the existing logic of using string arrays to check for inclusion. Sometimes making things shorter is not really necessary unless it improves something (performance or so).
Thanks a lot for taking your time to do this; great work!


Help Wanted: Build Web App

I want to Help you with this Issue, Can you please share more about this?

Thanks for the offer! We don't have a spec written out, but we need a simple web app to configure some electronics by reading and writing protobufs over the WebBluetooth API. How did you find this project? What's your level of web development experience? I see a React project on your profile?

I am a good enough web developer, I want to contribute to open source, Starting with your project.

I don't know anything about Nerf Modding but, I can research about it.
Currently researching about Protobuf

Hey @ahalekelly
Please Let me know more about this


socket.error when calling create_all()

I was trying to setup flask-s3 for my app but I kept getting a socket.error (either broken pipe or connection reset by peer) when running flask_s3.create_all(app) in a python shell as suggested in the docs. I had already made the bucket and was just trying to upload my static assets.
The fix appears to be to set the (undocumented) config variable S3_REGION to the region in which you created your bucket. However, this caused an error for me when the create_all() function tried to call conn.create_bucket(bucket_name). By changing this call to a get_bucket() call, I was able to execute create_all() without error.

Hi, could you by any chance give a full log and your code changes?

Hi, sorry for taking so long.
I've just tried to setup a simple app to illustrate this, but I've come up against a different problem. When I try to run create_all() I get TypeError: decoding Unicode is not supported with the following stack trace:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jacobfenton/Desktop/Python/create_all_test/venv/lib/python2.7/site-packages/flask_s3.py", line 290, in create_all
    _upload_files(s3, app, all_files, bucket_name)
  File "/Users/jacobfenton/Desktop/Python/create_all_test/venv/lib/python2.7/site-packages/flask_s3.py", line 187, in _upload_files
    bucket, hashes=hashes))
  File "/Users/jacobfenton/Desktop/Python/create_all_test/venv/lib/python2.7/site-packages/flask_s3.py", line 144, in _write_files
    asset_loc)
  File "/Users/jacobfenton/Desktop/Python/create_all_test/venv/lib/python2.7/site-packages/flask_s3.py", line 133, in _static_folder_path
    return six.u('%s/%s' % (static_url.rstrip('/'), rel_asset.lstrip('/')))
  File "/Users/jacobfenton/Desktop/Python/create_all_test/venv/lib/python2.7/site-packages/six.py", line 621, in u
    return unicode(s.replace(r'\\', r'\\\\'), "unicode_escape")
TypeError: decoding Unicode is not supported

I believe that is fixed in commit d8ab184531b650a416066987d552f67902ff7382 - I don't have a Py2 instance available currently, can you test it? Before I push to pypi.

No still getting the same error

Oh for fucks sake python 2, let me fix this then

Just tried again with the latest commit and got no errors, so it would appear the unicode problem has been solved as well as whatever my original problem was.


New 'Unable to convert to javascript' error message showing up

Describe the bug
Getting a python related error message with python experiment off in /beta
To Reproduce
Steps to reproduce the behavior:

Go to /beta editor
Click on open new project
while blocks are still loading, click javascript to change to javascript
See error

Expected behavior
no error
Desktop (please complete the following information):

OS: mac
Browser chrome

@pelikhan

Nice repro, looking into it.

I fixed the error message but underlying is still there. We don't disable the language toggle while the blockly editor.


Show tooltips for total plants and planting density even when there is data

previously we were showing these tooltips only when data was missing
Rachel provided feedback that we should show the tooltips for context all the time
note that total plants tooltip is only relevant for the site level observations

in case this was missed


[Fix] change test module name

It is trivial. change report name 'pr-nnstreamer-apptest'-> 'pr-audit-nnstreamer-apptest'
Signed-off-by: sewon.oh<EMAIL_ADDRESS>
[Template] PR Description
In general, github system automatically copies your commit message for your convenience.
Please remove unused part of the template after writing your own PR description with this template.
$ git commit -s filename1 filename2 ... [enter]

Summarize changes in around 50 characters or less

More detailed explanatory text, if necessary. Wrap it to about 72
characters or so. In some contexts, the first line is treated as the
subject of the commit and the rest of the text as the body. The
blank line separating the summary from the body is critical;
various tools like `log`, `shortlog` and `rebase` can get confused
if you run the two together.

Further paragraphs come after blank lines.

**Changes proposed in this PR:**
- Bullet points are okay, too
- Typically a hyphen or asterisk is used for the bullet, preceded
  by a single space, with blank lines in between, but conventions vary here.

Resolves: #123
See also: #456, #789

**Self evaluation:**
1. Build test: [ ]Passed [ ]Failed [*]Skipped
2. Run test: [ ]Passed [ ]Failed [* ]Skipped

**How to evaluate:**
1. Describe how to evaluate in order to be reproduced by reviewer(s).

Add signed-off message automatically by running **$git commit -s ...** command.

$ git push origin <your_branch_name>

:memo: Version: 1.20180820. Thank you for submitting PR #308. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://invain.mooo.com/.


Shorten the animation after completing a level

The animation of the commits falling down and bouncing around is cool, but also really annoying when progressing through the levels and you have to wait for it to complete before getting to the next level.
It looks like it executes the animation twice before continuing.  They all fall, bounce, and come to a stop, then they jump, bounce, and come to a stop again, then they finally jump back up into their starting position.

I don't necessarily want to slow down the first animation (since I actually get feedback that its a cute part of the demo), but I would be open to limiting it (or skipping alltogether) once you have solved a few levels.
I'm wondering if I could use browser session for this. How many levels will you solve at a time?

How about stopping it if the user clicks anywhere (or a specific “Next”
button maybe…)?
2015-08-26 23:01 GMT+02:00 Peter Cottle<EMAIL_ADDRESS>
I don't necessarily want to slow down the first animation (since I
actually get feedback that its a cute part of the demo), but I would be
open to limiting it (or skipping alltogether) once you have solved a few
levels.
I'm wondering if I could use browser session for this. How many levels
will you solve at a time?
—
Reply to this email directly or view it on GitHub
https://github.com/pcottle/learnGitBranching/issues/306#issuecomment-135169189
.

Wiring in interaction might be a bit hard since you need to interrupt the in-flight promises as they animate. I'd rather not kick it off all together.
It's also unfortunately dependent on screensize -- the larger your screen, the more distant the commits have to fly before they bounce (and lose energy).
What about the 2nd time the animation plays it only does one round, and the third time not at all?

The first round of animation doesn't take that long and is nice, but the second animation after they have already stopped once gets frustrating if doing more than a couple levels.
Another possibility would be to keep track of how many levels have been completed (and how many times the animation has played) in the last couple minutes and reducing or eliminating based on that.  So if someone is going through slowly, they will always see the full animation, but if you are going quickly (like I was doing just for a refresher) you will get less and won't be slowed down by the animations.
That wouldn't require any session storage.  Just save the time (and maybe level number) in local storage when it is completed, remove any times that are more than a minute or two old, and then determine how much animation to show based on how many times are left.
This could even be extended in the future to show the how long it took to complete the level along with how many commands it took.

Another thought I just had, maybe something like a 'hard mode' where the intro dialogs would be disabled along with success animation.  All you would get is the goal window showing you what you need to do and the dialog at the end that shows how many commands it took and lets you move to the next level.

https://dl.dropboxusercontent.com/u/15917930/finish.mov
Woohoo! What do you guys think of the above?

Basically I increment a counter each time you solve a level, transfer that to a speed multiplier that affects the gravity / drag / elasticity of the physics simulation, and then use that to scale up or down the length of the animation.
It's actually a really nice improvement -- I got annoyed waiting for the animation as well @admsteck when cruising through levels but needed the extra push to get this fix in.
I can even follow up and make it part of local storage too...


Ask for Sonarr's Language Profiles

Is your feature request related to a problem? Please describe.
No
Describe the solution you'd like
As you may know, Sonarr has the possibility of setting the desired Language profile for a show, as opposed to Radarr where you set the desired language in the Quality profile itself.
Can you consider (if possible at all), to ask for the Language profile in Sonarr when approving a request or even setting a default one in the desired Sonarr server/using it in the filters?
Describe alternatives you've considered
Now i just have to edit the language profile in Sonarr once the show is approved, which is not really a hassle, but just asking.
Additional context
None

Yeah this makes sense. We're hoping to roll out a full i18n and translation update in phase 2 so this is definitely something to factor into this.

Closing this issue as its covered by #267


Using user-friendly name for the type parameter in VolumeSnapshotClass

The type parameter in the VolumeSnapshotClass  will be either snapshot, backup, or "" after this change.
The ID in the VolumeSnapshotContent  will be either ss, bs.
longhorn/longhorn#2534

@PhanLe1010 I believe you push to the wrong branch.

pushed back the correct content

Let's use https://github.com/longhorn/longhorn-manager/pull/1240 instead. Closing this.


A proposal for chain initialization, main chain block inclusion, and deposit processing

On-chain contract
We create a deposit contract on the blockchain, with roughly the following code:
HashChainValue: event({prev_tip: bytes32, data: bytes[2048], value: wei_value, total_deposit_count: int128})
ChainStart: event({hash_chain_tip: bytes32, time: timestamp})

hash_chain_tip: public(bytes32)
total_deposit_count: int128

@payable
@public
def deposit(data: bytes[2048]):
    log.HashChainValue(self.hash_chain_tip, data, msg.value, self.total_deposit_count)
    self.total_deposit_count += 1
    self.hash_chain_tip = sha3(concat(self.hash_chain_tip, data, as_bytes32(msg.value), as_bytes32(self.total_deposit_count)))
    if self.total_deposit_count == 16384:
        log.ChainStart(self.hash_chain_tip, block.timestamp)

When a user wishes to move their ETH from the 1.0 chain to the 2.0 chain, they should call the deposit function, sending along 32 ETH and providing as data a SimpleSerialize'd object with the following arguments (in order):

pubkey: int256
proof_of_possession: [int256]
withdrawal_shard: int64
withdrawal_address: bytes20
randao_commitment: hash32

If they wish to deposit more than 32 ETH, they would need to make multiple calls.
[Governance note: when publishing the final version of this contract, it may be desirable to issue some kind of formal EIP that "enshrines" its privileged status, ąnd encourage CarbonVotes and other polls to vote on it, which if successful wouuld make clear to the community that this contract is "part of the protocol" and so the community is responsible for providing ongoing protocol improvements that eventually unlock any ETH sent into this contract]
Chain initialization
When a ChainStart log is published, this initializes the chain, setting the following parameters:

POW_CHAIN_HASH_ROOT (new parameter) = hash_chain_tip
GENESIS_TIME = time
PROCESSED_HASH_ROOT (new parameter) = hash_chain_tip

It runs on_startup with initial_validator_entries equal to the list of data records published as HashChainValue logs so far, in the order in which they were published (oldest to newest).
Chain updating
Define a validator's "view" as being the value obtained by calling DEPOSIT_CONTRACT_ADDRESS.get_hash_chain_tip() from the post-state of the block 512 blocks behind the current head of the PoW chain. Define a "valid view" (defined subjectively from the PoW of a validator) as a value which is a descendant of POW_CHAIN_HASH_ROOT and cannot be obtained by calling DEPOSIT_CONTRACT_ADDRESS.get_hash_chain_tip() from the post-state of a block that is part of the canonical PoW chain at least 512 blocks behind the head. Note that any valid view should be either equal to or an ancestor of the validator's view.
Blocks will have a new data field, hash_chain_tip_vote: hash32, which proposers are expected to fill with the following algorithm:

Let slot B be the last slot during which the POW_CHAIN_HASH_ROOT changed.
If all blocks since slot B contained a hash_chain_tip_vote that was either equal to the POW_CHAIN_HASH_ROOT or was an invalid view, vote the validator's view.
If there was at least one valid view published as a hash_chain_tip_vote since slot B, copy the first valid view.

Note that assuming >= 50% honest, this algorithm will converge to all honest proposers voting the same value, which is a descendant of the POW_CHAIN_HASH_ROOT. If the same value is voted for in >= 683 of the last 1024 blocks, set the POW_CHAIN_HASH_ROOT to this value.
Note that this is a vote, not a consensus rule; blocks with incorrect votes should not be rejected.
Deposit processing
Add a new type of SpecialObject, which consists of the entire hash-linked-list of HashChainValue logs since the previous PROCESSED_HASH_ROOT up to the POW_CHAIN_HASH_ROOT. If a valid such hash-linked-list is submitted, then we run add_validator with the given values for each record, and set PROCESSED_HASH_ROOT = POW_CHAIN_HASH_ROOT.
This does mean that deposit processing is not "automatic", in that deposits are not automatically read from the PoW chain; only a hash is automatically read, and the rest of the data must be manually imported by some block proposer. This is by design, to limit the amount of in-consensus communication between the PoW chain and the beacon chain required to a single hash value.

If I understand correctly, this allows the first proposer submitting a new valid vote to delay deposit processing by 511 additional blocks, as they can vote on the child of POW_CHAIN_HASH_ROOT instead of their actual view and all following proposers would do the same.
I'm wondering if we could fix this by counting votes for a view also towards all ancestors of that view. Then proposers wouldn't have to copy the first vote but could always vote for their view. POW_CHAIN_HASH_ROOT would be updated continuously (but still delayed by 512 blocks compared to the head) and there wouldn't be a special first vote. Sounds too simple, what's the catch?

If I understand correctly, this allows the first proposer submitting a new valid vote to delay deposit processing by 511 additional blocks, as they can vote on the child of POW_CHAIN_HASH_ROOT instead of their actual view and all following proposers would do the same.

Correct. But all that means is that if < 50% of proposers are dishonest, deposit processing will on average be delayed by < 512 additional blocks. So it's not that bad. As soon as you get lucky with one honest proposer, the chain can catch up fully.

I'm wondering if we could fix this by counting votes for a view also towards all ancestors of that view

The problem is that this would require the beacon chain to be aware of the concept of ancestors within that mechanism, which would add considerable complexity.

Minor suggestions for the deposit contract:

Update the size of pubkey and proof_of_possession for the BLS12-381 curve.
Do not cap the number of bytes in data to 300 to be more update-friendly.
Consider removing assert msg.value == as_wei_value(32, "ether") as this is not friendly to deposit top ups, deposit buffers, or other future changes related to the size of the deposit.
Consider changing the constant 10000 to a power of two for consistency (also, name it and add it to the list of constants in the spec).
Consider hashing total_deposit_count into hash_chain_tip so that the total_deposit_count can be readily used by light clients. Bonus: this removes the need for the ChainStart event and the self.total_deposit_count == 10000 check.

Should we consider adding a small proposer reward for adding a well-formatted SpecialRecord that triggers add_validator?

@JustinDrake fixed all except (1) [probably better to run through and update to BLS12-381 all at once at some point]
@djrtwo sounds good to me!

Here is an alternative that allows for the same structure to be used for accepting deposits from the PoW chain and from shards.
The code changes to:
HashChainValue: event({prev_tip: bytes32, data: bytes[2048], value: wei_value, total_deposit_count: int128})
ChainStart: event({hash_chain_tip: bytes32, time: timestamp})

receipt_tree: bytes32[int128]
total_deposit_count: int128

@payable
@public
def deposit(data: bytes[2048]):
    log.HashChainValue(self.hash_chain_tip, data, msg.value, self.total_deposit_count)
    index:int128 = self.total_deposit_count + 2**32
    self.receipt_tree[index] = sha3(concat(data, as_bytes32(msg.value), as_bytes32(block.timestamp))
    for i in range(32):
        index //= 2
        self.receipt_tree[index] = sha3(concat(self.receipt_tree[index * 2], self.receipt_tree[index * 2 + 1]))
    self.total_deposit_count += 1
    if self.total_deposit_count == 16384:
        log.ChainStart(self.receipt_tree[1], block.timestamp)

@public
@constant
def get_receipt_root() -> bytes32:
    return self.receipt_tree[1]

Now, any receipt can prove its membership separately from a Merkle tree. We also make a few changes:

A receipt can be used to deposit if the beacon chain time is less than max(receipt.timestamp, chain.genesis_time) + DEPOSIT_WINDOW (eg. 2^25 seconds ~= 1 year).
Withdrawn validator slots stay in the state for 1 year
The proof of possession is now a signature of not just the hash of the pubkey, but of the pubkey+data. This prevents attackers from griefing by submitting a deposit with the same pubkey but a different randao hash root or withdrawal address.
A rule is added that you cannot deposit a validator that has the same pubkey as any validator currently in the state. This prevents double-spending receipts.

Notice that this mechanism can also later be used unchanged to import validators from shards.

Log HashChainValue is still using hash_chain_tip. Should this be self.receipt_tree[1] in the 0th arg?

Why the one year? Is this to compensate for the fact that withdrawals can now happen fast in the beacon chain (if only a short queue) to prevent quickly reregistering of public key after withdrawal?

Log HashChainValue is still using self.hash_chain_tip. Should this be self.receipt_tree[1] in the 0th arg?

Ah, yes it should.

Why the one year? Is this to compensate for the fact that withdrawals can now happen fast in the beacon chain (if only a short queue) to prevent quickly reregistering of public key after withdrawal?

It could be any length. But if the length is short, then it creates risk of loss of funds if the beacon chain breaks for an extended duration.

closed via #107


Draft "registration logic" for ChainSync Jumping

The ChainSync Jumping optimization requires some sort of centralized (across all ChainSync clients) governor logic in order to

pick the "dynamo" (the peer among the non-ledger peers which engages in the full ChainSync protocol) in the happy path while syncing, and
decide on proactivity of other peers in either the sad path, or when caught up.

This introduces extra complexity, so care should be taken to determine which parts can essentially be communicated by simpler methods. Also, the BlockFetch decision logic can serve as a point of reference.

The current CSJ prototype now contains the concept of a Governor, which handles registration of new ChainSync clients, providing them ClientContext for further interaction (in particular, it contains the candidate fragment variable).
It can be seen as a natural generalization of bracketChainSyncClient, incorporating and extending the registration of the candidate fragments in varCandidates. This is non-invasive as the only other places to adapt are next to the existing tracking of the candidate fragments in NodeKernel, and sufficiently powerful.


DEV 8775. 6 Downgrade packer to 0.12.2

https://github.com/Shippable/pm/issues/8775

tested on u16 image. closing


Fix docs showing how to access underlying class

The way you access a class internally (GameObjectClass) is removed in the final files and is instead accessed via GameObject.class. I've found all the places that shows extending a class and updated them.

Thanks, @straker!


Sphinx docs skeleton

The Python ecosystem uses either Sphinx or MkDocs. The Astral folks use MkDocs, but Arrow and I suspect the majority of the Python ecosystem use Sphinx.
The most noticeable difference between the two is that MkDocs uses Markdown and Sphinx uses reStructuredText. I have used Sphinx too long to be bothered by rST but, IMO, a lot of people dislike the syntax at first.
Translating between rST and Markdown will be annoying so we should make a decision before too long. For now, I got the site started with the tool I knew, Sphinx, so that is what I have added here.

MkDocs API documentation example: https://github.com/pydantic/pydantic/tree/main/docs/api (renders as: https://docs.pydantic.dev/latest/api/base_model/)

And here's an example of auto-documenting a module in Sphinx: https://raw.githubusercontent.com/spiraldb/vortex/dk/python-documentation-struct-read-projection/pyvortex/docs/encoding.rst
If you want classes/functions to be in separate files, you'd need O(N_IDENTIFIERS) rST files.

and FWIW, this is what the PyData theme looks like applied to the docs I wrote so far: https://spiraldb.github.io/vortex/docs/

Let's stick with Sphinx + Arrow theme

bump @gatesn or @robert3005; need an approving review to merge :)


[FEATURE] Use file-scoped namespaces on the entire project

Is your feature request related to a problem? Please describe.
Now that samples are using file-scoped namespace (#28), it's better to apply it to the entire project to keep consistency and also improve readability.
Describe the solution you'd like

[ ] Update editor.config to prefer file-scoped namespaces and issue warnings for block-scoped namespaces.
[ ] Convert all *.cs files to file-scoped namespaces.

I'd be happy to create a PR for this.

Sounds like a good improvement, feel free to work on it


About CephFS Provisioner input/output errors

I also encountered this problem, but I don't know how to solve this problem. Is it necessary to upgrade the ceph version to mimic? I am using "quay.io/external_storage/cephfs-provisioner:latest", I see someone modifying cephfs_provisoner.py to remove the namespace part, I don't know if it can be solved, and I don't know how to modify this image
My k8s version is 1.10.3, the ceph version is luminous, using centos7.4
ls: cannot access kubernetes-dynamic-pvc-ec3f1e84-94a6-11e8-9879-0a580af40146: Input/output error
total 0
d????????? ? ? ? ?            ? kubernetes-dynamic-pvc-ec3f1e84-94a6-11e8-9879-0a580af40146

Upgrade the linux kernel to 4.10 and above or upgrade the ceph to mimic version


[Vmware] Add missing condition to cleanup nics if there are commands to send

Description
This PR allows cleaning up nics in case there are nicExpunge commands to be sent
Fixes: #5308
Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[x] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale

[ ] Major
[ ] Minor

Bug Severity

[ ] BLOCKER
[ ] Critical
[ ] Major
[ ] Minor
[ ] Trivial

Screenshots (if appropriate):
How Has This Been Tested?

@blueorangutan package

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1231

@blueorangutan test matrix

@nvazquez a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

Trillian test result (tid-2033)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 35646 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5449-t2033-kvm-centos7.zip
Smoke tests completed. 89 look OK, 0 have errors
Only failed tests results shown below:

Test
Result
Time (s)
Test File

Trillian test result (tid-2032)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 38321 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5449-t2032-xenserver-71.zip
Smoke tests completed. 89 look OK, 0 have errors
Only failed tests results shown below:

Test
Result
Time (s)
Test File

Trillian test result (tid-2034)
Environment: vmware-65u2 (x2), Advanced Networking with Mgmt server 7
Total time taken: 41973 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5449-t2034-vmware-65u2.zip
Smoke tests completed. 89 look OK, 0 have errors
Only failed tests results shown below:

Test
Result
Time (s)
Test File

code lgtm

@blueorangutan package

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan test matrix

@nvazquez a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

Trillian test result (tid-2075)
Environment: vmware-65u2 (x2), Advanced Networking with Mgmt server 7
Total time taken: 41992 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5449-t2075-vmware-65u2.zip
Smoke tests completed. 88 look OK, 1 have errors
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_02_list_snapshots_with_removed_data_store
Error
58.26
test_snapshots.py

Trillian test result (tid-2073)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 45062 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5449-t2073-xenserver-71.zip
Smoke tests completed. 87 look OK, 2 have errors
Only failed tests results shown below:

Test
Result
Time (s)
Test File

ContextSuite context=TestKubernetesCluster>:teardown
Error
101.31
test_kubernetes_clusters.py

test_01_vpc_site2site_vpn_multiple_options
Failure
537.73
test_vpc_vpn.py

Trillian test result (tid-2074)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 45303 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5449-t2074-kvm-centos7.zip
Smoke tests completed. 87 look OK, 2 have errors
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_nic
Error
135.33
test_nic.py

test_07_deploy_kubernetes_ha_cluster
Failure
3604.60
test_kubernetes_clusters.py

test_08_deploy_and_upgrade_kubernetes_ha_cluster
Failure
0.06
test_kubernetes_clusters.py

test_09_delete_kubernetes_ha_cluster
Failure
0.04
test_kubernetes_clusters.py

ContextSuite context=TestKubernetesCluster>:teardown
Error
77.05
test_kubernetes_clusters.py

@blueorangutan package

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1293

@blueorangutan test matrix

@nvazquez a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

Trillian test result (tid-2101)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 35271 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5449-t2101-kvm-centos7.zip
Smoke tests completed. 89 look OK, 0 have errors
Only failed tests results shown below:

Test
Result
Time (s)
Test File

Trillian test result (tid-2102)
Environment: vmware-65u2 (x2), Advanced Networking with Mgmt server 7
Total time taken: 38439 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5449-t2102-vmware-65u2.zip
Smoke tests completed. 89 look OK, 0 have errors
Only failed tests results shown below:

Test
Result
Time (s)
Test File

Trillian test result (tid-2100)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 40294 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5449-t2100-xenserver-71.zip
Smoke tests completed. 88 look OK, 1 have errors
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_02_list_snapshots_with_removed_data_store
Error
55.13
test_snapshots.py


[Bug]: No new lines in error decoder errors

Summary
Check out this error: https://react.dev/errors/418
It's formatted all on one line, but should have new lines.
Page
https://react.dev/errors/418
Details
No response

Hi, could you please assign this issue to me? I submitted a pull request separately and signed the CLA. Thank you

@rickhanlonii Good morning Ricky, just wanted to follow up on this issue, I submitted a pull request under #6735 and wasn't sure if there was any feedback for my proposed solution. Thanks!

Seems to include newlines now:

-- https://react.dev/errors/418
The only thing missing is handling empty strings. The hydration errors will always produce https://react.dev/errors/418?args[]= but that's interpreted as missing unfortunately.

I could reproduce it locally though it wasn't consistent. Should be fixed with https://github.com/reactjs/react.dev/pull/6767


vulnsrc/alpine: yaml parsing error skip all parsable ones

shadow commit on 2017/03/01 causes yaml parsing error.
This *yaml.TypeError was returned to the top level Updater making other parsable ones not be inserted into the database.
If an error is parsing error, since it lose more returning the error to the caller, it may be better logging it than break an update iteration.

- pkg:
  name: shadow
    secfixes:
      ** here should be a version, but nothing here **
      - CVE-2016-6252
      - CVE-2017-2616
- pkg:
  name: slock
    secfixes:
      1.3-r3:

I sent a report upstream and I'm going to see if I can help them avoid checking in breaking YAML.
This isn't the first time this has happened and I don't want to remove the schema from our parser if it can be avoided.

Natanael at Alpine Linux responded really quick and both fixed the issue and added some code to sanity check changes to avoid this in the future.


Error "The requested session access is denied"

When I run RDPCheck.exe and let it connect, it brings me to the account login page. But when I log in, it give this error. This also happens when I try to connect via Remote Desktop Connection(to <IP_ADDRESS>). Here is my RDPConf.exe. My version of windows is Windows 10 Home 1909 build 18363.1379.
The error I get:

My config:

btw I'm conecting from my main account to my alternate account

Full access with user's permission might be your problem. Click without

I am having the same issue. I am using Guest account to login. Anyone has an idea how to resolve this for Guest accounts? They are not member of "Remote Desktop Users" Built-In group which Microsoft claims that need to be used for RDP.

In RDP settings in Windows you can select which accounts are allowed to RDP into the computer

Thanks for the tip but where can I do that? I am using Windows Home version and it seems that these settings are missing from the OS by design due to licensing.

Actually this issue has already been resolved and is posted at the main page under Links section:
Adding «Remote Desktop Users» group:
FULL STEP BY STEP SOLUTION HERE: https://superuser.com/questions/680572/allow-non-administrator-users-to-access-windows-7-home-premium-via-remote-deskto
A small registry hack needs to be done:

This is working like a charm. Thanks for your assistance.


winston:create-logger diagnostic cannot be turned off

Please tell us about your environment:

winston version?

[ ] winston@2
[x] winston@3

node -v outputs: v10.16.3
Operating System? (Windows, macOS, or Linux) macOS
Language? (all | TypeScript X.X | ES6/7 | ES5 | Dart) Typescript

What is the problem?

import winston from 'winston';

const logger = winston.createLogger({
  level: 'debug',
  format: winston.format.json(),
  transports: [new winston.transports.Console()],
});

export default logger;

In the console I see a lot of messages coming from the diagnostic logger.
winston:create-logger Define prototype method for "error"
winston:create-logger Define prototype method for "warn"
winston:create-logger Define prototype method for "info"
winston:create-logger Define prototype method for "http"
winston:create-logger Define prototype method for "verbose"
winston:create-logger Define prototype method for "debug"
winston:create-logger Define prototype method for "silly"
winston:create-logger Define prototype method for "error"
winston:create-logger Define prototype method for "warn"
winston:create-logger Define prototype method for "info"
winston:create-logger Define prototype method for "http"
winston:create-logger Define prototype method for "verbose"
winston:create-logger Define prototype method for "debug"
winston:create-logger Define prototype method for "silly"
winston:create-logger Define prototype method for "error"
winston:create-logger Define prototype method for "warn"
winston:create-logger Define prototype method for "info"
winston:create-logger Define prototype method for "http"
winston:create-logger Define prototype method for "verbose"
winston:create-logger Define prototype method for "debug"
winston:create-logger Define prototype method for "silly"
winston:create-logger Define prototype method for "error"
winston:create-logger Define prototype method for "warn"
winston:create-logger Define prototype method for "info"
winston:create-logger Define prototype method for "http"
winston:create-logger Define prototype method for "verbose"
winston:create-logger Define prototype method for "debug"
winston:create-logger Define prototype method for "silly"

What do you expect to happen instead?

I expected none of these internal winston debug messages in my console. I tried using DEBUG=-winston:create-logger and DIAGNOSTIC=-winston:create-logger but neither worked.
Other information

I saw another user report the same issue https://gitter.im/winstonjs/winston/archives/2017/11/22
And another user report the same issue https://stackoverflow.com/questions/55573448/disable-winstons-verbose-startup-logs

I believe this is caused by https://github.com/3rd-Eden/diagnostics/issues/20

This is caused by line no 54 in 'create-logger.js' where the developer has kept a debug line open for us.
debug('Define prototype method for "%s"', level);
I did comment this line manually and the problem is solved.

Same question

We are also having an issue with that line. Commenting it out is not an option, realistically.

We are also having an issue with that line. Commenting it out is not an option, realistically.

I can confirm that setting: DEBUG=-winston:create-logger in your .env file works.

None of the following .env lines removes this debug output in<EMAIL_ADDRESS>nor<EMAIL_ADDRESS>Setting env on the command line does not work either.
DIAGNOSTICS=-winston:*
DEBUG=-winston:*
DIAGNOSTICS=-winston:create-logger:*
DEBUG=-winston:create-logger:*
DIAGNOSTICS=-winston:create-logger
DEBUG=-winston:create-logger
DIAGNOSTICS=-*
DEBUG=-*
Perhaps the act of initializing Winston in our code, or the Winston code itself, somehow negates the environment variables?

Are you using the @dabh/diagnostics package in any other part of your repo?  Are you using winston from somewhere where process.env isn't available? https://github.com/DABH/diagnostics?tab=readme-ov-file#adapters Haven't seen others report this issue so thinking it must be something special about your env or setup...


Performance improvement

The glob lookup for the file CertificateAuthorityCertificate.pem causes a fairly significant performance slowdown when issuing any request, and it worsens the more files are present in the project.  By replacing this with a simple, synchronous exists check, we can realize fairly significant performance improvements.
On a current project, I was seeing ~3-second delays for every interaction through the library, while the same commands via curl completed in tens of milliseconds.  This simple change cut cycle time for my tests from ~9 seconds per test to ~200ms.
But since you can't easily reproduce my results, we can use the test suite for this project as proof.  This is the result of running the grunt suite before the improvement:
brian:~/work_repos/mockserver-client-node$ time node_modules/grunt/bin/grunt
Running "exec:stop_existing_mockservers" (exec) task
done
...
Chrome 78.0.3904 (Linux 0.0.0): Executed 34 of 34 SUCCESS (4.607 secs / 4.622 secs)
TOTAL: 34 SUCCESS

Running "stop_mockserver" task

Done.

real    6m25.084s
user    5m34.983s
sys     0m28.594s

And here are the same tests after this change:
brian:~/work_repos/mockserver-client-node$ time node_modules/grunt/bin/grunt
Running "exec:stop_existing_mockservers" (exec) task
done
...
Chrome 78.0.3904 (Linux 0.0.0): Executed 34 of 34 SUCCESS (4.841 secs / 4.852 secs)
TOTAL: 34 SUCCESS

Running "stop_mockserver" task

Done.

real    0m37.302s
user    0m5.040s
sys     0m0.848s

This looks like a good improvement.  I'll be doing a new release in the next few days which will include this change.


Move Format enums and util functions

Draft PR that moves FormatEnums and their utility functions from ecschema-metadata to quantity package. First commits of  quantity and ecschema-metadata Formats refactoring.

@ColinKerr @StefanApfel-Bentley @AureMonke
is this pr still needed? if not can we close it, and if so can we get a backlog item linked to this pr?


Instructions to load and use CuBERT pretrained and fine-tuned models

Any instructions to know how to load the provided pre-trained models? I understand that these are probably just BERT models trained on Source code using different tokenizer but I can't figure out how to incorporate the Cubert tokenizer with original BERT files to load / use the pretrained models. Any help is appreciated. Also please correct me if I am wrong about my assumption above.
Thanks!

Hello @DNGros.
I read these comments but I can not understand how to use CuBERT and how to fine-tuned it on my special dataset.
Have a nice day!

try this:
import tensorflow.compat.v1 as tf
------------------ 原始邮件 ------------------
发件人: "Kamel @.>;
发送时间: 2022年3月2日(星期三) 下午3:15
收件人: @.>;
抄送: @.***>;
主题: Re: [google-research/google-research] Instructions to load and use CuBERT pretrained and fine-tuned models (#571)
Hi @DNGros ,
I tried to run cubert on TensorFlow 2, and I am getting the error below. TensorFlow 2 do not support the contrib. Should I run cubert on TensorFlow 1? or any sugestion?
from tensorflow.contrib import layers as contrib_layers ModuleNotFoundError: No module named 'tensorflow.contrib'
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.Message ID: @.***>

Hi @ElchaabiMohamed
Yeah, it was a bit complicated to fine tune Cubert. To solve this issue, you should run CuBERT in TensorFlow 1.14.
I would recommend to have a look to other models such as CodeBERT and CodeT5, these models build in PyTorch and much easier to fine-tune and modify their source code.

Hi @ElchaabiMohamed
Thank you for your answer, but i still have the same error even i run CuBERT in TF 1.14.
Any other suggestion ???

Hi @ElchaabiMohamed
I solved in such a way switching import tensorflow.compat.v1 as tf to  import tensorflow as tf

@Kamel773
It's working now, thank you very much


UX: inconsistency in error messages on sign up page

Harbor version 1.1.0-rc1
Sign up with a username that's taken:

Sign up with an email that already has an account:

The error messages should have consistent phrasing. For the second instance, "Email address is already used" or "Email address already exists" would be good.
This is a minor detail but it's nice to have consistent phrasing. Looks like the change belongs in https://github.com/vmware/harbor/blob/master/src/ui/static/i18n/lang/en-lang.json.

We have changed the messages, please check the latest code (master branch).
"EMAIL_EXISTING": "Email address already exists.",
"USER_EXISTING": "Username is already in use."


Domains dropdown error

Sometimes domains dropdown fails and shows the threshold value and edit icon when it is collapsed:

closed by https://github.com/samtecspg/articulate/commit/ebcab4eef6e5569aed5eb1a6c064785c8bac2491


Multiple reporters with the same name not handled

Description
I don't know if this can be considered a bug, but if you register a reporter with the same name as an already existing one, when you select that reporter both of them will run, ignoring priority. It happened to me after updating to 2.4.x (I'm a slowpoke, I know it), where the now built-in junit reporter conflicted with my previous hacked together junit reporter. I guess as a workaround I can rename my reporter and update the CI scripts though.
Steps to reproduce

Create a custom reporter
REGISTER_REPORTER("junit", 10, SomeRandomReporter);
Run tests with -r=junit, notice that both reporters' output are sent to stdout.

Extra information

doctest version: v2.4.4
Operating System: Gentoo Linux
Compiler+version: Clang 8.0.1, gcc 9.3.0

Thanks for reporting! I initially allowed multiple reporters with the same names but different priorities because it made my code easier but looking back in hindsight that might have been the wrong decision as it leads to problems such as yours. I'll keep this issue open as I might eventually fix this.

Thanks for reporting! I initially allowed multiple reporters with the same names but different priorities because it made my code easier but looking back in hindsight that might have been the wrong decision as it leads to problems such as yours. I'll keep this issue open as I might eventually fix this.

Relates to the case of missing reporter: https://github.com/doctest/doctest/issues/271


Plugin page fails after updating to 13.10.0

Description of the issue
When updating from 13.9.1 to 13.10.0 in the browser the plugin the plugin page fails to load.
The logs say "Failed to load the package cache"
Steps to reproduce the issue

npx govuk-prototype-kit create --version=13.9.1 upgrade-from-13.9.1
cd upgrade-from-13.9.1
npm run dev
Visit Manage prototype
Click Plugins
Click Upgrade next to GOV.UK Prototype Kit

Actual vs expected behaviour
Expected: The upgrade works, then the kit is fully working in the new version.
Actual: The upgrade works, but after the upgrade the kit is unable to load the Plugins page.  I haven't tested for further impact.
What's going wrong
It's logging to say Failed to load the package cache - failing to read a file system cache should lead to the cache either being ignored or recreated.  Either option fixes this bug, recreating it would be my suggestion.

This issue will be fixed with the following PR: Prototype will now start when a plugin is missing from node_modules


Multiple episodes support error

          Okay, thank you! I will try to build it myself, I just wanted you to know about it.

Originally posted by @motyafl in https://github.com/sweetbbak/toru/issues/10#issuecomment-1994129017
I built it and was trying to test the episodes support, but it just doesn't work. It doesn't give me a choise what episode to stream. I checked, my torrent has multiple media files, but it doesn't work.

alright, I will try another multimedia torrent today and check it out.

I am author of that commit. I also checked that and it works fine on my build. Are you sure, that you built latest possible version? And if that, can you describe how you stream your torrent file?

closed with ae5308909e76c006d8ca121d2061853ecaeb8df6


BOOM-2355: HubSpot API key - no logner supported

Fixes #4
Next steps:

Create private app in hubspot (done for Boom)
Update the library
Replace the current api key with the new api key

Changelist:

replaced hapiKey query parameter with Authorization header
increased minor version number

Nice, thanks!


rs-convert.exe

I'm currently using Intelrealsense D455 for keypoint detection.I detect the two-dimensional coordinates on the color images.I had some problems getting depth information from two-dimensional coordinates:1. Alignment problem between color images and depth images.
1. Alignment problem between color chart and depth chart. I recorded a 5 minute 5 second video using realsenseviewer. Then 2238 color maps and 2239 depth maps were obtained using rs-convert.exe.I obtained the two-dimensional coordinates of 2238 frames through the key point detection of the color graph.When I mapped 2,238 two-dimensional coordinates to the depth chart, I found that they were misaligned.

2.The images produced by parsing bag files by rs-convert.exe and python are badly inconsistent.Since the image produced by rs-convert.exe is unaligned, I used the official python program to parse the bag file and  align the color and depth maps.But python parses 4460 color images and 4460 depth images.I tried to map my 2238 2D coordinates to the corresponding alignment depth map, but I couldn't!

With method 1 (extracting the depth and color images and then mapping the coordinates together), the mis-alignment may be due to the different physical positions on the front of the camera of the depth and color sensors not being taken into account (with the color sensor being offset horizontally from the depth sensor).  The SDK's depth-color alignment would usually automatically make adjustments for that difference when mapping color and depth coordinates together by using extrinsics (the translation and rotation between two sensors).
Perhaps instead of recording a bag-file video of the individual color and depth streams in the 2D mode of the Viewer, you could map depth and color together in the Viewer's 3D mode.  Instead of recording a bag, you could use a video screen-capture tool to record an .avi or .mp4 video and then extract the images from that video file so that you have a set of perfectly matched images.

With method 1 (extracting the depth and color images and then mapping the coordinates together), the mis-alignment may be due to the different physical positions on the front of the camera of the depth and color sensors not being taken into account (with the color sensor being offset horizontally from the depth sensor). The SDK's depth-color alignment would usually automatically make adjustments for that difference when mapping color and depth coordinates together by using extrinsics (the translation and rotation between two sensors).
Perhaps instead of recording a bag-file video of the individual color and depth streams in the 2D mode of the Viewer, you could map depth and color together in the Viewer's 3D pointcloud mode. Instead of recording a bag, you could use a video screen-capture tool (such as Camtasia, Fraps or XSplit) to record an .avi or .mp4 video of the area of the Viewer's window that contains the pointcloud - or press the F8 keyboard key for a fullscreen view - and then extract the images from that video file so that you have a set of perfectly matched images.

Thanks for your quick reply, I have solved the problem.But when I get three dimensional coordinates, I always get zero. How do I solve for zero. There are no relevant papers to solve this problem.

What method are you using to get the three-dimensional coordinates, please?

What method are you using to get the three-dimensional coordinates, please?

I tracked the mouse's two-dimensional key points on the color images.Then align the color images with the depth images.Finally, the get_distance () function provided by IntelRealsense SDK2.0 is used to obtain the distance information.

You will not get XYZ coordinates without generating a 3D pointcloud, but you can retrieve the real world Z-distance in meters of a specific two-dimensional XY coordinate using get_distance().  A Python example of this can be found at https://github.com/IntelRealSense/librealsense/issues/9939

Hi @666tua  Do you require further assistance with this case, please?  Thanks!

Case closed due to no further comments received.


Better error message when no newuidmap and newgidmap installed

Add better error message (can be seen at http://vagga.readthedocs.org/en/latest/errors.html#uidmap) and docs /cc @lilianmoraru

You are fast :)
Thanks!


Only export vertex colors when the material calls for them

Currently, if one imports a model with vertex colors, they show up as a color attribute with a mix node:

But when exporting, if vertex colors are present at all, they are exported into the glTF, affecting the material.  For example if the user has accidentally created empty vertex colors, the material may export as all black, even though Blender does not preview that.  This has caused some confusion among artists (both at my company and externally), as Blender does not make the presence of vertex colors ("Color Attributes") particularly obvious, with the UI panel collapsed by default and the colors not influencing the material by default.
I'm starting to think that the exporter should avoid exporting vertex colors for any mesh unless the mesh bears a material that references the vertex colors from a node graph.  Thoughts?

I spent hours this morning trying to figure out why using Filament to open a Blender-exported glTF file crashed with At least one buffer slot was never assigned to an attribute.  Same problem even after deleting every material/texture and all but one mesh.
Turns out the mesh was being exported with Color Attributes (Face Corner/Byte Color, if that's relevant), COLOR_0 and COLOR_1. Filament will crash whenever the model has two Color Attributes, or a single Color Attribute with no associated material.
As far as Blender/glTF is concerned [0], I fixed the issue by unchecking "Vertex Colors" under the "Geometry" panel in the export window. This stopped those attributes from being exported.

I'm not sure I see a need for any change to the glTF exporter, given there's already an option to disable exporting vertex colors.
What about making the "Vertex Colors" option unchecked by default?  There might be more people negatively affected by checked-by-default (i.e. who don't need to export vertex colors but still run into these kind of problems), than those who would be negatively affected by unchecked-by-default (i.e who do need to export vertex colors, but are already explicitly working with them therefore should know they need exporting).
[0] Whether Filament is "correct" in crashing is a separate question that I'll raise on their repository (I suspect not - the spec doesn't require support for COLOR_ attributes but it shouldn't crash simply because they are present, but unused).

What about making the "Vertex Colors" option unchecked by default?

My thinking is that the exporter should attempt to match the visual that the user has created for their model, as opposed to asking the user to learn to use a larger set of checkboxes.
The glTF spec is such that if COLOR_0 is present, it gets multiplied with the base color, always.  If the user has something similar to this set up in their Blender material, we can best preserve that on export by including the vertex colors.  If the user has vertex colors assigned in Blender, but is not using them for the appearance of the model, then we must exclude them from the glTF export, because glTF offers no way to include disused vertex colors.

Some stuff to decide how/if to handle

Is there still a "Vertex Colors" option? What does it do?
What to do when a mesh has multiple materials that disagree about whether and which color attribute to use. Warn and pick one to "win"?
What to do when a color attribute is used for base color but not for alpha (fairly common?). Could export COLOR_0 as vec3 without alpha.
What to do when a color attribute is used for alpha but not base color. Could export COLOR_0 with one-filled RGB components.
Does an opaque material that has "multiply alpha by color attribute alpha" nodes count as using the color attribute for alpha?
What to do if different color attributes are used for base color and alpha. Could merge them together into COLOR_0 (similar to combining image channels).

Is there still a "Vertex Colors" option? What does it do?

It sounds like we would need this option.  When checked, Blender color attributes not used in the material would be exported as glTF custom vertex attributes, likely _COLOR_0 as suggested above.  When unchecked, I'm less sure, but given that's a change from the default behavior, maybe the user is trying to turn off all vertex colors regardless of material nodes?

What to do when a mesh has multiple materials that disagree about whether or which color attribute to use. Warn and pick one to "win"?

Just like textures do now, yeah.  I can't think of a better suggestion.

What to do when a color attribute is used for base color but not for alpha (fairly common?). Could export COLOR_0 as vec3 without alpha.

Yes. glTF allows vec3 here, and it results in some space savings over vec4.

What to do when a color attribute is used for alpha but not base color. Could export COLOR_0 with one-filled RGB components.

Is this a common case we must support?  I'm fine if the documentation just disallows this, unless people really want it.

Does an opaque material that has "multiply alpha by color attribute alpha" nodes count as using the color attribute for alpha?

Good point, no.  If blend mode is opaque, no alpha channel need be included, and you can get the vec3 smaller storage again.

What to do if different color attributes are used for base color and alpha. Could merge them together into COLOR_0 (similar to combining image channels).

If this is relatively straightforward to do, it would be great, similar to how this project combines base color & alpha textures.  But if this is too much for our existing schedules or too negative an impact on code complexity, then we can disallow it and ask users to combine it into a single color attribute before export.

Does there need to be a separate option for exporting color attributes different than the option for exporting all attributes?
If COLOR_0 is only for material use, does it make sense to always make it unorm16? (So vec3 and vec4 would be the same size.)

Does there need to be a separate option for exporting color attributes different than the option for exporting all attributes?

I think if we can see color attributes used in the material, and decide to export or not based on that, there's no need for an additional checkbox to control that.
I do think a (new?) checkbox could be helpful to control whether unused attributes get exported or not, defaulting to not.

If COLOR_0 is only for material use, does it make sense to always make it unorm16? (So vec3 and vec4 would be the same size.)

Oh, I forgot.  I think this project already uses unorm16 for them.  So... no size savings as I wrote above, sorry.

I think this project already uses unorm16 for them.

Currently only Byte Color attributes export as unorm16. Float color attributes export as floats.

Not sure why this ticket is still open, as we implemented it already. Closing


Add collections types

Thanks for the feedback. As soon as we have all of the weird typing issues figured out, I'll address the conflicts.

TODO added and merge conflict resolved.

Great, thanks!
Just a small info, I will not be able to review PRs here for the next week or two (I will need to catch up on writing plugins).