Presenting asdf-pydantic, create ASDF tags with pydantic models.

After pip install asdf-pydantic you can do something like this:

from asdf_pydantic import AsdfPydanticModel

class Rectangle(AsdfPydanticModel):
    _tag = "asdf://asdf-pydantic/examples/tags/rectangle-1.0.0"

    width: float
    height: float

# After creating extension and install ...

af = asdf.AsdfFile()
af["rect"] = Rectangle(width=1, height=1)

#ASDF 1.0.0
#ASDF_STANDARD 1.5.0
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.1.0
asdf_library: !core/software-1.0.0 {
    author: The ASDF Developers,
    homepage: 'http://github.com/asdf-format/asdf',
    name: asdf,
    version: 2.14.3}
history:
  extensions:
  - !core/extension_metadata-1.0.0
    extension_class: asdf.extension.BuiltinExtension
    software: !core/software-1.0.0 {
        name: asdf,
        version: 2.14.3}
  - !core/extension_metadata-1.0.0 {
    extension_class: mypackage.shapes.ShapesExtension,
    extension_uri: 'asdf://asdf-pydantic/shapes/extensions/shapes-1.0.0'}
rect: !<asdf://asdf-pydantic/shapes/tags/rectangle-1.0.0> {
    height: 1.0,
    width: 1.0}
...

Features

[x] Create ASDF tag from your pydantic models with batteries (converters) included.
[x] Validates data models as you create them and not only when reading and writing ASDF files.
[x] Preserve Python types when deserializing ASDF files.
[x] All the cool things that comes with pydantic (e.g., JSON encoder, JSON schema, Pydantic types)
[ ] Genereates ASDF schemas for you. (TBD, talk to me if you have ideas)

Rationale
Defining and prototyping composite types with Python standard types: UserDict, Dataclasses, TypeDict are okay, but nothing beats the flexibilty of pydantic. With ASDF you're already in the mental space of working with either YAML and JSON hierarchical structure. This feels natural with pydantic.
There are big potentials beyond integrating with the Python ecosystem. For example, Tags created with asdf-pydantic is already readily JSON-serializable (other existing tags, like astropy, may require custom serialization) and the JSON schema comes for free.  Initial adoption of JSON schema in another language (e.g., C++, C#, Java, javascript) could be much faster than with ASDF schema — and not to forget, this is the most common schema for HTTP APIs.

I am looking for commmunity interest and user feedback. Alongside contributors to incubate this project if ASDF is interested.

GitHub repository
Documentation

Thanks for sharing!
I'm excited to try out your project. I'm curious, did you encounter any issues with ASDF (including any non-intuitive API) while developing the project (or are there features that if added to ASDF would make this project easier/cleaner)?

I agree that this is a good idea. I have been playing with the idea of fusing  asdf to pydantic since I started working with ASDF.
Several thoughts:

The generation of ASDF schemas is a necessity for interoperability with non-python based ASDF libraries. One of the major points of using ASDF is that data being written and read from an ASDF file will be validated. pydantic would ensure this if a file is written using python; however, without a schema to transmit the information any non-python library would not have a method to handle validation of data. Note that I realize there is no current roadmap for a non-python based ASDF library, but that is one of the lofty principles that ASDF attempts to follow.

To this end, pytdantic does provide an interface for generating json-schemas: https://docs.pydantic.dev/usage/schema/.
This is a purely json-schema esque python dictionary, so tag information will need to be populated.
Also correctly referencing existing ASDF schemas will be difficult.

Versioning will need to be implemented. Since ASDF is a file format, anything of this type will need to support backwards compatiblity with opening older versions of the model objects. Thus if you tweak your AsdfPytdanticModel you have to have a system in place to to open previously written data using this model (ASDF is supposed to be an Archival format).

In theory, basic "rules" can be in place, wherein changes to AsdfPydanticModel based objects need to be done in a flexible enough way. However, there is no way to really ensure this. (I would be happy to see a way to ensure this).
Basically, schemas are a way to preserve previous iterations of object representations outside the code governing the Python objects the represent.

I am not sure how this would be able to handle non-builtin and non-AsdfPydanticModel based objects. In your example, would you be able to change it so that the width and height are astropy.units.Quantity objects? The hand-off between pydantic and ASDF, especially for validation, might be difficult.

How does subclassing work? That is if you subclass an existing model, one would have to explicitly set a new tag otherwise deserialization will become an issue (which object would it be deserialized to?). When generating schemas from the AsdfPydanticModel would the schemas be essentially duplicated or would they reference something in common?

Should there be an interface to "wrap" existing Python objects? Part of the reason for all the duplication, is that users of ASDF do not necessarily control the objects they are wishing write to ASDF files. It would be great to just define something like AsdfPydanticModel but declare it represents some other Python object for ASDF, wherein one basically defines something akin to the to_yaml_tree or from_yaml_tree interfaces of ASDF converters. Then one can at least combine creation of schemas and converters into a single object.

There are more things to consider once you dive in.

One part I decided to ignore was optimizing entry point installation as pointed out in various pages of the ASDF docs.
What resulted was asdf-pydantic implements a Converter that contains a dynamic mapping between tags and types to be referenced later when the user creates their Extension or extension manifest. I've yet tested if the performance loss is significant enough for me to change how users statistically specify URIs, types, etc.

Maybe a nice-to-have is the API or the docs does not encourage building generic Converter without adding your own fixings. I've no specific request here, but the limitations made me design *asdf-pydantic`'s converter as (1) a singleton and (2) stores a mapping between tags to types.

@WilliamJamieson Let me first address the questions that I can answer quickly then put more thoughts on the others (especially the schema and version question):

I am not sure how this would be able to handle non-builtin and non-AsdfPydanticModel based objects.

Oh I was so thrilled when this worked amazingly. See this passing test script that uses astropy.units.Quantity.
https://github.com/ketozhang/asdf-pydantic/blob/main/tests/patterns/astropy_types_test.py
Look through the other test scripts as well.

How does subclassing work? That is if you subclass an existing model, one would have to explicitly set a new tag otherwise deserialization will become an issue (which object would it be deserialized to?). When generating schemas from the AsdfPydanticModel would the schemas be essentially duplicated or would they reference something in common?

The rule to follow is:

Every defined AsdfPydanticModel should be a tag.
If a field's  type is an AsdfPydanticModel, it is referenced as a tag.
If a field's type is not an AsdfPydanticModel but is compatible with standard ASDF schema or installed extensions. The proper tag and/or serialization is used.
If a field's type ispydantic.BaseModel but not (2), then it's field are as (3).
If a field is neither (2), (3), or (4), then serialization will fail.

With those, subclassing follows (1) and a new tag needs to be defined (not yet enforced in asdf-pydantic).
For schemas, I've limited exposure to YAML schema inheritance and this does not exist in JSON schema.

Should there be an interface to "wrap" existing Python objects?

If we agree the minimum the user must do is to define the object's data model in aAsdfPydanticModel, then pydantic already provides the wrapper:
class MyForeignObjectAdapterModel(ASDFPydanticModel):
  ...

my_foreign_object = ...

af.AsdfFile({
  "obj": MyForeignObjectAdapterModel.parse_obj(my_foreign_object)
})

###
from pydantic import parse_obj_as
sequence_of_foreign_objects = ...

af.AsdfFile({
  "objs": parse_obj_as(list[MyForeignObjectAdapterModel], sequence_of_foreign_objects)
})

Now how complicated the adapter is depends on your object. All Python standard types adaptable for free. You may take advantage of the object's __dict__ if defined.

@ketozhang its great that yo have already considered many of the same things I have already. I'll take the time to browse through your code and play with it.
There are two more things that I think would be useful:

It would be nice to be able to treat these models as proxies for the AsdfFile object. Namely, each AsdfPytdanticModel instance contains all the information needed for this (assume the object is rooted with keyword root or something more specific). So one should be able to access the AsdfFile which would be created by serializing the object instance. This would make non-serialization related features, such as search, info, etc. immediately accessible as you generate datamodels. That is in your example there would be no need for:

af = asdf.AsdfFile()
af["rect"] = Rectangle(width=1, height=1)

Instead, it would already exist (or be generated on the fly and cached) with access given via something like Rectangle(width=1, height=1).asdf_file. Then the access to ASDF features becomes pretty direct.

The ability to store "object (not instance) specific metadata".

Right now for the Roman Space Telescope, we store descriptive metadata about all datamodel instances in the rad schemas. For example, we encode the MAST archive's database endpoints and metadata directly in the schemas, e.g. https://github.com/spacetelescope/rad/blob/2f909e39b46d2ebb6ddb50e033b8e11afcb40c7c/src/rad/resources/schemas/aperture-1.0.0.yaml#L23-L25. This information is never set by the datamodel instances as it remains constant across all instances. However, this information is still accessible through the schema_info command on the AsdfFile object.
The current implementation of using this information is awkward because it is easy to search out and find this data given a valid ASDF file, but it is rather difficult to interrogate the schemas independently of an ASDF file instance to back out the data in a way which can be related to a file instance (schemas are very flat in that they do not really define an inheritance structure). This "reverse interrogation" is important so that the MAST archive can be updated to accommodate new data.
What is nice about defining everything via these models is that Python defines a nice inheritance structure. This means when python imports the module containing a datamodel, the nested structure is essentially flushed out by Python before any instance is created. This can then be interrogated to figure out this metadata.

I am not sure how this would be able to handle non-builtin and non-AsdfPydanticModel based objects.

Oh I was so thrilled when this worked amazingly. See this passing test script that uses astropy.units.Quantity<EMAIL_ADDRESS>
I just had a chance to look over the source code for asdf_pydantic and it is pretty slick.
However, currently as your example tests stand ASDF is not actually validating the data as it is being written to the ASDF file nor is it validating the data is it is reading from the ASDF file. Granted, pydantic will validate when the data gets placed into the object, so it will throw an error if the data inside it is not correct. But this will not occur during the "validation" portions of the serialize/deserialize, instead it happens during the object creation/conversion from yaml. And more importantly, non-asdf-pytdantic objects contained within asdf-pydantic objects will experience no validation.
I had to dig really deep and "watch" ASDF run its validation on the data as it is reading/writing. Basically, asdf-pydantic does not throw validation errors via an oversight!
(I'm going to point at the latest PyPi release tag to explain what is going on; however, the current dev version is rapidly evolving in this area, see #1490 for what might be changing)
During tree validation ASDF validates from the root node down through all the leaf nodes in a depth-first-search.  When a "tag" is encountered during validation, as is the case for the tagged objects in your example, we hit the following block of code:
https://github.com/asdf-format/asdf/blob/efafac8e6efec81370bb6fbdf06c82e958185d7c/asdf/schema.py#L306-L312
In your particular case, you have a valid ASDF extension so it drops to:
https://github.com/asdf-format/asdf/blob/efafac8e6efec81370bb6fbdf06c82e958185d7c/asdf/schema.py#L307-L308
In your case schema_uris  variable happens to be an empty list. When it hits the code that validates further down the tree:
https://github.com/asdf-format/asdf/blob/efafac8e6efec81370bb6fbdf06c82e958185d7c/asdf/schema.py#L314-L320
It iterates over an empty list and so does not descend further. Moreover, it does not raise any kind of alarms. This means that ASDF cannot validate anything contained within the subtree result of your AsdfPydanticModel.
I don't think we quite recognized the consequences of this of allowing tags which don't have a schema to allow for ASDF to validate the contents. However, his makes it much simpler to just skip writing a schema for an object, but instead just defining a tag. @eslavich  might be able to comment on why this was allowed in the first place.
In this case, pydantic will ensure some data integrity, though only for objects defined via asdf-pydantic. But, flawed data in non-asdf-pydantic defined objects, which may "deserialize with out issue" will remain unrecognized as invalid.
Thus once asdf-pydantic can correctly create asdf-schemas, it should pass them as part of the extension process to ASDF to avoid these edge cases.

@WilliamJamieson I encourage you to submit an issue to the repo for those suggestions that asdf-pydantic not yet support.
You've brought up a lot of desires I need to put down in the documentation. For example,
Data model metadata

The ability to store "object (not instance) specific metadata".

This is already supported. A field defined with the type pydantic.ClassVar is automatically assigned to every instance and inherited by every subclass. By default, this field is not written to ASDF, it only exists in the Python object. We could choose to store all ClassVar fields in the ASDF schema as you've done, but I am not aware of if this is a standard practice in ASDF.
Versioning and backward compatibility
The constraint is you can only have one Python package version installed. If you want backward-compatibility, the general rule is your latest version of your Python package must be a superset of the extension, tag, and schema versions.
My take is because your Python package must claim to support an extension at some version, to be backward compatible, it must also register an older version of the extension in its manifest. You could have various strategies to build AsdfPydanticModel to be flexible, but it could much more stable way to archive the source code itself in your Python package (e.g., put it in model/archive/shapes_1_0_0.py).
The fallback in asdf is to parse it as Python native types. That works pretty well for a lot of archival purposes.
Another fallback is dynamic generation of models from an ASDF schema which is possible with pydantic but out of scope for asdf-pydantic right now. However, I am not sure how this is more helpful at asdf fallback. Again you're not going to get much use out of your AsdfPydanticModel if you don't have the old version of the class existing elsewhere.
I'm open to ideas. How has the community implemented backward compatibility?
Are AsdfPydanticModel fields being validated in ASDF?
@WilliamJamieson, replying to the comment just before this. ASDF is not doing any validation is because the tests I have, have not added a TagDefinition to the Extension.
From my tests, ASDF is validating the field of AsdfPydanticModel.  Try this example where I maliciously changed astropy.Quantity.value to a string in the YAML file.

Click for example
import asdf
import astropy.units as u
from asdf.extension import Extension
from astropy.units import Quantity

from asdf_pydantic import AsdfPydanticConverter

class DataPoint:
    _tag = "asdf://asdf-pydantic/examples/tags/datapoint-1.0.0"

    distance: Quantity[u.m]

AsdfPydanticConverter.add_models(DataPoint)

class TestExtension(Extension):
    extension_uri = "asdf://asdf-pydantic/examples/extensions/test-1.0.0"

    converters = [AsdfPydanticConverter()]  # type: ignore
    tags = [*AsdfPydanticConverter().tags]  # type: ignore

asdf.get_config().add_extension(TestExtension())

with open("test.asdf", "rb") as fp:
    asdf.open(fp)

#ASDF 1.0.0
#ASDF_STANDARD 1.5.0
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.1.0
asdf_library: !core/software-1.0.0 {author: The ASDF Developers, homepage: 'http://github.com/asdf-format/asdf',
  name: asdf, version: 2.14.4}
history:
  extensions:
  - !core/extension_metadata-1.0.0
    extension_class: asdf.extension.BuiltinExtension
    software: !core/software-1.0.0 {name: asdf, version: 2.14.4}
  - !core/extension_metadata-1.0.0 {extension_class: astropy_types_test.setup_module.<locals>.TestExtension,
    extension_uri: 'asdf://asdf-pydantic/examples/extensions/test-1.0.0'}
  - !core/extension_metadata-1.0.0
    extension_class: asdf_astropy._manifest.CompoundManifestExtension
    extension_uri: asdf://asdf-format.org/core/extensions/core-1.5.0
    software: !core/software-1.0.0 {name: asdf-astropy, version: 0.4.0}
positions:
- !<asdf://asdf-pydantic/examples/tags/datapoint-1.0.0>
  distance: !unit/quantity-1.1.0 {datatype: float64, unit: !unit/unit-1.0.0 m. value: "foobar"}
  time: !time/time-1.1.0 2023-01-01T00:00:00.000
- !<asdf://asdf-pydantic/examples/tags/datapoint-1.0.0>
  distance: !unit/quantity-1.1.0 {datatype: float64, unit: !unit/unit-1.0.0 m, value: 1.0}
  time: !time/time-1.1.0 2023-01-01T01:00:00.000
...

As you pointed out and I've read in the docs, schema-less tags are possible and not discouraged.

I welcome any further questions and feedbacks here until the thread is closed.
Do consider making an issue for the usual bug reports and suggestions in at https://github.com/ketozhang/asdf-pydantic/issues.

Versioning and backward compatibility
The constraint is you can only have one Python package version installed. If you want backward-compatibility, the general rule is your latest version of your Python package must be a superset of the extension, tag, and schema versions.
My take is because your Python package must claim to support an extension at some version, to be backward compatible, it must also register an older version of the extension in its manifest. You could have various strategies to build AsdfPydanticModel to be flexible, but it could much more stable way to archive the source code itself in your Python package (e.g., put it in model/archive/shapes_1_0_0.py).
The fallback in asdf is to parse it as Python native types. That works pretty well for a lot of archival purposes. Another fallback is dynamic generation of models from an ASDF schema which is possible with pydantic but out of scope for asdf-pydantic right now. However, I am not sure how this is more helpful at asdf fallback. Again you're not going to get much use out of your AsdfPydanticModel if you don't have the old version of the class existing elsewhere.
I'm open to ideas. How has the community implemented backward compatibility?

I'll open an issue directly in asdf-pytdantic for this, where further discussion can occur.
Honestly, I think the simplest for versioning AsdfPydanticModels is to simply require the version number of the model to match the version number of the package. This will properly reflect the possible differences in an object, and then one could possibly issue a warning if the version of a tag in an existing asdf file does not match the current version of the model, but the data is compatible with the model. If pytdantic's validation fails then the user will be able to quickly determine the version of the extension needed, and then manually extract and forward port the data (if possible). In practice, I doubt the data contained in any model will drastically change over time.


Unable to interpolate env_vars in airflow.cfg

I'm almost positive that pull request #467 has resulted in non working interpolated config vars in the airflow.cfg file. I'm not sure what the intent was of that pull request, so I've been unable to submit a patch.
The output of the error looks like this:
sqlalchemy.exc.ArgumentError: Could not parse rfc1738 URL from string '$DATABASE_URL'

example of what doesn't work:
[core]
sql_alchemy_url = $DATABASE_URL

@neovintage something weird is definitely happening here -- I will fix it.

@neovintage I got a similar error as you under Python 2 (when I wrote my comment) but since then I can't replicate it under Py2 or Py3. Are you sure you have a $DATABASE_URL env var set (which would be the other reason you'd see that error)? For example if you set sql_alchemy_conn = $PATH (or some env var you know exists) do you see it properly expanded in the error message?

@jlowin It's set and I've also tried changing it to something else. When I rollback to commit ba35124d75a3ad23266c324d729e80d5267e161f and then print the output (e.g. conf.get('core', 'SQL_ALCHEMY_URL')), I end up with the correct uri. At current head, when I print the conf var I end up with '$DATABASE_URL'. I've tried blowing away my virtualenv and starting from scratch. Is there any toher output that would be useful?

@neovintage ok, I was pretty sure you knew what you were doing but it never hurts to ask :). I suspect this is a weird corner case with the Py3 compatible basestring that for some reason I can never replicate (it popped up in one other issue). Could you please help me with one thing -- if you look at line 35, could you print the type of the env_var that is getting kicked out? If I'm right it'll be something like newstr:
def expand_env_var(env_var):
    if not isinstance(env_var, basestring):
        print(type(env_var))
        return env_var
    ...

Regardless, I will rework the trap a different way. Really sorry for the trouble!


Create quickstart.md

I think Getting Started Step 2 can be simplified further. It's probably as easy as:
docker run -it -p 80:80 arogi/arogi-demos

I agree, TSP code shouldn't exist anywhere in this docker

I created a new Docker image and updated the instructions.
Tested on:

[x] Ubuntu
[ ] Windows
[ ] macOS

Tests passed. Merging.


A binary/command to run on the profiler

The server currently executes forever, and requires a call from an external program.
It would be nice if there were a binary (or command to be passed to the server) that makes it execute a prompt from CLI and exit after that.
To use both the Nvidia Visual Profiler and the Samply profiler I needed to edit the code to make the process exit after answering a single request, that I started with curl.

This has been implemented in #119.


Remove c string limitation rebase2

rebased #104

Coverage increased (+0.2%) to 84.923% when pulling 793f0ff01e61291cef6070f4c375cdfed4266dea on hendrik-cliqz:remove-c-string-limitation-rebase2 into 11a2c8bd32571c9bcfa8b02d227435dc124e2886 on cliqz-oss:master.

Coverage increased (+0.2%) to 84.938% when pulling 0c498ef53c0d18a97530cc82995fc17d4bfae257 on hendrik-cliqz:remove-c-string-limitation-rebase2 into d7f8dbb756ad19cb49a796cfd9639a2ae486bb7f on cliqz-oss:master.


tekton pipeline 'setup' task returned error: fatal: unable to access URL using bad/illegal format or missing URL

How to Reproduce:

Log in to the cluster using:  icc [cluster name]
oc new-project tools
RELEASE=$(curl -s https://api.github.com/repos/IBM/ibm-garage-tekton-tasks/releases/latest | jq -r '.tag_name')
export NAMESPACE="tools"
kubectl apply -n ${NAMESPACE} -f "https://github.com/IBM/ibm-garage-tekton-tasks/releases/download/${RELEASE}/release.yaml"
oc create serviceaccount pipeline
oc adm policy add-scc-to-user privileged -z pipeline
oc adm policy add-role-to-user edit -z pipeline
oc new-project [your ReactJS application]-dev
git clone [your ReactJS application repo located in github.ibm.com]
cd [to your cloned ReactJS application folder]
oc sync [your ReactJS application]-dev
oc pipeline --tekton

Note: At this point it is expected that the pipeline is running.

On Openshift Container Platform -> Click Pipeline: Task Status: "Last run status: failed"

On Openshift Container Platform -> Click Pipeline ->  Click Task Status: "Log" says:

setup
STEP-GIT-CLONE
set +x
Cloning into '/source'...
fatal: unable to access<EMAIL_ADDRESS>:  URL using bad/illegal format or missing URL
NOTE:

xxxxxx  symbol in the URL represents my git Personal token. [Actual valued removed for security purposes - just here]
XXX  symbol in the URL represents my folder name.  [Actual value removed as well on purpose - just]

Thanks.

I find a way to resolve this:
First Step: SSH
I cloned the app repo with SSH instead of with HTTPS
Second Step: Openshift Secret
Updated git-credentials with Username & Token.
Then run the pipeline again and the Setup stage worked!

sorry I closed this by mistake @schijioke-uche

What's the root cause? That the URL ends with / like .git/ ?


mirror: add command to mirror to Bintray.

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[x] Have you successfully run brew tests with your changes locally?

Used for uploading imagemagick 6.9.4-3 in https://github.com/Homebrew/homebrew-core/pull/1308. CC @DomT4
Going to merge quickly as it's actually working.

Thanks for this. Have been meaning to write it into the pull command for a while but Andrew rewrote most of pull and I lost track of the logic in that command. Looks reasonable to me.

Nice one! :+1:

Neat! I didn't know we did this.
Should we also mirror files that are:

on known-flaky servers
in the archive.org wayback machine
known to delete or change URLs for old versions
?

@apjanke Yes but we need to not mirror too much stuff as if we're ever space-restricted then bottles are more impotant 👍


feat: add additional build files as examples

I have added further build files based on the integration sections on the preferences page of the Crashtest Security Suite.

Github recommends to put a linebreak in all files at the last line

Nice. I will fix this. I am just wondering: Where do I see the recommendation?


Login to analysis backend

how do I login after application is successfully deployed.
Can you also specify the steps to create user and password for login

Analysis needs to be set up as a client application using Auth0 (an authentication/identity service), then users can be added.  Additional notes are in the UI readme: https://github.com/conveyal/analysis-ui#configuration


✨ Sparsification metric and plot methods for Calibration Error

[x] Add Sparsification Error from Ilg et al., Uncertainty estimates and multi-hypotheses for optical flow. In ECCV, 2018.
[x] Create wrapper classes for BinaryCalibrationError and MulticlassCalibrationError to provide plotting methods.
[x] Write Tests
[ ] Update Documentation

Does #69 take precedence to test the workflows in passing?


Using LaTeX equations in markdown

I am unable to use LaTeX equations in the .Rmd or .Rmarkdown files. The markdown file renders the equation well, but the website shows the raw equation.
This issue was raised for the blogdown so I guess it is related to MathJax. I tried the solution suggested by yihui but it didn't work.
I'm new to Blogdown, and I learnt about this theme yesterday from your R-Ladies workshop on YouTube. So, if I missed something, please let me know.

Maybe not the best way to do it, but this solution by @Lauler worked!
I put the code below in footer.html (found at themes/hugo-apero/layouts/partials/footer.html).

@harshvardhaniimi I was working on this for my own site and can recommend a cleaner solution.
You're definitely on the right track by including a script for mathjax in the theme files, but it's best to avoid directly modifying the files in [site root]/themes/hugo-apero/... because those are the theme defaults. If you want to modify the content of those files, we can save your own versions in the site directory (mimicking the same tree structure as themes/hugo-apero/...) and Hugo will use those custom files instead of the defaults. For example, if I create my own [site root]/layouts/partials/footer.html, then Hugo will ignore the default [site root]/themes/hugo-apero/partials/footer.html and use my version instead. This lets us modify theme components modularly without corrupting the theme's defaults :)
So in the case of mathjax, we just need a way to insert the code in Yihui's example into a site "partial."
One way to do this is to copy the contents of (for instance) themes/hugo-apero/layouts/partials/footer.html into layouts/partials/footer.html and modify it like so:

    [ ... all of the other code is here ...]

which should give every page on your site the ability to render mathjax in a blogdown post. It's possible the first time you tried to use Yihui's solution you mentioned above because you didn't replace the YOUR_CDN_HERE component of the link. That piece of Yihui's solution is easy to miss at first, so I wanted to flag that.
The above should work, but if you wanted to be even more modular with this process, instead of putting the mathjax code directly into footer.html, you could call another partial that contains custom footer content: {{ partial "foot_custom.html" }}. That file could then contain the mathjax code or, indeed, further calls to other partials files. Because Hugo builds websites from lots of little pieces, it can be helpful to keep all of your custom little pieces organized in separate, well-labeled files for your own future comfort.
Hope this works

Thank you! Help here is greatly appreciated, if you'd be willing to offer this partial as a PR to the theme I'd happily merge it in! Mathjax is in my todo list

@mikedecr Moving footer.html to layouts/partials/ worked perfectly! Thanks for pointing to replace YOUR_CDN_HERE; I had missed it entirely.
I will put this as a pull request for everyone's benefit.

@apreshill I could give this a shot combined with something like {{if site.Params.enableMathjax }} to let users set an option in the global config? I will need to dig around in the theme to get used to editing and testing a theme from its source repo.

@harshvardhaniimi if I could make a slight suggestion to your site based on the commit you submitted, I would recommend you make your layouts/partials/footer.html look like this:

      {{ if ne site.Copyright "" }}{{ site.Copyright | safeHTML }}{{ else }}© {{ now.Format "2006"}}{{ if site.Params.orgName }} {{ site.Params.orgName }}{{ end }}{{ if site.Params.orgLocal }}, {{ site.Params.orgLocal }}{{ end }}{{ end }}

      Made with Hugo Apéro.

      {{ partial "shared/attribution.html" }}

    {{ if and (not .IsHome) (site.Params.socialInFooter) }}

      {{ partial "shared/social-links.html" . }}

    {{ end }}

      {{ range site.Menus.footer }}
      {{ .Name }}
      {{ end }}

In other words, place the calls to the two mathjax scripts within the  tags instead of creating a separate set of tags. And you definitely don't want to paste [ ... all of the other code is here ...] anywhere because that isn't HTML code, I was only typing that to stand in place of the rest of the code in that file!

Hi @mikedecr -
I think I would prefer something like what I did here:
https://github.com/hugo-apero/hugo-apero/blob/f761c96a11ff1c33a3aa722b4087b1ac2c6837de/layouts/partials/shared/comments.html#L8
If you want to start a branch with a PR, I'm happy to respond inline!
Thank you- Alison


[WIP] Pay UI Improvements v2

PR-Codex overview
This PR updates the Storybook configuration, adds new components and screens, improves styling, and refactors existing code.
Detailed summary

Added new components like MoveDownIcon and PayEmbedContainer
Updated Storybook configuration
Improved styling in various components
Refactored existing code for better readability

The following files were skipped due to too many changes: packages/thirdweb/src/react/web/ui/PayEmbed.tsx, packages/thirdweb/package.json, packages/thirdweb/src/stories/PayScreens/PayFlows/CryptoFlow.stories.tsx, packages/thirdweb/src/stories/PayScreens/PayFlows/OnRampOnly.stories.tsx, packages/thirdweb/src/stories/PayScreens/PayFlows/OnRampAndSwap.stories.tsx, packages/thirdweb/src/stories/utils.tsx, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/Buy/swap/SwapStatusScreen.tsx, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/Buy/tx-history/useBuyTransactionsToShow.ts, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/Buy/main/PaymentMethodSelection.tsx, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/Buy/fiat/PostOnRampSwap.tsx, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/Buy/main/TokenSelectedLayout.tsx, packages/thirdweb/src/stories/PayScreens/TransactionHistory.stories.tsx, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/Buy/tx-history/BuyTxHistory.tsx, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/Buy/confirmation/common.tsx, packages/thirdweb/src/stories/PayScreens/BuyWithCryptoFlowTest.tsx, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/Buy/main/BuyUIMainScreen.tsx, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/Buy/fiat/FiatStatusScreen.tsx, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/Buy/fiat/FiatScreenMain.tsx, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/Buy/confirmation/SwapConfirmationScreen.tsx, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/Buy/swap/SwapScreenMain.tsx, packages/thirdweb/src/stories/PayScreens/mockHistory.ts, packages/thirdweb/src/stories/PayScreens/BuyWithFiatFlowTest.tsx, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/Buy/confirmation/FiatConfirmationScreen.tsx, packages/thirdweb/src/stories/PayScreens/PayFlows/SwapFlowStory.tsx, packages/thirdweb/src/stories/PayScreens/PayFlows/FiatFlowStory.tsx, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/Buy/BuyScreen.tsx, packages/thirdweb/src/stories/PayScreens/mockQuotes.ts, pnpm-lock.yaml

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

TODO:

Add the loading state in fiat confirmation screen when initial onramp<>toToken info needs to be replaced with the postonramp quote
Create screens for swap Tx details / swap fiat details
Hook in the new screens and delete old ones
Test new flow


Gym form analysis

Adding and implementing the work from:
https://www.researchgate.net/publication/324759769_Pose_Trainer_Correcting_Exercise_Posture_using_Pose_Estimation

Can also look at the work from :https://github.com/capasitore/ai-sports-assistant


scikit-bio + simde v0.5.0: different numerical output

https://travis-ci.com/github/biocore/scikit-bio/jobs/351711278#L989
@@ -4,5 +4,5 @@
 number of groups                 2
 test statistic             1.03296
-p-value                       0.35
+p-value                       0.29
 number of permutations          99
 Name: PERMDISP results, dtype: object

Bit odd for plain SSE2 to go wrong. Does bumping up the submodule version help? Might be good to get something new from the CI.
FWIW, the PR is https://github.com/biocore/scikit-bio/pull/1709. Linking it so GitHuh would show a reference there too.

I'm hoping that 63d87d9604e025816aed575afe6cf6a46d79635d fixes the problem, but haven't checked yet.  Other than that it's hard to think of what the problem could be.
If that doesn't fix the issue I'll probably have to examine the results of every simde call to look for something different than the native results, which really isn't something I'm looking forward to.

@nemequ Thanks for that. I've rerun the tests with SIMDe 0.6.0 at https://travis-ci.com/github/biocore/scikit-bio/jobs/381064732
[doctest] skbio.stats.distance._permdisp.permdisp fails the exact same way as before: https://travis-ci.com/github/biocore/scikit-bio/jobs/381064732#L1135

☹.  Okay, I'll try to take a look soon.
What is the p-value measuring here? I'm wondering if it's possible that this could be "failing" because SIMDe's results are actually more accurate.  We have several functions which can be significantly more accurate than the native versions; for example, rcp might be implemented as 1/x instead of an approximation. IIRC our portable rsqrt is also more accurate than the native version…

I updated the scikit-bio PR to use a newer version of SIMDe and the issue went away. Thank you @nemequ and the other contributors!
The changelog of the differences since SIMDe 0.5.0 is at https://github.com/simd-everywhere/simde/releases/tag/v0.7.0-rc-1
Potential fixes could have been:
sse, sse2, sse4.1: fix ties-toward-even rounding 3208aeb @nemequ
sse4.1: better testing of _mm_round_ps b6a7310 @nemequ
sse: add simde_x_mm_round_ps with lax_rounding argument 24e5926 @nemequ
sse: fix NaN handling for _mm_max_ps, update test case 15aa0c4 @nemequ
sse2: add shuffle-based implementation of _mm_mul_epu32 e2da067 @nemequ
sse: add __builtin_nontemporal_store version of simde_mm_stream_ps 9a8001e @nemequ


add race, sex, age, ethnicity demographics for Delaware counties

Collects total_vaccine_initiated/completed by cumulative counts and by percentages for each demographic
Questions:

Percentages are calculated from populations of people > 16 years old. I've added variables to covid_demographics.csv to reflect this, but if this is not how we handle percentage-based data I can change this.
They report patient declined to disclose and data not reported for both race and ethnicity. How should these be handled? My thought was to maybe combine them together to make a general "unknown" variable. I have them removed at the moment so that the tests pass, but this needs to be addressed.

Thank you @smcclure17 I think for now we might want to track only the raw counts data.
I don't want to create any downstream inconsistencies between the percentages are calculated here and how they are computed for other geographies
Do you agree @TomGoBravo and @mikelehen ?

That makes sense! I can remove them if that's the conclusion

I think that's the right idea, thank you!


Can I get 68 key points(landmarks) from the obj file?

Hi,
Is this feasible? If it's feasible, how should I get it?
Thanks a lot!

I'm trying to map obj files to UV space, but I'm not sure if this is feasible. Has anyone tried anything like that?

Hi! Have you solved the problem?I'm trying to do the same thing.

I'm trying to map obj files to UV space, but I'm not sure if this is feasible. Has anyone tried anything like that?

Hi! Have you solved the problem?I'm trying to do the same thing.

Later I moved on to another project, and I didn't succeed in the attempt here

Author

hi~can u share which project did you attempt?


Exclude impulse wav files is hrtf is disabled

hrtf means "Head-related transfer function". It's to put the source
in a 3-D related position to the listener. Without this feature,
we can remove all the impluse wav resources.
This patch is backported from the crosswalk-lite branch
BUG=https://crosswalk-project.org/jira/browse/XWALK-3929
TEST=Build with/without flag disable_webaudio_hrtf and simple test on XWalkCoreShell.apk
SIZE_REDUCED=140k(Release build on ARM)

@halton

LGTM ping @darktears


Switch order of module pattern checks for more consistent UMD.

I was having errors thrown bundling loglevel in a library that was then being bundled in an application package and found that reversing these checks solves the issue for us.
Based on this conversation: https://github.com/umdjs/umd/pull/22 it seems that checking for AMD before CommonJS is a safer paradigm to follow.

That sounds good, but this change has broken all the tests. Can you take a look at why? I'm happy to merge a change like this, just as long as the tests pass :smile:

Hey - that's because I'm really good at breaking tests inadvertently. Apparently, my copy-paste skills are rusty and my amd definition did not properly parallel what already existed in code.
I've updated and local tests went from red to green, so with a little bit of luck, the travis ci build should do the same.

Looks great, thanks! Merged.

Thanks for this fix!
@pimterry do you plan do a minor release in the near feature? I would like to use loglevel with AMD as soon as possible :-)

All good now; this is now up and released as 1.4.1. Let me know if you have any other trouble.


Negative flatten

[ [ 'a', 'b' ], [ [ 'c', 'd', 'f' ] ] ]
What I need to get is [ [ 'a', 'b' ], [ 'c', 'd', 'f' ] ] regardless of deepness of the input.
Maybe I'm loosing something and I can easily archive this with existing functionality.. Atm I think that list.flatten(-1) would be natural way to do such thing.

Also I think about list.flatten(untilPredicate)

What about list.map((inner) => inner.flatten())?

@samwgoldman your point is valid, but

Alexey, I don't fully understand the behavior you're looking for. Could you supply some various examples of arbitrary nesting and the expected output of this operation? I'm curious for more examples of data of type:
type NestedValue = Array | String

That should be Array< NestedValue >

@leebyron I want to flatten list to List and not , Array< NestedValue > . I think it's obvious from my examples.

Sorry, I meant List, not Array. But also, it's not yet obvious to me from your examples, so please humor me with more input to understand the pattern.
Here are the examples you've provided so far:
[ [ 'a', 'b' ], [ [ 'c', 'd', 'f' ] ] ] => [ [ 'a', 'b' ], [ 'c', 'd', 'f' ] ]
[ [ [ 'a', 'b' ], [ [ 'c', 'd', 'f' ] ] ] ] => [ [ 'a', 'b' ], [ 'c', 'd', 'f' ] ]

I'm curious about:
[ [ 'a', [ 'b' ] ], 'c' ] => ?
[ [ [ 'a' ] ], 'b' ] => ?
[ [ [ 'a' ] ], 'b' ] => ?
[ 'a', [ [ 'b' ] ], 'c' ] => ?
[ 'a', [ 'b', [ 'c', [ 'd' ], 'e' ], 'f' ], 'g' ] => ?

For my concrete case I want
[ [ 'a', [ 'b' ] ], 'c' ] => [ [ 'a', [ 'b' ] ], 'c' ]
[ [ [ 'a' ] ], 'b' ] => [ [ [ 'a' ] ], 'b' ]
[ [ [ 'a' ] ], 'b' ] => [ [ [ 'a' ] ], 'b' ]
[ 'a', [ [ 'b' ] ], 'c' ] => [ 'a', [ [ 'b' ] ], 'c' ]
[ 'a', [ 'b', [ 'c', [ 'd' ], 'e' ], 'f' ], 'g' ] => [ 'a', [ 'b', [ 'c', [ 'd' ], 'e' ], 'f' ], 'g' ]
But I just propose, I think it would be useful if it could be controlled by list.flatten(untilPredicate).
If you know any other solution for my case, I will appreciate a lot..

What is your predicate in this case?
We would likely need a different kind of method or utility function for this sort of operation, since the flatten function is "greedy" to maintain an optimal O(N) performance.

list.flatten((v) => "List contains not Lists") . The predicate could signal on witch point it should stop flatten.

My real-life case is
static getMapPaths(map) {
        map = fromJS(map);
        getPaths = (map, result = new List([])) => (
            (!Map.isMap(map) || map.isEmpty()) && result.toArray() ||
            map.flip().toList().reduce(
                (res, k) => res.push(getPaths((map.get(k)), result.push(k))),
                new List()));
        return fromJS(getPaths(map).flatten().toArray());
    }

Maybe there is a better way to implement that kind of function, but in this particular implementation I get nested Lists and I need to flatten it somehow, but I don't want to loose separate paths, which are represented as Lists. Which is why I was required to separate actual "paths" by JS Arrays from nested Lists and convert it to Immutable data structures.
Original idea was:
static getMapPaths(map) {
        map = fromJS(map);
        getPaths = (map, result = new List([])) => (
            (!Map.isMap(map) || map.isEmpty()) && result ||
            map.flip().toList().reduce(
                (res, k) => res.push(getPaths((map.get(k)), result.push(k))),
                new List()));
        return getPaths(map).flatten(-1); // or .flatten((v) => "List that contains not Lists")
    }

I'm having trouble following the purpose and behavior of the getMapPaths function, but I think I better understand now what you're looking for from a flattening point of view.
Hopefully this code will help you out:
import { List, Iterable } from 'immutable';

function flattenIf(pred, list) {
  var flatList = List();
  function flatDeep(_list) {
    _list.forEach(function (item) {
      if (Iterable.isIterable(item) && pred(item)) {
        flatDeep(item);
      } else {
        flatList = flatList.push(item);
      }
    });
  }
  flatDeep(list);
  return flatList;
}

function isListOfLists(list) {
  return list.every(Iterable.isIterable);
}

var customFlatten = flattenIf(myDeepList, isListOfList)

I studied your getMapPaths function a bit and I think I understand what you're trying to do here, please correct me if I'm mistaken:
You want to provide a Map of Maps, like
var deepMap = { a: { b: true, c: true }, d: true, e: { f: { g: true } } };

And then produce a list of all paths to leafs, like
[ [ "a", "b" ], [ "a", "c" ], [ "d" ], [ "e", "f", "g" ] ]

If this is the goal, then you should be able to write a recursive reducer for doing this without a need for flattening afterwards. You should also avoid going back and forth between native JS arrays and objects and Immutable collections by using fromJS and toArray within utility methods like this - those require full copies and will add up to be quite poor on performance and should not be necessary.
Here's a recursive reducer version of getMapPaths which expects Immutable collections as input and returns Immutable collections as output without producing deeply nested lists.
function getMapPaths(deepMap) {
  return _reducePath(List(), deepMap, List());
}

function _reducePath(paths, map, path) {
  return !Map.isMap(map) || map.isEmpty() ?
    paths.push(path) :
    map.reduce(
      function (paths, subMap, key) {
        return _reducePath(paths, subMap, path.push(key))
      },
      paths
    );
}

Then to use it:
var deepMap = fromJS({ a: { b: true, c: true }, d: true, e: { f: { g: true } } });
var paths = getMapPaths(deepMap);
console.log(paths.toString());
// List [ List [ "a", "b" ], List [ "a", "c" ], List [ "d" ], List [ "e", "f", "g" ] ]

@leebyron Great! ) It recalled me on SICP courses ). Also I recognized that I need different algorithm and different output data structure for my problem.. Your explanation and examples did great job for me to understand it. ATM believe that my concrete request was provoked by wrong approach which was leaded to ~maybe~ wrong need (I'm about negative flatten param and/or predicate). If someday I catch myself on it I'll think twice until I'll be dead sure that we need it.
Thank you and your colleagues for what you're doing for open source, it's huge )

Thanks, I'm closing this task out since we're in a better place :)


Add missing permission name

Permission cms.change_page is missing name when created in migration 0010.
Added migration which update value of that Permission.

Coverage decreased (-0.01%) to 88.446% when pulling 8cd5b28c9c6d5094cb64dab678e6ad9bc3d659ae on piotrjakimiak:develop into 9c92ccb1aa713ddd6a7e4d0eaa43f5f82d7332a6 on divio:develop.

@piotrjakimiak Thank you ! Your commit fixed my bug - https://github.com/divio/django-cms/issues/5053

LGTM. I'll apply this to 3.1 (when first error appeared) and then I'll forward port this

Coverage remained the same at 88.46% when pulling f5a144522dc33da89b9db94399969717516853b5 on piotrjakimiak:develop into 9c92ccb1aa713ddd6a7e4d0eaa43f5f82d7332a6 on divio:develop.

@piotrjakimiak thanks for sharing this. I applied to 3.1, will forward port in the next days


Last login

When the user comes online we could send info about it to the server along with a current version of frontend app. This would be handy during testing compatibility with previous versions of the app.
It also might be useful to display this information on the admin page.

I like the idea, but I don't think that it should be sent right when the User is online.
I'm more in a favour of adding another key to the POST & GET requests with the version. This way we could actually distinguish similar requests that come from different versions of the frontend.
The reason behind it is that I don't think that we actually care if user gets newer version of our app when he gets access to the network as long as the newer version is backward compatible with the previous one.
And by backward compatible i mean if the forms will be accepted by the server.
So the only thing that matters to us in terms of compatibility is just the form data.
Here's an example - some Driver is long time rider and doesn't have internet connection very often, so he's syncing one time per month. He's sending the usual form (the 2020-02 one) which is without the version key, so the backend (now at let's say 2020-04) accepts it and treats a s a 'legacy' one.
Another Driver might have better luck accessing the web so he's using the later, 2020-04 version which sends POSTs with version: 1 key, and that lets us treat his form bit differently, according to the latest features.
As you said we can display this in admin page basing the Driver's app version on the schema of his last GET/POST request. This will also allow us to check how many drivers are using the old schema in order to not push the changes too far (as that would increase complexity in code).
Right now we have no idea who's using which version other than inspecting POST errors, and we certainly can do better than that :)
What do you think about it?

Adding the version to "existing" requests sounds more reasonable and also easier to implement 😄 sounds ok

Ok, so I actually did google this a bit and found out that the version key might not cut it, because GETs don't send any form (duh). So maybe custom header instead?:
https://restfulapi.net/versioning/

Guys, you need to remember, that tags are created (therefore named) after development has occurred. Also, we should figure out how to deprecate older versions, as at some point it would be beneficial to remove older endpoints, as their maintenance would increase complicity.

Guys, you need to remember, that tags are created (therefore named) after development has occurred.

That was just an example of and idea how we could approach this in the future (hence made up tag name).

Also, we should figure out how to deprecate older versions

Agree

as at some point it would be beneficial to remove older endpoints

Did you mean the API endpoints? We only have a few right now...
Or maybe git tags? Or different versions of the application?

This is blocked by #438

Ticket stays

it is useful for planning deploys
helps with debugging


ISINs in random mode always give GB as the country code

Bug Report
Steps to Reproduce:
{
  "schemaVersion": "0.1",
  "fields": [
    {
      "name": "isin"
    }
  ],
  "rules": [
    {
      "constraints": [
        {
          "field": "isin",
          "is": "ofType",
          "value": "ISIN"
        },
        {
          "not": {
            "field": "isin",
            "is": "null"
          }
        }
      ]
    }
  ]
}

Expected Result:
First two characters to differ
Actual Result:
First two characters in every row are "GB"
Related:
#1192

Full Sequential Mode also doesn't generate all country codes. For example, I think AZ is a valid country code, so should come first in Full Sequential Mode.


2D Guide couldn't be created in 4.0

Godot version
v4.0.beta1.official [20d667284]
System information
Windows 10, Vulkan, NVIDIA GeForce RTX 3050 Laptop GPU
Issue description
In 3.x, a 2d guide could be created by dragging the axis, but in 4.0 I tried the same method but nothing happened.
There is still a clear guide option so the feature isn't deleted.
Steps to reproduce
click and drag on the left-up corner to the 2d viewport can't create the guide.
Minimal reproduction project
No response

I can't reproduce this on Linux or Windows.

FYI, the guide won't be created when there's nothing in the scene, same for 3.x.

FYI, the guide won't be created when there's nothing in the scene, same for 3.x.

Could we lift this limitation somehow?


Incorrect rendering of front page on mobile devices

FYI, http://depsy.org/ does not render properly on smartphones and tables.  The page is right truncated (with a white background) when you scroll.  'Developer tools' in Google Chrome will allow you to reproduce and troubleshoot this.

Thanks! Yeah, we are an absolute mess on mobile right now...we took the very 2002-era, desktop-first approach in developing, wholly to save time and launch faster. This made sense for us since we still get around 90% of visitors from there at Impactstory, a similar project.
So, while improved mobile experience is very much on the roadmap, it's not our chief priority right now alas.


update anoushkashishodia's slide

This PR updates the caption to @Anoushkashishodia slide.

Looks good, added it to Caption Contest.


resources.relativePath does not work with windows like paths and not file scheme

If two resources have windows like paths and with non-file scheme then resources.relativePath is giving me wrong values.
resources.relativePath(URI.file('c:\\foo\\bar\\').with({scheme: 'vscode-userdata'}), URI.file('c:\\foo\\bar\\a1\\a2').with({scheme: 'vscode-userdata'}))

Expected: a1\a2
Actual: ../c:\foo\bar\a1\a2

I ran your example on Window and it results in a1/a2 which is correct.

URI.file('c:\foo\bar\') only works as you expect when ran on Windows. \ is a valid file name character on Linux/Mac, so URI.file would not try to convert '' to '/'
For non-file URI, the resource util uses posix path logic: Doesn't know about path roots (/C:/), nor does it treat \ as separators.

This is the sample that causes issues:
resources.relativePath(URI.file('c:\\foo\\bar\\').with({scheme: 'vscode-userdata'}), URI.file('C:\\foo\\bar\\a1\\a2').with({scheme: 'vscode-userdata'}))
We're missing the 'ignoreCase' argument on resources.relativePath


Add tests

We currently have neither any unit nor integration tests. The bigger and more complex the project grows, the more problematic this will become, so we should address this soon.

@zbazztian Hey I would love to contribute to this issue and see what requirements or prerequisites you have for this project so far!

@cloudguruab happy to collab with you on this effort! Could you help plan out the test cases here whenever you have a moment? Some things to consider:

We have users who use Jira Server.
We have users who use Jira Cloud.
Some users use the CLI.
Some users use GitHub Actions.
Whether using the CLI or Actions, some users use the issue_end_state_flag.

It'd be cool to create a GitHub Actions workflow to run integration testing on these test cases.

@cmboling Awesome, I will go ahead and allocate some time at the end of the week to tackle planning out these test cases! In the meantime, I will send over any questions if I come up with some.

@cloudguruab Noice thanks so much. For sure let us know if you have any questions!

Hello everyone.
This issue is open for quite sometime now, and I don't see a single test file in this repository.
Asking about what you prefer:

Integration tests only or integration tests and unit tests ?
Something else or pytest is fine ?

First I wrote formally out of respect for your work (really appreciate it). But then thought that no-one has the time to read essays.

I hope I get to work on this.


multithreading in scipy.fft disabled on Windows

This is an issue that I just found again. We haven't fixed it, so this will be a performance regression for scipy.fft functions on Windows for 1.9.0 in wheels and conda-forge packages (it's gated on Mingw-w64 usage, so the conda packages in the defaults channel or wheels from https://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy will not be affected).
Usage of pthreads is disabled here: https://github.com/scipy/scipy/blob/main/scipy/fft/_pocketfft/meson.build#L13-L17
The in-progress fix at https://github.com/mreineck/pocketfft/pull/3 does not yet work.

I'm sorry that I don't have positive news on this ... since I neither have access to Windows nor the required expertise with MINGW, I can't take an active role.

@peterbell10 and I discussed this, and agreed that a fix is nice to have but not a blocker for 1.9.0, because it is only a performance issue and not a correctness one.

@mreineck - can I help with setting up Windows access somehow?   I've done that on various machines, virtual remote, virtual local, and via dual boot (on an Intel Mac).

Unfortnately I don't think this would help much ... the main problem is not that I don't have access to Windows, but rather that I have no experience with it. The issue is not really a bug in the FFT code but some low-level quirk in MinGW, and I'm no use at all when it comes to locating and fixing (or working around) this :-/

Is there any update on this issue?
When I set workers via set_workers() and read them through get_workers() it always returns 1, I guess this means multithreading is disabled on my system..
Would rebuilding locally with MSVC help?

Yes, rebuilding with MSVC will give you multi-threaded behavior. You'll need a matching Fortran compiler though - Intel Fortran works best.
No other updates on this issue.


[SMALLFIX] Replace anonymous type with lambda

Java 8 Improvement: replace anonymous type with lambda in alluxio/core/common/src/main/java/alluxio/util/ExtensionUtils.java#listExtensions

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2601/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2602/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2604/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2612/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2613/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2614/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2615/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2617/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2618/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2619/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2627/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2628/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2629/
Test PASSed.

alluxio-bot, merge this please


v 6.+ 3d meme version

this is a last 3 month work update

[6.1 & 6.2 updates]

added PROXY message (uses recieved buffer as own gamebuffer)
fix: find file close descriptor added
new: added 3d display
added texture display
added runtime textures swap on f5
added field rotation along Y and Z axises
added lighting
added help page

[MAJOR Win 11 bug fix!]

now game runs on windows 11 devices!
background tex rotation added
& fix (invoked when game over)
game end sound bang effect
added runtime custom wav sound load
[Added mouse controls
help text reduced (due to its unnesesity)
added texture swap on music effect
added screen ratio calculations
added possibility to load custom textures on demand
added random end effect choose (4 variants)
removed depth loop from the version (unused)
[major send back fix]
3d 2d compile swap added
mouse controls bug detected (add switch off?)


scicalc: Add swipe up action

Swiping up has now the same effect as hitting the "<" button.
@berkenbu Hope this is ok with you. I find it easier to correct typos that quickly appear with a single swipe instead of navigating to the correct screen first.

To be honest having left swipe for undo, right swipe to calculate result and accordingly up/down swipe to cycle between screens would feel more natural for me. But I did want to touch the original input actions.

Original author of the calculator here:
I have this (arguably arbitrary) mental image of the 3 input screens being arranged side by side, hence the lr swipe gesture to switch between them.
That being said, I am not married to that idea, if others find your arrangement more natural I am not against changing it to that.

This PR looks good to me - as it's just adding the swipe up functionality rather than changing things - so merging now.
Although I can definitely see the reasoning behind having swipe left as delete, and right as calculate result. While arranging the screens left/right makes sense, I think most users probably expect left swipe=delete/back, and given that arranging the screens up/down instead probably is preferable.
... so I'd be up for changing it. It might also be an opportunity to use Bangle.setUI to get user input as well?

ok lets do it


WIP: Add recipe and helper for deploying media

This adds a recipe and inc file for deploying test media. Media is
extracted from a tarball specified by SRC_URI, and known media types are
extracted and grouped in to packages, installed in /usr/share.
Audio filed can be installed by including test-multimedia-audio, video
files are in test-multimedia-video, etc.

Looking for feedback on this, what do you think? The .inc-file should allow users to supply their own test media, and the default gstreamer test media is hosted by Pelagicore and free for anyone to use.

There were some functional issues with the previous patch:

Packages were created for file types with no files present
"meta" packages (called file packs in the code) for video, audio and picture extensions were generated even when those extensions did not have any files present

I fixed the above issues, and also fixed the URI issue pointed out by @rpannek. The cert issue is still present, and is being worked on, but does not affect the code in the patch.
I removed the WIP status, and I consider this ready for merging, so please re-review :)

Final fiddle from my part.. I moved the inc file to a bbclass. This makes it easier to add the "test media" capability to a recipe while retaining existing functionality. I added this as a separate commit to make it easier to read. When accepting this, please squash the commits together, or let me know and I will do it.


Two product production decision

Borderline Basic/Intermediate.  It's a basic exercise in the scheme of integer linear programming, but there isn't that much documentation on Z3 linear optimization

As a reminder we will also need to update the repository config.json file when we merge this

As a reminder we will also need to update the repository config.json file when we merge this

Satisfied in #54


Add new regression data download method and update github actions ubuntu version

Fix #688.
Fix #686

Just set up a test environment on ada and ran all the tests with no errors. So maybe it's a problem with the github workflows?

My first check would be how the python dependencies are installed in the test workflow, which seems to be just with a pip install . and the versions are not the same that's in the poetry.lock.
It has been so long that it might be installing different versions with unexpected results.
So changed the test workflow to also install using poetry.


Extension has ceased working entirely; is this abandoned?

Reviews on the Chrome Extension library show this is not an issue of user error; it seems Trello or Chrome updated in some way and it broke this extension.
I had migrated to using this extension when a different one with similar utility had this same issue and just broke one day... Has the same thing happened here? Is this project abandoned, or is a fix coming...?

Sad...same here

This is due to changes in the Trello HTML content so the cards are no longer being detected properly during card processing. I have just submitted a pull request that resolves this issue (as well as the removal from the Chrome extension marketplace due to non-compliance with manfest v3 see #36)  so hopefully it can be merged and a new version of the plugin promoted to the Chrome extension store.

The last update on GitHub was three years ago, so--sadly--my money is on "abandoned."


Encounter moved right... when the encounter did not move right.

Ever since updating to 1.5.1 I have noticed that I have seen the "encounter moved right" and it throws the ball to the right when the pokemon did not move to the right. It did eventually catch it this time, but I have seen in the past where it just wastes pokeballs.
10:11:48.274 INFO    | {'type': None, 'status': None, 'dex': 187, 'name': 'Hoppip', 'form': '???', 'shiny': False, 'iv': 84, 'atk_iv': 12, 'def_iv': 15, 'sta_iv': 11, 'cp': 12, 'level': 13, 'gender': '?', 'pvp_info': {'GL': {'dex': 188, 'name': 'Skiploom', 'rating': 94.29, 'cp': 1058, 'level': '50'}, 'UL': {'dex': 189, 'name': 'Skiploom', 'rating': 95.96, 'cp': 1789, 'level': '50'}}, 'screen_x': 0, 'screen_y': 0, 'latitude': 0, 'longitude': 0}
10:11:48.782 INFO    | Action: catch pokemon
10:11:48.783 INFO    | Current trial #1
10:11:49.123 INFO    | Action: feed berry
10:11:52.366 WARNING | No selectable berry.
10:11:52.368 INFO    | Action: select poke ball
10:11:52.368 INFO    | Skipped selecting ball for Hoppip (IV84 | CP12 | LVL13).
10:11:53.318 INFO    | Encounter moved right
10:11:53.318 INFO    | Action: throw pokeball
10:11:55.956 INFO    | Missed hitting...
10:11:55.956 INFO    | Current trial #2
10:11:57.810 INFO    | Encounter moved right
10:11:57.810 INFO    | Action: throw pokeball
10:12:00.476 INFO    | Missed hitting...
10:12:00.478 INFO    | Current trial #3
10:12:02.522 INFO    | Action: throw pokeball
10:12:05.130 INFO    | Hoppip (IV84 | CP12 | LVL13) was caught.
10:12:06.144 INFO    | Keep Hoppip (IV84 | CP12 | LVL13).
10:12:18.454 INFO    | Searching for pokestop to spin...

@MerlionRock, I can assist with this if given access to the code.

This was added to detect pokemon that moves. While it isn't 100% perfect, it's better than wasting 9 balls on Pokemon that move left and right all the time.
Alternatively, latest version of PGSharp now support 100% hit. And therefore this won't be an issue anymore.


[SMALLFIX] Remove synchronize on remove user.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/494/
Test PASSed.


Relay mode dimensionalised postfix collector

Since we were able to get https://github.com/disqus/postfix-stats/pull/4
successfully merged into the upstream postfix-stats project, we can now
allow the postfix diamond collector to work in relay mode as well as non
relay mode.

@gnufied I would appreciate your input here

@tsheasha can we also try and get tests we wrote for postfix-stats daemon merged into upstream repo and possibly get travis-ci setup going for it, since we depending on upstream project we kinda ensure that, it meets our quality requirements (at least as best as we can).

@gnufied go for it and poke the maintainers of postfix-stats when you are ready. I think that is orthogonal to this PR however.

+1
Pretty cool that postfix-stats accepted our changes. @gnufied, I agree what you propose would be very good. Not essential though, if they don't care to do so I'm not going to lose any sleep. Reason for this being we pin the version of their code anyway.

I am not sure where to leave my comments now but the code that got merged upstream into disqus is not threadsafe and the handlers are being called concurrently from multiple threads.
This will causes messages to be categorized under incorrect relay_statuses.

Left a message in upstream repo - https://github.com/disqus/postfix-stats/pull/4/files#diff-e05df3c3f065b9851bf4b21c023400b3R202

I pushed a fix for that https://github.com/disqus/postfix-stats/pull/5
A very simple fix

let me know what you think

@zachm My new pull request in disqus has been merged

+1


Issue 329 deepcopy

Description
Create function make_new_copy to replace deepcopy
Fixes #329
Type of change

[x] New feature (non-breaking change which adds functionality)

Key checklist:

[x] No style issues: $ flake8
[x] All tests pass: $ python run-tests.py --unit
[x] The documentation builds: $ cd docs and then $ make clean; make html

You can run all three at once, using $ python run-tests.py --quick.
Further checks:

[x] Code is commented, particularly in hard-to-understand areas
[x] Tests added that prove fix is effective or that feature works
[ ] Any dependent changes have been merged and published in downstream modules

@martinjrobins - have now updated to your suggestion above


Fixed install --partition and added scripts/run-install to use it in …

…a local container
Signed-off-by: Sven Dowideit<EMAIL_ADDRESS>

this should head towards fixing #1667 - but I've not tested it enough to know if it doesn't also break other things yet.

this has broken publishing to AWS, to I've got to work out how and why.


merge package template from Kronos-Tools/npm-package-template

package.json

chore(engines): update<EMAIL_ADDRESS>from template

.travis.yml

chore(travis): merge from template .travis.yml

Codecov Report

Merging #107 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master    #107   +/-   ##
======================================
  Coverage    90.9%   90.9%
======================================
  Files           1       1
  Lines          22      22
======================================
  Hits           20      20
  Misses          2       2

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cf6c2b3...b8e2766. Read the comment docs.

Coverage remained the same at 82.143% when pulling b8e276698cfbb6c679da7ddf7ed0889af99ae4ee on template-sync-1 into cf6c2b37e9b4668a170cf61b6b2fcf039e12bacb on master.


ibc: relayer can't find nodekey during packet proof query

Describe the bug
The relayer currently fails to process withdrawals between Penumbra chains with the following error:
2023-07-13T21:58:00.807818Z     error   Error assembling packet message {"path_name": "penumbra_path", "chain_id": "penumbra-devnet-local-986947bb", "client_id": "07-tendermint-0", "msg": {"type": "timeout_packet", "src_port": "transfer", "src_channel": "channel-0", "dst_port": "transfer", "dst_channel": "channel-2", "sequence": 1, "timeout_height": "0-28824", "timeout_timestamp":<PHONE_NUMBER>, "data": "CkFwYXNzZXQxOTg0ZmN0ZW53OG0yZnBsOGE5d3pndXpwN2ozNGQ3dnJhdnJ5dWhmdDgwOG55dDlmZGdncXhtYW5xbRIBMRqSAXBlbnVtYnJhdjJ0MXg0OGhkY2pxYTVqbGtneDJ3bmpzMHNnNjducWh3dHgwOGxreHVmNmo5YTh1cTk2MHlwbGNudnd1ZjJubm55NTM3OTJxZDg0bTIzY2o3azl0MjNlaDdlNmt3NTJsam4zcjNnZXZqbGttaGE5em54OXVxcXo3cWc0ZHQ4NmM5eTJuazA2bmh5IpIBcGVudW1icmF2MnQxa25lcWp4eDhndDJ0d2M4ODVnZm43bjJ0azY4bTczMDQzMmh4bHpuNDdocnptejI5YTR0aGNjejR5cWZrZnZqdm1qa3d4cHk0N2NscWhmNDVkOHpxOWo5ZWZ4ajk0YzJqeWhnem01OXNsZjhha3Bnd3cwNnh6ZWthNncyazA5ZnFhcWtncW4=", "ack": ""}, "error": "error querying packet proof: error querying tendermint proof for packet receipt: rpc error: code = Unknown desc = Missing node at NodeKey { version: 2338, nibble_path: 49c5 }."}

To Reproduce
Steps to reproduce the behavior:

Fire up a local devnet/preview connection, following docs in https://github.com/penumbra-zone/penumbra/tree/ed1cb0734aadb36059eb147c57b30694ff578002/deployments/relayer#running-a-local-devnet
Run the relayer via rly start
Attempt a withdrawal: pcli -n http://localhost:8080 tx withdraw penumbrav2t1kneqjxx8gt2twc885gfn7n2tk68m730432hxlzn47hrzmz29a4thccz4yqfkfvjvmjkwxpy47clqhf45d8zq9j9efxj94c2jyhgzm59slf8akpgww06xzeka6w2k09fqaqkgqn@penumbra-testnet-ganymede-be7bab59 upenumbra 1 channel-0

Note that you'll need to update the chain id in that address to be the actual chain id of testnet-preview. To find it: curl -s https://rpc.testnet-preview.penumbra.zone/status | jq -r .result.node_info.network.
Additional context
I made sure to update the to the latest penumbra protos in my local relayer build, and that did not resolve the problem. Let's dig around and figure out why the relayer is not happy looking up the packet proof.

Can we run the relayer with Debug so we can see the output of this debug statement? (in PenumbraProvider.QueryTendermintProof)
cc.log.Debug("Querying K/V", zap.String("ChainId", cc.ChainId()), zap.Int64("Height", height), zap.String("Key", string(key))

I think this error is returned by pd when we request a state key that doesn't exist

this error is emitted by our jmt's TreeReader:
https://github.com/penumbra-zone/jmt/blob/3d374ed7de884d50cbcd9559ef0a53adaacbf828/src/reader.rs#L13
we're querying for a packet commitment that doesn't exist (this is probably due to a channel id getting swapped, using the wrong channel id in the relayer due to an issue in our query or events api- note that the channels are different in your test, channel-0 vs channel-2)

I found one issue: our state path for packet receipts is wrong:
https://github.com/penumbra-zone/penumbra/blob/f5bfd6868c28ac1ce98c9443fa385602fc8909de/crates/core/component/ibc/src/component/state_key.rs#L78
it should instead be
receipts/ports/{}/channels/{}/sequences/{}

I think #2837 should fix this

Testing this today, will report back with results.

Unfortunately I'm still seeing the error, testing on the latest main, which includes #2837:
2023-07-14T18:51:02.267037Z     debug   Querying K/V    {"provider_type": "penumbra", "ChainId": "penumbra-testnet-ganymede-0b5ffff4", "Height": 1153, "Key": "receipts/ports/transfer/channels/channel-0/sequences/2"}
2023-07-14T18:51:02.312904Z     error   Error assembling packet message {"path_name": "penumbra_path", "chain_id": "penumbra-devnet-local-3a758144", "client_id": "07-tendermint-0", "msg": {"type": "timeout_packet", "src_port": "transfer", "src_channel": "channel-0", "dst_port": "transfer", "dst_channel": "channel-0", "sequence": 2, "timeout_height": "0-29165", "timeout_timestamp":<PHONE_NUMBER>, "data": "CkFwYXNzZXQxOTg0ZmN0ZW53OG0yZnBsOGE5d3pndXpwN2ozNGQ3dnJhdnJ5dWhmdDgwOG55dDlmZGdncXhtYW5xbRIBMRqSAXBlbnVtYnJhdjJ0MXF0Y2ZlNjNzaHFjaHV1ajB6bHRydHhuamF5aHdkY2VrdmVhbTRzNXB3OHNzZHhyMDMyaGc4YXJua2VwNzl6NmpwNzh3dWw3dDN1MDJuZHhtZmsyeWMzdnF1OHduNmp3Y2R1NWt1Y3E0Y3dueTU0cGN6enluNTdha2d3NDlqMHMzMGtteXB0IpIBcGVudW1icmF2MnQxaGs0aHFsMG4zdWpsczU1ajBkd3Q4Y2w0a3NhdTc0eTBmMzlheWV3aGVsdXFjcG10bmEzamx4MGduZjIzemt6cjA5aGhyNXg3Z2x1eTJqdDgwMGRxMGo3Y3p1bGtyYWdwbmVjaHY3Yzh0czVoZDRjOXZsZDVhdW42cWFrN2p3bXByd256YWw=", "ack": ""}, "error": "error querying packet proof: error querying tendermint proof for packet receipt: rpc error: code = Unknown desc = Missing node at NodeKey { version: 1153, nibble_path: ba78 }."}

so a bit more investigation is required.

This issue is resolved. In fact, it's so resolved that we have working IBC transfer between two Penumbra test chains:
$ pcli -n https://grpc.testnet-preview.penumbra.zone view balance
2023-07-20T21:33:36.601509Z  INFO penumbra_view::worker: starting client sync
Scanning blocks from last sync height 887 to latest height 888
[0s] ██████████████████████████████████████████████████       1/1       36/s ETA: 0s
 Account  Amount
 0        5transfer/channel-2/upenumbra

which is huge, and deserves its own writeup. I'm closing this issue as complete, but also cross-referencing related PRs that patched blocking bugs:

https://github.com/penumbra-zone/penumbra/pull/2816
https://github.com/penumbra-zone/penumbra/pull/2820
https://github.com/penumbra-zone/penumbra/pull/2828
https://github.com/penumbra-zone/penumbra/pull/2829
https://github.com/penumbra-zone/penumbra/pull/2837
https://github.com/penumbra-zone/penumbra/pull/2853


Unable to receive FSK packets after switching modems

Hi I'm trying to switch an sx1278 between lora to fsk to fsk direct mode in a loop, it doesn't seem to work properly, can you make a quick sketch to show me how it's done? Thanks

it doesn't seem to work properly
What exactly does that mean? All the methods return status codes, it might be helpful to know them. And it would be really helpful to know your code.

All the status codes return success but the receiver doesn't receive anything.
This is where all transmission call through:
https://github.com/Bambofy/FOSSASAT-1/blob/master/Code/Beacon/communication.cpp#L334

This is the code I used for testing
#include <LoRaLib.h>

#define DIO2  6

SX1278 lora = new LoRa(10, 2, 5);

void setup() {
  Serial.begin(9600);
  pinMode(DIO2, OUTPUT);
}

void loop() {
  Serial.println("LoRa");
  lora.begin();
  delay(1000);
  for(int i = 0; i < 5; i++) {
    Serial.println("Sending LoRa");
    lora.transmit("Hello World!");
    delay(1000);
  }

  Serial.print("FSK\t");
  Serial.println(lora.beginFSK());
  delay(1000);
  for(int i = 0; i < 5; i++) {
    Serial.println("Sending FSK");
    lora.transmit("Hello World!");
    delay(1000);
  }

  Serial.print("AFSK\t");
  Serial.println(lora.directMode());
  delay(1000);
  for(int i = 0; i < 5; i++) {
    Serial.println("AFSK tone 400 Hz");
    tone(DIO2, 400);
    delay(1000);
    Serial.println("AFSK tone 1000 Hz");
    tone(DIO2, 1000);
    delay(1000);
  }
  noTone();
  lora.packetMode();
}

It sends 5 LoRa packets, 5 FSK packets, transmits 10 FM tones and then sends RTTY string. On SDR, all transmissions are sent, although you are right that I can't receive FSK packets after the switch from LoRa modem to FSK.

You're also evaluating return codes wrong: e.g. here you rewrite the value of err_check over and over again, so the ony thing you know is setSyncWord return code, all the previsou are lost.
Why not just pass all the configuration to beginFSK()?

Yep will fix the err_check lazyness, i can't tell which part of the example code above is RTTY transmitted? I saw you used lora.directMode( INT VALUE FOR FREQ ) in a previous code snippet is that still usable? or do I have to use tone() to send data through directMode?

Sorry, I posted wrong code, here's the correct version:
#include <KiteLib.h>

#define DIO2  6

SX1278 lora = new Module(10, 2, 5);
RTTYClient rtty(&lora);

void setup() {
  Serial.begin(9600);
  pinMode(DIO2, OUTPUT);
}

void loop() {
  Serial.println("LoRa");
  lora.begin();
  delay(1000);
  for(int i = 0; i < 5; i++) {
    Serial.println("Sending LoRa");
    lora.transmit("Hello World!");
    delay(1000);
  }

  Serial.print("FSK\t");
  Serial.println(lora.beginFSK());
  delay(1000);
  for(int i = 0; i < 5; i++) {
    Serial.println("Sending FSK");
    lora.transmit("Hello World!");
    delay(1000);
  }

  Serial.print("AFSK\t");
  Serial.println(lora.directMode());
  delay(1000);
  for(int i = 0; i < 5; i++) {
    Serial.println("AFSK tone 400 Hz");
    tone(DIO2, 400);
    delay(1000);
    Serial.println("AFSK tone 1000 Hz");
    tone(DIO2, 1000);
    delay(1000);
  }
  noTone(DIO2);
  lora.packetMode();

  Serial.print("RTTY\t");
  Serial.println(rtty.begin(434.0, 183, 45));
  delay(1000);
  rtty.leadIn(500);
  rtty.println("Hello World!");
  lora.standby();
  delay(1000);
}

thanks i'll let you know how i get on with this.

I'm pretty confused by this bug. It seems that after switching when switching to FSK from LoRa modem, something in the chip changes and the FSK transmissions can no longer be received by a module with the same settings. Here's what I know so far:

I did a register dump before and after switching modems (FSK -> LoRa -> FSK). It looks like all the configuration registers are the same, with the notable exception of register 0x30 (RegPacketConfig1). Before the change, this register reads 0x90 (expected value) but afterwards, it reads 0x00. The weirdest part is that it seems stuck there - writing to the register is ignored, regardless of chip mode (standby/sleep).
When comparing the FSK spectrum before and after the modem change, it seems changed - there's one extra spike on the lower end of the band. I think this is the reason why the receiver doesn't pick up the packet, but I'm not sure.

I tried resetting the chip both by pulling the RESET pin low (as per datasheet), as well as power cycling and neither seems to resolve the issue.

The only attempt at solution that I can think of now is trying to configure every single SPI register each time user calls either one of the begin() methods. Right now, not all registers are configured, only the necessary ones (in previous versions of LoRaLib this wasn't an issue because it was assumed that user would only call begin() method once after startup, so I could rely on the default values). Downside is that I don't know if this actually resolves anything and even if it does, it will prolong execution time of the begin() methods.
I will update you once I know more.

First bit of progress: I managed to trace the issue to setBandwidth() method called inside begin() when switching to LoRa mode. When setBandwidth() isn't called, receiver can pick up FSK packets just fine. No idea why.

Does direct mod fsk take the setBandwidth() function? Might be because the bandwidth changing is not supported by the cake direct mode?

See the wiki
FSK modem does not support setBandwidth() that method is exclusive to LoRa, This is not a direct mode issue, it happens when configuration LoRa modem.

Weirdness intensifies: inside setBandwidth(), method setBandwidthRaw() is called. When I comment out everything from inside setBandwidthRaw(), it doesn't work. When I comment out call to setBandwidthRaw(), the issue magically disappears. The crazy part is this: when I take the code that was inside setBandwidthRaw() and just paste it into setBandwidth() instead of calling setBandwidthRaw(), it works.
I am seriously doubting my sanity right now. Is there something really obvious I'm missing? I will try debugging in Atmel Studio, maybe that will help.

Is it because you have defined setBandwidthRaw() as a protected member, but when calling it you are using the format SX1278::setBandWidthRaw() when it should be something like this::setBandWidthRaw() ?

I can just hotfix it for now by pasting the code from setBandwidthRaw() into setBandwidth() but that will only work for SX1278 and not the other modules. I don't like the idea of not fixing the root cause; it will probably rear its ugly head again somewhere further down the line ...

yep for sure, what happens when you change setBandwidthRaw() function to public?

It doesn't resolve the issue and as a bonus, end user gains the ability to set invalid bandwidth values ;)
I'll commit the hotfix but won't create a new release yet. I'd rather only release once it's been fully resolved.

very strange! ok sounds good.

Hotfixed in 5de5e1ba15806e95c45698cf0e3d360838c76c7d
You know what would be stranger? If it had absolutely nothing to do with any sort of SPI write/module configuration.
I just commented out this line and it works ... still no idea why.

Another tiny bit of a progress: seems like it is caused by the fact that bandwidth is saved as a float variable. When it is replaced by some other type (e.g. uint32_t) it works. I'm probably going to replace all floating point member variables with integers, just to be safe.

wtf haha, nice debugging! any idea why the type would matter? perhaps it's because float is a definition and uint32_t is a real type? anyway you should submit that bug report to Semtech or someone, that's a pretty big bug to be honest.

It's not a bug in Semtech's chip, it's in my code. I still don't know why exactly did this happen, and until I do, there's really not much to report here. Besides, I don't think the chips were designed with quick modem switching in mind.

true, thanks for the bug fix, great to see your debugging method. 👍

I AM AN IDIOT.
Of course the FSK spectrum looks different after I've applied Errata fixes that were intended for LoRa modem only ... Type seemed to make a difference because after I've changed it, the errata fix no longer executed, hence no more changed spectrum.

Fixed in v5.2.2, thanks for reporting it.


Use requirejs to load esrileaflet and proj4leaflet  throws error

leaflet:  version  1.1.0
esrileaflet:  version 2.1.0
proj4:  version 2.4.3
proj4leaflet: version 1.0.1
I  have problem use  requirejs to load leaflet ,esrileaflet ,proj4 and proj4leaflet.
After loading reference model,the tiledMapLayer can show correct，but dynamicMapLayer can not display correct and throws a error  "proj4 is not defined" ,then i use  "window.proj4=proj4" to solve this problem.But the dynamicMapLayer  still can not show,and throws another problem when first image add to dynamicMapLayer.But when i use

i just pushed up a sample which shows how to load proj4/proj4leaflet in requirejs for you to take a look at in a new branch called proj4
if this doesn't help you resolve your error, please provide a reproducible test case of your own with the bare minimum code necessary to demonstrate the problem. i'd be happy to take a look.

Thank you for your answer sincerely.I think i found my problem,first time, i use requirejs to load leafleat.css,then browser throws a error

I debug the code, it occurs  when the first image requert successfully from server and were  added to dynamicMapLayer .
But If i load css file by  tag ,the dynamicMapLayer  shows correct.
require.config({
   baseUrl:"javascript/extra/api/",
   map: {
       "*": {
           "css": "require/css.min"
       }
     },
     paths:{
           "lealeft": "leaflet-src",
           "esri-leaflet":"esri-leaflet/dist/esri-leaflet",
           "proj4":"proj4/proj4",
           "proj4leaflet":"proj4/proj4leaflet"
     },
     shim : {
           "lealeft":["css!./leaflet"]
     }
});
define(['lealeft','esri-leaflet','proj4','proj4leaflet'],
    function(leaflet,esri,proj4,Proj){
       window.proj4=proj4;
       var crs = new Proj.CRS('EPSG:27700', '+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +towgs84=446.448,-125.157,542.06,0.15,0.247,0.842,-20.489 +units=m +no_defs', {
           origin: [-5220400.0, 4470200.0],
           resolutions: [
<PHONE_NUMBER>505292,
             66145.9656252646,
             26458.386250105836,
             19843.789687579378,
             13229.193125052918,
             6614.596562526459,
             2645.8386250105837,
             1322.9193125052918,
<PHONE_NUMBER>526459,
<PHONE_NUMBER>010584,
<PHONE_NUMBER>505292,
             66.1459656252646,
             26.458386250105836,
             19.843789687579378,
             13.229193125052918,
             6.614596562526459,
             2.6458386250105836,
             1.3229193125052918,
             0.6614596562526459
           ]

         });

         var map = L.map('map', {
           crs: crs
         }).setView([53.386, -2.319], 7);

         // The min/maxZoom values provided should match the actual cache thats been published. This information can be retrieved from the service endpoint directly.
         esri.tiledMapLayer({
           url: 'https://tiles.arcgis.com/tiles/qHLhLQrcvEnxjtPr/arcgis/rest/services/OS_Open_Background_2/MapServer',
           maxZoom: 19,
           minZoom: 6,
         }).addTo(map);

         // Dynamic map layers are projected by ArcGIS Server itself before the image is retrieved
         esri.dynamicMapLayer({
           url: "https://sampleserver6.arcgisonline.com/arcgis/rest/services/WorldTimeZones/MapServer",
           layers: [0],
           opacity: 0.4
         }).addTo(map);
});

glad to hear it!


Fresh release?

@rly beat us all with the https://github.com/djc/rnc2rng/pull/34 fix, and it would be great if released rnc2rng was carrying this fix.

2.6.5 is on PyPI now, I think that should take care of things!

2.6.5 is on PyPI now, I think that should take care of things!

THANK YOU!

THANK YOU!


Use maven wrapper

This allows people to build the project without having to install maven themselves.
This is a small change that can be with one command, see: https://github.com/takari/maven-wrapper

Added this to my local master branch. How do I push this to master branch?


adding events and making create admin collection not public

What is the status of this @psiemens? Is there anything else that needs to be done with this to get it merged?

@joshuahannan Thanks for checking in! I'm just working on updating the tests to use the new deployment transaction. I'll try to have it ready for review by tomorrow.
The custodians have been made aware of this change and know that it's non-breaking, so we can take a day to make sure it's extra solid.

@joshuahannan This PR should be good to go now. I updated the deployment of LockedTokens to use a separate account as the admin, and also added a line to link the capability for the TokenAdminCollection resource.


Crypto Broker Voyager Issues Notice of Default to Three Arrows

Date
2022/06/27
Description
Crypto broker Voyager Digital Ltd. said it has issued a notice of default to Three Arrows Capital, the troubled crypto hedge fund, on a loan worth roughly $675 million based on Bitcoin’s price on Monday.
Links
https://www.bloomberg.com/news/articles/2022-06-27/crypto-broker-voyager-issues-notice-of-default-to-three-arrows?srnd=premium
Agreement

[X] I irrevocably agree to release my contribution under the CC-BY 3.0 License

Attribution name
RockyDog
Personal link
No response

Already covered in https://web3isgoinggreat.com/?id=voyager-digital-reduces-withdrawal-limit


Switch to new Chrome headless mode

Following https://github.com/alphagov/govuk-frontend/pull/3542 just a quick PR for:

https://github.com/alphagov/govuk-frontend/issues/3541

For more info on our headless mode deprecation issues see:

https://github.com/puppeteer/puppeteer/issues/10070

Since Chrome's "new headless" is slower I've split the Accessibility tests into 4 groups of 8 components each

Timing stats for { headless: true } (before) versus { headless: 'new' } (after)
Before
Accessibility tests: 1m 57s
Behaviour tests: 41s
Component tests: 59s
Percy screenshots: 54s
Before
Accessibility tests: 2m 27s
Behaviour tests: 46s
Component tests: 1m 4s
Percy screenshots: 1m 2s

Do you know if the Puppeteer team are aware that tests running under the new headless modes are slower? Is it something they're working to improve?
Sharding the accessibility tests is a neat idea (TIL that's a thing!) but I feel like our GitHub Actions are starting to get a little complex. Wondering if we should either wait until the performance of the new headless mode improves, or just accept that our tests will be a little bit slower for a while?

Do you know if the Puppeteer team are aware that tests running under the new headless modes are slower? Is it something they're working to improve?

From some of the comments they're aware of it (approx 30% slower), but appreciate it's now the same full browser for both headless and headful modes

Sharding the accessibility tests is a neat idea (TIL that's a thing!) but I feel like our GitHub Actions are starting to get a little complex. Wondering if we should either wait until the performance of the new headless mode improves, or just accept that our tests will be a little bit slower for a while?

Ha that's great. I'd always wanted to try --shard out and this seemed perfect for it. That said, I'm more than happy to drop the commit (especially as it changes required status checks again!) and even hold off "new headless" for a while
For context this PR was thinking our { headless: 1 } loophole to silence the deprecation warning would be closed:

https://github.com/puppeteer/puppeteer/issues/10070

But it's still holding strong

I've rebased with https://github.com/alphagov/govuk-frontend/pull/3491 and removed the jest --shard commit
No rush to approve this one but good to know it works when we need it

The second timing block in https://github.com/alphagov/govuk-frontend/pull/3543#issuecomment-1521486713 is the "After" one (both say "before")?

The second timing block in #3543 (comment) is the "After" one (both say "before")?

Ahh thanks, updated!

@36degrees I've gathered timing stats again after rebasing with Puppeteer v20:

https://github.com/alphagov/govuk-frontend/pull/3589

The switch to Chrome for Testing (not Chromium) has closed the gap 🙌

Although since the ~600 accessibility tests merged we've drifted a lot from the 2m 30s sweet spot we once had


Fullscreen doesn't work on Safari

The page becomes blank and we cannot do anything.

@wadjih-bencheikh18 do you have a mac to debug this?

Otherwise you can start webkit using npx playwright wk

do you have a mac to debug this?

no, i will use playwright wk


Failure handling

e.g. when failed to create a snapshot due to some IO errors, cube should just crash so we can minimize the impact.
https://github.com/matrixorigin/matrixcube/blob/7c9ded1709cdf0afd0e562f22d3d4cddc88eb365/raftstore/peer_job.go#L230
need to review all failure handling in raftstore.

Error handlings are being reviewed as a part of the current ongoing refactoring process.

some failure handling related refactoring has been completed in v0.2, more will come in v0.3.
move the milestone to v0.3.


ONNX converted model, Windows UI support

We have converted the InstaFlow-0.9B model to ONNX format and have added initial support for InstaFlow to OnnxStack
ONNX Model
https://huggingface.co/TheyCallMeHex/InstaFlow-0.9B-ONNX/tree/main
OnnxStack UI download (windows only)
https://github.com/saddam213/OnnxStack/releases/tag/v0.9.1-pre
We are currently working on other models now.
InstaFlow-0.9B | 1 Step

Thank you so much! I will check and add that to the README of the repo!

We now support the Rectified low step models
https://github.com/saddam213/OnnxStack/releases/tag/v0.10.0


Update README.md

Hi @markymouse, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. Real humans will now evaluate your PR.

TTYL, MSBOT;

Thanks MSBOT.Been waiting for the "Real Humans" to come up, and make a "World of difference" not a sum, dividend but "Difference" is all that matters. @msftclas


Build Prototype for Gathering Metrics from Existing Repo

Description:
This is a follow up story to #29, depending on the results of that story there may be more or less work to do, but the gist of it is take what was found in that spike and write some code that can gather metrics from the existing files in the repo to use for metrics thresholds
Update Information: The first story is unable to have a POC that is functional. The reason for this is that the libraries appear to only work when being used inside of a plugin. There is a small amount of code that was written but unfortunately, the Project object couldn't even be gotten due to it being in a static main method.
The documentation from the previous story is found here.
The general plan for this story is to make a prototype that writes the gathered metrics to a file just to check that everything is working. After that, make an arraylist of the FeaturesVectors for all the methods in the repository
Related Epic:
Backend
Number Of Story Points: [number]
Acceptance Criteria:

[x] Ready when #29 is done and this story is further elaborated
[x] Done when metrics gathering class can work with the other complexity, size, and keywords classes.
[x] Done when metrics gathering class is completed
[x] Done when the code is commented
[x] Done when a PR is made and merged in.
[ ] Done when the code is tested

Comments

We've punted the testing into another story due to the testing being more intensive then we thought it would be, ie it's hard to test an IntellIiJ Plugin


Ported Probabilistic PCA Tutorial from Edward

I have tried to port Edward's tutorial on Porbabilistic PCA as closely as possible to Turing. While running HMC I found that it was extremely slow with 5000 data points and hence I've run on only 50 data points
Please give feedback in order to improve this or if it looks okay I can add the markdown files for the website as well. Thanks

I think this model is a better fit for the model zoo at TuringExamples. Could you slim this down into a .jl file with just the data preparation, the model, and the sampling step without any I/O stuff like plotting or printing? After, can you submit that as a PR at the TuringExamples repo.

Can we close this PR?

Yeah, I think so.


nit: right associative example

See: https://prometheus.io/docs/querying/operators/#binary-operator-precedence
2 * 3 ^ 2 being equivalent to 2 * (3 ^ 2) doesn't actually demonstrate that ^ is right associative. It just follows from operator precedence. The significance of right associativity is better exemplified by something of same precedence level as ^.

Thanks!