Refactor KEB to not support 1.x Kyma related steps

Description
Refactor KEB to not support 1.x Kyma related steps.
Reasons
Those steps are obsolete and not used in the KEB provisioning flows.
AC

remove obsolete 1.x steps and if needed refactor existing ones to make provisioning, deprovisioning, update and upgrade flows clean
reduce complexity of KEB flows and code, I expected steps code will be reduced by half

Additional components collection step
To enable the setting of additional components per Kyma version and plan we need a new approach to collect them for SKR. One solution might be declarative creation of ConfigMaps and kustomize. We already have similar working functionality for collecting runtime components overrides.
The idea
Add additional-components.yaml to base folder where overrides for specific runtime version are stored (e.g. 2.1 base).
Additional-components.yaml should contain ConfigMaps declarations with additional components set for plans.
Example:
apiVersion: v1
kind: ConfigMap
metadata:
  name: additional-component-service-manager-proxy
  namespace: kcp-system
  labels:
    component: service-manager-proxy
    add-cmp-plan-azure: "true"
    add-cmp-plan-aws: "true"
    add-cmp-plan-openstack: "true"
    add-cmp-plan-gcp: "true"
    add-cmp-plan-azure_ha: "true"
    add-cmp-plan-aws_ha: "true"
    add-cmp-plan-preview: "true"
data:
  component.name: "service-manager-proxy"
  component.namespace: "kyma-system"

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: additional-component-sc-migration
  namespace: kcp-system
  labels:
    component: sc-migration
    add-cmp-plan-azure: "true"
    add-cmp-plan-aws: "true"
    add-cmp-plan-gcp: "true"
data:
  component.name: "service-manager-proxy"
  component.namespace: "kyma-system"
  component.source: "https://github.com/kyma-incubator/sc-removal/releases/download/0.5.1/sap-btp-operator-migration-v0.5.1.tgz"

kustomization.yaml contains runtime versions for which prepared ConfigMaps should be relevant (kustomization.yaml in base folder for 2.1).
In KEB we can provide a new type implementing this interface which would be responsible for collecting all runtime components. Additional components would be retrieved on the basis of runtime version and plan labels. Even better, we could seperate required components and additional components by providing two interfaces.

Remove Kyma 1 support: https://github.com/piotrmiskiewicz/control-plane/tree/remove-kyma1-from-keb

@szwedm - your idea is to have one config map per additional components, this is what I don't like.

@piotrmiskiewicz - thanks for feedback.
We can provide yaml for each additional component, this way we will keep consistency in the repo like it is with components overrides.

PR with new components provider:
https://github.com/kyma-project/control-plane/pull/1686
New components provider internally has two mini components providers, one for required components and the other for additional components. Required components are fetched only from the bucket associated with Kyma 2 (Kyma 1 is no longer supported). Contents from the yaml file with list of Kyma components are read and then unmarshalled to KymaComponent structs with the distinction of prerequisites. For additional components we have two scenarios, provider gets defaults from defined yaml file when there are no configmaps created for any additional component for given version and plan or overwrites the defaults when there is at least one configmap with additional component for given version and plan (similar to HTTP PUT). In biref, configmap with at least one additional component = no default list of additional components. For example the following configmap:
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    add-cmp-plan-azure: "true"
    add-cmp-version-2.2.0: "true"
    component: test
  name: additional-component-test-2.2
  namespace: kcp-system
data:
  component.name: test
  component.namespace: test-namespace

will cause the provider to include only this one test additional component in the additional components list for Kyma 2.2.0 and azure plan.

Fixed, https://github.com/kyma-project/control-plane/pull/1779/files


Step Log with HTML tags, when converted to PDF, whole HTML tag is visible

I used some Step Log and had used some HTML tags for formatting in them, when converted to PDF, the whole HTML tag is visible as it is. Can step logs be converted keeping the HTML formatting.

Can u add an example step with the HTML?


New API to set shader uniforms

Link to forum thread.
This pull request implements https://github.com/SFML/SFML/issues/538 and introduces a new API to set uniform variables in sf::Shader. The old setParameter() overloads are deprecated in favor of new setUniform() ones.
The sf::Glsl namespace is added to SFML. It contains type definitions for types that can be passed to Shader::setUniform(). Most types are typedefs to existing SFML types (such as sf::Vector2<T>), there are however a couple of new ones:
sf::Glsl::Mat3  // 3x3 matrix
sf::Glsl::Mat4  // 4x4 matrix
sf::Glsl::Vec4  // 4D float vector
sf::Glsl::Ivec4 // 4D int vector
sf::Glsl::Bvec4 // 4D bool vector

I provided implicit conversions from sf::Transform and sf::Color to these types where appropriate. Where necessary for ambiguity reasons, an explicit conversion must be used:
sf::Color color;
shader.setUniform("color", sf::Glsl::Vec4(color)); // pass as 4D vector
shader.setUniform("color", color); // same call, implicit conversion

sf::Transform transform;
shader.setUniform("matrix", sf::Glsl::Mat3(transform)); // pass as 3x3
shader.setUniform("matrix", sf::Glsl::Mat4(transform)); // pass as 4x4

In addition to basic GLSL types, arrays are now supported as well. They can be specified with Shader::setUniformArray() that takes a pointer to the first element and the length:
sf::Glsl::Vec2 positions[5] = ...;
shader.setUniformArray("positions", positions, 5);

Design rationale
After extensive thinking about the design (see 2 forum threads), I came to the conclusion that setUniform() overloads are more appropriate than named functions setUniformFloat(), for the following reasons:

The call is simpler and we don't force the user to type redundant information.
We allow polymorphism (a user can write a template to pass uniforms of any type, without knowing the function names).
The number of provided methods is smaller. We can work with conversions where possible. For example, we don't need to overload setUniformMat3() for sf::Glsl::Mat3, float* and sf::Transform separately, one method taking sf::Glsl::Mat3 is enough.
It is still possible to enforce type safety at compile time and at runtime. The former can be achieved with a wrapper function that explicitly requires the type to be specified.
There is no way around dedicated types anyway, otherwise we can't support GLSL arrays (or only in a very low-level way).

Try it out
A self-contained archive containing a C++ file, two GLSL shader files and an image can be downloaded temporarily from my homepage: www.bromeon.ch/sfml/feature-shader-uniforms.zip

:+1: for finalizing this task.
I have two comments so far:

What about replacing all those functors in Shader.cpp with a single RAII structure similar to TextureSaver, which would save/restore the program binding with its constructor/destructor? This way you can keep the code that sets uniforms in each setUniform function, and you can keep things a little simpler.

I would create a Glsl.inl file to keep Glsl.hpp clean -- especially since all this stuff is private code.

Like this? :smiley:

Of course, the moment I wrote that, I realized that I could as well have outsourced the entire class definitions, since a declaration is enough for typedef...
So it should be even cleaner now.

I haven't got time to check everything in details but I would have included the .inl at the end of the .hpp  (*) file from outside both sf and priv namespaces, and added those in the .inl file so that when reading this file we know without looking at any other file in which namespace things are declared.
(*) if I remember correctly it's file to define a typedef to an incomplete type if it's full definition is available when using it so in this case it should work I believe.

I haven't got time to check everything in details but I would have included the .inl at the end of the .hpp  (*) file from outside both sf and priv namespaces, and added those in the .inl file so that when reading this file we know without looking at any other file in which namespace things are declared.

I would have done it like this too (see Thor), but I looked at Rect.inl and adhered to that style... Laurent? ;)

(*) if I remember correctly it's file to define a typedef to an incomplete type if it's full definition is available when using it so in this case it should work I believe.

Yes, where "using" depends on how you're accessing the type -- it's a matter of whether the type is complete at that time or not. For example, you can declare pointers and references to incomplete types.

I would have done it like this too (see Thor), but I looked at Rect.inl and adhered to that style... Laurent? ;)

Good point. Also note that in Thread.inl, the priv namespace is added there.
But I'd say we should avoid this style, at least from now on. What @SFML/sfml think?

Also note that in Thread.inl, the priv namespace is added there.

In Thread.inl, the namespace priv surrounds local private stuff, not the class definition.

I would have done it like this too (see Thor), but I looked at Rect.inl and adhered to that style... Laurent?

Do you really need to ask if you should do like in all other files? ;)
Regarding the modifications: looks perfect now :+1:

Do you really need to ask if you should do like in all other files? ;)

No, I already did it like in all other files ;) the question is rather:

But I'd say we should avoid this style, at least from now on. What does @SFML/sfml think?

But it's a different question, so let's discuss it elsewhere and not pollute this pull request any further....

Regarding the modifications: looks perfect now :+1:

Okay, cool :)

I applied the suggestions and some other changes.

Within UniformBinder, I left out ensureGlContext(), because as Laurent pointed out, it has already been called at that point. I don't like "better safe than sorry" reasoning, we need to be clear about our logic. I emphasized this with a comment.
Renamed the internal functions to copyMatrix(). All overloads start with the source parameter, as that's more idiomatic C++ (STL algorithms).
Added a few lines of code to define all setUniform(), setUniformArray() and setParameter() overloads for OpenGL ES, instead of hundreds of lines of empty implementations.
Squashed the commits.

Something to keep in mind: the API currently provides an implicit conversion from sf::Color to sf::Glsl::Vec4. While being convenient, this disables future conversions from colors to other GLSL types, most notably sf::Glsl::Ivec4 (or a potential sf::Glsl::Uvec4 that may be supported one day). Even though multiple conversions can coexist, the call setUniform(string, color) will be ambiguous, breaking code that relies on the implicit conversion.
It would however be possible to add explicit conversions from sf::Color to other GLSL types. This however would not allow things like
sf::Glsl::Vec4 colors[3] =
{
    sf::Color::Yellow,
    sf::Color::Cyan,
    sf::Color::Red,
};

So the question really boils down to whether the syntax
shader.setUniform("color", sf::Glsl::Vec4(color));

is such a big issue, that we desperately need
shader.setUniform("color", color);

at the cost of future limitations as outlined above.

Within UniformBinder, I left out ensureGlContext(), because as Laurent pointed out, it has already been called at that point.

It still needs to be called (it was there before your modifications, it was for a good reason ;)), in case setUniform is called in a thread that has no active GL context. That's how all GL resources work: they call ensureGlContext() in every function that makes an OpenGL call, even if their constructor already did it.

Added a few lines of code to define all setUniform(), setUniformArray() and setParameter() overloads for OpenGL ES, instead of hundreds of lines of empty implementations.

Although it reduces the number of lines to type for you, it clutters the public header. I prefer hiding such details, even if it involves typing more useless code for us. It's not as if it was going to change often.

It still needs to be called

Okay, I misunderstood you. Thanks for the explanation! :grinning:

Although it reduces the number of lines to type for you, it clutters the public header. I prefer hiding such details, even if it involves typing more useless code for us.

It makes me sad how we have all kinds of powerful abstractions in C++, but resort to silly code duplication for such reasons :worried: are those 15 lines in the header a real problem? I'm not worried about typing (my IDE is powerful enough), but about code readability. Having a .cpp file of which the half are just empty dummy implementations is awful...

It makes me sad how we have all kinds of powerful abstractions in C++, but resort to silly boilerplate code

Don't forget that it is not true code, just a quick and dirty placeholder until we switch to OpenGL ES 2 and have a proper implementation of shaders for all platforms.

are those 15 lines in the header a real problem?

I like keeping public headers clean.

code readability

Nobody's supposed to "read" this code, it's empty.

Having a .cpp file of which the half are just empty dummy implementations is awful...

Would you feel better if we created a separate cpp file for the empty implementation, and leave the original Shader.cpp clean?

I like keeping public headers clean.

I don't see headers so much as the public interface, because they're in fact not. The documentation should be the primary source to learn about the API, headers are more tailored to developers/"power users" than average users. And as a developer, it may be more intuitive to see the same thing handled with less code, but it depends of course on the person.
Still, I totally understand your point, and generally I agree that it's a good idea to let .hpp files give a clean and concise overview of the class.

Would you feel better if we created a separate cpp file for the empty implementation, and leave the original Shader.cpp clean?

I thought about that too, but I don't think it's worth the special case for the configuration script. Handling it in code would require headers, and that's probably only more confusing. I'll just add the empty functions to the bottom of Shader.cpp :wink:

Sooner or later, the backend implementations using the underlying GL objects will probably have to be split anyway since supporting multiple specifications (GL vs GLES) and multiple versions of the same specification (2.1 vs e.g. 4.5 which supports DSA) within a single file will become a complete mess. This is also one of the reasons why I favoured keeping UniformBinder isolated in the .cpp file: when using DSA it becomes completely unnecessary.

@binary1248 Okay, good to know. I think we should keep such things in a separate future PR though, there's also some optimization potential.
I applied the changes as discussed above: reintroduced ensureGlContext() and all the empty definitions for OpenGL ES.

Something to keep in mind: the API currently provides an implicit conversion from sf::Color to sf::Glsl::Vec4. While being convenient, this disables future conversions from colors to other GLSL types, most notably sf::Glsl::Ivec4 (or a potential sf::Glsl::Uvec4 that may be supported one day). Even though multiple conversions can coexist, the call setUniform(string, color) will be ambiguous, breaking code that relies on the implicit conversion.
It would be possible to add explicit conversions from sf::Color to other GLSL types. This however would not allow things like
sf::Glsl::Ivec4 colors[3] =
{
    sf::Color::Yellow,
    sf::Color::Cyan,
    sf::Color::Red,
};

So the question really boils down to whether the syntax
shader.setUniform("color", sf::Glsl::Vec4(color));

is such a big issue, that we desperately need
shader.setUniform("color", color);

at the cost of future limitations as outlined above.
To demonstrate the difference, I added another commit that implements implicit conversions from sf::Color to both sf::Glsl::Vec4 and sf::Glsl::Ivec4. I don't think it's inherently bad if the user explicitly specifies whether he needs a conversion to float or integer 4D vectors. It would also be more consistent to sf::Transform and sf::Glsl::Mat3/4:
sf::Color color;
shader.setUniform("color", sf::Glsl::Vec4(color));
shader.setUniform("color", sf::Glsl::Ivec4(color));

sf::Transform transform;
shader.setUniform("transform", sf::Glsl::Mat3(transform));
shader.setUniform("transform", sf::Glsl::Mat4(transform));

What do you think? Or is it unrealistic that people use other GLSL types than vec4 for colors?

Or is it unrealistic that people use other GLSL types than vec4 for colors?

In an sfml-graphics only context (not sf::Context :wink:), it is probably unrealistic since users would not care about how colour data flows through the shader stages. They just accept the fact that colours are vec4s, even more so since SFML still supports legacy GL which provides gl_Color which is a vec4 itself. Just look at the example shaders. And to make matters worse, legacy GL also provides access to that horrible mat4 "color matrix" which I have never seen anybody use in a legitimate way.
If we are to stay somewhat future-proof, we should probably allow for other types as well. Already now, shaders can be used in a limited way as GL wrappers although I don't know whether people would actually bother using sf::Color in that case. In really advanced use cases, reducing the size of the colour attribute if you know your destination framebuffer only supports e.g. a 32-bit RGBA format (like SFML does with its FBO RenderTextures) might provide some sort of performance improvement.

Okay. So should I merge the two commits?
Other feedback?

So should I merge the two commits?

Sure.
I guess this looks fine now. Like I said, there is optimization potential, but I'll leave that for another time in the (hopefully not to distant) future. :wink:
:+1:

Now that the corresponding PR has been merged, I introduced the SFML_DEPRECATED macro, and used SFML_DOXYGEN (from https://github.com/SFML/SFML/pull/961) to hide implementation details.
I also improved the documentation in several parts, and provided a usage example.

Anything left? Final reviews?

Looks good to me. I can't spot anything that I would want to comment on right now, but from what I can tell, this should serve as a good starting point for future work anyway. :+1:

This PR has been added to my merge list, meaning it will be merged soon, unless someone raises any concerns.


Added pre-commit

Fixes #1
Target Release

0.0.1
Draft CHANGELOG entry

NEW FEATURES

Added pre-commit with:

go linting
general file linting
commit linting based on conventional commits

Right now, the commitlint hook is not running. I checked active issues in pre-commit and the commitlint-precommit-hook, but cannot find anything that could explain my issue.
gitlint could be a good alternative because it has pre-commit support, but it only supports Linux/Mac.


Thank you!

@rondale-sc just wanted to drop a quick thank you - ran into an issue recently and thanks to your mention on the podcast/ this addon I was able to re-register a controller we hacked together in some legacy code! Thanks for continuing to think about those of us still stuck on ember 1.13 :)

@toranb :heart:


Calculate fee estimates from bitcoin core rather than LND

Calculate fee estimates from bitcoin core (straight from the source!) rather than LND - which is probably doing something similar but maybe doing some processing of the numbers.

We'll have to extract raw channel open/close txs from LND and I don't even think that's possible. Regardless, it's too much work for too little impact now that we added custom fee support.


Upgrade Axum to 0.7

What type of enhancement is this?
Tech debt reduction, Other
What does the enhancement do?
Sooner or later we should upgrade it. I'd create this ticket to collect information, since this upgrade is a huge breaking changes from axum side - https://github.com/tokio-rs/axum/releases/tag/axum-v0.7.0
Implementation challenges
No response

Blocked by tonic uses http 0.2 and axum 0.6.

Other refs -

https://github.com/hyperium/tonic/pull/1670
https://github.com/etcdv3/etcd-client/pull/75

What is holding this?

@yurivict This is mainly blocked by https://github.com/hyperium/tonic/pull/1670.

tonic 0.12 has been released. We can move forward now.

substrait's prost version needs bump also.

greptime-proto's patch - https://github.com/GreptimeTeam/greptime-proto/pull/181

Blocked by

https://github.com/apache/arrow-rs/pull/6041
https://github.com/apache/datafusion/pull/11372

I'm afraid that we cannot upgrade until September's Arrow & DataFusion release and we catch up to it.
I don't know if the upgrade can be done separated, but at least cannot as a whole.
cc @waynexia @MichaelScofield

💯 I'm preparing the migration to arrow 53 and the latest datafusion

💯 I'm preparing the migration to arrow 53 and the latest datafusion

@waynexia Are you still working on this? I've encountered a grpc request timeout issue, and am trying to upgrade tonic and hyper to see there are any free lunch. Our tonic and hyper are too old to waste the efforts to tackle problems.


Use static imports

Currently the deploy plabook looks too complex:
- name: Hosted-Engine-Setup_Part_01
  hosts: localhost
  connection: local
  vars_files:
    - passwords.yml
  vars:
    he_install_packages: true
    he_pre_checks: true
    he_initial_clean: true
    he_bootstrap_local_vm: true
    ovirt_repositories_ovirt_release_rpm: "{{ ovirt_repo_release_rpm }}"
  roles:
    - role: oVirt.repositories
    - role: oVirt.hosted-engine-setup

- name: Hosted-Engine-Setup_Part_02
  hosts: engine
  vars_files:
    - passwords.yml
  vars:
    he_bootstrap_pre_install_local_engine_vm: true
  roles:
    - role: oVirt.hosted-engine-setup

- name: Hosted-Engine-Setup_Part_03
  hosts: engine
  vars_files:
    - passwords.yml
  vars:
    ovirt_engine_setup_hostname: "{{ he_fqdn.split('.')[0] }}"
    ovirt_engine_setup_organization: "{{ he_cloud_init_domain_name }}"
    ovirt_engine_setup_dwh_db_host: "{{ he_fqdn.split('.')[0] }}"
    ovirt_engine_setup_firewall_manager: null
    ovirt_engine_setup_answer_file_path: /root/ovirt-engine-answers
    ovirt_engine_setup_use_remote_answer_file: True
    ovirt_engine_setup_accept_defaults: True
    ovirt_engine_setup_update_all_packages: false
    ovirt_engine_setup_offline: true
    ovirt_engine_setup_admin_password: "{{ he_admin_password }}"
  roles:
    - role: oVirt.engine-setup

- name: Hosted-Engine-Setup_Part_04
  hosts: engine
  vars_files:
    - passwords.yml
  vars:
    he_bootstrap_post_install_local_engine_vm: true
  roles:
    - role: oVirt.hosted-engine-setup

- name: Hosted-Engine-Setup_Part_05
  hosts: localhost
  connection: local
  vars_files:
    - passwords.yml
  vars:
    he_bootstrap_local_vm_add_host: true
    he_create_storage_domain: true
    he_create_target_vm: true
  roles:
    - role: oVirt.hosted-engine-setup

- name: Hosted-Engine-Setup_Part_06
  hosts: engine
  vars_files:
    - passwords.yml
  vars:
    he_engine_vm_configuration: true
  roles:
    - role: oVirt.hosted-engine-setup

- name: Hosted-Engine-Setup_Part_07
  hosts: localhost
  connection: local
  vars_files:
    - passwords.yml
  vars:
    he_final_tasks: true
    he_final_clean: true
  roles:
    - role: oVirt.hosted-engine-setup

AFAIR this was because of delegate_to don't work with include_tasks, but it works with import_tasks, can we use that instead? It would be more user friendly, and maybe in future when deleage_to will work with include_tasks, we don' thave to change the interface how user will execute the playbook.

Could you please take a look?
@sandrobonazzola @irosenzw

I tried it, it didn't worked.
The BIG issue is that we are activating the role on 2 hosts in a serial fashion:
do some tasks on the Host -> then do some tasks on the Engine -> again on the Host and so on...
There is no option in Ansible (yet) to change the host inside a play,
you have to start another play to do it.
I searched for a solution for quite some time and I didn't find anything.

You can see here how I've solved it:
https://github.com/oVirt/ovirt-ansible-image-template/blob/master/tasks/qcow2_image.yml#L135

There is also another relevant caveat here:
we also have to consume this from the CLI and the cockpit interactive installers (to avoid any code duplication).
Interactive means that we are not just asking all the user input values in the beginning without any validation but we are going to run the flow though different steps validating next step values in a loop.
We are going to run, for instance, the first part of the role to bring up the local VM with an engine there.
Then we are coming back to the user asking for storage details in a loop using the engine to try them until we successfully have a valid storage domain and only then to the next step and so on.
This means that we definitively need the capability of running just a specific part of the playbook.
We can make this a bit simpler with import_tasks+delegate_to to mix host and engiene VM tasks inside a single execution step but we still need stepped execution.

Why you can't specify all parameters before role execution and then just execute it? This is standard way in Ansible, I haven't heard so far of any role, which would want use input in the middle of execution. So do we really need those validation in the middle? If so, can't we make the role either idempotent or force cleanup of all finished tasks, so the role can be executed again?


Return JSON from decode_vin_batch to fix parsing exception

Example of what resp.content looks like before the changes:
b'<Response xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><Count>1</Count><Message>Results returned successfully. NOTE: Any missing decoded values should be interpreted as NHTSA does not have data on the specific variable. Missing value should NOT be interpreted as an indication that a feature or technology is unavailable for a vehicle.</Message><SearchCriteria /><Results><DecodedVINValues><MakeID>515</MakeID><ModelID>2475...
https://vpic.nhtsa.dot.gov/api/Home/Index/LanguageExamples shows format needs to be in the body/data content form, not params.
Exception:
Traceback (most recent call last):
  File "/home/batman/.pyenv/versions/3.11.4/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3548, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-cfb612afbaaf>", line 3, in <module>
    vpic.decode_vin_batch(['58ADA1C1XRU044431'])
  File "/home/batman/.pyenv/versions/3.11.4/lib/python3.11/site-packages/vpic/typed_client.py", line 211, in decode_vin_batch
    vehicles = self._client.decode_vin_batch(vins)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/batman/.pyenv/versions/3.11.4/lib/python3.11/site-packages/vpic/client.py", line 153, in decode_vin_batch
    return self._request_post("DecodeVINValuesBatch", data={"DATA": ";".join(vins)})
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/batman/.pyenv/versions/3.11.4/lib/python3.11/site-packages/vpic/client_base.py", line 73, in _request_post
    results = resp.json()["Results"]
              ^^^^^^^^^^^
  File "/home/batman/.pyenv/versions/3.11.4/lib/python3.11/site-packages/requests/models.py", line 968, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

@davidpeckham can we get this merged? It fixes batched VIN queries!

@sshane Thank you for the fix.

thanks! a new pypi release would be great

@sshane I hope to have time later this week to fix the build so I can release your changes.
For your projects, do you need basic vehicle information like manufacturer, make, model, model year, series, and trim. Or do you need to know more about the vehicle (engine, restraints, ...)?

We mostly need the make, model, model year, electrification level, and body style (hatch or sedan). vpic can be quite slow, so your VIN library is enticing!
I've tried it on a few VINs and do get DecodingError that vpic decodes properly.

I added body style and electrification level in release v0.4.0 this evening. If you send a sample of VINs that failed to decode, I'll take a look.

That was quick, awesome work! I opened a few issues on the repo

@davidpeckham any way you can publish a new version with this change to PyPI?


docs: Fix namespaceRegexes in full-cluster-secret-store.yaml

Problem Statement
This fixes a typo on https://external-secrets.io/v0.9.20/api/clustersecretstore/, in which the property is incorrectly called namespacesRegex
Checklist

[x] I have read the contribution guidelines
[x] All commits are signed with git commit --signoff
[x] My changes have reasonable test coverage
[ ] All tests pass with make test
[ ] I ensured my PR is ready for review with make reviewable

hi @excalq ! Thanks for your contribution! 🥳


Different name for same userid

https://beta.hydroshare.org/resource/68313c40c7ff48cd9e065ceada3c8426/
The author and owner are the same user.  Why the different names?
004.zip

This is by design. When a resource is created, a snapshot of the user's profile information is created as the Author information. After initial creation, the author information should only be changed if an editor of the resource decides to do so. If Tony changed his profile information (i.e., his first name) after this resource was created, that change wouldn't be reflected in the Author information unless Tony specifically updates that for the resource. This is to enable the use case that a user publishes a resource with one set of Author information, but then changes affiliation later on - or to support Authors that are not HydroShare users. This is consistent with how academic journals handle author information.
Owners, on the other hand, must be HydroShare users and so owners are tied directly to the user's HydroShare profile.


Add acceptance test for watch query changed via mutation

I wrote a test to better understand how watchQuery should work if a mutation is performed. Maybe it's valuable to be added to the test suite? I could refactor it into a route unit test, too. Cheers ✌🏽

Great suggestions, also Anakin → Darth 😁
I've made the requested changes, let me know if they line up with what you're after.
Cheers!

Thanks @chadian :v:


open_file return empty

Hi, thank you for your amazing code. I try to implement a simply file dialog to selected multi designed suffix( under vs2013 and qt), but i found that the file vector(f.result ()) returns empty. So i wander whether where might be some incompatible with qt?

I am not aware of any incompatibilities with Qt. Can you maybe show some code?

std::string title = "File Selection";
std::string defaultpath = std::string (InputFileFolderPath.toLocal8Bit ());
auto f = portableDialog::open_file (title, defaultpath,
	       { "(.txt .Post *.PosT .txtEst)", "*.txt *.Post *.PosT *.txtEst","All files", "*.*" },
		portableDialog::opt::multiselect);

It's simple just like the examples shown in your code, and the f is return empty. I don't know why this happens.
I just try to replace the original QT5's file selection with portable-file-dialogs. Thanks

Thanks for the additional information. Does the system open a file dialog when you run that code? If so, can you tell whether it’s Zenity, or Kdialog, or something else?
I also suggest calling portableDialog::settings::verbose(true); and see whether anything interesting is reported in stdout.

Yeah, the file dialog is opened, And i tried the latest version of pfd , the open_file somehow works. Another question: how can the select_folder to select multi folders while pressing the CTRL button? Thanks


Please update to latest orbit-db

and also publish the new version on orbit.chat so https://github.com/orbitdb/orbit-web/commit/02c90befd410d5eb7d5a26cef27bb6c789dc0cf5 is also included.
That would be really appreciated.

@latenssi or @haadcode ?

Orbit Chat is not a priority for me atm. and I won't be able to work on this.
Happy to accept a Pull Request that addresses this.

this is broken and not working, please update


Release v1.0.0

TODO

[ ] Fix middleware specs
[ ] Add web specs
[ ] Eliminate {TODO,FIXME}s as much as possible
[x] Add codecov to CI

Welcome to Codecov :tada:
Once merged to your default branch, Codecov will compare your coverage reports and display the results in this comment.
Thanks for integrating Codecov - We've got you covered :open_umbrella:


Interpret System.ComponentModel.DefaultValueAttribute

As suggested in PR: https://github.com/aspnet/EntityFrameworkCore/pull/16986/files
However, note that this isn't even a data annotation attribute--it's just something from ComponentModel--and we would need to be sure that the interpretation we use for EF Core is not in conflict with how other parts of the ecosystem (e.g. WinForms/WPF/UWP/Xamarin visual designers)  use it.

Another idea I have is to add an property to the ColumnAttribute


💡 [IMPROVEMENT] - Make it clear what OpenTofu is without referring to Terraform

Summary
I ran into a post announcing "OpenTofu is going GA" on Lemmy and clicked the link out of curiosity to be brought to the OpenTofu website. The only context I had was that OpenTofu is a FOSS application, but not what kind of application. With this lack of context and no familiarity with Terraform, it is completely unclear what OpenTofu does. The 2 places I could eventually find this information were the docs and the Terraform website.
It is great to advertise the values of the project, but don't forget that visitors might only know the name of your product out of context. For someone who has never heard of either Terraform or OpenTofu, the website is trying to "sell" me an alternative to a product I don't know, for reasons I fully agree with. This is very confusing.
Basic Example
The only mention of OpenTofu's functionality is "The open source infrastructure as code tool." tagline. The FAQ question: "What is OpenTofu?" is answered with: "A drop-in replacement for Terraform".
If you look at the Terraform website it provides a quick overview of what it does and some common use-cases. The latter is especially useful for someone who stumbles upon your website.
Drawbacks
I suppose it could make people who are intimately familiar with Terraform have the opposite experience: they know what OpenTofu does, but not why they should use it instead of Terraform
Unresolved questions
No response

Hey @MetalPinguinInc ! I think you are right, the messaging so far is geared towards people who already use Terraform and there is a wider audience for the website.
Personally, I'd like to see a few changes, but none of these have been agreed upon:

Restructure the menu to make it easier to find documentation as the Manifesto, I believe, has served its purpose.
Add something to help newcomers understand how the whole state management works as that seems to be the most confusing aspect of OpenTofu and Terraform. (i.e. ideally, you are not supposed to touch TF-provisioned infrastructure by hand at all).
As you point out, rebuild the front page and FAQ to bring people from outside the ecosystem in.
Make the website as accessible as possible (See #259 and #260).

That being said, these are longer term plans and there's a lot that can be done right now. If there's, for example, a piece of text you think should be changed, please open an issue because that's easier to make a quick decision on.
Thank you so much for your detailed issue and your help. Your input is very valuable.


Add coinbase index to atxs table

Description
Added index by coinbase to atxs table as the number of atxs grew significantly and we can expect that it will be frequently queried from API.

@dshulyak it's better than before. Checked on macOS M1
With index
INFO	grpc	finished unary call with code OK	{"grpc.start_time": "2024-02-19T14:15:19+01:00", "system": "grpc", "span.kind": "server", "grpc.service": "spacemesh.v2alpha1.ActivationService", "grpc.method": "List", "peer.address": "<IP_ADDRESS>:57557", "grpc.code": "OK", "grpc.time_ms": 6906.889}

INFO	grpc	finished unary call with code OK	{"grpc.start_time": "2024-02-19T14:18:37+01:00", "system": "grpc", "span.kind": "server", "grpc.service": "spacemesh.v2alpha1.ActivationService", "grpc.method": "List", "peer.address": "<IP_ADDRESS>:58591", "grpc.code": "OK", "grpc.time_ms": 7242.017}

Without index
INFO	grpc	finished unary call with code OK	{"grpc.start_time": "2024-02-19T14:16:13+01:00", "system": "grpc", "span.kind": "server", "grpc.service": "spacemesh.v2alpha1.ActivationService", "grpc.method": "List", "peer.address": "<IP_ADDRESS>:57823", "grpc.code": "OK", "grpc.time_ms": 10101.229}

INFO	grpc	finished unary call with code OK	{"grpc.start_time": "2024-02-19T14:17:12+01:00", "system": "grpc", "span.kind": "server", "grpc.service": "spacemesh.v2alpha1.ActivationService", "grpc.method": "List", "peer.address": "<IP_ADDRESS>:58145", "grpc.code": "OK", "grpc.time_ms": 9029.339}

bors merge

bors merge


SCons (*.sc) lexer

(Original issue 298 created by None on 2007-10-01T08:47:41+00:00)
We use SCons for some projects at VMware and a bug report was filed against Review Board to syntax highlight the SCons (*.sc) files. These are just Python files so they simply need to be associated with the Python lexer.
Reported by guest

(Original issue was assigned to gbrandl)
(Original issue 298 last updated on 2007-10-14T12:56:58+00:00)
(Issue automaticaly closed due to status in Bitbucket: resolved)

(Original comment by gbrandl on 2007-10-14T12:56:58+00:00)
Okay, done in r451.


fix the rename_table_across_databases.test to make it can run successfully multiple times.

What problem does this PR solve?
Issue Number: close #5145
Problem Summary:
What is changed and how it works?
fix the select statement with appropriate operator precedence.
Check List
Tests

[ ] Unit test
[ ] Integration test
[ ] Manual test (add detailed scripts or steps below)
[ ] No code

Side effects

[ ] Performance regression: Consumes more CPU
[ ] Performance regression: Consumes more Memory
[ ] Breaking backward compatibility

Documentation

[ ] Affects user behaviors
[ ] Contains syntax changes
[ ] Contains variable changes
[ ] Contains experimental features
[ ] Changes MySQL compatibility

Release note

None

/run-all-tests

/cc @breezewish

/merge

/merge


SLING-7000

Applied SLING-7000-v2.patch attached to the issue. Unfortunately we lost the rename of PathInfoTest.java to ModifiableRequestPathInfoTest.java.

@Buuhuu This has been applied in the mean time. Can you close the PR?


#13

Confirmed that annotating ReactElement and DOMElement as described fixes all GWT cast exceptions for me.

Thanks a lot.


Styling Profile

Styling profile
separate view profile, update address and update profile components
set up a weather API to show the current weather of the customer
set up current data and time in the profile
set up user profile pic by using pixel API (not quite works but it works)
set up Admin profile pic

Looks super cool Kenny, thank you!!!


custom auto-tagging system

I think it would be nice to have the possibility to auto add custom tags based on regex. For example based on my #64 and #60 issues I would be able to create a custom tag instead of writing an issue (except maybe gwennen, as that was a bug).
Quick mock:

Performance still a problem - will close the Issue for now, sorry!


Feature request: retain model ensembling as an option, for molecular replacement usage purposes

Hello,
Although it is true that in most cases the 5/25 models generated by AF2 monomer/multimer are very similar to each other, we consistently find that - when using these models (or pieces thereof) for molecular replacement - a model ensemble is clearly superior to any individual model (including the top ranked one).
An example of this is this work that appeared online just yesterday, were we performed molecular replacement using ensembles of models whose relative RMSD ranged from 0.7 to 1.9 Å.
Keeping this observation in mind, and regardless of whether DeepMind retires this functionality in the future or not, I think it would be very useful for crystallographers if this capability was reinstated into the OF code, as a non-default option (number_of_models = # or something like that).
Thank you!

We do support multiple-model inference.

No, sorry, I meant multiple-model inference (I also got confused as we often use the term ensembles for MR model sets).
But how does one then tell OF how many models to make, if they want more than 1?

Just provide a comma-separated list of model parameters. E.g.
--jax_param_path <path_1>,<path_2> --openfold_checkpoint_path <path_3>,<path_4>
Right now the script only supports one config_preset at a time, but I'm trying to figure out a sensible way to allow users to specify per-checkpoint config presets.

Sorry, I think I did not explain myself well. By models I meant output PDB models, i.e. the equivalent of "ranked_0.pdb... ranked_4.pdb" of AF2 (monomer).

AF2 generates the different PDB files using different sets of model parameters (of which it runs 5 by default). This functionality can be replicated by just specifying 5 different sets of model parameters as I did above. Maybe I'm still misunderstanding you though.

Thanks for the explanation, it's nice that one can also generate PDBs with AF2 and OF models at the same time (based on 'Exactly one of --openfold_checkpoint_path or --jax_param_path must be specified to run the inference script." I thought it was either one or the other). Is there any way to also rank the resulting PBDs by chain pLDDT (as AF2 does)?

That "Exactly" message is outdated. I'll change it now. We don't have pLDDT ranking built in ATM, but I can add it reasonably soon.

Great, thank you!


io.Pipe{Reader,Writer}.Close resets error code set with CloseWithError

% go version
go version go1.7 linux/amd64
% go env
GOARCH="amd64"
GOBIN="/home/cyphar/go/bin"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/cyphar/.local"
GORACE=""
GOROOT="/usr/lib64/go"
GOTOOLDIR="/usr/lib64/go/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build573484999=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"

If you use io.Pipe you'll notice that calling .Close on the writer will un-do the error code set by .CloseWithError(err). I'm unsure if this is a bug, but I found this to be very unexpected. In particular when it's usual for you to use a defer writer.Close() code style. This is also available on play.golang.org: https://play.golang.org/p/zJDmmRTOLQ.
package main

import (
	"fmt"
	"io"
	"io/ioutil"
)

func main() {
	reader, writer := io.Pipe()

	go func() {
		defer writer.Close()

		fmt.Fprintf(writer, "some data to write before the error")
		writer.CloseWithError(fmt.Errorf("an error code"))
		fmt.Fprintf(writer, "some write that will fail")
	}()

	val, err := ioutil.ReadAll(reader)
	fmt.Printf("val: %s\n", val)
	fmt.Printf("err: %s\n", err)
}

If you comment out defer writer.Close() you see what I would expect.
What did you expect to see?
% go run pipe.go
val: some data to write before the error
err: an error code

What did you see instead?
% go run pipe.go
val: some data to write before the error
err: %!s(<nil>)

In particular, I did not expect for the error returned to change to EOF.

My proposed patch looks like this:
commit f3bce6cef511a1c64fb668fd452b2a5c364e3dc9
Author: Aleksa Sarai<EMAIL_ADDRESS>Date:   Mon Nov 21 02:23:53 2016 +1100

    io: pipe: make CloseWithError final

    CloseWithError would previously overwrite previously set error flags,
    making a defer'd pipe.Close() set the error to EOF (clearing the
    previous error flags). This was quite confusing to users, because it
    meant that io.Pipes we're very different to channels (double-closing a
    pipe would not produce an error and would actually affect the pipe).

    Fixes: golang/go#17992
    Signed-off-by: Aleksa Sarai<EMAIL_ADDRESS>
diff --git a/src/io/pipe.go b/src/io/pipe.go
index 7e98cd2eb788..a462e857493c 100644
--- a/src/io/pipe.go
+++ b/src/io/pipe.go
@@ -93,23 +93,29 @@ func (p *pipe) write(b []byte) (n int, err error) {
        return
 }

-func (p *pipe) rclose(err error) {
+func (p *pipe) rclose(err error) error {
        if err == nil {
                err = ErrClosedPipe
        }
        p.l.Lock()
        defer p.l.Unlock()
+       if p.rerr != nil {
+               return ErrClosedPipe
+       }
        p.rerr = err
        p.rwait.Signal()
        p.wwait.Signal()
 }

-func (p *pipe) wclose(err error) {
+func (p *pipe) wclose(err error) error {
        if err == nil {
                err = EOF
        }
        p.l.Lock()
        defer p.l.Unlock()
+       if p.werr != nil {
+               return ErrClosedPipe
+       }
        p.werr = err
        p.rwait.Signal()
        p.wwait.Signal()
@@ -138,8 +144,7 @@ func (r *PipeReader) Close() error {
 // CloseWithError closes the reader; subsequent writes
 // to the write half of the pipe will return the error err.
 func (r *PipeReader) CloseWithError(err error) error {
-       r.p.rclose(err)
-       return nil
+       return r.p.rclose(err)
 }

 // A PipeWriter is the write half of a pipe.
@@ -168,8 +173,7 @@ func (w *PipeWriter) Close() error {
 //
 // CloseWithError always returns nil.
 func (w *PipeWriter) CloseWithError(err error) error {
-       w.p.wclose(err)
-       return nil
+       return w.p.wclose(err)
 }

 // Pipe creates a synchronous in-memory pipe.

Double-Close isn't really defined either way, but reading the existing docs in the io package, it looks like the existing behavior matches what the docs say, even if it wasn't what you expected.
Changing it seems unwise. It seems just as likely to surprise people in the other direction and break existing code in the wild, so I'm inclined to not change behavior.


Unable to download or change Tonie content

Problem/Motivation
After successful flashing and adding a CC2300 Box to Teddycloud (running as this HA Addon Container on HAOS on an RPi 4) i wanted to verify, that an original Tonie (that is playable, i think only because it was already downloaded and played pre-flashing) gets automatically downloaded and cached inside Teddycloud (which enables the possibility to play it inside the browser.
Expected behavior
Download starts automatically and after it finishes, I am able to play it in the browser.
Actual behavior
Download started but after a second, this error is shown:

Additionally, when trying to change the source of a model to a radio stream, the following error happens:

with the following addon logs produced:
WARN |platform_linux.c:0292:socketReceive()| buffer does not contain null terminator

WARN |platform_linux.c:0292:socketReceive()| buffer does not contain null terminator

WARN |platform_linux.c:0292:socketReceive()| buffer does not contain null terminator

DEBUG|server.c:0284:httpServerRequestCallback()| No certificate authentication

DEBUG|server.c:0287:httpServerRequestCallback()| Started server request to /api/getTagInfo, request 2, by unknown/web

DEBUG|server.c:0289:httpServerRequestCallback()|  >> client requested '/api/getTagInfo' via GET
DEBUG|server.c:0534:httpServerRequestCallback()| Stopped server request to /api/getTagInfo, request 2

2024/11/05 15:51:56 [error] 272#272: *124 connect() failed (111: Connection refused) while connecting to upstream, client: <IP_ADDRESS>, server: ff7725ba-teddycloud, request: "GET /api/getTagInfo?ruid=c62ca116500304e0?overlay%3D10082C71290C HTTP/1.1", upstream: "http://[::1]:80/api/getTagInfo?ruid=c62ca116500304e0?overlay%3D10082C71290C", host: "<IP_ADDRESS>:8123", referrer: "http://<IP_ADDRESS>:8123/api/hassio_ingress/exC2FbRGkw-5V6_UBCRYiiPJj7BD0gzlb4kD_2DU2ps/web/tonies"
WARN |platform_linux.c:0292:socketReceive()| buffer does not contain null terminator

WARN |platform_linux.c:0292:socketReceive()| buffer does not contain null terminator

WARN |platform_linux.c:0292:socketReceive()| buffer does not contain null terminator

WARN |platform_linux.c:0292:socketReceive()| buffer does not contain null terminator

Logs when starting the Toniebox:
INFO |server.c:0959:server_init()| 1 open HTTPS API connections

DEBUG|server.c:0276:httpServerRequestCallback()| Certificate authentication:

DEBUG|server.c:0277:httpServerRequestCallback()|   Issuer:     'Boxine Factory SubCA 02'

DEBUG|server.c:0278:httpServerRequestCallback()|   Subject:    'b'10082C71290C''

DEBUG|server.c:0279:httpServerRequestCallback()|   Serial:     '1A48A7'

DEBUG|server.c:0287:httpServerRequestCallback()| Started server request to /v1/time, request 2, by b'10082C71290C'

DEBUG|server.c:0289:httpServerRequestCallback()|  >> client requested '/v1/time' via GET
INFO |server.c:0430:httpServerRequestCallback()| UA=TB/1620325289 SP/34471936 HW/1048921, FW=1620325289, SP=34471936, HW=1048921

INFO |handler_cloud.c:0041:handleCloudTime()|  >> respond with current time

INFO |mqtt.c:0699:mqtt_init_box()| Registered new box 'Toniebox_Lukas' (cn: '10082C71290C')

INFO |mqtt.c:0700:mqtt_init_box()| Using base path 'teddyCloud/box/10082C71290C' and id 'teddyCloud_Box_10082C71290C'

DEBUG|home_assistant.c:0073:ha_publish()| [HA] Publish
DEBUG|home_assistant.c:0073:ha_publish()| [HA] Publish
DEBUG|server.c:0534:httpServerRequestCallback()| Stopped server request to /v1/time, request

Logs when doing a freshness check:
INFO |server.c:0959:server_init()| 2 open HTTPS API connections

DEBUG|server.c:0276:httpServerRequestCallback()| Certificate authentication:

DEBUG|server.c:0277:httpServerRequestCallback()|   Issuer:     'Boxine Factory SubCA 02'

DEBUG|server.c:0278:httpServerRequestCallback()|   Subject:    'b'10082C71290C''

DEBUG|server.c:0279:httpServerRequestCallback()|   Serial:     '1A48A7'

DEBUG|server.c:0287:httpServerRequestCallback()| Started server request to /v1/freshness-check, request 2, by b'10082C71290C'

DEBUG|server.c:0289:httpServerRequestCallback()|  >> client requested '/v1/freshness-check' via POST
INFO |server.c:0430:httpServerRequestCallback()| UA=TB/1620325289 SP/34471936 HW/1048921, FW=1620325289, SP=34471936, HW=1048921

INFO |handler_cloud.c:0817:handleCloudFreshnessCheck()| Content (944 of 944)
INFO |handler_cloud.c:0825:handleCloudFreshnessCheck()| Found 59 tonies:
INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E0040350141673B9, nocloud: 0, live: 0, updated: 0, audioid: 59B80855 (2017-09-12 18:16:21)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E00403501429D870, nocloud: 0, live: 0, updated: 0, audioid: 646B633C (2023-05-22 14:42:36)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E0040350154901E6, nocloud: 0, live: 0, updated: 0, audioid: 59CB9CCA (2017-09-27 14:42:50)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E0040350134730AE, nocloud: 0, live: 0, updated: 0, audioid: 00000001 (special), audioid-server: 00000001 (special)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E00403501541DA58, nocloud: 0, live: 0, updated: 0, audioid: 641AC980 (2023-03-22 10:25:20)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E004035016A12CC6, nocloud: 0, live: 0, updated: 0, audioid: 5BD9B2B8 (2018-10-31 14:48:40)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E004035015A38D35, nocloud: 0, live: 0, updated: 0, audioid: 61113F95 (2021-08-09 16:45:41)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E0040350133C430C, nocloud: 0, live: 0, updated: 0, audioid: 5E5D1A11 (2020-03-02 15:37:05)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E0040350175FF0BA, nocloud: 0, live: 0, updated: 0, audioid: 6253C838 (2022-04-11 08:18:32)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E004035014DA3D5D, nocloud: 0, live: 0, updated: 0, audioid: 646B6355 (2023-05-22 14:43:01)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E004035017A66765, nocloud: 0, live: 0, updated: 0, audioid: 6253C3E3 (2022-04-11 08:00:03)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E00403501575D2F6, nocloud: 0, live: 0, updated: 0, audioid: 64CD3D47 (2023-08-04 20:02:47)

DEBUG|contentJson.c:0254:content_json_update_model()| Audio-id 64CD3D47 unknown but previous content known by model .

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E004035015A58B17, nocloud: 0, live: 0, updated: 0, audioid: 60339FDD (2021-02-22 13:13:17)

...
...
...
DEBUG|contentJson.c:0254:content_json_update_model()| Audio-id 57EE897A unknown but previous content known by model .

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid:<PHONE_NUMBER>000000, nocloud: 0, live: 0, updated: 0, audioid: 57EE897A (2016-09-30 17:49:14)

DEBUG|contentJson.c:0254:content_json_update_model()| Audio-id 57EE897A unknown but previous content known by model .

INFO |cloud_request.c:0200:web_request()| Connecting to HTTP server prod.de.tbs.toys:443...

INFO |cloud_request.c:0252:web_request()|   trying IP: <IP_ADDRESS>
DEBUG|cloud_request.c:0033:httpClientTlsInitCallbackBase()| Initializing TLS...

DEBUG|cloud_request.c:0070:httpClientTlsInitCallbackBase()| Initializing TLS done

Steps to reproduce

Flash CC2300 Box as described inside teddycloud step-by-step
add Box to teddycloud
The "Boxine" cloud connection Icon on the top right shows green.
put an original Tonie on the box (was previously downloaded already)
Click on the Download button in the middle

Here is my config:
configVersion=13
log.level=5
log.color=true
log.logFullAuth=true
core.server.http_port=80
core.server.https_web_port=8443
core.server.https_api_port=443
core.server.bind_ip=
core.host_url=http://localhost
core.certdir=certs/client
core.contentdir=default
core.librarydir=library
core.datadir=data
core.firmwaredir=firmware
core.cachedir=cache
core.sslkeylogfile=
core.new_webgui_as_default=true
core.server_cert.file.ca=certs/server/ca-root.pem
core.server_cert.file.ca_der=certs/server/ca.der
core.server_cert.file.ca_key=certs/server/ca-key.pem
core.server_cert.file.crt=certs/server/teddy-cert.pem
core.server_cert.file.key=certs/server/teddy-key.pem
core.server_cert.data.ca=
core.client_cert.file.ca=certs/client/ca.der
core.client_cert.file.crt=certs/client/client.der
core.client_cert.file.key=certs/client/private.der
core.allowOrigin=
core.boxCertAuth=true
core.allowNewBox=false
core.flex_enabled=true
core.flex_uid=
core.settings_level=3
core.tonies_json_auto_update=true
security_mit.warnAccess=true
security_mit.lockAccess=true
security_mit.httpsOnly=true
security_mit.onBlacklistDomain=true
security_mit.onCrawler=true
security_mit.onRobotsTxt=true
cloud.enabled=true
cloud.remote_hostname=prod.de.tbs.toys
cloud.remote_port=443
cloud.enableV1Claim=true
cloud.enableV1CloudReset=false
cloud.enableV1FreshnessCheck=true
cloud.enableV1Log=false
cloud.enableV1Time=false
cloud.enableV1Ota=false
cloud.enableV2Content=true
cloud.cacheOta=true
cloud.localOta=false
cloud.cacheContent=true
cloud.cacheToLibrary=true
cloud.markCustomTagByPass=true
cloud.prioCustomContent=true
cloud.updateOnLowerAudioId=true
cloud.dumpRuidAuthContentJson=true
encode.bitrate=96
encode.ffmpeg_stream_buffer_ms=2000
encode.ffmpeg_stream_restart=false
encode.ffmpeg_sweep_startup_buffer=true
encode.ffmpeg_sweep_delay_ms=2000
encode.stream_max_size=251658239
frontend.split_model_content=true
frontend.ignore_web_version_mismatch=false
frontend.confirm_audioplayer_close=true
toniebox.api_access=true
toniebox.overrideCloud=true
toniebox.max_vol_spk=3
toniebox.max_vol_hdp=3
toniebox.slap_enabled=true
toniebox.slap_back_left=false
toniebox.led=0
rtnl.logRaw=false
rtnl.logHuman=true
rtnl.logRawFile=config/rtnl.bin
rtnl.logHumanFile=config/rtnl.csv
pcap.enabled=false
pcap.filename=config/traffic.pcap
mqtt.enabled=false
mqtt.hostname=
mqtt.port=1883
mqtt.username=
mqtt.password=
mqtt.identification=
mqtt.topic=teddyCloud
mqtt.qosLevel=0
hass.name=teddyCloud - Server
hass.id=teddyCloud_Server
tonie_json.cache_images=true
tonie_json.cache_preload=false
debug.web.pcm_encode_console_url=false

The content of the Homeassistant addon dir:

The ngCfg.json on SD-Card:
{
    "general": {
        "activeImg": "ofw2",
        "_descWaitForPress": "Waits for an earpress on startup",
        "waitForPress": false,
        "_descWaitForBoot": "Waits for an earpress before firmware boot",
        "waitForBoot": false,
        "waitTimeoutInS": 60,
        "_descMinBatteryLevel": "Divide through 2785 to get voltage",
        "minBatteryLevel": 8869,
        "ofwFixFlash": "/sys/pre-img.bin",
        "_descSerialLog": "Logging only works with the debug build!",
        "serialLog": false,
        "_descLogLevel": "0:Trace - 5:Fatal",
        "logLevel": 0,
        "_descLogColor": "Use colors in log output",
        "logColor": false
    },
    "ofw1": {
        "checkHash": false,
        "hashFile": false,
        "watchdog": true,
        "bootFlashImg": true,
        "flashImg": "/sys/pre-img.bin"
    },
    "ofw2": {
        "checkHash": true,
        "hashFile": false,
        "watchdog": true,
        "ofwFix": true,
        "ofwSimBL": true,
        "patches": ["altCa.305", "altUrl.custom.305"]
    },
    "ofw3": {
        "checkHash": true,
        "hashFile": false,
        "watchdog": true,
        "ofwFix": true,
        "patches": ["altCa.305", "altUrl.tc.fritz.box"]
    },
    "cfw1": {
        "checkHash": false,
        "hashFile": false,
        "watchdog": true
    },
    "cfw2": {
        "checkHash": false,
        "hashFile": false,
        "watchdog": true
    },
    "cfw3": {
        "checkHash": false,
        "hashFile": false,
        "watchdog": true
    },
    "add1": {
        "checkHash": true,
        "hashFile": false,
        "watchdog": true,
        "ofwFix": true,
        "ofwSimBL": false,
        "patches": ["blockCheck.310", "noCerts.305", "noPass3.310", "noPrivacy.305", "uidCheck.307"]
    },
    "add2": {
        "checkHash": true,
        "hashFile": false,
        "watchdog": true,
        "ofwFix": true,
        "ofwSimBL": false,
        "patches": ["altCa.305", "altUrl.305"]
    },
    "add3": {
        "checkHash": true,
        "hashFile": false,
        "watchdog": true,
        "ofwFix": true,
        "ofwSimBL": false,
        "patches": ["altCa.305", "altUrl.305"]
    }
}

I double checked, that the ca.der inside teddycloud client dir is the one from the Toniebox backup.
Proposed changes
I have no current lead - however - the connection to the Boxine cloud server does not seem to work.

Hi mrueg
The download request looks like this:

Does the content/download/... resource needs no be rewritten to the $ingress_path/content/download... or something similar?
Maybe interesting: Inside the Teddycloud Code at the suspected content download resource https://github.com/toniebox-reverse-engineering/teddycloud/blob/9e05ef8fc4a7d66f1e38db6a444c4a0d49af991f/src/handler_api.c#L1749, there is an ToDo saying something about rewrite

i installed the edge addon - copied over the certs folder from the old addon dir to the new one, uninstalled the stable addon and started the edge one -
however, it does not seem to rewrite the URL for download - as it still is:
:/content/download/C62CA116/500304E0?overlay=10082C71290C
Setting the radio does seem to work, the POST to http://<IP_ADDRESS>:8123/api/hassio_ingress/ZqlUipbpq2X-KnZyoxcBlKBP8Yhen3U9pBfsJu-Qlho/content/json/set/c62ca116500304e0?overlay=10082C71290C responses 200. However, even after a freshness check, the actual Tonie content is being played.

Probably this is missing one or two changes in the javascript. Hopefully get some time next week.

I see
INFO |handler_cloud.c:0590:handleCloudContent()| Serve streaming content from https://dispatcher.rndfnk.com/rbb/fritz/live/mp3/mid
[20:47:04] INFO: Service teddycloud exited with code 256 (by signal 11)
[20:47:05] INFO: Starting TeddyCloud...

looks like there's some segfault?

I switched the base to ubuntu https://github.com/toniebox-reverse-engineering/teddycloud/issues/160 seems like it's working now :) Please give it another try.

Just investigated the filters: The http method is missing from my descriptions - but the download (and play) of content is a GET call. I suspect, this is why there is no rewrite currently:
https://github.com/mrueg/addon-teddycloud/blob/0f17acbe4602183bc1a55487a94facf3cd4f8e32/teddycloud/rootfs/etc/nginx/servers/ingress.conf#L36

Download still does not work.
I came across the code building the download URL: https://github.com/toniebox-reverse-engineering/teddycloud/blob/9e05ef8fc4a7d66f1e38db6a444c4a0d49af991f/src/handler_api.c#L2638 which is not considering any base - or configurable path.
That is how I found your issue on the teddycloud repo, which tackles the problem (https://github.com/toniebox-reverse-engineering/teddycloud/issues/245).
I think this is not an issue of this repo anymore - correct?
For now i would stick with dirty workaround of manually rewriting the download URL with DevTools.

yes, you're right I was fixing audioUrl, and it should have been downloadTriggerUrl. https://github.com/mrueg/addon-teddycloud/commit/af4889170e028e744ad8275f103ea7e31f95db3d this should add a filter for it as well.
If https://github.com/toniebox-reverse-engineering/teddycloud/issues/245 gets fixed, we won't need all the nginx rewrites anymore.

works now - thank you very much :)

@mrueg, can you release that last commit to stable?

@mmslakr v0.3.2 is released.


Defining a TooltipPanel or a TooltipLabel in a theme does not affect themed controls

Windows 10 64bit Godot 2.1-beta
When applying a theme with TooltipPanel or TooltipLabel defined to a Control with tooltip text - the tooltip has default styling if some of parent Controls have no theme (or is overriden).
Steps to reproduce:

Create a control of any type (eg. a panel)
Create a nested control of any type (eg. a button)
Create a theme and attach it to the button
Add Class Items TooltipPanel or TooltipLabel (any or both)
Modify properties so that they could be easily noticed
Add a tooltip text to the button
Run the scene, hover over the element, the tooltip is using default styling

Also, in the element preview area for theme editing there is no tooltip element. My suggestion - add a dedicated tooltip element or attach tooltip texts to every element previewed.
EDIT: I've encountered this issue with a more complex hierarchy, so perhaps my issue laid somewhere else. My tooltip refused to change unless I changed the theme for VBoxContainer, while my tooltipped buttons were contained deeper in panels (which were styled).

Managed to recreate this in a separate scene using a similar scene hierarchy. The Panel is themed with a proper TooltipLabel, which does not appear when hovering over the Button or the Panel itself.
tooltip.zip

Is this still valid in the current master branch?

Unfortunately the master branch currently does not compile. I'll check it as soon as possible (if I'll forget then just ping me)

The build issue was just fixed, so it should work now.

This is fixed, probably at the same time as #18478 (which was a duplicate?)

Closing, as it seems to be fixed in 3.1 according to the issue linked by @KoBeWi.


GET request on lighttpd are now working

GET request should not have content-length: 0 in headers, this cause a Bad
Request response in lighttpd.
This code sample is now working :
var request = require('request');
var util = require('util');

request({ uri: 'http://piratebay.se' }, function (error, response, body) {
  if (!error && response.statusCode == 200) {
    util.puts(body);
  } else {
    if (error)
      console.log("error :", error);
    console.log("statusCode :", response.statusCode);
  }
});

technically, according to the HTTP spec a GET request can have a body so we can't assume length=0 for all GET requests.

This is just to avoid GET methods with empty body and a header with content-length = 0. If a GET request has a body, it is already handled in the other part of the condition.
Also, when a GET request can have a body exactly ? I have read the spec, it is not explicitly forbidden, but, is there a case where that make sense ?
As far as I can see, lighttpd always return a Bad Request with a GET with a content-length

there is no good reason to ever have a body in a GET request, but it's allowed, and a few crazy people do it, probably WebDAV or some other insanity-over-HTTP.
this will set the content-type improperly if an object is streamed to a GET request. if (self.body) doesn't handle all conditions where a body could be in a request (most notably streaming), which is why you'll need to hook in before .end() and check if anything was written to properly handle this case.

closing until this handles streaming. will re-open once new code lands.


clustering: creates the package and implements DBSCAN

Verified against scikit-learn's implementation (gen_test.py)

Without some kind of dedicated data structure for spatial indexing? Such as k-d tree.

This patch does not include any support for spatial indexing, but once the implementation's correct it can be added and it will be useful for other clustering algorithms. Raised as #122


Perfromance degradation issue

I am getting less performance after conversion. Have you bench mark your code with before conversion and after conversion?

In order to test the accuracy of yolov4 on the embedded platform after model conversion under different frameworks. AlexeyAB provided test pictures, and I also uploaded my test results in the README.In some tensorflow1.x,tensorflow2.x and pytorch projects, after the model conversion, the accuracy dropped significantly. In my OpenVINO implementation, the accuracy is not much different from darknet, and even better than darknet in some scenarios.
Have you added --reverse_input_channels  when you run mo.py?
And have you set the same confidence threshold when you test your model？ In object_detection_demo_yolov3_async.py,you could set -t 0.2  , (default -t is 0.5)

Thanks for your reply. I have used --reverse_input_channels while converting yolov4-tiny model. yolov3-tiny performance is matches with before and after conversion performance. I have used https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_YOLO_From_Tensorflow.html for the conversion. Regarding yolov4-tiny I am getting reduced performance around 25-30%. I was wondering If you have compared results on full data.
Moreover, the test results in your README has different confidence score before and after conversion.


Access Token is empty

After successful OAuth2 verification, in the callback, I have used the Oauth2SessionManager instance to create the client.
get_access_tokens method does throw a
/opt/boxen/data/virturalenvs/rbox/lib/python2.7/site-packages/urllib3/util/ssl_.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings   InsecurePlatformWarning
The client object does not have an access token.
I use Python 2.7

I gave the base_url while creating the client. So, I was not getting the access token.


Config changes to reflect the default branch from master to main. Please merge to complete the default branch renaming job.

Auto branch renaming content.

Docs Build status updates of commit dfbc1cc:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.publish.config.json
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit f519173:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.github/workflows/VerifyAdvocateYAML.yml
:white_check_mark:Succeeded

.openpublishing.publish.config.json
:white_check_mark:Succeeded

README.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


ci: support for triggering long tests

At the moment we can't trigger the long tests on via our trybots.
We should add the ability to do so via cmd/cueckoo.

CUE_LONG=1 disappeared as of https://review.gerrithub.io/c/cue-lang/cue/+/1192938, so I think this is no longer needed. At least not until we need to re-introduce "long" or "slow" tests, and hopefully that doesn't happen anytime soon.


Update codegen to new schema

This was failing with:
Error [Graphcache Error]: Invalid Object type: The type `WalletWalletNFTsConnection` is not an object in the defined schema, but the GraphQL document is traversing it.
(Caused At: "EthMainnetWalletNFTsByAddress" query, "WalletByAddressFragment" Fragment)
https://bit.ly/2XbVrpR#3
    at invariant (/Users/johnmitsch/misc/qn_sdk_ts/node_modules/@urql/exchange-graphcache/src/helpers/help.ts:85:19)
    at expectObjectType (/Users/johnmitsch/misc/qn_sdk_ts/node_modules/@urql/exchange-graphcache/src/ast/schemaPredicates.ts:101:3)
    at getField (/Users/johnmitsch/misc/qn_sdk_ts/node_modules/@urql/exchange-graphcache/src/ast/schemaPredicates.ts:81:3)
    at isFieldAvailableOnType (/Users/johnmitsch/misc/qn_sdk_ts/node_modules/@urql/exchange-graphcache/src/ast/schemaPredicates.ts:47:5)
    at writeSelection (/Users/johnmitsch/misc/qn_sdk_ts/node_modules/@urql/exchange-graphcache/src/operations/write.ts:266:9)
    at writeField (/Users/johnmitsch/misc/qn_sdk_ts/node_modules/@urql/exchange-graphcache/src/operations/write.ts:408:3)
    at writeSelection (/Users/johnmitsch/misc/qn_sdk_ts/node_modules/@urql/exchange-graphcache/src/operations/write.ts:305:22)
    at writeField (/Users/johnmitsch/misc/qn_sdk_ts/node_modules/@urql/exchange-graphcache/src/operations/write.ts:408:3)
    at writeSelection (/Users/johnmitsch/misc/qn_sdk_ts/node_modules/@urql/exchange-graphcache/src/operations/write.ts:305:22)
    at writeField (/Users/johnmitsch/misc/qn_sdk_ts/node_modules/@urql/exchange-graphcache/src/operations/write.ts:408:3)
    ```

coverage: 99.58%. remained the same when pulling 69a88e52c51b61e809558de38615532e45ab6675 on john/update_codegen_0923 into 45ecf683298dba4123c15ff13ad5b07d0de5cdf7 on main.


[BUG]org.opensearch.integtest.config.ChimeNotificationConfigCrudIT failed when running integration tests with OpenSearch rpm/deb package

What is the bug?
There's an integration test failure when running tests with OpenSearch rpm/deb package, it seems that we do not handle the warnings correctly when calling _refresh API:
Warnings: [this request accesses system indices: [.opendistro_security], but in a future major version, direct access to system indices will be prevented by default, this request accesses system indices: [.opensearch-notifications-config], but in a future major version, direct access to system indices will be prevented by default]

, we need to dive deep on this and fix it.
What is the expected behavior?
Integration tests runs successfully with rpm/deb package.
Do you have any screenshots?

integ-test-failure is reserved for the bot. Please use a different label to track integration failures.


Keep lists of integers as lists when inspecting queries

@josevalim I'm opening this as a PR as I'm not sure about this change.
Is there a better way than to pattern match on the result of Macro.to_string/2?

@josevalim updated

:heart: :green_heart: :blue_heart: :yellow_heart: :purple_heart:


Some naming issues in the Dashboard swagger

[ ] The GrafanaResource is not a good name for a ARM resource, consider to rename it to another one, like ManagedGrafana .
[ ] All body parameters in a put or patch operation should be defined as required .
[ ] The naming workspaceName is weird here as the description is 'The name of Azure Managed Grafana' .

All concerns are solved.


feat: use valueTypeHandle receive target field type of parsed enum class

Codecov Report

Merging #128 (6ef31b0) into master (bd7419b) will decrease coverage by 0.081%.
The diff coverage is 28.070%.

@@               Coverage Diff               @@
##              master      #128       +/-   ##
===============================================
- Coverage     30.932%   30.851%   -0.081%
  Complexity       846       846
===============================================
  Files            192       192
  Lines           8994      9037       +43
  Branches        2229      2241       +12
===============================================
+ Hits            2782      2788        +6
- Misses          5618      5652       +34
- Partials         594       597        +3

Flag
Coverage Δ

unittests
30.851% <28.070%> (-0.081%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...nt/intellij/jvm/standard/StandardJvmClassHelper.kt
73.047% <ø> (ø)

...s/src/main/kotlin/com/itangcent/common/utils/KV.kt
46.429% <8.824%> (-25.571%)
:arrow_down:

.../itangcent/intellij/config/resource/URLResource.kt
47.059% <33.333%> (-11.275%)
:arrow_down:

...m/itangcent/intellij/psi/AbstractPsiClassHelper.kt
48.474% <62.500%> (-0.082%)
:arrow_down:

...om/itangcent/intellij/psi/DefaultPsiClassHelper.kt
20.000% <62.500%> (-0.690%)
:arrow_down:

...otlin/com/itangcent/intellij/jvm/PsiClassHelper.kt
50.000% <100.000%> (+50.000%)
:arrow_up:

...kotlin/com/itangcent/intellij/psi/ClassRuleKeys.kt
100.000% <0.000%> (+3.125%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bd7419b...6ef31b0. Read the comment docs.


feat!: drop node.js < 16

This should be fine based on the end-of-life previous versions: https://github.com/nodejs/Release


Add locale to user details

Fixes #60

Coverage remained the same at 100.0% when pulling ce1d42c4a6b372a0f468b00eb359bdcb3d939a8d on feature/add-locale-field into ae2ac34bf5ba6fdaaa0df69cf009c2ae52b0ab3d on master.

Coverage remained the same at 100.0% when pulling ce1d42c4a6b372a0f468b00eb359bdcb3d939a8d on feature/add-locale-field into ae2ac34bf5ba6fdaaa0df69cf009c2ae52b0ab3d on master.

Coverage remained the same at 100.0% when pulling ce1d42c4a6b372a0f468b00eb359bdcb3d939a8d on feature/add-locale-field into ae2ac34bf5ba6fdaaa0df69cf009c2ae52b0ab3d on master.


feat: Add before function callback for custom format

Hello, this allows me to do something like this (just replaced cb with before though)

In order to apply custom format:

Nice! Could you also put option documentation in README?

Just did that, what do you think ?

thanks!

@danymat can you show full code please.
What is the types.lsp.insertTextFormat.Snippet?

https://github.com/danymat/dotfiles/blob/d91534d08e5a3f6085f8348ec3c41982e9b74941/nvim/.config/nvim/lua/configs/cmp.lua#L35-L60

I made a function similar to @danymat that works with Ultisnips, see here


Remove vars that aren't actually needed for dual stack

CALICO_IPV6POOL_NAT_OUTGOING setting not needed because it defaults to
"false".
CALICO_IPV6POOL_IPIP setting not needed because node startup.go does
not honour that variable at all; instead it hardcodes "Never".

release-v3.11 isn't real, according to @djosborne - I think this should go into release-legacy for now IIUC.


Tween Accept Array as Target

Hello everyone.
I was thinking that tween should be able to accept array of objects as target.
E.g. :
Currently
game.add.tween(obj1).to({x:'-100'});
game.add.tween(obj2).to({x:'-100'});
game.add.tween(obj3).to({x:'-100'});

If accept Array of objects
game.add.tween([obj1,obj2,obj3]).to({x:'-100'});

I know laser are in developing, but tween was not write there yet.
Should i create a PR with this feature?

I like the idea, but I don't want to implement it into the API, sorry. No more sweeping changes like this really.

np.


Add sort by privilege level to TO and use it in TP

Add support for sorting roles by priv_level
Change TP to request roles ordered by priv_level
Add support to TO Go to order by descending order
Change TP to request roles in descending order

What does this PR (Pull Request) do?

[x] This PR fixes #3226

Which Traffic Control components are affected by this PR?

Traffic Ops
Traffic Portal

What is the best way to verify this PR?

Build and start TO and TP, open the Roles section of TP, and ensure the roles are listed in descending order by privilege level

The following criteria are ALL met by this PR

[x] This PR includes tests OR I have explained why tests are unnecessary
[x] This PR includes documentation OR I have explained why documentation is unnecessary
[ ] This PR includes an update to CHANGELOG.md OR such an update is not necessary
[x] This PR includes any and all required license headers
[x] This PR ensures that database migration sequence is correct OR this PR does not include a database migration
[x] This PR DOES NOT FIX A SERIOUS SECURITY VULNERABILITY (see the Apache Software Foundation's security guidelines for details)

Can one of the admins verify this patch?

So now that this has been done, it'll make any endpoints that use dbhelpers support ordering, right? That'll need to be documented on the affected API endpoints.

I didn't see ordering anywhere in the docs, but would I just add that under parameters to each of the endpoint documentation pages?

yeah, exactly

this PR should probably have a changelog.md entry as it introduces a  new api feature.

Since documentation for this and #3701 are both covered in #3740 due to weird RST tables things, I removed documentation from this PR


⚠ Bofrost.lu has degraded performance

In e651aed, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1825 ms

Resolved: Bofrost.lu performance has improved in e26d277 after 22 minutes.


aggregate_temporal -> labels clarification

Currently the definition of the labels parameter in aggregate_temporal is:

Labels for the intervals. The number of labels and the number of groups need to be equal.

One of the most common cases IMO, is that the labels are again valid timestamps. This way, the time dimension is mapped unto a time dimension, and not something entirely new.
Not sure if this should be a hard constraint, but for me it's at least a recommended suggestion. Also an example can be provided.
For me, it's also an option to use the first timestamp of each interval as the default label. That way the labels parameter is no longer mandatory, and the function becomes easier to use.

We discussed that for 0.4 we stick with the current proposal, but @jdries may just remove the ability to specify string-only labels as GepPySpark can't handle them. Moving the discussion to 0.5 or 1.0.

Let's revisit this issue: We now have rename_labels. I'm wondering whether we should just remove the labels parameter from aggregate_temporal completely and just handle it as in reduce: We just enumerate the labels by default based on the intervals (i.e. [ [ "2015-01-01", "2016-01-01" ], [ "2016-01-01", "2017-01-01" ], [ "2017-01-01", "2018-01-01" ] ] will have the labels [0, 1, 2]). Afterwards, a user can call rename_labels when required. @jdries @mkadunc
Note to myself: I would need to allow the temporal string types in the labels parameter of rename_labels.


Update changelog 4.4

new since rc1
new:

Improved [Obsolete] warnings - include the Nlog version when it became obsolete
Added Condition to AutoFlushWrappper
FileTarget - Close stale file handles outside archive mutex lock #1513

done


Added requirement for redirect URIs to be the same hostname as client

Continuation of https://github.com/solid/webid-oidc-spec/pull/16 because of a mess up with forked repos

@zenomt and @dmitrizagidulin can you review please?

Also, @kjetilk @Mitzi-Laszlo @timbl @justinwb @RubenVerborgh: two of you need to approve this before it can be merged.

it's not appropriate to place any constraint on the form of the client_id (for example, that it "must be identical to the value of the origin header") beyond what is already specified by OIDC

I wouldn't know why it is “not appropriate”, given that we also place constraints on OIDC to authenticate people with a WebID.
But I do think that the burden of proof is on @jaxoncreed to argue why indeed it needs to be a WebID, and to document that in this PR.

further, the redirect_uri should be entirely up to the application.

Why though? It seems perfectly reasonable to constrain this to the same domain, because this is the domain I'm giving permission.

placing a constraint on the client_id is inappropriate because the form of the client_id is an implementation detail of the OP (except for the requirement that it's unique for every registration). OPs are free to make the client_id be whatever is convenient (for example, a key to a database table, or a structured blob encoding information about the client, or anything, as long as it's unique). this is discussed a lot more in #12.
regarding the redirect_uri: since that's where the OP sends the credentials (or the code), i've argued in #12 and #23 that the redirect_uri should be the application identifier.

the form of the client_id is an implementation detail of the OP

But nothing stops us from adding more constraints on it for Solid-compatibility?
It would be a problem if making it a URL would be incompatible with the OIDC spec. But it does not seem to be incompatible.

the redirect_uri should be the application identifier

Very likely not (talking from a SemWeb perspective now); the URI of a a redirect page identifies a redirect page, not an app.

But nothing stops us from adding more constraints on it for Solid-compatibility?

requiring it to be a URL (or requiring any particular form) would shut out lots of existing OIDC Provider implementations that would otherwise be able to be a WebID-OIDC Provider. currently the changes to make an OIDC Provider be a WebID-OIDC Provider are backward-compatible additions, like "put the WebID in a new webid claim, otherwise if the sub is a URI, it's the WebID", and for the proof-token thing, "have a way to set a cnf claim in the id_token".  the other things that make it be WebID-OIDC are outside the OP, like that the WebID says the OP is an authorized issuer.
the uniqueness of the client_id is an important security aspect of OIDC/OAuth. if the client_id were a URL, it would still need to be a unique URL somehow (fragment?) for each registration.
for the OP to assign a client_id taking the form of a URL that has the same origin as the redirect_uri, that would require all redirect_uris being registered to be in the same origin (OIDC allows multiple redirect_uris, and they don't currently need to share an origin).
requiring all redirect_uris for a client to share an origin, and for that origin to be the same as something (there is no Origin header when redirecting to/loading the OP's login page, and dynamic registration could be done by a back-end server that could omit or set any Origin) would eliminate an entire class of possible application architectures, where different components of the application are in different origins (maybe some parts are in AWS Lambda/Azure Functions/OpenWhisk/etc, maybe logging in is a static github pages page, etc).

requiring it to be a URL (or requiring any particular form) would shut out lots of existing OIDC Provider implementations that would otherwise be able to be a WebID-OIDC Provider.

+1

if the client_id were a URL, it would still need to be a unique URL somehow (fragment?) for each registration.

This is a major blocker indeed; makes the suggestion incompatible with OIDC.

So, what do you say, @jaxoncreed ? Do you agree with @zenomt and @RubenVerborgh 's assessment?

requiring it to be a URL (or requiring any particular form) would shut out lots of existing OIDC Provider implementations that would otherwise be able to be a WebID-OIDC Provider. currently the changes to make an OIDC Provider be a WebID-OIDC Provider are backward-compatible additions, like "put the WebID in a new webid claim, otherwise if the sub is a URI, it's the WebID", and for the proof-token thing, "have a way to set a cnf claim in the id_token". the other things that make it be WebID-OIDC are outside the OP, like that the WebID says the OP is an authorized issuer.

Anything that hinders compatibility with other OIDC provider implementations should be considered a non-starter. Really important that we foster interop.

Yes, I actually do agree with @zenomt at the moment. It's reflected in another pull request I have (#27) for the spec but not this one. I believe this one can be thrown out.

@jaxoncreed to be clear - this pull request can be closed without merging in favor of #27 ?

See also https://github.com/solid/webid-oidc-spec/issues/12#issuecomment-502723571

OK; I understand that this can be closed. If that was wrong, we can always reopen :-)

You're correct. It can be closed.


add helper functions

Hi, i'm trying to use lexbor in my crystal wrapper, this functions helps. It seems this parser 15% faster and less memory used, also binary size with static library smaller by 1Mb.
Question, is it support skip whitespace nodes?
Also, waiting css selectors feature.

Hi, @kostya
Thanks for the PR!
Maybe you should make the functions inline (lxb_dom_node_tag_id, lxb_dom_node_next and so on)?
Like a:
/* node.h */
lxb_inline lxb_dom_node_t *
lxb_dom_node_parent(lxb_dom_node_t *node)
{
    return node->parent;
}

and for ABI:
/* node.h */
LXB_API lxb_dom_node_t *
lxb_dom_node_parent_noi(lxb_dom_node_t *node);

/* node.c */
lxb_dom_node_t *
lxb_dom_node_parent_noi(lxb_dom_node_t *node)
{
    lxb_dom_node_parent(node);
}

What do you think about it?
P.S.: Throughout the project, 4 spaces are used as indentation.

@kostya
Thanks!


Handle math

This might turn out to be really difficult, but as a start maybe we can provide a workable fallback if we are typesetting math stuff.

I think it shouldn’t be hard to populate the math related fields of the font metrics using HarfBuzz API and then let LuaTeX do the actual typesetting since this is handled by the engine itself and it has rather decent support for OpenType math layout.


Large size difference between v5 and v6 in webpack output

RxJS version:
6.2.0
Code to reproduce:
Can't provide
Expected behavior:
Expect the output file size to be the same size as v5 or smaller.
Actual behavior:
Output rxjs is about 4x larger
Additional information:
I have a TypeScript project that I build with webpack, emitting ES5 ultimately, and performs ES6 tree shaking via Babel. It currently uses v5 of RXJS, utilizing AsyncSubject, Observable, ReplaySubject, BehaviorSubject and several operators like bufferTime, filter, and others.
When looking at the output from the webpack-bundle-analyzer, below, I see that the parsed size of my output is 27.13 KB. I updated my project to utilize rxjs6 without using the compatibility library and re-built it with the same build pipeline. This time I got an output size of 116.43 KB, which is more than 4x larger. This primarily looks like it's because of whatever is in this _esm5 directory, but this alone preempts my adoption of v6.
What can be done to shrink this directory (or better, eliminate it so I can achieve sizes as small as what I got with v5)?
Thanks!
rxjs5

rxjs6

This seemingly looks like related with configuration / how to import modules around, but we can't figure out without reproducible code (or config and other environment surroundings). Curious if you already applied all suggestions at https://github.com/ReactiveX/rxjs/blob/d231053616ab4e5d2541818314110df626e6bf5a/doc/pipeable-operators.md#build-and-treeshaking .

@kwonoj I didn't see that document originally when I was reading through the documentation. I'll work through the suggestions there and get back to you.

@kwonoj I checked your solutions but none of them seems to work as I expected, bundle size keeps almost the same. I don't know why Rxjs 6 drops the support for something like:
import { switchMap } from 'rxjs/operators/switchMap';
This solution is flexible and adds control over your dependencies.

agreed , seeing same issues. this is thie biggest bundle size in my vendor chunk.
All I am using is these
import { mergeMap, delay, map, catchError } from 'rxjs/operators';
import { of, concat } from 'rxjs';
import {
  ActionsObservable,
  StateObservable,
  ofType,
  combineEpics,
} from 'redux-observable';

"redux-observable": "1.0.0",
"rxjs": "6.2.2",
and i have a bundle size of 167 KB.

Using version 6.2.2 and seeing the same issue myself. I've tried following the documentation on tree shaking as linked to above but it has made no difference.
When seen through bundle analyzer, I also see every single operator included in the bundle when only importing a single operator from 'rxjs/operators' and nothing else, as a trivial test case.
Example code:
index.js
import {filter} from 'rxjs/operators';

webpack.config.js
const config = {
    entry:  './src/index.js',
    output: {
        filename: 'bundle.js',
        path: path.resolve(__dirname, '../../dist')
    },
    mode: 'production',
    resolve: {
        alias: rxPaths()
    },
    module: {
        rules: [
            {
                test: /\.js$/,
                exclude: /node_modules/,
                loader: 'babel-loader'
            }
        ]
    },
    plugins: [
        new webpack.optimize.ModuleConcatenationPlugin()
    ]
};

Bundle analyzer visualisation looks exactly like the OP's screenshot. Adding the rxPaths() alias, ModuleConcatenationPlugin() seems to make no different to what's included or the bundle size.

@patrickkunkadazn I tried your snippet and created https://github.com/kwonoj/rxjs-webpack-test , npm run build do wepback and output is

8kb in total. Rxjs is 6.2.2, as same as you mentioned: https://github.com/kwonoj/rxjs-webpack-test/blob/master/package.json#L15

Hmm interesting my output is 292kb / 99kb minified.  Thanks for creating the repo - I will scan for differences now.

@patrickkunkadazn worth to share your repo as well, so anyone can try and dig differences.

So it looks like this is caused by the presence of the babel-preset-env package:
https://github.com/patrickkunkadazn/rxjs-webpack-test
I've added the package, and also a .babelrc file. Upon build, now the entirety RxJS 6 is included. Perhaps this is the expected behaviour, but it would be good to understand why.

@kwonoj would you like to open a separate issue for this?

I had a look through every babel plugin that was being included with env that could be breaking treeshaking, and eventually narrowed it down to this one:
https://www.npmjs.com/package/babel-plugin-transform-es2015-modules-commonjs
Without this plugin present, tree shaking works, but with it present, we end up including the whole library.

I think that's expected, if import syntax is transpiled into cjs require, there's no way to make tree shaking work.

OK - I assumed as much. Considering using babel-preset-env is part of the standard workflow of someone writing in modern javascript and transpiling to ES5 or similar for production, should this be flagged in the documentation somewhere?
From what I understand, and from my testing, we can simply remove this plugin from preset-env, and

tree-shaking will then work
Webpack is still able to parse ESM import statements and convert to an ES5-compatible bundle anyway

I'm assuming this is the root cause of the issues described by the other commenters in this issue, so probably worth flagging?

I'm going to close this as it's answered for some aspect and do not have additional cases other then configuration issue. We may update documentation for precaution later.

I've been running into this issue and the suggestions above with setting modules: false in Babel configs don't seem to work in the Babel 7.x world.