GRPC Proxyless support for EnvoyFilters

Describe the feature request
Istio grpc-proxyless feature to have support for EnvoyFilter
Currently there has been limited development for grpc-proxyless feature in istio but it is one of the most performant features of istio which we are using in our production setup and empowers our infra.
Istio is helping us empower our control planes while grpc is effectively handling load balancing for our setup. GRPC XDS is developing at a rapid pace and adding new features with every release. These features are supported via the Envoy XDS API, but not necessarily directly related to istio configurations.
Istio provides support for using EnvoyFilters to pass any custom configuration to envoy if something is not readily supported by istio in the current phase.
Keeping both things in mind, it would be great if the EnvoyFilter support is extended to GRPC Xds use-cases as an experimental approach, which will allow both the grpc-proxyless and istio development to be on differnet tracks yet being supportive of each other.
As an example, we wanted to try out outlier detection, but the algorithms supported by grpc-xds have currently an alternative in istio, hence a suggestion on one of the threads was to instead use EnvoyFilter.
Affected product area (please put an X in all that apply)
[ ] Ambient
[ ] Docs
[ ] Dual Stack
[ ] Installation
[X] Networking
[ ] Performance and Scalability
[X] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[X] User Experience
[ ] Developer Infrastructure
[X] GRPC Proxyless
Affected features (please put an X in all that apply)
[ ] Multi Cluster
[ ] Virtual Machine
[ ] Multi Control Plane
Additional context
I made some changes on my local setup in the istio source code to make this working and I would to contribute for this feature.
Mainly the changes involved were exposing the cluster patcher from clutser.go so it can be accessed in cds.go
Since this will be my first contribution to istio, it would be great if the community can help me better design the code to follow the code development guidelines properly.

As an example, we wanted to try out outlier detection, but the algorithms supported by grpc-xds have currently an alternative in istio, hence a suggestion on one of the threads was to instead use https://github.com/grpc/grpc-java/issues/10656.

Not sure how do you patch as the outlier detection type is not same with envoy's

The types are same with enovy, if you are confused with https://github.com/grpc/proposal/blob/master/A50-xds-outlier-detection.md#lb-policy-config
This is an internal representation which is created after grpc-xds has parsed the enovy configuration and is not provided by the XDS API and that is okay.
If you see this
https://github.com/grpc/grpc-java/blob/0987dc401ca55885b99e98a857705396ab47828e/xds/src/main/java/io/grpc/xds/EnvoyServerProtoData.java#L299
It is taking Envoy form of config and then translating it to internal object which is then used
https://github.com/grpc/grpc-java/blob/0987dc401ca55885b99e98a857705396ab47828e/util/src/main/java/io/grpc/util/OutlierDetectionLoadBalancer.java
https://github.com/grpc/grpc-java/blob/0987dc401ca55885b99e98a857705396ab47828e/xds/src/main/java/io/grpc/xds/XdsClusterResource.java#L219
I think these are enough to be able to navigate the code.
GRPC proxyless uses only what envoy supports, just adds its own sugar over it. So it doesn't have much things different in terms of the API, hence the above patching will work. I have locally tested it as well, or may be let me know if I got something wrong in understanding the question.

Do you have an example on outlier? I am asking because i am wondering whether it can be make it in tree

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: outlier-on-percentage
  namespace: istio-poc
spec:
  configPatches:
  - applyTo: CLUSTER
    match:
      cluster:
        service: grpc-server.istio-poc.svc.cluster.local
    patch:
      operation: MERGE
      value:
        outlier_detection:
          failure_percentage_threshold: 100
          enforcing_failure_percentage: 100
          failure_percentage_minimum_hosts: 0
          failure_percentage_request_volume: 2
  workloadSelector:
    labels:
      app: grpc-client

If I understand correctly you need the EnvoyFilter configuration for the same right, this when I tested with the changes in the linked MR, was propogating to the grpc client.
How the client handles these outliers remains to be tested, but my main goal here was first to let the client understand the config and assuming the client will handle the rest.

No it is in stable state as far as I am aware of, since traffic director is a feature from Google which is built over it

I have also created an MR for the same, need guidance on how we can properly implement this.

I am not strongly against it - but I would rather avoid it if possible. GRPC has a smaller surface, and I would prefer simpler/cleaner ways to expose them - annotations/labels are not perfect either, but far easier than filters. I also think features with multiple implementations ( envoy and N gRPC languages) deserve promotion to proper API and are more likely to be stable.
The concern is that it's too easy to add an envoy filter that generates something proxyless grpc will reject, and those multi-implementation features will be stuck in alpha. They are pretty strict - and while XDS is used the content is quite different. That's why we have a separate implementation - it is fully on-demand so scale is far better.
BTW, as an alternative to EnvoyFilter I was thinking of an Gateway-attachment-like model where a real Envoy Cluster CRD could attach to a service. For routes it's a bit more complicated but possible too. This is more for the gateway to backend communications - the customized bootstrap file can technically add full listener/clusters already for envoy but not grpc.

In terms of stability - grpc side is very solid and used in prod quite a
bit, but with traffic director as xds server. For Istio - it works pretty
well but I have not seen a lot of interest and usage. There are a couple
other projects I've seen consuming XDS directly for http/tcp or grpc - and
a very nice paper from Facebook ( hyperscale and minimal cost) on benefits
of proxyless style.
On Mon, Nov 20, 2023, 04:36 Anurag Agarwal @.***> wrote:

I have also created an MR for the same, need guidance on how we can
properly implement this.
—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/issues/47927#issuecomment-1818986222, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAUR2TGMSUPBEDEAGDHWWDYFNFFXAVCNFSM6AAAAAA7RDXXZ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJYHE4DMMRSGI
.
You are receiving this because you were mentioned.Message ID:
@.***>

@costinm although I do agree that there should be a precise contract between the two, but grpc as far as I know rejects what it doesn't attend but doesn't break.
Given the different languages available as far as I know they follow grfc and although they may be 1 feature ahead or behind the other they consume things in similar ways as per the gRFC document.
We just wanted to use istio for grpc proxyless just like a control plane that can communicate with kubernetes much like how traffic director behaves.
Although I do agree that over the period of time one can contribute to make the specs more precise like actually implementing the grpc supported algorithms for both envoy and grpc proxyless but meanwhile there can be a way for adhoc items.
I would also personally prefer to keep everything under destination rule instead of half things present as destination rule and other half as envoy filter, but before that I would like to have the feature available on day 1.
I don't see much documentation or development in grpc proxyless since it has released and has also be pointed out in multiple issues.

I don't disagree. Proxyless in istio is - like many other features - stuck in alpha, and for the most part had very few maintainers and not a lot of attention. I get some 20% time for istio - and grpc is rarely on the priority list.
The main reason ( besides time ) I have been reluctant about is the new Gateway API and ambient. Traffic Director support for proxyless is based on HttpRoute and K8S gateway API surface, and the integration with Ambient - based on opting out of interception specific ports - has been de-prioritized or had push back.
Moving proxyless grpc with Istio old API and without ambient integration doesn't seem useful.
Once ambient ships and we may reopen the discussion on selective interception - and the common API surface gets more settled - we can probably move proxyless forward.

One reason why I feel this feature is not widely used is because it is not documented even on the istio official website and the fact that people will have to move their services to use grpc.
Every article points to traffic director and there is just one very old blog post available for handling grpc proxyless.
But this is one of the key features which if given time and development can be invincible for the users. We ourselves in our organisation has saved 1000s of $$ with just this feature and made our lives simpler.
I can help with few contributions for the grpc proxyless since we have just adopted it and are eagerly using it in our production system.
Nonetheless what about this issue, does it sound okay to implement EnvoyFilters as starters.

I second the Anurag's words. We are having a hard time implementing the proxyless gRPC due to very poor documentation. We'd likely see the higher rate of feature adoption if it's documented and stated as mature. For now the only source of some docs is TrafficDirector guides.
Outlier detection support is one of the missing features to have a solid proxyless XDS setup.

@akamac we had actually started implementing the outlier detection in istio, but a lot of work has come up in our organisation becuase of which we have for now put it on hold for sometime.


userpie.com download file is other

http://userpie.com dowload link will download a zip file containing something that is not userpie files.

Seems this is still the case, the download link is userpie.com/latest.zip... but the actual file downloaded is userpress-2.0.12-1.zip

Still an issue, this downloads the Wordpress version.


want to help: using labels in github

Hi There :-)
Can you try to add labels in github to show where you need help, what you expect to have in 1.0, and so on.
I would like to help but for efficiency would like to filter which topics is more urgent than another.
regards
Cédric

Hej guys, thanks for the offer. I guess I don't have too grand a vision for how this guide should evolve, so every idea is welcome.
What helps most is engaging with issues, help me solve open questions and improve the existing explanations in the guide.
Regarding new features, here are a few ideas.

I removed the lncli-web section, as it was quite error-prone and I not really able to support it. Maybe there's an opportunity?
A good description on how to generate external SSL certificates in combination with DynDNS (eg. to use for external access for mobile wallets like Shango, ZAP or Lightning Joule
How to connect Lightning Joule

How exactly would you use lables? Create new issues with "Help wanted" for task, or something like that?


wayland uses non-native decorations and can't be resized or moved [arch+gnome]

The following is an image of a program I'm working on that uses winit with the wayland backend (via glutin via glium via conrod):

The window decorations are thin gray bars rather than the native gnome decorations that look like this:

Does anyone know why this might be? @vberger any ideas on what's happening or where I could look to fix it? Any advice appreciated :)

FWIW when running "gnome with xorg" the window decorations look normal, however I also run into other issues with HiDPI and would prefer to get wayland working nicely anyway.

Well, on wayland (especially gnome), decorations are drawn client side. As such, the decorations are currently handled for winit by my wayland-window crate.
For now, this crate only draws basic grey decorations, because I haven't yet done the work to make them prettier and include buttons to close/minimize, etc... But there is no simple way to make them look native. This would require either:

manually parsing the GTK configuration and reproduce the behavior of the GTK engine (this is kind of a lot of work, and similar work would need to be done for Qt for KDE environments).
using GTK to open the window (which kind of defeats the the purpose of winit).

However, resizing the window (by clicking and dragging the border) and moving it (by clicking and dragging the large top decoration) should work. If not, it is either a bug of wayland-window or of gnome.

I understand the fact that there is no decoration due to wayland policy of client decoration, but what about the resizing?
Also, when I use piston_window with SDL2 backend, the window has the GTK decoration. Do you mean that the libsdl is using the GTK lib to draw those decorations?

Just to clarify, by "no decoration" do you mean that the window is completely borderless, or are there just grey borders with no buttons, as in the first screenshot of this issue?
The resizing problem has been diagnosed as a bug in mutter : https://bugzilla.gnome.org/show_bug.cgi?id=781811
Regarding SDL, if I believe this discussion https://bugzilla.libsdl.org/show_bug.cgi?id=2710 it looks like SDL makes no attempt to draw any border. So maybe mutter recognises SDL windows and decorates them itself? Or maybe your version of libSDL2 does not have wayland support and uses XWayland, in this case mutter will decorate it in all cases.
A simple way to test this last hypothesis would be to run your program using SLD2 with the environment variable WAYLAND_DEBUG=1 set. If nothing is printed in the console, it's likely you aren't using wayland.

There are grey borders as in the screenshot, no problem.
Indeed, you were right about SDL : with the Wayland debug, there is no output. The libsdl2 provided by the Debian repository uses XWayland as you thought.
The lack of window decorations makes this crate not really useful under wayland, IMO. Maybe a solution exists to ask the decoration to the desktop… but I have no idea about the way Wayland works.

I think it would be worth making it an explicit goal to get native decorations working with wayland, even if it didn't have the highest priority.
I realise this is not a part of wayland's policy, however winit's aim is to provide a portable native window API across platforms - users will naturally be disappointed if they can get native windows on macos, windows and x11 but not on wayland.
I agree it'd probably take a lot of effort. At least if we maintain that this is something winit would like to have, a potential contributor may be more willing to come along and champion it, knowing that their contribution would be very welcome :)
That said, I wonder if this is even possible with winit which does not have access to any graphics context (OpenGL or otherwise). I guess they could be drawn using software rendering? Is this how decorations are normally drawn? 🤔

Regarding the decorations under wayland, If the goal is to have functional decorations, even if they do not match the look&feel of the desktop environment, this is just a question of implementing the missing parts in wayland-window (aka, the close/maximize/minimize buttons and the name in the titlebar, I think?). And it's certainly possible to adapt wayland-window to make the borders more pretty than these plain grey rectangle.
If the goal is to match the look&feel of the desktop environment, this is going to be much more difficult: there is no unified look&feel in linux, and even less with wayland where clients are supposed to draw their own decorations. Already GTK and Qt try to each provide the tools to mimic the styling of the other, with more or less success. So I see two possible alternatives:

"Simply" use GTK/Qt/etc... to create the window instead of using wayland-window
Try to find and parse any available GTK/Qt configuration to manually mimic it

The first choice add some large dependency to winit (GTK or Qt or...), the second seems plainly unrealisable to me.

KWin supposedly provides an API to make it draw window decorations for the application: https://blog.martin-graesslin.com/blog/2015/12/server-side-decorations-coming-to-kwinwayland/
Gräßlin planned to add something to the Wayland protocol so that it can be supported by other window managers besides KWin, but I don't know what happened with that plan.

While working with vulkano it seems I've encountered this problem. (see link to vulkano issue earlier)
On Debian 9 with Wayland and Gnome everything worked fine (i.e. with native controls), but after installation of libwayland-dev (which is in the libgtk-3-dev dependency tree) I've got the same gray window decorations as in the OP, also window could not be resized or moved. Also full-screen application got affected too, after launching app part of the rendered image appears, but nothing else.

If you had native controls prior to installation of libwayland-dev, it means that winit was actually connection to Gnome as an X11 client, using the compatibility layer. Installing libwayland-dev made it possible to use the wayland backend so it used it.
The gray decorations are what winit currently draws, as there is no "native decorations". Improving them is tracked by https://github.com/Smithay/wayland-window/issues/19 and https://github.com/Smithay/wayland-window/issues/4.
The move/resize issues have been tracked to a bug in mutter, see https://bugzilla.gnome.org/show_bug.cgi?id=781811. I don't know how long it will take for the bugfix to reach your repos.
Could you open a new issue for the fullscreen problem, possibly with a minimal reproducing example? It seems to me like it's probably an other issue, not clearly related to the decorations.

Is it possible to somehow add ability to enforce use of compatibility layer maybe through feature gate in the winit?
I'll try to write a minimal example for fullscreen issue next week.

If I'm not confusing things, starting with version 0.7.1 of winit you can set the environment variable WINIT_UNIX_BACKEND=x11 to force the use of the X11 backend of the wayland one (and thus use the compatibility layer).

Closing in favor of #306


BUG 2276533: osd: Legacy osds on lvm pvcs crash on resize init container

OSDs on LVM-mode PVCs are failing to come up and crashing in the expand-bluefs init container. To avoid the crash and allow the OSDs to start, a workaround was found to simply remove that init container. Now we disable the OSD resize for this case to avoid others hitting this during upgrade as well.
Signed-off-by: Travis Nielsen<EMAIL_ADDRESS>(cherry picked from commit acd7b4faa05330c66636c9f034c156c184a20deb) (cherry picked from commit d0fbec4b8d62b948986586b90fb96a3ab96fe893)
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2276533

Checklist:

[ ] Commit Message Formatting: Commit titles and messages follow guidelines in the developer guide.
[ ] Reviewed the developer guide on Submitting a Pull Request
[ ] Pending release notes updated with breaking and/or notable changes for the next minor release.
[ ] Documentation has been updated, if necessary.
[ ] Unit tests have been added, if necessary.
[ ] Integration tests have been added, if necessary.

/bugzilla refresh

/bugzilla refresh


feat: support Ronbosense lidar

PR Type

New Feature

Description
This change proposes Robosense Helios 16 、Helios32 、Bpearl 、Ruby Plus 、M1 Plus and E1 support for nebula.
Review Procedure
Remarks
We have tested these newly added codes using the corresponding Robosense Lidar and all tests have passed.
To meet our needs, we have added a function in the TcpDriver class (i.e. bool asyncSendReceive (std:: vector&buff, Functor func). By the way, the code we have added contains abbreviations for some proprietary terms, such as (MSOP: Main data Stream Output Protocol), which cannot pass spell checking. Can you solve this problem?
Pre-Review Checklist for the PR Author
PR Author should check the checkboxes below when creating the PR.

[ ] Assign PR to reviewer

Checklist for the PR Reviewer
Reviewers should check the checkboxes below before approval.

[ ] Commits are properly organized and messages are according to the guideline
[ ] (Optional) Unit tests have been written for new behavior
[ ] PR title describes the changes

Post-Review Checklist for the PR Author
PR Author should check the checkboxes below before merging.

[ ] All open points are addressed and tracked via issues or tickets

CI Checks

Build and test for PR: Required to pass before the merge.

Hello, I am from Robosense. This change proposes Robosense Helios 16, Helios32, Bpearl, Ruby Plus, M1 Plus and E1 support for nebula. Please review this change. If you have any questions, please feel free to contact<EMAIL_ADDRESS>


configd: dashboard hangs with ipsec status widget when configd is down

I have widgets for services and ipsec. I wanted to stop/start configd, but the stop command hangs. Configd stop works fine on the services diagnostics page.
Can we lower the "configd not there" timeout?

@fichtner Maybe we should lower the default in https://github.com/opnsense/core/blob/73f353c9c27e68de57c82593ce633bc3461e39bc/src/opnsense/mvc/app/library/OPNsense/Core/Backend.php#L76 to something like 20 seconds and pass the parameter with a higher value when needed. Most calls won't take that long or run in the background.

How about a response timeout (120 secs as defined) and a separate connect timeout (5-10 secs)?
Is connect instant or will it have to wait for the last configd command to finish?

I guess your right, adding another parameter for connect timeout would be better. If we can avoid configd calls during boot it might even be better to set it to a fixed default.

fixed sounds good to me

works as expected, thanks!


Add handling for ambiguous parsing options

This pull request addresses this bug report.
Let me know if there are any issues or places where I can improve it!

Thanks!

Thanks!

Any time! This was all really pleasant to work on—the Ruby community's reputation is deserved. Thanks!


Coinbase Pro Funding Type addition

Coinbase pro require type as a parameter in the trade history parameters

closing because another PR covered this


For a 0.1.3 release

Codecov Report

Merging #33 (490db4d) into master (ddda45a) will not change coverage.
The diff coverage is 87.50%.

@@           Coverage Diff           @@
##           master      #33   +/-   ##
=======================================
  Coverage   91.30%   91.30%
=======================================
  Files           4        4
  Lines         299      299
=======================================
  Hits          273      273
  Misses         26       26

Impacted Files
Coverage Δ

src/models.jl
89.28% <87.50%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ddda45a...490db4d. Read the comment docs.


Entity FX crash

Alright, we're getting a crash at server start (but not client side) that causes a crash, at first we thought it was another mod we were having trouble with, but after removing that mod the error happened again and we tracked it down to the latest version of Thaumic Bases (Because I clearly can't read error reports worth a damn. Don't judge me >.>)
Crash Report: http://pastebin.com/cnQhH6z7

Already fixed in 34ce428fee67c9f837968c850d922b6a6b91ec2d
Untill the version is released use the server-side fix in the Releases section here, on github.

Awesome, thanks for the quick response!


Integration with Mockk

Hello. Great library! Might there be any future integration with mockk for kotlin?

Awesome. Though I just discovered that koin already does integration with mockito. Thanks for the help.


How to report a Segfault?

Hi.
I'm experiencing a segfault in my test suite for my rather complicated and probably overly ambitious project at https://github.com/markburns/delfos
(Project aim - record method calls and runtime type information in a ruby application)
I'd like to know how best to report this, given that I now have very limited time for open source and have not followed the changes in depth for the move to ruby/spec. So apologies up front.
Admittedly non-trivial replication steps (sorry)
#install neo4j on your system

git clone git@github.com:markburns/delfos.git
git checkout origin/support_recording_class_method_inherited
bundle install

# BEWARE this currently deletes all data in the neo4j instance at localhost:7474

bundle exec rspec lib/delfos/integration_spec.rb

Output - https://gist.github.com/markburns/e26481b83c91c3726a0aa497f96b16ab
Friendly guidance request
I have some questions - because I didn't see mention in the CONTRIBUTING.MD:

Is here with a minimal case for replication the best place to report a segfault?
Is a PR with the above preferable?
What is the workflow? I.e. if a green build is required for a PR to be merged, does that mean a feature branch with a failing spec can only be merged after the underlying ruby issue is fixed?
A side question - is this suite being green a prerequisite for release of a new ruby version?

Hello,
You should report this to the MRI bug tracker at https://bugs.ruby-lang.org/


Generate additional namespace usages from the swagger definitions

Adds an option to generate additional namespace usages from the swagger definitions
Reason for the change:
My service and client projects both have dependencies on libraries that contain DTOs.
My swagger.json contains the namespaces in the definitions but there's currently no way to utilize those in NSwag (as far as I know).
These changes add an option to NSwag to generate namespace usages for DTOs directly from the swagger definitions (given that they're already included in the swagger.json, otherwise the option does nothing).

swagger.json:
...
  "definitions": {
    "My.Namespace.MyObject": {
       ...
    },
    "My.Namespace2.MyObject2": {
       ...
    },
    "My.Namespace2.MyObject3": {
       ...
    }
  }
...

client.cs:
...
using My.Namespace;
using My.Namespace2;
...

This solves a quite specific problem and also is a leaky abstraction (swagger should be self-describing and the server technology should not not leak into it).
Why do you need these usings/what in the generated code is using it? Do you exclude types or are not all types in the spec?

Currently, our web api is implemented in C# (AspNetCore) using DTO classes from a separate library of data models, e.g., My.DtoLib.SomeObject. We are trying to generate C# clients that will use those same model classes, so we have generateDtoTypes set to false and the excludedTypeNames set to an empty array.
Our thinking was that the "DRY-est" practice would be to have the data model definitions exist in one place. Since we will be calling the web api from our own server logic, we want to be able to use the client with our own types.
However, the generated client classes are referencing the data model classes without the full namespace, e.g. SomeObject. We couldn't find any setting for the nswag.json file that would add the full namespace, so we came up with this solution.
Any feedback you can give us would be greatly appreciated - thanks for taking the time to look at this

Can't you just add your namespaces using the AdditionalNamespaceUsages setting and import just import your existing types?

We were hoping for a solution where we wouldn't need to manually add new namespaces/dependencies, especially since those namespaces were already being automatically added to the swagger.json file we are generating. That being said, feel free to close this PR if you guys don't like the design, it does seem like a very specific problem. Thanks again for the feedback!


Implement missing metadata ui fields

This issue is imported from pivotal - Originaly created at Oct 7, 2019 by bstrooband
Show the relevant metadata in the generated XML (like service name, contact persons, signing details and so forth) Do so by using a DTO class.
This needs to be done and validated for all endpoints:

/authentication/stepup/metadata
/authentication/idp/metadata
/authentication/sp/metadata
/authentication/proxy/idps-metadata

It would be valuable to use the Entityfactory for the different flavours.

Added specific story for what I think is still missing (Thijs Kinkhorst - Dec 20, 2019)


[Bug] TotK-RstbCalculator does not reset the cached vanilla RSTB file when changing the dump version

When changing the game path setting to a different dump version and clicking save, TotK-RstbCalculator will keep the vanilla RSTB file from the previous dump in cache, and merging will continue to generate a resource table for the previous version (until closing and reopening TKMM)

The vanilla file is parsed every time a new RstbGenerator is constructed. Can you provide steps to create and verify that this is occurring?

Open TKMM, merge a first time, go to the settings, change the dump location to a different version of the game, then merge a second time. The second time it merges, it will still use the previous version for the resource table. RSDB files on the other hand are correct

Fixed in https://github.com/TKMM-Team/Tkmm/commit/f68c4c166de134badebf134f098eddc2d30985a9


Avoid creating a temporary column during fix_the_geom_type!

This change was introduced for imports but not for synchronizations (because synchronizations do not register new tables, so they must reproduce the cartodbfication and other stuff done in table registration): https://github.com/CartoDB/cartodb/commit/7f9b18b38051eac4e43fc3b8fd21e8ee2530dc8c
The change in how ST_Multi is applied in Table::set_the_geom_column! is replicated here into Synchronization::Adapter#fix_the_geom_type!.
Otherwise we may have a problem (as we already have had with some connector imports): if the import source produces a table where the geometry is of type geometry rather than geometry(Geometry,4326) and it contains geometries without an SRID (or with another SRID), it will fail when copied to the new column the_geom_simple of type geometry(Geometry,4326).
With the changes here, the problem is avoided, because no new column is used (so no change of geometry type), and the proper conversion of the geometry to geometry(Geometry,4326) is handled by the cartodbfication, performed afterwards.

/cc @rafatower for awareness. See https://app.clubhouse.io/cartoteam/story/102407/syncronization-error-in-data-table-view

Just to give a little bit more context: that the_geom_simple was introduced as part of https://github.com/CartoDB/cartodb/pull/7543 , and part of the reasoning behind it was to deal with some corner cases in syncs and after moving functionality to CDB_CartodbfyTable.


Adding class methods to yuupay integration

Fixes: The integration currently breaks Shopify because Yuupay.return doesn't exist.
@girasquid @j-mutter review please

:+1:

fix #38081  <-- so it auto-closes the issue

Should we just change the Universal module to define these, or is it not worth it if we're not going to be doing this for many gateways?

@j-mutter I don't see us doing this again for another gateway. It adds more work to integration universals.

Cool. :shipit:


Voided Transactions & Historical Balances Discussion

Hello and thank you for building medici! It has been a great use to me.
I have a use case where I need to have accurate historical balances of accounts, and sometimes may need to void past entries (due to errors).
After voiding a past entry, I would assume the historical balance to also change. IE the balance of an account 1 second after the initial (voided) entry, to be equal to the balance 1 second before the datetime of that journal (as if that initial entry never existed in the first place).
However this is not the case with the current implementation of void as currently, voiding a journal creates an opposite entry at the present date (new Date()), instead of at the date of the original entry.
Example:
State before the void:
// t0: balance of account1 100
// t1: add journal entry crediting 250 to account1
// t2: balance of account1 350

Expected state after the void:
// t0: balance of account1 100
// t1: add journal entry crediting 250 to account1
// t2: balance of account1 100         <--- this is what I expect
...
// tcurrent: void journal entry at t1
// tcurrent+1: balance of account1 100

Actual state after the void:
// t0: balance of account1 100
// t1: add journal entry crediting 250 to account1
// t2: balance of account1 350         <--- this is what I get
...
// tcurrent: void journal entry at t1
// tcurrent+1: balance of account1 100

I am aware of the difference between timestamp (when the journal was inserted) and datetime (when the journal happened in reality).
Questions:

Is my use case reasonable? Should I be able to expect to have accurate historical balances that may change if we void transactions?

In book.void there any reason we pass the null date (which becomes new Date()) to the new journal which voids an old journal entry, instead of just passing the date of the original journal (or the millisecond right after it?)

Do you see any potential ways of implementing this without changing the code of the library? Should I implement a mechanism for deleting the journals (and transactions) altogether? Besides violating the rules of good bookkeeping, is there any technical reason why we shouldn't delete journal entries and transactions in medici?

Assuming answers to 1. and 2. agree with this, are you happy for me to push a fix updating the date of the voiding journal entry to the same date as the original journal entry?

Again, thank you so much for this library, it has saved me hours of work. I really appreciate this small and mighty code base.

More on this, it doesn't seem like book offers an easy way to access journalEntries directly, if I take the delete route.
I can access transactions via book.ledger() and remove them, or just access the medici_journal via mongo directly, bypassing the book API - but not having a direct API just makes me doubt that journals and transactions are intended to be deleted. I am also not using balance snapshots, if this helps (i set balanceSnapshotSec to zero).

Hi!

Yes. Yes.
No reason.
Do NOT delete entries. Otherwise, it won't be ledger any more. Technical issue - balance caching. You'd need to clean the collection too.
Yes.

Thank you!

I have some different thoughts on this.
Imagine in the real world: Person X orders product A for 100 credits. You need now to move the product A from your assets to the sold assets ledger, person X anticipated payment needs to be booked into receivables. So if person X transfers the money to reduce the receivables and book the incoming money into the cash account or whatever.
If we make a balance after the person ordered but not yet paid, you could say: We have 100 credits in receivables.
Now, person X sends an E-Mail cancelling the purchase. Now the contract is void and you can void the transaction. In legal terms a legal act can be in effect ex tunc (with effect to the past) or ex nunc (with effect from now on). A contract cancellation could be ex tunc, with effect to the past. Is this relevant for the booking? I think not.
Why?
Imagine your booking year ends on 31st october. You get the order of person X on the 31st october and now you close your ledger. And on 1st september you have new books and now the cancellation happens. You cant modify your old ledger, as it closed.
So it is actually very very very correct that the new void entry has the curent date.
So:
I am -1 on voiding by default back to the transaction date.
I am +1 if you make it viable to set the void date to the transaction via an option. Maybe there is some case, where booking like that makes some kind of sense, and with that option, also that person would be happy.

Since I'm not as smart as @Uzlopak let's do what he said. :)

That's not a technical question it's an accounting question and generally
surging you never delete a transaction you just create a transaction to
offset it so that you have all the history...eg booked revenue at time of
order and booked contra revenue at time of cancellation
On Mon, Nov 18, 2024, 11:43 PM Vasyl Boroviak @.***>
wrote:

Hi!

Yes. Yes.
No reason.
Do NOT delete entries. Otherwise, it won't be ledger any more.
Technical issue - balance caching. You'd need to clean the collection too.
Yes.

Thank you!
—
Reply to this email directly, view it on GitHub
https://github.com/flash-oss/medici/issues/117#issuecomment-2484696725,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB6ARVNYAUN44WN4BBUSMW32BK6YFAVCNFSM6AAAAABRX3M3X6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOBUGY4TMNZSGU
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Dear @Uzlopak and @koresar, thank you for getting back to me.
Okay, it makes perfect sense to add an option for the date when voiding a transaction, to accomodate for different use cases.
For my specific use case there is a third option - changing the date on a journal entry (and underlying transactions), as the date was inputted incorrectly. But this may cause some issues with the balance caching. Let me figure out a solution to my use case and I let you know.

Okay, it makes perfect sense to add an option for the date when voiding a transaction, to accomodate for different use cases.

If so, I'm happy see a tiny little PR for exactly that and nothing else in it. :)

Do NOT delete entries. Otherwise, it won't be ledger any more. Technical issue - balance caching. You'd need to clean the collection too.

@koresar Thanks for the answers. I understand this goes against the principles of a ledger. But it may be necessary for our use case.
Just to clarify - if I delete the entry & all transactions involved, and I delete everything in the balances collection, from a purely functional perspective, medici should OK with that DB state?

I ended up adding the ability to delete journals and transactions. Okay thanks for that guys! I guess we can close this now. Thanks again for your input 🙏

Great to hear that you could adapt it for your needs. I would have not added deletion because it basically violates the ledger rules. But hey, we are not accounts ;)


[vcpkg baseline][sentencepiece] Fix flags, linkage, deps

Complete alternative to #42205.

Thanks!


x/sys/unix: Utsname.Version is too small?

Go version
go1.20.12 freebsd/386
Output of go env in your module/workspace:
Not relevant.

What did you do?

Built the following minimal reproducible Go program targeting FreeBSD, shown here:
https://github.com/prometheus/node_exporter/issues/2809#issuecomment-1914467114
Copied the file to a running instances of pfSense
Ran it
Got an "Cannot allocate memory" error
Cloned sys/unix locally and played around with the syscall_freebsd.go file, in particular I commented out sections of the Uname method until I found the section that caused the error. That was this section:

 mib = []_C_int{CTL_KERN, KERN_VERSION}
          n = unsafe.Sizeof(uname.Version)
          if err := sysctl(mib, &uname.Version[0], &n, nil, 0); err != nil {
                 return err
}

Upon further digging I found that uname.Version is set to 256 bytes here:
https://cs.opensource.google/go/x/sys/+/master:unix/ztypes_freebsd_amd64.go;l=639-645;bpv=0?q=Utsname&ss=go%2Fx%2Fsys
so then I,
6. Modified the above locally to a larger value, 1024.
7. Rebuilt the test binary with all code from step 5 uncommented.
8. Reran the binary on the pfSense OS,
9. No error.
What did you see happen?
Got "cannot allocate memory" until I made the Utsname.Version field larger.
What did you expect to see?
The Uname to be returned.
Apologies if this isn't the correct venue for this report. I'm not a Go developer, nor have I ever worked on FreeBSD or pfSense. I am willing to do what's needed to fix this, but I'm not sure exactly what's "wrong" here. I know the types_freebsd.go file is auto-generated, but I'm not familiar with the domain to know if:

pfSense is errantly reporting a KERN_VERSION larger than 256 bytes, or
the types_freebsd.go file needs to be regenerated/modified in some way.

Edit: Fwiw I was doing some reading and it looks like maybe this isn't the first time someone has encountered this issue in FreeBSD?
https://github.com/freebsd/freebsd-src/blob/82bebc7936581e9c4ff3604d4cb998f8cc017f50/sys/kern/kern_xxx.c#L351-L371

(attn @golang/freebsd)

Both FreeBSD 13 and 14 libc use 256 bytes for utsname struct fields (Or 32 bytes depending if the __KERNEL__ macro value is defined).
The underlying implementation via __xuname(int namesize, void *namebuf) however is variable size:
https://cgit.freebsd.org/src/tree/sys/sys/utsname.h?h=stable/13
https://cgit.freebsd.org/src/tree/lib/libc/gen/uname.c?h=stable/13
https://cgit.freebsd.org/src/tree/lib/libc/gen/__xuname.c?h=stable/13
https://cgit.freebsd.org/src/tree/sys/sys/utsname.h?h=stable/14
https://cgit.freebsd.org/src/tree/lib/libc/gen/uname.c?h=stable/14
https://cgit.freebsd.org/src/tree/lib/libc/gen/__xuname.c?h=stable/14
This is also true for pFSense AFIK: https://github.com/pfsense/FreeBSD-src/blob/RELENG_2_7_2/sys/sys/utsname.h

The following C reproducer returns ENOMEM on pfSense CE 2.7.2, while successful on FreeBSD 14.0-RELEASE:
#include <sys/utsname.h>
#include <sys/cdefs.h>
#include <sys/param.h>
#include <sys/sysctl.h>
#include <errno.h>
#include <stdlib.h>

int main(int argc, char **argv) {
        struct utsname u = {0};
        int mib[2], rval;
        size_t len;
        mib[0] = CTL_KERN;
        mib[1] = KERN_VERSION;
        len = sizeof u.version;
        char* q = (char *)u.version;
        rval = sysctl(mib, 2, q, &len, NULL, 0);
        printf("rval: %d, errno: %d\n", rval, errno);
        printf("u.version: %s\n", &u.version[0]);
        return 0;
}

rval: -1, errno: 12
u.version: FreeBSD 14.0-CURRENT amd64 1400094 #1 RELENG_2_7_2-n255948-8d2b56da39c: Wed Dec  6 20:45:47 UTC 2023
    root@freebsd:/var/jenkins/workspace/pfSense-CE-snapshots-2_7_2-main/obj/amd64/StdASW5b/var/jenkins/workspace/pfSense-CE-snapshots-2_7_2-main/sources/Fr

rval: 0, errno: 0
u.version: FreeBSD 14.0-RELEASE-p3 #0: Mon Dec 11 04:56:01 UTC 2023
    root@amd64-builder.daemonology.net:/usr/obj/usr/src/amd64.amd64/sys/GENERIC

Looks like we should suppress ENOMEM and truncate to match  the __xuname behavior in eg:
https://github.com/golang/sys/blob/2f2cc5dae4e240ccd9b4142205a71a311328419f/unix/syscall_freebsd.go#L190-L192

Change https://go.dev/cl/562617 mentions this issue: unix: supress ENOMEM errors from sysctl's implementing Uname(uname *Utsname) due to truncated fields.

@paulzhol I have tested the above patch on my pfSense machine, this works, thank you!


Fix NameError for Bundler::Plugin::API::Source

There is a missing require statement for the API source under some
circumstances. I noticed it when I was starting Foreman after updating
to Bundler 1.14.1. Here's the stack track:
↪ foreman start -f Procfile.pow
~/.rvm/gems/ruby-2.2.3/gems/bundler-1.14.3/lib/bundler/rubygems_ext.rb:45:in `full_gem_path': uninitialized constant Bundler::Plugin::API::Source (NameError)
  from ~/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/rubygems/basic_specification.rb:145:in `block in full_require_paths'
  from ~/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/rubygems/basic_specification.rb:144:in `map'
  from ~/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/rubygems/basic_specification.rb:144:in `full_require_paths'
  from ~/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/rubygems/basic_specification.rb:77:in `contains_requirable_file?'
  from ~/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/rubygems/specification.rb:949:in `block in find_in_unresolved'
  from ~/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/rubygems/specification.rb:949:in `each'
  from ~/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/rubygems/specification.rb:949:in `find_all'
  from ~/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/rubygems/specification.rb:949:in `find_in_unresolved'
  from ~/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:74:in `require'
  from ~/.rvm/gems/ruby-2.2.3/gems/bundler-1.14.3/lib/bundler/plugin/api/source.rb:2:in `<top (required)>'
  from ~/.rvm/gems/ruby-2.2.3/gems/bundler-1.14.3/lib/bundler/rubygems_ext.rb:45:in `full_gem_path'
  from ~/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/rubygems/basic_specification.rb:145:in `block in full_require_paths'
  from ~/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/rubygems/basic_specification.rb:144:in `map'
  from ~/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/rubygems/basic_specification.rb:144:in `full_require_paths'
  from ~/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/rubygems/specification.rb:1370:in `add_self_to_load_path'
  from ~/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/rubygems/specification.rb:1283:in `activate'
  from ~/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/rubygems/core_ext/kernel_gem.rb:67:in `block in gem'
  from ~/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/rubygems/core_ext/kernel_gem.rb:66:in `synchronize'
  from ~/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/rubygems/core_ext/kernel_gem.rb:66:in `gem'
  from ~/.rvm/gems/ruby-2.2.3/bin/foreman:22:in `<main>'
  from ~/.rvm/gems/ruby-2.2.3/bin/ruby_executable_hooks:15:in `eval'
  from ~/.rvm/gems/ruby-2.2.3/bin/ruby_executable_hooks:15:in `<main>'

The line in question is:
load Gem.activate_bin_path('foreman', 'foreman', version)

So it appears that the Gem.activate_bin_path method is the entrypoint
where this fails without the proper require.

There is another PR #5379 that is fixing this exact same issue. Can you post steps on how to reproduce this error and the output of bundle env
Can you also check to make sure that rubygems is up to date. gem update --system

Whoops, I didn't see that issue when I searched for a preexisting one.
I ran the gem update --system and it stopped the error from happening. I did this before running bundle env so this part is a little tainted:
bundle env:
Environment
Bundler   1.14.3
Rubygems  2.6.10
Ruby      2.2.3p173 (2015-08-18 revision 51636) [x86_64-darwin14]
GEM_HOME  /Users/michael/.rvm/gems/ruby-2.2.3
GEM_PATH  /Users/michael/.rvm/gems/ruby-2.2.3:/Users/michael/.rvm/gems/ruby-2.2.3@global
RVM       1.27.0 (latest)
Git       2.10.1
Platform  x86_64-darwin-14
OpenSSL   OpenSSL 1.0.1j 15 Oct 2014
rubygems-bundler (1.4.4)

Bundler settings
jobs
  Set for the current user (/Users/michael/.bundle/config): "7"
gem.test
  Set for the current user (/Users/michael/.bundle/config): "rspec"
gem.mit
  Set for the current user (/Users/michael/.bundle/config): "true"
gem.coc
  Set for the current user (/Users/michael/.bundle/config): "false"

I trimmed the Gemfile because I think the only relevant thing is the line in foreman, which was in the original comment (it bombs out on that line).

Do you know what version of rubygems we were using before?

The default one for ruby-2.3.1p112 ... Sorry, I should have noted it before updating!

@michaelherold It looks like you're not longer having this issue, can i close this?

Yep, that's fine with me.


google experiment not working

After setup experiment id, the function createAnalyticsScriptTag() never is called, therefore the experiment never load.
Also is pending a function that retireves the experiment value:
variationNumber = cxApi.getChosenVariation();
https://developers.google.com/analytics/devguides/collection/analyticsjs/experiments#cxjs-getchosen

@LucasIsasmendi - Did you add the Analytics service to at least a single run method as is described in the documentation? The service has to be instantiated at least once and that only happens if it is referenced by an angular.run method or a controller.

Thanks justinsa, I did the update and now that line of code runs, however, as per google docs, experiment call should return a variation number, where is that code? how can I get the callback from google to obtain the variation number?
An example from other script is:
angularLoad.loadScript('//www.google-analytics.com/cx/api.js?experiment=' + experimentId).then(function() {
var choosenVar = cxApi.getChosenVariation()
var chooseVar = cxApi.chooseVariation()
})

@LucasIsasmendi that isn't a callback from Google in the example you provide, but a thenable callback from a promise that happens to be for a script post-load. If you have Analytics in the angular run method then cxApi may already be available as a global by the time the method body executes. Ideally it will be but that depends on the script loading performed by the browser. For example, you may be able to do the following:
angular.run(function (Analytics) {
  var choosenVar = cxApi.getChosenVariation();
  var chooseVar = cxApi.chooseVariation();
});

The above might also result in a race condition because it isn't guaranteed that the api.js script will have definitely been loaded by then.

Great!, it works within the run section as you mention.
Thanks!


Upgrade ReactFlow version

Reactflow 12.0 introduced some breaking changes. We should upgrade to the latest available version to utilize the new features and improvements. Migrating may also fix the issue mentioned here

You can find all the information needed ont heir website


Produce a form to gather feedback from the backers and the community

Goal: get a sense for the students' expectations and gather qualitative feedback about improving our material and resources.
Questions that need answers:

Who is going to watch the course? Who's the student? (past experience)
What do you expect to get/learn from this 2d course series?
How can we improve tutorials for you, compared to the ones we make now?
Would you use assignments to put the techniques you learn in practice?

Done! I'll send it to backers first and share it publicly later


Getting ManagementForm data is missing or has been tampered with error on every save

Using v0.5.3 with Wagtail 2.9.0 and Django 3.0.7.
I'm trying to use CondensedInlinePanel in the ModelAdmin, but while it renders beautifully, it won't save.
Here's the panel config:
        CondensedInlinePanel(
            "menu_items",
            label="Menu Items",
            card_header_from_field="title",
            panels=[
                FieldPanel("day_of_week"),
                FieldPanel("title"),
                FieldPanel("description"),
                FieldPanel("portion_per_order"),
                ImageChooserPanel("image"),
                FieldPanel("price"),
                FieldPanel("allergens"),
                FieldPanel("menu_category"),
                FieldPanel("order_limit"),
                FieldPanel("highlight"),
                FieldPanel("include_in_portion_count"),
            ],
        ),

It's possible this is related to an interaction with ModelCluster and/or taggit. The traceback seemed to be getting stuck on my allergens field, which is a tag field.
If I comment out that field and the MenuItemAllergen model (ie. the through model for allergens), then the error goes away.
I'm not sure I can follow enough of the internals of this app to debug it in detail though.


RTL support?

hi
do u have a RTL support

Hi @Haizone! At this time, we don't have plans to support RTL, however, we're open to pull requests! :)


Cannot be required on Laravel 9 due dependency problems with symfony/process

Describe the bug

sail composer require nesk/puphpeteer

- nesk/rialto[1.2.0, ..., 1.3.0] require symfony/process ^3.3|^4.0 -> found symfony/process[v3.3.0-BETA1, ..., 3.4.x-dev, v4.0.0-BETA1, ..., 4.4.x-dev] but the package is fixed to v6.0.5 (lock file version) by a partial update and that version does not match. Make sure you list it as an argument for the update command.
- nesk/rialto 1.4.0 requires symfony/process ^3.3|^4.0|^5.0 -> found symfony/process[v3.3.0-BETA1, ..., 3.4.x-dev, v4.0.0-BETA1, ..., 4.4.x-dev, v5.0.0-BETA1, ..., 5.4.x-dev] but the package is fixed to v6.0.5 (lock file version) by a partial update and that version does not match. Make sure you list it as an argument for the update command.
- nesk/puphpeteer 2.0.0 requires nesk/rialto ^1.2.0 -> satisfiable by nesk/rialto[1.2.0, 1.2.1, 1.3.0, 1.4.0].
- Root composer.json requires nesk/puphpeteer ^2.0 -> satisfiable by nesk/puphpeteer[2.0.0].

Expected behavior
Composer should not fail on that.
Environment (please complete the following information):

OS: macOS
PHP version: 8.1
Laravel: 9

Duplicated https://github.com/rialto-php/puphpeteer/issues/170


Compact: Attempts to set retention result in "bucket with type  is not supported"

Hi all,
What happened:
We are trying to set retention limits on Thanos Compact with the following:
/opt/thanos/0.29.0/thanos compact --retention.resolution-1h=365d --retention.resolution-raw=365d --retention.resolution-5m=365d
What you expected to happen:
Expected retention to set to keep files for 1 year max. I could not find any requirements in documentation for retention, so I'm at a loss as to why this isn't working.
Full logs to relevant components:
However it does not seem to be working and we're receiving the following error:
level=info ts=2023-01-23T05:20:18.305969012Z caller=factory.go:52 msg="loading bucket configuration" level=error ts=2023-01-23T05:20:18.306412051Z caller=main.go:135 err="bucket with type  is not supported\ngithub.com/thanos-io/objstore/client.NewBucket\n\t/go/pkg/mod/github.com/thanos-io/objstore@v0.0.0-20221006135717-79dcec7fe604/client/factory.go:84\nmain.runCompact\n\t/app/cmd/thanos/compact.go:205\nmain.registerCompact.func1\n\t/app/cmd/thanos/compact.go:94\nmain.main\n\t/app/cmd/thanos/main.go:133\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594\npreparing compact command failed\nmain.main\n\t/app/cmd/thanos/main.go:135\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594"
Object Storage Provider:
I have the following thanos-objstore.yaml config:
type: S3 config:   bucket: "test-thanos"   endpoint: "s3.us-west-2.amazonaws.com"   insecure: false   signature_version2: false   put_user_metadata:     "X-Amz-Acl": "bucket-owner-full-control"   http_config:     idle_conn_timeout: 0s
Environment:

Reproduced issue on Thanos version 0.23.1 and 0.29.0
Ubuntu 20.04 LTS
Linux ip-10-208-9-70 5.15.0-1023-aws #27~20.04.1-Ubuntu SMP Wed Oct 26 20:02:26 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

How did you specify the config file for compactor? I don't think it is a bug, seems like an issue in your configuration file or how you passed the config file

How did you specify the config file for compactor? I don't think it is a bug, seems like an issue in your configuration file or how you passed the config file

Compact configuration is supplied under Object Storage Provider. Apologies, formatting isn't great.


add teams lifecycle

closes https://github.com/MicrosoftDocs/OfficeDocs-SkypeForBusiness/issues/8027

@JohanFreelancer9 Copy editing is needed for this PR. Thanks.

@SerdarSoysal Thank you for submitting feedback and contributing to the docs. We are currently investigating this.

@MikePlumleyMSFT Mike, can you help me getting this merged? Thank you


Use crk.altlabel.tsv to specify relabeling options?

Use initial rows in crk.altlabel.tsv to specify what relabeling options are made available, and what they will be named.

First row gives internal names for the various relabeling options.
Second row (SELECTION-LABEL) gives the names that iDict will use for showing relabling options.
Third row (SHOW-LABEL) will indicate whether the corresponding relabeling option will be shown (True) or not (False) in iDict.

E.g. for itwêwina, crk.altlabel.tsv contains the following first three rows, which already would support this modification:

ROW/COLUMN
1
2
3
4
5
6

1
FST TAG
LINGUISTIC (SHORT)
LINGUISTIC (LONG)
ENGLISH
NÊHIYAWÊWIN
EMOJI

2
SELECTION-LABEL
short linguistic labels
long linguistic labels
plain English labels
nêhiyawêwin labels
emojis

3
SHOW-LABEL
True
True
True
True
False

I will NOT implement this. I could be convinced to add configuration using a dedicated configuration file such as a .toml. However, I am not creating a mini-format in crk.altlable.tsv.

Configurability is my primary goal. So, ok, using .toml, would something like the following work:
[relabeling-1]
altlabelColumn = 1
morphodictLabel = "FST-style tags"
morphodictShow = false
morphodictOrder = 0

[relabeling-2]
altlabelColumn = 2
morphodictLabel = "short linguistic labels"
morphodictShow = true
morphodictOrder = 3

[relabeling-3]
altlabelColumn = 3
morphodictLabel = "long linguistic labels"
morphodictShow = true
morphodictOrder = 4

[relabeling-4]
altlabelColumn = 4
morphodictLabel = "plain English labels"
morphodictShow = true
morphodictOrder = 1

[relabeling-5]
altlabelColumn = 4
morphodictLabel = "nêhiyawêwin labels"
morphodictShow = true
morphodictOrder = 2

[relabeling-6]
altlabelColumn = 6
morphodictLabel = "emojis"
morphodictShow = false
morphodictOrder = 0

I don’t think we need this just yet, but when we do get to it, if I were to
apply the mallet of simplicity here, I would propose:

Labels come from the first row of the TSV file.
Relabelling order can come from the order of the columns in the TSV
file too.
Let’s not have multiple names for the same thing. Instead of supporting
two labels “Linguistic (short)” and “short linguistic labels”, pick one
and stick to it. That is, Internal name = external name because Python
strings and dict keys do support spaces.

This leaves only one really configurable thing, which is whether a column
should be shown in the UI or not. I propose adding this as a setting in
settings.py, where it could look like this:
# Which relabellings to show in the UI. These are opt-in so that people
# can add columns to the TSV file for testing without having them
# immediately show up on the web.
#
# It is a set, not a list, because the actual order comes from the .tsv
# file.
MORPHODICT_SHOWN_RELABELINGS = {
    "Linguistic (short)",
    "😎",
    …
}

Presumably admins could see all the relabelling options, with something to
indicate which are not yet public.


TypeError: the JSON object must be str, not 'bytes

Hi,
Thanks for the amazing repo.
I tried to use the given script to predict trees on custom image as follows:
import matplotlib.pyplot as plt
from deepforest import deepforest
from deepforest import get_data

test_model = deepforest.deepforest()
test_model.use_release()

#predict image
image_path = get_data("/home/bruno/python_scripts/detect_tree_sat/2019_YELL_2_528000_4978000_image_crop2.tiff")
image = test_model.predict_image(image_path = image_path)

#Show image, matplotlib expects RGB channel order, but keras-retinanet predicts in BGR
plt.imshow(image[...,::-1])
plt.show()

The image is:

I got the following error:

headers={'Accept': 'application/vnd.github.v3+json'},

File "/usr/lib/python3.5/json/init.py", line 312, in loads
s.class.name))
TypeError: the JSON object must be str, not 'bytes'

Has anyone had the same error?
Thanks,
Bruno

Hi Bruno, just to document for others. What made you close the issue?
How was install? I'm currently reworking all the install whls to make it
easier. What operating system are you on?
On Wed, Jan 8, 2020 at 7:34 AM Bruno Santos<EMAIL_ADDRESS>wrote:

Closed #64 https://github.com/weecology/DeepForest/issues/64.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/weecology/DeepForest/issues/64?email_source=notifications&email_token=AAJHBLHVV5DVWLHINSZJTVDQ4XXB5A5CNFSM4KEJR63KYY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOV266HZA#event-2931680228,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJHBLC6WZS4XOVQ2BZZ4K3Q4XXB5ANCNFSM4KEJR63A
.

--
Ben Weinstein, Ph.D.
Postdoctoral Fellow
University of Florida
http://benweinstein.weebly.com/

Hi Ben,
I closed the issue because I sorted it out my issue.
I installed from the source like this:
python setup.py install
I am working on Ubuntu 16.04.
Btw, I change use_release utilities.py the following,
_json = json.loads(urllib.request.urlopen(urllib.request.Request(
                'https://api.github.com/repos/Weecology/DeepForest/releases/latest',
            headers={'Accept': 'application/vnd.github.v3+json'},
             )).read().decode('utf-8'))
        asset = _json['assets'][0]

Thanks,
Bruno

What version of json package are you using on Ubuntu? I cannot replicate this error on osx
_json = json.loads(urllib.request.urlopen(urllib.request.Request(
        'https://api.github.com/repos/Weecology/DeepForest/releases/latest',
    headers={'Accept': 'application/vnd.github.v3+json'},
     )).read())

_json
{'url': 'https://api.github.com/repos/weecology/DeepForest/releases/21747526', 'assets_url': 'https://api.github.com/repos/weecology/DeepForest/releases/21747526/assets', 'upload_url':

3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 13:42:17)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)]
Python Type "help", "copyright", "credits" or "license" for more information.
import json
json.__version__
'2.0.9'

This looks like a known error on python3.5, which I thought was a touch outdated. I didn't build a .whl for linux for python3.5 for ubuntu. Would it be a pain to say, python version must be >3.5 for linux?
https://stackoverflow.com/questions/6862770/let-json-object-accept-bytes-or-let-urlopen-output-strings


Add Boiko School domain

https://boiko.top/ is the official website.
You can also check the information provided by https://kh.isuo.org/schools/view/id/11711.

@aquamarine121 First stage of verification passed, official domain is registry.edbo.gov.ua as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision. Proof of domain ownership: https://registry.edbo.gov.ua/institution/145941/

@aquamarine121 Thank you for your patience while waiting for a review and processing your request. Pull request merged.


[Do not merge] Adding templates to handle error responses

Initial PR

Only addresses default templates (not azure templates)
Needs stuff from the client runtime (Everything HttpRestException related)
Addresses cases when
No error model is defined
Error model is defined as a primitive type (HttpRestException<T>)
Error model is well defined

Stuff remaining

Azure templates
PRs for ClientRuntime and modeler
Reviewing if HttpRestException<T> for primitive types is a good idea?

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

# 🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged


[BUG] maintainers missing in codeowners

Describe the bug
Since https://github.com/opensearch-project/OpenSearch/pull/9173, @abbashus and @adnapibar were re-added to MAINTAINERS, but not to CODEOWNERS. Please re-add so that they can be notified on PRs as code reviewers.

Seems like that's not correct.

Seems like that's not correct.

Hmm... is that a permissions issue on the repo?
They both exist as GitHub users:

https://github.com/adnapibar
https://github.com/tlfeng

How is write access to the repository managed? I assume a project (or repo?) admin is able to manage that. As I'm neither, I don't know who does or doesn't have write access.

I talked with @prudhvigodithi  -- I think the permissions were changed?

@tlfeng @adnapibar I've sent invitations to rejoin as maintainers, sorry for the misaligned images:

@prudhvigodithi can you investigate audit and see if there was an operation error or if this was related to another change? Maybe this had to do with them leaving the OpenSearch-Project organization.

If these folks left the org they need to be readded as external collaborators and permissions reinstanted, or revert https://github.com/opensearch-project/OpenSearch/pull/9173 and https://github.com/opensearch-project/OpenSearch/pull/10681.

Hey @peternied I ran the audit again, the only drift now for OpenSearch repo is tlfeng to be part of the collaborators, should be solved once @tlfeng accept the invite as maintainers.

@peternied @prudhvigodithi I accepted the invitation, and thank you for your effort. Sorry for the late reply!

Thanks @tlfeng I can see the OpenSearch repo is complaint now, the collaborators, code owners and maintainers are in sync now.
@dblock can we proceed to close this issue?
Thank you

Looks like https://github.com/opensearch-project/OpenSearch/blob/main/.github/CODEOWNERS is valid now.


Fix deepFreezeAndThrowOnMutationInDev error caused by objects without a prototype

This PR fixes a bug in deepFreezeAndThrowOnMutationInDev which did not take into account that objects passed to it may have been created with Object.create(null) and thus may not have a prototype. Such objects don't have the methods hasOwnProperty, __defineGetter__, or __defineSetter__ on the instance.
I ran into an unrecoverable error in React Native when passing this type of object across the bridge because deepFreezeAndThrowOnMutationInDev attempts to call object.hasOwnProperty(key), object.__defineGetter__ and object__defineSetter__ on objects passed to it. But my object instance does not have these prototype methods.
Changes:

Modified calls to object.hasOwnProperty(key) to use Object.prototype.hasOwnProperty.call(object, key) (Per ESLint rule here)
Modified calls to deprecated methods object.__defineGetter__ and object.__defineSetter__ to instead use Object.defineProperty to define get and set methods on the object. (Per guidance on MDN)
Added a new test to deepFreezeAndThrowOnMutationInDev-test to verify the fix.

Test Plan
I tried to create a reproducible example to post to Snack by passing prototype-less objects to a Text component, in various ways, but they appear to be converted to plain objects before crossing the bridge and therefore they do not throw an error.
However, I was able to create a new test to reproduce the issue. I added the following test to deepFreezeAndThrowOnMutationInDev-test:
it('should not throw on object without prototype', () => {
    __DEV__ = true;
    var o = Object.create(null);
    o.key = 'Value';
    expect(() => deepFreezeAndThrowOnMutationInDev(o)).not.toThrow();
  });

The changes in this PR include this new test.
ESLint test produced 0 Errors and no change in Warning count (1014)
Related PRs
N/A
Other areas with possibly the same issue:
https://github.com/facebook/react-native/blob/c6b96c0df789717d53ec520ad28ba0ae00db6ec2/Libraries/vendor/core/mergeInto.js#L50
https://github.com/facebook/react-native/blob/8dc3ba0444c94d9bbb66295b5af885bff9b9cd34/Libraries/ReactNative/requireNativeComponent.js#L134
Release Notes
[GENERAL] [BUGFIX] [Libraries/Utilities/deepFreezeAndThrowOnMutationInDev] -Fix for compatibility with objects without a prototype.

Thank you for contributing, this PR looks great. I'm especially happy for the additional test!
We don't accept PRs against release branches. I tried to point this towards master but it won't rebase cleanly. Can you retarget this to the master branch?

Ah, I see https://github.com/facebook/react-native/pull/19598. I'm going to close this in favor of that.

@grabbou , can you merge this to 0.49 and publish?


Pod integration fail

Hi, i am getting the next error when execute "pod install"..
fatal error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/lipo: /var/folders/xf/b7gq41hs2jv_4243r8t57l4m0000gn/T/tmp.mZ7MzT8O/.build/iphoneos/lib/libsecp256k1.a and /var/folders/xf/b7gq41hs2jv_4243r8t57l4m0000gn/T/tmp.mZ7MzT8O/.build/iphonesimulator/lib/libsecp256k1.a have the same architectures (i386) and can't be in the same fat output file
Found a possible solution adding this 2 lines to build_secp256k1.sh file but it doesn't work...
// lines 23 and 24.
(cd src && make distclean > /dev/null)
(cd src && ./autogen.sh)
How can I fix it?
Thanks

Hey man, got solution on this?

Same Issue here.
Do you solved it?

No, i didnt try again, sorry.. Maybe SPM works better than cocoapods ( in my case, I prefere SPM )


To harmonize scRNA-Seq reference to align spatial data

Hi,
I am using Tangram to align scRNA-Seq reference onto spatial data. My scRNA-Seq reference is a collection of samples from different datasets. As a result, the raw gene expression of the scRNA-Seq reference showed a batch effect due to sample sources. May I ask if you have any suggestions to harmonize the batch effect when aligning to spatial data? Thanks.

Yes - map using cluster mode and pass as clusters the cell types (irrespective of dataset they come from).
See main github page, or tutorial, for how to do that (it's really just one argument to the mapping function).
What cluster mode does: it takes all cells with same label (the cluster) and average the gene expression across the cluster. The resulting averaged cell is the only cell, representative of that cluster, to be mapped to space (and clearly that can be mapped onto different spots during one alignment). The reason why this is a good idea for you, is that by doing so you average out the batch effects as the cluster will presumably contain cells from different batches.
Let me know if that is not clear and if it helps.

Not the right metric: training scores merely indicate how well the model fit the data: you look at TEST scores to see which mapping works better. Indeed, it is perfectly normal that in cluster mode you have lower training scores because you are only mapping average cells therefore you have less selection (as opposed to map all cell profiles in the sc). That does not mean that mapping is less good.
About getting test scores: The strategy consists of comparing gene expression of the mapped single cells vs the spatial data across all genes except those used to train Tangram. If your spatial data are Visium, typically you train on 1k markers and all the other ~29k can be used as test genes.  The plot you are after is the one with blue dots containing sparsity of spatial data vs scores. You can automatically fit the dots with a red curve and get the resulting area under the curve (AUC). Comparing the AUCs of the two mappings is what you need (you can plot the images here and I'll help you).
You can follow the tutorial for getting test scores. LMK how that goes and if you are stuck write again.

Hi @lewlin, thank you so much for your detailed explanation. Yes, I am able to produce the test scores using LOOCV and plot the AUC images. Below is the AUC of the "cells" mode.

And the image of the "clusters" mode is,

The plot is generated for about 8,000 MERFISH cells of 73 marker genes. The AUC values are comparable between the two modes, and the "clusters" mode showed a slightly higher AUC. Given the similar AUC values, it seems that both modes are good to train full genes. To compare the two modes w.r.t. the cell type annotation by the mapping probabilities, I compared the predicted cell type labels (the deterministic label by the most scores) of two modes, and it turns out they have produced quite different annotations. For example,

In the above contingency table, the row annotation is by the "clusters" mode, and the column annotation is by the "cells" mode. The ARI of the contingency table is 0.167, i.e., two annotations are quite different. In this case, may I ask how do two modes produce quite different deterministic cell type annotations? and which annotation should I use for the downstream? Thank you so much.

There you go: now that's a good comparison.
Cluster mode is giving you a much better mapping (as it should, b/c of your settings): ignore cell mode mapping and just use cluster. In general, always use mapping with the highest test score. More comments:

difference in AUCs slight but actually is very significant (we have not worked out yet a way to disentangle prediction of basal gene expression vs relevant).
Do not trust the AUCs "too much' because the fitting we use is very crude: to compare mappings, it is helpful to overlay markers and eyeball the trends (although in your case cluster is more solid than single cell).

Hi @lewlin, that's great. Your comments are very helpful. Thank you so much for all your help.


Update SIKE_P434_R2 IANA value in preparation for spec change

Please note that while we are transitioning from travis-ci to AWS CodeBuld, some tests are run on each platform. Non-AWS contributors will temporarily be unable to see CodeBuild results. We apologize for the inconvenience.
Issue # (if available):
Description of changes: Updates the SIKE_P434_R2 from 16 to 19, in anticipation of changes to the draft spec. (Current draft spec https://tools.ietf.org/html/draft-campagna-tls-bike-sike-hybrid-02)
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@0ee0b09). Click here to learn what that means.
The diff coverage is n/a.


Possible future issue with url structure

Now one page is created for one day. If this calendar is used for let's say 3 years there might be 1000 pages in the same root.
https://forum.getkirby.com/t/server-response-time/279

After lots of use cases with Kirby I can say that bottlenecks happen when you put a lot (like a couple hundred or even more than thousand) subpages in a single folder. That is when scanning the list of folders gets messy. Other than that Kirby can handle A LOT of content if it's structured more tree-ish.

A possible solution to it would be like this:
[your-calendar-page]/y-2017/m-03/d-14
It would be even nicer to have:
[your-calendar-page]/2017/03/14 but Kirby does not like pages starting with a number.
As a bonus I could get all the events from a month without a problem. It would be harder with the current solution.
Thoughts?

You are right, it's a performance issue to which I have already thought.
In the alpha version of calendar board there was an year folder between [your-calendar-page] and day page. I think it's enough to solve the issue, but I didn't add it in beta version because of this:

I didn't find (yet) an elegant solution to visualize year and month pages in the panel.
List of months and list of day? I don't like that, I would prefer to keep the plugin as clean as possible, maybe hiding those breadcrumbs.

I don't think it's possible with the current version of the panel. You can add CSS to the panel but I don't see how it could match custom breadcrumb sub pages.
There is javascript but it can't be used outside the field.
Maybe the forum have some ideas.
I agree that the simplicity of the plugin is perfect as it is. I also love that it's possible to add whatever stuff to the day structure field. Genius.

Another solution I thought about was to create a kind of auto-backup that every X years (or every X days created) moves (attention not deletes) a part of the folders outside the directory worlkflow.
Managing Calendar Board future events and activities past objects maybe aren't so important.

For daily usage the current version works best. For past events the year/month/day version works best.
Maybe have two setups? Always keep real data in a year/month/day folder strucure. Always keep new fresh data in the current setup until the data is 2 month old or so.
It feels a little risky. What if we need to look back at some data?
I don't have perfect idea here.

Another hackish solution could be to have a redirect field that redirect the pages that are year or month to the calendar. That way you would not need to design those pages.
It will not remove the main issue tho.

You can hide useless breadcrumbs (year-xxxx, month-xxxx,...) using:
ul.breadcrumb-list > li > a[title^="year-"]{display:none}
Do I like it? Not so much. You have insert this rule in a panel css and call it through config.php resulting with "spaghettify" the plugin.

Ahh, yes, that would probably work.
Do I like it? In fact I think it's the best workaround so far, because the result of it ends up being exactly what we want, which is to not show the whole hierarchy in the breadcrumb.
I know that the panel will get alot of love this year by Bastian and I hope it means controlling it. Maybe it would be possible for blueprint to set their own breadcrumbs yes I'm deaming now.
As I see it you have 3 choices:

Do nothing and wait for the future. This means that everyone that uses this plugin will need some conversion process when the structure is changed. (I know you don't have a user storm right now)
Change the structure and use the css "hack" and add a real solution later. Then a conversion process will not be needed in the future.
Ask the forum for a solution that no one of us have thought of yet.

Changed structure introducing year folder.
I've also added instruction to hide years breadcrumb.
I think it's enough for now.

It works, at least at the backend. Then it will work on the frontend as well. The content structure works as well. Close this task? Or are you not happy with this solution?


Assets #7 Create rpc call for  asset creation transaction

Create assetTx create rpc call:
Inputs:

Asset metadata json

Payment address: optional to specify the address paying for new asset fees. If not provided use wallet unspent inputs

Outputs: Asset metadata json with assetTx (see prior Assets issues) as one of the fields.

Requirements:
The transaction created needs to have fields for all of the above mentioned points
Ensure full backwards and forwards compatibility post changes. Including syncs from scratch and continuation post both graceful shutdowns and ungraceful shutdowns!
Create appropriate unit and possibly fuzz or other tests to ensure compliance and avoid unforeseen issues.

Live on testnet now!


Be able to configure background color of the canvas

Now there is no background color which makes it white. But I would like to be able 5o specify the color

Done https://github.com/JordyMoos/elm-pixel-boulder-game/pull/40/files


Testing GH Tab size

module.exports = {
	tabThere: 1,
  twoSpacesThere: true,
}

Managing your tab size rendering preference

It seems to me that that setting is applied only to the source code and not the code blocks in the issues.


loadFromPOM download jar with ivy2, not maven

I found %%loadFromPOM download jar with ivy2, which make it can not reused the jar in ~/.m2 directly.
is there any way to reuse the jar in ~/m2 dierctly

it reuse jar in ~/.ivy2
and you can try my custom gradle magic to load dependency, it reuse jar in ~/.gradle
https://github.com/jimmybow/docker-java-notebook/blob/master/data/jupyter/custom_IJava/custom.jshell


CP VC changes to fpgadiag 1.4.0 branch

mactest & fpgastats tools used in PAC card D5005.
CP changes to support network card testing.

It looks like the previous PR also added the fvlbypass option to fpgadiag, which is not included in this version.


F# support to AssemblyInfo

Support for AssemblyInfo task to create F# files.
Uses (and merges to the dll) FSharp.Compiler.CodeDom that is in public domain.

This change is

Any chance that this would get merged?

I'm might be repeating myself, but is there a reason why this couldn't be merged in?


Slim down Sanity moving some tests to Tier2 (execution time > 15 mins)

After agreement on Google Chat, this PR is moving some tests from Sanity/Tier1 to Tier2 because they need more than 15 mins to execute

As agreed in the chat, we'll merge this PR with Tier2 in order to release ods-ci 1.13, but we'll discuss in the automation meeting if Tier3 is more appropriate


Status LED option - not matching the Status Light setting from YI App

Hello,
In both 0.16 and 0.17 sw - the status LED option in Camera Options does not reflect the current Status Light in YI App.
Thanks for the amazing software!

Yes, it's correct.
If you save the option inside the app, the web page doesn't reflect the change.
Because I can't read the internal configuration in binary format.
But if you change the web setting, the app will reflect it.


improve context sharing for beforeEach/afterEach

re https://github.com/sindresorhus/ava/pull/106#issuecomment-150966370

// @vdemedes @schnittstabil

I think it's good to go!

There is an unnecessary(?) this.context = {} within the Test constructor in lib/test.js remaining.

@schnittstabil I didn't put it there, but I assume it's so that even tests without hooks can have a context, for consistency. @vdemedes Correct?

@sindresorhus Yep, correct!


Performance decrease since 0.6.x

We're using kubeconform v0.4.13 in our CI at https://gerrit.wikimedia.org/g/operations/deployment-charts/ to check a bunch of charts and "deployments". Today I've tried upgrading to v0.6.1 and realized that a complete check of all charts against two k8s versions went from ~20sec to ~70sec (we're running multiple processes in parallel, so those numbers are probably not very representative).
As v0.5.0 still has the same performance than v0.4.13 I would suspect https://github.com/yannh/kubeconform/pull/168 to be the culprit.

Hum. 70s sounds like a loooong time - I run this on every commit on quite a few kubeconfigs and it takes a lot less than that... I've not noticed a performance decrease though, esp not in that order of magnitude. Anything special about your use case - very long files, very large amount of files, .. ? I did run some manual performance regression tests when updating the library and couldnt find a decrease in performance :( Maybe I should look at this again.

Sorry for getting back so late. I don't think there is anything very special about the single files it's just that we have many of them. We're checking charts (multiple renderings per chart with different values) as well as rendered helmfile templates for all of those charts. I'd say the absolute number is what drives the total runtime so high.
If it helps I could maybe find a particular case/input where this is observable.

@jayme-github I'm sorry about this regression for you but I will close this ticket :( I did quite a bit of performance testing when swapping libraries and did end up with really good performance with the new one. If you ever figure out if that's what caused the issue or if it was another commit let me know.  Best,

Sure, thanks for letting me know. I unfortunately never came around to putting some time into this. I might when we at some point have to move to >= 0.6 :) - cheers


Fix for issue 4 - improve perf of metadata queries on SQL Server

Added an interceptor which appends "OPTION (MERGE JOIN)" to the SQL for the metadata queries executed during Reverse Engineering. Added a quirk so users can turn this off if they want to. (I thought this would be best, but let me know if you'd prefer the quirk to work the other way around i.e. users having to explicitly set an AppSetting to turn this feature on).

Looks good with the changes that @divega suggested. I'll leave Diego to do the final sign-off.

All changed wanted by @divega are now fixed. Going ahead with the merge.

Merged into master with fb2695c2b.


[Skill Calc] Fletching Unf/Finished

Finished bows are listed, xp wise, as finishing the second half of it, like only if you strung an unf. Add a tick box or something to display full bows/crossbows as the full experience for making them.
Ex. Magic longs are 91.5 for unf, 91.5 for strung, 183 total. Checkbox would show Magic Longbow as 183xp.

You can select multiple actions by holding shift, and the total will appear as the first option. You can use that for stuff like bows.

I feel this can just be achieved by using the shift highlight for the two parts.:
Unstrung Magic Longbow
String Magic Longbow

Good points, I hadn't even thought of that. With the multi-action merger thing there doesn't need to be anything else.


Automate adding function name and test number to the test title

A function should be implemented which adds the function name and a test number to the title of the tests. Additionally it should add a comment before the test such that it is displayed in the TOC in RStudio.
For example
test_that("duration and unit variable are added", {
...
}

should be transformed to
#---- derive_vars_aage, test 1: duration and unit variable are added ----
test_that("derive_vars_aage, test 1: duration and unit variable are added", {
...
}

If the function name and test number are already in the title, it should be updated if necessary.
Please note this is quite a large task and good for someone familiar with the RStudio API.

Hi, this issue is ready for a review. I have also added an RStudio Addin. Considering that admiraldev is being created, maybe it would make sense to move this function and the Addin there? Anyway, let me know what you think in the referenced PR. Cheers!

Unfortunately, the description and the unit test guidance differ. In the description we have derive_vars_aage, test 1 while the unit test guidance require derive_vars_aage Test 1. We have some unit tests following the guidance but no unit tests following the description. Should we update the description?
@bms63 , @thomas-neitmann , what do you think?

I like the Test 1 more than test 1.

At the moment the function name is derived from the test file name. But we have test files which test more than one function.
I would propose that we use two levels. The first level is the function and the second one the test. For example
# derive_vars_aage ----
## duration and unit variable are added ----
test_that("derive_vars_aage, test 1: duration and unit variable are added", {
...
}

## duration without unit variable is added ----
test_that("derive_vars_aage, test 2: duration without unit variable is added", {
...
}

The function name is derived from the last
# <function name> ----

comment.
If there is no such comment, the function name is derived from the test file name and a comment for the function name is added.
@thomas-neitmann , @bms63 , what do you think?

From my perspective: when a developer sees that a test is failing, it is better to actually show the name derived from the filename - this is easier to navigate - you know immediately in which file the error is.
If the name is derived from the tested function but no such file exists, you would need to find in which file the test actually is.

I think the test file name is displayed anyway. For example:
Failure (test-derive_merged.R:76:3): derive_vars_merged: merge last value and flag matched by groups

So if we have the function name in the test description, we have both the test file name and the function name in the error message.

You beat me to it @bundfussr!

ok I see, then maybe only a test number would be enough? It is unique within the file.

So this 2 level headers may be much more complex task, compared to the original issue (and probably also very error prone since user-defined headers will get easily mixed with those that are automatically created).
I have fixed the PR and it seems that the current version is working - please have a look and let me know if you find anything else there.
If you would like to change scope of the issue (= change the function name and use multiple-level headers), I am not sure I would have the capacity for that. Definitely not during hackR, it would take much more time. Please let me know what you think and what you decide for.


[sign tx] add signing with secrets and ValidationResponseCard

Sign Transaction

Sign Transaction's Import and Overview view
Signing with secret keys enabled
We're not saving secret keys in store

note: waiting on design for the validation response's error state

Preview is available here:http://laboratory-pr839.previews.kube001.services.stellar-ops.com/

Preview is available here:http://laboratory-pr839.previews.kube001.services.stellar-ops.com/

Preview is available here:http://laboratory-pr839.previews.kube001.services.stellar-ops.com/

Preview is available here:http://laboratory-pr839.previews.kube001.services.stellar-ops.com/

Preview is available here:http://laboratory-pr839.previews.kube001.services.stellar-ops.com/

Preview is available here:http://laboratory-pr839.previews.kube001.services.stellar-ops.com/

Preview is available here:http://laboratory-pr839.previews.kube001.services.stellar-ops.com/


Animated model causing scene to crash

here is the model in question.
http://www.katsbits.com/files/md5/animated-md5_bob/bob_lamp_update.zip
This model worked fine in 3.0 but has many issues in 3.1.
The 1st issue I noticed is that the animation controls don't play. The model is divided into parts like head , body , lamp etc. In 3.0 I could play each animation individually. In 3.1 none of the animations play until I get to the last control "lamp_top" then it plays but it also plays all of the other animations at the same time.
I also noted the default master animation control is not loading like it did in 3.0.
The 2nd issue is when I add the model to the scene. After making changes to the model such a moving it I can no longer save my scene then the sdk will start to slow down. If I shut down the scene and reload it most of my models will be absent and I get the error "bone matrixes cannot be null"
If I deleted the model from the scene and reload the scene my other models will reappear.

Hey!
Good News:
The second one is already fixed but you'd have to wait for beta2 and since it's hacky we are currently thinking about implementing it differently.
If you need it asap, you have to build the engine from https://github.com/MeFisto94/jmonkeyengine/tree/MatParams (using gradlew assemble dist) and then use mvn install:install-file -U -Dfile=dist/lib/jme3-core.jar -DgroupId=org.jmonkeyengine -DartifactId=jme3-core -Dversion=3.1.0-beta1 -Dpackaging=jar
This however requires you to have a running linuxoid system with maven available (you actually setup a local maven repo) and placing mavenLocal() {} inside the repositories in our gradle.build of the sdk.
Then you can compile your own patched sdk using ./gradlew buildSdk.
For the first Issue: I also have that problem currently and try to fix it as well, however I can't recall having a default master animation control at all. Anyway we are already on it and trying our best :)

I should say default blender control. When you load the model in 3.0 you get an empty shell control attached to the root of spatial. It's the default blender control and isn't always empty depending on who set up the animation. To be franks it's not a big issues but the controls not playing is.
I'm in the process of porting a huge project over to 3.1 hopefully by time I'm done we'll have found every possibe bug in the SDK :>

Regarding to 1st issue:
Tested your model using xbuf blender exporter and it plays OK in sdk. I do not know why that problem occurs just when using blender importer. :confused:

Because XBuf Importer only adds one AnimControl and not one per Geometry

Strange !! :confused:
but it plays OK when I import it using ogre exporter and convert it to .j3o

Did you make sure to only start it on one mesh?
If so, could you please upload the ogre j3o and the blender j3o?
I tried it on my model but there I can't even use the ogre importer :(

Yep, I am sure.
here you can download it : https://drive.google.com/open?id=0B35eWZpi0dBoNktmcE5FNTR4ckU

So @Ali-RS this was only an issue with the blender importer, right?

IIRC yes, it was an issue with blender importer and other importers appear to work fine.

Closing as we have gltf now and in case it happens there as well, we can create a new issue.
Also it's most likely something for the engine anyway AND we have the new Animation System anyway.


npm run build fails "Cannot find module .\Playbook\build\build.js" in Windows 10

After doing a fresh clone of the repo in Windows 10, with current node versions 10.14.1 LTS and 11.3.0  the npm run build cmd fails with following messages:
PS C:\Users\lfreeman\testGitPlaybook> npm run build

<EMAIL_ADDRESS>build C:\Users\lfreeman\testGitPlaybook
node build/build.js; node build/build.search.js

internal/modules/cjs/loader.js:582
throw err;
^
Error: Cannot find module 'C:\Users\lfreeman\testGitPlaybook\build\build.js;'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:580:15)
at Function.Module._load (internal/modules/cjs/loader.js:506:25)
at Function.Module.runMain (internal/modules/cjs/loader.js:741:12)
at startup (internal/bootstrap/node.js:285:19)
at bootstrapNodeJSCore (internal/bootstrap/node.js:739:3)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>build: node build/build.js; node build/build.search.js
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR!      C:\Users\lfreeman\AppData\Roaming\npm-cache_logs\2018-12-05T15_03_40_671Z-debug.log

I have the same issue

Interestingly if one types the build script commands directly at the command prompt they work, i.e. node build/build.js. Must be some environmental thing.

@lecniers try replacing 'node build/build.js; node build/build.search.jswithnode build/build.js && node build/build.search.js`

thanks @anywhichway , updated package.json and pushed it. I cant test on windows. Assuming this fixes it, if not please reopen.


TS emits an error for BoolQuery type

TS expects a should property, despite it being an optional prop in BoolQueryConfig:
interface BoolQueryConfig {
    must?: Query | Query[];
    filter?: Query | Query[];
    should?: Query | Query[];
    must_not?: Query | Query[];
    minimum_should_match?: number | string;
    boost?: number;
}

Doesn't work:
const query: BoolQuery = {
      bool: {
        filter: { match_all: { boost: 1 } },
      },
 }

TS error:
Type '{ filter: { match_all: { boost: number; }; }; }' is not assignable to type 'BoolQueryConfig'.
  Types of property 'should' are incompatible.
    Type 'Assertion' is not assignable to type 'Query | Query[] | undefined'.
      Type 'Assertion' is missing the following properties from type 'Query[]': pop, push, concat, join, and 25 more.

Works:
    const query: BoolQuery = {
      bool: {
        should: undefined,
        filter: { match_all: { boost: 1 } },
      },
    }

TS version: 4.0.3

Hi, guys... I was facing the same error and noticed this happens when I ran this command  npm i -D @types/chai , so I uninstalled @types/chai in order to fix the problem, and works again.

Ah, nice find @mgonzalez121690, yeah if you take a look at the very bottom of https://unpkg.com/browse/<EMAIL_ADDRESS>it looks like the chai types are adding a should property to literally all objects via the Object interface.
I'm not sure there's anything we can do about that here, chai adding that property to all object types is very broad.
Most likely if you still want to include @types/chai in your project, you would want to adjust the types property in your tsconfig.json to not include @types/chai for your main app code that uses elastic-ts, and have a separate tsconfig.json that you use for your tests. Something like:
tsconfig.json
{
  "compilerOptions": {
    "types": [] // or ["node"] etc, basically a list of global types to evaluate
  }
}

tsconfig.test.json
{
  "compilerOptions": {
    "types": ["chai"] // or ["node", "chai"] etc
  }
}

Yeah, it's a case with Chai.Assertion polluting the global Object interface. You can neutralize the effect in a declaration file, like so:
// global.d.ts
interface Object {
  should?: any
}