Static all loggers

Switch to use static loggers (so they can be used by static methods and also so we don't instantiate a bunch of them).

Done in rdf4j storage

Unfortunately, this is giving me grief in Eclipse. I'm not sure why yet but running compliance tests from my Eclipse now regularly rails with this kind of error:
java.lang.IllegalAccessError: class org.eclipse.rdf4j.sail.memory.MemoryStore tried to access private field org.eclipse.rdf4j.sail.helpers.AbstractSail.logger (org.eclipse.rdf4j.sail.memory.MemoryStore and org.eclipse.rdf4j.sail.helpers.AbstractSail are in unnamed module of loader 'app')
	at org.eclipse.rdf4j.sail.memory.MemoryStore.initializeInternal(MemoryStore.java:245)
	at org.eclipse.rdf4j.sail.helpers.AbstractSail.init(AbstractSail.java:187)
	at org.eclipse.rdf4j.sail.helpers.AbstractSail.initialize(AbstractSail.java:176)
	at org.eclipse.rdf4j.sail.helpers.SailWrapper.initialize(SailWrapper.java:96)
	at org.eclipse.rdf4j.sail.inferencer.fc.SchemaCachingRDFSInferencer.initialize(SchemaCachingRDFSInferencer.java:211)
	at org.eclipse.rdf4j.sail.Sail.init(Sail.java:60)

Funnily enough I'm not getting this on the command line or when running in IntelliJ, so it might be something specific to my Eclipse environment. However, there's also the fact that this enhancement comes at a cost, namely that we lose the option of allowing different applications in the same application context to use their own logger (see http://www.slf4j.org/faq.html#declared_static).
I wonder if the (fairly minimal) overhead reduction is worth it.

Being able to log from a static context is nice too.

Being able to log from a static context is nice too.

True of course. There's no rush with this but I'll try and take a closer look at what's going on with the classloader in Eclipse - it might be a build path config issue rather than an actual problem with the code. I also want to take a closer look at static vs instance on a case by case basis - for example in the AbstractSail there are no static methods, plus it is intended to be inherited from, so a different approach might be useful there.


Move placeholder behind caret

Hey again @wobsoriano, hope you're doing well.
I found that the placeholder appears on top of the caret. By swapping the order of these slots the placeholder appears behind the caret.

Before
After

Let me know if I can improve or if you'd like these PRs in a different format. I expect there may be more like this if that's cool with you.

This looks good to me. Thank you!


Prefer entryPoint for Summarizer

Description
Makes the default entryPoint for the summarizer clients the summarizer. This reduces the coupling of the summarizer on the request pattern. Once this change saturates, we can entirely drop the request pattern for the summarizer.
Also removes some long deprecated code, and an unused and non-functional class(SummarizerHandle)

@alexvy86  i won't be around next week, so if you approve feel free to merge it as well, or take it over and make any modifications, as it would be good to get this in before the next major.

@alexvy86 i won't be around next week, so if you approve feel free to merge it as well, or take it over and make any modifications, as it would be good to get this in before the next major.

Sure thing.

Closing in favor of #14681


[Snyk] Security upgrade ubuntu from xenial to xenial-20210416

This PR was automatically created by Snyk using the credentials of a real user.![snyk-top-banner](https://github.com/andygongea/OWASP-Benchmark/assets/818805/c518c423-16fe-447e-b67f-ad5a49b5d123)
Snyk has created this PR to fix 2 vulnerabilities in the dockerfile dependencies of this project.
Keeping your Docker base image up-to-date means you’ll benefit from security fixes in the latest version of your chosen image.
Snyk changed the following file(s):

Dockerfile

We recommend upgrading to ubuntu:xenial-20210416, as this image has only 93 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.
Vulnerabilities that will be fixed with an upgrade:

Issue
Score

Allocation of Resources Without Limits or Throttling SNYK-UBUNTU1604-SYSTEMD-1320131
  614

Allocation of Resources Without Limits or Throttling SNYK-UBUNTU1604-SYSTEMD-1320131
  614

Allocation of Resources Without Limits or Throttling SNYK-UBUNTU1604-SYSTEMD-1320131
  614

Allocation of Resources Without Limits or Throttling SNYK-UBUNTU1604-SYSTEMD-1320131
  614

Improper Check for Dropped Privileges SNYK-UBUNTU1604-BASH-542609
  536

[!IMPORTANT]

Check the changes in this PR to ensure they won't cause issues with your project.
Max score is 1000. Note that the real score may have changed since the PR was raised.
This PR was automatically created by Snyk using the credentials of a real user.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
📜 Customise PR templates
🛠 Adjust project settings
📚 Read about Snyk's upgrade logic

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Allocation of Resources Without Limits or Throttling

Olá!
Obrigado por enviar o pull request para corrigir as vulnerabilidades no Dockerfile. No entanto, sua descrição não atende aos critérios estabelecidos para uma boa descrição de pull request.
Você não explicou claramente o motivo do pull request e não incluiu um passo a passo para testar as mudanças propostas. É importante fornecer um contexto histórico, justificativa ou racional da mudança, além de instruções claras de teste.
Por favor, revise sua descrição e adicione essas informações para melhorar a qualidade do seu pull request. Se precisar de ajuda, estou à disposição.
Obrigado pela colaboração!
Esse comentário foi gerado por inteligência artificial


added support for PoD and CoA packet codes

I have added new packet codes (for Packet-of-Disconnect and Change-of-Authorization)

ops, accidentally pushed 2nd patch with vendors.


Fonts need to be more fully specified

Default Myriad Pro font is provided only as a TTF. While most browsers can handle the web font specification, Internet Explorer expects appropriate MIME type configuration. Without that the site reverts to Calibri, Arial, 'sans serif', rather than an alternative Myriad Pro web-font filetype. Properly, there should be access to EOT, WOFF, WOFF2, TTF, SVG font file types if possible.
As it is, a deployment of this skeleton to government application sites will otherwise require MIME type entries for over 90 applications for the Transportation Ministry alone. Can this be addressed at source instead?
ERROR Addressed:
CSS3114: @font-face failed OpenType embedding permission check. Permission must be Installable.

Hey Gavin!  I'll ask our graphics department about those other font formats. As for the MIME Type configuration, as per your point, this template is generic on purpose for scalability; trying to keep it as minimal as possible as other ministries won't want the overhead and more control specific to their builds.  For example, why the majority of the JS has been removed...

Justin, thanks.
This has really nothing to do with JS, It should just be a few more lines to the main.css and inclusion of the alternative web font formats for Myriad Web Pro.   Also, keep in mind that IE does not support TTF web fonts prior to version 9.  We would do this ourselves but this font is licensed from Adobe so we can't acquire those files ourselves.  IE asks the server for TTF files using certain parameters different from other browsers, so the workaround would be a MIME entry to support that request.  That MIME entry would need to be applied for all applications using this package.
Rather than a workaround that is imposed on all consumers of this package, let's complete the work and adhere to full standards by cleaning this up.  See https://www.w3.org/Fonts/

Hey Gavin!
I've uploaded the other font types made available to me that we have licence to use.  Do you have any particular preference for the CSS declarations?  As a community resource, I'll leave it up to those that fork this one to decide.  Here's some reading I looked into with a quick google.
Cheers,
J

I’ll refresh my fork and let our devs have a say.
Gavin McLeod
250-896-4437 (Cell)
778-698-3014 (Desk, Lync)

User Experience / Information Architect
Enterprise Architecture Services
Information Management Branch
Ministry of Transportation and Infrastructure
<EMAIL_ADDRESS>From: Justin Hewitt<EMAIL_ADDRESS>Sent: Wednesday, March 8, 2017 4:35 PM
To: bcgov-c/Gov-2.0-Bootstrap-Skeleton
Cc: Gavin McLeod; Author
Subject: Re: [bcgov-c/Gov-2.0-Bootstrap-Skeleton] Fonts need to be more fully specified (#3)
Hey Gavin!
I've uploaded the other font types made available to me that we have licence to use. Do you have any particular preference for the CSS declarations? As a community resource, I'll leave it up to those that fork this one to decide. Here's some readinghttps://css-tricks.com/snippets/css/using-font-face/ I looked into with a quick google.
Cheers,
J
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/bcgov-c/Gov-2.0-Bootstrap-Skeleton/issues/3#issuecomment-285216476, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AC3TUGMYhHX42wxuuFPLecsbPAshuMbVks5rj0k2gaJpZM4MVzso.


fix: allow operationId in handler.yml

#60  👈 (View in Graphite)
main

This stack of pull requests is managed by Graphite. Learn more about stacking.


Change in httpFilter

Fixes #
My proposition related to issue #870

@brahimbim Thank. I have a few questions:

If I have some connection with the user, for example, comments -> users, will I be able to sort by his balance? And filtering by one-time password recovery code?
I added a couple of tests in order to make sure that filtering and sorting works by json, could you check this by making a rebase master?

Why was this not Merged? Are there any issues?


Add setAjaxHeaders method to Viewer and TiledImage

See #1748 for background.

I haven't yet tested this at all.
There is a breaking change for those who directly use new TiledImage(...) (instead of the recommended Viewer.open(...) or Viewer.addTiledImage(...)) and set ajaxHeaders options for both Viewer and TiledImage. Previously, new TiledImage(options) didn't merge options.ajaxHeaders with Viewer.ajaxHeaders, but now it does.

Unless you object, I think this PR is ready to be merged.


Logging in Debug causes Bus Endpoint to Fail Validation when Consumer Definitions interact with IReceiveEndpointConfigurator

Is this a bug report?
Yes.
Can you also reproduce the problem with the latest version?
Unsure, definitely found on 6.3.2 as can't upgrade yet to 7+.
Environment

Operating system: Windows 10
Visual Studio version: 2019 Enterprise
Dotnet version: .Net Core 2.1

Steps to Reproduce

Configure basic RabbitMq Bus
Add console logging provider
Add 1 Consumer and Definition which modifies a receive endpoint properties (Durable is triggering it for me)
Create a bus instance, and Boom!

Expected Behavior
It works as normal.
Actual Behavior
Configuration validation throws following exception.
The service bus was not properly configured: [Failure] [EndpointName].Durable was modified after being used
This seems to be triggered by how change tracking is done on endpoint configuration and how logging seems to exercise it.
From quick inspection property change tracking triggers validation errors if any setter is executed after property InputAdrress getter is called, but logging at 'Debug' seems to be trigger this behavior earlier than expected.
Reproducible Demo
Coming soon.

I want to say this is fixed in v7, I seem to remember seeing it and fixing it. If you can reproduce with the latest v7 release, let me know.

Appreciate your quick response.
Do you recall which direction you went about to fix it?
Going to have a look at the code anyway.
If the fix can safely be isolated, would you be willing to entertain a PR to 6.3.2?
Doing a migration and RabbitMq.Client version which other components depend is preventing me from going straight to 7+.

I don't update older versions once a major version has been released.

This was the commit that moved the log message: https://github.com/MassTransit/MassTransit/commit/3ab23e6dd72c05b37766b68acb8f0b5315bee27d#diff-bc08ba6e1b41c8037ccaf38546fa32cffc392d6f9260814b8d9ba64c751a7b7e

Thanks for the info.
For now going to run with a logging filter the makes sure MassTransit category doesn't run with LogLevel bellow Information as a guardrail to avoid the issue.
In future will consider upgrading MassTransit to version 7+.
From my perspective, it this ticket can be closed.


Common atmos aliases

Describe the Feature
For each of these cheatsheet commands, an alias can be used
https://github.com/cloudposse/atmos/blob/master/website/docs/cli/cheatsheet.md
e.g.

Command
Potential Alias

atmos help
ahp

atmos version
av

atmos completion
ac

atmos describe affected
ada

atmos describe component
adcm

atmos describe config
adcn

atmos describe dependents
add

atmos describe stacks
ads

atmos terraform
at

atmos terraform clean
atc

atmos terraform deploy
atd

atmos terraform generate backend
atgb

atmos terraform generate backends
atgbs

atmos terraform generate varfile
atgv

atmos terraform generate varfiles
atgvs

atmos terraform shell
ats

atmos terraform workspace
atw

atmos helmfile
ah

atmos helmfile generate varfile
ahgv

atmos validate component
avc

atmos validate stacks
avs

atmos vendor pull
avp

atmos workflow
aw

atmos aws eks update-kubeconfig
aaeuk

atmos atlantis generate repo-config
aagrc

Expected Behavior
Install aliases using an atmos command
Or
A zsh plugin that creates all the aliases
Use Case
Less verbose commands
Describe Ideal Solution
See Expected Behavior
Alternatives Considered
Creating my own aliases or my own zsh plugin
Additional Context
N/A

Thanks for considering @aknysh. I meant shell aliases, not golang cobra aliases, but that would be a nifty feature too.
E.g. This works for me
alias aagrc="atmos atlantis generate repo-config"

I use a few of these locally but I've created the aliases manually. I'd rather use a zsh plug-in like the one for terraform which populates my shell aliases. It also creates new aliases for me when terraform releases a new subcommand.
https://github.com/ohmyzsh/ohmyzsh/tree/master/plugins/terraform

We discussed internally and are leaning towards introducing command aliases via the atmos.yaml so individuals can customize as they see fit. I think it will be too hard to agree on a convention for aliasing that will meet everyone's requirements.
e.g.
In the atmos.yaml, you would add:
aliases:
  tp: terraform plan
  ta: terraform apply
  agrc: atlantis generate repo-config

Then to use the aliases, you would run atmos tp as a shortcut for atmos terraform plan

[!TIP]
Use Shell Aliases
In your shell, you can do alias a=atmos, then run a tp as a shortcut for atmos terraform plan

Awesome, tyvm


Create developer page (#263)

First try to port http://elementaryos.org/developer

[ ] We'll need to link this page in the header template. Probably next to "Get Involved"?
[ ] Can we set the background color/image/etc at html, body in developer.css? That way we'll get the "transparent" navbar. CSS changes in master should fix the footer color if you'd like to merge master in ;)
[ ] Can we set the links to the same as their currently set on the main/old site? (We can just reference main/old site for now for the sake of making things work).
[ ] The left dashed border looks a little funky on mobile. Can we add a media query that only sets that left border width when the links are actually to the right? (Looks like that happens at about 1200px)

Otherwise, great port!

The styles in your media query are backwards ;p

[ ] there should be a line break between html, body in your css.
[ ] Since you're only using 100, probably don't need to load other weights of Raleway


Can't access load balancer in UI, but can via compose

If I click on a load balancer that has been activated, I get an "Unknown link" error. I can successfully interact with the balancer via rancher-compose, but not via the link to it in the UI.
If I create the balancer and click on it before it is activated, I can reach it. But as soon as it is active, the page reports the error and I can no longer examine it via the web UI.

@ndarilek Can you confirm that you are logged in as a user and not an admin?

I'm confused, aren't all admins users, so I couldn't be logged in as an
admin if I wasn't a user?
I can probably still answer this question. My installation only has a
single user at this time, my own, and it is an admin. I was able to do
this under 0.37, but not under 0.39.1. And it can access an inactive
load balancer, but as soon as that balancer is active, the link to it
errors out. If I then deactivate the balancer via rancher-compose  down, I can visit the link again.

we have users and admins that have different access. admins get access ot the admin tab.
But @aruneli has confirmed that it's for all users/admins. @ndarilek Sorry for the confusion.
@aruneli will provide more details on how to reproduce.

Issue reproduced in v0.40.0 -rc1
Steps to Reproduce:

Create a LB
Start the LB
Click on LB name

Result: Directed to "Error, Unknown Link" page (outside of Rancher)
Expected Result: Details of LB page

This is blowing up because there are no containerStats available for balancer services for some reason.  Changed to not blow up for now, but the stats should be added as a followup.

Component 	Version
Rancher 	v0.40.0-rc2
Cattle 	v0.96.0
User Interface 	v0.54.0
Rancher Compose 	beta/latest

Verified that I can access load balancer page but since stats is empty, assigning it to @wlan0 for the fix.


skip TestFloat64Dtype tests

float64 not often supported on integrated GPUs, so probably good to skip related tests as discussed here: https://github.com/geohot/tinygrad/issues/1087

Would merge ripping out float64 entirely.


Limiting rewrites?

I found a degenerate case using rules similar to the math example. When confronted with an expression like (* x 0) it seems to first apply both zero-mul and comm-mul to get that (* x 0) = (* 0 x) = 0. After that, each iteration seems to run more and more instances of assoc-mul. I think this makes arises since the x is equivalent to a multiplication, leading to: (* x 0) = (* x (*x 0)) = (* x (* x 0)).
The result is that this simple expression times out after 10 seconds. Is there a way to prevent this expansion? I wonder if it would be enough to add guards on the associativity rules that the nested expression isn't a constant?
If there is a way, I'm happy to figure out and add a reasonable case to the existing tests for it.

You could indeed write guards on such an associativity rule. But associativity is a known hard problem in this context, and the name of the game is just managing explosion. Saturating something like this is going to be very tricky indeed.
Your best bet for limiting which rules apply is the scheduler API. egg uses the BackoffScheduler by default (which you can customize a fair bit). You can change the scheduler with the with_scheduler method of Runner.

That makes sense. Would it make sense for me to add something along those lines to the math.rs test (since it is mentioned as examples of things)?

I'm always happy to take patches! It is already using the BackoffScheduler, but if you can come up with something that the current math setup cannot do, it'd be great to have an example and a way to make it work.

The tests for math.rs seem to be using a test runner that exits as soon as an equivalence is found. So writing a test there like "(* x 0)" => "0" terminates really quickly (since it is found after the first iteration). But, when I use this attempting to reach saturation it loops forever. Is there a way to write the test and say that it should reach saturation?
I also noticed that the definition of is_const relies on finding an equivalent constant node in the graph. Is there a reason it couldn't use the analysis? It seems like that would avoid the search through all possible substitutions, wouldn't it?

re: is_const, yes you could use an analysis! But it only scans the e-nodes in one e-class, which is pretty quick.
You'll have to just write your own test function if you don't want to use the test_fn macro. You can check the stop_reason field on Runner to see if it saturated.

Oh. I think I see what I missed. The ConstantFold is pruning the non-leaf (non-constant) nodes. I think this answers many of my questions about the interaction between the ConstantFold pass and the rules  like (* x 0) as well as (probably) solves my problem.
That makes sense. I took a stab at it, and it looks like the pruning is actually preventing the blow-up here. Specifically:
After applying associativity to find (* x 0) = 0, the pruning rules kick in and delete the other nodes. Something similar happens with (* x 1) = x which rewrites: (* x 1) => (* (x 1) 1) => (* x (* 1 1)) => (*x 1).
An somewhat related thought -- would it make sense to take advantage of pruning within the implementation of is_const? It seems like instead of using iter.all(...) it could check just the first element of the iterator and then ensure there are no more elements. The pruning should have eliminated the non-constant nodes).
Poking at that looks like it reduces the L2 Accesses in the benchmarks by 20% to 40% in the math cases.

Ah that's a nice catch. I'd be a little nervous to rely on it though, since if they behavior changes this would be a tricky one to see.
Also, it's an any, not an all. Is that what you were referring to?

Ah yeah, that's the one. My experiment changed it to:
let iter = egraph[subst[var]].nodes.iter();
if let Some(n) = iter.next() {
  matches!(n, Math::Constant(..))
} else {
  false
}

I could try using the analysis data -- it seems like that would be cleaner than relying on the pruning behavior, and should offer a similar reduction in L2 Access since it doesn't need to search through the iterator.

Yeah, the analysis should be easier. Not sure why i didn't do that initially. Something like:
egraph[subst[var]].data.is_some()

Closing for now, feel free to re-open!


Fixed the function that returns the current month for the credit card descriptor

Fix compiler warnings: use non-deprecated constants

Thanks for the pull request @danielnegri . Merging.


[Java][FlightSQL] Inconsistent GET_PRIMARY_KEYS_SCHEMA

Describe the bug, including details regarding any error messages, version, and platform.
The protobuf documentation ^1 for the returned schema of FlightSQL CommandGetPrimaryKeys is
 *  catalog_name: utf8,
 *  db_schema_name: utf8,
 *  table_name: utf8 not null,
 *  column_name: utf8 not null,
 *  key_name: utf8,
 *  key_sequence: int32 not null

org.apache.arrow.flight.sql.FlightSqlProducer.Schemas#GET_PRIMARY_KEYS_SCHEMA ^2 is defined as
    public static final Schema GET_PRIMARY_KEYS_SCHEMA =
        new Schema(asList(
            Field.nullable("catalog_name", VARCHAR.getType()),
            Field.nullable("db_schema_name", VARCHAR.getType()),
            Field.notNullable("table_name", VARCHAR.getType()),
            Field.notNullable("column_name", VARCHAR.getType()),
            Field.notNullable("key_sequence", INT.getType()),
            Field.nullable("key_name", VARCHAR.getType())));

Note the reordering of key_name and key_sequence.
Should this be considered a bug in the implementation or the documentation? Or, is field order "unimportant" in this context?
Component(s)
Java

C++ agrees with Java:
https://github.com/apache/arrow/blob/2bbd67dc42c64bb5185b6b9147b777d9230e76e9/cpp/src/arrow/flight/sql/server.cc#L1302-L1308
So does Go:
https://github.com/apache/arrow-go/blob/56b794f52a9bfb45fb54a4e070002a69b38fb018/arrow/flight/flightsql/schema_ref/reference_schemas.go#L46-L53
I'd consider this a bug in the documentation.


linux: Enable P2P DMA, NVMe Target Passthrough, Fabrics In-Band Auth and TCP TLS.

Description of changes
These changes enable the following:

P2PDMA: Allows devices to DMA to another device without round-trip through the CPU if possible. Mainly useful for RDMA and NVMe Target, but I saw uses in AMDGPU and the Intel Xe driver as well.

NVMe Target Passthrough allows bypassing the block layer and thus more efficient when exposing an existing NVMe device directly, especially in combination with P2PDMA and RDMA/RoCE NICs.

NVMe over Fabrics In-Band Auth allows challange-response authentication. Enabled it for both client and target.

NVMe over Fabrics TCP TLS support - with the help of ktls-utils, TLS encrypted sessions are possible. This, in combination with In-Band Auth, enables secure storage transport.

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

@K900 Changed back to master, anything else I can do to get this merged soon?
Since it seems very unlikely this breaks anything, any chance for a backport as well?
The NVMe-oF parts hinder my development/CI a bit for https://github.com/vifino/nvmetcfg and the P2PDMA enablement should increase performance in some situations.


Refresh ship status under certain conditions

When formation changes, or any ship is repaired / destroyed.
连续出击的时候十分方便

hahah

coffee script 的锅。。我也不想。。


Update of the schema 41910to42000.sql for compatibility with MariaDB version 10.3.38.

Description
This PR changes two column renames in schema-41910to42000.sql. Using MariaDB version 10.3.38-MariaDB, some lines are not applying the expected changes to the column names in the tables. This issue results in errors during the version transition process.
The modifications made to the schema-41910to42000.sql ensure that changes to the tables are applied correctly. With these fixes, the transition between versions should proceed successfully, without errors.
Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[x] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)
[ ] build/CI
[ ] test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale

[ ] Major
[x] Minor

Bug Severity

[ ] BLOCKER
[ ] Critical
[ ] Major
[x] Minor
[ ] Trivial

How Has This Been Tested?
I made the mentioned changes, and the transition between versions went as expected.

@blueorangutan package

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 11111

@blueorangutan LLtest

@DaanHoogland a [LL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

code lgtm
@JoaoJandre @DaanHoogland
If the issue exists with mariadb 10 and is fixed by this PR, we should merge this.

@LayonRibeiro
can you update the title with a short summary of the issue ?

@JoaoJandre , for people using mariadb 10 this is actually a blocker. I don't know if mariadb10 is well supported otherwise, but I think we should include this.

@JoaoJandre , for people using mariadb 10 this is actually a blocker. I don't know if mariadb10 is well supported otherwise, but I think we should include this.

Yes we are including this for sure. I'm just waiting for the CI results and we can merge this.

@blueorangutan package

@rohityadavcloud a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 11130

Do we need explicit testing if Github Actions smoketests have passed? (that I reckon runs against MySQL 8.x)? @JoaoJandre @DaanHoogland

I think we can merge. cc @weizhouapache

Do we need explicit testing if Github Actions smoketests have passed? (that I reckon runs against MySQL 8.x)? @JoaoJandre @DaanHoogland

I think we can merge. cc @weizhouapache

@DaanHoogland shouldn't we wait for the CI results?

Do we need explicit testing if Github Actions smoketests have passed? (that I reckon runs against MySQL 8.x)? @JoaoJandre @DaanHoogland

I think we can merge. cc @weizhouapache

@DaanHoogland shouldn't we wait for the CI results?

It will not give us more information than the GitHub actions in this case, as @rohityadavcloud implies.

Do we need explicit testing if Github Actions smoketests have passed? (that I reckon runs against MySQL 8.x)? @JoaoJandre @DaanHoogland

I think we can merge. cc @weizhouapache

@DaanHoogland shouldn't we wait for the CI results?

It will not give us more information than the GitHub actions in this case, as @rohityadavcloud implies.

Sorry, I missed that message :sweat_smile:


[TIR] Add missing narrow down of index within conditions

Hi, there. This change to NarrowDataType pass want to also narrow down int types in index conditions like in buffer indices.
Eg, T.Int64(1) < i:int64 * T.int64(3)  => 1 < i:int32 * 3

in the future, fundamentally speaking, we might want to make sure all values are int64 in the higher-level TIR, and use this pass to narrow dtypes

off the topic: in the future, fundamentally speaking, we might want to make sure all values are int64 in the higher-level TIR, and use this pass to narrow dtypes

What if values come from graph level IR？ （eg，K：i32 of topk）

we have to cast all indices to int64 at first, otherwise affine analysis could fail because it ignores casting

we have to cast all indices to int64 at first, otherwise affine analysis could fail because it ignores casting

Yeah, want to add this change just because failures on predicate if we want to use affine analysis ：）
Very happy to know we have plan to make index type uniform!

It's pretty much engineering intensive though. Would be great to have some helping hands from the community @wrongtest

@cyx-6

I don't have more comments :-) @Hzfengsy would you like to take a second look?


[no merge!] Re-enable 50_60 for Devnet

Please remove the italicized help prompts before submitting or merging
Provide a general summary of your changes in the Title above
Pull requests without a rationale and clear improvement may be closed
immediately.
Please provide clear motivation for your patch and explain how it improves
Dash Core user experience or Dash Core developer experience
significantly:

Any test improvements or new tests that improve coverage are always welcome.
All other changes should have accompanying unit tests (see src/test/) or
functional tests (see test/). Contributors should note which tests cover
modified code. If no tests exist for a region of modified code, new tests
should accompany the change.
Bug fixes are most welcome when they come with steps to reproduce or an
explanation of the potential issue as well as reasoning for the way the bug
was fixed.
Features are welcome, but might be rejected due to design or scope issues.
If a feature is based on a lot of dependencies, contributors should first
consider building the system outside of Dash Core, if possible.

Issue being fixed or feature implemented

Why is this change required? What problem does it solve?
If it fixes an open issue, please link to the issue here.

What was done?
Describe your changes in detail
How Has This Been Tested?
Please describe in detail how you tested your changes.
Include details of your testing environment, and the tests you ran
to see how your change affects other areas of the code, etc.
Breaking Changes
Please describe any breaking changes your code introduces
Checklist:
Go over all the following points, and put an x in all the boxes that apply.

[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have added or updated relevant unit/integration/functional/e2e tests
[ ] I have made corresponding changes to the documentation
[ ] I have assigned this pull request to a milestone (for repository code-owners and collaborators only)

Guix Automation has began to build this PR tagged as v21.0.0-devpr6139.0ef641a4. A new comment will be made when the image is pushed.

I run build, the docker image would be available in ~1 hour
btw:
quorum llmq_60_75 doesn't fit because useRotation = true
quorum devnet_dip0024 doesn't fit too because useRotation = true
Seems as llmq_50_60 is the only choice, because it is not rotating and has proper size.


Various core deprecations

For #3869
Mostly unused and undocumented methods.
See inline comments.

Coverage decreased (-2.5%) to 81.126% when pulling 5ca4add20376483de56a97ef43680dc24fd77ef3 on x/core-deprecations into 412ef8a4d480497d3b4e02ede281e972e14ed0f3 on master.


fixFix alter column stats not forward to master bug.

Alter column stats operation need to write bdbje, so it should be forwarded to master to execute.

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


Increment version for textanalytics releases

Increment version for textanalytics releases

Can one of the admins verify this patch?


Feature/blish support

Changes:
-Added Native BlishHUD Support (For custom mumble link)
-Added option to start accounts always with custom mumble-link (with a warning) in advanced account settings.
-Added options to start TacO or Blish automatically with an account in account settings
Disclaimer: BlishHUD-Killproof module currently not working when multi-boxing, u have to disable it to run multiple instances of BlishHUD
#191 - should also no problem anymore

I wanted to note that this PR would help a lot with the Blish HUD project (GitHub, Website) and ensuring users have a seamless experience using it in multi-box mode with LaunchBuddy.
+1 from us


Unable to see video stream on the Web from my custom iOS app.

Unable to see video stream on the Web from my iOS app.
Description
In my project I need to stream live videos (conferencing) with some custom camera controls. I had setup a custom AV stack in order to have the full control over my camera (change resolution, fps, etc.). As a streaming solution I've picked Twilio. According to the documentation and some examples, if I use a custom AV stack, then I need to use Sinks.
The problem is with the following: when I stream from iPhone and run a VideoQuickStartiOS example app on the simulator on my Mac, then I can see the video and hear audio from my device, which is absolutely great. But, when I connect to the same room from the Web interface, then I can ONLY hear my audio and video is not visible.
Room is configured using Twilio CLI and the interesting fact is, when I connect from the Video Community example app to the room the video is visible on the web.
Trying to figure out what is the difference and why it's not working using TVIVideoSink from my device.
Steps to Reproduce

Generate a new room from the React example
Setup a simple AVFoundation stack and use code snippets from bellow
Wonder why Video is visible in the VideoQuickStart app and not on the Web.

Code
//Twilio basic setup
private func setupTwilio() {

        if (audioTrack == nil) {
            audioTrack = LocalAudioTrack(options: nil, enabled: true, name: "Microphone")
        }

        // Create a video track which captures from the camera.
        if (videoTrack == nil) {
            guard let device = activeInput?.device else { return }

            videoTrack = LocalVideoTrack(source: self)

            let config = CameraConfigFactory().makeCameraConfigFactory(captureDevice: device)
            self.requestOutputFormat(config.outputFormat)        }

        // Preparing the connect options with the access token that we fetched (or hardcoded).
        let connectOptions = ConnectOptions(token: accessToken) { (builder) in

            if let audioTrack = self.audioTrack {
                builder.audioTracks = [audioTrack]
            }
            if let videoTrack = self.videoTrack {
                builder.videoTracks = [videoTrack]
            }

            builder.roomName = "Test"
        }

        // Connect to the Room using the options we provided.
        room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)
    }

private func videoOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
    guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {
        debugPrint("Can't create pixel buffer")
        return
    }

    let time = CMSampleBufferGetPresentationTimeStamp(sampleBuffer)
    let frame = VideoFrame(timestamp: time, buffer: pixelBuffer, orientation: VideoOrientation.up)
    sink?.onVideoFrame(frame!)
}

// MARK:- VideoSource
extension CameraController: VideoSource {
    var isScreencast: Bool {
        // We want fluid AR content, maintaining the original frame rate.
        return false
    }

    func requestOutputFormat(_ outputFormat: VideoFormat) {
        if let sink = sink {
            sink.onVideoFormatRequest(outputFormat)
        }
    }
}

//Camera config
struct CameraConfig {
    let outputFormat: VideoFormat
    let inputFormat: VideoFormat
}

class CameraConfigFactory {
    func makeCameraConfigFactory(captureDevice: AVCaptureDevice) -> CameraConfig {
        var targetSize: CMVideoDimensions {
            return CMVideoDimensions(width: 544, height: 480)
        }

        var frameRate: UInt {
            return 24 // With simulcast enabled there are 3 temporal layers, allowing a frame rate of {f, f/2, f/4}
        }

        let cropRatio = CGFloat(targetSize.width) / CGFloat(targetSize.height)
        let preferredFormat = selectVideoFormatBySize(captureDevice: captureDevice, targetSize: targetSize)
        preferredFormat.frameRate = min(preferredFormat.frameRate, frameRate)

        let cropDimensions: CMVideoDimensions

        if preferredFormat.dimensions.width > preferredFormat.dimensions.height {
            cropDimensions = CMVideoDimensions(
                width: Int32(CGFloat(preferredFormat.dimensions.height) * cropRatio),
                height: preferredFormat.dimensions.height
            )
        } else {
            cropDimensions = CMVideoDimensions(
                width: preferredFormat.dimensions.width,
                height: Int32(CGFloat(preferredFormat.dimensions.width) * cropRatio)
            )
        }

        let outputFormat = VideoFormat()
        outputFormat.dimensions = cropDimensions
        outputFormat.pixelFormat = preferredFormat.pixelFormat
        outputFormat.frameRate = 0

        return CameraConfig(outputFormat: outputFormat, inputFormat: preferredFormat)
    }

    private func selectVideoFormatBySize(captureDevice: AVCaptureDevice, targetSize: CMVideoDimensions) -> VideoFormat {
        let supportedFormats = Array(CameraSource.supportedFormats(captureDevice: captureDevice)) as! [VideoFormat]

        // Cropping might be used if there is not an exact match
        for format in supportedFormats {
            guard
                format.pixelFormat == .formatYUV420BiPlanarFullRange &&
                    format.dimensions.width >= targetSize.width &&
                    format.dimensions.height >= targetSize.height
                else {
                    continue
            }

            return format
        }

        fatalError()
    }
}

Expected Behavior
Video should be visible on the web version of the room
Actual Behavior
Video is not visible on the web version of the room. Looks like it is turned OFF, but it's visible in the iOS VideoQuickStart app.
Reproduces How Often
Always
Logs
Debug level logs are helpful when investigating issues. To enable debug level logging, add the following code to your application:
Versions
All relevant version information for the issue.
Video iOS SDK
pod 'TwilioVideo', '4.0.0-beta1'
Xcode
Version 11.7 (11E801a)
iOS Version
iOS 13.7
iOS Device
iPhone X

@vpryimachenko,
To make sure I am understanding the issue:
You have created an iOS app that uses CameraController, which is a class that implements the VideoSource protocol. When you run this application and connect to a room, another iOS device connects to the room running the iOS Quickstart application and you are able to render the video.
However, if you join the same room using the Javascript Video App example application, the video does not render, but you are connected to the same room because you have audio flowing?
If you run the iOS Video App example application with the Javascript Video App example application, the video does render.
My first thoughts are that if the iOS Quickstart application is able to render the video that is being sent from your CameraController there is no reason why it shouldn't also work with the Javascript implementation.
To get us going on investigation, would you be able to send over some Room Sids where you have connected your iOS application and the web client to the same room when you have no video being rendered? Also the logs from both the iOS and Web side would be very useful to determine what is happening during this call.
Let me know if I am understanding the issue correctly and if you can send the log files over. Also, I noticed that you are running the 4.0.0-beta1 version of the SDK. While this should certainly work, I was wondering if you could also try this scenario with 3.7.1 of the SDK to ensure that it isn't something introduced in this beta.
Thank you,
Ryan


Handle tag errors on import

Prior to this commit we had implemented passing tag errors up the stack,
but didn't have any specs around it and sure enough, the implementation
wasn't flexible enough. This commit fixes our implementation and also
adds the missing spec.

Good catch


re: feedback request on linkedin

Here are some notes I jotted down will perusing your code, broken down by "make it easier to use", "make it easier to develop", and "make it easier to distribute". I'll also open a couple quick PRs to address a couple of these points after I open this issue.
make it easier to develop

main branch should build with stable rust
put all nightly feature usage into a lightweight "nightly-changes" (or
whatever you want to call it) branch
create sub-crate that offers a nice public library api
consider using an existing library for command line argument parsing

this will reduce the time you spend working on your own custom argument
parsing and make it
clap to simplify your life when it

prioritize documenting your types, especially obviously important traits like DataSink and Finalizable
push docs to docr.rs (once you've written some)
push crates to crates.io

make it easier to use

commit to some kind of stable but evolveable command line api

main command line api should require few options to get good average use
case behaviors
make it easy to stray outside the lines with compression, encryption, and
network parameters
think of the difference between git porcelain and git plumbing
flags like --release, --backup, and --check could be subcommands

argument parsing should be grouped by applicability to different
subcommands/modes
subcommands make it easier to extend the main cli with additional
high-level behaviors

make it easier to distribute

look into minimizing binary size

adding the following to your Cargo.toml brings the binary size down to
~682KB:

[profile.release]
lto = true
codegen-units = 1
strip = true

allowing for a no_std build could significantly reduce it further (i'm not
sure though), would be worth a try
consider if std is really necessary for all use cases and offer a std feature
flag or something (you mentioned a target use case is embedded environments
which may not have standard library could be good for embedded systems, but
what about embedded systems without access to std?

Just opened #5 and #6


Joining properties does not seem to work

Readme promises to understand expressions like (.property1 | .property2 | .propertyN).
I try JSPath.compile('.aa | .bb') and JSPath.compile('(.aa | .bb)') and get Unexpected token "|" and Unexpected token "(" respectively.
0.2.14.

At present there's no way to use join in top level.
But you could easily get what you want:
JSPath.compile('.(.aa | .bb)')

Is it hard to fix?

You aren't first who asks for this so I try to implement.

Sweet! It's logical to have this.

Will be implemented in 0.3

@hoho I've released 0.3.0, could you check whether it works as you expected?


Parity violating contributions to molecular energy.

From PR https://github.com/pyscf/pyscf/pull/2459


Handle Spotinst timeout and throttle exceptions

Spotinst is currently having issues where requests will just hang until they timeout. They are hanging for up to 15mins so set a reasonable timeout in requests so we don't wait that long and then catch those exceptions.
Added retry logic to automatically keep retrying with jitter. This will happen either when Spotinst throws a rate exceeded error (http code 429) or if the request hangs.

build


Issue 312 - Add simple server README

closes #312

I would combine the readmes for this and curlz, since they belong together as part of an end-to-end setup.
I would also integrate cobra in for a nicer CLI experience.
Finally, I would highlight the most important sections of the code in the README. In this case that would be the bits that show how to tie ziti into the http server and http requests. If you prefer, I can do that bit in a second pass.


New mangling: fix re-mangling of ImplFunctionType with generic signature

@swift-ci Please smoke test and merge


How do you add a new where clause to an existing sql statement?

I am unable to figure how to construct an Expr to pass to AddWhere method. Lets say I have a query  like "SELECT * FROM product WHERE price < 500". Now how do I add "discount <= 10"? This is the closest I came to a solution:
	left := sqlparser.NewStrVal([]byte("price"))
	right := sqlparser.NewStrVal([]byte("?"))
	comp := &sqlparser.ComparisonExpr{
		Operator: sqlparser.LessEqualStr,
		Left:     left,
		Right:    right,
	}
	fmt.Printf("%v\n", sqlparser.String(comp))
	sel := stmt.(*sqlparser.Select)
	sel.AddWhere(comp)
	fmt.Printf("%v\n", sqlparser.String(sel))

But this seems to quote the left in single quotes, because I assume it is considering it as a value instead of a column name. So how do I add a column name expr?

I think you'll want to use something like this: https://github.com/vitessio/vitess/blob/v18.0.1/go/vt/sqlparser/ast_test.go#L91
That will provide a Column name instead of a String value type.


[JS] [Regression] Model googleai/gemini-1.5-pro-latest not found

Describe the bug
Set up a .prompt file like this:
---
model: googleai/gemini-1.5-pro-latest
config:
  temperature: 0.85
  candidateCount: 4
input:
...

This used to work before the 0.9 update, but now it's broken.
I tried using the stable version model: googleai/gemini-1.5-pro, but it performs poorly (doesn't return JSON according to my schema). The latest version is much better at respecting the schema.
Please bring back the ability to use the latest versions!
To Reproduce

Run the prompt
Error Model googleai/gemini-1.5-pro-latest not found

Expected behavior
Prompt should complete successfully
Screenshots
If applicable, add screenshots to help explain your problem.
Runtime (please complete the following information):

OS: MacOS
Version 0.9

** Node version

run node --version at paste here
node --version
v18.20.5

Additional context
Add any other context about the problem here.

Just one point of clarification, the model names have been streamlined a bit; gemini-1.5-pro will always correspond to the current stable version. If you are OK with preview versions, you can still do that as well by providing config.version.
---
model: googleai/gemini-1.5-pro
config:
  version: gemini-1.5-pro-latest
...
---

I'm going to close this as answered, and also opened https://github.com/firebase/genkit/issues/1335 to make this clearer in the docs.


Ability to print out OIDC tokens with Debug logging

Currently it's super hard to figure out what my OIDC token contains and how that maps to roles the user is in.
It would be great to have a CLI flag that enables the server to print out the tokens to allow users to debug the exact path they need to specify in the userClaimsRules

OK while debug printing to find out why my mapping wasn't working I discovered the doco is actually wrong and the mapping key is actually 'claimMapping' not 'userClaimsRules' so I've PR'd the fix in #79


Anti collision feature conflicts with name tag plugin

Hello,
I have tried Aero, the most interesting feature to me was the anti-collision.
Unfortunately it doesn't work when using a name tag plugin such as NametagEdit
The player have a nametag but they can push each other.
Do you think it is possible to integrate Aero with NametagEdit, or better, with your Nametags plugin to have both the nametag and the anti collision work at the same time ?
Thanks !

Hello, first thanks for reporting this issue. I'm currently  on vacation and won't be able to reply a proper answer until Friday. I will answer again to this issue request on Friday.


Possible file descriptor leaks?

I have been debugging issues with file descriptors leaking in an application, and looking at Elastigo I'm wondering if there's a chance that Elastigo may be leaking some FDs.
For example, In lib/coreget.go:94 inside of ExistsBool, an HTTP request is executed, but the Body of the response is never closed, presumably leaving a dangling connection for some period of time. It looks like this may also be an issue in some other functions, i.e. get itself calls DoCommand which returns the Body of a Response, but it is never closed.
Am I mistaking, or are these resource leaks?

hi @johnwchadwick we wrapped the http stuff in an api that looks a bit like the Go stuff so while it looks like core Go http calls, it's actually Elastigo code. follow the path to https://github.com/mattbaird/elastigo/blob/f2efa92a8bd8e8976f45ffe0cfbdab7cbf4b5a63/lib/request.go#L121
and you'll see the defer close on the actual underlying connection.
cheers!

Oh, I missed that bit. Thanks for the explanation.


Feature/add font display for font face

Added the option to have a use a different way to show the font-space before it's loaded.

Can you add it to the description as well.
And wouldn't it be better to use swap as default value? Not sure but good to investigate.

Added the description. I would leave the default value as it is (auto). Othertwise it could cause side effects when upgrading it on excisting projects.


pipenv install doesn't work when PIP_USER=yes

I was having trouble using pipenv without --site-packages. I couldn't find anything online so this might help someone even if this behaviour doesn't get changed.

Expected result
pipenv to work like normal, maybe ignoring $PIP_USER
Actual result
$ pipenv install requests --verbose
Creating a virtualenv for this project…
⠋Using base prefix '/usr'
New python executable in /home/orestis/.local/share/virtualenvs/tmp-XVr6zr33/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /home/orestis/.local/share/virtualenvs/tmp-XVr6zr33
Creating a Pipfile for this project…                                                                                Installing requests…
⠙Installing 'requests'
$ "/home/orestis/.local/share/virtualenvs/tmp-XVr6zr33/bin/pip" install   --verbose    "requests" -i https://pypi.python.org/simple --exists-action w                                                                                Exception information:
Traceback (most recent call last):
  File "/home/orestis/.local/share/virtualenvs/tmp-XVr6zr33/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/home/orestis/.local/share/virtualenvs/tmp-XVr6zr33/lib/python3.6/site-packages/pip/commands/install.py", line 251, in run
    "Can not perform a '--user' install. User site-packages "
pip.exceptions.InstallationError: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.

Error:  An error occurred while installing requests!
Can not perform a '--user' install. User site-packages are not visible in this virtualenv.

Steps to replicate

export PIP_USER=yes
Run pipenv install …

Thanks for the report. Pipenv respects all pip configurations including the possibility that you are passing —system. If you mean not to indicate to pip that you want a user install you should unset this as other users take advantage of this and other pip configuration passthrough quite extensively. Closing for now, have a good one!

@techalchemy I might me missing something but I'm not clear why pipenv should respect this flag. If pipenv always creates virtual environments, shouldn't it ignore the user and system flags?
For context, I'm setting this in my pip.conf to avoid accidentally polluting the global site in my pyenv versions. I can get around this for pipenv by aliasing it to PIP_USER=0 pipenv, but that seems hacky for something seemingly incompatible with pipenv's design goals.


Cursor keys don't work in Emacs

In Black Screen 0.36.7, the cursor keys don't work in Emacs: instead, up types &, down types (, left types %, and right types '. In iTerm on the same machine, however, Emacs cursor keys work as expected. What would have to happen for this to work in Black Screen?

Fixed. Thank you for the report, @marnen.

Confirmed, although the cursor doesn't always display properly. Thanks for the quick fix!

@marnen, yeah, it's a recently introduced regression with the caret. I'll try to fix it tomorrow and will let you know.

@marnen, now the caret should be displayed correctly. You should also remember that Black Screen is currently highly unstable, don't rely on it in your everyday work.

@shockone Again, thanks! I know it's somewhat unstable, but I want to play around, see how far I can get with it, and perhaps contribute to it.


Types in structs

{
  (t list) list
  (v nil) nil
  (v cons) cons
}

What about opaque types (can values in the struct refer to the types in the struct) (just implement proper existentials instead)? What about recursion? What about GADTs? What do the types of such structs look like?

I think the best solution is to require type fields to be concrete, and implement existentials for when you want them opaque. This way you can also use existentials with things other than structs.
(exist a * {(t a) a})


BUG - Import from grid long-term stats not capturing import used for battery charging

Hi,
I use Octopus Agile overnight cheap rates to charge my battery, however the sensors available for import in the Energy dashboard does not capture the kW used for this.   In this screenshot you can see that I've imported 1.8kWh to date today (7am) but that the battery charged is 5.3kWh.  I've actually imported 7.1kWh overnight, as you can see from the second image (growatt dashboard).  I'd like to be able to accurately track the import used alongside the tariffs imported from Octopus, for the purposes of cost tracking.  Currently, if I charge my battery, that doesn't work.

Hi @jasongoff - I have a few questions:

I made a lovely template for 'bugs', when you create a new issue it should present you with a template to complete, did that not happen? It would give me all the information I need to debug this further i.e. what type of system you have
Is there a value on the Android app anywhere that would give you this value? If there is can you please send me a screengrab from the app so that I can pinpoint the specific state that needs to be added.

Oops! Sorry Chris - I didn't read that properly and just deleted it before I started typing!  Copy/pasted below, but to answer your 2nd question sensor.ouctcgp05j_import_from_grid shows the point-in-time values correctly, including the import when I was charging.  I can't see any numbers in the mobile app, just the graphs etc.  Where would I look do you know?
Describe the bug
As above!
To Reproduce
Go to Energy Dashboard, view the OUCTCGP05J Import from grid today (load + charging) value.  It does not include anything imported to charge the battery.
Growatt Device Type
Device(s): SPH5000 Hybrid Inverter
Home Assistant information
Home Assistant 2023.1.5
Supervisor 2022.12.1
Operating System 9.4
Frontend 20230110.0 - latest

Growatt Server API Integration version:
The one included in Core - couldn't find a version number

Additional context
N/A

Thanks @jasongoff - You're forgiven.
Note - if you're still using the version of the integration from Core then I recommend switching to the HACS one so that as/when I fix things you can get them quicker. But it's totally your preference.
I have the same type of inverter and I haven't noticed the same issue, however it doesn't mean that it's not specific to just yourself.
Now, with reference to the specific problem you're having:
In order for values to appear on the Energy Dashboard they have to be in "kwh" and an increasing sum e.g. "total lifetime" or "total today" i.e. not an instantaneous value.
The particular value you're after has always been a bit of a tricky one to get hold of for hybrid systems, I did have a go at calculating it manually for a while.
You'll have to leave it with me, I'll check over the values on the API and come back to you, but it may take a few days (I need to find a gap to look at this again).
The good news is that I have exactly the same system so it shouldn't be too hard to compare against.

Can you do me a favour and post this same screenshot but from your phone/app:

It will allow me to compare the values, note that my value for "Imported from grid" (bottom right) looks approximately correct when compared to the value on the graph.

I wrote a script a while back for exactly this purpose, but discarded it because I found a value on the API that provided what was needed.
Anyway, it looks like they've changed the value of that API call. I've just run the script again now and these were the results:
    *TODAY TOTALS BREAKDOWN*
      Self generation total (batteries & solar - from API) (kwh): 7.2
      Load consumed from solar (kwh): 1.6
      Load consumed from batteries (kwh): 5.6
      Self consumption total (batteries & solar - from API) (kwh): 7.2
      Load consumed from grid (kwh): 4.4
      Total imported from grid (Load + AC charging) (kwh): 4.4
      Load consumption (calculated) (kwh): 11.6
      Load consumption (API) (kwh): 11.6
      Exported (kwh): 0
      Solar battery charge (calculated) (kwh): 1.2
      AC battery charge (calculated) (kwh): 7.5

Note the massive difference in the values between: Total imported from grid (Load + AC charging) (kwh): 4.4 which is from the API call that should have provided the combined value and Load consumption (calculated) (kwh): 11.6 which is calculated with some rather clunky maths (but is far more accurate).
Assuming you're seeing the same problem then I think it's time I re-introduced this logic into the Integration and took it for a spin.
If you're willing to test it when it's ready you'll need to switch over to the HACS version of the Integration and we can go from there.
I'll keep you posted, NOTE - I'm unlikely to be able to take a look at this before the weekend.

Can you do me a favour and post this same screenshot but from your phone/app:
It will allow me to compare the values, note that my value for "Imported from grid" (bottom right) doesn't look correct when compared to the graph.

Bizarrely, I'm in a data deadspot where I work (Swindon town centre!) and the firm's wifi seems to be blocking access to the app!  I'll check when I get home.

Assuming you're seeing the same problem then I think it's time I re-introduced this logic into the Integration and took it for a spin. If you're willing to test it when it's ready you'll need to switch over to the HACS version of the Integration and we can go from there.
I'll keep you posted, NOTE - I'm unlikely to be able to take a look at this before the weekend.

Hi Chris, yes that looks exactly like the issue I'm seeing.  Happy to test, I was planning to switch over to the new version soon anyhow.  Thanks!

No hurry whatsoever, like I said, I'm unlikely to be able to work on, it issue a fix for this until the weekend at the earliest.
The key thing is that it looks like we're both experiencing a similar issue only I had been ignorant to it.
I'll keep you posted on the situation and when I've got a beta release for testing.
Whenever you get the chance to post a screenshot would be helpful just to eyeball that's all.

Hi @jasongoff - As you can guess I haven't got around to implementing the fix for this yet. I didn't have time over the weekend and I've just spent the evening 'administrating' the hell out of this repo.
We now have all the bugs from the Core repo duplicated over here so that I can actually work my way through them.
This one is next on my list to fix, hopefully in the next day or so.
Sorry for the delay, if it makes you feel any better I have the same system, so I'm experiencing the same bug and it's nagging at me!!

Hi @muppet3000, no problem!  You're doing this voluntarily alongside your proper* job!
Cheers
Jase

If yours is anything like mine, tinkering is more fun than the stuff we get paid for :)

@jasongoff - If you're feeling adventurous, please try using the 'dev' build of the Integration through HACS (if you can't work out how to do it just let me know here and I'll post screen grabs).
I've pushed the logic for calculating the combined 'load + charging' value to be calculated as accurately as we can given the data available.
Let me know how you get on.
I'm hoping to do a release in the next few days with the logic if you'd prefer to wait for that. There's just 1 more bugfix I want to include in it as well.

Hi @muppet3000, All installed so I'll monitor/compare when I charge the battery tonight.  Thanks very much :)

Hi @muppet3000, All installed so I'll monitor/compare when I charge the battery tonight. Thanks very much :)

In theory if you have an in-home display for your electricity consumption you should see how much you've consumed from the grid so far today. The reading in home-assistant should match up (or be damn close).
Let me know how you get on.

FYI - I pushed another tweak to this earlier which you should notice no difference in, it just improves the logic by which this value is calculated to accommodate for when some of the graph data is missing. It still has to make quite sweeping assumptions but it's better than what was there before.
One more bug to squash and then I'll do a proper release that contains this.

Hi @muppet3000, either I've set it up incorrectly, or I'm not seeing the changes.  I installed DEV release in HACS, but I'm not seeing the intended results.  Battery charged overnight for 6kwh, but import+load showing as 1.5kwh rather 7.5.

Did you remember to restart Home Assistant after installing the Dev version? (If you go to the HACS dashboard in HA it normally has a big red warning if you haven't restarted since the update)
If you did then I'm not sure what the issue could be. If a restart of HA doesn't fix it would you consider sharing access to your system with me so I can help debug? There are steps on how to share them on the FAQ page for this repo.

Hi, yes did a restart. Sent you a personal email. Cheers!
Sent from Outlook for Androidhttps://aka.ms/AAb9ysg

From: Chris Straffon @.>
Sent: Friday, February 3, 2023 7:12:15 AM
To: muppet3000/homeassistant-growatt_server_api @.>
Cc: Jason Goff @.>; Mention @.>
Subject: Re: [muppet3000/homeassistant-growatt_server_api] BUG - Import from grid long-term stats not capturing import used for battery charging (Issue #7)
Did you remember to restart Home Assistant after installing the Dev version? (If you go to the HACS dashboard in HA it normally has a big red warning if you haven't restarted since the update)
If you did then I'm not sure what the issue could be. If a restart of HA doesn't fix it would you consider sharing access to your system with me so I can help debug? There are steps on how to share them on the FAQ page for this repo.
—
Reply to this email directly, view it on GitHubhttps://github.com/muppet3000/homeassistant-growatt_server_api/issues/7#issuecomment-1415191889, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABEFTA4OOQ6CIE2DUYCWX4LWVSVU7ANCNFSM6AAAAAAT6WQC4Y.
You are receiving this because you were mentioned.Message ID: @.***>

Hi @jasongoff I think you've misunderstood, I don't need access to your HA installation, I'd like to use your credentials for Growatt on my Home Assistant test instance to debug the issue.
You can find steps on how to do that here: https://github.com/muppet3000/homeassistant-growatt_server_api/blob/dev/FAQ.md#how-do-i-share-my-credentials-with-you-for-testingadding-new-features-for-my-system-type

Hi, done!  email should be coming your way from Growatt.  Let me know if not.

From: Chris Straffon @.>
Sent: 03 February 2023 8:07 AM
To: muppet3000/homeassistant-growatt_server_api @.>
Cc: Jason Goff @.>; Mention @.>
Subject: Re: [muppet3000/homeassistant-growatt_server_api] BUG - Import from grid long-term stats not capturing import used for battery charging (Issue #7)
Hi @jasongoffhttps://github.com/jasongoff I think you've misunderstood, I don't need access to your HA installation, I'd like to use your credentials for Growatt on my Home Assistant test instance to debug the issue.
You can find steps on how to do that here: https://github.com/muppet3000/homeassistant-growatt_server_api/blob/dev/FAQ.md#how-do-i-share-my-credentials-with-you-for-testingadding-new-features-for-my-system-type
—
Reply to this email directly, view it on GitHubhttps://github.com/muppet3000/homeassistant-growatt_server_api/issues/7#issuecomment-1415292642, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABEFTA4DC4CAC42RJVX6UXDWVS4CXANCNFSM6AAAAAAT6WQC4Y.
You are receiving this because you were mentioned.Message ID: @.***>

Thanks I've got it, I'll try and take a look.
The maths doesn't quite add up, I agree. I've got a little helper app that I wrote to debug this sort of stuff so I'll run it through that tonight and see what comes out!

I got the email, have tried it all out with your credentials and on my
system it's calculating your current usage for the day at 7.5kwh, which
looks approximately correct.
At the risk of sounding devastatingly patronising (and that's not my
intention), are you sure you're using the HACS version of the integration
rather than the one in Home Assistant Core?
The one circled here:
[image: image.png]
In the integrations page it will show you what you're using currently:
[image: image.png]
If you happen to have accidentally chosen the old one (the one just called
"Growatt") then that's the one from Home Assistant Core.
As far as I can tell, the logic is working fine, this is the only thing I
can think of now.
C
--
Chris Straffon - DevOps Engineer
Aldershot, Hampshire
Mobile: +44 (0)7921 629628
On Fri, 3 Feb 2023 at 08:56, Jason Goff @.***> wrote:

Hi, done! email should be coming your way from Growatt. Let me know if not.

From: Chris Straffon @.>
Sent: 03 February 2023 8:07 AM
To: muppet3000/homeassistant-growatt_server_api @.>
Cc: Jason Goff @.>; Mention @.>
Subject: Re: [muppet3000/homeassistant-growatt_server_api] BUG - Import
from grid long-term stats not capturing import used for battery charging
(Issue #7)
Hi @jasongoffhttps://github.com/jasongoff I think you've misunderstood,
I don't need access to your HA installation, I'd like to use your
credentials for Growatt on my Home Assistant test instance to debug the
issue.
You can find steps on how to do that here:
https://github.com/muppet3000/homeassistant-growatt_server_api/blob/dev/FAQ.md#how-do-i-share-my-credentials-with-you-for-testingadding-new-features-for-my-system-type
—
Reply to this email directly, view it on GitHub<
https://github.com/muppet3000/homeassistant-growatt_server_api/issues/7#issuecomment-1415292642>,
or unsubscribe<
https://github.com/notifications/unsubscribe-auth/ABEFTA4DC4CAC42RJVX6UXDWVS4CXANCNFSM6AAAAAAT6WQC4Y

.
You are receiving this because you were mentioned.Message ID: @.***>

—
Reply to this email directly, view it on GitHub
https://github.com/muppet3000/homeassistant-growatt_server_api/issues/7#issuecomment-1415392834,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACQ62ZEYXDIPXOSACJ2TRKTWVTB4PANCNFSM6AAAAAAT6WQC4Y
.
You are receiving this because you were mentioned.Message ID:
@.***
com>

Sent you an email. I accidentally sent it here too but I've deleted it from here so you probably have the same email twice.

Hi @muppet3000 ,
Now I've actually got the dev branch installed properly, I've checked this morning and it is indeed working out the numbers correctly!  Thanks very much, and apologies for the confusion!
Cheers
Jase

Awesome, thanks for the update, I was beginning to think my logic was twitchy!!
I've got 1 more bug to squash and then I'll wrap it up in a formal release.

Hi @muppet3000
I downloaded the latest update and my mobile account has locked out. Should i have changed something in the setup?

I recommend taking a look over here: https://github.com/muppet3000/homeassistant-growatt_server_api/issues/15
There have been various issues with the Growatt servers today and they seem to have started locking accounts for some reason.
My suggestion from over on that other thread is to log into the growatt website (server.growatt.com) and create yourself a different user for use with Home Assistant (and potentially one for your phone as well) to work around this issue. i.e. follow the same steps you did to create a user for me, but just create multiple for yourself so that you don't get locked out on your phone again.
I am also re-adding the ability to specify "server.growatt.com" back into the integration (available on Dev now) as that endpoint is working again and may be more stable.

Thanks Chris,
I'd already thought about creating separate accounts to preserve the master one, so will get that sorted tomorrow.
Cheers
Jase
Sent from Outlook for Androidhttps://aka.ms/AAb9ysg

From: Chris Straffon @.>
Sent: Tuesday, February 7, 2023 9:34:43 PM
To: muppet3000/homeassistant-growatt_server_api @.>
Cc: Jason Goff @.>; Mention @.>
Subject: Re: [muppet3000/homeassistant-growatt_server_api] BUG - Import from grid long-term stats not capturing import used for battery charging (Issue #7)
I recommend taking a look over here: #15https://github.com/muppet3000/homeassistant-growatt_server_api/issues/15
There have been various issues with the Growatt servers today and they seem to have started locking accounts for some reason.
My suggestion from over on that other thread is to log into the growatt website (server.growatt.com) and create yourself a different user for use with Home Assistant (and potentially one for your phone as well) to work around this issue. i.e. follow the same steps you did to create a user for me, but just create multiple for yourself so that you don't get locked out on your phone again.
I am also re-adding the ability to specify "server.growatt.com" back into the integration (available on Dev now) as that endpoint is working again and may be more stable.
—
Reply to this email directly, view it on GitHubhttps://github.com/muppet3000/homeassistant-growatt_server_api/issues/7#issuecomment-1421484204, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABEFTA2O3SAGJG3OZQWBOZ3WWK5XHANCNFSM6AAAAAAT6WQC4Y.
You are receiving this because you were mentioned.Message ID: @.***>

Hi,
I can't see where to update my account details in the HA integration.
Sent from Outlook for Androidhttps://aka.ms/AAb9ysg

From: Chris Straffon @.>
Sent: Tuesday, February 7, 2023 9:34:43 PM
To: muppet3000/homeassistant-growatt_server_api @.>
Cc: Jason Goff @.>; Mention @.>
Subject: Re: [muppet3000/homeassistant-growatt_server_api] BUG - Import from grid long-term stats not capturing import used for battery charging (Issue #7)
I recommend taking a look over here: #15https://github.com/muppet3000/homeassistant-growatt_server_api/issues/15
There have been various issues with the Growatt servers today and they seem to have started locking accounts for some reason.
My suggestion from over on that other thread is to log into the growatt website (server.growatt.com) and create yourself a different user for use with Home Assistant (and potentially one for your phone as well) to work around this issue. i.e. follow the same steps you did to create a user for me, but just create multiple for yourself so that you don't get locked out on your phone again.
I am also re-adding the ability to specify "server.growatt.com" back into the integration (available on Dev now) as that endpoint is working again and may be more stable.
—
Reply to this email directly, view it on GitHubhttps://github.com/muppet3000/homeassistant-growatt_server_api/issues/7#issuecomment-1421484204, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABEFTA2O3SAGJG3OZQWBOZ3WWK5XHANCNFSM6AAAAAAT6WQC4Y.
You are receiving this because you were mentioned.Message ID: @.***>

You have to delete the integration and then re-add it, one of the weird quirks of HA, it's actually really difficult to write the logic to allow people to change their username/password, just easier to delete and re-add.
Since it's the same instance you're pointing to it'll still have all your previous data.

You have to delete the integration and then re-add it, one of the weird quirks of HA, it's actually really difficult to write the logic to allow people to change their username/password, just easier to delete and re-add.
Since it's the same instance you're pointing to it'll still have all your previous data.
From my phone but you get the idea...

Closing - fixed in https://github.com/muppet3000/homeassistant-growatt_server_api/releases/tag/v1.0.2


Signing, object vs string

If payload is not a buffer or a string, it will be coerced into a string using JSON.stringify.

Shouldn't these result in the same output or am I missing something?
jwt.sign({x: 1}, "secret")
// 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ4IjoxLCJpYXQiOjE0MjE4NjE1ODB9.YBvFd7_yRFKGRs09T1I6ChZze-6OmY7SpiCXxVzknVc'

jwt.sign(JSON.stringify({x: 1}), "secret")
// 'eyJhbGciOiJIUzI1NiJ9.eyJ4IjoxfQ.XOAA7XIKOu4yBszS83Yc_js4QvGVrdubYLQdNnpQ8J4'

If it's an object, the library will add the "typ" header as well.
https://github.com/auth0/node-jsonwebtoken/blob/master/index.js#L16
In addition it'll generate and add an iat to the token's payload in case of an object.
This doesn't happen in case of a string.


[R-package] Accept data frames as inputs

Summary
It should be possible to use R data frames directly as inputs in {lightgbm}, without converting them to matrices.
Motivation
The data.frame is a very common data structure in R, and many statistics and machine learning projects accept data frames as inputs, including but not limited to {caret}, {randomForest}, and the packages listed in https://github.com/microsoft/LightGBM/issues/4295#issuecomment-841753885.
Allowing the use of data.frame objects would reduce friction for R users working with LightGBM.
Description
I think this work can be broken down into the following components

[ ] accept data.frame inputs for data argument to lgb.Dataset()
[ ] accept data.frame inputs for data argument to lightgbm()
[ ] accept data.frame inputs for data argument to lgb.cv()
[ ] automatically encode categorical-looking column types (factor, character) in a LightGBM-friendly way
[ ] accept data.frame inputs for predict.lgb.Booster() / Booster$predict() / Predictor$predict()

once "automatically encode categorical-looking column types" is implemented, this should also including converting categorical-looking columns with the same rules

[ ] allow passing a column name for init_score in lgb.Dataset() if data is a data.frame
[ ] allow passing a column name for label in lgb.Dataset() if data is a data.frame
[ ] allow passing a column name for weight in lgb.Dataset() if data is a data.frame

References
See #4207 for a prior proposal and https://github.com/microsoft/LightGBM/pull/4207#pullrequestreview-659266142 for reference on this issue.
This feature precedes #4295.

I've added this feature request to #2302, per this project's standard approach for managing its backlog.
If you are interested in contributing to this feature or have additional information to add, please leave a comment and the issue can be re-opened.

Want to add that when this feature is picked up, the R package should support categorical features for data frames using the same interface as the Python package:

setting categorical_feature = "auto" in Dataset (the default) means "automatically detect categorical features"
if categorical_feature in Dataset is a vector of feature indices or names, use that list of features for categorical_feature instead

https://github.com/microsoft/LightGBM/blob/8a90ea3f267a81a529e3f069cc13e0f6320e7989/python-package/lightgbm/basic.py#L1127
https://github.com/microsoft/LightGBM/blob/8a90ea3f267a81a529e3f069cc13e0f6320e7989/python-package/lightgbm/basic.py#L534-L540
https://github.com/microsoft/LightGBM/blob/8a90ea3f267a81a529e3f069cc13e0f6320e7989/python-package/lightgbm/basic.py#L555-L556

The tricky thing here is that we would need to store factor levels in the Booster object in order to allow for save predictions. This could be a named list like xlevs = list(gender = c("m", "f"), age = c("young", "old")).

Sorry, this was locked accidentally. Just unlocked it. We'd still love help with this feature!


Key Vault support for .NET RSA Crypto primitives

Many applications in .NET need an RSA derived object for their crypto functions. Today, there is no built-in way of doing this with keys/certificates stored in Key Vault. I have written an implementation for this, but I believe it belongs in this SDK.
Key Vault does have its own crypto abstractions, but that does not address the need to use Key Vault in places that are not "key vault crypto" aware.
I'm happy to discuss API names, shapes, parameters, but I have this working already here:
https://github.com/novotnyllc/RSAKeyVaultProvider
The RSAKeyVault enables you to use secrets and certificates stored in an
Azure Key Vault for performing signing and decryption operations. (Encrypt and verify
can be done locally with the public key material.) The type derives from RSA so can
be used anywhere an AsymmetricAlgorithm can be, including with SignedXml types.
I would like to contribute this as a PR into the Key Vault SDK but wanted to start a discussion here before sending a PR.

Ping?

Definitely something that would be worth having in the Azure SDK :+1:

@onovotny Hi Oren, we understand what you need but I think we'd implement this differently than you have. For example, most likely I'd modify the KeyVaultKeyResolver and KeyVaultKey from the AKV Extensions package to return a key that subclasses RSA (and also implements IKey). With this model, we could allow for Secrets that contain either public/private certificate or for direct references to Keys. So, basically:
IKey key = keyResolver.ResolveKeyAsync( keyIdentifier );
RSA rsa  = key as RSA;
We are making some changes to the SDK at the moment and may be able to combine this work. I don't think we'd accept your code as-is because we already have the KeyResolver model.

Thanks. Is the work to do this the way you're suggesting something you'd do or is it something I can help with?
What I have is just one way it could work.

We have some work going on in the SDK at present that this work should either merge with or build on top of. No ETA at present as we are resolving some compatibility issues.

This should now be implemented with PR #7662 which builds on #7176. Please feel free to reopen if not along with what you feel is missing.

@heaths I just looked at the PR/code and it's not clear to me how I'd get an RSA-derived object back from the client that I can pass into other API's?

The private parameters are only necessary if you need the private key, e.g. signing, decrypting, and unwrapping (for RSA). If you just need a raw RSA object, you can use KeyClient to get one, though the private key wouldn't be returned. For that, there's the new CryptographyClient we're introducing in preview 4. Not only does it provide simple methods for signing and encrypting with an RSA private key on Key Vault, for all corresponding public-key operations it can do so locally: better for throughput and controlling costs. Sending every request to the KV could get expensive and will certainly affect performance.
Seems like what you're wanting, then, is a wrapper around CryptographyClient that extends RSA?

Seems like what you're wanting, then, is a wrapper around CryptographyClient that extends RSA?

Yes, exactly.

@heaths that's also what I've already written https://github.com/onovotny/RSAKeyVaultProvider/tree/master/RSAKeyVaultProvider -- with the added support for providing the public certificate and carrying that around efficiently. I think that kind of thing would be best provided by the SDK instead of a 3rd party library like mine.

Reopening. Apologies for the misunderstanding. I think what would make sense for both performance and cost-savings is to provide a wrapper around CryptographyClient that extends AsymmetricAlgorithm or SymmetricAlgorithm since we support RSA, EC, and - soon - AES.

@clairernovotny what was the point of taking, as input, either the JWK or a X509 certificate when all that should be in KV already? I'm working on this over the holiday (now that another woodworking project is wrapping up) and want to make sure requirement are met based on your existing code from which you ported.
The CryptographyClient will attempt to download the key. It won't be able to download the private key by design*, but as long as the caller has "get" permissions on the key it can perform any public key operations locally; if not - or if requiring the private key - the operations are done remotely.
* It's actually stored as a secret and one can download a full certificate with CertificateClient.DownloadCertificate, but that's not a best practice and not something I'd want to enable in this workflow. Won't work with Managed HSM anyway.

@clairernovotny does #33186 fit requirements (at least for RSA, but it's still a draft)?


Event Registration [Android]

Hello, I am using 1.0.0 #dev version and I am getting this error on android
undefined is not an object (evaluating 'taskProvider()(data).then')
I think it has to do with event registering because when I do not setup the player the bug goes away.
Is the new event registering working now ? the one described on #332

If you use old method registerEventHandler then open file index.js in lib folder and change line 109 from
return function() {
to
return async function() {

@tuantai0625 it worked, thank you!

@tailt94 @vbspace I'm using the new method according to the example app. Any idea on how to fix this issue?

TypeError: undefined is not an object (evaluating 'taskProvider()(data).then')


tags: implement functionality to install/switch tag

it should be possible to install a specific tag, like it is possible to switch a branch.
however pushing to tags should be disabled
Update: or perhaps it is okay to push to a tag, as it would just create a branch?

Should we close this?

yeah


feat: add dbt models for ADP Payroll [DE-412]

cc @royconst, all we need to do now is add some dbt operators to the Payroll DAG and these models will do the following:

starting with _raw_payroll, generate an ephemeral model casting all the fields to appropriate datatypes. Note that for the fields that reflect dollar values, I took the liberty of chopping off the leading $  and converting them to floats, and appending _USD to the field names. This is how serious people store money-data in a database (so that they can be treated as numbers you know?)
update incremental model payroll_base using the date-partition-replacement strat we use for tables that run a full refresh each day (note that this means this dbt should run every day regardless of whether new data was ingested in the sync).
materialize payroll_base into a matview in the partner project called payroll that runs SELECT * from payroll_base.

This PR also includes a schema yaml to run a single dbt test that checks for not-null and unique on the _hashid field in payroll_base that was calculated by hashing the combo of all of the non-dollar-amount fields present in _raw_payroll.
This dbt runs locally and you can see the _base table and the partner matview in BigQuery! I will issue a separate PR that will add the dbt tasks to the ADP Payroll DAG.

(converted this to draft because I remembered this table updates incrementally, not full refresh! oops)

okay, NOW this is ready for review - when there's a good time that is (so maybe next week lol) @royconst @jitoquinto @kanelouise

@jitoquinto @kanelouise @royconst - per my recent post in slack, this dbt now prepends _ to three field names that start with a number in the ADP Payroll source data. RIP my afternoon lmao.

My one concern has been answered and the rest lgtm! Glad people will be able to do all the money maths they want now 💸
Left a comment to your response just asking if we can document somewhere (probably on the dag and not the dbt) that we should be truncating in the event we have to run a backfill

I added notes about Backfill mode (including the advisory to truncate the _base table) in the airflow-dags PR: https://github.com/community-tech-alliance/airflow-dags/pull/496/commits/608cd0754b251f1dcaee6c113899478192021b33
Gonna merge this one 🎉


Property cleanup

What kind of change does this PR introduce? (check one with "x")
[x] Bugfix
[] Feature
[] Code style update (formatting, local variables)
[] Refactoring (no functional changes, no api changes)
[x] Other... Please describe:

Fixes:
Fix code to remove extra hop to deprecated method. Update property documentation.

I think we also need to add _ "github.com/project-flogo/core/app/propertyresolver" to the main.go of the engine, right ? I tested the without it and I got the error Failed to create engine: Unsupported property resolver type - env. Resolver not registered. But adding the above import to main.go works.

The question is should we load the built-in ones by default or have them add that import to the engine.json?  If we think it will always be used then we should definitely add the import, if not .. I'm not sure...  @mellistibco any thoughts?

I'm inclined towards loading it by default. Since we are providing those as  built-in, I think it's fair to have them without having to explicitly add that in engine.json/flogo.json .

I think it makes sense to have it build-in, either adding in main.go or import.go or elsewhere that we can import automatically.


bwdb-6515 update popup close button

PR Checklist
Check these before submitting your pull request:

[x] Visual and behavioral components are separated
[x] Exported components are documented with examples
[x] Props have JSDoc comments
[x] All relevant visual sub-components can be overridden via props
[x] Any extra props are spread into the outermost rendered element

Breaking Changes

No changes should be breaking

Changes to Existing Components

Moved the close button on SplitContent.Popup to the middle of the popup, slightly restructuring the DOM to achieve overflow.

New Visual Components

Content component of Popup

New Behavioral Components

None

@safronenko23 I went ahead and did this since Shaun mentioned it.


How to make batch update using FROM (VALUES) ?

How to write this PostgreSQL query with gorm?
UPDATE nodes SET removed_at = _data.removed_at
FROM (VALUES ('1bc80b35-c53c-4773-a7ad-bbaf66d0e956', '2023-09-11 13:00:16.740'),('36fc4528-2e98-48c2-884b-bd02119b8d3a', '2023-09-10 13:00:16.740'))
AS _data (node_uid, removed_at)
WHERE nodes.uid = _data.node_uid

Without using Raw()

Hi!
Please let me know if you still need help with this question.

@anoruxylene hi! It's still relevant :)

@AlexeyBazhin
One of the solutions to execute your PostgreSQL query without using Raw() might be the following one:
package main

import (
	"log"
	"time"

	"gorm.io/driver/postgres"
	"gorm.io/gorm"
)

const dsn = "YOUR_DNS"

type Node struct {
	UID       string `gorm:"primaryKey"` // string for the sake of simplicity
	RemovedAt time.Time
}

func (n *Node) TableName() string {
	return "nodes"
}

func main() {
	db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
	fatalOnError("Failed to init gorm", err)

	nodes := []Node{
		{UID: "1bc80b35-c53c-4773-a7ad-bbaf66d0e956", RemovedAt: time.Now()},
		{UID: "36fc4528-2e98-48c2-884b-bd02119b8d3a", RemovedAt: time.Now().Add(-time.Hour)},
	}
	err = db.Save(&nodes).Error
	fatalOnError("Failed to update nodes", err)

	log.Println("Updated nodes!")
}

func fatalOnError(message string, err error) {
	if err != nil {
		log.Fatalln(message, err)
	}
}

I use Save() method to update existing nodes. This approach executes on conflict update under the hood.

Please let me know if you have any questions regarding the solution.

@anoruxylene Thank you so much!
But I got a bunch of other fields. One of them (not primaryKey) is uuid type. So i have an error "ERROR: invalid input syntax for type uuid: "" (SQLSTATE 22P02)". I presume that Save() method updates all fields of db record, which is not good for my case.

@AlexeyBazhin, Could you please clarify if nodes.uid is a primary or unique key?

@anoruxylene primary varchar :)
I was talking about another field (nodes.cluster_id) - not null foreign key of uuid type

@AlexeyBazhin In this case, let's try the following query instead of Save()
	res := db.Clauses(clause.OnConflict{
		Columns:   []clause.Column{{Name: "uid"}},
		DoUpdates: clause.AssignmentColumns([]string{"removed_at"}),
	}).Create(&nodes)
	fatalOnError("Failed to update nodes", res.Error)

@anoruxylene Thanks, but already know about this one. Can't do this in my project.

@AlexeyBazhin You should have mentioned it in the description of the task. It could have saved us time. So, what is the reason you can't use the approach in your project? And what is the reason not to use Exec() in your case?
Your clarifications are crucial for finding the best solution.

Any one able to find a solution for this in gorm?
As a workaround, I have executed the query by preparing and running with exec method.
        query := `update %s.products AS p set score = v.score from (VALUES %s) AS v(id, score) where v.id = p.id`;
	var placeholders []string
	for id, score := range scoreMap {
		value := fmt.Sprintf("('%s', %d)", id, score)
		placeholders = append(placeholders, value)
	}

	query := fmt.Sprintf(query, schemaName, strings.Join(placeholders, ", "))
	result := c.db.Exec(query)


ks delete default fails

Is this a BUG REPORT or FEATURE REQUEST?:

Uncomment only one, leave it on its own line:

bug

/kind feature

What happened:
ks delete default throws an error after following the kubeflow user guide with kubernetes cluster on minikube
What you expected to happen:
ks delete default deleting the app from the cluster
How to reproduce it (as minimally and precisely as possible):
follow user guide for kubeflow
Anything else we need to know?:
Environment:

ksonnet version (use ks version):
ksonnet version: 0.9.0
jsonnet version: v0.9.5
client-go version: 1.8
Kubernetes version (use kubectl version):
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.3", GitCommit:"d2835416544f298c919e2ead3be3d0864b52323b", GitTreeState:"clean", BuildDate:"2018-02-07T12:22:21Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"", Minor:"", GitVersion:"v1.9.0", GitCommit:"925c127ec6b946659ad0fd596fa959be43f0cc05", GitTreeState:"clean", BuildDate:"2018-01-26T19:04:38Z", GoVersion:"go1.9.1", Compiler:"gc", Platform:"linux/amd64"}

Would you mind posting the error message if you can?

Sorry. Here it is.
Following the user guide on a minikube cluster, ks delete default resulted in Error strconv.Atoi: parsing "": invalid syntax for ksonnet v0.8.0 and in Error Parse major version failed for for ksonnet v0.9.0.
It's related to ksonnet/ksonnet#272 and should have been fixed for v0.9.0.

This appears to be an issue where ksonnet is failing if the Major / Minor Server Version isn't provided. We're not currently handling parsing of the GitVersion - I'll look into getting this fixed.


[5.8] Fix appending path to inline Blade views

When appending the view path to Blade views with only one line and opening tags we should take into consideration for closing the PHP tag first before appending the path. I've added tests which should cover quite a few situations.
Thanks to Sisve for making me aware of this: https://github.com/laravel/framework/pull/28117#issuecomment-480473139

@bzixilu @sisve Feel free to comment if you see any more discrepancies.

Short tags (<?=) would be another discrepancy. And, if short_open_tag is enabled, <? also.

This would be valid PHP code to have in a Blade view too, but would also cause a syntax error with the proposed change:
Hello world
<?php echo 'foo';
/* <?php echo 'bar'; ?> */

Short tags (<?=) would be another discrepancy. And, if short_open_tag is enabled, <? also.

@fitztrev good call. Added this.

This would be valid PHP code to have in a Blade view too, but would also cause a syntax error with the proposed change:

@fitztrev I get an error when I try to render this in a blade view. I believe closing tags are always needed which makes some of my tests unnecessary actually.

Here's valid code that does not work under the latest change.
<?php echo '<?xml version="1.0" encoding="UTF-8"?>';

Here's a failing test for it:
'Echo XML declaration' => [
    '<?php echo \'<?xml version="1.0" encoding="UTF-8"?>\';',
    '<?php echo \'<?xml version="1.0" encoding="UTF-8"?>\'; ?><?php /**PATH foo ENDPATH**/ ?>',
],

@fitztrev Pushed a commit which should take into account all scenarios now. Made use of the tokenizer extension to check for the opening and closing tags. I didn't add a test for the short tags because it will fail if they're not enabled but they should be covered as well now.

@fitztrev thanks for your help with this!


Merging dev branch with codecoverage

Description

Initial merge of dev branch into master to integrate code coverage setup and configuration.

Welcome to Codecov :tada:
Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.
:information_source: You can also turn on project coverage checks and project coverage reporting on Pull Request comment
Thanks for integrating Codecov - We've got you covered :open_umbrella:


Template registration on example data (T2) is not correct

Could be the steps that are not good, or the straightening that produces a wrong warping field...

On another subject, the registration went well so it is not completely related to the straightening:

My guess is that because we know are using only translation (xy) as a first step during the straightening, the deformation needed by the ANTs' landmark-based bspline method is bigger and the mesh size is not high enough to make it work properly at some points along the spinal cord, as you can see on the image below:

Using a pre-registration based on translation followed by rotation, the results are much better.
Results with only translation:

Results with translation + rotation:

However, the intervertebral disks seem to be wrongly oriented. They should be perpendicular to the spinal cord instead of perpendicular to the xy plane.

with gapxy=40 in sct_straighten_spinalcord: 192 sec (162 before)

fixed


Small question.

Hi!
Thanks for this library! After running through it with debug I have a better understanding of crsf. But I have a small question. In the map_chan function I see some "magic" numbers. Could you tell me a little about them? Or maybe you can give me links to articles about crsf parsing?
const fn map_chan(x: u16) -> u16 {
    (1000 + (((x as u32).saturating_sub(191)) * (2000 - 1000) * 2 / (1792 - 191) + 1) / 2) as u16
}

In Betaflight I found a function with some comments explaining the "magic" numbers. Maybe it will be useful for someone.
https://github.com/betaflight/betaflight/blob/master/src/main/rx/crsf.c
STATIC_UNIT_TESTED float crsfReadRawRC(const rxRuntimeState_t *rxRuntimeState, uint8_t chan)
{
    UNUSED(rxRuntimeState);
    if (channelScale == CRSF_RC_CHANNEL_SCALE_LEGACY) {
        /* conversion from RC value to PWM
        * for 0x16 RC frame
        *       RC     PWM
        * min  172 ->  988us
        * mid  992 -> 1500us
        * max 1811 -> 2012us
        * scale factor = (2012-988) / (1811-172) = 0.62477120195241
        * offset = 988 - 172 * 0.62477120195241 =<PHONE_NUMBER>6418548
        */
        return (channelScale * (float)crsfChannelData[chan]) + 881;
    } else {
        /* conversion from RC value to PWM
        * for 0x17 Subset RC frame
        */
        return (channelScale * (float)crsfChannelData[chan]) + 988;
    }
}

Sorry for the confusion. I mapped those values this way because that's how betaflight does it. Now that I think about it, it might be better to return the raw values and leave the mapping to the user.

I think you can do two methods. One can return raw values, and the second can return the values as they are now.

I just released a new version with a better mapping system :))