MINOR:  Add validation to the set_idempotent_data method in GrpcPlacementService and fill in the test case

What type of enhancement is this?
Refactor
What does the enhancement do?
Add validation to the set_idempotent_data method in GrpcPlacementService and fill in the test case.
Tips:
You can implement the ValidateExt trait on SetIdempotentDataRequest, then invoke validate_ext function to achieve the validation functionality.

you can locate the ValidateExt trait in the file: src\placement-center\src\server\grpc\validate.rs
you need to add validation to the set_idempotent_data method in this file:  src\placement-center\src\server\grpc\service_placement.rs
you need to fill in the test case in this file: src\grpc-clients\tests\placement_test.rs

references:

https://github.com/robustmq/robustmq/pull/548)https://github.com/robustmq/robustmq/pull/548

Implementation challenges
No response

please assign me.thanks.


Fix banting issue with multiple versions of angular 2

This fixes https://github.com/CanopyTax/single-spa-angular2/issues/2.
See https://github.com/angular/angular/issues/14343, which is what tipped me off to what was happening.
Changes I made:

Put angular into a separate shared bundle for app2 and app3 with webpack's CommonChunkPlugin.
Got rid of SystemJS because there is no need for it in this case.

I plan on making similar changes for this to the original repository https://github.com/joeldenning/simple-single-spa-webpack-example to hopefully help other people avoid this.

@joeldenning Appreciate all the help! I will give this a try. It does look like how it was before, using ES6 imports. That works on our end too. Where we get the issue is when using SystemJS to load the module locally or over HTTP. Do you have any clues as to why the problem shows up when using SystemJS?


fix type-o

Net::Braintree::ErrorCodes::MerchantAccount::ApplicantDetails::Address had a type-o in the package declaration.  Not sure if it caused any practical issues, but it creates a dependency failure when packaging to RPM w/ rpmbuild.

@resynthesize We have merged this PR and your changes will be included in the next release of our library.
Thank you!

This was published in https://github.com/braintree/braintree_perl/releases/tag/0.29.0


Improvements for "codemon fetch"

When inputs and outputs are written in the .in and .op files, there's an additional newline at the top of each file, which can cause problems in diffing between outputs during codemon listen, therefore, we need to avoid adding that newline either during write or be remove post write.
Improve all verbose messages and use consistent coloring, currently, some of them are green and some are yellow, in a very inconsistent way. Maybe stick to white or some other bright colour which will be good for a basic terminal.
Maybe get rid of codemon fetch <contestName> ? This needs discussion. Currently there are 3 ways to leverage the fetching functionality: codemon fetch inside contest directory, codemon init -f <contestdir>, codemon fetch <contestName>. The last one creates a directory with given contest name and contains only inputs and outputs and no code files. I believe we have already served that purpose with codemon init -f <contestDir>, therefore we can get rid of this command.

@ankingcodes I agree, the command does seem redundant to the current functionality. I think we should remove it.  There are also a few inconsistencies in CodemonFetch module that I want to fix. I'll fix all this in a new PR.
As for the second one I think it can be a separate issue, make coloring of all verbose messages consistent and also add better verbose messages for a better user experience. What do you think?

@Enigmage - Yeah, I agree with this. But I'm not a big fan of long messages. Make them verbose wherever required, but try to keep them concise and to the point.


If last selection from list was “other”, selecting other again does not trigger the prompt to add the name for other (Minister’s Issue).

@calebissharp This might not be an issue once we deal with #338 but can we just make sure that we can enter two “other” issues in a row here?

@calebissharp was this fixed by #34w? If yes, can you assign yourself and move it to In Progress please?


Fix the name in metadata

What type of PR is this?
bug
What this PR does / Why we need it?
The name in the metadata must be the same as the directory name.
Which Jira/Github issue(s) does this PR fix?
Resolves #
Special notes for your reviewer
Pre-checks (if applicable)

[ ] Validated the changes in a cluster
[ ] Included documentation changes with PR

/retest

/lgtm


How to turn on and off without uninstalling.

I'm experimenting with this to see how well I like it. The only way I can see to turn it off is to uninstall it. Is there another way?

Not now sadly :(
Once https://github.com/jwkvam/jupyterlab-vim/pull/40 was implemented then we could have a toggle for this.
You could also disable it:
jupyter labextension disable jupyterlab_vim


bower install command not found

Hi :)
I'm struggling trying to install Bower.
I alredy installed Bower with this command : [$ sudo npm install -g bower]
But when I try to check with: "bower --help" it says that the command is not found.
I tried to add "/usr/local/bin/npm" to my PATH doing: [$ export PATH="/usr/local/bin/npm:$PATH"]
but it is still not working.
Any idea ?
Thankkkssss

Hey. Please ask support questions on StackOverflow. This is development issue tracker.

Thank you


Checksums and/or signatures for release binaries?

Are there checksums or signatures available for release files? I haven't been able to find anything of the sort on the releases page.
Including at least a SHA256 checksum for each release file would be a nice courtesy since a lot of people's deposits will be riding on this code.

just came here to suggest this too. very important!

It's something we've been discussing for a while now (see #103 and https://github.com/ethereum/eth2.0-deposit/issues/156). We plan to implement it soon. Just trying to work out the logistics of who has signing privileges.


Add naming restriction for naming

In order to comply with vague GEANT requirements.
Seems like this one is the restriction:
The practical restrictions of NCName are that it cannot contain several symbol characters like :, @, $, %, &, /, +, ,, ;, whitespace characters or different parenthesis. Furthermore an NCName cannot begin with a number, dot or minus character although they can appear later in an NCName.

My bad, it is not GEANT restriction, it is a part of the schema.


Remove pimcore5 branch from readme, is outdated

Pimcore5 is supported by default in master branch

Thank you!


[WIP]fix(region): optimized azure reset linux pasword

这个 PR 实现什么功能/修复什么问题:

[ ] 功能、bugfix描述
[ ] 冒烟测试
[ ] 单元测试编写

是否需要 backport 到之前的 release 分支:

release/3.5

/area region
/cc @zexi

Codecov Report

Merging #9856 (5c413db) into master (81cb23f) will increase coverage by 2.08%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #9856       +/-   ##
==========================================
+ Coverage    5.98%   8.06%    +2.08%
==========================================
  Files         849     107      -742
  Lines      131946   12991   -118955
==========================================
- Hits         7893    1048     -6845
+ Misses     123351   11823   -111528
+ Partials      702     120      -582

Flag
Coverage Δ

aFlag
8.06% <ø> (+2.08%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/compute/models/scaling_group_resource.go

pkg/util/rbacutils/rbac.go

pkg/cloudprovider/retry.go

pkg/compute/models/guestjoints.go

pkg/multicloud/aws/instance.go

pkg/cloudcommon/db/statusstandalone.go

pkg/compute/models/scalinggroup_network.go

pkg/keystone/models/users.go

pkg/compute/models/natgatewayresource.go

pkg/mcclient/options/k8s/pod_template.go

... and 733 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 81cb23f...5c413db. Read the comment docs.

Codecov Report

Merging #9856 (5c413db) into master (81cb23f) will increase coverage by 2.08%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #9856       +/-   ##
==========================================
+ Coverage    5.98%   8.06%    +2.08%
==========================================
  Files         849     107      -742
  Lines      131946   12991   -118955
==========================================
- Hits         7893    1048     -6845
+ Misses     123351   11823   -111528
+ Partials      702     120      -582

Flag
Coverage Δ

aFlag
8.06% <ø> (+2.08%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/compute/models/scaling_group_resource.go

pkg/util/rbacutils/rbac.go

pkg/cloudprovider/retry.go

pkg/compute/models/guestjoints.go

pkg/multicloud/aws/instance.go

pkg/cloudcommon/db/statusstandalone.go

pkg/compute/models/scalinggroup_network.go

pkg/keystone/models/users.go

pkg/compute/models/natgatewayresource.go

pkg/mcclient/options/k8s/pod_template.go

... and 733 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 81cb23f...5c413db. Read the comment docs.


[BUG] - Unknow weird behavior using NextUI

NextUI Version
2.1.13
Describe the bug
Certain items in NextUI just aren't behaving as expected on my project. One example is inputs, this is what it appears for me on Firefox (I've tried it on Chrome and it's the same)

Other elements have other behavior, the Skeleton doesn't have it's 'shimmer' effect it doesn't display anything, the NavbarMenuTrigger doesn't display at all either. I've tried re-importing my files into a new clean project with no avail, I've tried other computers, I don't have any overwriting css files I only use Tailwind and I cannot figure it out... In the case of that picture it seems the Input background is being set by something, if I set background of input's to 'none' it appears fine but I want to fix the rest of my broken elements. Has anyone run into this issue or have an idea?
Your Example Website or App
No response
Steps to Reproduce the Bug or Issue

Use an Input inside of a Modal
Hovering shows input has a different background

Expected behavior
I expected the UI to mirror the examples in the documentation.
Screenshots or Videos

Operating System Version
Windows 11
Browser
Firefox

All seems to be normal with that version

So maybe you have conflicting css classes in your project


⚠ Bofrost.at has degraded performance

In 1c09b26, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 3109 ms

Resolved: Bofrost.at performance has improved in b5119b2 after 30 minutes.


[region-isolation] Add a test specifically for ownership.

Just expanding code coverage.

@swift-ci smoke test


Exasol

The numberic type is not found with Exasol DB,
i found the solution and its works for me:
odbc.cpp:
switch ((int) column.type) {
case<PHONE_NUMBER>:
case SQL_INTEGER :
add column type<PHONE_NUMBER> as integer
Garry Lachman

Sorry...
The Real Solution:
add "SQL_TYPE_TIMESTAMP" to case
change in SQLColAttribute
from SQL_DESC_TYPE  to SQL_DESC_CONCISE_TYPE
for support odbc 2.x
Note from MSDN:
"To work against ODBC 2.x drivers, use SQL_DESC_CONCISE_TYPE instead"

Interesting. I wonder if there is a way to detect if the driver is an ODBC 2.x driver. node-odbc targets ODBC 3.x. Maybe switching to SQL_DESC_CONCISE_TYPE would be fine though. Will have to test.


Woodwork initialization can fail if datetime format is inferred incorrectly

During initialization Woodwork infers the format string for any columns identified as Datetime columns and then converts these columns to datetime64[ns] dtypes using this format string. If the most common format string does not work for all of the data in the series, initialization will fail with:
TypeConversionError: Error converting datatype for dates from type object to type datetime64[ns]. Please confirm the underlying data is consistent with logical type Datetime.

Code Sample, a copy-pastable example to reproduce your bug.
import pandas as pd
import woodwork as ww

dates = pd.Series(["01/01/2017"]*4 + ["13/12/2017"])
df = pd.DataFrame({
    'id': range(5),
    'dates': dates
})

df.ww.init()

The inference and conversion process should be updated so that it does not fail. There are at least two circumstances that could cause this failure:

The sample of data used to infer the format is not large enough to capture the correct format
The most commonly inferred format is not the correct format (as in the example above)

We might need to update the process to be something like this:

Try initialize using the format inferred on a sample of data as we are doing today
If this fails, use all of the data instead of a sample to infer the format strings
If the most common format fails, try all other identified format strings until finding one that works
If no valid format strings are found, raise the error above

Note, for performance reasons we want to try to infer on a sample first, and only fallback to using more data if the sample approach fails.

Another possibility here would be to try the conversion with the inferred format in Datetime.transform, and if that fails raise a Warning and do the conversion without the specifying the format string, which should succeed because at this point we have already identified that this column contains valid datetime data. If we took this approach we would not have knowledge of the format string that was used to convert the data, but the initialization would no longer fail.

perhaps, there are two separate things going on here

did we infer the correct format? or did the user supply the correct datetime
what should we do if we try to parse a series using a particular format and some rows fail

i think this issue is mainly about 2. in that case, i think the expected behavior would be that the rows that cannot be parsed get converted to nan or NaT.
could we accomplish this by using pd.to_datetime(...,errors="coerce"). looking at docs here

@kmax12 The issue here is that we attempt to parse a series using a particular format inferred by Woodwork and some rows fail.
I think using coerce could be good if the user specified the format and some data is not compatible with that format, using NaT for those rows makes sense.
I'm not as sure we should coerce n the case where Woodwork infers a format that is incompatible with all of the data. My reasoning - if we have a series such as the dates series in the example code above that has a "true" datetime format of day/month/year, but Woodwork infers the format month/day/year and we use the inferred format to coerce the data, then we will convert some rows incorrectly and replace others with NaT, when the "right" approach would be to convert all rows correctly using day/month/year.
I guess if we do use coerce with a Woodwork inferred format that causes NaT to be added, we should at least warn the user that this happened so they can review and fix if needed by manually specifying the correct format on initialization.

that is why im trying to separate 1. did we infer or did the user provide the format correctly?  from 2. what happens if the inference is wrong?
the situation you describe seems to me like we need to improve the inference algorithm, but that coerce would be fine. we could also expose the coerce parameter if we wanted the user to have flexibility on this behavior


The requestTimeout setting is invalid

DTM in saga mode, requestTimeout setting does not work when calling dtmsvr/submit. It can be set in the code, right? Can you support it?

https://github.com/dtm-labs/dtm/blob/55f5f9559581ced20439cd9c0f7a47c0c4e1b11b/dtmcli/dtmimp/trans_base.go#L47

ok, I'd fix the issue, and you should use requestTimeout from the transaction.option. The requestTimeout's naming should use snake_case, and the outer layer will omit it because it is not from storage.


SQL editor frame color doesn't change when switching between PROD and DEV connections

System information:

Operating system (distribution) and version: Windows 10 64 bit
DBeaver version: <IP_ADDRESS>206051253

Connection specification:

Database name and version: MySQL RDS 5.7.33
Driver name: MySQL 5.7
Do you use tunnels or proxies (SSH, SOCKS, etc)?  No

Describe the problem you're observing:
When switching from a production connection to a development connection in an SQL editor, editor frame color stays red
Steps to reproduce, if exist:

Create two connections: one with red color for PROD and one with white color for DEV
open SQL editor for the PROD connection and switch to DEV connection

Thanks for the bug report @yoni-yad2

fixed ib #17309


Header consistency in OS

Gray headers should be same size page to page, currently they are smaller on grants and works than on the profile page.
Desktop (please complete the following information):

MacOS
Safari

@Vensberg @wrenaria
This should be live now on https://experts.ucdavis.edu/.


global logger

This PR implements an application-wide logger and migrate print statements to logger calls. Closes #158 .

Since there is only one logger with the name 'application', I think the name parameter in the log format is not needed.
I'd also consider adding the module and function names to the log messages, which may be helpful to have in a large project like this with many different modules. The log format would then be the following (note the capital 'N' in funcName):
'%(asctime)s - %(levelname)s - %(module)s.%(funcName)s - %(message)s'

I like your proposal @keithchev , how do you feel about it @edyoshikun ?

I like that structure because it's easier to localize the call. Thank you both


Nuxt Generate and AWS S3

Hi,
I've read in the GitHub issues that it's supposed to be possible to use npm run generate to create a static site that runs on GH pages or AWS S3.
I've done this and uploaded the files to the bucket. All files are being found/loaded properly after I set the router.base in the config, the json file to populate the page is being loaded as well but none of the interactions are going:
https://s3-eu-west-1.amazonaws.com/thuisverpleging-vlaanderen/index.html
Is this not supposed to work?

You are having the same issue that I am having, which is a routing issue look at your URL. Your URL went to index.html. https://s3-eu-west-1.amazonaws.com/thuisverpleging-vlaanderen/index.html for the interactions . You are having same issues no interactivity and CSS breaking

You are having the same issue that I am having, which is a routing issue look at your URL. Your URL went to index.html. https://s3-eu-west-1.amazonaws.com/thuisverpleging-vlaanderen/index.html for the interactions . You are having same issues no interactivity and CSS breaking.
This is post about this situation #940


FSDP integration enhancements and fixes

What does this PR do?

Fixes #516
integrates new features of FSDP to auto wrap transformer blocks and support for mixed precision

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


[FEAT] I want to add handwritten notes and projects in navbar section also their resources.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

Important Announcement: Closure of GitHub Repository Issues, Website Redesign, and Discord Channel Invitation
We apologize for the inconvenience caused, but we must inform you that all current issues in our repository will be closed. This decision aims to improve our development process and enhance organization.
Excitingly, we are announcing a complete redesign of our website, focused on improving user experience and incorporating new features aligned with our goals.
Please note that approximately half of the existing issues are redundant and will be closed accordingly.
We also invite you to join our Discord channel for project collaboration and communication. Visit GSSoC server and assign yourself Web Resource Hub project role using the self-roles feature.
Join here
Your feedback and support are invaluable to us. We look forward to hearing your thoughts on the upcoming changes.
Thank you for your understanding and continued support.
Sincerely,
Project Admin


Slide bug

Ionic version:  (check one with "x")
[x] 1.x
[ ] 2.x
I'm submitting a ...  (check one with "x")
[x] bug report
[ ] feature request
[ ] support request => Please do not submit support requests here, use one of these channels: https://forum.ionicframework.com/ or http://ionicworldwide.herokuapp.com/
Current behavior:

I use Slide component in my home page as following:
ts:
export class HomePage {
@ViewChild('Slides') slides: Slides;
constructor(){
}
slideChanged() {
	let currentIndex = this.slides.getActiveIndex();
	console.log("Current index is", currentIndex);
}

}
html:
<ion-slides (ionSlideDidChange)="slideChanged()">

Slide 1

Slide 2

Slide 3

I have only three , the index of slides should be 0,1,2. but it show number 3 sometimes.
Expected behavior:

the index of slides should be 0,1,2
Steps to reproduce:

Ionic info: (run ionic info from a terminal/cmd prompt and paste output below):
insert the output from ionic info here

Hello! Thanks for opening an issue with us! As this seems like more of a support question i will urge that you ask this question on our forum or on our slack channel. Thanks for using Ionic!


TLS v1.1 not supported with kong-ingress + cert-manager

NOTE: GitHub issues are reserved for bug reports only.
For anything else, please join the conversation
in Kong Nation https://discuss.konghq.com/c/kubernetes.

Summary
I think this is an issue with kong-ingress-controller because when I look at the config for kong-proxy containers it shows the nginx_http_directives ssl_protocols as TLSv1.1 TLSv1.2 TLSv1.3.
Kong Ingress controller version
0.6.0
Kong or Kong Enterprise version
1.3
Kubernetes version
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.9-eks-c0eccc", GitCommit:"c0eccca51d7500bb03b2f163dd8d534ffeb2f7a2", GitTreeState:"clean", BuildDate:"2019-12-22T23:14:11Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}
Environment

Cloud provider or hardware configuration: AWS EKS
OS (e.g. from /etc/os-release): amazon-linux2
Kernel (e.g. uname -a):
Install tools: eksctl for cluster, helm for kong ingress
Others:

What happened
kong-ingress-controller paired with cert-manager supports only TLS 1.2+ and not TLS 1.1, which is configured in kube-proxy
Expected behavior
TLS 1.1 to be supported
Steps To Reproduce

follow kong + cert-manager guide and then use ssllabs to verify TLS versions accepted.

TLS 1.1 is enabled by default.
You should use this property instead: https://docs.konghq.com/2.0.x/configuration/#nginx_http_ssl_protocols
And specifying that environment variable, you need to use KONG_ prefix.

Yes sorry, I did set the environment var in the helm chart (confusuing that the KONG_ prefix isn't needed here until I dug in) but I'm still only getting 1.2 and 1.3 which can be verified by ssl labs. https://www.ssllabs.com/ssltest/analyze.html?d=webapi-dev.blackbookcloud.com&s=<IP_ADDRESS>&latest
Since TLSv1.1 TLSv1.2 TLSv1.3 is default I added TLSv1 also to confirm that the block was updated in the .conf - it was - but still only 1.2 and 1.3 are showing up in ssllabs, and a 1.1 call fails to the site.
The architecture is NLB (level 4) with two tcp listeners on 80 and 443 (no tls) into kong-ingress-controller which terminates tls on 443 to all http services/routes.
Not sure if you guys are dog-fooding with konghq.com, but it also does not support 1.1

For further validation I created a blank EKS cluster.

installed external-dns via helm
installed kong-ingress using this values.yaml
installed cert-manager and created a ClusterIssuer
created the echo service mentioned in the cert-manager guide (curl -sL bit.ly/echo-service | kubectl apply -f -)
created the ingress

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: echo-bbc-com
  annotations:
    kubernetes.io/tls-acme: "true"
    cert-manager.io/cluster-issuer: dns-letsencrypt-prod
spec:
  tls:
  - secretName: echo-bbc-com
    hosts:
    - echo.blackbookcloud.com
  rules:
  - host: echo.blackbookcloud.com
    http:
      paths:
      - path: /
        backend:
          serviceName: echo
          servicePort: 80

cannot use TLSv1 or TLSv1.1
TLSv1
❯ curl -v -I --tlsv1.0 --tls-max 1.0 https://echo.blackbookcloud.com
* Rebuilt URL to: https://echo.blackbookcloud.com/
*   Trying <IP_ADDRESS>...
* TCP_NODELAY set
* Connected to echo.blackbookcloud.com (<IP_ADDRESS>) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS handshake, Client hello (1):
* TLSv1.0 (IN), TLS alert, Server hello (2):
* error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure
* stopped the pause stream!
* Closing connection 0
curl: (35) error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure

TLSv1.1
❯ curl -v -I --tlsv1.1 --tls-max 1.1 https://echo.blackbookcloud.com
* Rebuilt URL to: https://echo.blackbookcloud.com/
*   Trying <IP_ADDRESS>...
* TCP_NODELAY set
* Connected to echo.blackbookcloud.com (<IP_ADDRESS>) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.1 (OUT), TLS handshake, Client hello (1):
* TLSv1.1 (IN), TLS alert, Server hello (2):
* error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure
* stopped the pause stream!
* Closing connection 0
curl: (35) error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure

TLSv1.2
❯ curl -v -I --tlsv1.1 --tls-max 1.2 https://echo.blackbookcloud.com
* Rebuilt URL to: https://echo.blackbookcloud.com/
*   Trying <IP_ADDRESS>...
* TCP_NODELAY set
* Connected to echo.blackbookcloud.com (<IP_ADDRESS>) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=echo.blackbookcloud.com
*  start date: Feb 25 23:31:38 2020 GMT
*  expire date: May 25 23:31:38 2020 GMT
*  subjectAltName: host "echo.blackbookcloud.com" matched cert's "echo.blackbookcloud.com"
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
> HEAD / HTTP/1.1
> Host: echo.blackbookcloud.com
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Content-Type: text/plain; charset=UTF-8
Content-Type: text/plain; charset=UTF-8
< Connection: keep-alive
Connection: keep-alive
< Date: Wed, 26 Feb 2020 00:51:23 GMT
Date: Wed, 26 Feb 2020 00:51:23 GMT
< Server: echoserver
Server: echoserver
< X-Kong-Upstream-Latency: 2
X-Kong-Upstream-Latency: 2
< X-Kong-Proxy-Latency: 0
X-Kong-Proxy-Latency: 0
< Via: kong/1.4.3
Via: kong/1.4.3

<
* Connection #0 to host echo.blackbookcloud.com left intact

This test was using the default ingress (elastic/classic load balancer), I also destroyed the kong-ingress helm installation and recreated using an nlb with the same results.

Can you set the environment variable on proxy container:
KONG_NGINX_HTTP_SSL_PROTOCOLS="TLSv1.1 TLSv1.2 TLSv1.3"

What happens then?

Its already set from the helm chart
<<K9s-Shell>> Pod: kong-ingress/kong-kong-64bbf6789f-88jtj | Container: proxy
/ $ echo $KONG_NGINX_HTTP_SSL_PROTOCOLS
TLSv1 TLSv1.1 TLSv1.2 TLSv1.3

Konga connected via kube-proxy shows that this is being consumed correctly.

This is the cluster / ingress that's hosting the site that I provided ssl labs for previously [here].(https://www.ssllabs.com/ssltest/analyze.html?d=webapi-dev.blackbookcloud.com&latest)

@dndx @fffonion Any idea what is causing this problem?

Sorry, fat fingered and closed the issue.

Could you share the kong error log when sending tls1.1 and tls1.0 requests?

sure, is log_level just set as an environment variable?

2020/02/26 19:13:17 [info] 24#0: *31375 SSL_do_handshake() failed (SSL: error:1417A0C1:SSL routines:tls_post_process_client_hello:no shared cipher) while SSL handshaking, client: <IP_ADDRESS>, server: <IP_ADDRESS>:8443
This is the only line that appears when curling.

Wondering if https://docs.konghq.com/2.0.x/configuration/#ssl_cipher_suite is the cause?
If TLS 1.1 is a requirement, the suite will need to be set to "old". The default of "modern" (Kong 1.3) or "intermediate" (Kong 2.0) will cause the observed behavior. https://wiki.mozilla.org/Security/Server_Side_TLS

@rparsonsbb please try setting env.ssl_cipher_suite="old" and let us know if this resolves it?

env.ssl_cipher_suite="old" has corrected the TLS issues, when configured now with both nginx_http_ssl_protocols="TLSv1 TLSv1.1 TLSv1.2 TLSv1.3" and ssl_cipher_suite="old" kong-ingress will serve TLSv1 through TLSv1.3.
Thanks so much to all of you for your help, I'll write up a guide and submit a PR in case anyone else runs into this issue again.


Update pymysensors version to 0.7

Description:
Update pymysensor to version 0.7 that allows for mysensors api version 2.0.0
Related issue (if applicable): fixes #
Pull request in home-assistant.io with documentation (if applicable): home-assistant/home-assistant.io#
Example entry for configuration.yaml (if applicable):

Checklist:
If user exposed functionality or configuration variables are added/changed:

[ ] Documentation added/updated in home-assistant.io

If code communicates with devices, web services, or a:

[ ] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[ ] New dependencies have been added to the REQUIREMENTS variable (example).
[ ] New dependencies are only imported inside functions that use them (example).
[ ] New dependencies have been added to requirements_all.txt by running script/gen_requirements_all.py.
[ ] New files were added to .coveragerc.

If the code does not interact with devices:

[ ] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[ ] Tests have been added to verify that the new code works.

Hi!
You need to run script/gen_requirements_all.py also, to update requirements_all.txt.
Do you want to finish this, or should I? I'm working on several PRs for mysensors in home assistant.

Hi!
I'd like to help but when I can! did the requirements change? I saw in the log that the attribute 'reset' is missign in the latest commit. That's why i closed the PR.

Since you have changed the requirements in the component, you also have to update the list of all requirements by running the script gen_requirements_all.py, and include the result in your commit.
What reset attribute do you mean?

I'll try it again and report on the maybe missing attribute :)

ok seems to work now! Maybe I had an issue with my virtualenv and the version of pymysensor

DAMN IT:
the error came back:
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
    self.run()
  File "/home/hass/.homeassistant/deps/mysensors/mysensors.py", line 628, in run
    response = self.handle_queue()
  File "/home/hass/.homeassistant/deps/mysensors/mysensors.py", line 347, in handle_queue
    reply = func(*args, **kwargs)
  File "/home/hass/.homeassistant/deps/mysensors/mysensors.py", line 177, in logic
    ret = self._handle_set(msg)
  File "/home/hass/.homeassistant/deps/mysensors/mysensors.py", line 82, in _handle_set
    if self.sensors[msg.node_id].reboot:
AttributeError: 'Sensor' object has no attribute 'reboot'

all clear - sorry for the mess!
You have to delete the pickle file. Maybe we should add a version-number to the pickle file and destroy it when it doesn't match.

Thanks for the report! I'll have to look into this, and see if we can fix it for a seamless experience.

Do you still have an old pickle file to test with? I've made a PR here with a fix:
https://github.com/theolind/pymysensors/pull/64
My tests show good results but a live test would be great. Could you test that branch and see if it fixes it for you too?

Conclusion to the thread: Hotfix and bumped version to 0.7.1 in https://github.com/theolind/pymysensors/pull/66.


Add norm(::EmptyStorage)

Add norm(::EmptyStorage) = 0.0.

Codecov Report

Merging #699 (61dfb67) into main (145d530) will decrease coverage by 1.81%.
The diff coverage is 60.00%.

@@            Coverage Diff             @@
##             main     #699      +/-   ##
==========================================
- Coverage   79.51%   77.70%   -1.82%
==========================================
  Files          81       81
  Lines        8910     7525    -1385
==========================================
- Hits         7085     5847    -1238
+ Misses       1825     1678     -147

Impacted Files
Coverage Δ

src/NDTensors/empty.jl
56.04% <33.33%> (-22.01%)
:arrow_down:

src/NDTensors/tensor.jl
68.59% <100.00%> (-6.97%)
:arrow_down:

src/NDTensors/tensorstorage.jl
64.28% <100.00%> (-10.14%)
:arrow_down:

src/NDTensors/octavian.jl
0.00% <0.00%> (-100.00%)
:arrow_down:

src/NDTensors/similar.jl
33.33% <0.00%> (-30.31%)
:arrow_down:

src/NDTensors/dims.jl
42.85% <0.00%> (-28.58%)
:arrow_down:

src/broadcast.jl
60.16% <0.00%> (-25.56%)
:arrow_down:

src/physics/site_types/tj.jl
53.22% <0.00%> (-13.45%)
:arrow_down:

src/qn/qnindexset.jl
86.66% <0.00%> (-13.34%)
:arrow_down:

... and 70 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 145d530...61dfb67. Read the comment docs.


Can't access Google Drive files.

I'm trying to get VCR data in google drive address: gs://merlotreserve/finetune_data/vcr/,  and I find AccessDeniedException because I cannot access to your Google Cloud Storage bucket using my personal account. Is there any way to get permission to access the VCR data?

hey! sorry for the delay but I just checked and the permissions seem to have been public the whole time. lmk if you weren't able to get access and we can troubleshoot 😄


feat: add arrow-import runtime semantic

The import dependency of programs is not a tree, but a DAG (Directed Acyclic Graph). For example, an utility function may be used by many scopes. A tree structure cannot handle this, as there's only one parent for a tree node.
Now we can add an arrow from one scope to another to mark a manual import relation.

Using arrows occasionally or in short distances is fine. But

the repo may be prone to become go-to.
drawing arrows is a laborious mouse operation, unscalable
too many arrows make the canvas messy.

So besides arrows, we can have use some code-based solution. For example,  enrich the @export decorator, like @export('siblings') or @export('recursive')


How to generate the mask gt_map?

I'm just a beginner,when I want to read your code, I couldn't find the file which is used to generate mask gt_map by the ann. Could you help me ?

I'm just a beginner,when I want to read your code, I couldn't find the file which is used to generate mask gt_map by the ann. Could you help me ?

have you found it?


[WIP] topology: support exclusion of metadata fields

This issue related to:

removing some of the K8s fields (to save storage space; and reduce UI contention)
issue discussing removing flow fields https://github.com/skydive-project/skydive/issues/1439

One can filter out fields from graph.Node.metadata using the following:
analyzer:
  topology:
    exclude_fields:
    - a.b.c
    - x.y.z

since metadata no longer is a multi-level map this implementation is not longer relevant.


Widgets panel keeps loading forever until you go to another tab and then go back

STR

Create a ticket with 2 layers (one with points [A0], another with polygons [B0], for example).
In the layers list, click over one of the sources (A0) and drag out in order to create another layer pointing to that A layer.
Something like this:

Reload the browser.
Open the chrome console.
Go to widgets panel.
Some requests are "cancelled" and loading state doesn't disappear.

Already fixed with last changes about editor initialization.


Write middleware

Write the middleware that will check up an user's role and give an access or not.

It's done.


Add ability to add custom env vars to the environment.

There is currently no way to add custom env vars to the environment that the services use that doesn't require a code change.
This feature should be added to the cli, such that new env vars can easily be added or changed if custom services are being used with the cli.

This will be included in #145


Add task title as slug to URL

Task URLs can be shared using:

https://automaat.test/#/task/11
https://automaat.test/#/task/12

This works, but if you are sharing with others, and potentially share multiple tasks, it makes it easier to understand what is what by having the task title slug be part of the url:

https://automaat.test/#/task/11-customer-capabilities
https://automaat.test/#/task/12-find-customer-uuid

We should probably add a slug column to the database, to guard against breaking links when renaming tasks (#37). Another solution would be to add another table to keep track of historical task names, and support redirecting from old names to new names, as long as no task takes the name historically used by another task.


Fast removing letters from the search field makes the browser go back in history

This is actually caused because the backspace key makes the browser go back in history, and because removing a letter causes an immediate search, the input gets blocked and stops receiving events, thus the backspace key triggering a "back in history" event.
One possible solution is to delay the start of the filter by a few ms.

This is fixed in 6.x version.


Preparation for 1.2.0 release

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 95.014%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
343

Relevant Lines:
361

💛  - Coveralls


FEATURE ShopifyExtension - allow CartOptions to be overwritten or reconfigured

Ideally we'd be able to update the Cart Options to allow us to override styles, colors, etc.

Done in https://github.com/dynamic/silverstripe-shopify/pull/138 with extension points


Enable processing from the registered schema in the generator package's renderer (for internal use only)

implement one of  https://github.com/pdfme/pdfme/issues/246
ref: https://github.com/pdfme/pdfme/issues/213

What was done in this PR are two things.
The first was creating something called rendererObj, which allows for the future expansion of PDF creation processes.
The second was the creation of the generator/src/renderers folder to separate the PDF rendering processes for each schema.

I am aware that this PR conflicts with the following PR, but I would like to merge this PR first if the fix is still time-consuming.
https://github.com/pdfme/pdfme/pull/256
Similar adjustments will need to be made to the UI, and I believe that those changes will be significant, similar to this PR.

Hey @peteward @steffancarrington ,
The review is optional. I have no intention of taking up your valuable time.
However, since it is a significant revision, I have assigned it to you because I want you to understand what I am trying to do and to let me know if there is anything amiss.
Thank you.

I am aware that this PR conflicts with the following PR, but I would like to merge this PR first if the fix is still time-consuming. #256

Hi @hand-dot, I can work on the fixes to #256 later today / tomorrow.  If you still want to merge this PR first then we will rebase and work it out!
I will read this PR now. 👀


WIP: Support caching dbt ls cache in Airflow Variable

Cache only the dbt ls as an Airflow Variable. This is an alternative to #992, when we cached the pickled DAG/TaskGroup into the local Airflow node.
For the example DAGs tested, it reduced the task queueing time significantly (from 20s to 1s), and the total DAG run time for JaffleShop from 1 min 25s to 40s
To enable this feature, users should set the environment variable AIRFLOW__COSMOS__EXPERIMENTAL_CACHE=1.
Users can purge the cache via Airflow UI.
As a reference:

A dbt project with 100 models can lead to a dbt ls output of 257k characters when using JSON. Gzip could compress it by 20x.
Another real dbt project with 9,285 models led to a dbt ls output of 8.4 MB, uncompressed.
Maximum cell size in Postgres: 20MB

What is missing (why this is draft PR):

Add logic to decide if cache changed or not, similar to #992
Allow users to customize their own calculate_current_version
Write tests
(TBD) Enable the feature by default before 1.5 release (TODO: log ticket)
Variables should be prefixed with Cosmos
Add a timestamp when the cache was last updated
Reduce the size of the variable in dbt (it's currently quite verbose) - check options to compress it

Future work:

How this will affect the Airflow db in the long term
How does this performance compare to ObjectStorage?

Closes: #990
Task queue times in Astro before the change:

Task queue times in Astro after the change, on the second run of the DAG:

This experimental feature is available in astronomer-cosmos==1.5.0a4.

Looks promising!

@kzajaczkowski, thanks a lot! The feature is currently available in the 1.5.0a7 release. Would you be interested in testing it out and giving early feedback?

@kzajaczkowski, thanks a lot! The feature is currently available in the 1.5.0a7 release. Would you be interested in testing it out and giving early feedback?

@tatiana, we did some preliminary testing using 1.5.0a4 release and observed one of a simple dags go from 00:13:45 to 00:02:11. Queuing times went almost to zero. The  Experienced also pytest dag tests go from a few minutes to seconds.

@tatiana A little late to this PR, but not late to 1.5.0:
I'm thinking the cache_identifier and cache_dir should go in the render_config, since the render config is supposed to handle all the options for things like this.
I think, going into something like Cosmos 2.0, it makes sense to consolidate the DbtGraph() init function's signature to something simple like:
class DbtGraph:

    load_method_mapping: dict[LoadMode, Callable[[], None]] = {}

    def __init__(
            self,
            project_config: ProjectConfig | None = None,
            profile_config: ProfileConfig | None = None,
            execution_config: ExecutionConfig | None = None,
            render_config: RenderConfig | None = None,
            **kwargs
    ):

as per the code example in #895.
Adding more kwargs means either we cannot do this, or we need to add more deprecations.
I think doing this will also help make the API cleaner, more consistent, and easier for users to reason about. Right now there are just a lot of things for users to tweak and it can be overwhelming. Keeping it consistent and locked inside the configs can reduce confusion.
WDYT?

@tatiana A little late to this PR, but not late to 1.5.0:

Hi @dwreeves, Thanks a lot for all the very relevant feedback! I'm sorry I missed it, as it was made after the PR was merged.

I'm thinking the cache_identifier and cache_dir should go in the render_config, since the render config is supposed to handle all the options for things like this.

We can expose them there as part of a follow-up PR / future version of Cosmos. Just added: #1110
I'm also happy with the proposal to refactor the DbtGraph interface and #895. Would you be interested in working on this?
I created a PR to address the feedback on the dead code: #1111.
Your proposal to refactor the load implementation in #1001 is very good as well. Would you like to work on it?

Your proposal to refactor the load implementation in https://github.com/astronomer/astronomer-cosmos/issues/1001 is very good as well. Would you like to work on it?

I would, but I've been busy... still trying to find time to contribute to this project. ☹ The time will eventually come.


New package: Unibo.Alchemist version 30.0.5

[ ] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?
[x] This PR only modifies one (1) manifest
[x] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[x] Does your manifest conform to the 1.6 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

Automatic Validation ended with:

No errors to post.

(Automated response - build 729.)

Hi @Zimbrando,
This package installed normally on a Windows 10 VM. But when launching, it does not appear to provide any output - to the console, nor to the system logs, nor any log files in its directory that I could find. Is this expected? Is this application supposed to be used in a different way?

Hi @Zimbrando,
This package installed normally on a Windows 10 VM. But when launching, it does not appear to provide any output - to the console, nor to the system logs, nor any log files in its directory that I could find. Is this expected? Is this application supposed to be used in a different way?

Hi,
It is expected to run from command line and provide an help output that explains how to use it. I am trying to figure out how to add it in the system path correctly, so yes right now it's not working as expected. Do I have to mess with the manifest file or the msi package to accomplish this?

@microsoft-github-policy-service agree

@stephengillie There seems to be some errors in the configuration and I have figured out how to add the the path in the environment variable. If it is ok, you can approve the merge and then later on i will update the msi package to a newer version. Otherwise we can close this pull request.

I'll need to review this further. But if the package has a configuration issue, then that issue should be resolved, and then the package can be submitted.

Hi @Zimbrando,
Sorry about the delay.

@ stephengillie There seems to be some errors in the configuration and I have figured out how to add the the path in the environment variable.

These seem essential for normal program operation, so we shouldn't add the manifest until after they've been added.

If it is ok, you can approve the merge and then later on i will update the msi package to a newer version. Otherwise we can close this pull request.

I'll close this PR for you. Having a version which requires this level of post-install configuration might be a bad experience for users. They might form a negative opinion of this application based on this version.


Special-case JSDoc @example in intellisense popover

VSCode Version: Code 1.13.1 (379d2efb5539b09112c793d3d9a413017d736f89, 2017-06-14T18:13:05.928Z)
OS Version: Darwin x64 16.7.0
Extensions: a tonne

Steps to Reproduce:

Add @example then a string on the next line to the JSDoc
Check out the popover

Reasoning:
There are ~700-800 uses of this in definitely typed if you exclude test code

It'd be great to have them with a code font, and with syntax highlighting.

What confuses me is that sometimes this turns into a single line too, I assume this is some markdown processing issue:
        /** Just a simple sugar function for:
         *
         * @example
         * let valueReturned = false;
         * jest.fn(() => {
         *   if (!valueReturned) {
         *     valueReturned = true;
         *     return value;
         *   }
         * });
         */
        mockReturnValueOnce(value: any): Mock<T>;

When highlighting gives:

Which you can force to be mulitline via the triple backticks surrounding the code:

Still seeing the string left-trimmed though, so indentation doesn't work.

Yes the fenced code block approach should work but it looks like the TypeScript server which powers our JS and TS language support is stripping the indentation from the code block. I've opened https://github.com/Microsoft/vscode/issues/30062 to track this upstream

I have a work around for this:
you can try the following comment and it will indent it.
/** This is a description
* with each line
* on it's own
*
* @example Test
* javascript      * let valueReturned = false;      * jest.fn(() => {      *           if (!valueReturned) {      *             valueReturned = true;      *             return value;      *             //no idea why it requires me to have so many spaces...      *           }      * });      *
*/
so you can indent it, but it takes quite a few spaces before it kicks in.

so it isn't entirely true that it is trimming it, I am actually not exactly sure what the process is on it, but you can make it work with a few more extra spaces.

Based on the jsdoc documentation, I agree that we should display @example tag contents as code


Update flink cluster to use a supported dataproc version (1.2 -> 1.5)

Tested the gcloud command and it starts a supported 1.5.53 (12/18) cluster. 1.2.x line was deprecated for while.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

Run Load Tests Python ParDo Flink Batch


[Event Request] Codeunit 7313 Create Put-away - OnAfterGetDefaultBin

In the Create Put-away codeunit (7313) is it possible to have a new event in the code procedure as follows:
//Existing Code
end else begin
Clear(Bin);
if WMSMgt.GetDefaultBin("Item No.", "Variant Code", "Location Code", Bin.Code) then
Bin.Get(Location.Code, Bin.Code);
QtyToPutAwayBase := RemQtyToPutAwayBase;
CalcQtyToPutAway(false, false);
end;
//New Code
end else begin
Clear(Bin);
if WMSMgt.GetDefaultBin("Item No.", "Variant Code", "Location Code", Bin.Code) then
Bin.Get(Location.Code, Bin.Code);
OnAfterCodeGetDefaultBin("Item No.", "Variant Code", "Location Code", Bin) then
QtyToPutAwayBase := RemQtyToPutAwayBase;
CalcQtyToPutAway(false, false);
end;
Currently there are events to handle the selection of a bin for directed pick and put-away scenarios but it is impossible to influence the bin selection for non directed pick and put-away scenarios The proposed event will allow the default bin to be altered by passing the selected bin record.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


feature: aerospace-sensible-terminal to open Terminal via AppleScript

The example config currently suggests to open a new Terminal using the following command:
open -n /System/Application/Utilities/Terminal.app
However, every time this command is run the Dock becomes more polluted with Terminal icons, which doesn't go away, even if you exit out of a Terminal.
A better example would be to use AppleScript to open Terminal like so:
osascript -e 'if application "Terminal" is running then' -e 'tell application "terminal"' -e 'do script ""' -e 'activate' -e 'end tell' -e 'else' -e 'tell application "Terminal"' -e 'activate' -e 'end tell' -e 'end if'
This has a couple of advantages:

Only one Terminal icon will be in the Dock at all times.
If you check 'Command Key' visibility in the title by going to Terminal->Settings->Profiles->Window->Title, you'll be able to jump between Terminal windows, even in a tabbed or stacked layout, by using the keyboard shortcut ⌥⌘N, where N is the Terminal number.

The disadvantage is that you now have ugly AppleScript within the config.
i3 has the script i3-sensible-terminal to launch a terminal based on what is defined in $TERMINAL and I think aerospace would benefit to have this feature as well.
You could hide the details of opening Terminal with AppleScript inside of it and the example config would then simplify to:
alt-enter = 'exec-and-forget aerospace-sensible-terminal'

Thanks for the well-written issue!
And special thanks for listing the AppleScript monster. I'd not bother trying to write it/Google myself
Your idea makes sense, but I'm not sure that I want to abstract the AppleScript monster in a special aerospace sensible-terminal command. The potential problem is that this monster might not be handling all the cases, and further bugfixes would need to be done in the script

No problem. Thanks for all your effort in making MacOS actually usable for those of us used to working in i3.
I'll leave the expanded AppleScript below in case anyone finds this issue and wants to add it to their bin directory:
#!/usr/bin/osascript

if application "Terminal" is running then
    tell application "terminal"
        do script ""
        activate
    end tell
else
    tell application "Terminal"
        activate
    end tell
end if


Components of a big data architecture

In text accompanying the big data architecture diagram, I dont see an explanation for the machine learning box and it's two way links with batch processing and stream processing. Please provide!

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 953d7d0d-651f-a566-1d28-6946c824e1ae
Version Independent ID: 8f5b4264-dcc9-ad28-36fe-45463d15f52a
Content: Big data architectures - Azure Architecture Center
Content Source: docs/data-guide/big-data/index.md
Service: architecture-center
Sub-service: azure-guide
GitHub Login: @EdPrice-MSFT
Microsoft Alias: pnp

@datascience-dst Thanks for the feedback! We will review and update accordingly.

#please-close


file_mtime should handle list of file names, too

Is your feature request related to a problem?
ImageList has a method to load many image files at once. It gets a list of file paths (strings). file_mtime should be able to handle this list of strings.
Desired solution
Return a tuple of last modified times.
Possible alternatives (optional)
No response
Screenshots (optional)
No response
Additional Context (optional)
No response

:tada: This issue has been resolved in version 0.12.0 :tada:
The release is available on:

v0.12.0
GitHub release

Your semantic-release bot :package::rocket:


Revises FS mount

So it will be more idiomatic Rust than C.

I actually think the MountOps should be an enum like
rust enum MountOpts { HostFs(HostFsMountOpts), Devfs(DevfsMountOpts) ... }
Having it this way would make it impossible to forget an option or provide one that isn't accepted by the FS. Also, we could get rid of all the unwraps, readability +1000. What do you think?

I actually think the MountOps should be an enum like enum MountOpts { HostFs(HostFsMountOpts), Devfs(DevfsMountOpts) ... } Having it this way would make it impossible to forget an option or provide one that isn't accepted by the FS. Also, we could get rid of all the unwraps, readability +1000. What do you think? I can refactor it in my PR.

The problem is mount options usually passed from the user as a string. That mean we need some mechanism to parse it from the string. I think your current solution is nullfs is good enough unless we find a better than that.

Oh, ok. The mount data could easily be an enum though, right?

Oh, ok. The mount data could easily be an enum though, right?

It could but I don't think it is better than Any in this case. Trait object should be the right solution here.

Something like this:
struct Mount {
    data: Arch<dyn Filesystem>,
}

trait Filesystem {}

Actually I already tried something like that with this PR but found out it is a massive changes so I decided to revert it otherwise you are going to have a big problem with your nullfs PR.


minor improvements

Add load_feature_annotation(). https://github.com/MoTrPAC/MotrpacRatTraining6moData/issues/43
Use signif() instead of round() to display values in plot titles. #41
Add counts argument to transcript_normalize_counts() and atac_normalize_counts()
to allow user-supplied data.  #43
Add tissues, assays, and cluster arguments to enrichment_network_vis() to
provide an alternate way for users to specify results from MotrpacRatTraining6moData::GRAPH_PW_ENRICH.
Add tests for enrichment_network_vis().  #45
Add note about reproducibility issues to documentation for transcript_normalize_counts().  https://github.com/MoTrPAC/MotrpacRatTraining6moData/issues/41
Move MotrpacRatTraining6moData from Imports to Depends. This means MotrpacRatTraining6moData is also attached (not just loaded) when MotrpacRatTraining6mo is attached.  #40
Replace fetch_object(name_as_string) with get(name_as_string, envir=as.environment("package:MotrpacRatTraining6moData")).  #40
Change URLs for load_methyl_feature_annotation() and load_atac_feature_annotation().  https://github.com/MoTrPAC/MotrpacRatTraining6moData/issues/43
Fix bug in plot_feature_logfc() that prevented epigenetic features from being plotted.
Speed up plot_feature_logfc() for differential epigenetic features.
In get_rdata_from_url(), calculate log fold-change standard errors for METHYL
differential analysis results: logFC_se = logFC/zscore https://github.com/MoTrPAC/MotrpacRatTraining6moData/issues/44

just waiting for next version of MotrpacRatTraining6moData to be ready for doc dependencies

now waiting for MotrpacRatTraining6mo v1.8.0 to be ready


Django admin site not logging in after production

I am deploying my. django app on digital ocean.
The site works fine, but I am not able to login to my admin page even with superuser credentials
I am being redirected to the django admin login page itself.
Please help me out

Hey have you upgraded the django version to maybe 3.2.x?
There is a problem with the upgraded django version in admin site  - I remember it being something to do with 'subtitle' being added.
Downgrade django to 3.1.11 and it should work, I hope?

I'm closing this issue as I'm assuming the original issue was solved. Feel free to reopen with more details if the problem persists.

Some enabled settings in the production.py, such as SECURE_SSL_REDIRECT, SESSION_COOKIE_SECURE, CSRF_COOKIE_SECURE, etc, will get 302 and forward superusers to admin login even with correct credentials.


Run build-ea workflow with latest EA JDK

This will require using the https://github.com/oracle-actions/setup-java

I am OK waiting for Adoptium EA releases. No hurry here ;)

Test early, test often? Would be good to, in this case, start testing with JDK 19.

@mnriem I think I can maintain this workflow ;)


【BUG】Profile does not record while querinh the hive partitions that do not exist

Steps to reproduce the behavior (Required)
1、HIVE node execute:
CREATE TABLE sales (
product_id INT,
product_name STRING
)
PARTITIONED BY (sale_year INT, sale_month INT)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS TEXTFILE
LOCATION 'xxxxx';
2、starrocks fe execute:
CREATE EXTERNAL CATALOG hive_catalog_test
PROPERTIES ("hive.metastore.type"  =  "hive",
"hive.metastore.uris"  =  "thrift://ip:7004",
"type"  =  "hive"
)
set enable_profile=true;select * from sales where sale_year=2023 and sale_month=1;
you can not get the profile in the web ui
This may be the cause of the problem:

If  isProfileEnabled,  the task should be accept even through It's executed in FE.
Expected behavior (Required)
get the profile info
Real behavior (Required)
StarRocks version (Required)
3.3.2-8151398

If I'm locating the problem correctly, I'll fix it

if this sql use FeExecuteCoordinator, then it will not have profile right now. you can add profile for FeExecuteCoordinator. Our Profile has some levels:
Query:
Summary:
Planner:
Execution:
Execution is Be's profile, which is not need for FeExecuteCoordinator. so you can keey other profiles which are built in StmeExecutor::buildTopLevelProfile


tinytga improvements

Hi! Thank you for helping out with Embedded Graphics development! Please:

[ ] Check that you've added passing tests and documentation
[ ] Add a simulator example(s) where applicable
[ ] Add a CHANGELOG.md entry in the Unreleased section under the appropriate heading (Added, Fixed, etc)
[ ] Run rustfmt on the project
[ ] Run ./build.sh (Linux/macOS only) and make sure it passes. If you use Windows, check that CI passes once you've opened the PR.

PR description
This PR is a collection of improvements to tinytga:

Port to nom 5
Color map support
Tests for all image types
Optional footer
Fixes #216

@jamwaffles Should tinytga get it's own changelog or should the updated tinytga version and the associated benefits be mentioned in e-g's changelog?

Add it to the single CHANGELOG.md for now and I'll figure something out when a release happens :)


Monitoring error: "BT_HCI: hci write adv params error"

Hey all,
I am trying to use nkolban's SampleRead.cpp on esp-idf-template on Ubuntu.
I have done the following:

Cloned esp-idf-template to "ble-test" folder
Created components folder in "ble-test" and copied cpp_utils to components folder
Edited and copied main.cpp & SampleRead.cpp into main folder
deleted main.c
modified set(COMPONENT_SRCS "main.c") to set(COMPONENT_SRCS "main.cpp") in CMakeList.txt in main folder
ran make menuconfig and enabled bluetooth & c++ exceptions

There was an error while compiling. I noticed the extra brackets at line 99 & 100 in BLEUUID.cpp and deleted them. Then I was able to compile successfully.
I used nRF on my android phone and was able to connect to my ESP32 board and see that the time increases every time I read on my phone.
Doing 'make monitor' returns the following error:
Toolchain path: /home/james/esp/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc
Toolchain version: crosstool-ng-1.22.0-80-g6c4433a
Compiler version: 5.2.0
Python requirements from /home/james/esp/esp-idf/requirements.txt are satisfied.
MONITOR
--- idf_monitor on /dev/ttyUSB0 115200 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:6164
load:0x40078000,len:10168
load:0x40080400,len:6552
entry 0x40080764
I (28) boot: ESP-IDF v3.2-beta1-151-g960c24057 2nd stage bootloader
I (28) boot: compile time 17:11:40
I (28) boot: Enabling RNG early entropy source...
I (34) boot: SPI Speed      : 40MHz
I (38) boot: SPI Mode       : DIO
I (42) boot: SPI Flash Size : 4MB
I (47) boot: Partition Table:
I (50) boot: ## Label            Usage          Type ST Offset   Length
I (57) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (65) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (72) boot:  2 factory          factory app      00 00 00010000 00100000
I (80) boot: End of partition table
I (84) esp_image: segment 0: paddr=0x00010020 vaddr=0x3f400020 size=0x3c5f4 (247284) map
I (180) esp_image: segment 1: paddr=0x0004c61c vaddr=0x3ff80000 size=0x00000 (     0) load
I (180) esp_image: segment 2: paddr=0x0004c624 vaddr=0x3ff80000 size=0x00000 (     0) load
I (186) esp_image: segment 3: paddr=0x0004c62c vaddr=0x3ffbdb60 size=0x02dd0 ( 11728) load
I (200) esp_image: segment 4: paddr=0x0004f404 vaddr=0x3ffc0930 size=0x00000 (     0) load
I (204) esp_image: segment 5: paddr=0x0004f40c vaddr=0x40080000 size=0x00400 (  1024) load
0x40080000: _WindowOverflow4 at /home/james/esp/esp-idf/components/freertos/xtensa_vectors.S:1685

I (213) esp_image: segment 6: paddr=0x0004f814 vaddr=0x40080400 size=0x007fc (  2044) load
I (223) esp_image: segment 7: paddr=0x00050018 vaddr=0x400d0018 size=0xa44d0 (672976) map
0x400d0018: _flash_cache_start at ??:?

I (466) esp_image: segment 8: paddr=0x000f44f0 vaddr=0x40080bfc size=0x11060 ( 69728) load
I (496) esp_image: segment 9: paddr=0x00105558 vaddr=0x400c0000 size=0x00000 (     0) load
I (496) esp_image: segment 10: paddr=0x00105560 vaddr=0x50000000 size=0x00000 (     0) load
I (503) esp_image: segment 11: paddr=0x00105568 vaddr=0x50000000 size=0x00000 (     0) load
I (522) boot: Loaded app from partition at offset 0x10000
I (522) boot: Disabling RNG early entropy source...
I (523) cpu_start: Pro cpu up.
I (527) cpu_start: Starting app cpu, entry point is 0x4008103c
0x4008103c: call_start_cpu1 at /home/james/esp/esp-idf/components/esp32/cpu_start.c:245

I (0) cpu_start: App cpu up.
I (537) heap_init: Initializing. RAM available for dynamic allocation:
I (544) heap_init: At 3FFAFF10 len 000000F0 (0 KiB): DRAM
I (550) heap_init: At 3FFB6388 len 00001C78 (7 KiB): DRAM
I (556) heap_init: At 3FFB9A20 len 00004108 (16 KiB): DRAM
I (562) heap_init: At 3FFBDB5C len 00000004 (0 KiB): DRAM
I (569) heap_init: At 3FFCAB08 len 000154F8 (85 KiB): DRAM
I (575) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (581) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (587) heap_init: At 40091C5C len 0000E3A4 (56 KiB): IRAM
I (594) cpu_start: Pro cpu start user code
I (277) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
I (347) BTDM_INIT: BT controller compile version [ac67b34]
I (357) system_api: Base MAC address is not set, read default base MAC address from BLK0 of EFUSE
I (427) phy: phy_version: 4000, b6198fa, Sep  3 2018, 15:11:06, 0, 0
W (967) BT_BTC: btc_gatts_arg_deep_copy 8, NULL value
I (967) BLEDevice: create advertising
E (977) BT_HCI: hci write adv params error 0x30

Any suggestions on how to fix this error are appreciated.

I have suggestion. It is related to uninitialized value in advParam variable in BLEAdvertising class. You can find answers here:
https://github.com/espressif/esp-idf/issues/2714#issuecomment-439736822
I will fix it soon in Kolbans library and devs will help with esp-idf. Most issues related to this library can be posted in kolbans repository. Thanks.


Talking of Dataloader as a Cache confuses developers

Hi, I'll keep this fairly brief for now, but try to expand it later.
A common issue I've seen with people using Dataloader is that they either use it as an application-wide cache, or they don't understand how dataloader works at a lower-level, and misunderstand why they are using it.
I feel like this is mainly a documentation issue. Dataloader is a data loading mechanism, where you load that data from is arbitrary, all that Dataloader ensures is that a signal value by a given key always resolves to the same value.
That is, I've seen people create a dataloader instance at application level, and think that this is safe because they're calling userLoader.clear() after changing a user. The problem with this is that that userLoader.clear() isn't broadcast to all other instances in your cluster.
Say you have a node.js server using dataloader, and you horizontally scale it — if you retain data between requests than you can get a case where server1's dataloader has a different local state to server2 which could have a different state from the database (e.g., server3 has written to the database) — this issue is encountered as soon as you go from one process to more than one process, because the dataloader "cache" is in-process-memory, not some sort of shared memory for all processes. (e.g., using memcached for caching would be sharing that memory between servers / processes)
I think there's a better way to communicate what dataloader is doing, and it should be highly promoted that dataloader is best suited to per-request caching (such that all values by a given key resolve to the same value within the one request, such that you don't get data-drift within the one request — sort of a snapshot of the world at that given time).
if the storage for the "cached" state for dataloader actually came from a shared datasource (memcached, redis, etc), then this issue would be avoided completely; but then you're also by-passing the point of dataloader in some ways.
I'd be willing to restructure / rewrite the documentation to be clearer as to split the "what it does" from the "api for doing it" – then again, I'd be fairly inclined to say clear or reset are anti-patterns, and shouldn't be generally used — i.e., don't use them to clear state between requests, only between the instance of that object in one request, e.g.,
1 request start
2  -> userLoader.get(1)
3    -> Users.update(1, { new: "data" })
4       -> userLoader.clear(1);
5          respond with userLoader.get(1)
6          -> request end

i.e., that second userLoader.get call shouldn't return the result from line 2, instead, it should return the fresh result from the database.
Perhaps a better API would be userLoader.get() and userLoader.getFresh() — i.e., we're not talking about a cache, as it's not a "cache" but more a call identity memoization.
I'd be interested in hearing others thoughts here, in particular, let me know if this makes any sense at all, and whether you'd be interested in a PR to redocument this project to help users avoid misunderstandings

I was considering a single dataloader instance at app level, and wanted to use IPC to sync data between worker processes. But i haven't considered about the case when there are multiple servers. So yes, now i have got a clear decision on this. Thank you!

@ThisIsMissEm this is good point, needs to be better documented
That is, I've seen people create a dataloader instance at application level, and think that this is safe because they're calling userLoader.clear() after changing a user. The problem with this is that that userLoader.clear() isn't broadcast to all other instances in your cluster.

Not just that, but dataloader is really designed for just the single request lifecycle – to handle cases where, say you have a article with comments, that you don't fetch the user details for every single user who's commented, and instead, you just fetch once per unique user.
i.e., because of the documentation, people are using dataloader for things which it was not intended, and hence, getting odd & unintended side-effects

@ThisIsMissEm, why would using DataLoader with something like Redis or Memcached go against what DataLoader was intended for? Why not use DataLoader both to batch requests (to avoid the n+1 problem) and also as an application-wide cache using a distribute key-value store?
As long as the business model layer is handling authorization correctly it shouldn't be an issue to share the cache between requests, right?

Indeed! So, essentially we shouldn't talk of DataLoader as a cache, and instead talk of it as a batcher for requests to external services (databases, other services, etc).
If you have a piece of code that has a batch operator that is cache aware, then sure, use that, but talking of dataloader itself as a cache is confusing

Hey @ThisIsMissEm I'd be happy to accept PRs that improved the documentation. Though I'm curious what you would suggest we use instead of "cache".
It's absolutely true that DataLoader does not and should not replace Redis in an architecture. However maybe the confusion is that "cache" is an overloaded term? For instance most would not suggest you can ditch Redis because Browsers have a cache, but also most would not suggest that browser-cache be called something other than a "cache". It seems like the root of the issue is that "cache" can mean a lot of different things based on context. It's really important for the large majority of (appropriate) uses of DataLoader that it caches loads to avoid repeat loads, but improving confusion around appropriate use of DataLoader and it's cache would be awesome.
I'm open to ideas.

I just pushed some initial changes to Readme that hopefully helps make it more clear that Caching behavior should be used in a per-request way.
I'm going to close this issue since we're all in agreement about what needs to be done, though feel free to continue conversation or to keep linking people here who are looking for more information. Any PRs that further improve the language would be really helpful.

@leebyron thanks for the response! I'll try to rework the documentation soon, but it may be a week or two before I get to it.

@leebyron @ThisIsMissEm @migueloller
Let alone the implementation, I think the APIs exposed by dataloader is also suitable for application-level cache, this might be the reason which makes people use it that way.
What do your think of this package that wraps dataloader using redis as the backend, and make the clear method a promise.
I think this lib resolves the problems @migueloller brought out on his comment.


Please, add gradient colours for the line depending on Y value for the LineChart (for example if y > 10 I need to set one color for the line, if y > 100 it should be another color)

I mean to have something like that, when the color of the line depends on the y value of the spot

not sure what you mean. Could adding Stops to your gradient solve your problem? You could use them to define hard edges. https://css-tricks.com/books/greatest-css-tricks/hard-stop-gradients/

Yes, you can add stops to your gradient to achieve the result.
Check the LinearGradient and stops property. You can use it in the gradient property of LineChartBarData


feat: refactor prql-java and add JDBC driver

Closes #1643
Hi @max-sixty / @snth, could you help on the following items?

[ ] error handling in prql-java/prql-api/src/main/rust/src/lib.rs - now a simple query fr0m table will crash the JVM
[ ] build multi-platform native binaries in .github/workflows/build-java.yaml so that we can pack into JDBC driver and publish for others to use - for testing or demonstration, you may use prqlc command line instead
[ ] sonatype account, registered maven group id org.prqllang, and gpg key for publishing signed artifacts to maven central
[ ] misc

beautiful PRQL logos, which will be later added into DBeaver - examples
platform to support - I only added mac, windows, and linux in the build file, but it's up to you on platforms should be supported
official docker image containing prqlc, I assume it's prql/prql in current implementation but it might be different

Key changes:

prql-api module for JNI(call rust lib) and CLI(wrapper of prqlc) integration
prql-jdbc module is the JDBC wrapper
rust lib is moved to prql-java/prql-api/src/main/rust

Build instructions:
cd prql-java
mvn clean package
# if you don't have Java and maven installed, try below using docker(should be similar using podman)
# docker run --rm -it  -v `pwd`:/data -w /data maven:3-openjdk-17-slim mvn clean package
ls -alF prql-jdbc/target/prql-jdbc-0.5.0-SNAPSHOT.jar

To use in DBeaver:

Add JDBC driver in Database -> Driver Manager -> New

Note: driver class is org.prqllang.jdbc.PrqlDriver.

Create new connection for testing

URL: jdbc:prql:ch:https://explorer@play.clickhouse.com:443/default (no user name and password)
Properties:

set prql.api to cli (as native lib is unstable)
prql.custom.classpath -> { full path to ClickHouse JDBC driver, URL like https://repo1.maven.org/maven2/com/clickhouse/clickhouse-jdbc/0.4.1/clickhouse-jdbc-0.4.1-http.jar works too }, only needed when the JDBC driver is not in class path
prqlc.path -> { full path to prqlc }, only needed when it cannot be found in PATH

Issue query
I hope you're not surprised that both PRQL and SQL will work. This is because 1) sometimes the underlying JDBC driver may issue SQL query so that wrapper has to support; 2) PRQL does not support DDL and mutation, so it's more convenient to support both instead of openning two windows one for PRQL and the other for SQL.
As to the error message shown below, it's not friendly for PRQL user, so I'll add an option later to override that(show PRQL compilation error instead).

Also just saw the full PR description, I'll respond to that now

Hi @max-sixty / @snth, could you help on the following items?
* [ ]  error handling in `prql-java/prql-api/src/main/rust/src/lib.rs` - now a simple query `fr0m table` will crash the JVM

I added a comment above, I think we can call something on the rust side to throw an error in Java rather than crashing the JVM. I can look more later if needed

* [ ]  build multi-platform native binaries in `.github/workflows/build-java.yaml` so that we can pack into JDBC driver and publish for others to use - for testing or demonstration, you may use prqlc command line instead

Yes, are there other projects that do this? Often this is the sort of thing we take from other libraries.
It looks like it's working on Ubuntu, is that correct? But not yet on Windows.

* [ ]  sonatype account, registered maven group id `org.prqllang`, and gpg key for publishing signed artifacts to maven central

Yes, I can do this later, and add them to the repo

I think we can call something on the rust side to throw an error in Java rather than crashing the JVM. I can look more later if needed.

Exactly. The was link is probably outdated. You may check more in https://github.com/exonum/exonum-java-binding. The idea is to throw a Java exception(java.lang.IllegalArgumentException if the PRQL is invalid, or java.lang.IllegalStateException if somehow failed to compile a valid query) with meaningful error message.

Yes, are there other projects that do this? Often this is the sort of thing we take from other libraries.

zstd-jni was the one I mentioned earlier but I didn't find a rust project, until I realized Deno is written in rust :)

It looks like it's working on Ubuntu, is that correct? But not yet on Windows.

Just tried to rebuild failed jobs at my end. Windows is the only one failing according to https://github.com/zhicwu/prql/actions/runs/4243658539, which is probably due to a naming issue - it's libprql_java*.* on mac and linux, but maybe prql_java.dll on windows.

Yes, I can do this later, and add them to the repo

Thank you. Please use the same repository secrets as used in build_java.yaml, if there's no concern.

Hmm... the build will never success as pre-commit kept replacing testng to testing - it's not a type but a popular testing framework :)

Thank you @zhicwu . This looks great!
I was going to say that I don't have a java environment setup but I see you already thought of that with your docker command:
docker run --rm -it  -v $(pwd):/data -w /data maven:3-openjdk-17-slim mvn clean package
I will give that a try.

I will give that a try.

In addition to build by yourself, you may download JDBC driver(bundled with JNI libs for macos, windows and linux) at https://github.com/zhicwu/prql/actions/runs/4257825932 - scroll down to bottom and download prql-jdbc in Artifacts list, unzip and prql-jdbc-0.5.0-SNAPSHOT.jar is the driver.

@zhicwu this is looking really good. Thanks for adding tests.
Is there somewhere that describes where the Java code comes from? Is lots of it autogenerated? Could we explain that in the readme?

I will give that a try.

If you can @snth, this would be great. Otherwise I can this weekend...

Is there somewhere that describes where the Java code comes from? Is lots of it autogenerated? Could we explain that in the readme?

Did you mean prql-jdbc? VSCode helped to generate empty methods but I had to fill in all the blanks :) Do you want me to add author and copyright to each of the files?
Understood it's time-consuming to review a PR with a few thousands line of code. Do you think it's better to meet on Slack so that I can walk you through? And you may want to review the features implemented so far as well.

Did you mean prql-jdbc? VSCode helped to generate empty methods but I had to fill in all the blanks :) Do you want me to add author and copyright to each of the files?

For example, this file: prql-java/prql-api/src/main/java/org/prqllang/internal/Utils.java. Thanks for adding one link there for a function that was taken from Netty. What about the remainder of the file? We don't have to hit 100% on every piece of code, but if there are large amounts of code that we've copied then it's helpful for future contributors to know where it came from.

VSCode helped to generate empty methods but I had to fill in all the blanks

How about prql-java/prql-jdbc/src/main/java/org/prqllang/jdbc/PrqlCallableStatement.java? Or the 1000 line file prql-java/prql-jdbc/src/main/java/org/prqllang/jdbc/PrqlResultSet.java. It looks auto-generated — but maybe it was lots of typing? 😄
If it was auto-generated, let's make a note, so someone knows not to edit it, and how to generate it again.
We can put the details at the top of a file, or in the Readme.
I recognize this is 5K lines of Java, and we have 20K lines of Rust, so it'll be a big part of the repo, and so it's helpful to know where stuff comes from.
Is that reasonable? I'm really not trying to create more work, I very much appreciate the contribution.

Do you want me to add author and copyright to each of the files?

I'm not sure of the exact rules here. If the original file has a notice, we shouldn't remove it. But I don't think we need to write new copyright notices on everything we copy.
I'm asking because it helps the maintenance of this repo.

Re Slack — what timezone are you on? I'm on PT. We can set something up if we need to discuss live. I'm not really on IM platforms most of the day though.

Thank you @zhicwu !

What about the remainder of the file?

That'd be me :)

It looks auto-generated — but maybe it was lots of typing?

Yes, methods were initially generated by VSCode, but I have to implement each of the methods - that's why I said it's not complex but do have some workload ;)

Is that reasonable? I'm really not trying to create more work, I very much appreciate the contribution.

Of course and I totally understand. Does it make sense to create a separate repo for prql-java or just prql-jdbc?

I'm not sure of the exact rules here. If the original file has a notice, we shouldn't remove it. But I don't think we need to write new copyright notices on everything we copy.

As far as I know, many opensource projects under Apache 2.0 license have copyright header in each source file, but it's really up to you. I was just trying to align with what we have here.

Re Slack — what timezone are you on? I'm on PT. We can set something up if we need to discuss live. I'm not really on IM platforms most of the day though.

I'm on China Standard Time(UTC +8), so probably we can meet at early morning my time before I head to work or weekend.

Yes, methods were initially generated by VSCode, but I have to implement each of the methods - that's why I said it's not complex but do have some workload ;)

Oh right — so lots of typing from you!
Could we just add at the top of the file that that was how it was created? This is the sort of explanation I was hoping for.
Same for files like https://github.com/PRQL/prql/pull/1929/files#diff-3ceb4a7cb5c600632a0681b0ef28bea171f39e43c8e8a4db7c16b840d2a37f76R1. So if someone else came along and wanted to make a small change, they would have some context on the origin and purpose of the code.

No need to make a separate repo (unless you have a strong preference to, in which case that would be OK)
And I think we're all fine on the licenses. If there's any code that's copied from elsewhere, we should link to it, but otherwise great.

I'm on China Standard Time(UTC +8), so probably we can meet at early morning my time before I head to work or weekend.
How about this coming weekend? I'm 16 hours behind, so anytime your morning?

A question here, how to handle parameters in PreparedStatement?  Now ? is not legal parameter placeholder in PRQL.
PreparedStatement stmt = conn.prepareStatement("from employee | filter name == ?");
//  SELECT * FROM employee WHERE name = ?
stmt.setString(1, "John");

I wrote an issue here: https://github.com/PRQL/prql/issues/1987

Since C ABI was implemented in prql-lib, it's probably better to use JNA instead(10 times slower than direct JNI in general), with no additional rust code.
As to JDBC wrapper, I'll create a separate repo for that as it can be extended to support more query languages in addition to SQL and PRQL, for instance (schemaless) GraphQL.

As to JDBC wrapper, I'll create a separate repo for that as it can be extended to support more query languages in addition to SQL and PRQL, for instance (schemaless) GraphQL.

Ok great, this could be a nice project.
I thought it was a great idea to have this for PRQL; the PRQL project could still help spread the word about it if you create a different repo.

What are good next steps for the prql-api module? Would it be helpful to chat at the weekend?
@snth let us know if you got a chance to try this out; otherwise I'll spend some time testing it.

I thought it was a great idea to have this for PRQL; the PRQL project could still help spread the word about it if you create a different repo.

That's very kind of you! And sorry for constantly changing my opinion. It's all started when I was trying to add PRQL into ClickHouse JDBC driver as preferred language(instead of SQL) for manipulating data from multiple external datasources, and then I realized it's probably better to make it a JDBC wrapper to bring PRQL to all JDBC drivers, not just ClickHouse. Now I think it can be further abstracted to support more query languages in the same way. I'm going to add GraphQL in the next following days for PoC and release the first version in case others may interest.

What are good next steps for the prql-api module?

I tried #1971 implemented by @linux-china and it works well, but maybe we don't need additional rust code for consistency, that's why I was thinking JNA even it's a bit slow. Or we probably need both and let user to choose. Let me add JNA implementation later today for comparison.

Would it be helpful to chat at the weekend?

Yes, let's meet in the coming weekend. Ping me on Slack when you're available.

Yes, let's meet in the coming weekend. Ping me on Slack when you're available.

OK, is there a good Slack workspace or should I create a new one?

OK, is there a good Slack workspace or should I create a new one?

This is the invite of my personal workspace, but you may create yours or an official workspace for PRQL :)

OK great, I joined and will be around this weekend. If I'm not on, feel free to ping me on GH, or Discord if you get access.
Something that would be good to think about — what could we merge from this work to make incremental progress. There's a lot of code here, lots looks very good. There are lots of tests, and it publishes to Maven, and the API looks quite complete. The code is high quality — I just made a change in #1971 and then I see the code is already here at https://github.com/PRQL/prql/pull/1929/files#diff-d661b1ff3ea2758cbe9719c1ce84e1f2a4280b48ee645533466fb8e3f5dd3a2bR34.
So it would be great to merge some of this, even if the JDBC driver becomes separate. Could we start merging the parts that are easier / pass tests?

@zhicwu  I think we should use more parameters instead of Option class for JNI method.  Option object requires more invocations between Java and JNI, and not good for performance.   If you want to make API clean, and you can wrap up a new PRQL class to call JNI.
#[no_mangle]
#[allow(non_snake_case)]
pub extern "system" fn Java_org_prqllang_internal_JniImpl_compile0<'local>(
    mut env: JNIEnv<'local>,
    _class: JClass<'local>,
    query: JString<'local>,
    option: JObject<'local>,
) -> jstring {
}

Parameters instead of Option object.
#[no_mangle]
#[allow(non_snake_case)]
pub extern "system" fn Java_org_prql_prql4j_PrqlCompiler_toSql(
    env: JNIEnv,
    _class: JClass,
    query: JString,
    target: JString,
    format: jboolean,
    signature: jboolean,
) -> jstring {

Option object requires more invocations between Java and JNI, and not good for performance. If you want to make API clean, and you can wrap up a new PRQL class to call JNI.

Thanks @linux-china for pointing that out! It was a copy-n-paste without thinking :p Yes, there's a PrqlCompileOptions for the same purpose.

@zhicwu  could you share your prql-jdbc?  It's good implementation for PRQL over JDBC.   I can not find the code on your fork. thanks.

@zhicwu could you share your prql-jdbc? It's good implementation for PRQL over JDBC. I can not find the code on your fork. thanks.

Sorry I discarded all the changes trying to come up with a new PR for pushing prql-java to maven central. I'll create a personal repo jdbc-wrapper for the JDBC implementation. Will update here once it's released.

@zhicwu  I got your code from commit, and it works well.

@zhicwu I got your code from commit, and it works well.

Thanks @linux-china. It has glitches handling jni and cli, and the driver does not support dialect mapping. I fixed some of the issues and tried schemaless graphql for testing. Will add a few more changes(e.g. templating and scripting etc.) before publishing to maven central.


Inconsistency (?) between netopeer2 and sysrepo

Hello,
I found a little bit of an inconsistency between Netopeer2 and Sysrepo. I have this schema:
module example-schema {
    prefix aha;
    namespace "http://example.com";

    list animalWithColor {
        key 'name color';
        leaf name {
            type string;
        }
        leaf color {
            type string;
        }
    }
}

and this data:
<animalWithColor xmlns="http://example.com">
  <name>Dog</name>
  <color>brown</color>
</animalWithColor>
<animalWithColor xmlns="http://example.com">
  <name>Dog</name>
  <color>white</color>
</animalWithColor>
<animalWithColor xmlns="http://example.com">
  <name>Cat</name>
  <color>grey</color>
</animalWithColor>

Now, I want to get all the name leafs inside the list (to tell the user, which values are valid for name when addressing animalWithColor). That means all nodes with this xpath:
/example-schema:animalWithColor/name
I want to do this through Sysrepo and also through Netopeer2. I created a sample program which does this:
#include <iostream>
#include <libyang/Tree_Data.hpp>
#include <sysrepo-cpp/Session.hpp>
extern "C" {
#include <nc_client.h>
}

int main(int argc, char* argv[])
{
    std::cout << "SYSREPO" << std::endl << "==============" << std::endl;
    auto conn = std::make_shared<sysrepo::Connection>("name");
    auto sess = std::make_shared<sysrepo::Session>(conn);
    auto items = sess->get_items("/example-schema:animalWithColor/name");
    for (unsigned int i = 0; i < items->val_cnt(); i++) {
        std::cout << items->val(i)->xpath() << " = " << items->val(i)->val_to_string() << std::endl;
    }

    std::cout << std::endl << "NETCONF" << std::endl << "==============" << std::endl;
    nc_client_init();
    auto session = nc_connect_unix("/home/vk/git/netconf-cli/build/netopeer2-server.sock", nullptr);
    auto rpc = nc_rpc_getconfig(NC_DATASTORE_RUNNING, "/example-schema:animalWithColor/name", NC_WD_ALL, NC_PARAMTYPE_CONST);
    uint64_t msgid;
    nc_send_rpc(session, rpc, 10000, &msgid);
    struct nc_reply_data* reply;
    nc_recv_reply(session, rpc, msgid, 100000, 0, (struct nc_reply**) &reply);
    auto dataNode = libyang::create_new_Data_Node(reply->data);
    for (const auto& it : dataNode->tree_for()) {
        for (const auto& node : it->tree_dfs()) {
            std::cout << node->path();
            if (node->schema()->nodetype() == LYS_LEAF) {
                libyang::Data_Node_Leaf_List leaf(node);
                std::cout <<  " = " << leaf.value_str();
            }
            std::cout << std::endl;
        }
    }
}

This is the output of the program:
SYSREPO
==============
/example-schema:animalWithColor[name='Dog'][color='brown']/name = Dog
/example-schema:animalWithColor[name='Dog'][color='white']/name = Dog
/example-schema:animalWithColor[name='Cat'][color='grey']/name = Cat

NETCONF
==============
/example-schema:animalWithColor[name='Dog'][color='brown']
/example-schema:animalWithColor[name='Dog'][color='brown']/name = Dog
/example-schema:animalWithColor[name='Dog'][color='brown']/color = brown
/example-schema:animalWithColor[name='Dog'][color='white']
/example-schema:animalWithColor[name='Dog'][color='white']/name = Dog
/example-schema:animalWithColor[name='Dog'][color='white']/color = white
/example-schema:animalWithColor[name='Cat'][color='grey']
/example-schema:animalWithColor[name='Cat'][color='grey']/name = Cat
/example-schema:animalWithColor[name='Cat'][color='grey']/color = grey

Sysrepo gives me the name leafs only (that's what I wanted), but Netopeer2 also gives me the color leafs, which are also present in the list, but they definitely do not fit the /example-schema:animalWithColor/name xpath. How would I go about getting only the name leafs with Netopeer? Is this the expected behavior? Should I just filter the Netopeer results by the xpath? Or is there a better approach?

Hi,
okay, this took me a second to figure out but it all makes sense. Actually, the data returned are the same, your output differences are caused by different data representations and their transformation for display.
If you look at the first line of SYSREPO output
/example-schema:animalWithColor[name='Dog'][color='brown']/name = Dog

you see that there is a list instance (/example-schema:animalWithColor), its first key with value (name='Dog') and its second key with value (color='brown'). In the NETCONF output you start with a data tree representation, so this are the same data
example-schema:animalWithColor (1)
     /               \
name='Dog' (2)   color='brown' (3)

The difference appears when you call path() method for each data node because it always prints a full path of each node
/example-schema:animalWithColor[name='Dog'][color='brown'] (1)
/example-schema:animalWithColor[name='Dog'][color='brown']/name = Dog (2)
/example-schema:animalWithColor[name='Dog'][color='brown']/color = brown (3)

So, how to achieve what you want? The thing is that even sysrepo internally start with this data tree, then uses the XPath filter and gets the XPath in the sr_val_t structure by calling path() on each selected node. So, you can do the same, call XPath filter (find_node()) again on the returned data.
I guess the confusing thing is that you have to use the filter twice. However, in other word, the reason is that you want just the name node but it cannot really exist without its parent list, which must also always include all keys. I hope I have explained it enough to be understandable.
Regards,
Michal

Okay, I understand. Thank you for the explanation!


Deploy on Azure Function with private python package

Hi
I am using Azure Python Function for tasks that need to be done once a day. They are triggered by MS Flow using http.
Some code is shared between functions and functionapps. This code is now contained in a python package host in a private Azure DevOps repository.
When i’m using Azure Function locally. I have no issues.
But when I am deploying the app from my laptop using : Function azure FunctionApp publish  --build-native-does, I have got the following error : no matching distribution found for
How can I set an access to my private azure repos during Azure Function deployment ?
Best regard
Frantche

To install a private package, you'll need to setup a DevOps pipeline. Here's a sample template - https://github.com/Azure/azure-functions-devops-build/blob/master/sample_yaml_builds/linux/consumption/python-consumption.yml
You can use the az functionapp devops-build command to help you create the pipeline.