drone 0.4 database ping failed

First of all allow me to tell you that I think drone is an exquisite piece of technology that's very easy to understand and use. Thank you for sharing :).
I am trying to deploy drone 0.4 with Postgres Database located in another container.
$ docker run -d --name drone-postgres -e POSTGRES_USER=drone -e POSTGRES_DB=drone -e POSTGRES_PASSWORD=drone postgres

I have verified that the database is up and running with another docker container.
$ docker run -it --link drone-postgres:postgres --rm postgres sh -c 'exec psql -h "$POSTGRES_PORT_5432_TCP_ADDR" -p "$POSTGRES_PORT_5432_TCP_PORT" -U drone'

After entering drone password I am inside a psql shell. I suppose it is working.
Next I have run drone with environment variables.
$ docker run --rm -it -e DATABASE_DRIVER=postgres -e DATABASE_CONFIG=postgres://drone:drone@<IP_ADDRESS>:5432/drone drone/drone:0.4

INFO[0000] using database driver postgres
INFO[0000] using database config postgres://drone:drone@<IP_ADDRESS>:5432/drone
INFO[0000] database ping failed. retry in 1s
INFO[0001] database ping failed. retry in 1s
INFO[0002] database ping failed. retry in 1s
INFO[0003] database ping failed. retry in 1s
INFO[0004] database ping failed. retry in 1s
INFO[0005] database ping failed. retry in 1s
INFO[0006] database ping failed. retry in 1s
INFO[0007] database ping failed. retry in 1s
INFO[0008] database ping failed. retry in 1s
INFO[0009] database ping failed. retry in 1s
INFO[0010] database ping failed. retry in 1s

The IP Address is from docker inspect drone-postgres. Could you please tell what have I done wrong or another way to deploy drone with Postgres, both in docker containers.

Have you also tried DATABASE_CONFIG=postgres://drone:drone@postgres:5432/drone? As per default you can use the link name as host to connect to.

@tboerger just tried it, same result.
$ docker run --rm -it --link drone-postgres:postgres -e DATABASE_DRIVER=postgres -e DATABASE_CONFIG=postgres://drone:drone@postgres:5432/drone drone/drone:0.4  INFO[0000] using database driver postgres
INFO[0000] using database config postgres://drone:drone@postgres:5432/drone
INFO[0000] database ping failed. retry in 1s
INFO[0001] database ping failed. retry in 1s
INFO[0002] database ping failed. retry in 1s

I don't really know but maybe this is apso related to the scratch base image. I'm using my own custom docker image...

@valentinbud can you docker pull drone/drone:0.4 and confirm you have the latest docker image?
We had an open issue that I believe might be related:
https://github.com/drone/drone/issues/1269
We merged a PR a few days ago to fix:
https://github.com/drone/drone/pull/1306

@bradrydzewski sure. I have just removed the dev machine and created a new one with docker-machine, so I have no images.
$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE

docker pull drone/drone:0.4
0.4: Pulling from drone/drone
a751b1edea64: Pull complete
a39ef86716f9: Pull complete
5fd6e7ab8fa4: Pull complete
f005ec171ab3: Pull complete
3e96393070da: Pull complete
036eaaa9dd63: Pull complete
b509b5ab561b: Pull complete
40dea3261655: Pull complete
Digest: sha256:2ec06956976eadeaec2517e334eed82a4e744273154d2e852816aa9ae3eb3829
Status: Downloaded newer image for drone/drone:0.4

Same result with both postgres and IP Address in the Postgres connection string.
$ docker run --rm -it --link drone-postgres:postgres -e DATABASE_DRIVER=postgres -e DATABASE_CONFIG=postgres://drone:drone@postgres:5432/drone drone/drone:0.4
INFO[0000] using database driver postgres
INFO[0000] using database config postgres://drone:drone@postgres:5432/drone
INFO[0000] database ping failed. retry in 1s
INFO[0001] database ping failed. retry in 1s
INFO[0002] database ping failed. retry in 1s
INFO[0003] database ping failed. retry in 1s
INFO[0004] database ping failed. retry in 1s
INFO[0005] database ping failed. retry in 1s
INFO[0006] database ping failed. retry in 1s

@tboerger based on Alpine, Debian, Busybox?

@valentinbud can you confirm the /etc/hosts file is created by Docker inside the running Drone container?

@valentinbud also you mentioned you verified connectivity, however, the example you provided used the IP address instead of the hostname. Are you able to verify connectivity using linked containers and the hostname using the same command below?
$ docker run -it --link drone-postgres:postgres --rm postgres sh -c 'exec psql -h "$POSTGRES_PORT_5432_TCP_ADDR" -p "$POSTGRES_PORT_5432_TCP_PORT" -U drone'

@bradrydzewski I have tried it both with IP address and hostname of linked container. Same result.
docker run -it --link drone-postgres:postgres --rm postgres sh -c 'exec psql -h "$POSTGRES_PORT_5432_TCP_ADDR" -p "$POSTGRES_PORT_5432_TCP_PORT" -U drone'
Password for user drone:
psql (9.5.0)
Type "help" for help.

drone=#

I have just received a new error before the drone container finally gave up and exited.
INFO[0029] database ping failed. retry in 1s
ERRO[0030] pq: SSL is not enabled on the server
FATA[0030] database ping attempts failed

Should SSL be enabled and properly configured on the Postgres server?

you could try adding ?sslmode=disable per this stackoverflow:
http://stackoverflow.com/questions/21959148/ssl-is-not-enabled-on-the-server

@bradrydzewski I am still wandering how could I check inside the container the if /etc/hosts exists, since the container doesn't even have cat or anything else for that matter.
Although I have used docker inspect. I am running drone/drone:0.4 in a separate tmux pane interactively.
$ docker inspect modest_colden | grep Hosts
    "HostsPath": "/mnt/sda1/var/lib/docker/containers/619e421ae826df3a8696fd0384425679b19bf09781ef947254fa25ffff03b772/hosts",
 $ docker-machine ssh dev sudo cat /mnt/sda1/var/lib/docker/containers/619e421ae826df3a8696fd0384425679b19bf09781ef947254fa25ffff03b772/hosts
<IP_ADDRESS>      619e421ae826
<IP_ADDRESS>       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
<IP_ADDRESS>      postgres dbae54a6ef34 drone-postgres

@bradrydzewski thanks! It works with hostname and IP address once I add ?sslmode=disable.
Just to answer myself, security wise, TLS should be properly configured and enabled throughout the infrastructure.

Just to answer myself, security wise, TLS should be properly configured and enabled throughout the infrastructure.

You should be fine as long as your are running Drone and Postgres on the same machine and not exposing Postgres. If you decide to expose Postgres or run on a separate server, requiring a remote connection, you should definitely consider TLS mode.
This would require you to setup certificates with Postgres and then to mount those certificate files into the Drone container using Docker volumes. You can then reference these certificate files using the following parameters in your Postgres connection string:

sslcert - Cert file location. The file must contain PEM encoded data.
sslkey - Key file location. The file must contain PEM encoded data.
sslrootcert - The location of the root certificate file. The file must contain PEM encoded data.
Valid values for sslmode are:

More information at https://godoc.org/github.com/lib/pq#hdr-Connection_String_Parameters

@bradrydzewski thank you for the extra valuable information. For now they both run on the same machine and Postgres doesn't expose its port. I am definitely going to configure TLS on all services once I go multi machine.

@bradrydzewski Any idea why this would also be happening with sqlite3? Same error on multiple containers using default settings and a host mounted volume.

@Dids let me guess, you are using some kind of Centos/Fedora/Atomic/Redhat? Than your volume just needs a Z flag, e.g. http://www.projectatomic.io/blog/2015/06/using-volumes-with-docker-can-cause-problems-with-selinux/
This is related to SELinux.

@tboerger Just regular old Ubuntu. I'm running HAProxy and two instances of the Drone container for both Github and Bitbucket.

@Dids the only known scenario that can cause the sqlite3 database connection failure is when the volume is not properly mounted. The root cause of such issues can be the following:

volume is not correctly mounted per the documentation
volume is not correctly mounted due to OS restrictions (known issues with SE linux per @tboerger)
volume is correctly mounted, but database configuration environment variables are misconfigured

There are of course other scenarios, unrelated to drone or docker, that could cause sqlite database connections to fail. For example, two separate OS processes both trying to lock the database file for writes.
To further troubleshoot we need to see the actual database error message, which is not included in the message pasted above. Drone will attempt to connect to the database 30 times and will then print the database error message, from sqlite. Without this message we can only speculate.

@bradrydzewski Here's what's essentially my Docker compose file (I use Tutum/Docker Cloud):
misc-drone-bitbucket:
  image: drone/drone:0.4
  expose:
    - "8000"
  environment:
    - HTTP_CHECK=HEAD / HTTP/1.0
    - VIRTUAL_HOST=http://bitbucket.drone.my-domain.com
    - VIRTUAL_HOST_WEIGHT=2
    - REMOTE_DRIVER=bitbucket
    - REMOTE_CONFIG=https://bitbucket.org?client_id=my-client-id&client_secret=my-client-secret
    - DATABASE_DRIVER=sqlite3
    - DATABASE_CONFIG=/var/lib/drone/drone.sqlite
    #- DEBUG=true
    #- GIN_MODE=debug
  volumes:
    - /data/drone_bitbucket:/var/lib/drone
    - /var/run/docker.sock:/var/run/docker.sock
  tags:
    - misc
  restart: always

I never saw any sqlite errors, but I did notice that if I redeploy the container, it correctly creates the directories on the second launch/deploy. I've seen this before with Tutum though, so I'm assuming it's a Docker Compose issue (or a Tutum issue for that matter).
At any rate, it's not letting me go any further, still stuck on 503 Service Unavailable with no error messages in the container logs.

Solved all of my issues by moving to a custom Rancher setup. No issues with volumes and everything just works.

Hi there,
So I just went to try and get Drone working on my personal server and I've gotten the docker image downloaded and everything set up EXACTLY as http://readme.drone.io/setup/overview states. When I do the docker run command I get 2 errors back after a bit of a wait:
time="{timecode}" level="error" msg="unable to open database file"
time="{timecode}" level="fatal" msg="database ping attempts failed"

I've tried making an empty database file while defining the sqlite3 database driver. Is there something I'm doing wrong?
To clarify I'm pretty well retyping EXACTLY what is in the guide and getting these errors. I have created the dronerc file and populated it with my Github credentials.

unable to open database file happens when sqlite3 cannot create the database because the parent directory does not exist. The default directory for the database, /var/lib/drone, does not existing in the container and must be mounted as a volume, per the docs. This can be done with the following flag when starting the drone container --volume /var/lib/drone:/var/lib/drone

@firrae Where are you deploying the containers? Docker Cloud (previously Tutum) has volume issues when deploying, esp. on the first deployment.

@firrae as you stated that you are deploying on your own server, which OS do you have? Maybe something from the RedHat family? There are known issues where the volume needs a speciic additional flag, e.g. --volume /var/lib/drone:/var/lib/drone:z or --volume /var/lib/drone:/var/lib/drone:Z, explained at http://www.projectatomic.io/blog/2015/06/using-volumes-with-docker-can-cause-problems-with-selinux/

@bradrydzewski I have the exact line in my command as per http://readme.drone.io/setup/overview/#create-and-run-a-drone-container:68f80267fa3a50980dbb745a782b8dca. I am using that docker command verbatim. I have made sure to also create the directories where they didn't pre-exist.
@Dids I am deploying this on a private virtual server in our office. To also answer @tboerger I'm running Ubuntu 14.04 (kernel 4.X). That Ubuntu instance is virtualized on V-Sphere.
Using Ubuntu I figured would have me safe from any issue like the ones SELinux has.

the unable to open database file is a low level error from sqlite because it cannot create the database file. The two most common root causes (lack of mounted volume and SELinux) are mentioned above, however, I'm not familiar enough with the internals of sqlite to advise further. Perhaps it makes sense to post to the sqlite issue tracker to get more assistance.

I've seen lots of permission related issues with Docker and volumes when creating/modifying files as a non-root user. Could this be an issue here also?

I am using Docker via a user with sudo level access, so I'm not sure where the issue might end up. I'll go check what the sqlite issue tracker says, but sqlite seems to be running and working fine otherwise. Is there a create script I could use to premake the sqlite file so it doesn't have to technically create it?

No no, I was referring to the Drone dockerfile running a service as a different user which creates/modifies the sqlite file. On the host system (using volumes) this user doesn't exist, so the file permissions are wrong and cause issues.

@Dids drone runs as root inside the container. You are correct that running drone as non-root would cause issues, in which case drone would be broken for thousands of people :)

I am using Docker via a user with sudo level access

Doesn't the Docker deamon run as root [1]? Have you made any custom configuration to your Docker installation or to its default deamon settings? Have you added any additional runtime flags?
[1] https://docs.docker.com/engine/security/security/#docker-daemon-attack-surface

create script I could use to premake the sqlite file

It is unlikely this would make a difference. If libsqlite cannot create the database it is unlikely it will be able to open the file with a write lock.

@bradrydzewski Thanks Brad, that's actually good to hear. :)
My setup is somewhat identical to @firrae, as I'm running a VM (Ubuntu 14.04), then followed the Drone docker setup word for word. Nothing crazy going on that could be causing it, apart from using AUFS with Docker.

@bradrydzewski I installed it via the instructions on the Docker site (https://docs.docker.com/engine/installation/linux/ubuntulinux/). From there I'm able to run Hello World and a few other very basic things so I figured it was all good and tried installing Drone.

@Dids @firrae have you tried doing something like docker run --volume=/var/lib/drone:/var/lib/drone -t -i busybox /bin/sh which drops you to a prompt and then ls /var/lib/drone? and touch /var/lib/drone/foo.txt to test the ability to access the volume and write files?

@bradrydzewski I can try that this evening when I have physical access to the box again.

@bradrydzewski Yup, I've seen it create files on the host side, so it shouldn't be that either. Other containers on the same host are using volumes just fine too.

I'm able to consistently reproduce this issue under three conditions:

incorrectly mount the volume
instruct sqlite3 to create the database in a folder that does not exist (usually related to number 1)
invalid permissions to write the file (usually related selinux or apparmor)

I would recommend you do some additional debugging to try to uncover the root cause. Here are some things I can suggest:

try using docker volume create to create a container volume instead of a host volume
try using a container volume like --volume=/var/lib/drone
try using DATABASE_CONFIG=/ to avoid volumes all together and to create the database inside the container root directory (not this will not persist and should be used for debugging only)
try compiling your own version of drone and add debugging code. Add Go code that checks the directory and tries to write a file. Perhaps Go will provide more meaningful error messaging.

Aside from the above suggestions I'm not sure how much more I'm going to be able to help simply due to inability to reproduce. The issue also seems to be isolated to a small number of installations which makes this even more difficult to troubleshoot ... it would be much easier if everyone was having this issue, although I'm glad they aren't :)

Oh, apparmor, I have that installed... The official Docker install instructions says you need to have it installed. Could that be the issue?
And that's fair, this seems to be a fairly rare issue, and in my part it's likely due to me not being very advanced in my knowledge in Docker/sysadmin.

OK, well I restarted the VM again and now it works... That was anticlimactic...

If you are using the drone from the rancher catalog, use database instead of localhost.  Look at the examples below.

See this in the logs:
4/1/2017 2:39:11 PMtime="2017-04-01T19:39:11Z" level=info msg="using database driver postgres"
4/1/2017 2:39:11 PMtime="2017-04-01T19:39:11Z" level=info msg="using database config postgres://<user>:<pass>@database:5432/postgres?sslmode=disable"
4/1/2017 2:39:11 PMtime="2017-04-01T19:39:11Z" level=info msg="database ping failed. retry in 1s"
4/1/2017 2:39:12 PMtime="2017-04-01T19:39:12Z" level=info msg="database ping failed. retry in 1s"
4/1/2017 2:39:13 PMfatal error: runtime: out of memory

The container is stuck in a reboot cycle, and the public facing IP serves a 503.

Any ideas why this is happening? Drone is the only thing running on a t2.small ec2 instance (2gb RAM), and i've tried configuring everything at least a dozen times.
Can we get more verbose logging?

Can we get more verbose logging?

No because Go does not instrument the standard library or database drivers with logging. There is no magic flag that can automatically start dumping verbose logs (like with Java). This is the best we have.

Any ideas why this is happening?

The error claims the server is out of memory. The error message is coming from the Go standard library which is claiming it cannot allocate memory. Why would the Go runtime not be able to allocate memory? Have you checked the server memory usage?
4/1/2017 2:39:13 PMfatal error: runtime: out of memory

what are the "Generate" buttons for, next to github and database config? They seem to just generate nonsense.

You mean the generate button on the secrets tab? You paste your secret file and it generates the encrypted version.

I see.
How much memory should be required of the host?
Also, I am still getting the ping database issue before the memory issue.

How much memory should be required of the host?

the drone server usually allocates 8mb memory to start and with cached data usually runs around 12mb idle

Also, I am still getting the ping database issue before the memory issue.

the database ping failure simply means that the postgres driver cannot establish a connection with the postgres server. Generally this happens when perhaps the username, password, address or some other parameter in the database connection string is incorrect. Or that the server cannot be reached due to network configuration issues. Unfortunately without direct server access and knowledge of your configuration, this is not something I can troubleshoot. You can read the official postgres driver documentation for more detail https://github.com/lib/pq which may provide more insight.

I definitely have that much RAM free at runtime; it's on a 2GB machine where the only other container alive is postgres.
I have quadruple checked the configuration by reinstalling everything running on the host:

postgres from the catalog, dbname postgres, user and pass both postgres
drone from the catalog, with the postgres secret key being postgres://postgres:postgres@database:5432/postgres?sslmode=disable and the linked service being postgres-lb.

This is as vanilla as I can get the installation, and it still repeatedly tries to connect and then throws the memory error.

To confirm there are no issues with drone:0.4 and postgres I just downloaded and setup and instance using docker-compose file. This is as vanilla as it gets and runs without issue.
This is the exact file / configuration I used. Hopefully this helps
version: '2'

services:
  drone-server:
    image: drone/drone:0.4
    ports:
      - 8080:8000
    volumes:
      - ./drone:/var/lib/drone/
    restart: always
    environment:
      - DRONE_OPEN=true
      - REMOTE_DRIVER=gogs
      - REMOTE_CONFIG=${GOGS_URL}
      - DATABASE_DRIVER=postgres
      - DATABASE_CONFIG=postgres://postgres@postgres:5432/postgres?sslmode=disable

  postgres:
    image: postgres
    environment:
      - POSTGRES_USER=postgres


How do I say required but could be default?

Package version eg. v8, v9:
v9
Issue, Question or Enhancement:
How do I write a rule that says the key is required but the value could be default? For a string, empty is fine but null is not. How do I do that? I couldn't figure that out from docs and no helpful examples available also.
Code sample, to showcase or reproduce:

@aneeskA an example would really help here, I'll attempt to answer based on my understanding of the question though:
There are a number of issues around this that I've left open, the TL;DR of it is you can use a *string instead of a string which can capture all the states you need to deal with eg. nil, blank/empty and another validation and then can use something like:
omitempty,eq=""|othervalidation
let me know if this helps

@aneeskA Thank you for your question. It appears as if your concern has been addressed. Please let us know if you have any follow-up questions.


[stable/kibana] Add config file for properties

Add config file for passing configuration values to properties instead
environment variables. Environment varibales can not work if the
property name has (dots as well as) underscroes in the name. This use-
case comes up while using some plugins whose property names have been
defined with dots and underscores in them.

What this PR does / why we need it:
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:

/assign @scottrigby

LGTM

@compleatang Can you please review this once again. Included @josdotso's suggested changes to achieve the same functionality.

@rustycl0ck have added two small comments, otherwise GTG from my end.

@compleatang Thanks for the quick response. Both the suggestions have been taken care of now.

cool. i'm not a formal approver but this LGTM

/assign @unguiculus

/ok-to-test

/lgtm

/ok-to-test


Tool shouldn't create manifests for configurations that don't exist

I don't have any oauth or secret configurations in my ocp3 config, and that led to manifests being created that have empty data.
[dymurray@pups cpma]$ cat data/manifests/100_CPMA-cluster-config-oauth.yaml
apiVersion: config.openshift.io/v1
kind: OAuth
metadata:
  name: cluster
  namespace: openshift-config
spec:
  identityProviders:
  - null
[dymurray@pups cpma]$
[dymurray@pups cpma]$ ls data/manifests/
100_CPMA-cluster-config-oauth.yaml    100_CPMA-cluster-config-secret-.yaml
[dymurray@pups cpma]$ cat data/manifests/100_CPMA-cluster-config-secret-.yaml
apiVersion: ""
kind: ""
type: ""
metadata:
  name: ""
  namespace: ""
data: null

We should have some better error checking so that we aren't creating null manifests.

Thanks and agree. I believe both, @gildub and @alexander-demichev, are aware of this one.

Indeed and agreed, it's just because we're only working on OAuth as the first bit.

I will fix it after we add more config parts to migrate. It should be checked and skipped if empty here https://github.com/fusor/cpma/blob/master/ocp4/ocp4.go#L33

This has been addressed by https://github.com/fusor/cpma/pull/85


should karate.compareImage() throw an exception on a failure

refer comments on this stack overflow issue: https://stackoverflow.com/a/76586756/143475
@jkeys089 perhaps when karate.compareImage() is used, it should always return the delta as the mismatchPercentage and leave it to users to decide how to handle it, that's how karate.match() works by the way
karate.match() returns a JSON with { pass: boolean } in it - I know that should be arguably { passed: boolean } but lets stick to pass. so should the karate.compareImage() also return a pass: boolean property.

@ptrthomas if you don't want compareImage to throw, just configure mismatchShouldPass: true.

@jkeys089 agreed for the keyword. not sure if the JS API should use that default, that's all

@ptrthomas it currently matches the behavior of karate.abort() / abortedStepsShouldPass so I think it is consistent in that way.


Bug: bottom to top icon in footer needs to align because it overlaps footer text

What happened?
Bottom to top icon in footer needs to align because it overlaps footer text
Add screenshots

What browsers are you seeing the problem on?
Chrome
Record

[X] I have checked the existing issues
[X] I have read the Contributing Guidelines
[X] I agree to follow this project's Code of Conduct
[X] I want to work on this issue

Hello @Saurav-Pant please assign this issue to me under gssoc 24

Go ahead @kapilG0


Add basic CMP messaging

Task/Issue URL: https://app.asana.com/0/1125189844152671/1202192141265631
Description
This PR integrates the autoconsent library and adds a first basic messaging two-way comms between the library and the app.
Steps to test this PR

[x] Install the app and go to SERP/
[x] In the dax dialog click Hide an. Hide tips forever.

Eval handled

[x] Navigate to https://www.lufthansa.com/de/en/homepage.
[x] You should see a privacy settings popup and a new Dax Dialog on top of tip.
[x] Click on Manage Cookie Pop-ups.
[x] Both the dialog and the privacy settings popup should disappear.

Basic handled

[x] Navigate to http://privacy-test-pages.glitch.me/features/autoconsent/.
[x] You should see a new Dax Dialog and in the website a button that says Reject all
[x] Click on Manage Cookie Pop-ups.
[x] Dax dialog should disappear and the button should say I was clicked.
[x] Refresh the page.
[x] This time click on No Thanks
[x] Dax dialog should disappear and button should still say Reject all.
[x] Refresh the page.
[x] This time dismiss the dialog without clicking any of the buttons.
[x] Dax dialog should disappear and button should still say Reject all.

Current dependencies on/for this PR:

develop

PR #2128   👈

PR #2129

This comment was auto-generated by Graphite.

@marcosholgado - this is tested but if you want I can leave it non approved as it is the base branch? in case it helps avoiding merging by mistake (?)

Give free all


Invalid default data for parsing in binary mode

Describe the Bug
When parsing a CE from HTTP binary message, this library fails to parse the CE correctly. It uses a {} as a default value for data, even if data isn't given, nor have content type like text/plain.
Steps to Reproduce

Parse CE from HTTP:const ce = HTTP.toEvent({ headers: req.headers, body: req.body })

Send an HTTP like:POST /events HTTP/1.1
Content-Length: 0
Host: localhost:8080
ce-id: xuPh9kai
ce-source: local
ce-specversion: 1.0
ce-type: ping
content-type: text/plain

Actual Behavior
The event is parsed incorrectly into (notice the {} in data):
{
  "id": "xuPh9kai",
  "time": "2023-04-19T10:04:25.088Z",
  "type": "ping",
  "source": "local",
  "specversion": "1.0",
  "datacontenttype": "text/plain",
  "data": {}
}

Expected Behavior
The event should be parsed correctly into:
{
  "id": "xuPh9kai",
  "time": "2023-04-19T10:04:25.088Z",
  "type": "ping",
  "source": "local",
  "specversion": "1.0",
  "datacontenttype": "text/plain",
  "data": ""
}

Additional context
This incorrect parsing prevents interoperation with other languages. For example, in https://github.com/openshift-knative/showcase/pull/24, I use a Rust built WASM module (using cloudevents-rust lib) to print the given CE as structured JSON.
/kind bug

Hmm, this is even worse. Sending request like:
POST /events HTTP/1.1
Content-Length: 12
Host: localhost:8080
ce-id: Kea7feem
ce-source: local
ce-specversion: 1.0
ce-type: ping
content-type: test/plain

Hello World!

Results in the following CE:
{
  "id": "Kea7feem",
  "time": "2023-04-20T13:01:34.972Z",
  "type": "ping",
  "source": "local",
  "specversion": "1.0",
  "datacontenttype": "test/plain",
  "data": {}
}

It should be (data is missing, despite having content type of test/plain):
{
  "id": "Kea7feem",
  "time": "2023-04-20T13:01:34.972Z",
  "type": "ping",
  "source": "local",
  "specversion": "1.0",
  "datacontenttype": "test/plain",
  "data": "Hello World!"
}

@cardil I am not able to recreate this issue. This module does not actually implement a server that receives incoming requests - that is up to the users of the module. However, we do have extensive tests around incoming CloudEvent data. For example, see this test for text/plain.
https://github.com/cloudevents/sdk-javascript/blob/1cf8f8acae6678ab62b25f82cc11f0e756d87a5c/test/integration/message_test.ts#L116-L127
The [faas-js-runtime](https://github.com/nodeshift/faas-js-runtime) module, however does implement a server and parses incoming CloudEvents, for which there are also tests. For example, here we test your specific use case:
https://github.com/nodeshift/faas-js-runtime/blob/deb9f99444630eb440ea1d270e73ea47b4c745a1/test/test-http-body.js#L52-L71
Can you provide some code that enables me to reproduce this locally?

@cardil I took a look at the showcase app and I believe the problem is in your app. Specifically, the /events endpoint is not configured for text/plain only for application/json see here: https://github.com/openshift-knative/showcase/blob/ad31193312c61458de0380979fb038d0db13001e/expressjs/src/routes/events/endpoint.js#L22-L34 and here: https://github.com/openshift-knative/showcase/blob/ad31193312c61458de0380979fb038d0db13001e/expressjs/src/routes/events/endpoint.js#L78-L99

I think you are right @lance!
I think this Express middleware automatically parses everything as JSON: https://github.com/openshift-knative/showcase/blob/latest/expressjs/src/middleware/json.js#L7
Ref.: https://www.geeksforgeeks.org/express-js-express-json-function
I'll confirm it, and close this issue.
Probably, the Express JSON middleware should NOT parse the body, if the content type isn't JSON.

I think this Express middleware automatically parses everything as JSON: https://github.com/openshift-knative/showcase/blob/latest/expressjs/src/middleware/json.js#L7

Yes I think you are right. You might consider using express.raw().

@cardil have you been able to confirm this assessment on your end?

@cardil based on your PR here https://github.com/openshift-knative/showcase/pull/25 I am assuming you have resolved this issue on your end. If you still find that there is an issue, please reopen this one or start a new one.


[BUG]: Uncertain Code and Folder Structure

Describe the bug
The code and folder structure are messy, causing confusion. It's hard to maintain and debug.
To Reproduce
Steps to reproduce the behavior:
Go to the project repository.

Open the code and folder structure.
Try to locate specific files or understand the organization.
See the disorganization.

Expected behavior
A clear and consistent structure to make maintenance and debugging easier.
Additional context
I am SSOC'24 Contributor.

Hey, I have updated the folder structure and ready to create the PR.
We should consider this first

@mdkaifansari04 Let the folder structure be as it is right now.You can organize the public folder in the client folder if you want. Do let me know on this.

I have structured all the component based on their use case
Here is the screenshot of it

I only have structured the client folder not the server one and also have removed some unused codes. It's completely fine and wanted to push it first and then work on further issues


Release/0.3.2

0.3.2
New

Add code of conduct. [Ben Dalling]

Fix

CVE-2024-5171. [Ben Dalling]

Accidentally raised against develop.


WS1004

CloudFormation Lint Version
cfn-lint 0.86.0
What operating system are you using?
WSL2 Ubuntu on Windows
Describe the bug
cfn-lint incorrectly reports:

[cfn-lint] WS1004: Lambda function xxx does not have a corresponding log group with a Retention property

When there is a !Ref to a log group with a retention property.
Expected behavior
When there is an explicit !Ref to a defined log group with the RetentionInDays property set then WS1004 should not be flagged.
Reproduction template
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31

Resources:

  LambdaProxyFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: src/
      FunctionName: MyFunction
      Handler: index.handler
      LoggingConfig:
        LogFormat: JSON
        LogGroup: !Ref LambdaProxyLogGroup
      MemorySize: 512
      PackageType: Zip
      ReservedConcurrentExecutions: 1
      Runtime: nodejs18.x
      Timeout: 10
      Tracing: Active

  LambdaProxyLogGroup:
    DeletionPolicy: Retain
    UpdateReplacePolicy: Retain
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupClass: STANDARD
      RetentionInDays: 180

cfn-lint itself does not vend WS1004. Seems to be from this popular rule pack:
https://awslabs.github.io/serverless-rules/rules/
https://github.com/awslabs/serverless-rules/issues


REST Api Parser does not find a file if there is an accent in its name

Hi
I used the parse api with this encoding :
http://<IP_ADDRESS>:9090/services/rest/parser?path=%2Fusr%2Fshare%2FDocuments%2F2016%2FPrêtAltea.pdf
My file has an accent in its name.
I receive an error from the parser which is :
java.io.FileNotFoundException: /usr/share/Documents/2016/PrêtAltea.pdf (No such file or directory) - /usr/share/Documents/2016/PrêtAltea.pdf (No such file or directory) - com.jaeksoft.searchlib.webservice.parser.ParserImpl.putMagic(ParserImpl.java:203)
if i delete the accented character from the name, it works.
Is there any option to activate or is it a bug ?
thanks
Marc

Hi everybody,
I answer to myself,
The only thing to do, is to select the right language in start.sh script
Instead of en_US.UTF_8, i put fr_FR.UTF_8, so accents will exists in the charset.
Maybe there is a more "reliable" solution instead of hardcode a language..
But that worked for me...
Thanks
Marc

Hi Marc,
First, thanks for the workaround.
I've read some interesting Q&A issues:
http://stackoverflow.com/questions/3072376/how-can-i-open-files-containing-accents-in-java
http://jonisalonen.com/2012/java-and-file-names-with-invalid-characters/
http://stackoverflow.com/questions/12109520/java-cant-see-file-on-file-system-that-contains-illegal-characters
I think  that one interesting suggestion would be to move to java.nio. I currently investigate this issue.


Code / Documentation

Where can i find the code or the documentation to achieve the same results as the demo video provided. I couldn't find the main file where all the classes and helper function were called.

Please add the code for reproducing the demo, thanks

Hi @sarthakg2002 and @chumpblocckami, most of the code was stored in a separate repository. It was moved here yesterday. I'm still working on documenting it, but you can find it here.

hi @SkalskiP, thanks a lot for the update! :)


username is not showing for oauth users

after commenting, the username is shown for users from local registration as there is username field. However users from oauth does not show their username in comment.

Do you have a screenshot?

the comment is done by the user registered from local accounts package. However reply is done by oauth user.

Please show me the oauth user document, might be that the username field isn't set on that user.

Yes the username field is not set. But that is oauth. Shouldn't it be
username or profile.name?
On Sep 28, 2015 1:48 AM, "Matteo De Micheli"<EMAIL_ADDRESS>wrote:

Please show me the oauth user document, might be that the username field
isn't set on that user.
—
Reply to this email directly or view it on GitHub
https://github.com/ARKHAM-Enterprises/meteor-comments-ui/issues/31#issuecomment-143591071
.

probably, I'll have a look at this soonish

This is fixed in the latest version 0.2.13


ke

@yujikosuga Was this created by accident?

@elmindreda Yes, it was. Thanks for mentioning this to me as I was completely unaware of this. Let me close the ticket.


Add support for Marker create, finalize, activate messages

Description
Add support for Marker creation, finalization, and activation messages.

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

[ ] Targeted PR against correct branch
[ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
[ ] Added a relevant changelog entry to the Unreleased section in CHANGELOG.md
[ ] Re-reviewed Files changed in the Github PR explorer

@khigh-figure I added the full list of all message Request protos from provenance, and alphabetized all the supported protos by package. Can we get this merged ASAP and the concurrent changes made on test wallet service?


SocketException only via VS Code - not Android Studio

Type: Bug
During an http REST call using http package, I receive the error:
"SocketException (SocketException: Failed host lookup: 'somedomain.com' (OS Error: No address associated with hostname, errno = 7))".
this is strictly happening on an x86 emulator when running/debugging (F5) in VS Code (VSC).

it does NOT happen with the SAME emulator running through Android Studio (AS).
it does NOT happen on any APK install built from VSC or AS when run on a PHYSICAL device.
it does NOT happen when run from command prompt using "flutter run".

App has Internet access and the setting in AndroidManifest.xml ('') in both manifest files.
Others have indicated the issue is related to how VSC is running the app.  I've looked at the launch.json, tried deleting it altogether, tried various settings to no avail.  Current launch.json is:
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "my_project",
"request": "launch",
"type": "dart"
},
{
"name": "my_project (profile mode)",
"request": "launch",
"type": "dart",
"flutterMode": "profile"
},
{
"name": "my_project (release mode)",
"request": "launch",
"type": "dart",
"flutterMode": "release"
}
]
}
Another piece of info which may help... if I use an x86_64 emulator (which I guess is 64-bit), and set the "run/debug" setting in VSC to either "profile" or "release" (option defined in this launch.json file), the app runs fine without error via VSC.  It's only when I run the other emulator (x86) through VSC I get the error.

The emulator runs in normal way if you uncheck "Uncaught Exception"

Yes!  That was around the time I started experiencing it.  Sorry for you, but also happy that others see same thing - that way it's not just my end and hopefully can be addressed.
  I must say it makes me a bit uneasy for other things slipping through, but I'll take it for now and fingers crossed a solution is found soon.

Please file this on the dart extension repo

Please file this on the dart extension repo

Happy to!  Can you please point me where to do that?

I have the same issue since august 15th. The emulator runs in normal way if you uncheck "Uncaught Exception"

They closed this issue and asked to re-post on "dart extension repo".  I've Googled and have no idea where to do this.  You?


Users don't know when syncing is done

Description
As a user, I do not have sufficient logging to tell me when my sync is fully completed. The world state sync is relatively clear in its progress with percentages and clear logs, but the block import step is much more unclear. We do not show how far from head we are and do not tell the user what percentage of blocks are imported.
Acceptance Criteria

Progress bar or percentage complete for block import step that is similar to world state download step
Clear logging when world state is complete that the block import is not yet completed
Clearer logging around Fork Choice Update WHILE syncing to not confuse the user. Could move to TRACE or DEBUG until sync is complete. Remove FCU calls for users while syncing or make them very clear in what they are saying for Engine-API calls.

Provided are some logs
2022-11-07 17:05:36.277-05:00 | EthScheduler-Services-43 (importBlock) | INFO  | FastImportBlocksStep | Completed importing chain segment 6364401 to 6367600 (3200 blocks in 10187ms), Peers: 21
2022-11-07 17:05:38.376-05:00 | EthScheduler-Services-3 (batchPersistAccountData) | INFO  | SnapsyncMetricsManager | Worldstate download in progress synced=99.74%, accounts=15440824, slots=215959150, codes=5240820, nodes=267968473
2022-11-07 17:05:44.342-05:00 | EthScheduler-Services-28 (requestCompleteTask) | INFO  | SnapWorldDownloadState | Running world state heal process from peers with pivot block 7911686
2022-11-07 17:06:02.958-05:00 | EthScheduler-Services-43 (importBlock) | INFO  | FastImportBlocksStep | Completed importing chain segment 6367601 to 6370800 (3200 blocks in 10362ms), Peers: 21
2022-11-07 17:06:35.194-05:00 | EthScheduler-Services-43 (importBlock) | INFO  | FastImportBlocksStep | Completed importing chain segment 6370801 to 6378200 (7400 blocks in 10144ms), Peers: 21
2022-11-07 17:06:38.441-05:00 | EthScheduler-Services-26 (batchPersistTrieNodeData) | INFO  | SnapsyncMetricsManager | Healed 108189 world state nodes
2022-11-07 17:07:00.413-05:00 | EthScheduler-Services-28 (requestCompleteTask) | INFO  | SnapWorldDownloadState | Pausing world state download while waiting for sync to complete
2022-11-07 17:07:10.632-05:00 | EthScheduler-Services-43 (importBlock) | INFO  | FastImportBlocksStep | Completed importing chain segment 6378201 to 6384600 (6400 blocks in 10127ms), Peers: 22
It is not here that SnapSync is complete and waiting for Backwards sync when block import is completed.

closing for changes to sync logging in 22.10.x


Enter / Select

Type: Bug
Since the latest version (1.76.0) I have the problem that when entering an element, it is no longer selected for Enter = Insert.
VS Code version: Code 1.76.0 (92da9481c0904c6adfe372c12da3b7748d74bdcb, 2023-03-01T10:22:44.506Z)
OS version: Windows_NT x64 10.0.19045
Modes:
Sandboxed: No
Extensions (11)

Extension
Author (truncated)
Version

vscode-markdownlint
Dav
0.49.0

vscode-eslint
dba
2.4.0

parameter-hints
Dom
0.2.7

vscode-firefox-debug
fir
2.9.8

vscode-language-pack-de
MS-
1.76.2023030109

isort
ms-
2022.8.0

python
ms-
2023.4.0

vscode-typescript-next
ms-
5.1.20230302

material-icon-theme
PKi
4.24.0

LiveServer
rit
5.7.9

quokka-vscode
Wal
1.0.527

Thanks for creating this issue! We figured it's missing some basic information or in some other way doesn't follow our issue reporting guidelines. Please take the time to review these and update the issue.
Happy Coding!

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.
Happy Coding!


Issue after configuration

Hi!
I'm trying to configure that component but I get an error (I'll pass it at the end). I also have the sems portal component, but Il prefer this one, to get the data from the inverter directly.
What am i doing bad?
Thanks for you help!
Error:
Logger: homeassistant
Source: custom_components/goodwe/sensor.py:104
First occurred: 9:20:05 (1 occurrences)
Last logged: 9:20:05

Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/config/custom_components/goodwe/sensor.py", line 98, in async_refresh
    inverter_response = await self.entity.inverter.read_runtime_data()
  File "/config/custom_components/goodwe/goodwe_inverter.py", line 1000, in read_runtime_data
    raw_data = await self._read_from_socket(self._READ_DEVICE_RUNNING_DATA)
  File "/config/custom_components/goodwe/goodwe_inverter.py", line 407, in _read_from_socket
    return await command.execute(self.host, self.port)
  File "/config/custom_components/goodwe/goodwe_inverter.py", line 374, in execute
    await on_response_received
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/config/custom_components/goodwe/sensor.py", line 104, in async_refresh
    raise PlatformNotReady
homeassistant.exceptions.PlatformNotReady

Which inverter model ?
And/or can you check the connection (and provide the logs) by executing the inverter_test.py ?

My inverter model is: GW5048-EM
I'm ussing dockers. In wich docker I must to execute this inverter_test.py?
I'm trying to execute on host and on homeassistant but it return an error "import custom_components.goodwe.goodwe_inverter as inverter"

You need to checkout/download whole directory structure (resp. the custom_components/goodwe folder) to be able to run the inverter_test.py.
Anyway, looking at the code, it smells like network issue, either it can't connect to the inverter or there are too many drop-outs.
The inverter_test.py output would help.

Yes, this inverter is connected by an Ethernet module, not the WiFi one.
Before the change of the module we have problems with the connection, but
now is all perfect and correct. Could the Ethernet module be the problem?
If I tried to use one of the examples to request to 48899 port as the omnik
inverter from a python script I have a response without problems, with the
name of the Ethernet module; and the app of the inverter works fine too.
Tell me what I can do to get more information to solve the problems.
Thanks
El lun., 5 oct. 2020 17:30, mletenay<EMAIL_ADDRESS>escribió:

Something very odd is happening on your network and it is truncating the
response packets.
You should have received response for the "version" request with length of
86 bytes, but in 3 attempts you got (same content) various lengths - 84, 19
and 86.
Same problems for the "info" request - never got the full-size response.
UDP is unreliable, but this is way too much.
Can you check your network ? Do you have sufficient wifi signal strength ?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/mletenay/home-assistant-goodwe-inverter/issues/10#issuecomment-703708464,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJKE7WYJOXFR5S63IPPDAULSJHRCRANCNFSM4SEUWUWQ
.

I'm not sure what is the source of the problems, if it is IP stack of the HA machine, or some docker network issue or some actual network problem. Under normal circumstances, you should not get incomplete udp packets.
There's very little I can do on the python code level, actually the only thing I can do is to increase the re-try count from 3 to some larger number. And I can improve the exception handling.
But at best case you'll get very unreliable readings from the inverter.
I suggest to use the inverter_test.py to somehow trace the source of the network problems. In normal situations, you should get the packets with right size and only very seldom some re-try attempts.

I've tried again the inverter_test.py and I have this output
2020-10-06 08:32:04,359 __init__(53) - DEBUG: Using selector: EpollSelector 2020-10-06 08:32:04,359 discover(471) - DEBUG: Probing inverter at <IP_ADDRESS>:8899 2020-10-06 08:32:04,360 connection_made(315) - DEBUG: Send: 'aa55c07f0102000241' 2020-10-06 08:32:04,534 datagram_received(326) - DEBUG: Received: 'aa557fc001824d31303130394757353034382d454d20ffffffffffffffffffffffffffffffff3935303438454d553139335730303433333630303431302d30343030302d3130033431302d30323033342d3039101f64' 2020-10-06 08:32:04,535 discover(485) - DEBUG: Detected ES inverter GW5048-EM, S/N:95048EMU193W0043 2020-10-06 08:32:04,535 __init__(53) - DEBUG: Using selector: EpollSelector 2020-10-06 08:32:04,536 connection_made(315) - DEBUG: Send: 'aa55c07f0106000245' 2020-10-06 08:32:05,083 datagram_received(326) - DEBUG: Received: 'aa557fc00186850a010001020dcc00010201ff0000005000dc0005007d000000000900096401000001091c0001021d138801091c0003021d1387' 2020-10-06 08:32:05,083 datagram_received(334) - DEBUG: Unexpected response length: expected (142, 149), received: 58 2020-10-06 08:32:05,083 connection_made(315) - DEBUG: Send: 'aa55c07f0106000245' 2020-10-06 08:32:05,560 datagram_received(326) - DEBUG: Received: 'aa557fc0018685' 2020-10-06 08:32:05,560 datagram_received(334) - DEBUG: Unexpected response length: expected (142, 149), received: 7 2020-10-06 08:32:05,560 connection_made(315) - DEBUG: Send: 'aa55c07f0106000245' 2020-10-06 08:32:06,046 datagram_received(326) - DEBUG: Received: 'aa557fc00186850a000001020dd000010201ff0000005000dc0005007d0000000009000964010000010922000102161388010922000302251388010200e700000000000148cc0000' 2020-10-06 08:32:06,046 datagram_received(334) - DEBUG: Unexpected response length: expected (142, 149), received: 72 2020-10-06 08:32:06,046 connection_made(315) - DEBUG: Send: 'aa55c07f0106000245' 2020-10-06 08:32:06,529 datagram_received(326) - DEBUG: Received: 'aa557fc001868506410001020d9b00000201ff0000005000dc0005007d0000000009000964010000010920000102131388010920000302241387010200e700000000000148cc000022460026003b0001668b001301003002000100010000fc7f000800060000140a06081e3700000000000000000000000000000000000000000000000000007b08000070c510ef' vpv1: PV1 Voltage = 160.1 V ipv1: PV1 Current = 0.1 A ppv1: PV1 Power = 16 W pv1_mode: PV1 Mode = 2  pv1_mode_label: PV1 Mode = Working,PV has a power output  vpv2: PV2 Voltage = 348.3 V ipv2: PV2 Current = 0.0 A ppv2: PV2 Power = 0 W pv2_mode: PV2 Mode = 2  pv2_mode_label: PV2 Mode = Working,PV has a power output  ppv: PV Power = 16 W vbattery1: Battery Voltage = 51.1 V battery_temperature: Battery Temperature = 22.0 C ibattery1: Battery Current = 0.5 A pbattery1: Battery Power = 26 W battery_charge_limit: Battery Charge Limit = 125 A battery_discharge_limit: Battery Discharge Limit = 0 A battery_status: Battery Status = 0  battery_soc: Battery State of Charge = 9 % battery_soh: Battery State of Health = 100 % battery_mode: Battery Mode = 1  battery_mode_label: Battery Mode = Spare  battery_warning: Battery Warning = 0  meter_status: Meter status = 1  vgrid: On-grid Voltage = 233.6 V igrid: On-grid Current = 0.1 A pgrid: On-grid Export Power = -531 W fgrid: On-grid Frequency = 50.0 Hz grid_mode: Work Mode = 1  grid_mode_label: Work Mode = The inverter is generating  vload: Back-up Voltage = 233.6 V iload: Back-up Current = 0.3 A pload: On-grid Power = 548 W fload: Back-up Frequency = 49.99 Hz load_mode: Load Mode = 1  load_mode_label: Load Mode = Inverter and the load is disconnected  work_mode: Energy Mode = 2  work_mode_label: Energy Mode = Normal(On-Grid)  temperature: Inverter Temperature = 23.1 C error_codes: Error Codes = 0  e_total: Total PV Generation = 1863.6 kWh h_total: Hours Total = 8774  e_day: Today's PV Generation = 3.8 kWh e_load_day: Today's Load Consumption = 5.9 kWh e_load_total: Total Load = 2625.1 kW total_power: Total Power = 19 W grid_in_out: On-grid Mode = 2  grid_in_out_label: On-grid Mode = Inverter gets power from grid  pback_up: Back-up Power = 1 W plant_power: Plant Power = 549 W diagnose_result: Diag Status = 524294  house_consumption: House Comsumption = 573 W
Well, as you can see there is all the data of the inverter. I think, the problem could be the time that the script is waiting the response, or de number of the attemps, as you said before.
Maybe, changing the handling of this in the script, thouth it translate to less accuracy or less reliability; it can fix the error on hassio.
P.S. Sorry about my english, it isn't very good.

I have improved the exception handling, so you should get data in HA, at least from time to time.
But until the packet truncation problem is resolved, it will be unreliable.
Download the updated sources (mostly goodwe_inverter.py) and try to execute that several times to see how often you get correct results.
Then increase the "timeout: int = 2" value on line 308 in the goodwe_inverter.py and test it again if it has any effect on the packet truncation and data retrieval reliability.

Thanks  @mletenay, it's seems to work without problems. Finally, I didn't modify the timeout, maybe, It can have lost packets, but in the front I see all the sensors well.
Thanks for your work!


Fix binary Range column partition test before v5.1.0

What problem does this PR solve?
Binary Range column partition is not supported before TiDBv5.1.0
What is changed and how it works?

Check TiDB version when test
Admin check when table is exists

Check List
Tests

Integration test

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/merge


Got stuck unable to interact or move.

Died and the mod just... did not do anything. No idea why.

Can you send the logoutput.txt file?

Also, when you say that you were unable to move, do you mean the camera was in the vanilla spectate mode or that the camera was entirely frozen in place?

I was able to swap between vanilla and poltergeist, but was entirely frozen in poltergeist.

Can you send the logoutput.txt file?

Sorry, I only realized I should put in the bug report a while after it happened.

Hm, okay. Please send the file if this happens again. Unfortunately, I can't do much based on the information available right now, but I'll keep an eye on this issue in case you respond again.

Lmk if anything more comes of this, but I'm going to close it for now due to not being able to replicate.


ExcelStyleDateFormatter incorrect Format result

There is a number for example 44736.1070318287. In Excel with the following format strings it gives the following results:

dd.mm.yyyy hh:mm:ss.000 => 24.06.2022 02:34:07.550
dd.mm.yyyy hh:mm:ss.00 => 24.06.2022 02:34:07.55
dd.mm.yyyy hh:mm:ss.0 =>24.06.2022 02:34:07.5
dd.mm.yyyy hh:mm:ss =>24.06.2022 02:34:08

For new DataFormatter().FormatCellValue(cell); the results are as follows:

dd.mm.yyyy hh:mm:ss.000 => 24.06.2022 02:34:07.550 => test OK
dd.mm.yyyy hh:mm:ss.00 => 24.06.2022 02:34:07.55 => test OK
dd.mm.yyyy hh:mm:ss.0 =>24.06.2022 02:34:07.5 => test OK
dd.mm.yyyy hh:mm:ss =>24.06.2022 02:34:07 => test FAILED (!)

As far as I can see in the source code, the value of milliseconds of the output date-time string never affects the value of seconds. This differs from the behavior of Excel, which, if you do not specify milliseconds in the formatting string, always tries to round the available equal or more than 500 milliseconds to the nearest larger second if there are no milliseconds in the formatting string.
Maybe it's worth changing the ExcelStyleDateFormatter.Format method so that it takes into account this rounding of seconds if there are no milliseconds in the formatting string, or should it be fixed in some other way, what do you think?

This looks to be a potential minor bug. Are you willing to create a PR?

我也遇到了这个问题，在我本地也就是 win10 里面会出现这个问题，然后发布到server2012里面，这个问题就不会出现了


alter_table does not work

The gem defines an alter_table operation, but it appears to be unfinished. It calls an undefined method 'construct_xml_stream'.

You're absolutely right and I believe the method is a remnant of the previous hbase-ruby library I based my earlier work on. I'll be implementing this method soon but for now, I'm having it throw a NotImplementedError. Stay tuned!


website: implement home page

Created an introductory landing page. This page features critical details about capa and its features, It offers a comprehensive introduction to capa's capabilities.
The page can be found at https://deeyasingh.github.io/


Initial pull requst for review for adding a new QuantumCircuit constructor

Summary
First commit on #8709
This is a first commit for adding a new QuantumCircuit constructor QuantumCircuit.from_instructions.
It uses a rough interface, so it is missing bookkeeping and set tracking. The code for testing the method from_instruction can be found in circuit_draw.py.
Details and comments
Method for the constructor
@classmethod
    def from_instructions(
        cls, instructions: Iterable[CircuitInstruction], *, name=None, metadata=None, global_phase=0
    ):
        """Return a circuit from instructions

        Returns:
            QuantumCircuit: a circuit from instructions
        """

Hi @BramDo, welcome to the community! Take a look to the contributor guidelines if you didn't already. Also, considering starting with a good first issue as help wanted issues tend to assume your are familiar with the Qiskit internals and code-style.
You can also pass by #qiskit-pr-help on Slack if you need extra guidance.

I dont think this works with classical bits (let allow with conditions). Testing is always needed, but even more for this kind of code with so many corner cases.
The comment #8709 (comment) provides really good initial guidance on the direction.

Thank you for the review and the helpful changes. I will investigate  how to implement this with classical bits according to the mentioned guidance. Good point about the conditions, that part is still open.  I will go to slack for further questions.

This is still work in progress.

Bram has dropped out of the QAMP project this PR was for. I will take over this issue. Please close this PR and I'll open a new one.


Error in testing phase due to missing type conversion

Faced the following build error :

:~/netobserv-new$ make build
### Check if prerequisites are met, and installing missing dependencies
### Formatting code
### Linting code
WARN [runner] The linter 'structcheck' is deprecated (since v1.49.0) due to: The owner seems to have abandoned the linter.  Replaced by unused.
WARN [runner] The linter 'deadcode' is deprecated (since v1.49.0) due to: The owner seems to have abandoned the linter.  Replaced by unused.
WARN [runner] The linter 'varcheck' is deprecated (since v1.49.0) due to: The owner seems to have abandoned the linter.  Replaced by unused.
WARN [linters_context] structcheck is disabled because of generics. You can track the evolution of the generics support by following the https://github.com/golangci/golangci-lint/issues/2649.
### Testing code
# github.com/netobserv/netobserv-ebpf-agent/pkg/grpc [github.com/netobserv/netobserv-ebpf-agent/pkg/grpc.test]
pkg/grpc/grpc_test.go:72:24: cannot use 0xaabbccdd (untyped int constant<PHONE_NUMBER>) as int value in argument to assert.EqualValues (overflows)
pkg/grpc/grpc_test.go:83:24: cannot use 0x88776655 (untyped int constant<PHONE_NUMBER>) as int value in argument to assert.EqualValues (overflows)
pkg/grpc/grpc_test.go:84:24: cannot use 0xddccbbaa (untyped int constant<PHONE_NUMBER>) as int value in argument to assert.EqualValues (overflows)
# github.com/netobserv/netobserv-ebpf-agent/pkg/exporter [github.com/netobserv/netobserv-ebpf-agent/pkg/exporter.test]
pkg/exporter/kafka_proto_test.go:53:24: cannot use 0xaabbccddeeff (untyped int constant<PHONE_NUMBER>02975) as int value in argument to assert.EqualValues (overflows)
pkg/exporter/kafka_proto_test.go:54:24: cannot use 0x112233445566 (untyped int constant<PHONE_NUMBER>6582) as int value in argument to assert.EqualValues (overflows)
pkg/exporter/kafka_proto_test.go:55:24: cannot use 0xC0010203 (untyped int constant<PHONE_NUMBER>) as int value in argument to assert.EqualValues (overflows)
?   	github.com/netobserv/netobserv-ebpf-agent/cmd	[no test files]
ok  	github.com/netobserv/netobserv-ebpf-agent/e2e/cluster	0.036s	coverage: 2.2% of statements in ./...
?   	github.com/netobserv/netobserv-ebpf-agent/e2e/cluster/tester	[no test files]
?   	github.com/netobserv/netobserv-ebpf-agent/examples/flowlogs-dump/server	[no test files]
?   	github.com/netobserv/netobserv-ebpf-agent/examples/ipfix-collector	[no test files]
?   	github.com/netobserv/netobserv-ebpf-agent/examples/performance/server	[no test files]
ok  	github.com/netobserv/netobserv-ebpf-agent/pkg/agent	0.089s	coverage: 19.8% of statements in ./...
?   	github.com/netobserv/netobserv-ebpf-agent/pkg/ebpf	[no test files]
FAIL	github.com/netobserv/netobserv-ebpf-agent/pkg/exporter [build failed]
ok  	github.com/netobserv/netobserv-ebpf-agent/pkg/flow	0.226s	coverage: 6.8% of statements in ./...
FAIL	github.com/netobserv/netobserv-ebpf-agent/pkg/grpc [build failed]
ok  	github.com/netobserv/netobserv-ebpf-agent/pkg/ifaces	0.051s	coverage: 5.3% of statements in ./...
?   	github.com/netobserv/netobserv-ebpf-agent/pkg/pbflow	[no test files]
?   	github.com/netobserv/netobserv-ebpf-agent/pkg/test	[no test files]
FAIL
make: *** [Makefile:97: test] Error 2
pravein@tcnode7:~/netobserv-new$ vi pkg/grpc/grpc_test.go

Is it specific to a version of go? I don't have this failure on my side

Myself and @shachee tried using  go 18.4, golangci-lint 1.50.1 since the recent versions supports 1.18. Which version is recommended?

1.18 should be fine. idk what the problem is.
in any case, your fix looks good, thanks

Fixed and merged.


Dont pass image on bridge

Closing this for now. Once we investigate the performance issues on Android, we'll revisit.


Improved AlertWord Visibility

Split from https://github.com/jocosocial/swiftarr/issues/246
A new personal category called "Alert Word Posts" that goes to a list of the words (similar to forum threads). When you tap on a word you get sent to the post search endpoint with the word pre-filled. This has the benefit of re-using the markasread functionality we already have. Could populate various components with the read/unread/totals from the usernotificationdata.

Apparently we already do this! They get appended at the bottom of the /forums page under a section called "Alert Keywords" that only appears if there is a hit.


feat: Schema update

Describe the feature you'd like to request
As a user I want to be able to write reviews, leave ratings, how much I spent, and the date I went somewhere.
Describe the solution you'd like to see
A new schema to keep track of: restaurantName, reviewDescription, stars, price, date, user, and displayName.
Additional information
The old schema needs to be updated.

Schema was fully updated, after many iterations and has been modified to be in a working state for milestone 1. For future reference, whenever you create a new schema if you do not run pnpm run db:push everything will break on your pnpm dev environment.


Need submissions for additional language support

Soliciting submissions for additional language support.
So far only English and Danish (Thanks Anders Damsgaard) are being worked on.

Languages added so far:
English
Spanish
Italian
German
French


Plugin gives incorrect timer. The kitten timer should be 24 minutes according to the OSRS Wiki

https://oldschool.runescape.wiki/w/Kitten#Hunger
"24 minutes after the kitten is last fed, the player is warned that the kitten is hungry. If the kitten is not fed after 3 minutes, the player gets a second warning indicating it is very hungry ("Meeeooowww!"). If then it is not fed within the next 3 minutes, the kitten will run away. Every time the kitten is fed, the timer resets to 24 minutes."
"A fresh kitten can go without attention for 25 minutes before it warns the player that it needs attention. If its needs are not met after 7 minutes, the kitten will cry out it's feeling lonely ("Meeeooowww..."). Finally, if its cries for attention are not heard in another 7 minutes, the kitten runs away."
The plugin how ever resets to 30 minutes on feed and attension! the 5 minute error caused me to lose my kitten.

too embarassing! What I read is not true. However, I did have a situation where the timer was incorrect and caused me to lose my cat. I now suspect this is caused by mobile client de-sync.


Month view with weekday columns

The current month view always has the first day in the top left and each day follows in sequence thereafter.
When using the month view it would be great to have the option for the columns to be set according to weekday (with the Weekday as a header) and for the first day to start in the relevant column for the weekday that it falls on.
e.g.  given that:

October 2024 starts on a Tuesday
I choose my week to start on a Monday
then:
1 October would start in row 1 column 2

Hey that would be realy nice so that one row is = one week.
I have the Problem in my PC it is 7 day a raw but on the tablet only 5 days.
Maybe somebody has a work around

Id like to see that type of view as well.
@lonelylupus It depends on your screen resolution. You can use a CSS media query to change the width for smaller resolutions.

There are no plans to add a view like this. It's too big a change for the current code base. This card was created as a simple overview, not a full calendar. But it is a possible feature for version 2.0 (which I'm not planning yet, so it might be a while). I'll keep this issue open for now.


[Accessibility] Merge all accessibility changes to master.

Merge all accessibility changes to master.
Also in this PR :

Moved the "beta" logo so it's in a better tabbing order.
Made the Facebook, Twitter and couple of other new buttons accessible.

I tested this PR with Adafruit :
Everything looks good except the tutorial that is fixed in one of the PR below.
I tested this PR with Chibitronics :
Everything looks good except the tutorial and the hidden menu that are fixed in PRs below.
Important
Since we changed the way the markdown is parsed, the tutorial was not starting anymore because it was not parsing the documentation correctly. We must be sure to merge the following PR from Adafruit, Chibitronics to Master at the same time :

Target
PR to approve and merge with the current one

Adafruit
https://github.com/Microsoft/pxt-adafruit/pull/318, https://github.com/Microsoft/pxt-adafruit/pull/287

Chibitronics
https://github.com/Microsoft/pxt-chibitronics/pull/66, https://github.com/Microsoft/pxt/pull/2911

SparkFun
https://github.com/Microsoft/pxt-sparkfun/pull/48

DO NOT merge this without the PRs above.

@veler,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

Build's not passing..

Pull latest master.

Build looks working now.

Add tutorial fixes to this PR:
https://github.com/Microsoft/pxt/commit/56b821f7e76ecf6b8e73c334f4faaacb960c1e2b
and
https://github.com/Microsoft/pxt/commit/260057c59be34eaf6bc4f46e0ae3d20d8484a061

As the way we read the markdown for the tutorial has changed, now the tutorial doesn't start on Adafruit. We should make the adafruit's tutorial MAS compliant too right?

Yes. And test both changes together..

When we update Adafruit, yes -- we need to fix any tutorials

Pull latest master. You shouldn't have changes to vs/editor files

Even after a pull from master, I still see vs/editor files. You should jake clean, and do an npm install..

It must be fixed now

Good work!


Circle and eclipse selectors don't have a widget

I add the plugin of toolbar and selector pack , for the defaul selector rect and polygon they have the widget but the circle and eclipse selectors doesn't have it  , how can I add it to them ?
edit : in the basic example it work good but by adding anything to the anno init , the eclipse and circle stop having widget , right now i just tried to add formatter of shape label and it didn't work , always there is error n.reduce is not a function.
here my code :

    <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/<EMAIL_ADDRESS>

Not sure if this is a copy and paste error. But are you passing the undefined anno variable to the SelectorPack before you are initializing anno?

I tried to pass it after the init like this :
 var anno = Annotorious.init({
                image: 'hallstatt',
                widgets: [
                  ColorSelectorWidget

                ],
                formatter: ColorFormatter
              });
              Annotorious.SelectorPack(anno);
              Annotorious.Toolbar(anno, document.getElementById('my-toolbar-container'));

but still circle and eclipse don't show any widget .

Are any error messages showing up in the browser console?

yes there is this error :
Uncaught TypeError: n.reduce is not a function
    at _e (index.js:43:1)
    at new u (index.js:43:1)
    at a.createEditableShape (index.js:43:1)
    at AnnotationLayer.js:436:48
    at kS.emit (index.js:36:20)
    at ToolsRegistry.js:57:46
    at a.emit (index.js:43:1)
    at index.js:43:1
    at i.mouseUp (index.js:43:1)
_e @ index.js:43
u @ index.js:43
(anonymous) @ index.js:43
(anonymous) @ AnnotationLayer.js:436
emit @ index.js:36
(anonymous) @ ToolsRegistry.js:57
emit @ index.js:43
(anonymous) @ index.js:43
i.mouseUp @ index.js:43

and these ones
Error: <ellipse> attribute ry: Expected length, "undefined".
Ke @ index.js:43
Je @ index.js:43
e @ index.js:43
(anonymous) @ index.js:43
(anonymous) @ index.js:43
(anonymous) @ AnnotationLayer.js:161
index.js:43 Error: <ellipse> attribute ry: Expected length, "undefined".
Ke @ index.js:43
Je @ index.js:43
e @ index.js:43
(anonymous) @ index.js:43
(anonymous) @ index.js:43
(anonymous) @ AnnotationLayer.js:161
index.js:43 Error: <path> attribute d: Expected number, "…99 h-706 z M439 NaN a 2 NaN 0 1 …".

maybe i did mistake importing the scripts :


feat(ci): upgrades ftw to python 3 supported version

Signed-off-by: Felipe Zipitria<EMAIL_ADDRESS>This PR upgrades ftw to 1.2.3, which has py3 support.

Thank you!
This is p ending the ftw release (should happen in the next few hours).

@fzipi : This has some tests failing. Otherwise it would be good to go, would not it?
And are the failing tests failing because of the changes to FTW?

@fzipi : This has some tests failing. Otherwise it would be good to go, would not it?
And are the failing tests failing because of the changes to FTW?

@dune73 Yes, this is becauseof the changes regarding the py3 full support. Some things are handled differently by py2 and py3, in particular string vs. bytes and encoding.
With @fgsch were in the mix for getting a proper version but it might be a breaking change for some users (might be). So we cannot merge this one yet.

@dune73 Yes, this is becauseof the changes regarding the py3 full support. Some things are handled differently by py2 and py3, in particular string vs. bytes and encoding.
With @fgsch were in the mix for getting a proper version but it might be a breaking change for some users (might be). So we cannot merge this one yet.

OK. So this is pending for the time being. Thank you.

OK. So this is pending for the time being. Thank you.

Any update on this front @fzipi and @fgsch?

Looks like python 3.9 fails with the 1.2.3 version, but 3.8 works. Will then fix the pipeline run to 3.8 for now.

Great work @fzipi. Thank you very much.

Finally, it is working. We might want to merge! @dune73


BUG: Sparse matrix multiplication causes segmentation fault

I am getting segmentation fault when I multiply a scipy sparse matrix by its transpose. Same question also posted also to http://stackoverflow.com/questions/32321375/sparse-matrix-python-segmentation-fault

    >>> import cPickle
    >>> fs = open('vec.pickle', 'rb')
    >>> vec = cPickle.load(fs)
    >>> vec
    <3020x512 sparse matrix of type '' with 26008 stored elements in Compressed Sparse Column format>
    >>> vec.max()
    10.0
    >>> vec.min()
    0.0
    >>> vec * vec.T
    Segmentation fault: 11

I do not think this is memory issue since the dimension is small. The vec object is created by gensim, if that information helps.
I also do not think this is overflow issues since the range of element is [0.0, 10.0]
The pickle object is here:
https://drive.google.com/open?id=0B3DJbsn85XMvdmFYT0MzZVFjOVU

It looks like your matrix is malformed. I tried running vec.toarray() and got:
ValueError: row index exceeds matrix dimensions

Thanks for pointing out. The matrix does look strange when I call check_format() on that. However, I would expect SciPy to throw exception rather than seg faulting.

Looks like I provided wrong input to gensim which caused creation of bad matrix. However, it would be nice if SciPy could help with error check so that such matrix cannot be created. This ticket can be closed if the SciPy community thinks the behavior is fine the way it is.

The sparse matrix __init__ calls self.check_format(full_check=False), where full_check=True should enable the index checks. The check on construction in principle has a runtime cost, so some bencmarking at least would be useful before deciding to enable it by default.

@pv good point.


bf: close/end readable/response streams on errors

@ironman-machine r+

Hello @rahulreddy
"r+": Success

:hourglass: PR is now pending. CI build url: http://ci.ironmann.io/gh/scality/Integration/18340

:sunny: :+1: circleCI test succeeded!


Allow to configure the HEARTBEAT_TIMEOUT

What is your Test Scenario?

A test case needs long time to run. And sometimes heartbeat does not send periodically with HEARTBEAT_TIMEOUT
What are you suggesting?

Allow to configure the HEARTBEAT_TIMEOUT. A longer timeout can work around the heartbeat bug.
What alternatives have you considered?

Make sure that the heartbeat can be sent.
Additional context

Hi @allayli
It looks like a TestCafe problem. Would please provide an example of your test case for which you want to increase HEARTBEAT_TIMEOUT?


Any way to contribute to this project

Thank you for this amazing library.
In order to help reach this library reach version 4, any way I can help on this project? Can you share the roadmap and your overall plan? Any pending APIs, types that needs to be covered?

@mistyharsh - thanks, however v4 is now available


NIFI-5661: Adding Load Balance config UI

Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[x] Has your PR been rebased against the latest commit within the target branch (typically master)?

[x] Is your initial contribution a single, squashed commit?

For code changes:

[ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[ ] Have you written or updated unit tests to verify your changes?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

Will review...

@mcgilman Thank you for reviewing! Incorporated the comments.

Thanks for the update. We do not want to reference another module directly via nf.. This would only be working currently because we have not enabled a module loader yet. However, the codebase currently supports one. This can be seen at the top of any of the JS files. If a module loader was turned on, this would fail because the nf. namespace would no longer be in global scope.
If you want to reference something in another module, we'll need to import it. Alternatively, you could import it in a module that is already imported in both. In the policy case, the combo options are in nfCommon and a utility function was added to access them.

@mcgilman I understand that. I moved the combo options to nfCommon. Hope this PR looks good now. Thanks!

Thanks @ijokarumawak! This has been merged to master.


vscode run to show  configuration.jsonValidation.url  error

This error is reported every time it is started, no solution is found.
The error message is as follows：
[tht13.python]: "configuration.jsonValidation.url" 必须是绝对 URL 或者以 "./" 开头，以引用扩展中的架构。
[tht13.python]: " configuration.jsonValidation.url 'must be an absolute URL or start with'. / 'to refer to the schema in the extension.

Can you file this against the tht13.python extension? Thanks!


cat: move cat to clap

Yet another port to clap, current test suite passes, also fixed a one pretty small clippy warning.

Thanks! :)


TypeError: Array.prototype.slice called on null or undefined

Running the etcher site with landr locally installed throws on the server render (not on compile). Interestingly it doesn't happen on the Balena site.

This is because of an incompatible version of color-converter and color. Etcher is pulling in another version and it's resolving to that, I assume npm shrinkwrap would fix this.


feat(analytics): Revert api_event metrics and filters back to merchant_id authentication

Type of Change

[x] Bugfix
[ ] New feature
[ ] Enhancement
[ ] Refactoring
[ ] Dependency updates
[ ] Documentation
[ ] CI/CD

Description

Previously modified api_event_metrics and api_event_filters to authenticate through AuthInfo where there are MerchantLevel, ProfileLevel and OrgLevel authentication. (https://github.com/juspay/hyperswitch/pull/5729)
But ApiEvents table is not populated with organization_id and profile_id, hence reverting back.
Additional Changes

[ ] This PR modifies the API contract
[ ] This PR modifies the database schema
[ ] This PR modifies application configuration/environment variables

Motivation and Context

Fix the issues on sandbox analytics related to api_events.
How did you test it?

Checklist

[x] I formatted the code cargo +nightly fmt --all
[x] I addressed lints thrown by cargo clippy
[x] I reviewed the submitted code
[ ] I added unit tests for my changes where possible

Make sure FE has also reverted the changes.
@lsampras, @tsdk02.


Numpy bitwise_and operation added

Description
Numpy element-wise operator bitwise_and added
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.

[x] Changes are complete (i.e. I finished coding on this PR)
[x] All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
[x] Code is well-documented
[x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

[x] Added numpy element-wise operator: bitwise_and

Comments
@haojin2

Waiting for GCD PR's  INT_TYPE_SWITCH to merge. Will use it to restrict the input dtype.

This operator will be directly moved to the master branch. See #16009.


[Feature Request] Specific Song

Select specific song out of library in spotify modal

landed in https://github.com/Aetheryx/aethcord/commit/ac757b5f1174d6741bf36b31ac11f497912aa242


Use OJ for JSON

This is primitive implementation and will not work with JVM. But we can use oj optionally when it is available. What your thoughts? Am I missing something here?
Maybe we should provide something like json_engine config.
Ticket: #117

Codecov Report

Merging #118 into master will increase coverage by 0.38%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #118      +/-   ##
==========================================
+ Coverage   91.62%   92.01%   +0.38%
==========================================
  Files          61       61
  Lines        1469     1452      -17
==========================================
- Hits         1346     1336      -10
+ Misses        123      116       -7

Impacted Files
Coverage Δ

lib/logstash-logger/formatter/json.rb
100% <100%> (ø)
:white_check_mark:

lib/logstash-logger/formatter/json_lines.rb
100% <100%> (ø)
:white_check_mark:

lib/logstash-logger/multi_logger.rb
63.88% <0%> (-8.34%)
:x:

lib/logstash-logger/buffer.rb
71.42% <0%> (-0.62%)
:x:

lib/logstash-logger/device/base.rb
77.5% <0%> (+1.89%)
:white_check_mark:

lib/logstash-logger/device/connectable.rb
82.25% <0%> (+3.99%)
:white_check_mark:

lib/logstash-logger/device/tcp.rb
100% <0%> (+4%)
:white_check_mark:

lib/logstash-logger/device/kinesis.rb
89.47% <0%> (+4.47%)
:white_check_mark:

lib/logstash-logger/silenced_logging.rb
97.05% <0%> (+5.16%)
:white_check_mark:

lib/logstash-logger/formatter/cee_syslog.rb
100% <0%> (+7.69%)
:white_check_mark:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f97c6be...e878dbc. Read the comment docs.

Hi, thanks for the thoughtful contribution.
I'd prefer not to introduce the complexity of supporting multiple JSON engines. JSON seems to be "fast enough" for most use cases.
I'm a fan of Oj and it's definitely faster. The recommended way to take advantage of Oj is to use the oj_mimic_json gem.
If you really need a different JSON engine, you can always write your own formatter that inherits from the built in JSON formatter that calls any engine you need.
I'll document this in the readme.

I'm a fan of Oj and it's definitely faster. The recommended way to take advantage of Oj is to use the oj_mimic_json gem.

I tried it, but to_json still goes through Rails calls. I profiled with stackprof, and it appears that to_json doing biggest number of object allocations.
Yes json is "fast enough", but it is not "memory friendly enough". Ruby will eventually clean up all unclaimed object with GC, but this results in memory spikes if you use a lot of logging. Big number of allocations increase fragmentation of memory.

I'll consider adding support for MultiJSON if it's done in a clean, configurable way. Do you want to take a stab at it?

Just to make sure we understand what is the problem https://github.com/stereobooster/ruby-json-benchmark
You are claiming to use JSON. But to_json inside Rails project is not the same as JSON.generate. So even JSON.generate instead of to_json will improve situation. Will look into MultiJson later.

UPD: actually usage of JSON.generate is enough. To get Oj simply use oj_mimic_json. We can add this as a note to Readme.

I updated PR ^

There are actually quite substantial differences between JSON.generate and ActiveSupport::JSON.encode. One of the more important ones is support for as_json to customize object output. So simply switching to JSON.generate could be a breaking change for a lot of people.
What I have in mind is making the LogStashLogger JSON serializer configurable. In Rails it would default to ActiveSupport::JSON, and otherwise it would default to JSON. MultiJson would also be a built in serializer.

Is this a case for Logstash::Event? Specs from official repo use dump and load see LogStash::Json for LogStash::Json.load, LogStash::Json.dump.

Logstash code is unrelated to LogStash::Event code. LogStash::Event is a separate Ruby gem. I checked, and internally Logstash::Event uses JSON.parse and #to_json. LogStashLogger does its own processing of JSON as well.

My bad. Confused gems. Can you direct me to source code of LogStash::Event. Because https://rubygems.org/gems/logstash-event/ directs me to https://github.com/elastic/logstash

I don't think the source code is available on Github anymore since Logstash rearranged everything. I looked at the source from the installed gem using bundle open logstash-event.

JSON.generate(@event.respond_to?(:as_json) ? @event.as_json : @event)

There are more differences than just #as_json. Instead of a potentially breaking change upon which hacks and workarounds are piled, I would prefer to have JSON serialization be customizable / configurable (with #to_json continuing to be the default).

I added compatibility test to benchmarks.  And it is more then I thought.

Update: was able to setup Oj in super compatible mode with to_json. And it is faster and use less memory. Check my benchmark repository.

This is looking much better. I'll spend some time reviewing this and will most likely merge it in soon. Thanks!

@stereobooster Was wondering why you closed this PR. Do you think it's no longer necessary due to the compatibility fixes in Oj?

We do not use this gem in our setup anymore


Improve Performance

This is currently the achilles heel in using this library in a production environment. CSS (and the Less, and Sass that can generate it) get all the advantages of being optimized for the browser. Using just JavaScript here means Descartes can't exploit any of those optimizations: how can we improve the performance here? Here are a few ideas and considerations:

Generate a stylesheet. Descartes could generate CSS just like other preprocessors so that basic styles can be cached, served, and rendered. The major issue is making sure that Descartes could accurately do so because of dynamic values: since it uses inline styling and actually does style calculation at the node level, it's more specific than traditional CSS would be.
Fine tune the code. This should happen anyways, but I have a comparatively naive understanding of JavaScript performance where others in the OS community could really contribute. The library is highly recursive, requires a lot of O(n) run throughs multiple times, and doesn't take full advantage of some of JavaScript's concurrency or caching capability. The more ideas, the better.
Eat the cost. Maybe I'm overthinking this? What is really the cost of speed here? Is the flexibility provided by the library to do event-based, highly context-based, and largely cascade free styling worth the cost of performance? In Less and Sass (which also have similar JS libraries that compute style at load time) the cost is not justified because it's mainly used for debugging or development agility, but they do not provide many of the features that Descartes does with DOM-based styling, better computation, and dynamic values. Is there enough in Descartes to tip the scale?

Want to hear people's thoughts.

In my opinion, generating a stylesheet would undermine one of the main values of using Descartes, which is having dynamic styles directly accessible by Javascript after the page has been rendered.
That said, generating the CSS could be an optional configuration for those who want it. Maybe having a cached stylesheet and the JS styles overwrite them once they're loaded? Might affect overall performance, but still have faster initial loading.

@dfosco I think you're spot on, especially with your second point. I think generating a stylesheet and having dynamic styles post-render are not mutually exclusive, precisely because Descartes uses inline styles and would overwrite anything declared by the stylesheet. The stylesheet in this scenario would be relegated to serving a caching and backup function: it would render at least the majority of the styles needed to get the (perceived) performance, then the actual dynamism of the library would kick in as it currently does.

@JonHMChan are you saying that Descartes would split the styles? Generate the base css which is not dynamic and have js part have only the dynamic styles. That sounds like the fastest option.

@krivinarius yes, that's exactly what I'm suggesting. I think having the base styles that are not dynamic (without using a function) would be generated and used for the initial load.

@krivinarius you may be interested to see that https://github.com/JonHMChan/descartes/blob/master/src/descartes.js#L441 has been implemented to generate the base CSS, still no JS generated since I think I may refactor how event listeners are declared to make it a bit easier to bind and more performant overall.


Add AppliedType to the SemanticDB schema

At this point, type application is SemanticDB is modelled within TypeRef, whch carries a prefix, a symbol and type arguments. This is directly inspired by a very similar data structure in Scalac, also called TypeRef.
This, however, means loss of information. Scala syntax allows type application of anything, whereas TypeRef only allows type application of type refs. For example, SemanticDB cannot currently model the signature of type alias T defined below (the code below doesn't compile with Scalac, but it does compile with Dotty, so we'll have to support it):
class ann extends scala.annotation.StaticAnnotation

object M {
  type T = (List @ann)[Int]
}

Therefore, I propose to change the protobuf schema as follows:
message TypeRef {
+  reserved 3;
  Type prefix = 1;
  string symbol = 2;
-  repeated Type type_arguments = 3;
}

+message AppliedType {
+  Type tpe = 1;
+  repeated Type type_arguments = 2;
+}

The final remark is about compatibility. The code that can be modelled with TypeRef+AppliedType is a strict superset of the code that can be modelled with TypeRef alone. Old-style typerefs can always be converted to a combination of a new-style typeref and a new-style appliedtype, but not the other way around.

I'm going to unassign the 4.0.0-RC1 milestone from this ticket. The situation when AppliedType is going to be necessary is quite esoteric, so let's wait out until we learn more about this use case, so that we don't have to rush and change a core aspect of SemanticDB types for nothing.


Fix off-screen items order

Turned out that incorrect items order may cause some issues. And it cannot be fixed by sorting indexes on the user side, because unshift will break their order anyway.
In my case it was an embedded YouTube video that would autoplay multiple times in some conditions. Seems like the wrong items order caused react to re-create DOM elements.

Released in 0.30.1.


Proxy is hardened so it can directly face the internet

We expect several features to be add-ins that plug in as connection middleware. Add a connection middleware to the sample in this repo.
Here's one example from Http2:
https://github.com/dotnet/aspnetcore/blob/09bb7b4ca5a4fbde0283c294c35fac8b485c0074/src/Servers/Kestrel/samples/Http2SampleApp/Program.cs#L41-L54
Other things we expect to need to plug into here:

Sniff SNI and rate limit
Rate limit SSL handshakes by IP

We don't have to write those components as part of this task, just demonstrate that a connection middleware has access to the necessary inputs and controls (e.g. IPs, sniffing the data stream, drop connections, etc.).
Specifically, this involves being able to filter connections based on information contained in the ClientHello TLS frame. Examples include:

Cipher Suite selection
Protocol Version
Server Name Indication (SNI)
Application-Level Protocol Negotiation (ALPN; used for HTTP/2 and HTTP/3)

I did a similar connection middleware component here that we should be able to model this on.

I know that this is outside the scope of the issue, but it would be great to have some middleware to block traffic.  Some simple ones being blocking user agents like curl and GeoIP2 country lookup using MaxMind DB Reader.  Not looking to build an IDS/IPS, but some simple tools would be nice.  At some stage would it be technically feasible to create a connector for Libmodsecurity WAF?

I know that this is outside the scope of the issue, but it would be great to have some middleware to block traffic.  Some simple ones being blocking user agents like curl and GeoIP2 country lookup using MaxMind DB Reader.  Not looking to build an IDS/IPS, but some simple tools would be nice.  At some stage would it be technically feasible to create a connector for Libmodsecurity WAF?

@mguinness I'd suggest moving that question to https://github.com/microsoft/reverse-proxy/discussions.
Blocking like that would be applied at the connection, server, or middleware layer before reaching any of the YARP logic at the routing layer or proxying layers. These scenarios are highly custom and I don't expect we'd be building them as part of the YARP project.

@mguinness I'd suggest moving that question to https://github.com/microsoft/reverse-proxy/discussions.
Blocking like that would be applied at the connection, server, or middleware layer before reaching any of the YARP logic at the routing layer or proxying layers. These scenarios are highly custom and I don't expect we'd be building them as part of the YARP project.

Triage: All is possible using TlsFrameHelper: https://github.com/microsoft/reverse-proxy/blob/main/src/ReverseProxy/Utilities/TlsFrameHelper.cs
We should work with users to create deployment cookbook / how tos.


TronWeb and ethers lib generate different addresses

Hello,
This documentation says: "The TRON address in Hex format removes the leading 41 to get the Ethereum address."
https://developers.tron.network/docs/account#account-address-format
Following source code generates different addresses for same mnemonic phrase and derivation path:
import { HDNodeWallet, ethers } from "ethers";
import { TronWeb } from "tronweb";

  // MNEMONIC
  const mnemonic = "patch left empty genuine rain normal syrup yellow consider moon stock denial";
  const path = "m/44'/195'/0'/0/0";

  // TronWeb
  const keyPair = TronWeb.fromMnemonic(
    mnemonic,
    path
  );
  const tronAddress = keyPair.address;
  const tronHexAddress = TronWeb.address.toHex(tronAddress);
  // 4196bb8957ad294a8920ced289a5ac85872cee8b23

  // ethers
  const masterNode_a = HDNodeWallet.fromMnemonic(ethers.Mnemonic.fromPhrase(mnemonic));
  const etherAddress = masterNode_a.derivePath(path).getAddress();
  // 0x0d31Fc78F7bCCB8C4D32B60E0ed395C0cbD2c483

packages.json
  "dependencies": {

    "ethers": "^6.6.0",

    "tronweb": "^6.0.0-beta.0"
  }

TronWeb returns address 4196bb8957ad294a8920ced289a5ac85872cee8b23 which without first byte is
96bb8957ad294a8920ced289a5ac85872cee8b23. But ethers lib returns 0x0d31Fc78F7bCCB8C4D32B60E0ed395C0cbD2c483
I suppose according to Tron documentation (https://developers.tron.network/docs/account#account-address-format) they must be the same ?
Correct me if I wrong.
Thank you.

You need to pass the path to HDNodeWallet.fromMnemonic as the second parameter. Because when you use HDNodeWallet.fromMnemonic, it already derived a default path for you. You can check out ethers code for more details.


validation for starting the gateway with domains

in managed domains it says --domains

https://github.com/threefoldtech/tfgateway/pull/52


How can I remove the central circle?

I am using the PointerSpeedometer and I need to completely remove the central circle, try putting the color transparent, however it also appears in black. Thanks for the help.

May you take a look at this issue #42 is that what you want?

I will fix this in the next version, thanks for contribution.

fixed in V 1.3.0.


ci(docs): fix remaining import workflow

Pull Request Test Coverage Report for Build 408955215

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.02%) to 48.516%

Totals

Change from base Build 408905216:
0.02%

Covered Lines:
1357

Relevant Lines:
2797

💛  - Coveralls


Roll Dart to 2.3

Should fix https://github.com/grpc/grpc/issues/16580

cc @jtattermusch it's not clear to me why tests are failing, e.g.:
Using default tag: latest
Error response from daemon: pull access denied for grpctesting/grpc_interop_dart_4c2921fdf63f6b338ed13faa0b148ed745e93343, repository does not exist or may require 'docker login'

@mit-mit when updating the docker images, one must run tools/dockerfile/push_docker_images.sh.
I've done so just no, so I'll respawn the tests and see how that goes.

cc @sigurdm, looks like interop tests are still failing with Dart 2.3

Problems:

looks like jwt_token_creds test is failing for dart
some problem with dart client for a portion of tests:

gRPC Error (14, Error connecting: type 'String' is not a subtype of type 'bool')
#0      Tester.clientStreaming (package:interop/src/client.dart:379:22)
<asynchronous suspension>
#1      Tester.runTestCase (package:interop/src/client.dart:125:16)
<asynchronous suspension>
#2      Tester.runTest (package:interop/src/client.dart:108:11)
<asynchronous suspension>
#3      main (file:///var/local/git/grpc-dart/interop/bin/client.dart:107:20)
<asynchronous suspension>
#4      _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:299:32)
#5      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:172:12)

and
Unhandled exception:
Bad state: Future already completed
#0      _AsyncCompleter.complete (dart:async/future_impl.dart:39:31)
#1      Tester.cancelAfterFirstResponse.<anonymous closure> (package:interop/src/client.dart:1138:17)
#2      _RootZone.runUnaryGuarded (dart:async/zone.dart:1314:10)
#3      _BufferingStreamSubscription._sendError.sendError (dart:async/stream_impl.dart:358:15)
#4      _BufferingStreamSubscription._sendError (dart:async/stream_impl.dart:373:16)
#5      _DelayedError.perform (dart:async/stream_impl.dart:602:14)
#6      _StreamImplEvents.handleNext (dart:async/stream_impl.dart:707:11)
#7      _PendingEvents.schedule.<anonymous closure> (dart:async/stream_impl.dart:667:7)
#8      _microtaskLoop (dart:async/schedule_microtask.dart:41:21)
#9      _startMicrotaskLoop (dart:async/schedule_microtask.dart:50:5)
#10     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:116:13)
#11     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:173:5)

@mit-mit @sigurdm  can you take a look at the failures and try to resolve them? We need this fix quickly because currently the entire interop build is failing on our CI.

I have made https://github.com/grpc/grpc-dart/pull/207 and https://github.com/grpc/grpc-dart/pull/208
So now
tools/run_tests/run_interop_tests.py -l dart -s dart --use_docker

Succeeds for me locally.

There are still two issues:
first one is against grpc-dotnet
https://source.cloud.google.com/results/invocations/d3ce41d9-91d6-40e5-a284-7265c5e69222/targets/github%2Fgrpc%2Finterop_test/tests
and the dart jwt_token_creds interop tests seems to be broken
https://source.cloud.google.com/results/invocations/8203e6c9-2740-45b1-b8ee-0afd162da7d9/targets/github%2Fgrpc%2Finterop_test/tests;group=tests;test=cloud_to_prod_auth:dart:default:jwt_token_creds:tls;row=1
I will still merge this PR as it at least improves the situation (the build now succeeds).


Private key generation

Pretty straight forward, adds Data.Krist.PrivateKey.makeKWPrivateKey, which returns a private key for any KristWallet password.

this is :good: :+1:

:cool: man thanks :white_check_mark:

is that like a poor man's ack

yes


Printing API Example is Absent

There is no example using Printing API. I am struggling to get started.

https://github.com/timsneath/win32/issues/262