Roundtripping a commit via oci image looses the commitmeta which makes signed ostree break

This was moved here from https://github.com/coreos/rpm-ostree/issues/4973
I created an oci image based on an automotive image that used signed composefs, when I rebased to it everything works, but on reboot the signatures fails:
[    0.419101] ostree-prepare-root[528]: ostree-prepare-root: Error loading signatures from repo: No commitmeta for commit 24e46035399c03969d5e6f610425a9adfc7a9959507fe9c32f6ccd12b3467573
Seems the detached metadata is lost somewhere along the way, which means that the signature is not available in the deployed repo.

This is a confusing topic, and I would probably agree this is a bug, but it was intentional in the code today.
There are two different codepaths.  Basically this (should) work today when using ostree container encapsulate and ostree container unencapsulate.
When following that path, the container metadata is discarded (manifest, config) - there is nothing else stored except the ostree commit.  The fact that there was a container image involved in transport is just discarded.
But when using ostree container image (a distinct layer in this code) and specifically the ostree container image pull/deploy functionality which is used by bootc (and rpm-ostree's container bits), we e.g. actually retain distinct refs for each container image layer (the ostree/container namespace) and we also create a merge commit.
The thing is, when using the container flow today we currently default to expecting to handle derived layers.
We used to have code which tried to split the difference here...and arguably, it is a bug or at least a design misfeature that we do a merge commit in this case, breaking any ostree signatures at deployment time.
This issue touches on https://github.com/ostreedev/ostree-rs-ext/issues/388 too - if we encourage people to do that, we can start to lean on using ostree signatures in encapsulated container images.
However...IMO, the direction we want to go is less around making ostree-containers special; we should create an opinionated binding between signed composefs and generic OCI.  I will file something about this...

In the short term...I think we can change the store.rs code to go back to not creating a merge commit, but then we'll need to store the manifest and config $somewhere else.  (This isn't hard of course, we could stick them in a lookaside somewhere like /ostree/repo/ext/container or whatever, or maybe better special refs like ostree/container/meta...the downside is just that we'd "leak space" if one ever did a transition from container -> ostree and stopped using the ostree-container stack.  But maybe we should stop caring about that...

This issue (among others) made me feel the need to do a spike on really rethinking how we do a sane container storage, which led to https://github.com/containers/composefs/pull/286 FWIW

Also cc https://github.com/ostreedev/ostree-rs-ext/issues/630 and https://github.com/cgwalters/composefs-oci


Store repo configs in subdirectories named after each repo's org

This PR reorganizes the repo configurations from a flat directory of files like rust.json and log.json into a hierarchy that uses the organization name in the path: ORG/REPO.json. Now we have files like rust-lang/rust.json and rust-lang-nursery/log.json.

Primary benefit: this tidies things up and makes it easier to track down the repositories that belong to a repo config.
Secondary benefit: this makes it possible to support repos with the same name. This is perhaps moot, generally, but as an example, I can't currently use my own davidalber/highfive repository for testing without clobbering the existing highfive.json file.

I'm going to go ahead and merge this. If you have any concerns after it's done, please let me know.


span context

- added span_context and associated tests
- added configuation files and package.json that will be needed

There's a bit odd assortment of files, e.g. partial thrift factories, but no span factory. The main question I have is what data type we're getting from Thrift for trace ID.

When we want to serialize to thrift we have been using es5 Buffers.  When we convert from string to id we convert to a Buffer.  Es6 may have better types for this, but I want to verify which types can be reported through a  udp/thrift binary protocol reporter first.

:shipit:


0.46 Breaks app with ionic storage (0.45 works fine)

Short description of the problem:
The ionic conference app works fine with app-scripts 44 and 45. After I installed 46 I got the following error:  Cannot find module "localforage"
After rolling back to 45 everything works as expected again.
What behavior are you expecting?
Steps to reproduce:

Clone the ionic conference app
Update app scripts to 0.46
Run ionic serve

Which version are you using?
"@ionic/storage": "1.1.6"
"ionic-angular": "2.0.0-rc.3"
"@ionic/app-scripts": "0.0.46"

I also have similar problem. After updating to 0.0.46 webpack can't find some modules. Revert to 0.0.45 fix that problem

An update of ionic@latest and to latest nodejs LTS fixed it for us, dont know which one did it

Also experiencing the same. For some reason firebase (which should be available as global) is not defined.

Can you please try with 0.0.48 ionic app scripts?

This should be resolved.
Thanks,
Dan


GUNA : Error is occured on set sidebar widget: weather

GUNA : Error is occured. Please, see the trace-back message in Sublime Console.
GUNA : ERROR ________________________________________________
Traceback (most recent call last):
  File "C:\Users\pouco\AppData\Roaming\Sublime Text 3\Packages\Guna\core\engine.py", line 747, in jDsqIU_oxSmbzL
  File "./python3.3/urllib/request.py", line 156, in urlopen
  File "./python3.3/urllib/request.py", line 475, in open
  File "./python3.3/urllib/request.py", line 587, in http_response
  File "./python3.3/urllib/request.py", line 513, in error
  File "./python3.3/urllib/request.py", line 447, in _call_chain
  File "./python3.3/urllib/request.py", line 595, in http_error_default
urllib.error.HTTPError: HTTP Error 404: Not Found
============================================================

You have correct settings like this ?
"weather": {
"appid": "751842343244355e36",
"city_name": "seongnam,KR",
"proxy": ""
},

I'm not using or have a proxy.

You don't need to fill proxy if you don't use, but appid, city_name should be filled.

I'm using the same configuration as the previous version
"sidebar_widget": [ "weather" ],
"weather": {
"appid": "e9e31a15a6bfaa***",
"city_name": "3456160"
}

city_name is weird.  once test with your browser.
http://api.openweathermap.org/data/2.5/weather?q=3456160&APPID=e9e31a15a6bfaa***
{"cod":"404","message":"city not found"}

Agora está fucionando perfeitamente.
"city_name": "Nova Iguacu"


Add ability to configure CidrIP of security group rules

This adds .spec.securityGroup.*Rules.CidrIp so that the created security group rules can optionally allow a target IPv4 CIDR range instead of only being allowed for the cluster's own master and worker security groups. If a CIDR range is specified, it will take precendence and be used, while the cluster's own master and worker security groups act as a fallback default.
Basically adds flexibility to the hardcoded <IP_ADDRESS>/0 that was added in #144
OSD-15465

Codecov Report

Merging #152 (d964e88) into main (3f49ed7) will increase coverage by 2.22%.
The diff coverage is 64.81%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #152      +/-   ##
==========================================
+ Coverage   41.75%   43.97%   +2.22%
==========================================
  Files          29       29
  Lines        1588     1594       +6
==========================================
+ Hits          663      701      +38
+ Misses        846      811      -35
- Partials       79       82       +3

Impacted Files
Coverage Δ

api/v1alpha2/vpcendpoint_types.go
100.00% <ø> (ø)

controllers/vpcendpoint/helpers.go
44.01% <64.81%> (+7.65%)
:arrow_up:


Update out of date setup instructions

Updates the README to be up to date with a change made in https://github.com/usdigitalresponse/usdr-gost/pull/294 and https://github.com/usdigitalresponse/usdr-gost/pull/319
Also links to the official nvm setup instructions because brew's install instructions for nvm didn't work correctly, and their webpage indicates it isn't officially supported by nvm (https://formulae.brew.sh/formula/nvm)
Ticket
Description
Screenshots / Demo Video
Testing
Checklist

[ ] Provided ticket and description
[ ] Your PR title contains the ISSUE ticket number e.g "86: ..."
[ ] Provided screenshots/demo
[ ] Provided testing information
[ ] Provided adequate test coverage for all new code
[ ] Added PR reviewers
[ ] Ensure at least 1 review before merging

@mattbroussard sorry I added a couple additional things I found while working through the rest of setup.
brew is no longer supported for managing nvm so I updated those instructions, and the seed filename was changed too


[Codegen] Extract RootTag case of translateTypeAnnotation from the flow and typescript folders in parsers-primitives

Summary

This PR aims to reduce code duplication by extracting emitRootTag logic from the flow and typescript folders into a shared parsers-primitives file. It is a task of #34872:

Extract the content of the case 'RootTag' (Flow TypeScript) into a single emitRootTag function in the parsers-primitives.js file. Use the new function in the parsers.

Note that #34898 should be merged first. I rebased on it's branch.
Changelog

[Internal] [Changed] - Extract RootTag case of translateTypeAnnotation from the flow and typescript folders in parsers-primitives
Test Plan

All tests are passing, with yarn jest react-native-codegen:

Thanks for taking this. 👏 That's amazing!
We can improve the quality even further byadding a unit test. The steps should be:

add the __tests__ subfolder in the packages/react-native-codegen/src/parsers.

add a parser-primitives-test.js where we unit test this fuction:

we test the output when the nullable parameter is false
we test the output when the nullable parameter is true

An example of unit test can be found here.
Thank you so much! 🙏

Thanks for the review!
I thought this case was already covered in these files (flow and typescript). But you're right, it's clearer and safer to test emitRootTag function 😁
It's done if you want to re-review it!

@MaeIg Just a heads up - looks like there are some conflicting changes  in the new main, have to be resolved before merging.

@MaeIg Just a heads up - looks like there are some conflicting changes in the new main, have to be resolved before merging.

@rshest  I rebased. Thanks 🙂


Work around for 0 valid bits

Here's an example file and what I think the fix is for #50. I zeroed the valid bits in a hexeditor on waveformatextensible-32bit-48kHz-stereo.wav to create the example. It fails the same way in my code as the example I found in the wild. It may need to get added to more tests.

Thanks for opening a pull request!

It may need to get added to more tests.

Yeah, could you also add a test to specifically test that the reported bits per sample is as expected? It could be similar to this one:
https://github.com/ruuda/hound/blob/fa88d577f08370e915a5bb3af6b772de68c53e20/src/read.rs#L966-L983

I've added the requested test.


Str stri val implementation

This contains a fix to better represent floats in a double-only world.  It also implements the global string functions Str(), StrI(), and Val().

Handles most of #50 , but I've still got some work to do to get optional parameters working for functions in the stdlib tree.

Added e2e test, which I didn't realize existed.  Thanks for pointing them out.


[doc] build_docs.py failure on Mac

If the filesystem on Mac is not Apple Filesystem (case sensitive), the build_docs.py fails as the repo already has BUILD file.
We may revise build_docs.py to create docs under other directory (maybe scratch?)

@HU90m @hcallahan-lowrisc do you know whether this is still relevant or can we close it?

Yep, this can be closed


Update to CanvasLayer sorting (issue #25384)

This change implements explicit RID comparison for VisualServerViewport::Viewport::CanvasKey to ensure proper and consistent sorting when one CanvasLayer is nested. Changes were tested on Windows 10 and appear to exhibit identical behaviour to 4.0.alpha build.
This change is not simple to cherrypick from master since broad changes to folder structure have since been made.
Bugsquad edit:

Fixes #25384

I think the fix looks good.
For context, in the master branch RID comparisons have been changed to use the id directly in core:
https://github.com/godotengine/godot/blob/9d732aa0cfcfd638c28ac8eb8149d912a5a3a642/core/templates/rid.h#L46-L48
While in 3.x it compares RID_Data* pointers:
https://github.com/godotengine/godot/blob/9ddeeb8b1afec963e52a3dee559994fea2c0dcee/core/rid.h#L70-L72
It's arguable that RID comparisons in 3.x are actually bogus, since pointer comparisons would always be non-deterministic. So this bug could also be fixed by changing RID comparison operators to match the master branch - but as documented in our guidelines, we recommend to prefer local solutions so the fix implemented here is the preferred approach.
There could be a case for changing RID in another PR, but the implications are big and this might not be worth doing at this stage of development, given that it's already refactored for future releases in master.

Change does look good regarding repeatability.
I'm not absolutely sure that the RID id has any relation with the position in the scene tree though (other than the chance of one being created before the other, presumably when you load the first objects is created first, but if you move at runtime this may not be the case). So the complete fix for that issue may require a bit more.


Use with Fargate and ALB?

Hi, this is great, but I'm having huge issues keeping targets healthy in a load balanced ECS configuration with an Application Load Balancer.
Does anyone have any config for listeners and healthchecks in this sort of setup?
Many thanks

Hi,
sorry for answering so late ...
Here is on example how to implement something like this (it's my pet project): https://github.com/aws-samples/reactive-refarch-cloudformation/blob/master/services/tracking-service/service.yaml


[main] Upgrade to latest dependencies

Cron. -knative-prow-robot
/cc knative/operations-wg-leads
/assign knative/operations-wg-leads
Produced via:
./hack/update-deps.sh --upgrade --release v0.22
./hack/update-codegen.sh

Details:
bumping knative.dev/caching 3aa75bb...7582ffe:
  > 7582ffe upgrade to latest dependencies (# 441)
bumping knative.dev/hack 58edbdc...b6ab329:
  > b6ab329 Add a comment to explain why we have a long wait_until_pods_running function (# 60)
bumping knative.dev/eventing c039526...3ebce0d:
  > 3ebce0d relax malformed ce response code to 4xx (# 5154)
  > 8ff609f add tests for various missing required fields (# 5144)
  > 979f99e Changing cluster role's name check to check only suffix for source spec conformance testing (# 5105)
  > 724f4ed adding role and role selector test for channels (# 5141)
  > 116a571 Fix comment on podspecable-binding clusterrole (# 5142)
  > c820f42 Channel Conformance Tests, CRD (# 5138)
  > d3e856a upgrade to latest dependencies (# 5140)
  > b5b547f adding todo stubs for channel conformance tests (# 5127)
  > 5947002 add test for rejecting malformed CE (# 5135)
  > f9de675 broker.status.address.url is not required (# 5134)
  > 04ab20f rename downstream-test-go to actions-downstream-test (# 5133)
  > 9e8f2df add conformance test for checking GET method gets rejected (# 5132)
  > 88b3c90 More unit tests for pingsource (# 5106)
bumping knative.dev/pkg dfeeb18...78ca070:
  > 78ca070 upgrade to latest dependencies (# 2070)
  > b558677 Use TB interface, rather than T pointer when doing test stuff (# 2064)
  > 9bf9eeb rename downstream-test-go to actions-downstream-test (# 2065)
bumping knative.dev/serving 3e2b15e...5dbe774:
  > 5dbe774 upgrade to latest dependencies (# 11036)
  > 2081525 Fix HTTP2 feature-gate threading in queue-proxy (# 11022)
  > 2490284 use-new-helpts (# 11018)
  > d55482e Bypass tracing handler if tracing is disabled (# 11016)
  > 37dedfa Skip flaky HTTP2 probe test (# 11027)
  > d5d489c Reduce allocations to store config on context in Activator (# 11013)
  > 1a0d326 Prevent MaxScaleLimit being exceeded on update (# 11010)
  > 4682e3e Use service-host-name helper in Activator creation (# 11021)
  > fb5af24 upgrade to latest dependencies (# 11020)
  > c9296e1 Add HA test for DomainMapping controller (# 10816)
  > b5d1348 Add a benchmark for the entire handler chain of the activator (# 11014)
  > ccf35f1 Update net-istio nightly (# 11002)
  > 069c635 add to mon reviewers (# 10972)
  > 0e8410a Format markdown (# 11001)

/lgtm

/unhold


DRAFT: introduce HCL configuration

This PR aims to add on @apparentlymart s work https://github.com/perkeep/perkeep/issues/1134#issuecomment-406877102 by compilling the high level HCL configuration to a low-level configuration that can be consumed in the same way as the low-level json configuration.
We start by adding a shim to decouple the configuration consumers from the json configuration. Next we add implementations for the old jsonconfig and the new hclconfig. Next, once the hclconfig is stable and feature complete we should add a small tool to translate existing configurations ( if possible ) to the HCL schema and deprecate the json configuration. Finally we should be able to remove the json configuration alltogether hopefully.
For some notes please see this file where i'll try to log discoveries and decisions regarding the generation of the low-level config from the HCL schema.

Why not just convert the HCL to JSON?
HCL has a spec for it, and Caddy has good examples of usage of adapting several config formats to JSON.
That way almost everything could left as-is (using JSON), only the input side (convert from HCL to JSON) would change.

Why not just convert the HCL to JSON? HCL has a spec for it, and Caddy has good examples of usage of adapting several config formats to JSON.
That way almost everything could left as-is (using JSON), only the input side (convert from HCL to JSON) would change.

Hey @tgulacsi I agree! Eventualy that shim will go away, i just added it to get a feeling on whats expected from the compiled configuration and to have some compiler help when removing the current low-level json configuration eventually.

I think it's worth noting that the HCL JSON syntax is a mapping from JSON's infoset to HCL's infoset, rather than the other way around. I expect it would be hard (possibly impossible) to implement Perkeep's existing JSON-based config language in terms of HCL JSON, because HCL JSON is opinionated about exactly how each HCL infoset feature may be written in JSON.
That doesn't mean that you can't in principle translate from an HCL-based language to an arbitrary JSON-based language as long as they represent compatible ideas; just that there isn't really anything in the HCL codebase that will directly help with that, and so it would need to be some custom logic that understands Perkeep's languages in particular.

Hey @apparentlymart

I think it's worth noting that the HCL JSON syntax is a mapping from JSON's infoset to HCL's infoset, rather than the other way around. I expect it would be hard (possibly impossible) to implement Perkeep's existing JSON-based config language in terms of HCL JSON, because HCL JSON is opinionated about exactly how each HCL infoset feature may be written in JSON.
That doesn't mean that you can't in principle translate from an HCL-based language to an arbitrary JSON-based language as long as they represent compatible ideas; just that there isn't really anything in the HCL codebase that will directly help with that, and so it would need to be some custom logic that understands Perkeep's languages in particular.

Basically, to start with an incremental change i wanted to take your schema and write a small compiler that builds the lowlevel JSON as a start to use HCL. Do you see problems with that approach? I assume that eventually we could start refactoring the handler factories to receive hcl.Blocks and reading their config from there, but thats a much more invasive change, that i would prefer further down the road.

I think that broadly makes sense, though there is one rough edge with the code I prototyped before:
When building the hcl.EvalContext for expression evaluation I made it read all of the environment variables from the environment just before evaluating the configuration, which therefore assumes that the process evaluating the configuration is the same process that will use the configuration. If you generate the configuration on one system then that system's env.HOME would presumably get baked in to the generated JSON, rather than it being evaluated dynamically on the final system.
IIRC Perkeep JSON has its own convention for encoding dynamic references to environment variables, but HCL doesn't have any built-in support for translating an expression to some other expression syntax without evaluating it first. It would in principle be possible to do it by interrogating the expression AST directly, but that would be a kinda-unusual custom thing, not something intentionally supported by HCL's API.
I attempted something along those lines in an abandoned side-project, so I do have an example of directly analyzing the HCL AST to produce a higher-level intermediate representation and then ultimately CloudFormation JSON. Doing that only for direct references like env.HOME and template interpolation sequences like "${env.HOME}/example" would be simpler than that, but would presumably still follow a similar principle. (hclsyntax.TemplateExpr and hclsyntax.ScopeTraversalExpr would be the only two cases to treat as special in that case, I think.)

I think that broadly makes sense, though there is one rough edge with the code I prototyped before:
When building the hcl.EvalContext for expression evaluation I made it read all of the environment variables from the environment just before evaluating the configuration, which therefore assumes that the process evaluating the configuration is the same process that will use the configuration. If you generate the configuration on one system then that system's env.HOME would presumably get baked in to the generated JSON, rather than it being evaluated dynamically on the final system.
IIRC Perkeep JSON has its own convention for encoding dynamic references to environment variables, but HCL doesn't have any built-in support for translating an expression to some other expression syntax without evaluating it first. It would in principle be possible to do it by interrogating the expression AST directly, but that would be a kinda-unusual custom thing, not something intentionally supported by HCL's API.
I attempted something along those lines in an abandoned side-project, so I do have an example of directly analyzing the HCL AST to produce a higher-level intermediate representation and then ultimately CloudFormation JSON. Doing that only for direct references like env.HOME and template interpolation sequences like "${env.HOME}/example" would be simpler than that, but would presumably still follow a similar principle. (hclsyntax.TemplateExpr and hclsyntax.ScopeTraversalExpr would be the only two cases to treat as special in that case, I think.)

Ah i think there was a misunderstanding, the plan was to parse the HCL into in-process representation of the low-level json (basically a fancy map[string]interface{}) so that everything is already expanded usable within the process. So no two step HCL -> JSON -> point server at JSON procedure. That should at least solve the environment issue.

Oh yes, that does sound better indeed. Sorry... the mention of JSON made me assume we were talking about a serialized artifact on disk, but a "JSON-like" in-memory data structure would avoid that problem indeed.


Set the working directory that tasks are run from?

At the moment, we do slurm submissions from the folder that fractal server runs in. That folder is often not accessible to users on the cluster (or at least, they don't have write access to it).
On the FMI side, that then leads to jobs being run from the /tmp folder.
Our jobs may write something into their working directory. I don't think ours currently do (if they did, we appear not to rely on it). But some user tasks in development rely on libraries that do write some logs to the working directory. In general, I'd focus on not having tasks write there. But they are using an old library that nobody really knows how to debug, but it appears to be running. Thus, would be good to allow that.
In the specific case of the RDCNet task, it wants to write a luigi.log file (hard-coded name) and if multiple users end up running this job, we start getting permission errors.
A solution is that users specify the working directory during job submission.
They can set the working directory for a given workflow submission by adding the following to the worker initialization:
chdir /path/where/you/want/the/working/dir/to/be
This solves the problem, but requires users to always set this. Would it make sense to change the working directory by default to a user-specific folder? I was thinking about using the cache_dir (if set) as the working directory. Would there be any issues with that?

We already log the following in our main workflow log:
2023-10-31 16:41:33,792 - WF106_job424 - DEBUG - job.working_dir: /path/to/fractal_logs/proj_0000054_wf_0000106_job_0000424_20231031_154133
2023-10-31 16:41:33,792 - WF106_job424 - DEBUG - job.working_dir_user: /path/to/different/user/isde/.fractal-cache/proj_0000054_wf_0000106_job_0000424_20231031_154133

But the job.working_dir_user is not used as the actual working directory for the job after all? I think the job is still submitted from job.working_dir (which is not accessible to users at the FMI). Given we already seem to define a job.working_dir_user, can we just pass this with the relevant sbatch option to submit from there?


Add config to mount a scratch directory

This adds an optional config to mount a scratch directory into your ocm container.

/lgtm
/approve


Switch base image from base to debian8

The newest debian images for jessie (gcr.io/google-appengine/debian8:latest) now has all the extra env variables and ca-certs that were in base.
cc @dlorenc

cc @broady

Hey, friendly ping.

@broady


Update deploy-intro.html

Grammar fix in a tutorial.

@anvera Please try follow instructions here to debug CLA status if you have signed it.

Sorry for the noise, but I'll have to close this for now until I figure out if I can sign the CLA.


clickhouse-backup: 1.5.2 -> 1.6.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/AlexAkulov/clickhouse-backup/releases.
meta.description for clickhouse-backup is: "Tool for easy ClickHouse backup and restore with cloud storages support"
meta.homepage for clickhouse-backup is: "https://github.com/AlexAkulov/clickhouse-backup"
meta.changelog for clickhouse-backup is: ""
Updates performed

Golang update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.6.0 with grep in /nix/store/3ckb88393akxdiy19baxw5rkl7kw72rh-clickhouse-backup-1.6.0
found 1.6.0 in filename of file in /nix/store/3ckb88393akxdiy19baxw5rkl7kw72rh-clickhouse-backup-1.6.0

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
clickhouse-backup

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/3ckb88393akxdiy19baxw5rkl7kw72rh-clickhouse-backup-1.6.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A clickhouse-backup https://github.com/r-ryantm/nixpkgs/archive/f71f2cb03d2149929bedc9cdf722674612ce16ab.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/3ckb88393akxdiy19baxw5rkl7kw72rh-clickhouse-backup-1.6.0
ls -la /nix/store/3ckb88393akxdiy19baxw5rkl7kw72rh-clickhouse-backup-1.6.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

clickhouse-backup

Maintainer pings
cc @Ma27 for testing.

Result of nixpkgs-review pr 188642 run on x86_64-linux 1

1 package built:

clickhouse-backup


jamfcpr 4.0.2 is showing us less than half of the packages in the GUI

Hi, jamfcpr 4.0.2 is showing us less than half of the packages in the GUI.
In the log, the JSON looks complete and it starts loading the metadata until it can't talk to the server anymore.
First try with User+PW :

20240206_165708 [JamfPro.getToken] Generate token for source server: https://XYZ.jamfcloud.com
20240206_165709 [JamfPro.getToken] Use existing token from Optional(https://XYZ.jamfcloud.com/api/v1/auth/token)
20240206_165709 [endPointByID] Response: nil.
ken for source server: https://XYZ.jamfcloud.com
20240206_165709 [JamfPro.getToken] Use existing token from Optional(https://XYZ.jamfcloud.com/api/v1/auth/token)
20240206_165709 [endPointByID] Issue communicating with the https://XYZ.jamfcloud.com.
20240206_165709 [endPointByID] Response: nil.
20240206_165709 [ApiAction.getPackages] Found package record missing the filename in xml.  Package id: 2651
20240206_165709 [endPointByID] Issue communicating with the https://XYZ.jamfcloud.com.
20240206_165709 [endPointByID] Response: nil.
20240206_165709 [ApiAction.getPackages] Found package record missing the filename in xml.  Package id: 41
20240206_165709 [endPointByID] Issue communicating with the https://XYZ.jamfcloud.com.
20240206_165709 [endPointByID] Response: nil.

and so on, this is written for all packages, which are of course not displayed in the GUI afterwards.
Second try with API Client:

20240206_175218 [JamfPro.getToken] Generate token for source server: https://XYZ.jamfcloud.com
20240206_175218 [ApiAction.getPackages] looking up: JustAnotherPackage.pkg
d.com/JSSResource/packages/id/2067
pi/oauth/token)
20240206_175218 [JamfPro.getToken] Generate token for source server: https://XYZ.jamfcloud.com
20240206_175218 [endPointByID] fetching XML from: https://XYZ.jamfcloud.com/JSSResource/packages/id/2518
pi/oauth/token)
20240206_175218 [JamfPro.getToken] Generate token for source server: https://XYZ.jamfcloud.com
20240206_175218 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
://XYZ.jamfcloud.com/api/oauth/token)
20240206_175218 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
20240206_175219 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
20240206_175219 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
20240206_175219 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
20240206_175219 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
... literally over 500 times ...
20240206_175301 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
20240206_175301 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
20240206_175302 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
20240206_175302 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
20240206_175302 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
20240206_175302 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
20240206_175350 [JamfPro.getToken] token response error from https://XYZ.jamfcloud.com.  Verify url and port
20240206_175350 [endPointByID] Unable to get valid token.
20240206_175350 [ApiAction.getPackages] Found package record missing the filename in xml.  Package id: 1963
20240206_175351 [JamfPro.getToken] token response error from https://XYZ.jamfcloud.com.  Verify url and port
20240206_175351 [endPointByID] Unable to get valid token.
20240206_175351 [ApiAction.getPackages] Found package record missing the filename in xml.  Package id: 2510
20240206_175400 [JamfPro.getToken] token response error from https://XYZ.jamfcloud.com.  Verify url and port
20240206_175400 [endPointByID] Unable to get valid token.
20240206_175400 [ApiAction.getPackages] Found package record missing the filename in xml.  Package id: 2639

During the API Client try I also got a "...verify url and port" dialog in the GUI in a continuous loop and had to close it forced.
The packages mentioned at the end of the log look completely normal and are in use, assume it is output incorrectly due to the previous issues.
btw. there is another report that sounds similar, but I could not identify from the information whether it is the same problem
https://github.com/BIG-RAT/jamfcpr/issues/18
Tested with:
JamfCPR 4.0.2
Jamf Pro Cloud Prod Instance with 11.1.1 (it´s not one of the sandbox Instances from my other open Issue)

v4.2.1 addresses some API Client issues.  See if it helps with yours.

probably meant v4.1.0 , unfortunately unchanged with this behavior.
but i had a new issue and had to use 4.0.2 to add the server again because i couldn't do it in 4.1.0.
when i enter everything in the "New Server:" dialog in v4.1.0 and click connect a token is created in the log, but the dialog does not close and the server is not saved in the list.
I also noticed that "API Client" is checked when opening the dialog, but it shows "Username" and "Password". After unchecking and checking "API Client" it is displayed correctly.

Nice catch on the use API client button - thanks.
Do you have your JCDS set as principal (Settings - Server - Cloud distribution point)?
Still seeing the token response errors?

yes it is set as principal.
i see no response error in the v4.1.0 log, but also no other errors, just again this after some hundred packages:

...
20240214_163846 [ApiAction.getPackages] looking up: ASUPERNICEPACKAGE.pkg
d.com/JSSResource/packages/id/2067
20240214_163846 [JamfPro.getToken] Attempting to retrieve token from Optional(https://XYZ.jamfcloud.com/api/oauth/token)
20240214_163846 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
urce/packages/id/2518
20240214_163846 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
20240214_163846 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
20240214_163846 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
    ... several hundred times later ...

20240214_163913 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
20240214_163913 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
20240214_163913 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
20240214_163913 [ApiAction.getPackages] Found package record missing the filename in xml.  Package id: 2571
20240214_163913 [ApiAction.getPackages] Found package record missing the filename in xml.  Package id: 2715
20240214_163913 [ApiAction.getPackages] Found package record missing the filename in xml.  Package id: 2649
    ... several hundred times later ...

20240214_163926 [ApiAction.getPackages] Found package record missing the filename in xml.  Package id: 1701
20240214_163926 [ApiAction.getPackages] Found package record missing the filename in xml.  Package id: 2523
20240214_163926 [ApiAction.getPackages] Found package record missing the filename in xml.  Package id: 1324
20240214_163926 [JamfPro.getToken] new token created for https://XYZ.jamfcloud.com
20240214_163929 [ApiAction.getPackages] Found package record missing the filename in xml.  Package id: 1689
20240214_163929 [ApiAction.getPackages] Found package record missing the filename in xml.  Package id: 2084
20240214_163929 [ApiAction.getPackages] Found package record missing the filename in xml.  Package id: 189
    ... several hundred times later ...

20240214_163944 [ApiAction.getPackages] Found package record missing the filename in xml.  Package id: 2507
20240214_163944 [ApiAction.getPackages] Found package record missing the filename in xml.  Package id: 2656
20240214_163944 [ApiAction.getPackages] Found package record missing the filename in xml.  Package id: 2116
20240214_163945 [ViewController.reloadFileList] source: https://XYZ.jamfcloud.com
20240214_163945 [ViewController.reloadFileList] display packages from source JCDS
...

Thanks for the details. I need to figure out why there is one attempt to get a token:
20240214_163846 [JamfPro.getToken] Attempting to retrieve token from Optional(https://xyz.jamfcloud.com/api/oauth/token)
but hundreds of new tokens created:
20240214_163846 [JamfPro.getToken] new token created for https://xyz.jamfcloud.com/
20240214_163846 [JamfPro.getToken] new token created for https://xyz.jamfcloud.com/

Or are there many more attempts to retrieve a token, just removed from your post.

no, i have only shortened where there was no change in the output (except date and package id).
I just see more attempts to get a token, earlier in the log, where it looks pretty normal, I think...
...now that i think about it, it doesn't look so normal... if i see correctly, it requests a new token for each access, while it should still have a valid one, if i understand the log correctly.
token lifetime is on default

20240214_165723 [endPointByID] fetching XML from: https://XYZ.jamfcloud.com/JSSResource/packages/id/2487
20240214_165723 [JamfPro.getToken] Attempting to retrieve token from Optional(https://XYZ.jamfcloud.com/api/oauth/token)
20240214_165723 [ApiAction.getPackages] looking up: ANOTHERNICEPACKAGE.pkg
d/2567
20240214_165723 [JamfPro.getToken] Attempting to retrieve token from Optional(https://XYZ.jamfcloud.com/api/oauth/token)
20240214_165723 [endPointByID] fetching XML from: https://XYZ.jamfcloud.com/JSSResource/packages/id/2593
20240214_165723 [endPointByID] endpoint passed to endPointByID: packages/id/2667
ps://XYZ.jamfcloud.com/api/oauth/token)
20240214_165723 [endPointByID] fetching XML from: https://XYZ.jamfcloud.com/JSSResource/packages/id/2667
20240214_165723 [JamfPro.getToken] Attempting to retrieve token from Optional(https://XYZ.jamfcloud.com/api/oauth/token)
20240214_165724 [ApiAction.getPackages] looking up: ANDANOTHERNICEPACKAGE.pkg

What has changed in the log is that a token is only requested at the beginning and no more attempts.
But now no packages are listed in the gui anymore and again hundreds of "Found package record missing the filename in xml. Package id:" in the log - apparently now for every package.
Maybe you know that already, but I wanted to mention it, it starts these messages when it has already gone through all packages in alphabetical order (as listed in json format in the log).
I had just realized that all these "record missing" ID's are from packages that he had already processed.
I have run a sync with jamf admin into a local folder to make sure that we don't have a problem in general and have looked at some xml's via api for anomalies... but nothing
if it's ok for you, i would send you a complete log in macadmins slack next week, maybe it will help to narrow down

Reading my mind - you can find me as lnh_slack.

v4.1.1 has been posted.  Please re-open the issue if the behavior persists.


Eeception "Disabling core dumps" was thrown while running "./CarlaUE4.sh"

Configures:
Ubuntu 18.04 LTS desktop
GTX 1660Ti 6G
Nvidia-driver-440
opengl4.6
vulkan1.1
Carla  release <IP_ADDRESS>, 0.9.8, <IP_ADDRESS> and 0.9.6
Exception as follow :
4.24.3-0+++UE4+Release-4.24 518 0
Disabling core dumps.

Problem: I couldn't key any command about Carla in either the terminal window or other one after above exception appeared
Then enter "CTRL+C" to end up, only the following usually appears
FUnixPlatformMisc::RequestExitWithStatus
FUnixPlatformMisc::RequestExit

Key "nvidia-smi" into the other terminial while running ./CarlaUE4.sh call

The ways tried & Corresponding results(ranked):

To run commands "./CarlaUE4.sh", "./CarlaUE4.sh -vulkan", "./CarlaUE4.sh -opengl" and "DISPLAY=" followed by one of
those respectively.

that can't help improve the matter anymore.

To switch among all previous releases.

the same error still appears.

To switch among various graphics driver versions including 440, 435 and 418 that make sure the successful operation to CUDA 10.1  .

the same error still appears.

To install Vulkan and its dependencies and run "vulkaninfo" call successfully.

be alike.
Some ways that haven't been tried:

some people think that the issue is caused by the CarlaUE4.sh script does not pass in the arguments to the underlying binary ,refering to nyukhalov's comment, which dosen't match my situation from my view.

the other people tried to delete some material in maps and then re-compile it, please refer to #2917, in my opinion, which have impact on completeness of files and further cause some unexpected issue, in addition, that solution isn't suitable for the new like me.

the way to edit allocation of memory size for running ./CarlaUE4.sh call is proposed from my understand towards error reports and the first way above.

I spent clear 3 days doing all above and reading all about the error under issues tag, I'd appreciate it if anyone could help me, I'll clear my head during the next few days.
hey  @bernatx @germanros1987
I'm still new to the OS & Carla, could you help me with the problem or assign anyone to do?

I have the same issue. Also the window with a car doesn't show up. only one window without car (looks like an FPP of a drone) opens up.

I built the new master branch on Ubuntu 18.04 and packaged it. It also throws the same error.
4.24.3-0+++UE4+Release-4.24 518 0 Disabling core dumps. X Error of failed request:  BadDrawable (invalid Pixmap or Window parameter)   Major opcode of failed request:  149 ()   Minor opcode of failed request:  4   Resource id in failed request:  0x7c0003e   Serial number of failed request:  365   Current serial number in output stream:  376 terminating with uncaught exception of type std::__1::system_error: mutex lock failed: Invalid argument Signal 6 caught. Segmentation fault (core dumped)
GPU: 1060 6GB
I tried running /PythonAPI/utils/config.py
 File "config.py", line 307, in <module>     main()   File "config.py", line 192, in main     client = carla.Client(args.host, args.port, worker_threads=1) AttributeError: module 'carla' has no attribute 'Client'

@tinmodeHuang sorry for the delay.
As it has been mentioned already, the "disabling core dumps" message is just letting you know that CARLA won't generate a core dump. However, I understand from your message that you are having problems running the server or client (this part wasn't clear to me).
In order to help you, could you please provide us with more information? First, for simplicity, we are going to consider CARLA <IP_ADDRESS> and vulkan (-vulkan).

Are you running the server in a local machine connected to a physical screen?
Does the server window show up? Can you interact with it?
If you run the manual_control.py script, does it work?

@tinmodeHuang sorry for the delay.
As it has been mentioned already, the "disabling core dumps" message is just letting you know that CARLA won't generate a core dump. However, I understand from your message that you are having problems running the server or client (this part wasn't clear to me).
In order to help you, could you please provide us with more information? First, for simplicity, we are going to consider CARLA <IP_ADDRESS> and vulkan (-vulkan).

Are you running the server in a local machine connected to a physical screen?
Does the server window show up? Can you interact with it?
If you run the manual_control.py script, does it work?

I won't modify posted description of the issue anymore in order to help those being in the same situation realise that message isn't a matter.
my answer to the first two questions is "yes",

I have met the same problem with released pack carla0.9.6, which prints
4.22.1-0+++UE4+Release-4.22 517 0 Disabling core dumps. sh: xdg-user-dir: command not found Signal 11 caught. Malloc Size=65538 LargeMemoryPoolOffset=65554  CommonUnixCrashHandler: Signal=11 Malloc Size=65535 LargeMemoryPoolOffset=131119  Malloc Size=44304 LargeMemoryPoolOffset=175440  Engine crash handling finished; re-raising signal 11 for the default handler. Good bye.
For some reason, we are not able to run docker on server, so we have tried several machines we can borrow. We can only success with a Ubuntu16 server with 4 V100 GPUs. On the server we are using which is a Centos 7 server, it always return the above message. We have tried -opengl, -opengl3 and even -no-rendering and nothing changes.
All these server has a GPU driver version 418.67 and cuda10.1
Does anyone knows what we can do next to make this issue fixed? We can do nothing because there are no coredump file to debug.
Many thanks!

I have met the same problem with released pack carla0.9.6, which prints
4.22.1-0+++UE4+Release-4.22 517 0 Disabling core dumps. sh: xdg-user-dir: command not found Signal 11 caught. Malloc Size=65538 LargeMemoryPoolOffset=65554 CommonUnixCrashHandler: Signal=11 Malloc Size=65535 LargeMemoryPoolOffset=131119 Malloc Size=44304 LargeMemoryPoolOffset=175440 Engine crash handling finished; re-raising signal 11 for the default handler. Good bye.
For some reason, we are not able to run docker on server, so we have tried several machines we can borrow. We can only success with a Ubuntu16 server with 4 V100 GPUs. On the server we are using which is a Centos 7 server, it always return the above message. We have tried -opengl, -opengl3 and even -no-rendering and nothing changes.
All these server has a GPU driver version 418.67 and cuda10.1
Does anyone knows what we can do next to make this issue fixed? We can do nothing because there are no coredump file to debug.
Many thanks!

perhaps I couldn't have make it clear in the last comment, which the exception doesn't matter as germanros1987 said, thus you can open another terminal to run something relevant.

use that
./CarlaUE4.sh -opengl

use that
./CarlaUE4.sh -opengl

yes ! it works.but why does './CarlaUE4.sh'  fail to present the CarlaUE4 window continuesly?
thank you for your time

use that
./CarlaUE4.sh -opengl

worked for me

I am still facing this problem and cannot figure out the reason!

@Derekabc Have you solved the problem? I have met the same problem and I find it has no error information. So I don't know where to fix

@tinmodeHuang  Have you found any solution because I am also facing the same issue.

@tinmodeHuang Have you found any solution because I am also facing the same issue. Thanks!

I have tried all the above but still cant find the solution. I am trying to run the Docker container for carla 9.13 on a windows 10 host. Maybe the issue is that I am using windows?

./CarlaUE4.sh -opengl -carla-server -benchmark -fps=10

Have you solved this problem? I'm having the same problem.

使用 ./CarlaUE4.sh -opengl

works to me, thx


npm release

Any chance on a new npm release?

@CrackerakiUA can we make a new version? It seems that a minor version is needed due to the Webpack fix.

ok, sure.

@AdriVanHoudt i have made publish, @1.0.5
please verify that everything is ok, as i didn't make compile bcs i don't have ruby installed on this environment.

@CrackerakiUA nice 🎉
I opened a PR where I ran gulp and gulp build for you https://github.com/CrackerakiUA/ui-cropper/pull/52
If you want I can do some more work like updating deps and removing the ruby dependency if you'd like

Any positive work is welcome, i have Ruby on my personal environment, when i am working on plugin. Any way i never understood why Ruby is needed, sass can be used with sass-middleware, this was made by Author of the cropper.

<EMAIL_ADDRESS>published

@CrackerakiUA awesome thanks, it seems like you didn't push the version changes to github yet?

Fixed.

Could you also update Bower please? It's currently still on 1.0.4. Thanks.

Isn’t it that bower searches for the latest version?
If you look at the bower.json there’s no version present.

I guess it's checking releases, i will make new one.

1.0.6 release is out.

afaik this issue now definitely closed ;)


Initial jwt implementation

Description
Add description...
Checklist

[ ] Description added to PR
[ ] PR is linked to issue
[ ] Solution can start locally

Codecov Report

Merging #35 (e213b52) into main (5713902) will decrease coverage by 5.14%.
The diff coverage is 83.15%.

@@            Coverage Diff             @@
##             main      #35      +/-   ##
==========================================
- Coverage   98.98%   93.83%   -5.15%
==========================================
  Files          21       26       +5
  Lines         197      292      +95
==========================================
+ Hits          195      274      +79
- Misses          2       18      +16

Impacted Files
Coverage Δ

MinimalApi/OptionsSetup/JwtBearerOptionsSetup.cs
23.52% <23.52%> (ø)

Domain/Auth/RefreshToken.cs
57.14% <57.14%> (ø)

Application/Common/Options/Auth/JwtOptions.cs
100.00% <100.00%> (ø)

Infrastructure/Auth/JwtProvider.cs
100.00% <100.00%> (ø)

MinimalApi/OptionsSetup/JwtOptionSetup.cs
100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Fix no lifecycle

what

No lifecycle

why

If no transition* var is set, it will throw an error

references

Previous PR https://github.com/cloudposse/terraform-aws-efs/pull/94

/test test/terratest

/test all


How many tracker can be used?

Hi,
I want to ask How many tracker can be handled by traccar service ? suppose that each tracker sends position every one minute.
IF there is a limitation, please suggest a solution to solve this issue.
Thanks

Traccar has been tested with 30k devices. It can handle 8k messages per second or more. It all depends on your hardware.
How many device do you need to handle?

Hi,
I'm going to handle 20k devices, what is HW specifications you suggest ?

I'd say at least 8GB RAM and 4 cores CPU. I would recommend to test different options. If you don't have devices yet, you can do a load testing with this script:
https://github.com/tananaev/traccar/blob/master/tools/test-performance.py

Hi,
I have added 30 TopflyTech model  T8803 Tracker devices and now i have problem location  of devices changing dynamically and are not stable at one location. Can you please explain how to solve this issue. MY VPS configuration  is Windows 8 64Bit Os, 4GBRam,  XEON 2.5 Ghz Processor  with 60 GB HDD.
Thanks

@digitaltrackit, please provide more details about the problem. Are you saying that locations are incorrect?

can i provide you  vps ip so u can check ? how is it

I want to know what the problem is first.

The location of the trackers are moving rapidly on google map. I observe that it is the trackers' previous/history location.

Tracker live location is incorrect its seems it's playing trackers' history where where tracker travel .

Maybe it's uploading history?

its uploading 24 hr. history and it taking long time to set device current location

I don't think anything can be done about it. Do you think there is some server-side problem?

Is it possible for you to see portal  personally so u will more understand problem i will provide ip and password  if its ok .

Yes, you can send me details by email (address is in profile).

ok thank you very much
going to send you details

plz. check your email i sent all loging details
Thanks  :)


Strong Password Generator Function

Hi @suryapratapsinghsuryavanshi , I guess my logic is different from the existing one as it will create the password by taking length as a parameter. Please have a look and let me know if any changes are required.
issue - #32

@suryapratapsinghsuryavanshi , I have updated the README file for emailValidator and strongPasswordGenerator functions.


rtz: 0.5.3 -> 0.7.0

Description of changes

Changes
Breaking Changes
New data has been ingested (see README for sources and dates). This does not break functionality, but may break any tests that rely on the results of lookups via the data in this package.
Major Improvements

Upgraded all dependencies.
Added default open CORS policy to the web server.
Added ability to automatically push web analytics to Application Insights.
Reworked alignment for bincode assets.
Fixed issues with using double-precision feature.
Stabilized to stable Rust 1.80.

Full Changelog: https://github.com/twitchax/rtz/compare/v0.5.3...v0.7.0.

Things done

Built on platform(s)

[X] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[X] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[X] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[X] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[X] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Ping @figsoda

@samueltardieu, since no more nightly features are used, env.RUSTC_BOOTSTRAP can be removed (tested it builds without it)

since no more nightly features are used, env.RUSTC_BOOTSTRAP can be removed (tested it builds without it)

Good catch, updated.

Note: additionally, I've proposed that rtz use the  vendored feature flag for utoipa-swagger-ui which would let us remove the hardcoded swagger-ui version (https://github.com/twitchax/rtz/pull/9).

Anything else to do? (ping @alyssais)

Seems like it should be marked broken on x86_64-darwin — would you mind opening a followup PR for that? I'm not going to hold this one back for that, so that we can unbreak it for Linux users.

Sure, I've opened #338589 to try and fix it.


Setting the icons

Fixes #10 .
Changes proposed in this pull request:

create a set_icon method in deer_gui
load all the icons' size available
do not crash when an icon can't be loaded, this is not a critical functionnality of the application

Hello Maxime,
Thanks for your contribution to Deer and we wish to continue contributing to it.
Currently, Deer is not compiling. BTW, I think we can use a different approach instead of storing each icon two times and reduce lines of code as this way is not efficient.
I suggest:

keeping only icons list as a data member of GUI class
In set_icons method use an array of icon names and iterate over it to load icons
Since you do not handle error in gdk_pixbuf_new_from_file, just remove error pointer and pass NULL
According to GTK reference [1], you need to use the return value of g_list_append, instead of ignoring it
gtk_window_set_icon_list method takes GtkWindow pointer as a first parameter, but window pointer is a GObject. Use GTK_WINDOW(window) to cast it
set_icons method has no type, make it void
In g_list_free_full [2], you have to write a function that will be called on each element to deallocate it
You need to copy resources directory to build/bin directory to be able to load icons from resources/Deer-...png. Here [3] is an example for how to accomplish that

Again, We are happy with your contributions to Deer and feel free to ping us if there is something not clear in the suggestions.
[1] https://developer.gnome.org/glib/stable/glib-Doubly-Linked-Lists.html#g-list-append
[2] https://developer.gnome.org/glib/stable/glib-Doubly-Linked-Lists.html#g-list-free-full
[3] https://stackoverflow.com/questions/13429656/cmake-how-to-copy-contents-of-a-directory-into-build-directory-after-make

Last point (copying resource directory) is done now, you can escape it.


The URL of the public signing key has changed

As per https://github.com/sgerrand/alpine-pkg-glibc

The URL of the public signing key has changed! warning
Any previous reference to https://raw.githubusercontent.com/sgerrand/alpine-pkg-glibc/master/sgerrand.rsa.pub should be updated with immediate effect to https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub.

But in the Dockerfile we are still using old one

wget -O "/etc/apk/keys/sgerrand.rsa.pub"
"https://raw.githubusercontent.com/andyshinn/alpine-pkg-glibc/master/sgerrand.rsa.pub" && \

Updated in the latest commit


Formatted with: gofmt -w *

CC @cmonr

@deepikabhavnani Will bring in #29 in first since this PR is simple to regenerate.

@cmonr - Re did it


feat/ add FTX Perpetual connector

Before submitting this PR, please make sure:

[x] Your code builds clean without any errors or warnings
[x] You are using approved title ("feat/", "fix/", "docs/", "refactor/")

A description of the changes proposed in the pull request:
Move FTX Perpetual connector into Hummingbot Foundation repository.
Refer to the details of this proposal:
HGP-18 [DG] Add FTX perpetual connector to boost FTX fee rebates
Tests performed by the developer:
Currently there's not unit tests available for FTX Perpetual.
Tips for QA testing:
Ensure that FTX Perpetual connector is working as intended.

Hi @CrimsonJacket, can you add to the body of the place order request, the key externalReferralProgram with the value "hummingbot1"?
Is already implemented in the spot connector. Line 84 define the class variable | Line 583 add the param to the request.

@rapcmia @nikspz @arnelhbot This PR should be top priority to get into the 1.7 release

the issues showing for PR:

error traiding pairs autometion wont work on the initial start
does not list available ftx perpetual markets when "tab" on markets parameter
Failed to start a perpetualMM strategy, stuck on unexpected error occurred when listening to orderbook streams. (don't place orders to OPEN position)

Steps:
Clone and install feature branch
Start the bot
Actual:
Bot showing ERROR - Connector ftx_perpetual failed to retrieve its trading pairs. Trading pairs autocompletion won't work.
logs_hummingbot.log
2022-07-21 06:49:52,871 - 137420 - hummingbot.core.utils.trading_pair_fetcher - ERROR - Connector ftx_perpetual failed to retrieve its trading pairs. Trading pairs autocompletion won't work.
Traceback (most recent call last):
  File "/home/ubuntu/ftxp5438/hummingbot/core/utils/trading_pair_fetcher.py", line 69, in call_fetch_pairs
    pairs = await fetch_fn
  File "hummingbot/connector/exchange_base.pyx", line 85, in all_trading_pairs
    mapping = await self.trading_pair_symbol_map()
  File "hummingbot/connector/exchange_base.pyx", line 65, in trading_pair_symbol_map
    async with self._mapping_initialization_lock:
  File "hummingbot/connector/exchange_base.pyx", line 67, in hummingbot.connector.exchange_base.ExchangeBase.trading_pair_symbol_map
    await self._initialize_trading_pair_symbol_map()
  File "hummingbot/connector/exchange_base.pyx", line 365, in _initialize_trading_pair_symbol_map
    raise NotImplementedError
NotImplementedError

Steps:
create a perpetualMM strategy using ftx_perpetual
try to set up market pair for ftx_perpetual
Actual:
does not list available ftx perpetual markets when "tab" on markets parameter
Expected:
Bot list available ftx perpetual markets like it showed for other connectors

Steps:
create a perpetualMM strategy using ftx_perpetual
set up actual market pair
start the perpetualMM strategy using ftx_perpetual
Actual:
Failed to place an OPEN orders, stuck on unexpected error occurred when listening to orderbook streams.
(tried ETH-USD, ETH-PERP, recommended BTC-USD Pair also)
Expected:
Bot place orders to OPEN position
logs_conf_perpetual_market_making_2.log
conf_perpetual_market_making_2.zip
2022-07-21 07:27:37,112 - 137420 - hummingbot.connector.derivative.ftx_perpetual.ftx_perpetual_api_order_book_data_source - ERROR - Unexpected error occurred when listening to order book streams. Retrying in 5 seconds...
Traceback (most recent call last):
  File "/home/ubuntu/ftxp5438/hummingbot/core/data_type/order_book_tracker_data_source.py", line 79, in listen_for_subscriptions
    ws: WSAssistant = await self._connected_websocket_assistant()
  File "/home/ubuntu/ftxp5438/hummingbot/core/data_type/order_book_tracker_data_source.py", line 182, in _connected_websocket_assistant
    raise NotImplementedError
NotImplementedError

PR update:

Checked the latest commit 82ec19fbf06 but still getting same errors below:

error traiding pairs autometion wont work on the initial start
does not list available ftx perpetual markets when "tab" on markets parameter
Failed to start a perpetualMM strategy, stuck on unexpected error occurred when listening to orderbook streams. (don't place orders to OPEN position)

Tagging @mhrvth for possible update this week. Thank you!

Hello @rapcmia. The connector should now be updated to the latest development branch structure, and should be usable. I have also added more unit tests.

Test performed:
Cloned and compiled feature branch = OK
Started client and created password = OK
Connected API Keys with no issues = OK
Created/started PerpetualMM strategy = OK
Filled and cancel orders work accordingly = OK
Filled orders are tracked via datadog = OK
Filled orders on History, CSV Data file matched with the exchange = OK
Current Issues (commit a8b330f52f4ea9d2f94683ecd9d79493a4063306)
Checked status - status --live not updating
Data integrity verified synced with exchange - order_book --live not updating
Checking history - History not showing position type (showing NIL instead of OPEN or CLOSE)
Fees matched with the exchange - Failed (exchange maker 0.02% and taker 0.07% vs bot use flat fees 0.07% for all)

Upon checking after latest dev merged bot failed to connect/check balance for ftx_perpetual
Steps to reproduce:

Clone and install branch
start the bot
connect ftx_perpetual

Actual:
Bot showing error:
Unhandled error in background task: cannot import name 'FundingInfo' from 'hummingbot.core.event.events' (/home/ubuntu/ftx5438p/hummingbot/core/event/events.py)
Expected:
no errors and could successfully connect Ftx perpetual
logs_hummingbot.log
2022-08-16 05:15:13,021 - 128968 - hummingbot.core.utils.async_utils - ERROR - Unhandled error in background task: cannot import name 'FundingInfo' from 'hummingbot.core.event.events' (/home/ubuntu/ftx5438p/hummingbot/core/event/events.py)
Traceback (most recent call last):
  File "/home/ubuntu/ftx5438p/hummingbot/core/utils/async_utils.py", line 9, in safe_wrapper
    return await c
  File "/home/ubuntu/ftx5438p/hummingbot/client/command/balance_command.py", line 85, in show_balances
    all_ex_bals = await asyncio.wait_for(
  File "/home/ubuntu/miniconda3/envs/hummingbot/lib/python3.8/asyncio/tasks.py", line 483, in wait_for
    return fut.result()
  File "/home/ubuntu/ftx5438p/hummingbot/user/user_balances.py", line 130, in all_balances_all_exchanges
    await self.update_exchanges(client_config_map)
  File "/home/ubuntu/ftx5438p/hummingbot/user/user_balances.py", line 126, in update_exchanges
    results = await safe_gather(*tasks)
  File "/home/ubuntu/ftx5438p/hummingbot/core/utils/async_utils.py", line 22, in safe_gather
    return await asyncio.gather(*args, **kwargs)
  File "/home/ubuntu/ftx5438p/hummingbot/user/user_balances.py", line 100, in update_exchange_balance
    return await self.add_exchange(exchange_name, client_config_map, **api_keys)
  File "/home/ubuntu/ftx5438p/hummingbot/user/user_balances.py", line 76, in add_exchange
    market = UserBalances.connect_market(exchange, client_config_map, **api_details)
  File "/home/ubuntu/ftx5438p/hummingbot/user/user_balances.py", line 22, in connect_market
    connector_class = get_connector_class(exchange)
  File "/home/ubuntu/ftx5438p/hummingbot/client/config/config_helpers.py", line 470, in get_connector_class
    mod = __import__(conn_setting.module_path(),
  File "/home/ubuntu/ftx5438p/hummingbot/connector/derivative/ftx_perpetual/ftx_perpetual_derivative.py", line 44, in <module>
    from hummingbot.core.event.events import (
ImportError: cannot import name 'FundingInfo' from 'hummingbot.core.event.events' (/home/ubuntu/ftx5438p/hummingbot/core/event/events.py)


Add link to create release in unreleased-commits

If the feature detects unreleased commits, it makes sense to allow creating a new release from there, if you have access to the repo:

Originally posted by @fregante in https://github.com/refined-github/refined-github/issues/6052#issuecomment-1275709514

Renamed due to:

https://github.com/refined-github/refined-github/pull/6575
https://github.com/refined-github/refined-github/pull/6576


Can O' Spam: Watch webbeesite.com

Can O' Spam requests the watch of the watch_keyword webbeesite\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

webbeesite\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Jeff Schaller in Charcoal HQ


Inconsistent type of 'date-time' Model properties on instances

So I was writing/running some tests today and I was seeing some strange, inconsistent behavior. My app is a single-page app, so it makes a sort of "initialize" API call to start things off. This call does the following:

checks the cache for the necessary data for the requesting User

If it's found, user gets is value by being parsed something like const user = User.fromJson(dataFromCache);
If it's not found, user gets its value from a query that is made with a bunch of eager clauses for the User and other relations that I need.

Let's say that User has a HasManyRelation relation to Roles, and each Role has a created_at property which is defined on the schema as a string with the date-time format. I had some logic that was looking through the User.roles array for the Role with the most recent created_at time. The particulars of that logic, when I was writing/testing it originally, had found that typeof role.created_at was an object.
When I today found this issue, in my investigation it turns out that typeof role.created_at was actually string. Upon further investigation, the difference (at least in this particular case) between when typeof role.created_at was an object vs when it was a string depended on whether or not the User.roles were found in cache and loaded via User.fromJson(...) or whether they were loaded via User.query().eager('roles')....
That was my "eureka" moment.
It turns out that Objection (or something it uses) is sometimes making created_at a Date object and sometimes making it a String. This is a problem! I wrote a gist here to reproduce this issue, and also highlight a bunch of other scenarios where it comes out as one type or another:
https://gist.github.com/newhouse/c3b052a3548396274210827c6be66875
As you'll see, depending on how you are obtaining your instance, the time field comes out as a different type. In short:

Instances returned through a single (vs graph) insert return it as a string
Instances returned through a graph insert underneath a parent return it as a string (though time fields in the parent are objects!!!)
Instances created from SomeModel.fromJson({...}) return it as a string.
All other ways I tried return it as an object.

There are some interesting inconsistencies in the gist.
Needless to say, I'm not sure which way it "should" be, but at least if it were consistent, one could code with the assumption that it will always be a string (or object or whatever). This has caused me a lot of headaches.
Is this an Objection issue?

Objection does no type conversions whatsoever. Neither does knex. It's the db driver that does that. With postgres you can use pg-types library to change this behaviour. You can also implement the $fromDatabaseJson to make the conversion.

class BaseModel extends Model {
  $parseDatabaseJson(json) {
    json = super.$parseDatabaseJson(json);

    const propSchemas = this.jsonSchema.properties;

    Object.keys(propSchemas).forEach(prop => {
      const schema = propSchemas[prop];
      const value = json[prop];

      if (schema.format === 'date-time' && value instanceof Date) {
        json[prop] = value.toISOString();
      }
    });
  }

  return json;
}

class User extends BaseModel {
  ...
}

class Role extends BaseModel {
  ...
}

Or with pg-types:
const types = require('pg').types
const moment = require('moment')
const TIMESTAMPTZ_OID = 1184
const TIMESTAMP_OID = 1114
const parseFn = (val) => {
   return val === null ? null : moment(val).toISOString()
}
types.setTypeParser(TIMESTAMPTZ_OID, parseFn)
types.setTypeParser(TIMESTAMP_OID, parseFn)

Actually, this is all you need in $parseDatabaseJson
class BaseModel extends Model {
  $parseDatabaseJson(json) {
    json = super.$parseDatabaseJson(json);

    Object.keys(json).forEach(prop => {
      const value = json[prop];

      if (value instanceof Date) {
        json[prop] = value.toISOString();
      }
    });

    return json;
  }
}

class User extends BaseModel {
  ...
}

class Role extends BaseModel {
  ...
}

Thanks, @koskimas and thanks for the suggestions!
Since I may go bring this up over at pg, is there any insight you can provide based on my examples and your knowledge of what's going on under the hood query-wise that might point to specific things that bring out the different behaviors?
For example:
Why might a graph insert produce a parent with a Date, while its children have string types?  If you cannot, or don't want to spend the time, that's totally understandable!
Thanks again much for all of your hard work on Objection. It's a pleasure to work with!

Oh, nevermind, I think I understand it a bit more and it's still something that's definitely in pg land, and isn't a bug there, either. It's simply a side-effect of pgs "expected" behavior combined with particular things that Objection/knex do/don't do in each scenario (namely using the value returned from the DB vs the value provided by the User/code), right?

I'm pretty sure pg always returns dates as Date instances, but you can send pretty much whatever you like to it and it will convert it correctly. If you inserts stuff using objection, it doesn't fetch the data from pg (except for primary key) and simply returns the object you inserted. Maybe that causes the inconsistent types? You can use returning('*') or ***AndFetch variants of the queries.

One doesn't need to hardcode the constants, they are already provided by pg-types:
import { DateTime } from 'luxon'
import pg from 'pg'

pg.types.setTypeParser(
	pg.types.builtins.TIMESTAMPTZ,
	val => (val === null ? null : DateTime.fromSQL(val)),
)

Dates can also be transformed to objects.
class ExtendedModel extends Model {

  static get dateTimeProps() {
    const attributes = [];
    const props = Model.jsonSchema.properties || {};

    for (const propName of Object.keys(props)) {
      const prop = props[propName];

      if (prop.type === 'string' && prop.format === 'date-time') {
        attributes.push(propName);
      }
    }
    return attributes;
  }

  $afterInsert(ctx) {
    if (super.$afterInsert) super.$afterInsert(ctx);

    const attrs = this.constructor.dateTimeProps;
    for (const attr of attrs) {
      this[attr] = this[attr] && new Date(this[attr]);
    }
  }
};

However this doesn't work for Model.fromJson. Is there any hook to modify the instance just after it is created?

@javiertury $parseDatabaseJson is the hook you are looking for.

@koskimas unfortunately that suggestion doesn't work.
Insert sometimes returns an incomplete object(id only) which is the only thing $parseDatabaseJson can work with. Whereas $afterInsert gives you access to all of the instance fields.
Also, Model.fromJson(data) fails using any solution. If you use a date object, validation fails. If you use a string in the input JSON, you cannot transform the property to a date object later on. The reason is that $afterValidate() cannot modify instance properties.

    json = super.$parseDatabaseJson(json);

    Object.keys(json).forEach(prop => {
      const value = json[prop];

      if (value instanceof Date) {
        json[prop] = value.toISOString();
      }
    });

    return json;

When I add this to my BaseModel, my queries time out