GH-37969: [C++][Parquet] add more closed file checks for ParquetFileWriter

Rationale for this change
Operations on closed ParquetFileWriter are not allowed, but should not segfault. Somehow, ParquetFileWriter::Close() also reset its pimpl, so after that, any operators, those need this pointer will lead to segfault
What changes are included in this PR?
Adding more checks (assert actually, because it will throw exception instead of return arrow::Status) for closed file.
Are these changes tested?
Yes.
Are there any user-facing changes?
No.

Closes: #37969

(I didn't go through the issue carefully, but would CheckClosed function like S3 Filesystem better here?)

(I didn't go through the issue carefully, but would CheckClosed function like S3 Filesystem better here?)
e.g. https://github.com/apache/arrow/blob/main/cpp/src/arrow/filesystem/s3fs.cc#L1249

Thanks for your hint.  Refactored the checking into separate method. Use "Assert" here because it will throw instead of return arrow::Status. As operation on closed object is not a normal application flow.
Please help take a look again, @mapleFU

Use "Assert" here because it will throw instead of return arrow::Status. As operation on closed object is not a normal application flow.

Ooops, I think you're choosing the right way, but thats maybe because parquet lib uses exception, and arrow uses Status. So I think we'd better use Exception here. If we are do it in parquet/arrow directory we need use Status

  Status Close() override {
    if (!closed_) {
      // Make idempotent
      closed_ = true;
      if (row_group_writer_ != nullptr) {
        auto row_group_writer = std::move(row_group_writer_);
        PARQUET_CATCH_NOT_OK(row_group_writer->Close());
      }
      PARQUET_CATCH_NOT_OK(writer_->Close());
    }

Actually this change LGTM. But I'm not fully understand why row_group_writer_ will be double-closed. For example, when call FileWriter::Close twice, the closed_ will be true, so row_group_writer_ will only close once. Did I miss something?

Actually this change LGTM. But I'm not fully understand why row_group_writer_ will be double-closed. For example, when call FileWriter::Close twice, the closed_ will be true, so row_group_writer_ will only close once. Did I miss something?

you are right, normally, it will not be called twice. However, in test case for use-after-close with writer->Close() and then writer->NewBufferedRowGroup():
https://github.com/apache/arrow/blob/450175b2dbfc7834be72883c5a0e6ac93e8ed5aa/cpp/src/parquet/arrow/arrow_reader_writer_test.cc#L5227-L5237
To be honest, I think I need to study a bit more to fully understand state diagram of these there attributes {writer_, row_group_writer_ and closed_ } of FileWriterImpl. If you have hints for me, that will be great?
https://github.com/apache/arrow/blob/450175b2dbfc7834be72883c5a0e6ac93e8ed5aa/cpp/src/parquet/arrow/writer.cc#L479-L483
Thank you so much @mapleFU

Update:,

Revised MR (simplified) so that it focuses on the fix for writer->Close(); writer->WriteTable(..)
Double closing and other cases of use after close will need further understanding. Will open a new issue with your help to clarify the intended flow of the code.
How do you think? Thank you so much for guiding me along the way; I don't mind to document it down, get your review along with MR for that. Thanks @mapleFU

Parquet is about two parts in this library

parquet: the core part of parquet, mostly it handles types like parquet defined physical-type and logical type.
parquet/arrow: the arrow part of parquet. It read / write arrow array, and maintaining some other infos.

The writer has the structure below:
arrow::parquet::FileWriter (Holding parquet::ParquetFileWriter, and some arrow-related schema/metadata)
parquet::ParquetFileWriter ( holds fileMetadata and at most one active row-group writer)
- parquet::FileSerializer extends parquet::ParquetFileWriter::Contents
parquet::RowGroupWriter (Holding some ColumnWriter)
- parquet::RowGroupSerializer extends parquet::RowGroupWriter::Contents
ColumnWriter
- May hold encoder and write multiple pages

Also, there are some buffered and unbuffered RowGroupWriter. You can go through the doc for that.

Thank you, @mapleFU, for providing a clear and detailed guideline. I've applied these changes and added back two simple tests.
I have one final comment regarding the WriteTable function:
We currently have two options for handling errors in this function:

(A.) We can let writer->WriteTable throw a ParquetException. This is because, in this part of the code, it calls this->properties, which directly returns a ParquetException from writer_.
(B.) Alternatively, we can add RETURN_NOT_OK(CheckClosed()); to the beginning of WriteTable, similar to NewRowGroup and NewBufferedRowGroup. This would consistently return arrow::Status::Invalid.

I personally lean towards option (B.) for the sake of consistency. However, considering that use-after-close is not a typical flow (usually requiring user code revision to fully handle this case), using an exception might be a viable option here.
This part is making me uncertain, so I would appreciate your input on this matter. Thanks mappleFU.

Sorry for delaying reply.
Nice catch, I missed this->properties() call here. I prefer we can also add a CheckClosed in WriteTable, this is much more easier... Personally properties might be called frequently, which might be hard to debug and reason.

CI failed in unrelated
Will wait other committers review

@pitrou Should this better for adding it on
  Status WriteColumnChunk(const std::shared_ptr<ChunkedArray>& data, int64_t offset,
                          int64_t size) override {...}

  Status WriteRecordBatch(const RecordBatch& batch) override;

I think other methods like Init and schema() is not so neccessary here?

Everywhere where we rely on something that's destroyed by Close.

Well I think adding in functions I mentioned above is ok
Previously I think NewRowGroup is OK, because without WriteRecordBatch, every write operation will call New{Buffered}RowGroup, but this change also LGTM

Thanks @mapleFU and @pitrou
Added:

WriteColumnChunk(.. ) // Actual implementation; other two overloads all delegate to this.
WriteRecordBatch(.. )

Reviewed, safe to skip:

Init()  // Internally implemented and called only within FileWriter::Make, with a newly created writer.
schema() // Independent of the writer.
properties() // Internal implementation, called by WriteTable, WriteRecordBatch, both already validated.
metadata() // In the current implementation of ParquetFileWriter::Content, it's safe to be called when closed. Open to adding this.

update: 3 failing tests are unrelated, I believe.

Indeed, the CI failures are unrelated.

Thank you @quanghgx ! I will merge this PR now.


adding search scope

Docs Build status updates of commit 6912a47:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

dynamicsax2012-technet/docfx.json
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit db5c0da:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

dynamicsax2012-technet/docfx.json
:x:Error

Details

dynamicsax2012-technet/docfx.json

Line 74: [Error] Unexpected end of content while loading JObject.

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 43a5e2c:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

dynamicsax2012-technet/docfx.json
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Update ruff to 0.3.0

I made the mistake of formatting before fixing (locally) leaving some files to be formatted after fixing, therefore it failed the CI


This is an outdated fork

The project has since migrated off Google Code and onto GitHub where it continues to receive regular updates.
https://github.com/drewnoakes/metadata-extractor/

@drewnoakes - thanks, apologies it took me so long to take this down.


[GraphQL Client] Fix digests

This PR fixes digests to use the respective types. One thing that is confusing is the difference between CheckpointDigest and CheckpointContentsDigest.

A proper fix for the Checkpoint data is in #62.


i tried in garage

it didnt work

nvm

You might have to refresh. It depends on the browser you are using.


Only draw images, if loading didnt fail.

Mainly reduces spamming the console, if the image url is not valid, e.g. loading the url gives a 404.

Hello @helt, seems like a sensible addition. Let me check this with @jacomyal before merging.

bump :) Merge or close pls

Hello again @helt. Can you please edit your code a little to converge with the edited's file current style please?

Updated

Thanks @helt. Merging right now.


chore: cjxd-test1 to 0.0.1

chore: Promote cjxd-test1 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign
You can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Understanding if my usecases can be implemented using casbin or casdoor

Hi, I'm new to casbin/casdoor and wanted to understand if below usecases can be implemented using casbin or preferably casdoor. First, I'd like to explain in brief about my application, it's a multi-tenant app for which we've created separate schemas in our postgres db to store information related to each client separately.
Usecases:

Roles and permissions shared by all clients.
Custom roles, which clients can create for themselves and they'll be specific to the client that created them.
User groups. I want to assign permissions based on individual users and user groups inside an organisation.
I want to provide a User access to individual entities. Entities may be created in a hierarchy.
I want to create/modify/revoke user access, manage roles and user groups from my application's UI and not casdoor UI.
I want to maintain everything in postgres db.
For authentication, I'm using azure AD B2C.

I tried to implement this using casbin, but casbin only allows me to use a single enforcer and expects every policy (roles, user groups, access policies) to be in a single place so that it can match based on matcher config. Is it possible to keep user groups, roles and access policies in separate DB tables and enforce using data from multiple tables?
Also, can this be achieved using casdoor, considering I want to use my own application UI to perform these actions?

All the usercases can be done by Casdoor


hostNetwork?

Didn't found a way to set hostNetwork.

Hi @utegental,
you can set the hostNetwork as one of the pod: properties.
As an concrete example, the following should do the trick for an example Deployment myapp:
hull:
  objects:
    deployment:
      myapp:
        pod:
          hostNetwork: true
          containers:
            app:
              ...

Some explanations following:
Any workload based object instance includes the hull.PodTemplate.v1 schema which provides the pod field based on the hull.Pod.v1 schema.
Within hull.Pod.v1 you can work with the HULL based properties (here overwriting the Kubernetes objects properties):
initContainers
containers
volumes

however the documentation of hull.Pod.v1 also points to  the Kubernetes API JSON schema for pods where you can find all the other original Kubernetes properties you can directly set on the pod: property (e.g. for Kubenretes 1.27 the podspec-v1-core):
activeDeadlineSeconds
affinity
automountServiceAccountToken
...
hostNetwork
...

This is the composition idea of HULL. You can generally work with all existing Kubernetes properties on the object specifications. Only for particular aspects/properties HULL provides specific means of handling them (in a hopefully improved way). Here for the pod: specification it allows you to manage containers, initContainers and volumes as a dictionary to improve the merging of configuration layers instead of leaving them as arrays. On rendering everything is of course converted to Kubernetes spec conforming YAML.
Hope this helps, if you need further assistance please let me know. Thanks!

@gre9ory, that is very helpful! I'm moving services from deprecated 'k8s-at-home' and from my first experience hull is very simple.


Use codecov alongside coveralls

Codecov has a pretty nice interface, and this is a fairly minimal change.

Changes Unknown when pulling 0e8f5e0130dc66d57e441a67994853c3cf207ff4 on fw/codecov into ** on master**.


Update LimitedAccessFileType WebPreviewableFiles option control-access-from-unmanaged-devices.md

https://github.com/MicrosoftDocs/OfficeDocs-SharePoint/issues/1855

@kaarins please double check it. Thanks!

@JohanFreelancer9 Copy editing is needed for this PR. Thanks.

@kaarins PR has been copyedited and is ready for final review, could you please check and merge? Thanks!


pyroute2: init at 0.4.12

Motivation for this change
Things done

[x] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] Linux

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

could you put this in a separate nix file in python-modules.nix

Is this the new policy for python?

Let's make it a policy. The main file is getting huge and we have to do something about it.

@FRidh do you plan to add this to the Contributing Guidelines for python or decide this on a per package base?

Would be good to have it in the guidelines, although, many typically don't read the latest version but only the stable one. I would like to have a bot that comments on PR's with notes specific to the files that are touched. That works much better I think.

@FRidh bot is good, but I guess this will require some time to setup. Even if they do not read the guidelines immediately, we can quote them in reviews. The next release is although not far away.


How to use it?

I've take 78 pictures, when I use it, the prompts come: "No more images available."
It could only handle two pictures.

Hi, did you manage to run it?

Yes, I've downloaded the dataset, and run dataset Machine Hall 02, but failed.
Here's some runtime prompts:
2024-01-15 18:35:41,603 - INFO - pointer 68 c 81
2024-01-15 18:35:41,671 - INFO - Pydegensac found 118/141 inliers
2024-01-15 18:35:41,671 - INFO - median_match_dist: 79.83
camera 0
2024-01-15 18:35:41,672 - INFO - Frame accepted. New_keyframe image_id: 149
2024-01-15 18:35:41,695 - INFO - [Keyframe Selection] | features extraction=0.049, matching=0.019, innovation check=0.001,
2024-01-15 18:35:41,696 - INFO - DYNAMIC MATCHING WINDOW: 1
2024-01-15 18:35:41,696 - INFO - Initialize an empty database
2024-01-15 18:35:41,696 - INFO - Feature extraction
2024-01-15 18:35:43,174 - INFO - Sequential matcher
2024-01-15 18:35:44,493 - INFO - MAPPER
2024-01-15 18:35:46,180 - INFO - Convert model from binary to txt
2024-01-15 18:35:46,323 - INFO - [COLMAP] | DATABASE INITIALIZATION=0.000, FEATURE EXTRACTION=1.478, SEQUENTIAL MATCHER=1.319, MAPPER=1.687, MODEL CONVERSION=0.143,
True
keyframe<PHONE_NUMBER>651666432.jpg
Traceback (most recent call last):
File "E:\Coding_Packages\COLMAP_SLAM\main.py", line 601, in
if k.keyframe_name ==  None:
AttributeError: 'NoneType' object has no attribute 'keyframe_name'

In theory should be a bug that rarely happen. I work on it. Let me know if instead it happens every time you try to run it


Fix plugin detection if app runs from jar

I was able to fix this issue by adding environment variable LOADER_PATH to the Dockerfile (see https://docs.spring.io/spring-boot/docs/current/reference/html/executable-jar.html#appendix.executable-jar.property-launcher) and applying the workaround mentioned in the issue above.


Fix an offense creation for Lint/EmptyFile

Closes #8669

Can you explain why those changes are needed? Personally, I don't quite get it.

I thought the HTML formatter was the culprit (or maybe even my rewriter code)

The original code
range = source_range(processed_source.buffer, 1, 0)
add_offense(range)

adds an offense to non existent (empty) source range - 1st line, 0-th column.
When generating html report (I didn't tested, but this can be a problem for other formatters), this https://github.com/rubocop-hq/rubocop/blob/9bc7908b1ad171b33ede24908acced724db6e8d4/lib/rubocop/formatter/html_formatter.rb#L118-L126
119 and 124 lines fails here
https://github.com/whitequark/parser/blob/c25585d8d91f3470d9cdfd6f3b6a553ad9a809fb/lib/parser/source/buffer.rb#L269-L271
because of fetch on empty source_lines.
add_global_offense adds a global offense, roughly speaking, not attached to any specific location and overrides source_line to empty string, so this eliminates the problem.

Got it. Might be nice to integrate those global offenses with expect_offence, though, as I'm certain people will wonder why are the tests using it.

Might be nice to integrate those global offenses with expect_offence

syntax_spec.rb uses the same style, as this test file, so also needs an update.
Wdyt about introducing it in a separate PR?

I think:
source_range(processed_source.buffer, 1, 0)
# should be
source_range(processed_source.buffer, 1, 0, 0)

FWIW, this helper is terrible. There are too many of these multiple argument public methods... The cop in charge of that should really have different settings for private (I don't care much, really) and public (should be max 1 imo).
If you prefer to go the add_global_offense route (seems a bit more complicated), I think the odd spec is fine. add_global_offense was really meant for non-parseable files tbh.

source_range(processed_source.buffer, 1, 0)

should be
source_range(processed_source.buffer, 1, 0, 0)
Just tested, the same error as in the original issue.

Well, I guess the suggested fix will do for now. Thanks!


Parse zipkin v2 high trace id

https://github.com/jaegertracing/jaeger/pull/566 with added test cc @moolex and thanks

Coverage remained the same at 100.0% when pulling dce954d44d26faae200774b39d8a5efa9ab1b404 on pavolloffay:zipkin-v2-hightraceid into 2974dce6c2aa9d0f59bdd004a18b6a2bf62274e5 on jaegertracing:master.


[Feat]: Add Movies tools in DB

What feature?
I have an idea to add one movie tool to the database named cataz.net, one of the best websites to stream your favourite series online.
Add screenshots

Code of Conduct

[X] I agree to follow this project's Code of Conduct

@JasonDsouza212 If you want these changes, please assign me this issue under GSSoC'23


Fails if outputFile specifies a non-existing directory

After upgrading to 4.2.1 our ember builds started failing (at least on macOS).
>> (node:52291) UnhandledPromiseRejectionWarning: Error: ENOENT: no such file or directory, open '/var/folders/30/3l01y1ld18l598kxr5srm7kr0000gn/T/broccoli-52291zjOJHNK9FLVI/out-541-simple_concat_concat/assets/codemirror/codemirror.js'
>>     at Object.openSync (fs.js:443:3)
>>     at Object.writeFileSync (fs.js:1194:35)
>>     at Object.handleFs (/Users/kevinansfield/code/ghost/Ghost/core/client/node_modules/broccoli-concat/node_modules/broccoli-output-wrapper/dist/index.js:45:44)
>>     at Concat._doPatchBasedBuild (/Users/kevinansfield/code/ghost/Ghost/core/client/node_modules/broccoli-concat/concat.js:166:17)
>>     at Concat.build (/Users/kevinansfield/code/ghost/Ghost/core/client/node_modules/broccoli-concat/concat.js:104:19)
>>     at resolve (/Users/kevinansfield/code/ghost/Ghost/core/client/node_modules/broccoli/dist/wrappers/transform-node.js:68:45)
>>     at new Promise (<anonymous>)
>>     at TransformNodeWrapper.build (/Users/kevinansfield/code/ghost/Ghost/core/client/node_modules/broccoli/dist/wrappers/transform-node.js:53:16)
>>     at Promise.resolve.then.then.then (/Users/kevinansfield/code/ghost/Ghost/core/client/node_modules/broccoli/dist/builder.js:99:36)
>>     at process._tickCallback (internal/process/next_tick.js:68:7)

The relevant part of our ember-cli-build.js where we use broccoli-concat is as follows:
let jsTree = concat(tree, {
    outputFile: 'assets/codemirror/codemirror.js',
    headerFiles: ['lib/codemirror.js'],
    inputFiles: ['mode/**/*'],
    sourceMapConfig: {enabled: false}
});

Changing the outputFile to remove the directory allows that step of the build to pass. Downgrading to 4.0.0 and keeping the directory in outputFile also allows works fine.

This is the culprit -- changing from fs-extra's outputFileSync which creates missing directories to wherever that writeFileSync points, which apparently does not.

@SparshithNR - Can you look into this? I think we'll need to make some changes in broccoli-output-wrapper to accommodate...

@rwjblue https://github.com/SparshithNR/broccoli-output-wrapper/pull/10

Closing as this appears to have been resolved. Thanks! 😄


Created release 1.1.0

Description
Update to master of the last release made.
Status
READY
Type of change

New features

Checklist:

[x] I have made corresponding changes to the documentation (if applicable)
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

Already merged to develop. Need a newer pull request


lxd/storage: Fix regression in truncate handling

Signed-off-by: Stéphane Graber<EMAIL_ADDRESS>

Testsuite passed


JAVA-151: Make it possible to register for SchemaChange Events.

This commit contains a substantial amount of code originally contributed by Robert Stupp (@snazy).

Since SchemaChangeListener is a new type, it would be nice to have onRegister and onUnregister, like we added for other listeners on the 2.2 branch. I'm going to add it.

Added a test and a small change to fix an issue I was running into.  Looking good to me :+1:


Add keyword "const"

The keyword was introduced in 2016.
See Microsoft/dafny@6e97c13f5d0f7157771c41969517078f552ac5e4

Thanks!


https://github.com/users/BSCPGROUPHOLDINGSLLC/projects/1#column-18309490

https://github.com/users/BSCPGROUPHOLDINGSLLC/projects/1#column-18309490

THEY DONT HAVE RECEIPTS FOR THOSE GROSS INCOMES AND RETURNS. 100% GUARANTEED
FOR NONE OF THE BUILDINGS....


yquake2: 7.43 -> 8.00

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/yquake2/versions.
meta.description for yquake2 is: "Yamagi Quake II client"
meta.homepage for yquake2 is: "https://www.yamagi.org/quake2/"
meta.changelog for yquake2 is: ""
Updates performed

Version update

To inspect upstream changes

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

Warning: no invocation of /nix/store/vlxj7nr5m8n64yyqav4d6849rm48x3a7-yquake2-8.00/bin/yquake2 had a zero exit code or showed the expected version

Warning: no invocation of /nix/store/vlxj7nr5m8n64yyqav4d6849rm48x3a7-yquake2-8.00/bin/yq2ded had a zero exit code or showed the expected version

0 of 2 passed binary check by having a zero exit code.

0 of 2 passed binary check by having the new version present in output.

found 8.00 with grep in /nix/store/vlxj7nr5m8n64yyqav4d6849rm48x3a7-yquake2-8.00

found 8.00 in filename of file in /nix/store/vlxj7nr5m8n64yyqav4d6849rm48x3a7-yquake2-8.00

directory tree listing: https://gist.github.com/55e4a8a937a3ea5a636c5f3f64e0f00f

du listing: https://gist.github.com/9172634f90ad47d997165883bde102c1

Rebuild report (if merged into master) (click to expand)

8 total rebuild path(s)

5 package rebuild(s)

5 x86_64-linux rebuild(s)
1 i686-linux rebuild(s)
1 x86_64-darwin rebuild(s)
1 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
yquake2
yquake2-all-games
yquake2-ctf
yquake2-ground-zero
yquake2-the-reckoning

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/vlxj7nr5m8n64yyqav4d6849rm48x3a7-yquake2-8.00 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A yquake2 https://github.com/r-ryantm/nixpkgs/archive/117ea32791b044dd971e44decab3a878ddff8ee0.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/vlxj7nr5m8n64yyqav4d6849rm48x3a7-yquake2-8.00
ls -la /nix/store/vlxj7nr5m8n64yyqav4d6849rm48x3a7-yquake2-8.00/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

5 packages built:

yquake2
yquake2-all-games
yquake2-ctf
yquake2-ground-zero
yquake2-the-reckoning

Maintainer pings
cc @tadfisher for testing.

Result of nixpkgs-review pr 133941 at 117ea327 run on x86_64-linux 1

5 packages built successfully:

yquake2
yquake2-all-games
yquake2-ctf
yquake2-ground-zero
yquake2-the-reckoning

2 suggestions:

warning: missing-phase-hooks
installPhase should probably contain runHook preInstall and runHook postInstall.
Near pkgs/games/quake2/yquake2/default.nix:48:5:
   |
48 |     installPhase = ''
   |     ^

warning: unclear-gpl
gpl2 is a deprecated license, please check if project uses gpl2Plus or gpl2Only and change meta.license accordingly.
Near pkgs/games/quake2/yquake2/default.nix:61:7:
   |
61 |       license = licenses.gpl2;
   |       ^


update(scale-out): docs for redis backplane

The sticky session was a little bit vague so that one might think even if they use just WebSocket for all the clients and turn off negotiations on clients, and use Redis backplane they still might need "sticky sessions". There was a clarification by @davidfowl here

@shahabganji  thanks for the PR. Nice addition.

@Rick-Anderson Thanks for the great job you guys are doing. Keep up the good work :+1:


[AutoPR applicationinsights/resource-manager] Dev ApplicationInsights microsoft.insights 2019

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/7539

This PR has been merged into https://github.com/Azure/azure-sdk-for-python/pull/9020


Semantic folder in dataset seems incorrect

I inspected the semantic folder generated by the generate_dataset.py. I think it should be the ground truth of semantic ID of each image. However, when I randomly printed them out, I found it seemed not very correct.
For example, it is 200.png in 5LpN3gDmAk7_1. However, the values in the semantic matrix are all 17, which are ceilings.

Hi @sunyyyl0106, sorry for the late reply. Did you solve the problem in the end and how did you solve it?

Hi @sunyyyl0106, sorry for the late reply. Did you solve the problem in the end and how did you solve it?

I referred to the official Habitat repo to generate the ground truth.

Great, it would be nice if you can share some details and close the issue!

Actually, there is a tiny issue with the generate_scene_data function in the generate_dataset.py. You can replace the following with the original one.
def generate_scene_data(save_dir: Union[Path, str], config: DictConfig, scene_path: Path, poses: np.ndarray) -> None:
    """
    config: config for the sensors of the collected data
    scene_path: path to the Matterport3D scene file *.glb
    poses: (N, 7), each line has (px, py, pz, qx, qy, qz, qw)
    """
    sim_setting = {
        "scene": str(scene_path),
        "default_agent": 0,
        "sensor_height": config.camera_height,
        "color_sensor": config.rgb,
        "depth_sensor": config.depth,
        "semantic_sensor": config.semantic,
        "move_forward": 0.1,
        "turn_left": 5,
        "turn_right": 5,
        "width": config.resolution.w,
        "height": config.resolution.h,
        "enable_physics": False,
        "seed": 42,
    }
    cfg = make_cfg(sim_setting)
    sim = habitat_sim.Simulator(cfg)

    # get the dict mapping object id to semantic id in this scene
    obj2cls = get_obj2cls_dict(sim)

    # initialize the agent in sim
    agent = sim.initialize_agent(sim_setting["default_agent"])
    pbar = tqdm(poses, leave=False)
    for pose_i, pose in enumerate(pbar):
        pbar.set_description(desc=f"Frame {pose_i:06}")
        agent_state = habitat_sim.AgentState()
        agent_state.position = pose[:3]
        agent_state.rotation = pose[3:]
        agent.set_state(agent_state)
        # Get agent state
        agent_state = agent.get_state()
        obs = sim.get_sensor_observations()
        save_obs(save_dir, sim_setting, obs, pose_i, obj2cls)

    sim.close()


Chat: update logic to select Deep Cody as default model when available

CLOSE https://linear.app/sourcegraph/issue/CODY-3988/move-enrollment-out-of-models-getter that addresses comment in https://github.com/sourcegraph/cody/pull/5687/files#r1788676940
Main change
Eemove the logic that mutate user-selected model from the "getter" in ChatController on models changes & add new logic handles setting Deep Cody as the default on first sync:

Set Deep Cody as default chat model on first sync when it is available.
Add logic to set the user's selected chat model to Deep Cody during the first sync.

This also fixes the issue where Deep Cody is showing up in the Edit model dropdown as we updated to set data.preferences!.defaults.edit and data.preferences!.defaults.chat separately:

Before

Test plan

Manual Test Plan:

Log into S2 and make sure you can see Deep Cody is selected as the default model for you in the Chat model dropdown list.
Verify you can switch chat model
Log into DotCom to ensure everything works as expected

Added new unit test for syncing Deep Cody model
Changelog

I'm not able to fully grok the complexities of observables and the different edge cases, but I do have the rudimentary understanding of the code. The way I tested this PR is by checking if it logically does what it claims to do:

When Deep Cody is available, then it's set as the default chat model on the first sync ->  I logged into S2.
And I got deep cody in my chat and In the edit flow of S2, it did not show DeepCody as a model(expected behaviour)

For dotcom, I don't see DeepCody as a model, both in chat and in edit, which I believe is expected behavior.

So, approving the PR because it does what it claims to do.


TTS Steaming is Broken for livekit-plugins-elevenlabs v0.4.0

This is a more clear description + RCA of #279
I think the issue is related to the TTS streaming implementation of livekit-plugins-elevenlabs. The reason I think that is because when I comment out this code from the assistant.py code, the very first assistant.say call is sent down to the agent playground.
Debugging further, I was able to verify that OpenAI is sending down the correct response to voices, but it wasn't getting streamed as audio to the playground.
I also tried downgrading to all the 0.4.0 dev versions, and none of them fixed the issue. 0.3.0 didn't work at all with the other packages.
The product code I'm using is the official agents quickstart guide.

I am facing the same limitation. Calling the _synthesize_task with  raw text _SpeechData object works perfect though.
Somehting else that I don't understand about this, is that _play_speech_if_validated is supposed to be validating that the streaming information is available in po_tx, however, po_tx seems to be empty when entering the _synthesize_task.

@theomonnom i commented out the exact code you're commenting in your PR and it didn't fix it. Your change is prob good, but doesn't fix the root cause of my issue.

Agree, this is something different. It seems to be that po_tx is empty when we arrive at that point of the code. And that shouldn't happen as po_tx was already validated as containing data. Something is not working with the streaming on the plugin side.

Facing same issues, trying to run the minimal voice assistant demo. I can see that audio is properly detected, transcribed, sent to the OpenAI api, but no speech is returned

I logged what you get in the receiver after sending the first chunk (here), and I obtained the following message:
{"message":"Unusual activity detected. Free Tier usage disabled. If you are using a proxy/VPN you might need to purchase a Paid Plan to not trigger our abuse detectors. Free Tier only works if users do not abuse it, for example by creating multiple free accounts. If we notice that many people try to abuse it, we will need to reconsider Free Tier altogether. \nPlease play fair and purchase any Paid Subscription to continue.","error":"detected_unusual_activity","code":1008}

Any idea if this is related to the implementation (for example how the websockets are being opened and closed), or any other reason? I am under the trial period, and this service should work. Same API key works fine in the Rest API.

I logged what you get in the receiver from elevenlabs after sending the first chunk (here), and I obtained the following message:
{"message":"Unusual activity detected. Free Tier usage disabled. If you are using a proxy/VPN you might need to purchase a Paid Plan to not trigger our abuse detectors. Free Tier only works if users do not abuse it, for example by creating multiple free accounts. If we notice that many people try to abuse it, we will need to reconsider Free Tier altogether. \nPlease play fair and purchase any Paid Subscription to continue.","error":"detected_unusual_activity","code":1008}

Any idea if this is related to the implementation (for example how the websockets are being opened and closed), or any other reason? I am under the trial period, and this service should work. Same API key works fine in the Rest API.

I tried using OpenAI TTS and had the same issue

I had this same error as @fjprobos and @gullerg . Seems the Free Tier has limits. I subscribed to ElevenLabs and now it works fine. Here is the error from the agent:
{"asctime": "2024-05-17 13:48:41,051", "level": "ERROR", "name": "livekit.plugins.elevenlabs", "message": "11labs connection failed\nTraceback (most recent call last):\n  File \"/Users/*****/git/livekit-test/.venv/lib/python3.11/site-packages/livekit/plugins/elevenlabs/tts.py\", line 367, in _run_ws\n    await asyncio.gather(send_task(), recv_task())\n  File \"/Users/theRealMarkCastillo/git/livekit-test/.venv/lib/python3.11/site-packages/livekit/plugins/elevenlabs/tts.py\", line 342, in recv_task\n    raise Exception(\"11labs connection closed unexpectedly\")\nException: 11labs connection closed unexpectedly\n", "job_id": "****", "pid": ****}

okay, thanks @theRealMarkCastillo. I tried switching to the OpenAI TTS, but streaming didn't seem to be supported. Anyone had any luck using OpenAI TTS?

Is there any progress on this? I'm also facing an issue where Eleven Labs isn't sending the streaming to the client.

I was able to make it work after upgrading to Elevenlabs paid subscription. They don't communicate that the free trial has this limitation.

{"asctime": "2024-05-22 10:48:38,712", "level": "WARNING", "name": "livekit.plugins.elevenlabs", "message": "failed to connect to 11labs, retrying in 0s", "taskName": "Task-12", "job_id": "AJ_Xz4RqLvwVwUu", "pid": 46658}
Data is returned when directly calling the api interface, but an error is reported through the official website case, saying that the connection has timed out？

most likely related to: https://github.com/livekit/agents/issues/305
Re: OpenAI TTS. Streaming is supported with our StreamAdapter class. Examples coming soon

@keepingitneil great!! Look forward to the OpenAI TTS. Commented the below on another issue (closed now though).
Just pulled latest from main and tried to run the OpenAI TTS. Now, I get the following error:
{"asctime": "2024-05-22 18:15:32,198", "level": "WARNING", "name": "livekit.agents", "message": "Running <Task pending name='Task-17' coro=<VoiceAssistant._synthesize_task() running at /Users/***/dev/livekit/venv2/lib/python3.11/site-packages/livekit/agents/voice_assistant/assistant.py:737> wait_for=<Future pending cb=[Task.task_wakeup()]>> took too long: 4.99 seconds", "job_id": "AJ_25Hy5b7Cj9xj", "pid": 49646}
{"asctime": "2024-05-22 18:15:32,204", "level": "ERROR", "name": "livekit.plugins.openai", "message": "openai tts main task failed in chunked stream\nTraceback (most recent call last):\n  File \"/Users/***/dev/livekit/venv2/lib/python3.11/site-packages/livekit/plugins/openai/tts.py\", line 89, in _run\n    async with self._session.post(\n  File \"/Users/***/dev/livekit/venv2/lib/python3.11/site-packages/aiohttp/client.py\", line 1197, in __aenter__\n    self._resp = await self._coro\n                 ^^^^^^^^^^^^^^^^\n  File \"/Users/***/dev/livekit/venv2/lib/python3.11/site-packages/aiohttp/client.py\", line 437, in _request\n    data = payload.JsonPayload(json, dumps=self._json_serialize)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/***/dev/livekit/venv2/lib/python3.11/site-packages/aiohttp/payload.py\", line 396, in __init__\n    dumps(value).encode(encoding),\n    ^^^^^^^^^^^^\n  File \"/Users/***/.pyenv/versions/3.11.1/lib/python3.11/json/__init__.py\", line 231, in dumps\n    return _default_encoder.encode(obj)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/***/.pyenv/versions/3.11.1/lib/python3.11/json/encoder.py\", line 200, in encode\n    chunks = self.iterencode(o, _one_shot=True)\n             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/***/.pyenv/versions/3.11.1/lib/python3.11/json/encoder.py\", line 258, in iterencode\n    return _iterencode(o, 0)\n           ^^^^^^^^^^^^^^^^^\n  File \"/Users/***/.pyenv/versions/3.11.1/lib/python3.11/json/encoder.py\", line 180, in default\n    raise TypeError(f'Object of type {o.__class__.__name__} '\nTypeError: Object of type _LiteralGenericAlias is not JSON serializable\n", "job_id": "AJ_25Hy5b7Cj9xj", "pid": 49646}

This is my code:
import asyncio
import logging

from livekit import agents

from livekit.agents import JobContext, JobRequest, WorkerOptions, cli
from livekit.agents.llm import (
    ChatContext,
    ChatMessage,
    ChatRole,
)
from livekit.agents.voice_assistant import VoiceAssistant
from livekit.plugins import deepgram, nltk, openai, silero

# This function is the entrypoint for the agent.
async def entrypoint(ctx: JobContext):
    # Create an initial chat context with a system prompt
    initial_ctx = ChatContext(
        messages=[
            ChatMessage(
                role=ChatRole.SYSTEM,
                text="You are a voice assistant created by LiveKit. Your interface with users will be voice. Pretend we're having a conversation, no special formatting or headings, just natural speech.",
            )
        ]
    )

    # VoiceAssistant is a class that creates a full conversational AI agent.
    # See https://github.com/livekit/agents/blob/main/livekit-agents/livekit/agents/voice_assistant/assistant.py
    # for details on how it works.

    openai_tts = openai.TTS(
            model=openai.TTSModels,
            voice=openai.TTSVoices
    )
    STREAM_SENT_TOKENIZER = nltk.SentenceTokenizer(min_sentence_len=20)
    tts = agents.tts.StreamAdapter(tts=openai_tts, sentence_tokenizer=STREAM_SENT_TOKENIZER)

    assistant = VoiceAssistant(
        vad=silero.VAD(), # Voice Activity Detection
        stt=deepgram.STT(), # Speech-to-Text
        llm=openai.LLM(), # Language Model
        tts=tts,
        chat_ctx=initial_ctx, # Chat history context
    )

    # Start the voice assistant with the LiveKit room
    assistant.start(ctx.room)

    await asyncio.sleep(3)

    # Greets the user with an initial message
    await assistant.say("Hey, how can I help you today?", allow_interruptions=True)

# This function is called when the worker receives a job request
# from a LiveKit server.
async def request_fnc(req: JobRequest) -> None:
    logging.info("received request %s", req)
    # Accept the job tells the LiveKit server that this worker
    # wants the job. After the LiveKit server acknowledges that job is accepted,
    # the entrypoint function is called.
    await req.accept(entrypoint)

if __name__ == "__main__":
    # Initialize the worker with the request function
    cli.run_app(WorkerOptions(request_fnc))

Hey, the arguments are not valid, these must be strings
            model=openai.TTSModels,
            voice=openai.TTSVoices

Hey this should now be fixed in the newer versions. It was due to 11labs changing their pricing policy

Anyone knows how to change the voice for elevenlabs to a custom voice? I couldn't find in documentation :/


Deleting a bullet still remains indented

What is the problem?
In the release of July 22, 2018, an indent still remains after deleting a bullet.
How can I reproduce it?
Type -  (a dash and a space, for an empty bullet point) then delete it. The cursor will be indented two spaces, rather than indented zero spaces, like it was before.
Is there any error message in the console?

No

Please restore behavior to before, or add an option to do so. Thanks!

Yes, this is an intentional change. I will introduce a new option for it.

One more question, are you trying to delete the list marker and then press enter to start a new line?
You can directly press enter, empty list item will be removed automatically.

Okay I'll keep that in mind


not_x_content_exception: Compressor detection

I am trying to run the latest version to see if performance is improved (I had the previous release running fine) but now I am getting this error on bulk inserts:
not_x_content_exception: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes
I have my bulk insert size set to only 10 for testing.  Here's the full error:
warn  2018-10-23T14:23:55.612 thread-13 exceptions.hpp:37             handle_elasticsearch ] elasticsearch exception, bulk exception, line 24, 3230006 bulk_fail_exception: Perform bulk get non zero errors
bulk perform errors: {"took":50,"errors":true,"items":[{"index":{"_index":"action_traces","_type":"_doc","_id":"rT1PoWYBpAN5--iPhhWo","status":400,"error":{"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"not_x_content_exception","reason":"not_x_content_exception: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"}}}},{"index":{"_index":"action_traces","_type":"_doc","_id":"rj1PoWYBpAN5--iPhhWo","status":400,"error":{"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"not_x_content_exception","reason":"not_x_content_exception: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"}}}},{"index":{"_index":"transaction_traces","_type":"_doc","_id":"rz1PoWYBpAN5--iPhhWo","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":108,"_primary_term":1,"status":201}},{"index":{"_index":"action_traces","_type":"_doc","_id":"sD1PoWYBpAN5--iPhhWo","status":400,"error":{"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"not_x_content_exception","reason":"not_x_content_exception: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"}}}},{"index":{"_index":"transaction_traces","_type":"_doc","_id":"sT1PoWYBpAN5--iPhhWo","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":118,"_primary_term":1,"status":201}},{"index":{"_index":"action_traces","_type":"_doc","_id":"sj1PoWYBpAN5--iPhhWo","status":400,"error":{"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"not_x_content_exception","reason":"not_x_content_exception: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"}}}},{"index":{"_index":"transaction_traces","_type":"_doc","_id":"sz1PoWYBpAN5--iPhhWo","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":119,"_primary_term":1,"status":201}},{"index":{"_index":"action_traces","_type":"_doc","_id":"tD1PoWYBpAN5--iPhhWo","status":400,"error":{"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"not_x_content_exception","reason":"not_x_content_exception: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"}}}},{"index":{"_index":"transaction_traces","_type":"_doc","_id":"tT1PoWYBpAN5--iPhhWo","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":109,"_primary_term":1,"status":201}},{"index":{"_index":"action_traces","_type":"_doc","_id":"tj1PoWYBpAN5--iPhhWo","status":400,"error":{"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"not_x_content_exception","reason":"not_x_content_exception: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"}}}}]}
{"text":"{"took":50,"errors":true,"items":[{"index":{"_index":"action_traces","_type":"_doc","_id":"rT1PoWYBpAN5--iPhhWo","status":400,"error":{"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"not_x_content_exception","reason":"not_x_content_exception: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"}}}},{"index":{"_index":"action_traces","_type":"_doc","_id":"rj1PoWYBpAN5--iPhhWo","status":400,"error":{"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"not_x_content_exception","reason":"not_x_content_exception: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"}}}},{"index":{"_index":"transaction_traces","_type":"_doc","_id":"rz1PoWYBpAN5--iPhhWo","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":108,"_primary_term":1,"status":201}},{"index":{"_index":"action_traces","_type":"_doc","_id":"sD1PoWYBpAN5--iPhhWo","status":400,"error":{"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"not_x_content_exception","reason":"not_x_content_exception: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"}}}},{"index":{"_index":"transaction_traces","_type":"_doc","_id":"sT1PoWYBpAN5--iPhhWo","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":118,"_primary_term":1,"status":201}},{"index":{"_index":"action_traces","_type":"_doc","_id":"sj1PoWYBpAN5--iPhhWo","status":400,"error":{"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"not_x_content_exception","reason":"not_x_content_exception: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"}}}},{"index":{"_index":"transaction_traces","_type":"_doc","_id":"sz1PoWYBpAN5--iPhhWo","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":119,"_primary_term":1,"status":201}},{"index":{"_index":"action_traces","_type":"_doc","_id":"tD1PoWYBpAN5--iPhhWo","status":400,"error":{"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"not_x_content_exception","reason":"not_x_content_exception: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"}}}},{"index":{"_index":"transaction_traces","_type":"_doc","_id":"tT1PoWYBpAN5--iPhhWo","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":109,"_primary_term":1,"status":201}},{"index":{"_index":"action_traces","_type":"_doc","_id":"tj1PoWYBpAN5--iPhhWo","status":400,"error":{"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"not_x_content_exception","reason":"not_x_content_exception: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"}}}}]}"}
Any help would be appreciated.

Never seen this before. Seem's like action trace index all failed, can you offer more information. like what exactly the mapping of action_trace index looks like. also version of your elasticsearch.

I'm using the source direct from this repo, so here's the action_traces mapping:
static const std::string action_traces_mapping = R"(
{
"mappings": {
"_doc": {
"properties": {
"createAt": {
"type": "date"
},
"updateAt": {
"type": "date"
},
"receipt": {
"enabled": false
},
"act": {
"enabled": false
},
"account_ram_deltas": {
"enabled": false
}
}
}
}
}
)";
ES version is 6.4.2.

I did a long run today (over 5 million blocks processed) and I never got a single document in the action_traces index.
Here's the error from elasticsearch's logs if it's helpful:
[2018-10-24T20:07:36,633][DEBUG][o.e.a.b.TransportShardBulkAction] [action_traces][3] failed to execute bulk item (index) BulkShardRequest [[action_traces][3]] containing [116] requests
org.elasticsearch.index.mapper.MapperParsingException: failed to parse
        at org.elasticsearch.index.mapper.DocumentParser.wrapInMapperParsingException(DocumentParser.java:171) ~[elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:72) ~[elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:263) ~[elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:725) ~[elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.index.shard.IndexShard.applyIndexOperation(IndexShard.java:702) ~[elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.index.shard.IndexShard.applyIndexOperationOnPrimary(IndexShard.java:682) ~[elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.bulk.TransportShardBulkAction.lambda$executeIndexRequestOnPrimary$2(TransportShardBulkAction.java:560) ~[elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.bulk.TransportShardBulkAction.executeOnPrimaryWhileHandlingMappingUpdates(TransportShardBulkAction.java:579) ~[elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.bulk.TransportShardBulkAction.executeIndexRequestOnPrimary(TransportShardBulkAction.java:558) ~[elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.bulk.TransportShardBulkAction.executeIndexRequest(TransportShardBulkAction.java:141) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.bulk.TransportShardBulkAction.executeBulkItemRequest(TransportShardBulkAction.java:247) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.bulk.TransportShardBulkAction.performOnPrimary(TransportShardBulkAction.java:124) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:111) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:73) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:1017) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:995) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:101) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:356) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:296) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:958) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:955) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:271) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:238) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationPermit(IndexShard.java:2249) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryShardReference(TransportReplicationAction.java:967) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction.access$500(TransportReplicationAction.java:97) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:317) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:292) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$1.doRun(SecurityServerTransportInterceptor.java:251) [x-pack-security-6.4.2.jar:6.4.2]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:309) [x-pack-security-6.4.2.jar:6.4.2]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:66) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:665) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:723) [elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.4.2.jar:6.4.2]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Caused by: org.elasticsearch.common.compress.NotXContentException: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes
        at org.elasticsearch.common.compress.CompressorFactory.compressor(CompressorFactory.java:56) ~[elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.common.xcontent.XContentHelper.createParser(XContentHelper.java:69) ~[elasticsearch-6.4.2.jar:6.4.2]
        at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:65) ~[elasticsearch-6.4.2.jar:6.4.2]
        ... 38 more

Here's an example of the conversation between the client and elasticsearch: https://drive.google.com/file/d/10tZWtKSyQXLyT9kbApowqeBM7CFyo5JC/view?usp=sharing

Here's an example of the conversation between the client and elasticsearch: https://drive.google.com/file/d/10tZWtKSyQXLyT9kbApowqeBM7CFyo5JC/view?usp=sharing

{"index":{"_index":"action_traces","_type":"_doc","retry_on_conflict":100}}
[{"receipt":{"receiver":  ..................... }]

It seems that the elasticsearch exception caused by extra square brackets of the action_trace source line. It's very strange that the document get serialized to that form. In my case it look perfectly normal:
{"index":{"_index":"action_traces","_type":"_doc","retry_on_conflict":100}}
{"receipt":{"receiver":"eosio","act_digest":"f95697d1abf649...........................}

what is your eosio version, I test the 1.3.0, 1.4.1 it's both ok. Also, nodeos running in what env. I run it in ubuntu 18.04.

I am currently running 1.4.1 on CentOS 7.  I had previously run on 1.3.1 and it worked, so I will try rebuilding that version and see if helps.

I reverted to eos 1.3.2 with the current version of elasticsearch_plugin and it still could not insert any action_traces.
By reverting to eos 1.3.2 with elasticsearch_plugin commit id  7bfdf7aeba89f8aa40ab16e4cb30cb331663d297 it works again.  So it seems to be something related to the changes that came in with the multithreading.

I rebuilt eosio 1.4.1 with the latest elasticsearch_plugin on Ubuntu 18.04 and it's successfully inserting action_traces, so this appears to be some CentOS 7 related issue.


wait: consider respecting also READY container count

Currently --wait flag only awaits Status == Running condition to be true. But in some cases, this still evaluates to true even no ready containers are there. For example,
NAME                  READY   STATUS    RESTARTS       AGE
foo-7d6d8499f5-4hjz9  0/1     Running   0              1m

Helm --wait does not respect the READY. I think the correct condition would be STATUS == Running and all containers is ready == 1/1.
Output of helm version: v3.11.1
Output of kubectl version: v1.23.7
Cloud Provider/Platform (AKS, GKE, Minikube etc.): -

Actually helm does wait for ready:
https://github.com/helm/helm/blob/e007c900ce5f2233c8583565d5ba1b0433b1a213/pkg/kube/ready.go#L274-L281
What you're probably seeing is the maxUnavailable being met.

Since I answered this and there's been no further questions, I'm going to go ahead and close this.


some fields in tauri.conf.json cannot be allowed while build

I followed the pre-requisite instruction but failed to build.
Here is the full error message.

Error `tauri.conf.json` error on `build`: Additional properties are not allowed ('devUrl', 'frontendDist' were unexpected
Error `tauri.conf.json` error: Additional properties are not allowed ('app', 'bundle', 'identifier', 'mainBinaryName', 'productName', 'version' were unexpected)

I did some search but only found those are not supported in tauri v2, but  I am using tauri v1 (tauri-cli 1.6.4 for specific)
Can you help me with this problem?

I did some search but only found those are not supported in tauri v2, but I am using tauri v1 (tauri-cli 1.6.4 for specific)

Main is on tauri 2, but it's kinda broken since I started a rewrite to support linux & other stuff, but life got in the way. Sorry about that (I considered not merging the semi-broken state, but figured no one is gonna use this. I was wrong 😅).
You should be able to run in by checking out 2dbfe91661b158955a0acd78650123af14a6af33 (which is pre-tauri2 migration).


Docs: (subjectively) nicer looking accordion example

the use of .btn-link currently just looks a bit odd, subjectively (as in v5 we now force the underline to always be there, while in v4 at least it was only on hover/focus).
.btn-light seems more fitting

Current look in v5:

After change:

I suggest more additional classes :)

Add a p-0 to the .card-header
Add a p-3 and rounded-0 to the button

Before:

After:

the example with more classes does look good. wonder though if it will obscure the point of the example?

There's a PR to make a dedicated accordion component at #30042. Maybe we explore that?

I think this PR with the suggestions @ysds made is good to land way before #30042

Added extra classes as suggested by @ysds

Also squeaked in a tiny clarification in the accessibility note there about what bootstrap doesn't support (the optional keyboard interactions)

any more thoughts on this?


Fade out the sound

The sound now sounds not natural, need to fade it out.

@chai00xxdd please whatsapp me the current ding dong sounds you are using and i will fade them out and send them back to u.


pkg/build: fix getBuilder for fuchsia.

This CL changes the supported vmType for fuchsia to be QEMU, instead of
GCE.
This was broken by commit b4e5a74e4, which changed the behavior of
getBuilder. It was part of a 6-commit PR trying to add support for
FreeBSD.
That commit has caused syzkaller to stop being able to build the fuchsia
image, getting the following error:
kernel build failed: unsupported image type fuchsia/amd64/qemu

Codecov Report

Merging #1055 into master will not change coverage.
The diff coverage is 0%.

@@           Coverage Diff           @@
##           master    #1055   +/-   ##
=======================================
  Coverage   67.25%   67.25%
=======================================
  Files         117      117
  Lines       20564    20564
=======================================
  Hits        13831    13831
  Misses       6081     6081
  Partials      652      652

Impacted Files
Coverage Δ

pkg/build/build.go
21.62% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2881fc2...c6ae65f. Read the comment docs.


Spurious OSC 11 when using tmux over Windows' SSH

Windows Terminal version
1.23.2771.0
Windows build number
10.0.26100.0
Other Software

OpenSSH_for_Windows_9.5p1, LibreSSL 3.8.2
tmux 3.5a

Steps to reproduce

Either

ssh to a Linux box
ssh to WSL (you can get the WSL2 VM IP by running e.g. ip a inside WSL)

Run tmux

Expected Behavior
Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
lhecker@server ~>

Actual Behavior
Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
lhecker@server ~> ]11;rgb:0000/0000/0000\

The issue does not occur when ssh'ing to localhost inside WSL.

I am so ready to blame win32-openssh already heh

This "issue" occurred only after I upgraded tmux from 3.4 to 3.5a


Is there an example sh script for running this repository?

Hello! Is an example sh script available to run this repository? It would help set up and test the code. Thank you!

We will provide you with the example command line for the experiment. Thank you.
python3 main.py --data HAPT --repr TSCP2 --gpu 0 --seed 0


QueryBuilder Issue: Incorrect query build when using joins with params

What steps will reproduce the problem?
DB-Schema:
Table-users
id, email,
Table-orders
id, user_id, status
** For test I created two users only, One user have 2 orders other have none.
$query = (new Query())
            ->select([
                'users.id',
                'users.email',
                new Expression('COUNT(orders.id)')
            ])
            ->from('users')
            ->join(
                'LEFT JOIN',
                'orders',
                'users.id=orders.user_id', [
                    'status' => 1
                ]
            )
            ->groupBy(['users.id', 'users.email'])
            ->where(['users.email' => 'rummykhan@localhost.com']);

        dd($query->all());

What is the expected result?
"id" => "1"
"email" =><EMAIL_ADDRESS>"COUNT(orders.id)" => "2"

What do you get instead?
SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens
The SQL being executed was: SELECT users.id, users.email, COUNT(orders.id) FROM users LEFT JOIN orders ON users.id=orders.user_id WHERE<EMAIL_ADDRESS>GROUP BY users.id, users.email
Additional info
While troubleshooting what I saw that Query Builder L-1182 It is not considering $params at all. whereas it should have because this QueryBuilder->buildCondition method is receiving one call from QueryBuilder->buildJoin as well. Due to this reason my parameter with the joins are not getting attached and hence the invalid numbers of tokens.

Q
A

Yii version
2.0.13?

PHP version
7.1

Operating system
Windows 10 & Ubuntu 16.04

Hi @samdark I actually fixed this on my local, should I push it for review?

If I get you right, correct query should be like this:
        $query = (new Query())
            ->select([
                'users.id',
                'users.email',
                new Expression('COUNT(orders.id)')
            ])
            ->from('users')
            ->join(
                'LEFT JOIN',
                'orders',
                'users.id=orders.user_id AND status = :status', // <- changed here
                ['status' => 1]
            )
            ->groupBy(['users.id', 'users.email'])
            ->where(['users.email' => 'rummykhan@localhost.com']);

Hey, I suspect you’re right, let me give it a try.. thanks

Yes it worked perfectly. Thanks @Ni-san and @samdark


Want to translate

Good morning!
Awsome product and works pretty good so far! Thank you, you just saved me a awfull lot of work.
But, we are a french college, I want to translate the display text.  Could you give me a pointer of where to look? Also after it is translated, are you interested in having it?
Thank you.

All translation strings are located in the config file... Please refer to the readme. And yes! I'd love to host a French translation file :)


Enhance documentation for hooking into events

I am using TabOverride with another plugin called autosize. TabOverride is the best plugin I've found so far for this issue, but I ran into a problem when implementing TO with autosize. The autosize plugin taps into the keypress event on the textarea, but the problem comes in that TO intercepts the keypress event when you press Enter and does event.preventDefault(), so autosize would not resize the textarea when I pressed Enter. I carefully read the documentation, then started digging into the JS code, but I simply could not understand how to hook into the function that TO calls when keypress is triggered. What I need to do is run a function when TO intercepts the keypress event that would tell the autosize plugin to update. That's it.
The section in the documentation titled Hooks is where I got completely lost. How do you use tabOverride.addExtension? And then addDelegatedListeners is used after addExtension?
I read over the JSDoc, but I'm still lost. I think the documentation could use some more specific examples of how to hook into TO events or functions.
Thank you for saving us a ton of time with this plugin!

Hi! There isn't a hook available for that function, but I can see it being useful, so maybe it should be added. If the only issue is with the keypress event, you can try just registering the event handler function for keydown instead of using tabOverride.set():
$('textarea').on('keydown', tabOverride.handlers.keydown);

If that doesn't work, then you could always try something like this:
$('textarea')
    .on('keydown', tabOverride.handlers.keydown)
    .on('keypress', function (e) {
        tabOverride.handlers.keypress.call(this, e);
        // do something else here
    });

To answer your question about hooks, there are currently only three "hooks" in the core library (set, addListeners, and removeListeners). Basically, you can register a callback function for when any of those events occur. It's essentially what you are looking for, except there is no available hook for the "keypress" event handler function.
The addExtension method is what is used to register the callback function. The addDelegatedListeners hook is available in the jQuery plugin and functions similarly to the addListeners hook in the core library. There are a few examples of using hooks in the Tab Override extensions:

https://github.com/wjbryant/taboverride.delay/blob/master/src/taboverride.delay.js#L73-L75
https://github.com/wjbryant/taboverride.escape/blob/master/src/taboverride.escape.js#L69-L71
https://github.com/wjbryant/taboverride.style/blob/master/src/taboverride.style.js#L413-L438

Thank you for your suggestions! Let me know if either of those examples works for you.


Fix openapi

This way it produces a 100% valid docs.json

@patrickelectric I've just resurrected this one


lib\Varien\File\Uploader.php:259

This fixed worked for me. Wasn't sure if you wanted to update this into your code.
https://magento.stackexchange.com/questions/105550/how-can-solve-error-magento-1-xx-on-php-7/105604#105604
File uploading will not work. Magento extends Mage_Core_Model_File_Uploader from above class, so we need to override this class and rewrite _validateFile() function replace below line
$params['object']->$params['method']($this->_file['tmp_name']);
with

$params['object']->{$params['method']}($this->_file['tmp_name']);

+1, had this issue too and the fix above worked for me. I'm leaving my fix in place for now but would be nice if this module could include this fix.
(When I clicked to upload it just gets stuck on "Uploading...")

This seems to be already fixed in the latest Magento 1.9.3.x, see the 2.1.0 Branch of this repository where the overwrite of the models and the specified class was removed from the repository because of the fixes. I just looked in my installation 'lib/Varien/File/Uploader.php' and it was already fixed there too.


[WIP][DNM] Stabilize CI

Working branch to verify and stabilize CI for cosmos.

Seems CI is using the deprecated dcos-launch

Default master branch has more failures than #792, closing this PR and using #792 to stabilize instead.


streaming rows blocks http up-to 20 seconds

hi guys, i am not sure if this is expected behaviour or a bug. if i use the streaming row approach it seems to block http from sending a 2nd request.
steps to reproduce

run the first code snippet
send a request to the http server (wait 1 second)
send a another request to the http server
observe the mysql-server process list (you will notice that the second connection does not appear until about 20 seconds have passed

here are some basic sample code you can reproduce the problem
var http = require('http');
const PORT=3000;
var server = http.createServer(handleRequest);
var Client = require('mariasql');

function handleRequest(request, res){
    var c = new Client();
    c.connect({
        host: "localhost",
        user: "root",
        password: "",
        db: "test"
    });
    var query = 'SELECT SLEEP(30)';
    var q = c.query(query);
    var rows = [];
    q.on('result', function(res) {
        res.on('data', function(row) {
            rows.push(row);
        }).on('end', function(info) {
        });
    }).on('end', function() {
        c.end();
    });
}
server.listen(PORT, function(){
    console.log("Server listening on: http://localhost:%s", PORT);
});

this behaviour can only be observed when using http. when using net package the behaviour does not occur
var net = require('net');
var Client = require('mariasql');
var server = net.createServer(function(socket) {
    var c = new Client();
    c.connect({
        host: "localhost",
        user: "root",
        password: "",
        db: "test"
    });
    var query = 'SELECT SLEEP(30)';
    var q = c.query(query);
    var rows = [];
    q.on('result', function(res) {
        res.on('data', function(row) {
            rows.push(row);
        }).on('end', function(info) {
        });
    }).on('end', function() {
        c.end();
    });
});

server.listen(1337, '<IP_ADDRESS>');

however if switched to query(q,callback) approach the behaviour does not occur.
var http = require('http');
const PORT=3000;
var server = http.createServer(handleRequest);
var Client = require('mariasql');

//We need a function which handles requests and send response
function handleRequest(request, res){
    var c = new Client();
    c.connect({
        host: "localhost",
        user: "root",
        password: "",
        db: "test"
    });
    var query = 'SELECT SLEEP(30)';
    c.query(query, function(err, rows) {
        if (err)
            throw err;
        res.end("hello " + util.inspect(rows));
    });
}
server.listen(PORT, function(){
});

once again this maybe behaviour by design when using the streaming row approach. i thought to write up here incase if anyone else ran into the same issue.

This problem is unrelated to the addon and is instead about the behavior of the client (browser in your case).
One easy way to tell that the process itself is not blocked is to add a simple interval timer that does something simple like console.log('hello world') every second and you'll see that "hello world" is printed while the streamed query is executing.
So now that it's been determined that the process (event loop) is not being blocked, you'll find that if you use multiple browsers (instead of multiple tabs within the same browser) in parallel, that you will see http requests get answered just fine (you can also hit the server with a bunch of curl requests too). Also, you will further find that if you append unique parts to the url for each http request (e.g. http://localhost/?123456, http://localhost/?7890, etc.), the browser will send them immediately.
I haven't done any digging to find out the reason behind this behavior, but now you know it's a client-side issue and not a server-side issue.

I should also note that in my testing, streaming vs. buffered queries made no difference.


server: support return simplified metrics response

What is changed and how it works?

Issue Number: ref #12355
What's Changed:

This PR is split from #12417 by only filter counter with 0 value and histogram with 0 samples. This will make it much easier to review and can be released in v6.1.
In my benchmark: By applying this simple logic, the /metrics response size can be reduced to about 35% with tpcc workload. When simplify-metrics is set to true, there is no much effect for the grafana charts, the only user-awareness change is some counter value show no-vaule instead of 0 value.
Related changes
Check List
Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression

Consumes more CPU
Consumes more MEM

Breaking backward compatibility

Release note
Please add a release note.

Please refer to [Release Notes Language Style Guide](https://pingcap.github.io/tidb-dev-guide/contribute-to-tidb/release-notes-style-guide.html) to write a quality release note.

If you don't think this PR needs a release note then fill it with None.
If this PR will be picked to release branch, then a release note is probably required.

@BusyJay @5kbpers PTAL thanks~

@BusyJay @innerr @5kbpers PTAL, thanks~

/build

Any side effects of it?

Any side effects of it?

The only side effect is that(as described in the PR desc): for counter-type charts, the displayed value will be non-value instead of 0 if simplify-metrics is set to true.

/test

/test

/test

/test

/merge


Gaps not working

I'm using this code from https://workspacer.org/config/#gaps,
#r "C:\Program Files\workspacer\plugins\workspacer.Gap\workspacer.Gap.dll"

using workspacer.Gap;

var gap = 20;
context.AddGap(
    new GapPluginConfig()
    {
        InnerGap = gap,
        OuterGap = gap / 2,
        Delta = gap / 2,
    }
);

but there are no gaps.

any solution?


Add code coverage

Also a Codecov badge.

Blocked by https://github.com/jruby/jruby/issues/6355, I guess.


Unhandled case of FlightGear supported a subset of the models

sitl_run.sh script does not handle a case of unsupported models.

Exactly - not sitl_run.sh but sitl_targets.cmake

Fixed in PR.


Research - Additional NC Programs

[ ] CCDF
[ ] TANF

Desired outcome of research:
How do we add these programs?
What steps do we need to do in collaboration with policyengine to actually implement?

initial research was completed here: https://app.zenhub.com/workspaces/myfriendben-652eb93b7dc40e0935103f4c/issues/gh/gary-community-ventures/benefits-calculator/1156

Here are sample PRs from Colorado:

CCDF: https://github.com/PolicyEngine/policyengine-us/pull/3002
TANF: Series of PRs https://github.com/PolicyEngine/policyengine-us/pulls?q=is%3Apr+is%3Aclosed+tanf+colorado

This is the PR for adding NC Tanf. Only checking on age and monthly income currently.


Add release with CI

Pull Request: Enhancements and CI Integration
Description
This PR introduces the following changes:

Ignored environment-specific files: Adjustments made to the .gitignore to exclude files specific to development environments.
CI Integration: Added Continuous Integration (CI) using GitHub Actions for building and delivering binaries.
Message package Improvements: Replaced the broken LocalWallet with PrivateKeySigner in the test suite.
Permissive CI Tests: Tests within the CI are currently permissive while further improvements are under development.
Workspace Enablement: The workspace remains enabled for the time being.

Known Issue
There is a test failure that needs to be addressed:
---- tests::test stdout ----
thread 'tests::test' panicked at message/src/lib.rs:398:9:
assertion `left == right` failed
  left: "{\"r\":\"0xfa6f7fd6825c953b355c8970fd2c9322162987bfb6898aa78f74f2be6bf8b10c\",\"s\":\"0x9a2018a7e31b623a91802147e6f8d5c658e17191e69f6663052efda71db72e2\",\"yParity\":\"0x1\"}"
 right: "{\"r\":\"0x1551fdb98e7afaedf0f626029e525ff7ca4b8e8bad81f2d7299979d76ed438b3\",\"s\":\"0x726fc8b549d8578868775530eee59c9c2b57923baef9df981280739f1b76289e\",\"yParity\":\"0x0\"}"
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

This starts anvil before cargo test?

This starts anvil before cargo test?

For sure! You can check out:
https://github.com/Calindra/paio/blob/89e45fe8f996d780fc288352eb7f70ba487603ad/.github/workflows/release.yml#L35-L36
https://github.com/foundry-rs/foundry-toolchain/tree/v1

Odd this test failure happens - what rustc are you on / cargo version? Didn't happen on mine

Apply this and tests should work again:
index f39a0e8..1ac70ee 100644
--- a/message/src/lib.rs
+++ b/message/src/lib.rs
@@ -396,7 +396,7 @@ mod tests {

         let signature = signer.sign_typed_data_sync(&v, &DOMAIN).unwrap();
         assert_eq!(
-            r#"{"r":"0xfa6f7fd6825c953b355c8970fd2c9322162987bfb6898aa78f74f2be6bf8b10c","s":"0x9a2018a7e31b623a91802147e6f8d5c658e17191e69f6663052efda71db72e2","yParity":"0x1"}"#,
+            r#"{"r":"0x1551fdb98e7afaedf0f626029e525ff7ca4b8e8bad81f2d7299979d76ed438b3","s":"0x726fc8b549d8578868775530eee59c9c2b57923baef9df981280739f1b76289e","yParity":"0x0"}"#,
             serde_json::to_string(&signature).unwrap()
         );
         let signed_tx = SignedTransaction {
@@ -407,7 +407,7 @@ mod tests {
         let ret = serde_json::to_string(&signed_tx).unwrap();

         assert_eq!(
-            r#"{"message":{"app":"0x0000000000000000000000000000000000000000","nonce":0,"max_gas_price":0,"data":"0x48656c6c6f2c20576f726c6421"},"signature":{"r":"0xfa6f7fd6825c953b355c8970fd2c9322162987bfb6898aa78f74f2be6bf8b10c","s":"0x9a2018a7e31b623a91802147e6f8d5c658e17191e69f6663052efda71db72e2","yParity":"0x1"}}"#,
+            r#"{"message":{"app":"0x0000000000000000000000000000000000000000","nonce":0,"max_gas_price":0,"data":"0x48656c6c6f2c20576f726c6421"},"signature":{"r":"0x1551fdb98e7afaedf0f626029e525ff7ca4b8e8bad81f2d7299979d76ed438b3","s":"0x726fc8b549d8578868775530eee59c9c2b57923baef9df981280739f1b76289e","yParity":"0x0"}}"#,
             ret
         );

@@ -429,7 +429,7 @@ mod tests {
         assert_eq!(signer, recovered);

         assert_eq!(
-            r#"{"name":"CartesiPaio","version":"0.0.1","chainId":"0x539","verifyingContract":"0x0000000000000000000000000000000000000000"}"#,
+            r#"{"name":"CartesiPaio","version":"0.0.1","chainId":"0xaa36a7","verifyingContract":"0x0000000000000000000000000000000000000000"}"#,
             serde_json::to_string(&DOMAIN).unwrap()
         );
     }```

Apply this and tests should work again:
index f39a0e8..1ac70ee 100644
--- a/message/src/lib.rs
+++ b/message/src/lib.rs
@@ -396,7 +396,7 @@ mod tests {

         let signature = signer.sign_typed_data_sync(&v, &DOMAIN).unwrap();
         assert_eq!(
-            r#"{"r":"0xfa6f7fd6825c953b355c8970fd2c9322162987bfb6898aa78f74f2be6bf8b10c","s":"0x9a2018a7e31b623a91802147e6f8d5c658e17191e69f6663052efda71db72e2","yParity":"0x1"}"#,
+            r#"{"r":"0x1551fdb98e7afaedf0f626029e525ff7ca4b8e8bad81f2d7299979d76ed438b3","s":"0x726fc8b549d8578868775530eee59c9c2b57923baef9df981280739f1b76289e","yParity":"0x0"}"#,
             serde_json::to_string(&signature).unwrap()
         );
         let signed_tx = SignedTransaction {
@@ -407,7 +407,7 @@ mod tests {
         let ret = serde_json::to_string(&signed_tx).unwrap();

         assert_eq!(
-            r#"{"message":{"app":"0x0000000000000000000000000000000000000000","nonce":0,"max_gas_price":0,"data":"0x48656c6c6f2c20576f726c6421"},"signature":{"r":"0xfa6f7fd6825c953b355c8970fd2c9322162987bfb6898aa78f74f2be6bf8b10c","s":"0x9a2018a7e31b623a91802147e6f8d5c658e17191e69f6663052efda71db72e2","yParity":"0x1"}}"#,
+            r#"{"message":{"app":"0x0000000000000000000000000000000000000000","nonce":0,"max_gas_price":0,"data":"0x48656c6c6f2c20576f726c6421"},"signature":{"r":"0x1551fdb98e7afaedf0f626029e525ff7ca4b8e8bad81f2d7299979d76ed438b3","s":"0x726fc8b549d8578868775530eee59c9c2b57923baef9df981280739f1b76289e","yParity":"0x0"}}"#,
             ret
         );

@@ -429,7 +429,7 @@ mod tests {
         assert_eq!(signer, recovered);

         assert_eq!(
-            r#"{"name":"CartesiPaio","version":"0.0.1","chainId":"0x539","verifyingContract":"0x0000000000000000000000000000000000000000"}"#,
+            r#"{"name":"CartesiPaio","version":"0.0.1","chainId":"0xaa36a7","verifyingContract":"0x0000000000000000000000000000000000000000"}"#,
             serde_json::to_string(&DOMAIN).unwrap()
         );
     }```

Ok, i'll apply and test here!

Done @stskeeps


can envoy support dubbo or not now?

if yes, there is some samples for this feature?

https://www.envoyproxy.io/docs/envoy/latest/configuration/listeners/network_filters/dubbo_proxy_filter.html?highlight=dubbo
https://www.envoyproxy.io/docs/envoy/latest/api-v2/config/filter/network/dubbo_proxy/v2alpha1/dubbo_proxy.proto.html?highlight=dubbo

@CodingSinger Envoy currently supports dubbo filter.

@CodingSinger Envoy currently supports dubbo filter.
have some samples about this?

Can i pull the image which support the dubbo filter by cmd docker pull envoyproxy/envoy-dev:latest ?
@zyfjeff


Provide response fields for checkout

Hi there,
when comparing to the Sumup SDK for Android the iOS implementation does not provide response fields (like result code, result message, etc.) for checkouts.
Would it be possible to include those fields, similar to the ones for Android?
The provider of the Sumup plugin for Flutter refuses to include those fields as long as they are only provided by the Android SDK implementation.
Thanks for considering my request.

Hello,
i would really like to support this request :)
i would also need this to be supported in iOS


socket leak

I used sole in my program to generate uuid , and found a sock leak in this line code;
https://github.com/r-lyeh-archived/sole/blob/master/sole.hpp#L604
It could be fixed by using this line code.
if (std::strcmp("lo", ifap->ifa_name) == 0)  { close(s); continue;}
Thanks for your work.

Good call! Ty!


Automate RH Community bundle creation

Add Makefile recipe to build RH community bundle
Automate RH community bundle release with Github Action

/hold
wait for https://github.com/medik8s/.github/pull/4

/lgtm

/retest

I think we don't need to wait for CI tests, as this change is not tested by any of them, we could just merge it


Building for the AUR

Hi, I'd like to build your app for the AUR, but I am unsure of how exactly to package this app.
so far this is what I've got
pkgname=Winds
pkgver=2.1.143
pkgrel=1
pkgdesc='Podcasts and RSS feeds in one app'
arch=(x86_64)
url='getstream.io/winds/'
license=BSD-3-Clause
depends=(redis mongodb)
makedepends=(yarn pm2)
source=("https://github.com/GetStream/${pkgname}/v${pkgver}.tar.gz")
sha512sums=('9caced2e2fb764b0c9cdb8667a6b51a21198d59f55466bdd5c8dbb516b4d6994ff5d6de5ddec955099a7c40ca8c98f10fd979b1af12e9f38a02bb3111596d2f3')

prepare() {
        yarn global add babel-cli
}

build() {
        cd api
        yarn
        cd ../app
        yarn
}

package() {

}

any idea how I would get this into a package? What files should be installed to the system and are there any dependencies I am missing? Not a fan of AppImages, would be nice if you guys supported flatpak, but ultimately I'd like this to run natively.

I'm not too familiar with AUR but are you opposed to installing with snap?

yaourt -S snapd
sudo systemctl enable --now snapd.socket
sudo snap install winds

I don't use systemd, IDK if I can even use snap tbh. I'd like to get this running natively so it can be posted to the AUR, once I figure this out I can also package it for Void Linux. I don't think snap is available on Void MUSL either, so snap won't do. Flatpak would work but not snap or AppImage.

Do you know how I can install this into a package? I've never seen anyone use yarn before for anything.

I'm really not sure. I asked around the team and did a bit of research. Doesn't look like electron builder (the package we use to build for linux) supports your package type. Very sorry.


Secure RSS feed for Activity

Is your feature request related to a problem? Please describe.
Knowing what is happening Activity-wise is important, but I'm not going to sit and eyeball the NetBird UI all day long, that's not realistic.
Describe the solution you'd like
It would be great to be able to subscribe to a private RSS feed and get Activity via it. A lot of software provides this.
Describe alternatives you've considered
Email notifications for specific event types like someone joining, but I'm more keen on RSS as emails can get lost, whereas with RSS, I'll get a notification if I can't access the feed to be able to check for new stuff, and likewise, I can auto-poll the feed and drag in notifications alongside all my other important notices in my existing feed reader. The world needs less email. :- )

Thank you for the feedback @HammyHavoc
Have you tried the event streaming feature?
https://docs.netbird.io/how-to/activity-event-streaming-to-siem-systems


Docker Hub Pyrsia Node Release Versioning Improvement to Distinguish Stable Release Version

Suggestion details
We need to have some kind of distinguishing tagging between stable & nightly releases in Docker Hub for Pyrsia Node.
For example 0.2.3-2805 is a stable release while 0.2.3-2806, 0.2.3-2819, 0.2.3-2825 are not.
Additional details
No response

Need to make sure the Docker build uses the correct version of the install script.  Right now the docker builds are using install.sh for both nightly and prod


使用url上传带预标注结果的数据 无法正常显示预标注结果（When using urls to upload data with premarked results, the premarked results cannot be displayed properly）

我使用zip文件方式上传 可以正常上传 并且显示出预标注结果，但是同一个zip文件 我使用url方式上传时却不能显示预标注结果了
When I upload a zip file, I can upload it normally and display the pre-annotated results. However, when I upload the same zip file using the url, I cannot display the pre-annotated results
zip方式上传
zip upload

url方式上传
url upload

我是用的Xtreme版本是v0.6.1
I am using Xtreme version v0.6.1

Please try the latest v0.8.1.

我重新部署了一套v0.8.1 依然存在这个问题
I redeployed it of v0.8.1 and still had this problem

而且我还发现v0.8.1的一个bug,由于他上传的文件结构从image改成了image_0，所以我修改了我的zip里面的image文件夹的名称，然后我上传之后发现 我的图片的名称不是正确的了。
Moreover, I also found a bug in v0.8.1, because the file structure he uploaded was changed from image to image_0, so I changed the name of the image folder in my zip, and then I found that the name of my image was not correct after uploading.
this is v0.8.1

这个是v0.6.1版本
this is v0.6.1

@wanghaowish v0.8.1已经修复该问题


Button href missng target property

Duplicates

[X] I have searched the existing issues

Latest version

[X] I have tested the latest version

Steps to reproduce 🕹
Hi, I found issue that really looks like mine : https://github.com/mui/material-ui/issues/22699
Steps:

Developing in typescript
Use <Button with href
add target="_blank" property to it
I get error ;
TS2769: No overload matches this call.
Overload 1 of 3, '(props: { href: string; } & ButtonOwnProps & Omit<ButtonBaseOwnProps, "classes"> & CommonProps & Omit<Omit<DetailedHTMLProps<AnchorHTMLAttributes, HTMLAnchorElement>, "ref"> & { ...; }, "color" | ... 24 more ... | "startIcon">): Element', gave the following error.
Type 'string | undefined' is not assignable to type 'string'.
Type 'undefined' is not assignable to type 'string'.

Trying to set strictNullChecks: true in tsconfig.json does not solve the problem
Current behavior 😯
typescript transpile error
Expected behavior 🤔
No response
Context 🔦
No response
Your environment 🌎
System:
OS: Linux 5.19 Ubuntu 22.04.3 LTS 22.04.3 LTS (Jammy Jellyfish)
Binaries:
Node: 16.18.0 - /usr/local/bin/node
Yarn: 1.22.19 - ~/.yarn/bin/yarn
npm: 9.6.4 - /usr/local/bin/npm
Browsers:
Chrome: 117.0.5938.132
npmPackages:
@emotion/react: ^11.11.1 => 11.11.1
@emotion/styled: ^11.11.0 => 11.11.0
@mui/base:  5.0.0-beta.17
@mui/core-downloads-tracker:  5.14.11
@mui/icons-material: ^5.14.11 => 5.14.11
@mui/material: ^5.14.11 => 5.14.11
@mui/private-theming:  5.14.11
@mui/styled-engine:  5.14.11
@mui/system:  5.14.11
@mui/types:  7.2.4
@mui/utils:  5.14.11
@types/react: ^18.2.12 => 18.2.12
react: ^18.2.0 => 18.2.0
react-dom: ^18.2.0 => 18.2.0
typescript: ^4.9.5 => 4.9.5

@kmoyse-apizee I can't reproduce this: https://codesandbox.io/s/https-github-com-mui-material-ui-issues-39287-whkjn3
Could you provide a repro? You can fork my sandbox or this template: https://mui.com/r/issue-template-latest

can you assign it to me ?

@legendary0911 Are you able to reproduce the issue?

Hi,
Thanks for your answer and the reproduction.
I forked it to reproduce my problem : https://codesandbox.io/s/https-github-com-mui-material-ui-issues-39287-forked-hc4nwk
If I copy paste the Demo.tsx code into a component under my vs-code I reproduce the issue. It does not on codesandbox...
It seems to be more related to the 'href' property rather than the 'target' itself.
import React, { useState } from "react";
import Button from "@mui/material/Button";

export default function TextButtons() {
  const [href, setHref] = useState<string>();
  return (
    <Button href={href} target="_blank" rel="noopener">
      Link
    </Button>
  );
}

The error :
No overload matches this call.
Overload 1 of 3, '(props: { href: string; } & ButtonOwnProps & Omit<ButtonBaseOwnProps, "classes"> & CommonProps & Omit<Omit<DetailedHTMLProps<AnchorHTMLAttributes, HTMLAnchorElement>, "ref"> & { ...; }, "href" | ... 24 more ... | "variant">): Element', gave the following error.
Type 'string | undefined' is not assignable to type 'string'.
Type 'undefined' is not assignable to type 'string'.
Seems happens because my href constant coulld be undefined...
I don't know why the issue shows on vs-code and not on  codesandbox ? my tsconfig has the same configuration...

@kmoyse-apizee The initial state in your snippet is undefined, you can fix it by initializing with an empty string:
-const [href, setHref] = useState<string>();
+const [href, setHref] = useState<string>('');

Yes but in my application logic the variable in the href can actually be undefined. To fix the problem I had to do :
<Button href={invitationShortLink ?? "#"} target="_blank" rel="noopener" disabled={!invitationShortLink}>Link</Button>
Which is fine because if not defined the buttons becomes disabled.
I still don't get why same code works on condesandbox and cannot be compiled by typescript in my environment.

Yes but in my application logic the variable in the href can actually be undefined. To fix the problem I had to do :
<Button href={href ?? "#"} target="_blank" rel="noopener" disabled={!href}>Link</Button>
Which is fine because if not defined the buttons becomes disabled.
I still don't get why same code works on condesandbox and cannot be compiled by typescript in my environment.

@kmoyse-apizee - I do not think that this code will be maintainable in the long time run. I'd advise you to use to create a conditional wrapper for this component.
I'm struggling to reproduce your issue locally and the sandbox is not working for me for some reason.

While not a solution, and I think this should still be addressed, I have a workaround for you @kmoyse-apizee.  If you conditionally render the property, it should avoid typescript issues.  Below is an example from my own project I'm working on, where I'm creating my own Button component and rendering out MUI's button below:
<MUIButton
    {...(href ? { component: 'a', href } : {})}
    sx={{ ...customSx }}
    variant={variant}
    startIcon={ItemIcon}
    onClick={onClick}
    disabled={disabled}
    type={type}
    fullWidth={fullWidth}
    {...(target ? { target } : {})}
  >

As you can see, I'm also doing the same thing for the href, with some additional renderings for the component.  I ran into the same issue you did and I was just looking for a way to avoid TS errors.


DELETE statements with a LIMIT do not work

Version
Other
What happened?
I tried to use LIMIT in a DELETE query but tool didn't not detect the query.
I searched about sqlite available clauses for DELETE. By default, SQLite doesn't support LIMIT clause on DELETE statement, but it can be available if sqlite is compiled with a custom option ([https://sqlite.org/compile.html#enable_update_delete_limit])
Tool might support this clause for UPDATE or DELETE statements, also it might generate a warning message about this usage instead of ignoring the query.
Relevant log output
# package db
error parsing queries: no queries contained in paths /home/<sqlc config path>/query.sql

Database schema
CREATE TABLE challenges (
        status TEXT
);

SQL queries
-- name: DeleteChallenge :exec
DELETE FROM challenges WHERE status = ? LIMIT 1;

Configuration
version: "2"
sql:
- schema: "schema.sql"
  queries: "query.sql"
  engine: "sqlite"
  gen:
    go:
      package: "db"
      out: "db"

Playground URL
https://play.sqlc.dev/p/031d55af21ebcebb19764888c1f2056714e6e48681930286d0841d2740ce20a6
What operating system are you using?
Linux
What database engines are you using?
No response
What type of code are you generating?
Go

I laid some groundwork for this in https://github.com/kyleconroy/sqlc/pull/2365, but the sqlite parser needs to be changed to accept it and pass it to the sqlc ast.

To clarify: after #2365 this does work in MySQL, but not yet in sqlite.


[internal] Use OTEL helper instead of tlscfg

Which problem is this PR solving?

Part of #4316

Description of the changes

How was this change tested?

Checklist

[ ] I have read https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
[ ] I have signed all commits
[ ] I have added unit tests for the new functionality
[ ] I have run lint and test steps successfully

for jaeger: make lint test
for jaeger-ui: yarn lint and yarn test

@yurishkuro --- FAIL: TestBuilderWithCollectors (0.00s)     collector_proxy_test.go:73:          	Error Trace:	/home/runner/work/jaeger/jaeger/cmd/agent/app/reporter/grpc/collector_proxy_test.go:73         	            				/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.4.linux-amd64/src/runtime/asm_amd64.s:1700         	Error:      	Received unexpected error:         	            	grpc: the server has been stopped         	Test:       	TestBuilderWithCollectors


Don't allow publishing to an exchange that is not open.

Prevents nasty issues like callbacks that are not fired when publisher confirmations are turned on, caused by broker-client acknowledgement sequencing mismatch.
Whenever a connection error occurs, if publisher confirmations are enabled, all currently unacked messages emit 'ack error' instead of not calling the callback altogether.
There also are two tests that don't pass for me on master, as well as on this branch.
test/test-connection-blocked.js:
assert.js:93
  throw new assert.AssertionError({
        ^
AssertionError: 1 == 0
    at process.<anonymous> (/Users/vlad/Projects/node-amqp/test/test-connection-blocked.js:58:12)
    at process.emit (events.js:95:17)
FAIL

test/test-flow.js:
/Users/vlad/Projects/node-amqp/node_modules/longjohn/dist/longjohn.js:195
        throw e;
              ^
Error: NOT_IMPLEMENTED - active=false
    at Connection._onMethod (/Users/vlad/Projects/node-amqp/lib/connection.js:518:15)
    at self.parser.onMethod (/Users/vlad/Projects/node-amqp/lib/connection.js:136:12)
    at AMQPParser._parseMethodFrame (/Users/vlad/Projects/node-amqp/lib/parser.js:366:10)
    at frameEnd (/Users/vlad/Projects/node-amqp/lib/parser.js:93:16)
    at frame (/Users/vlad/Projects/node-amqp/lib/parser.js:78:14)
    at header (/Users/vlad/Projects/node-amqp/lib/parser.js:64:14)
    at AMQPParser.execute (/Users/vlad/Projects/node-amqp/lib/parser.js:137:21)
    at Connection.<anonymous> (/Users/vlad/Projects/node-amqp/lib/connection.js:174:21)
    at Connection.emit (events.js:95:17)
---------------------------------------------
    at module.exports.run (/Users/vlad/Projects/node-amqp/test/harness.js:49:23)
    at Object.<anonymous> (/Users/vlad/Projects/node-amqp/test/test-flow.js:1:84)
    at Module._compile (module.js:456:26)
    at Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Module._load (module.js:312:12)
    at Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
FAIL

Might help with https://github.com/postwait/node-amqp/issues/373.