Details about cost / token usage etc

Hello!
This is an extremely intriguing project. Thank you for building it!
I was curious if there was any benchmarking on cost and latency when it comes to extraction with this tool? Aside from the mentioned problem with graphrag ("GraphRAG did not address our core problem: It’s primarily designed for static documents and doesn’t inherently handle temporal aspects of data."), graph rag is also slow, and costs a lot (atleast when building knowledge graphs for static documents). For incremental data, how does graphiti do when it comes to building knowledge graphs?
Thanks!

Thanks for the insightful question. Cost and latency are very important considerations to us, and it's something we spend a lot of time tracking and evaluating internally. In the future, we also plan to share analysis of these metrics publicly. You can keep an eye out on our blog post for future updates: https://blog.getzep.com.
Now to answer your questions. The RAG (and more broadly IR) domain has two fundamental components: data ingestion, and data retrieval. The two components have different latency requirements, and so we will discuss our approach to each below.
Ingestion:
During Ingestion, Graphiti not only extracts the nodes and edges from the episodes, it also deduplicates that data with information already in the graph, extracts timestamps, invalidates existing edges, and generates summaries for the entity nodes. We have many approaches to speed this up (as can be read in our blog), but ultimately we are using an LLM to perform these tasks so it is going to be slow compared to non-LLM-based ingestions techniques. We optimize our prompts for small models, which tend to be faster and cheaper. GPT-4o-mini is the default OpenAI model that we use and it costs $.15/1M input tokens and $.60/1M output tokens. That means that you could add 2000 pdf pages to Graphiti for about a dollar.
Another note on this, in contrast to many other GraphRAG approaches, we don't feed the entire Graph back into the LLM when extracting more data, so once a certain saturation is reached the time it takes to add new data data to the graph won't grow indefinitely with graph size.
Now, Graphiti allows for the option to build communities, which summarizes closely connected entities. This process is very similar to the process in Microsoft's GraphRAG paper and has a similar associated cost and latency. However, we note that unlike with Microsoft's GraphRAG, Graphiti is fully functional and very powerful even without using communities, so if this is a barrier we recommend trying Graphiti out on some test cases without using communities.
Retrieval:
Retrieval has much stricter cost and latency requirements as we understand that users have their own business logic that they have to execute once Graphiti returns results, and that could be long and expensive for agents applications.
Our retrieval is very fast and very cheap, especially when compared to Microsoft's GraphRAG. This is because we don't use text-to-text LLMs as part of our retrieval flow (although depending on the search method being employed we do use sentence embedder and cross-encoders). This means that while using the default Graphiti flow, it's not uncommon for >50% of the latency to come from OpenAI's sentence embedder. In this case you will be getting results back in a few hundred ms at negligible costs.
This contrasts Microsoft's GraphRAG where they use an LLM-based Map reduce style summarization flow over all communities on every query. Even if you choose to employ communities in Graphiti, we use other search techniques to identify which communities are most relevant to the query, and simply return those summaries to the user, rather than continually summarizing over the entire graph.

Thank you for the detailed answer! Let me digest and I'll post for more questions. And thank you for the blog links, super fascinating topic.


Unable to receive e-mail on IMAP mail server

Expected behavior
Mail account should connect to mail server and send/receive e-mail messages.
Actual behavior
When I try to configure the incoming IMAP server of my domain, I am told:
Cannot connect to server.
(Unable to connect)
If I click "Continue", setup continues as per normal... However afterwards, I am only able to send e-mail messages; receiving e-mail messages does not work.
No further error messages are shown.
E-mail settings have been tested as working just fine (sending and receiving) with other e-mail clients, all of which do not show any error messages.
Steps to reproduce

Attempt to configure e-mail account - IMAP over SSL.
Click "continue" at error message.
Attempt to send an e-mail, then reply back to that e-mail from my computer.

Environment
K-9 Mail version: 5.403
Android version: 7.0
Account type (IMAP, POP3, WebDAV/Exchange): IMAP

There's already #1058 to improve error reporting.
If you want help in setting up your account please use the mailing list. Make sure to include more details, like what mail server you're trying to connect to, which server settings you entered in K-9 Mail, and which server settings work in what other client.


[DEV-173] Refactored Node Health Monitoring

Release Checklist
Pre-Release

[ ] Check migrations

No database related infos (sqldb-xxx)
Impact on GS (new/removed columns)

[ ] Check for linter errors (in PR)
[ ] Test basic user operations

Login/logout
Registration
Update user data
Create/delete payment routes

Post-Release

Test basic user operations
Monitor application insights log

looks good! generally cleaner solution, I like it!


RG353VS（01252023） can't find and pair up bluetooth 5.2 device

After update to 01252023，RG353VS can't find and pair up bluetooth 5.2 device，such as TWS device（onplus budspro 2），I have try many times and turn on/off bluetooth，or repair tws device，can‘t be solved.but rollback to 01152023，it works fine.
also，bluetooth 5.0 TWS （SONY wf1000XM4）may works fine on 01252023.

There were no changes that impacted bluetooth pairing between 01152023 and 01252023.    Even if there was, I can not gaurantee that all bluetooth headphones will work since I don't own many of them.  Only advice I have  is if you don't see your bluetooth device show up after verifying you put your head phones in pairing mode, try the search more than once.

but I try to rollback 0152023，repair TWS，it's work correctly.And reupdate to 01252023，active exsting bluetooth，then the device name is disappear. once again，can't find and pair up. This is annoying and confusing


chore: add core-js@3 example in readme

Types of changes

[ ] Bug fix (a non-breaking change which fixes an issue)
[ ] New feature (a non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Description

Checklist:

[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly. (PR: #)
[ ] I have added tests to cover my changes (if not applicable, please state why)
[ ] All new and existing tests are passing.

Codecov Report

Merging #5633 into dev will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##             dev   #5633   +/-   ##
=====================================
  Coverage   95.7%   95.7%
=====================================
  Files         81      81
  Lines       2633    2633
  Branches     670     670
=====================================
  Hits        2520    2520
  Misses        95      95
  Partials      18      18

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9fb720d...5973867. Read the comment docs.

Codecov Report

Merging #5633 into dev will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##             dev   #5633   +/-   ##
=====================================
  Coverage   95.7%   95.7%
=====================================
  Files         81      81
  Lines       2633    2633
  Branches     670     670
=====================================
  Hits        2520    2520
  Misses        95      95
  Partials      18      18

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9fb720d...5973867. Read the comment docs.


fix npm start assets

they don't work (very likely caused by https://github.com/PrismarineJS/prismarine-web-client/pull/227)
cc @Saiv46

short term solution will be a revert of https://github.com/PrismarineJS/prismarine-web-client/commit/8c88ec638361e0fca6f3a770f3cbd4314b9f2433

ERROR in ./node_modules/jose/dist/browser/runtime/env.js 6:15-22
Module not found: Error: Can't resolve 'process/browser' in '/workspace/prismarine-web-client/node_modules/jose/dist/browser/runtime'
Did you mean 'browser.js'?
BREAKING CHANGE: The request 'process/browser' failed to resolve only because it was resolved as fully specified
(probably because the origin is strict EcmaScript Module, e. g. a module with javascript mimetype, a '*.mjs' file, or a '*.js' file where the package.json contains '"type": "module"').
The extension in the request is mandatory for it to be fully specified.
Add the extension to the request.
resolve 'process/browser' in '/workspace/prismarine-web-client/node_modules/jose/dist/browser/runtime'
  Parsed request is a module
  using description file: /workspace/prismarine-web-client/node_modules/jose/dist/browser/package.json (relative path: ./runtime)
    Field 'browser' doesn't contain a valid alias configuration
    resolve as module
      /workspace/prismarine-web-client/node_modules/jose/dist/browser/runtime/node_modules doesn't exist or is not a directory
      /workspace/prismarine-web-client/node_modules/jose/dist/browser/node_modules doesn't exist or is not a directory
      /workspace/prismarine-web-client/node_modules/jose/dist/node_modules doesn't exist or is not a directory
      /workspace/prismarine-web-client/node_modules/jose/node_modules doesn't exist or is not a directory
      /workspace/prismarine-web-client/node_modules/node_modules doesn't exist or is not a directory
      looking for modules in /workspace/prismarine-web-client/node_modules
        existing directory /workspace/prismarine-web-client/node_modules/process
          using description file: /workspace/prismarine-web-client/node_modules/process/package.json (relative path: .)
            using description file: /workspace/prismarine-web-client/node_modules/process/package.json (relative path: ./browser)
              Field 'browser' doesn't contain a valid alias configuration
              /workspace/prismarine-web-client/node_modules/process/browser doesn't exist
      /workspace/node_modules doesn't exist or is not a directory
      /node_modules doesn't exist or is not a directory
 @ ./node_modules/jose/dist/browser/runtime/asn1.js 1:0-57 44:14-33 44:39-47 47:13-21 104:14-33 104:39-47
 @ ./node_modules/jose/dist/browser/key/export.js 1:0-60 2:0-62 5:11-23 8:11-24
 @ ./node_modules/jose/dist/browser/index.js 21:0-69 21:0-69 21:0-69 21:0-69
 @ ./node_modules/prismarine-auth/src/TokenManagers/XboxTokenManager.js 8:22-37
 @ ./node_modules/prismarine-auth/src/MicrosoftAuthFlow.js 11:25-68
 @ ./node_modules/prismarine-auth/index.js 8:12-46
 @ ./node_modules/minecraft-protocol/src/client/microsoftAuth.js 2:37-63
 @ ./node_modules/minecraft-protocol/src/createClient.js 10:22-55
 @ ./node_modules/minecraft-protocol/src/index.js 6:21-46
 @ ./node_modules/mineflayer/lib/loader.js 1:11-40
 @ ./node_modules/mineflayer/index.js 7:0-43
 @ ./index.js 18:19-40

after reverting that commit
so we need to fix jose as well...


DBZ-2975 schema recovery

Unrelated test failure tracked via https://issues.redhat.com/browse/DBZ-4223.

Applied; thx, @mikekamornikov!


MultiLine className

When having a className, in a .tsx, that spans multiple lines, if the last class name, in any line, doesn't have a space after it, it doesn't get replaced during the obfuscate-build.
fixed it with "regex = new RegExp(([\\s"'\\\]|^)(${keyUse})(?=$|[\s"'\`]|\\n)`, 'g');" in utils.js

The obfuscator does not directly work with the files inside the src. It only obfuscates code under the .next folder, which is the NextJS build folder. Version 2.0.0 will be out this weekend and you will get better docs, demos and config. Please wait for it.

v.2.0.0 has been released.

Sorry. i did mean in the .next folder. see how gap-2 has a "\n" in the compiled js.

2.0 might have fixed this.

Why there are \n in the compiled JS? This is not supposed to happen. May I have the related source code?

maybe its just a mac thing. idk if it happening on linux.
This is just a normal .tsx file compiled with "next build"

ok, so setting line 151 in utils.js, in 2.0, to this regex = new RegExp(`([\\s"'\\\`]|^)(${keyUse})(?=$|[\\s"'\\\`]|\\\\n)`, 'g'); fixes it still. but idk if i will break any thing else. prob not.
without change:

with change:

I just updated the npm package. Check out the v.2.0.1


Add connect,idle timeout

Description
Add sample like this:

tcp connect timeout in DestinationRule setting.
http Idle Timeout in DestinationRule setting.

Prepare a non-existent destination and check the timeout.

No comments from me. The comments and suggestions from @akitok are great enough  😄


Added new streams to ResourceTracker so that they are disposed once n…

…o longer needed
Changed stream in ContentWriter from MemoryStream to VirtualStream

Excellent! Thanks!


Use order by on the HighWaterMarkChangeDetectionPolicy field with the search service API

Hello,
I'm currently work on solution of azure cognitive search with multiple cosmos db indexers.
I use .NET SDK to configure my indexes and indexers.
I set up the HighWaterMarkChangeDetectionPolicy on the _ts filed of my cosmos documents.
I set up the assumeOrderByHighWaterMarkColumn configuration to true like described in this section.
https://learn.microsoft.com/en-us/azure/search/search-howto-index-cosmosdb#incremental-indexing-and-custom-queries
But it remains an incomprehension on the finality. In this section give an example with a SQL like query but in my case I would like to use the search service API with a post request with the body:
{
	"count":true,
	"filter":"type eq 'collection'",
	"queryType":"full",
	"orderby":"_ts",
	"select":"id,collectionName,coverThumbnailURL,type,artists",
	"skip":0,
	"top":10
}

I specify the orderby property with _ts field, but this call gives me this error:
{
	"error": {
		"code": "",
		"message": "Invalid expression: Could not find a property named '_ts' on type 'search.document'.\r\nParameter name: $orderby"
	}
}

I didn't understand how to use order by on the HighWaterMarkChangeDetectionPolicy field with the search service API.
Thanks in advance for your help.
Regards.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: e1c2c641-bd55-a7aa-b747-bfb75671a7cc
Version Independent ID: 616f0123-6306-f0b3-806b-30c1fbfe702a
Content: Simple query syntax - Azure Cognitive Search
Content Source: articles/search/query-simple-syntax.md
Service: cognitive-search
GitHub Login: @BevLoh
Microsoft Alias: beloh

@julienFerreira
Thanks for your feedback! We will investigate and update as appropriate.

Hi @julienFerreira
The _ts field is not accepted by the search service API, according to the error notice you received. In Azure Cosmos Database, the last changed time of a document is tracked by the system property _ts field.
You must map the _ts field to a field in your search index before utilising the search service API to query data from Azure Cosmos DB. In order to accomplish this, you can create a field mapping in your indexer definition that associates the _ts field with a field in your search index.
Here is an example of how to create a field mapping for the _ts field in your indexer definition:
{
"name": "my-cosmosdb-indexer",
"dataSourceName": "my-cosmosdb-data-source",
"targetIndexName": "my-search-index",
"fieldMappings": [
{
"sourceFieldName": "_ts",
"targetFieldName": "lastModified",
"mappingFunction": {
"name": "unixTimestamp",
"parameters": {
"format": "s"
}
}
}
],
"outputFieldMappings": []
}
The _ts field in this illustration is mapped to a field called lastModified in the search database. The _ts value is transformed into a Unix timestamp format using the unixTimestamp mapping function so that it can be sorted and filtered.
When you've finished creating the field mapping, you can use the lastModified field in your search queries to sort and filter the data according to when the documents were last changed.
Here is an example of how to use the lastModified field in your search query:
{
"count": true,
"filter": "type eq 'collection'",
"queryType": "full",
"orderby": "lastModified desc",
"select": "id,collectionName,coverThumbnailURL,type,artists",
"skip": 0,
"top": 10
}
In this example, the lastModified field is used in the orderby clause to sort the data in descending order based on the last modified time of the documents.
If there are any further questions regarding the documentation, please tag me in your reply and we will be happy to continue the conversation.

@julienFerreira
We are going to close this thread if there are any further questions regarding the documentation, please tag me in your reply and we will be happy to continue the conversation.

I add a field in the description of index :
            [SimpleField(IsSortable = true)]
            public int lastModified { get; set; }

And I add a FieldMappings in my search indexer.
            SearchIndexer cosmosDbIndexer = new SearchIndexer(
                name: ENVIRONEMENT + "-" + collection_name + "-indexer",
                dataSourceName: cosmosDbDataSource.Name,
                targetIndexName: indexName)
            {
                Schedule = new IndexingSchedule(TimeSpan.FromHours(1)),
                Parameters = parameters,
                FieldMappings =
                {
                    new FieldMapping("_ts") { TargetFieldName = "lastModified" }
                }
            };

I finaly can use lastModified field in the body of request on seach API
{
	"count":true,
	"filter":"type eq 'collection'",
	"queryType":"full",
	"orderby":"lastModified",
	"select":"id,collectionName,coverThumbnailURL,type,artists, lastModified",
	"skip":0,
	"top":10
}

Thank you, regards.


{pv,mosh}: fix on aarch64-darwin

Motivation for this change
These fix some trivial incompatibilities with the Apple Silicon toolchain using patches that are already in Homebrew.
I also bumped the (tragically orphaned for >2 years) pv package while I was at it.
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[x] aarch64-darwin

[x] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 137605 at 97813ef8 run on aarch64-linux 1

3 packages built successfully:

mosh
pv
sanoid

1 suggestion:

warning: maintainers-missing
Package does not have a maintainer. Consider adding yourself?
Near pkgs/tools/misc/pv/default.nix:27:5:
   |
27 |     maintainers = with lib.maintainers; [ ];
   |     ^


Find an error in mnist_softmax.py

System information

OS Platform and Distribution (Mac os):
TensorFlow installed from (binary):
TensorFlow version (master):

Describe the problem
In mnist_softmax.py,there is a difference between github an the web of Tensorflow(https://www.tensorflow.org/versions/r0.12/tutorials/mnist/beginners/index.html),and the code on github has error in line 57,58.
Source code / logs
I replaced cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(y, y_)) with cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1])).And it works.

Could you link to the file with the mistake?


chore: Solve the quotient and remainder before applying constraints.

Description

Problem*

Resolves
Summary*

Documentation

[ ] This PR requires documentation updates when merged.

[ ] I will submit a noir-lang/docs PR.

[ ] I will request for and support Dev Rel's help in documenting this PR.

Additional Context

PR Checklist*

[ ] I have tested the changes locally.
[ ] I have formatted the changes with Prettier and/or cargo fmt on default settings.

The bigger issue here is the fact that we are liberally creating witnesses out of thin air. An abstraction over this like range_directive would help here

An abstraction over this like range_directive would help here

I can add this in this PR if you're not working on this already.

An abstraction over this like range_directive would help here

I can add this in this PR if you're not working on this already.

Sorry forgot to reply -- yeah this would be good, I have a few things in the backlog, so it would be quicker if you did it

Easiest to do it here. Reordering the opcodes is only a substantive change in the context of #1729 so we can just make this PR be primarily about the new abstraction layer.

@kevaundray if you give an ok in the comments here, I can approve + merge.

Looks like we should add the range constraints into the helper as well.

LGTM


Can you please add a top winners page

Can you please add a page with the top 20 winners (in terms of total value won over all games)

apikey = "YOURAPIKEY"; // put your API key here
                                    $user->domain = "http://csbox.pl"; // put your domain

                                    class user {

                                        public static $apikey;
                                        public static $domain;

                                        public function GetPlayerSummaries($steamid) {
                                            $response = file_get_contents('http://api.steampowered.com/ISteamUser/GetPlayerSummaries/v0002/?key=' . $this->apikey . '&steamids=' . $steamid);
                                            $json = json_decode($response);
                                            return $json->response->players[0];
                                        }

                                    }

                                    $mysql_server = "MYSQLADRESS";
                                    $mysql_admin = "MYSQLADMIN";
                                    $mysql_pass = "MYSQLPASSWORD";
                                    $mysql_db = "MYSQLDATABASE";
                                    mysql_connect($mysql_server, $mysql_admin, $mysql_pass) or die('Niestety nie udalo nam sie polaczyc z baza danych.');
                                    mysql_select_db($mysql_db) or die('Nie mozemy odnalesc podanej bazy danych.');
                                    $db = mysql_connect($mysql_server, $mysql_admin, $mysql_pass);
                                    mysql_select_db($mysql_db, $db);
                                    $getFarciarz = mysql_query("SELECT potPrice,userPutInPrice,winnerSteamId64 FROM history Order by potPrice DESC LIMIT 3");
                                    while ($row = mysql_fetch_array($getFarciarz)) {
                                        $potPrice = $row['potPrice'];
                                        $putIn = $row['userPutInPrice'];
                                        $chanse = $putIn / $potPrice * 100;
                                        $chanse = number_format($chanse, 2, '.', '');
                                        $potPrice = $potPrice / 100;
                                        ?>
                                    <img alt='avatar from steam user' src="<?php echo $user->GetPlayerSummaries($row['winnerSteamId64'])->avatar; ?>" style="width: 100px; height: 100px; border-radius: 100px;">

<?php echo $user->GetPlayerSummaries($row['winnerSteamId64'])->personaname; ?>

                                        Wygrał/a: $<?php echo $potPrice ?>

                                        Szansa: <?php echo $chanse ?>%

                                    <?php
                                }
                                ?>

@Xeroner
I think it would be better if the players stats are like
Games Played: x
Overall won: x
Highest win: x

@GreYzZ  So do it :) :+1:

unfortunately i can't

Am 04.12.2015 um 23:38 schrieb Jacek<EMAIL_ADDRESS>@GreYzZ So do it :)
—
Reply to this email directly or view it on GitHub.


Add CORAL for aligning C_1 and \hat{C}_2

Contributions
My contributions were to example/sl-prediction:

I added model selection (using GridSearchCV) to train_and_predict.py. We now automatically select C for the linear SVM and n_estimators (number of trees) for the random forest. Note that by default we do the grid search, so runtimes will be slower.
I added a normalize_embeddings step (and script) that occurs after HANDL and before featurization/training/prediction. There are currently two supported normalizations: "none" and "coral".

Testing
You can test using the toy data, which is currently set up to use CORAL normalization:
snakemake --configfile=configs/toy.json

Hi - no coral does not require unit variance and I don’t think we want that.   Also, sqrtm returns the “principal” square root which is the only symmetric positive definite square root.   I spent a bit of time learning about this and can update you on our call.


changing zagg target to service IP from skydns provided local dns name

Approved by @twiest in #2149


KYLIN-3632 Add configuration that can switch on/off preparedStatement cache

Add configuration that can switch on/off preparedStatement cache in kylin server

Can one of the admins verify this patch?

Codecov Report

Merging #290 into master will decrease coverage by <.01%.
The diff coverage is 0%.

@@             Coverage Diff              @@
##             master     #290      +/-   ##
============================================
- Coverage     21.29%   21.29%   -0.01%
+ Complexity     4444     4442       -2
============================================
  Files          1087     1087
  Lines         69983    69984       +1
  Branches      10109    10109
============================================
- Hits          14903    14902       -1
- Misses        53676    53677       +1
- Partials       1404     1405       +1

Impacted Files
Coverage Δ
Complexity Δ

...va/org/apache/kylin/rest/service/QueryService.java
0% <0%> (ø)
0 <0> (ø)
:arrow_down:

.../java/org/apache/kylin/common/KylinConfigBase.java
14.66% <0%> (-0.03%)
36 <0> (ø)

...he/kylin/dict/lookup/cache/RocksDBLookupTable.java
72.97% <0%> (-5.41%)
6% <0%> (-1%)

...a/org/apache/kylin/dict/Number2BytesConverter.java
81.74% <0%> (-0.8%)
17% <0%> (-1%)

...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java
70.21% <0%> (+0.6%)
7% <0%> (ø)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5f6007f...3fcb1d5. Read the comment docs.

Pull Request Test Coverage Report for Build 3769

0 of 2 (0.0%)  changed or added relevant lines in 2 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.0003%) to 23.304%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java
0
1
0.0%

server-base/src/main/java/org/apache/kylin/rest/service/QueryService.java
0
1
0.0%

Files with Coverage Reduction
New Missed Lines
%

core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java
1
81.08%

Totals

Change from base Build 3768:
-0.0003%

Covered Lines:
16309

Relevant Lines:
69984

💛  - Coveralls


flet==0.25.0.dev3526 apk "Android" error kiwisolver: dlopen failed

Duplicate Check

[X] I have searched the opened issues and there are no duplicates

Describe the bug
After building apk with kiwisolver package and running it on Android device simulator, import kiwisolver fails with the following error:
dlopen failed: library "libc++_shared.so" not found: needed by /data/data/com.flet.kiwisolver_app/files/flet/python_site_packages/kiwisolver/_cext.cpython-312.so in namespace classloader-namespace
Code sample
Code
import flet as ft

def main(page: ft.Page):
    page.title = "Flet libs test"

    def kiwisolver_tests(e):
        def test_basic():
            try:

                from kiwisolver import Solver, Variable

                page.add(ft.Text("kiwisolver: import - OK"))
                x1 = Variable("x1")
                x2 = Variable("x2")
                xm = Variable("xm")

                constraints = [x1 >= 0, x2 <= 100, x2 >= x1 + 10, xm == (x1 + x2) / 2]
                solver = Solver()
                for cn in constraints:
                    solver.addConstraint(cn)

                solver.addConstraint((x1 == 40) | "weak")
                solver.addEditVariable(xm, "strong")
                solver.suggestValue(xm, 60)
                solver.updateVariables()

                assert xm.value() == 60
                assert x1.value() == 40
                assert x2.value() == 80

                page.add(ft.Text("kiwisolver: test_basic - OK"))
            except Exception as e:
                page.add(
                    ft.Text(f"kiwisolver: test_basic - error: {e}", selectable=True)
                )

        test_basic()

    page.add(
        ft.SafeArea(
            ft.Column(
                [
                    ft.ElevatedButton(
                        "Run kiwisolver tests", on_click=kiwisolver_tests
                    ),
                ]
            )
        )
    )

ft.app(main)

pyproject.toml:
[project]

name = "kiwisolver_app"
version = "1.0.0"
description = ""
authors = [{name = "Flet Developer", email =<EMAIL_ADDRESS>dependencies = ["flet==0.25.0.dev3526", "kiwisolver"]

[tool.flet]
android.adaptive_icon_background = "#ffffff"

To reproduce
flet build apk
install apk on android device
click on "Run kiwisolver tests" button
Expected behavior
Expected to see import - ok and Basic test - ok Texts
Screenshots / Videos

Captures
[Upload media here]

Operating System
macOS
Operating system details
15
Flet version
0.25.0.dev3526
Regression
No, it isn't
Suggestions
No response
Logs
Logs
[Paste your logs here]

Additional details
No response

i try solution but not work
please you can create a google cloab link set all the dependencies in this colab and then any buddy upload the flet code and generate apk
i think this is best solution


Update backup.md

Added copy to clipboard ( {% include copy-clipboard.html %} ) to executable blocks of backup.md

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/8067b5bcd22e689d56611745e097cd950db0b555/

Closing in favor of https://github.com/cockroachdb/docs/pull/3365.


Update Boss Attacks

Did you mean to update the version field in modlinks? If not, Scarab won't know that an update is available

Thanks for the good catch!
Let me try to cancel this and I will retry tomorrow.


feature(nav-bar): Created the Navbar, missing the smooth scroll behavior.

What I did

Created the Navbar implementing all it's funcionalities, except the smooth scroll behavior and the links.

Logo on the left side of the screen, sections in the middle and button on the right side of the screen implemented, all with the hover transitions.

How to test

Open the terminal and go to the folder of this project

In terminal:
git checkout develop
git pull
git checkout <branch name>
git pull origin <branch name>
yarn install
yarn start

In your browser go to http://localhost:port

Check if the component looks just like the mockup in different screen sizes

Read the changed files section on GitHub

Test the things that you think are worth testing, even the ones that are not in this description

Boa, João, brabo demais! 🚀


Acrylic effect could be stronger (more transparent)

Files

Settings App

@R3voA3 We don't have any plans to change to opacity ourselves, but depending on how #3809 is implemented, you might be able to override the opacity with a custom theme.


Using Docling with costume layout and table recognition models

Is it possible to use the docling with costume models for layout and table recognition.
I would like to use the pipeline by replacing the existing models with my own models for layout and table recognition. I am wondering if the documentation has any example about using costume AI models.

The choice of the models is done at the Pipeline level. For example, the PDF pipeline (called StandardPdfPipeline) is defined in docling/pipeline/standard_pdf_pipeline.py.
You can make your own pipeline with different models, or simply extend with others. We have an example which extends the PDF pipeline with an image understanding model. See https://ds4sd.github.io/docling/examples/develop_picture_enrichment/.

@ALIYoussef You can of course also provide extension to docling via a PR.

I appreciate your support. I will give it a try and keep you posted!

Super, looking forward what you cook up. A few pointers:

Look into the table 1 of DocLayNet

Look into the dp-bench

@PeterStaar-IBM , have you benchmarked docling_v2 on the dp-bench? How good it is on tables?

the team is doing the evals in the next weeks

yes, same model but we have some updated weights, so I expect the numbers with the current weights to be better actually.

closing for now!

Hello, @PeterStaar-IBM , I see the the OTSL paper have some numbers (95.5) but they are on PubLayNet tables, which tables do dp-bench uses? Will you share the new results on dp-bench now with other tools including TableFormer+OTSL?

@PeterStaar-IBM , any update to this?


Add IranSans Font

Please add IranSans font in your package
Thanks :)

Hello @ar8118 and thank you for your suggestion;
But unfortunately I cannot add "Iran Sans" fonts since they're considered a proprietary software.
Feel free to suggest any other fonts to add or reopen this issue if something's missing!


TypeError: elm.focus is not a function

I get this error when i put Upload component inside FormItem. Probably  cannot find the focus bcs of Upload uses button. How can i resolve this?

Error source:

Have you tried to use the solution in #83 ?

@jsun969   Yes, I did, but it didn't work. I found the solution, tho. React Hook Form tries to focus on the error state. Since upload components have no focusable elements, it throws an "elm.focus is not a function" error. If you set shouldFocusError: false in your useForm, it fixes the problem, but you can't use the focus-on-error functionality anymore. Could you please add this problem to the .md file of this package so that everyone who wants to use uploads in their form can resolve this issue?

Yea we should add it in readme. Do you want to make a PR for it?

Yeah, sure.


Merge brooklyn.parameters in YAML files

Implements behaviour required to merge brooklyn.parameters sections of specifications in YAML files, allowing addition of new parameters to entities without having to specify the complete list again.

+1 for the change. I've personally felt the pain of not being able to merge parameters.
I think that we need to discuss with a broader audience how the merge happens - suggest mailing to the dev list.
What I'm concerned about is mostly how the parameters look in the UI when merged so the input of blueprint authors will be very valuable. Should override parameters be appended or prepended? Should they be merged with those coming from the class?

@neykov I have updated this with the suggested changes

@grkvlt Changes look good and can be merged as is. Still think it's a good idea to gather some input on the mailing list, or at least increase exposure of the upcoming change before merging. For example - are we prepending or appending parameters (matters for UI).

@neykov I implemented your suggestion (first add from spec if no spec parameters then merge with explicit parameters) and it also now places the current entities params at the start of the list. If there is no objection from the mailing list, we should be able to merge soon.

See message on<EMAIL_ADDRESS>for discussion.

Thanks @grkvlt. Merging.


[Investigation] Understand how the AR2 API maps to OpenAssetIO concepts.

What
Determine if there are any "semantic incompatibilities" between the USD AR2 API and OpenAssetIO.
Why
We need to ensure that we can implement an AR2 plugin that serves as an OpenAssetIO API host, allowing users to write a single plugin to support USD and OpenAssetIO asset resolution and publishing.
ACs

Wiki page identifying:

The purpose and function of AR2 entry points.
Mappings to OpenAssetIO equivalents.
Callouts of incompatible or missing functionality.

An Issue with notes for any improvements to OpenAssetIO API documentation.

Notes
Due to the "in progress" nature of the Python -> C++ migration, the methods of ManageInterface, the sum total of it's methods are split in Doxygen:

C++ ported methods (the basics)
Python to-port methods (the advanced stuff)

Final validation being done through an MVP https://github.com/OpenAssetIO/usdOpenAssetIOResolver/issues/8


Resolve TypeScript type error that occurs when defining a variable of type ChatCompletionMessageParam

Problem
Users cannot define a variable of type ChatCompletionMessageParam without causing a TypeScript type error, even when the variable has valid fields according to this type.
Steps to Reproduce
Copy and paste the following into a file:
import { ChatCompletionMessageParam, TokenJS } from './index'

// Create the Token.js client
const tokenjs = new TokenJS()

const main = async () => {
  const messages: ChatCompletionMessageParam = [
    {
      role: 'user',
      content: 'Hello!',
    },
  ]
  // Create a model response
  const completion = await tokenjs.chat.completions.create({
    // Specify the provider and model
    provider: 'openai',
    model: 'gpt-4o',
    // Define your message
    messages,
  })
  console.log(completion.choices[0])
}
main()

Expected Result
No type error should occur for the messages variable
Actual Result

Closed because this was a mistake on my part. It should be const messages: Array<ChatCompletionMessageParam> (note the Array


Stuff to add to the revised swarm cookbook

Stuff to consider adding or editing in the new revised Swarm cookbook

Imports: You're right, it's important to explicitly mention the necessary imports, especially for beginners. This could be added to the "Basic Usage" section (6.1):

from openai import OpenAI
from swarm import Swarm, Agent

client = OpenAI()
swarm = Swarm(client=client)

OpenAI API Key: While mentioned in section 5.3, it might be worth reiterating the importance of setting up the API key in the "Basic Usage" section.

Error Handling: The guide could benefit from a section on error handling, particularly for API-related errors and rate limiting.

Examples: While there are good code snippets throughout, it might be helpful to have a complete, runnable example that demonstrates a multi-agent interaction with handoffs.

Best Practices: A section on best practices for designing agent instructions and functions could be valuable.

Limitations: It might be useful to mention any known limitations of the Swarm framework.

Version Information: Including the version of Swarm that this guide is based on would be helpful.

Consistency: Ensure consistent formatting throughout the document, especially in code blocks.

Clarification on "Routines": The term "routines" is mentioned in the introduction but not extensively used later. It might be worth clarifying if this is interchangeable with "agents" or if it has a specific meaning.

Parallel Tool Execution: This feature is mentioned briefly in section 9.6, but given its importance, it might warrant more explanation or an example.

Context Variables: While explained well, an example of how these are passed and updated through a multi-agent interaction could be beneficial.

Testing: The testing section is good, but it could be expanded with more examples of different types of tests (unit tests, integration tests, etc.).

Troubleshooting: This section (11) is valuable, but could be expanded with more specific examples of common issues and their solutions.

API Reference: The API reference is comprehensive, but it might be helpful to include more details on the return types and possible exceptions for each method.

added "imports" as new section 6.1

re-introduced "routines" as new section 4.7


Cannot add / edit time sheets after contract end

It is not possible to add any new time sheets or edit existing time sheets of a contract if the current date if after the end of the contract.
It should be possible to add / edit all time sheets.

The issue is, that new/create on Timesheets is only allowed when user.current_contract.any? is true. We need a clever way to incorporate the corresponding contract into the ability check.

i removed the user.current_contract.any? check, since the TimeSheet model already validates that the associated contract is valid in the selected month.


A mac port

What feature do you want to get added on the P-Slice engine?
You had the time to add a linux port why not add a mac port it will make it easier for people to play!

I don't have a Mac to properly test
Here is Mac version: https://github.com/Psych-Slice/P-Slice/actions/runs/11420650132

Thanks!


[review only] APEXMALHAR-1988: Updating cassandra batch fetch logic to use Cassandra Paging feature

@bhupeshchawda and @chandnisingh  Please review.
Looks like the travis error is due to kafka module which is not relevant to these changes.

Failing due to kafka project test failures, closing and opening PR to re-run tests.

Changes look fine to me. Will merge it if there are no other comments.

I am okay with the changes. Will merge this in a day if there are no other comments.

@bhupeshchawda can you please merge the changes?


'target' option is heavy related on file system

I've found this issue when I was using gulp plugin gulp-minify-css to fix my urls. I'm getting wrong urls when target directory doesn't exist.
Here is the small example to reproduce the issue:
var CleanCss = require('clean-css')
var del      = require('del')
var mkdirp   = require('mkdirp')

var css = '.save { background: url(../images/save.png) }'

function createClean() {
    return new CleanCss({
        relativeTo: 'app/',
        target: 'dist/styles/'
    })
}

// remove dir
del.sync('dist')

var minified1 = createClean().minify(css)
console.log('-dir\n' + minified1.styles) // url is ../images/save.png

// create dir
mkdirp.sync('dist/styles')

var minified2 = createClean().minify(css) // url is ../../images/save.png
console.log('+dir\n' + minified2.styles)

I've discovered it when I was running build on my ci machine. It works fine locally because I'm not cleaning my working copy on every build, But ci makes clean checkout and target directory is missing when build is running. I thought it was a bug but then I found explicit code check in the source. So now I'm not sure if it's expected behavior or not.
Also I'm a bit of confused about relativeTo option. I thought that it's a part of website url but it seems it's also fs-related. It works fine for my 'app/' example, but when I set this property to '/' (and I expect it to rebase urls against http://host**/**) it resolves urls against fs root. It was very surprising for me.
OS: Windows 8 x64
clean-css version: 3.2.10

@rooks Well, it probably shouldn't assume the directory is in place as when using API you are in full control of the output - i.e. there's no way of setting target in CLI - maybe it should say sth like options.target && (!fs.existsSync(options.target) || fs.statSync(options.target).isDirectory()) ? options.target : path.dirname(options.target).
Re relativeTo here's are the rules - https://github.com/jakubpawlowicz/clean-css#how-to-rebase-relative-image-urls - need to give / a try as can't tell if it's a bug or not.

I've fixed target to handle cases when it points to a missing folder.
Re relativeTo option it will cause URL rebase to whatever you set it to, so if that's root of you project you should leave it empty or set it to and empty string. I think that's an expected behavior.


Adding nested tags for slice of strings

As pointed out in the issue #37 (faker slice of urls don't work), struct fields do not support faker tags for the underlying field data type in case of slices. Taking a subsection of the mentioned issue
type Entity struct {
    ...
    Uris []string        `faker:"slice_len=3, url"`  // makes all structure empty. I just need slice with 3 urls
    ...
}

In the above code taken from issue #37 the Uris slice will not take into account the url tag.
For the mentioned issue, I suggest an extension to the faker struct tags syntax to allow nested tags with following syntax:
type SomeType struct {
    ...
    SomeSliceOfStrings []string `faker:"<tags for slice>, [<tags for string>]"`
    ...
}
// Example
type Emails struct {
    EmailList []string `faker:"slice_len=3, [email]"`
}

The PR is still a work in progress but I would like to hear more and discuss on the idea of using nested tags.

Codecov Report
Attention: Patch coverage is 76.92308% with 6 lines in your changes are missing coverage. Please review.

Project coverage is 88.97%. Comparing base (6d0a05f) to head (7a75357).

Files
Patch %
Lines

faker.go
76.92%
4 Missing and 2 partials :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #49      +/-   ##
==========================================
- Coverage   89.01%   88.97%   -0.05%
==========================================
  Files          13       13
  Lines        1812     1832      +20
==========================================
+ Hits         1613     1630      +17
- Misses        142      144       +2
- Partials       57       58       +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Hi @HARDY8118
Thanks for the idea. I don't see any harm having the nested (yet).
Maybe can you add more usecases?
And wondering from user experience, will this okay? As you can see from the issue, people expect it differently 🤔

Hi @bxcodec
As I mentioned initially in the PR, the idea is still a work in progress, I created this PR to get some initial thoughts on the overall idea of having such feature. As you can notice from the code change, currently I have implemented the changes just for slice of strings just to get some comments on the idea and have a discussion about it (while we are discussing on this PR I am drafting some more changes and adding tests for more use cases).
From user experience standpoint I think this is am okayish approach as it clearly defines what the user expects in both the contexts, first being from the context of slice (properties like slice_len) and other being from the context of the underlying data which the slice consists of (like in this case language and type of string). This will help moving forward if there exists some overlapping properties providing clarity and control over how each part behaves.
However I would love to hear from you and others as well on what can be done better for improving the user experience.

@rubemlrm  any opinion from your side?


Debian 10 packages are incomplete

I've just tested the packages for Debian 10, and I'm sorry to have to report that they're not working correctly. The packages for Debian 9 appear to be fine.
The key issue seems to be that we're missing the fullstaq-rbenv and fullstaq-ruby-common packages from the Debian 10 repository. Here's the output on Debian 9:
deploy@steve-test-01:~$ sudo apt install fullstaq-ruby-2.6.5
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  fullstaq-rbenv fullstaq-ruby-common
The following NEW packages will be installed:
  fullstaq-rbenv fullstaq-ruby-2.6.5 fullstaq-ruby-common
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 6,804 kB of archives.
After this operation, 19.7 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 https://dl.bintray.com/fullstaq/fullstaq-ruby-apt debian-9/main amd64 fullstaq-rbenv all 1.1.2-1 [12.7 kB]
Get:2 https://dl.bintray.com/fullstaq/fullstaq-ruby-apt debian-9/main amd64 fullstaq-ruby-2.6.5 amd64 1-debian-9 [6,790 kB]
Get:3 https://dl.bintray.com/fullstaq/fullstaq-ruby-apt debian-9/main amd64 fullstaq-ruby-common all 1.0-0 [1,038 B]
Fetched 6,804 kB in 2s (2,951 kB/s)
Selecting previously unselected package fullstaq-rbenv.
(Reading database ... 30746 files and directories currently installed.)
Preparing to unpack .../fullstaq-rbenv_1.1.2-1_all.deb ...
Unpacking fullstaq-rbenv (1.1.2-1) ...
Selecting previously unselected package fullstaq-ruby-2.6.5.
Preparing to unpack .../fullstaq-ruby-2.6.5_1-debian-9_amd64.deb ...
Unpacking fullstaq-ruby-2.6.5 (1-debian-9) ...
Selecting previously unselected package fullstaq-ruby-common.
Preparing to unpack .../fullstaq-ruby-common_1.0-0_all.deb ...
Unpacking fullstaq-ruby-common (1.0-0) ...
Setting up fullstaq-rbenv (1.1.2-1) ...
Setting up fullstaq-ruby-2.6.5 (1-debian-9) ...
Setting up fullstaq-ruby-common (1.0-0) ...

And the output from Debian 10:
deploy@steve-test-01:~$ sudo apt install fullstaq-ruby-2.6.5
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libgdbm6
Recommended packages:
  fullstaq-ruby-common
The following NEW packages will be installed:
  fullstaq-ruby-2.6.5 libgdbm6
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 6,812 kB of archives.
After this operation, 20.2 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://mirrors.digitalocean.com/debian buster/main amd64 libgdbm6 amd64 1.18.1-4 [64.7 kB]
Get:2 https://dl.bintray.com/fullstaq/fullstaq-ruby-apt debian-10/main amd64 fullstaq-ruby-2.6.5 amd64 1-debian-10 [6,747 kB]
Fetched 6,812 kB in 0s (14.1 MB/s)
Selecting previously unselected package libgdbm6:amd64.
(Reading database ... 28665 files and directories currently installed.)
Preparing to unpack .../libgdbm6_1.18.1-4_amd64.deb ...
Unpacking libgdbm6:amd64 (1.18.1-4) ...
Selecting previously unselected package fullstaq-ruby-2.6.5.
Preparing to unpack .../fullstaq-ruby-2.6.5_1-debian-10_amd64.deb ...
Unpacking fullstaq-ruby-2.6.5 (1-debian-10) ...
Setting up libgdbm6:amd64 (1.18.1-4) ...
Setting up fullstaq-ruby-2.6.5 (1-debian-10) ...
Processing triggers for libc-bin (2.28-10) ...

As you can see, fullstaq-ruby-common is recommended, but is not installed, and fullstaq-rbenv is nowhere to be seen. If we try to install those packages separately:
deploy@steve-test-01:~$ sudo apt install fullstaq-rbenv fullstaq-ruby-common
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package fullstaq-ruby-common is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Unable to locate package fullstaq-rbenv
E: Package 'fullstaq-ruby-common' has no installation candidate

I know almost nothing about Debian packaging, but I'll take a look and see if I can understand what's going wrong here. Hopefully it's not too big a problem!

Thanks for reporting this. Looks like I made a mistake in the release process.
 I thought I didn't have to reupload them, but apparently I do, and I even have to explicitly specify that I want those packages to be included in the debian-10 repo.
So it should be fixed now.

Perfect! I can confirm that this fixes the problem. Saves me trying to build the packages again to figure out what went wrong (which I left running overnight).
It's likely we'll be switching to fullstaq-ruby from our current custom rbenv install; I'll amend our Ansible roles for next time we provision new machines.


update contributing.md

Hey ! With reference to issue #23
There is no loss of info in the file . I have just sorted things a bit and added few things such as

Squashing commits
Adding an upstream
Updating local and forked repo when changes are made in the upstream
Thank You
@DARK-art108 have a look!

Hi @DARK-art108 could you pls ass a GSSOC label and a level label to the same

@all-contributors please add @khareyash05  for Tutorials

Hi @DARK-art108  Could you pls add a level label and a GSSOC label to the PR as well because adding a level label to the PR is only being counted
Thanx!

Hi @DARK-art108 Could you pls add a level label and a GSSOC label to the PR as well because adding a level label to the PR is only being counted
Thanx!


upload image

From where can i get Person.obj file
File is produced from blender or by coding

Hi!,
Sorry for the late answer. About your question, the .obj files help us to save 3D data. In my case, I used SMPL model to get vertices and faces, then I export these two values in a .obj file with the library  PyWavefront. But, to use 3d-body-measurements library, the important values that you need are vertices and faces. SMPL model is a good option, you will need shape and pose params to enter on the SMPL model to generate vertices and faces, then, use 3d-body-measurements library to get body measurements.

Hello, Can you please share the code for SMPL model pose of given input image to generate A pose?

Hello, Can you please share the code for SMPL model pose of given input image to generate A pose?

Got the code?

Can you please share the code for SMPL model pose of given input image to generate A pose?


Page for Database Provider "PlanetScale"

to replace https://github.com/prisma/prisma/issues/7292
Additionally to the measures defined in https://github.com/prisma/docs/issues/2448

Thank you @Jan, for the draft content for the "Database Provider: PlanetScale" in Notion !
Is the thought to have this as a new type of doc, under Guides? Or what is your thought on where this material fits? To me, it seems perfect content for an entire guide on developing with PlanetScale...

We do not have similar content yet, and it is definitely "guide"y so the Guides section sounds like a good fit.
We talked about this in the past as "Using Prisma with x" or "Database Providers" (I actually started an issue for these to collect potential database services to document: https://github.com/prisma/docs/issues/2667).
Right now the priority should only be to get the PlanetScale information into the docs - if finding the right place takes more time, we can just put it under "Guides" with the title "Prisma + PlanetScale" and move it later.

I feel this Issue and Issue #2450 are the same... we ned a guide for PS.


TakeExactly and TakeAtMost

A thought inspired by https://github.com/JuliaFolds/Transducers.jl/issues/481:

Take(n)
Take n items from the input sequence.

This makes me think 1:3 |> Take(10) would error. To make the behavior more self-explanatory it could be TakeAtMost(10) for the permissive version and TakeExactly(10) for the error?

I'm not sure if this would be necessary. 1:3 |> Take(10) in the current implementation could still know about its length.
Not because of Take(10) alone but because the input reducible also knows its length. If there is no transducer that modifies the length depending on the input data, one should still be able to compute the length of the whole pipeline without materializing it; in this example the Take transducer could simply return min(length(1:3), 10).


Product Info Card Snippet

Description
Product Info Card With Slideout Add to Cart, Colour and Size Tabs
Screenshots
https://user-images.githubusercontent.com/64855541/147500533-2e357d1e-ed84-426e-850a-f3c55ceae73d.mp4
Additional information
Assets:
Image-

I will contribute to this one

I would like to take this one.

Assigned to @DGreat49251 👍


fix: NetworkTransform / NetworkAnimator Default Client Authority True

While server authoritative may be desirable, the reality that's borne out in the support activity in Discord makes it clear that users are not coming in the door with server authoritative character controllers.  They're either making up something simple or adapting something from a single player game or one of Unity's controllers, none of which are suitable for server authority.  When they try to get started, and slap on a Network Transform, and movement sync doesn't work as they expect, they're frustrated and have no idea where to look.  It's not until they get much more knowledgeable about Mirror and networking in general that they start looking at the server moving anything.
Additionally, before these components had this checkbox, they were client authoritative by default, and were in UNet too, so even those migrating from UNet are baffled as to what's broken.
We don't have a proper controller to offer them either.  The ones in the examples are client authoritative as well.  This change won't alter existing components already attached...only new usage of them.
So here it is, changing them back to client auth.  We tried it, and it's just not delivering on "ease of use" in this case.

I like this idea. While a server authoritative environment is the way to go for serious network development (and preventing abuse by malicious players), I can understand that if it's seen as "not working" then the immediate response to a newbie is to get flustered and upset, possibly going full tilt and lash out in the support channel.
I feel it's better to make it so that people who are newcomers and are following the videos or documentation can follow along. Avoids them getting pissy when we say "just tick that checkbox".
When the training wheels come off, then they can disable the "Client Authority" checkbox, and off we go.

sorry but closing this for three reasons:

Mirror should be server authoritative by default, or not. If we go down this road then we will have to make everything client authoritative by default (NetworkAnimator etc.). Which is a bad idea, since we want it to be cheat safe.

While this would reduce support load for people with client authoritative projects, it would increase support load for people with server authoritative projects that don't understand why the server doesn't sync it anymore

If someone adds this to a player owned object while in production (e.g. a pet), we want them to feel safe about it by default.


Report third party plugin configurations not being lazy

If a third-party plugin is not using a Provider based API for a configuration should be great to report it.

Thanks for the issue. The focus of this plugin is on analyzing plugin code that the user themself owns, not third-party plugins (see also https://github.com/autonomousapps/gradle-best-practices-plugin/issues/12). Furthermore, this suggestion raises a lot of questions about how to know which classes of a plugin to analyze. Looking at all tasks would be somewhat straightforward, but not necessarily custom DSL extensions. While they would probably appear in the plugin jar with the suffix _Decorated, it's not very clear which properties should fall under analysis and which shouldn't. I think this plugin's goal should be to catch the most egregious issues. Looking for all possible issues is kind of impossible, unless work on this somewhat generates income for me, or there is an active contributor community.

Note: _Decorated suffix is generated at runtime: org.gradle.internal.instantiation.generator.AsmBackedClassGenerator#decorateAndInject when we use Instantiator (i.e. ObjectFactory.newInstance or indirectly TaskContainer.register/ExtensionContainer.add/etc.).

For discovery of extensions a two-pass analysis could probably work: find all ExtensionContainer.add/create calls, and analyze class hierarchies of the passed-in classes. Noting it here in case someone picks this up to contribute.

Thanks @TWiStErRob. Maybe one approach would be to look for all public properties on classes discovered via your suggested algorithm, and then warn for those that are raw types (not Propertys). I could be willing to provide some implementation guidance on the class analysis if anyone found this to be an interesting project.


Python: No support for bodies in Http outbound request

Howdy, I am starting to play with Spin, looks very interesting! Is there a timeline to allow bodies in HTTP outbound requests?
Thanks!
What you were doing:
Trying to create my first WASM component :)
The component tries to do an HTTP Post request in Python
What did you expect:
Post is handled successfully
What happened:
Got error: NotImplementedError: todo: handle outgoing request bodies
Additional info:
Traced it to spin-python-sdk/src/spin_sdk/http
/init.py
if request.body is not None:         raise NotImplementedError("todo: handle outgoing request bodies")
Minimal example:
from spin_sdk import http
from spin_sdk.http import Request, Response, send

class IncomingHandler(http.IncomingHandler):
    def handle_request(self, request: Request) -> Response:
        body = b'{"test": "test"}'
        resp = send(Request("POST", "https://random-data-api.fermyon.app/animals/json", {}, body))

Spin version : spin 2.4.2 (340378e 2024-04-03)

Hey @camba1, thanks for reporting this. I moved it into the SDK repo. I will take a look into it and get back.

Thanks @karthik2804 !

Hey @camba1, the fix has been merged and a new release of the SDK is out. Please upgrade to spin-sdk v3.1.0 and give it a try. Let us know if you run into any issues.

Hi @karthik2804 ,
It upgraded to 3.1.0 and now it works as expected. Thank you!


fs: add initial syscall.Flock abstraction

excluive lock, only only actually implemented for linux.
it will compile and pretend to lock on windows

Summary by CodeRabbit

New Features

Introduced a new flock package for file locking functionality across different operating systems (Linux and Windows).
Added methods for locking, opening, and closing files to ensure safe access in concurrent environments.

Platform-Specific Enhancements

Implemented file locking for Linux systems.
Added a placeholder implementation for file locking on Windows systems.

@rabbitcodeai review again

The package is explicitly about flock but I wonder about splitting this into:

flock.go
syscall.go
syscall_linux.go
syscall_windows.go

refactored into x/fs/syscall and x/fs/flock, and Lock() function renamed to LockEx()

to shadow or not to shadow, that is always the question with darvaza.org/x

not to shadow. renamed to x/fs/fssyscall

<EMAIL_ADDRESS>tagged


Fixed #30899 -- Lazily compiled import time regular expressions.

Ticket 30899

Just import time re.compile are changed.
Tests are excluded.
django.utils.translation.trim_whitespace_re did not include because django.utils.translation already imported in django.utils.text.(Circular dependency)
django.core.handlers.wsgi._slashes_re did not include because it compile bytes re.compile(br'/+') and _lazy_re_compile does not handle bytes(add byte handling to _lazy_re_compile?)

django.utils.translation.trim_whitespace_re did not include because django.utils.translation already imported in django.utils.text.(Circular dependency)

Perhaps we should put the utility in django.utils.regex_helper instead, if that fixes it.

.django.core.handlers.wsgi._slashes_re did not include because it compile bytes re.compile(br'/+') and _lazy_re_compile does not handle bytes(add byte handling to _lazy_re_compile?)

Please add byte handling - seems like it would be straightforward!

@adamchainz Thanks for the comment.
I moved _lazy_re_compile to django.utils.regex_helper, and changed below line to use _lazy_re_compile : https://github.com/django/django/blob/6bbf9a20e2c5865e01b537e8cd34dfca06621a4a/django/utils/translation/init.py#L331
Also, Added bytes handling to _lazy_re_compile  and changed the below line to use _lazy_re_compile :
https://github.com/django/django/blob/6bbf9a20e2c5865e01b537e8cd34dfca06621a4a/django/core/handlers/wsgi.py#L12

I note from the ticket that this change is intended to reduce delays during import.
Could you show an example from before and after using -X importtime? Probably something like python -X importtime -m django runserver on a project that imports some of these regex-containing modules.

@pope1ni Thanks for the comment.
I ran the test on master against this patch on my local machine.
Here is the report for some changed modules:
import time: self [us]  | cumulative | imported package
------------------------|------------|-----------------------------
import time:      2861  | 3181       | django.utils.text (before)
import time:       491  | 824        | django.utils.text (after)
import time:      2392  |       2392 | django.utils.dateparse (before)
import time:      123   |       123  | django.utils.dateparse (after)
import time:      2743  |       8098 | django.utils.html (before)
import time:      448   |       3467 | django.utils.html (after)

Wow. That looks like a much bigger delta than I would have expected.
Did you run multiple iterations of before and after to rule out any I/O?
Otherwise, this is neat. :sunglasses:

Fantastic! "A few milliseconds ✖100 times a day ✖every django developer from now on" should come out pretty big :)

@hramezani Thanks, great job :rocket:


Creation of the model of DBItems

Background
Once the source files has been identified they need to be changed to a tree of the model representing all kind of DBItems
Feature
Provided the list of files and DB connection build up the model, resulting in a tree of `DBItems, returning the tree root.
It's up to your consideration what will be the function signature. If to accept a list of strings as file names. Or to get another structure like a Map[String, Seq[String] + rootPath where the keys of the map are the schemas and sequences the files within.
Also up to your consideration if related 'DBItem's are created recursively or explicitly by this function.
E,g, the DBUserItem which is the owner of a function will be returned as part of the DBFunctionFromSource creation.
Or explicitly called in this function because it known a function has an owner and list of users. (Probably the slightly preferred option is the recursive approach).
Proposed Solution
Sets for collections seem to be an effective way to do, to ensure same items are not there multiple times.
NB!
Database creation has to be done on a different connection than all the other steps. But perhaps we can skip the DB creation at the moment or do it explicitly before calling this function.
NB!
Issue of schema merging. (Let's discuss when we get to it)
Depends on #

Tables part still TODO.


Purification Chamber inputs

After updating to 8.1.3, had to spend over an hour trying to figure out why my assembly line halted. Could NOT the shards from the Chemical Injection Chamber to the Purification Chamber inputs---any more. Going back to 8.1.2 fixed the problem. Something happened in the update to 813 that nerfed the inputs. Just fyi.

At least its not just me going crazy, I've almost pulled my hair out trying to see what im doing wrong with the Input to Purification Chamber..

Thus is explained on the wiki. You have to pump mekanism oxygen gas into the purifier. May I recommend reading the wiki article on ore processing.

Check it out here.
http://wiki.aidancbrady.com/wiki/Ore_Processing

May I recommend you reading what I initially wrote. It worked fine---before the update. Since it worked fine before, that means the oxygen was already going to it. Since going back to 8.1.2 fixed the problem, that means there is something amiss in the update.

@hehe3301
Oxygen present
http://gyazo.com/adbdb92b56f3f7b6337e284c0f831828
Input Slots
http://gyazo.com/07bdf46494770c6554435d56ad3d1a76
Ender IO Item Conduit
http://gyazo.com/b3abc1bdd478fcef65f7c98ec3fadce3
http://gyazo.com/bb751835a7ae895042f3287cdf7f3cc9
http://gyazo.com/50330cd42a09f56faa481cfea9f83b3a
Chest with items
http://gyazo.com/dc41d63ed6da585764c21105b968ed3c
Also, Mekanism Copper dust has no smelting recipe anymore

Hopper above worked before update as well.  No method that I have tried (vanilla, Ender IO, Mekanism transporters as well as colored transporters) of automatically putting items in the purifier seem to work.  Performing 2x for now and manually feeding the few ores I need 3x processing for into the purifier.

Upgrading to a factory did fix the issue.

Getting the same issue here. Purification Chamber does not accept any automatic inputs - I tried vanilla hoppers, Mekanism logistical transporters, and having the machines adjacent to each other. As billm said, switching to a Basic Purification Factory worked, and this is only since the 8-1-3 update. I just finished rolling back to the 8-1-2 on my server and it worked normally, so it's definitely something with 8-1-3

Also getting a similar issue with the Chemical Injection Chamber, it's not accepting automatic inputs either. Switching to a Basic Injecting Factory also still works.

Same here. Oxygen fine, energy fine, putting things in manually, all is fine. Just not anything automatic.

Can verify the following:

The latest revision does not fix the issue (compiled it from scratch)
The factory version still works

@Elvarion, I can't reproduce the missing tin and copper smelting recipes with only Mekanism installed, there may be some kind of conflict.  Looking into the purification chamber issue now.

Fixed!

playing on bevo tech pack reborn latest version on the ATLauncher and the purification chamber does not accept any automatic inputs! Wondering if there is a fix for this yet?

This is a very old bug report. If you're running an v8 series build out of necessity, make sure you're on the latest v8 or you're out of luck.


fpga: ignore EBUSY error when reading afu_id

Device descovery can get EBUSY error when AFU is being programmed.
It causes plugin to crash.
This error should be ignored as this is valid use case.
This is harmless as afu will be rediscovered on the next run, when
reprogramming is done.

Codecov Report

Merging #38 into master will decrease coverage by 0.18%.
The diff coverage is 92.68%.

@@            Coverage Diff             @@
##           master      #38      +/-   ##
==========================================
- Coverage   79.77%   79.58%   -0.19%
==========================================
  Files           5        5
  Lines         613      622       +9
==========================================
+ Hits          489      495       +6
- Misses        115      117       +2
- Partials        9       10       +1

Impacted Files
Coverage Δ

cmd/fpga_plugin/devicecache/devicecache.go
96.1% <92.68%> (-1.83%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 87daf21...54fd4f6. Read the comment docs.

@rojkov not 100% sure in case of device plugins, but I think it is only really checked during scheduling or container creation. if device become unhealthy in the middle of execution, I don't remember any code that are dealing with that.


Haiku updates

@bors: r+

:pushpin: Commit 17c74f4 has been approved by alexcrichton

:hourglass: Testing commit 17c74f499d6dac53ee260738537bd10617426a97 with merge 64501712c9c5b99ce727cfe339a4278574429cd3...

:broken_heart: Test failed - status-travis

@bors: retry
On Tue, Sep 5, 2017 at 10:49 AM, bors<EMAIL_ADDRESS>wrote:

💔 Test failed - status-travis
https://travis-ci.org/rust-lang/libc/builds/272097754?utm_source=github_status&utm_medium=notification
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/rust-lang/libc/pull/756#issuecomment-327218655, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAD95NyzNe7e4DRUiyo3Ojyd-Mzg1ZEtks5sfW10gaJpZM4PLTyu
.

:hourglass: Testing commit 17c74f499d6dac53ee260738537bd10617426a97 with merge 0b7f8aa29897b34ba60d7140e3fa54ef27c1cfec...

I've been looking at the travis builds, the ones that fail all seem to have error: missing key: 'url' in the logs, is that why they're failing?

:broken_heart: Test failed - status-travis

:hourglass: Testing commit 17c74f499d6dac53ee260738537bd10617426a97 with merge 655c45e100817faec73b102da4280b1d23b278c7...

:broken_heart: Test failed - status-travis

@bors: retry

:hourglass: Testing commit 17c74f499d6dac53ee260738537bd10617426a97 with merge 6e62d718c65f1aa58a9d36f84549a188058eed94...

:broken_heart: Test failed - status-travis

@bors: retry
On Wed, Sep 6, 2017 at 12:51 AM, bors<EMAIL_ADDRESS>wrote:

💔 Test failed - status-travis
https://travis-ci.org/rust-lang/libc/builds/272345059?utm_source=github_status&utm_medium=notification
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/rust-lang/libc/pull/756#issuecomment-327382037, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAD95BwtHNrNo3PuebjmLfLAHvZVxjGdks5sfjLtgaJpZM4PLTyu
.

:hourglass: Testing commit 17c74f499d6dac53ee260738537bd10617426a97 with merge 8e4178768a6271b57d32c846c24daba4df150b35...

:broken_heart: Test failed - status-travis

@bors: retry
On Wed, Sep 6, 2017 at 10:49 AM, bors<EMAIL_ADDRESS>wrote:

💔 Test failed - status-travis
https://travis-ci.org/rust-lang/libc/builds/272509416?utm_source=github_status&utm_medium=notification
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/rust-lang/libc/pull/756#issuecomment-327527471, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAD95A0l79hXE9qjiYzO13O7H8lgZ90Xks5sfr71gaJpZM4PLTyu
.

:hourglass: Testing commit 17c74f499d6dac53ee260738537bd10617426a97 with merge bb985eea37deb61389c158dd837c7e29411f5aa6...

:broken_heart: Test failed - status-travis


[Bug]: Salesforce upsert transform: selection of single stream field is not working

Apache Hop version?
2.3.0
Java version?
jdk-<IP_ADDRESS>
Operating system
Windows
What happened?
The selection of a single stream field in the fields table (second column) is not working, as if there was no input stream but the "Edit mapping" and "Get fields" are working (showing correctly all the fields) and the "Show input fileds" function gives back all the fields in the stream.
Issue Priority
Priority: 2
Issue Component
Component: Hop Gui

Tested on Hop 2.5 and everything works. I'm going to close the issue.


add rounds table, insert initial rounds on game start

 [L] As many optional new rounds as you want


[SPARK-10004] [shuffle] Perform auth checks when clients read shuffle data.

To correctly isolate applications, when requests to read shuffle data
arrive at the shuffle service, proper authorization checks need to
be performed. This change makes sure that only the application that
created the shuffle data can read from it.
Such checks are only enabled when "spark.authenticate" is enabled,
otherwise there's no secure way to make sure that the client is really
who it says it is.

Merged build triggered.

Merged build started.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40936/
Test FAILed.

Test build #1618 has started for   PR 8218 at commit 17eb187.

Test build #1618 has finished for   PR 8218 at commit 17eb187.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds no public classes.

My understanding is that the network package is not a public library, so I added a mima exclude for the whole package to get the build going.

Merged build triggered.

Merged build started.

Test build #40971 has started for   PR 8218 at commit c68deab.

Test build #40971 has finished for   PR 8218 at commit c68deab.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40971/
Test FAILed.

retest this please

Merged build triggered.

Merged build started.

Test build #40976 has started for   PR 8218 at commit c68deab.

Test build #40976 has finished for   PR 8218 at commit c68deab.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40976/
Test PASSed.

Merged build finished. Test PASSed.

I'm not an expert on this part of the code, but it looks sane.  I just left a few minor comments

Merged build triggered.

Merged build started.

Test build #41049 has started for   PR 8218 at commit 292a299.

Merged build triggered.

Merged build started.

Test build #41057 has started for   PR 8218 at commit fadff27.

Test build #41049 has finished for   PR 8218 at commit 292a299.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41049/
Test FAILed.

Test build #41057 has finished for   PR 8218 at commit fadff27.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41057/
Test FAILed.

Merged build triggered.

Merged build started.

Test build #41156 has started for   PR 8218 at commit 3cc9321.

Test build #41156 has finished for   PR 8218 at commit 3cc9321.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41156/
Test FAILed.

Let's try pinging a couple of people. @rxin @aarondav

Merged build triggered.

Merged build started.

Test build #41329 has started for   PR 8218 at commit 4d19ed5.

Test build #41329 has finished for   PR 8218 at commit 4d19ed5.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41329/
Test PASSed.

Merged build triggered.

Merged build started.

Test build #41386 has started for   PR 8218 at commit 8153497.

Test build #41386 timed out     for PR 8218 at commit 8153497     after a configured wait of 175m.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41386/
Test FAILed.

Merged build finished. Test FAILed.

retest this please

Merged build triggered.

Merged build started.

Test build #41460 has started for   PR 8218 at commit 8153497.

Test build #41460 timed out     for PR 8218 at commit 8153497     after a configured wait of 175m.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41460/
Test FAILed.

retest this please

Merged build triggered.

Merged build started.

Test build #41481 has started for   PR 8218 at commit 8153497.

Test build #41481 has finished for   PR 8218 at commit 8153497.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41481/
Test FAILed.

Merged build triggered.

Merged build started.

Test build #41540 has started for   PR 8218 at commit d25c6cf.

@pwendell here's an example of more timeouts; last timed out build took 154m just for Java / Scala tests; the failed build above took 124m for the same tests.

Test build #41540 has finished for   PR 8218 at commit d25c6cf.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41540/
Test PASSed.

Ping?

ping @aarondav

Looks alright to me - would be good if @aarondav takes a look too.

Merged build triggered.

Merged build started.

Test build #41777 has started for   PR 8218 at commit b491ac7.

Test build #41777 has finished for   PR 8218 at commit b491ac7.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41777/
Test FAILed.

Merged build finished. Test FAILed.

retest this please

@aarondav do you have any comments to add? Otherwise I really want to merge this soon.

Merged build triggered.

Merged build started.

Test build #41835 has started for   PR 8218 at commit b491ac7.

Test build #41835 has finished for   PR 8218 at commit b491ac7.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41835/
Test FAILed.

I triggered the tests again. I think for this one, since it is so early in the release cycle for 1.6, we can also optimistically merge it for now and do post-hoc reviews, provided that @vanzin is not going to disappear :)

Test build #1707 has started for   PR 8218 at commit b491ac7.

Test build #1707 has finished for   PR 8218 at commit b491ac7.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

These pyspark tests fail on every other PR. I'll merge this since I haven't seen any more feedback.

@vanzin sorry I couldn't review this in time. While looking through the code I was wondering about two things:
(1) See my inline comment
(2) Looks like the app ID is relatively easy to spoof? Aren't they listed in the standalone Master / RM UI? Should we use something that's more like a secret, like what we do in YarnShuffleService? The only problem there is that in standalone mode I'm not sure if we can securely pass secrets from drivers to executors.

Hi @andrewor14,
This patch is not very useful outside YARN. On Standalone, all apps run as the same user, and authenticate using the same user and secret. So there's no way to prevent one app from reading another's shuffle files (either through the shuffle service or reading them directly from disk).
On YARN, each app authenticates itself using the app's ID as the user name, and a secure, per-app secret (see SecurityManager::generateSecretKey). Authentication is not based on the app simply saying who it is - the app needs to know that secret. After the SASL auth occurs, then we just do simple matching of what app the shuffle file belongs to and what app the connection was authenticated as. You can't spoof it.

I see, so it seems then that there are two kinds of authentication, one during the handshake where we use some secure secret (i.e. the one used passed from ExecutorRunnable to YarnShuffleService), and another for reading each shuffle block where we use the app ID.
For the latter, could we use the same shuffle secret instead of the app ID? That would require us to pass the secret to the executor JVM securely, which could be difficult. This patch is already a strict improvement as is, so I'm just wondering whether it's worth it to strengthen the security guarantees further.

it seems then that there are two kinds of authentication

No, there's one kind of authentication. I don't know what's this "handshake" you talk about. Whenever you open a connection to read shuffle blocks and spark.authenticate is enabled (no matter whether you are connecting to a block manager directly, or to an external shuffle service), you will perform SASL authentication and provide the user name (= app id) and secret for your application.
The secret is already distributed securely on YARN; it's stashed in the credentials held by the UserGroupInformation object used to start the application.

No, there's one kind of authentication. I don't know what's this "handshake" you talk about.

What? I'm talking about the shuffle secret the executor container needs when it initially registers with the shuffle service: https://github.com/apache/spark/blob/6cd98c1878a9c5c6475ed5974643021ab27862a7/yarn/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala#L116. The whole motivation for adding that in the first place is for app-level authentication during the registration.

Yes, and if you look at that code, that secret is the return value of SecurityManager.getSecretKey(), which on YARN is stored in the UserGroupInformation object. Here's the code to make it clear:
  val secretKey = SparkHadoopUtil.get.getSecretKeyFromUserCredentials(sparkSecretLookupKey)
  if (secretKey != null) {
    logDebug("in yarn mode, getting secret from credentials")
    return new Text(secretKey).toString
  } else {
    logDebug("getSecretKey: yarn mode, secret key from credentials is null")
  }
  val cookie = akka.util.Crypt.generateSecureCookie
  // if we generated the secret then we must be the first so lets set it so t
  // gets used by everyone else
  SparkHadoopUtil.get.addSecretKeyToUserCredentials(sparkSecretLookupKey, cookie)

Is the secret doesn't yet exist (i.e. before the app is submitted), then a new one is created and stashed in the user's credentials. If it already exists (e.g. for the AM and all executors), the it's used. Authentication works fine just as it was originally designed. This patch is not about authentication. It's about authorization.


tiny optimization to make the ol' git clone and cd work for bash and zsh

Almost not worth it, but you know, why not.  I tested a quick copy paste of the updated command and I think it's comfy in two universes now.

Thank you.


Fix RoslynPad.Avalonia on Windows

Fixes RoslynPad.Avalonia.sln on Windows (tested in Rider).
The problem is that on Windows the following is being added in Directory.Build.props:
<UseWPF Condition=" $(MSBuildProjectName.Contains('Windows')) ">true</UseWPF>
<TargetFramework Condition=" '$(UseWPF)' == 'true' ">$(LtsTargetFramework)-windows</TargetFramework>

RoslynPad.Runtime.csproj is configured with TargetFrameworks:
<TargetFrameworks>$(EarliestSupportedTargetFramework);netstandard2.0</TargetFrameworks>
<TargetFrameworks Condition=" '$(DefaultTargetFramework)' != '$(EarliestSupportedTargetFramework)' ">$(DefaultTargetFramework);$(TargetFrameworks)</TargetFrameworks>

It seems that <TargetFramework> has higher priority than <TargetFrameworks>. Because of this the project is always being compiled using .NET 8.0 instead of the EarliestSupportedTargetFramework.
This PR resets the value of <TargetFramework> in RoslynPad.Runtime.csproj, causing the build to succeed.

Thanks for the fix!


Show All Contributors table on the Readme

Issue
In order to satisfy the All Contributors specifications, we need to show a table with the contributors. The .all-contributorsrc file is present, but is not being shown on the README file.
Idea
Let's use the docs at All Contributors to add the table. We may want to use the bot or the CLI to do this.

I'd like to help with this.

Hi @novellac
I have raised PR for this https://github.com/VueVixens/docs/pull/93
Please have a look and let me know you are expecting anything else or you want some correction in this.

We actually have this on our Teams page. I think we're good


Add tests for new pages

Reference item # 5 from: https://docs.google.com/document/d/1SUOdjqZWGWBWxdTN8wQnIN2vjzuTuAP0sWWUTJ7CjT4/edit

Future tests for user registation. MVP test all pages


Problem with second!

Hi, I've already run the command export PYTHONPATH=$PYTHONPATH:/home/giorgio/second.pytorch/ to add second to the path but this error persist:
python create_data.py nuscenes_data_prep --data_path=NUSCENES_TEST_DATASET_ROOT --version="v1.0-mini" --max_sweeps=10 --dataset_name="NuscenesDataset"
Traceback (most recent call last):
File "create_data.py", line 7, in
import second.data.kitti_dataset as kitti_ds
ModuleNotFoundError: No module named 'second'
I have also a question about how to use nuscene mini?

@Giofist  did you able to uses v1.0-mini. If yes please help me how to create data from v1.0-mini.

@Giofist did you able to uses v1.0-mini. If yes please help me how to create data from v1.0-mini.

No I ended up using full dataset

by the way your probelm is not related to the dataset but to the fact that second is not found in your system. Fix taht before

Hi,
How did you resolve ModuleNotFoundError: No module named 'second'?
I am facing the same error. I have updated PYTHONPATH = path_to_repo/second.pytorch but still I get ModuleNotFoundError: No module named 'second'.
Can you help me here?


Store channels in array format

Preliminary support for #412.

Nice addition overall, consider this a 👍 as soon as my comment have been addressed (whether we decide to add a warning or not).

@maxpoulin64 added a message and fixed a bug along with it.

This is a nice addition, thanks!
One comment I have is: couldn't we wrap this behind a user setting, similar to the MOTD one? Not sure what name we could give it, but if it's a generic info checkbox, I would also put the "Enabled capabilities" there (if possible, of course).
Also (unrelated to this PR, but displayed by it), I noticed a line saying [403]	Freenode No such channel, are we trying to join the lobby somewhere when first connecting?

@astorije Didn't you merge #453 to fix that issue (#450)? 😄

@dgw: no, that one just prevents joining the server window.

@maxpoulin64 I think he meant [403] Freenode No such channel, which indeed fixed that.

@xPaw Right, I did mean the 403 numeric.

Sorry for the noise, commented on the wrong PR...
👍 and merging this, discussion in comment can continue post-merge.


Improve Type Safety of Monads methods (sequence, traverse) using AnyM

I.e. instead of
  public static <T,T1>  AnyM<Stream<T>> sequence(Class<?> c, List<T1> seq)

The method signature can become the more type safe
    public static <T1>  AnyM<Stream<T1>> sequence(List<AnyM<T1>> seq)

This will require List / Collection based conversion methods for currently supported types (Stream, Optional, CompletableFuture)

See https://github.com/aol/cyclops/pull/25

Available on Master now


Ensure stack semantics

Something like "pop => stack size -= 1"
Check and assert if run in some kind of debug-mode.
Can also highlight the stack (e.g. what we're going to pop) during runtime.

Not sure this is super useful. It might be nice but the amount of work feel too great for the gain. We also haven't done it. Will close if not updated.


Build broken for 32-bit OpenSSL 3.3.0 on AIX 7.1

Got the following build error while trying to build 32-bit OpenSSL 3.3.0 on AIX 7.1:
ssl/rio/poll_immediate.c: In function 'SSL_poll':
ssl/rio/poll_immediate.c:62:9: error: 'SSL_POLL_ITEM' has no member named 'rtnevents'
ssl/rio/poll_immediate.c:72:23: error: 'SSL_POLL_ITEM' has no member named 'reqevents'
ssl/rio/poll_immediate.c:88:21: error: 'SSL_POLL_ITEM' has no member named 'rtnevents'
ssl/rio/poll_immediate.c:88:21: error: 'SSL_POLL_ITEM' has no member named 'rtnevents'
ssl/rio/poll_immediate.c:100:17: error: 'SSL_POLL_ITEM' has no member named 'rtnevents'
ssl/rio/poll_immediate.c:100:17: error: 'SSL_POLL_ITEM' has no member named 'rtnevents'
ssl/rio/poll_immediate.c:107:13: error: 'SSL_POLL_ITEM' has no member named 'rtnevents'
ssl/rio/poll_immediate.c:107:13: error: 'SSL_POLL_ITEM' has no member named 'rtnevents'
ssl/rio/poll_immediate.c:112:13: error: 'SSL_POLL_ITEM' has no member named 'rtnevents'
ssl/rio/poll_immediate.c:112:13: error: 'SSL_POLL_ITEM' has no member named 'rtnevents'
ssl/rio/poll_immediate.c:115:13: error: 'SSL_POLL_ITEM' has no member named 'rtnevents'

The root cause is ‘events’ and ‘revents’ are defined as macro for 32 bit in /usr/include/sys/poll.h, so in ssl/rio/poll_immediate.c, any reference of ‘events’ and ‘revents’ are replaced by ‘reqevents’ and ‘rtnevents’ respectively, which is unexpected.
Ref: Snippet of usr/include/sys/poll.h
…
#ifdef  __64BIT__
/* pollfd.fd is always a 32 bit entity (XOPEN) */
struct  pollfd
{
        int     fd;                     /* file descriptor or file ptr  */
        short   events;                 /* requested events             */
        short   revents;                /* returned events              */

};
#else /* __64BIT__ */
struct  pollfd
{
        long    fd;                     /* file descriptor or file ptr  */
        ushort  reqevents;              /* requested events             */
        ushort  rtnevents;              /* returned events              */
};
#define events  reqevents               /* SVR3,4 pollfd member name    */
#define revents rtnevents               /* SVR3,4 pollfd member name    */
#endif /* __64BIT__ */
…

That seems....rather unkind for AIX to globally define such a generic name like this.  They've basically restricted any using code from using the term events/revents here.
It should be fairly easy to fix, but IBM shouldn't do that.


Correctly indicate data source URL for VS

This is on purpose, since VS provides information on their website, but delivers their XLS by email which then gets uploaded to GitHub. VS asked us to provide their website as source.