zeek.as crash for "table" type name

Reported by @simeonmiteff , using zeek.as() with just "table" as type_name argument causes a TypeError and sometimes crash.
$ cat examples/z.js
zeek.on('zeek_init', () => {
        let s = zeek.as('table', ["<IP_ADDRESS>/24"]);
        console.log(s);
});

# RelWithDebInfo build
$ zeek examples/z.js
/home/awelzel/corelight-oss/zeekjs/examples/z.js:2
        let s = zeek.as('table', ["<IP_ADDRESS>/24"]);
                     ^

TypeError: Unsupported index size<PHONE_NUMBER>8087 for type
    at /home/awelzel/corelight-oss/zeekjs/examples/z.js:2:15

Node.js v21.2.0

# ASAN build:
$ zeek examples/z.js
AddressSanitizer:DEADLYSIGNAL
=================================================================
==3608162==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000050 (pc 0x7f4e9a4c046c bp 0x7ffc82948f80 sp 0x7ffc82948f80 T0)
==3608162==The signal is caused by a READ memory access.
==3608162==Hint: address points to the zero page.
    #0 0x7f4e9a4c046c in std::vector<zeek::IntrusivePtr<zeek::Type>, std::allocator<zeek::IntrusivePtr<zeek::Type> > >::size() const /usr/include/c++/11/bits/stl_vector.h:919
    #1 0x7f4e9a4ba7fe in ZeekValWrapper::ToZeekVal(v8::Local<v8::Value>, zeek::IntrusivePtr<zeek::Type> const&) /home/awelzel/corelight-oss/zeekjs/src/Types.cc:723
    #2 0x7f4e9a49dcf9 in plugin::Nodejs::Instance::ZeekAs(v8::Local<v8::String>, v8::Local<v8::Value>) /home/awelzel/corelight-oss/zeekjs/src/Nodejs.cc:498
    #3 0x7f4e9a49c491 in plugin::Nodejs::Instance::ZeekAsCallback(v8::FunctionCallbackInfo<v8::Value> const&) /home/awelzel/corelight-oss/zeekjs/src/Nodejs.cc:277
    #4 0x7f4e9330cadc in Builtins_CallApiCallbackGeneric (/opt/node-21.2.0/lib/libnode.so.120+0x110cadc)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /usr/include/c++/11/bits/stl_vector.h:919 in std::vector<zeek::IntrusivePtr<zeek::Type>, std::allocator<zeek::IntrusivePtr<zeek::Type> > >::size() const
==3608162==ABORTING
Aborted (core dumped)

Thanks @awelzel.
I also found that 'let x = zeek.as("vector", [1,2,3]);' segfaults when zeek::Type::Tag() is called on a null pointer.

A related problem (please let me know if I should open a separate issue for this): zeek.as() won't accept the type name in set[count] syntax - it has to be count_set (and so on for other index types).


Fix bug in deregisterPreconfer

I think the original code won't work as expected if the preconfer to be deregistered is the last preconfer.

Agreed. This was an issue. Thanks for spotting!


SelectField has no defaultValue

This is very annoying.
You can't set the value by props, and intercept the new value selected to set it in the state...
Can you fix that, all the other form field have this "defaultValue" props.

SelectField seems to can switch between uncontroled and controled value without throwing an error.
My bad sorry.

@Kornflexx I am not sure I understand what you mean. I have the same problem, I need to use a SelectField as uncontrolled component, and  SelectFields should support the defaultValue attribute, according to React's docs, which it currently does not. Unless I am mistaken, could you re-open this issue ?

Google brought me here.. I also want to use an uncontrolled SelectField.
Any chances?

Can anyone please tell me how to use defaultValue on an uncontrolled SelectField?

@Zregnateews if you are using "next" version you can do the following:
 <Select native defaultValue={30}}>             <option value="" />             <option value={10}>Ten</option>             <option value={20}>Twenty</option>             <option value={30}>Thirty</option>           </Select>
https://material-ui-next.com/demos/selects/
Hope it helps you!.

Ran into this issue myself i will just leave this here for the next dev who uns into this issue

pass in a displayEmpty prop to the Select component , this prop is a boolean so it must be set to true.
pass in a renderValue prop which is a function  that returns the value that should be displayed value should be displayed.

ie :
<Select       displayEmpty       defaultValue="value"       renderValue={ () => dashboardState.month}                     onChange={e => handleChange(e.target.value) } >                  { ...options }       </Select>


C-style comments are stripped from Markdown pages

markdown files with /* C-style block comments */ are not properly handled. the comments are inserted into the page file as they are, so they are not rendered. this problem became apparent when I switched from using brace-escape around code{}, but it applies everywhere, not just in code{}. it used to be possible to escape comments, but I removed that some time ago because it seemed silly. now though it would be useful.

what i did fixed it for blocks with formatted text but not ones like code{}... still need to think of something else

now I've fixed it by allowing the comments to be escaped again


MySQL streaming protocol support

The current MySQL protocol only uses the non-streaming version of VTGate (Execute). We also need to support the streaming version (StreamExecute).
Why not stream always? Execute gives you some benefits that Stream does not:

It can have shorter timeouts and limit rowcount. These two features protect the databases from accidental abuse.
Because Execute does everything in memory, it may be able to fulfill some queries that StreamExecute may not.

So, it's a good idea to support both options. There are multiple ways this can be supported:

SET STREAMING=1 // or 0: This option allows you to initialize a connection for streaming at the beginning, or switch back and forth as needed. The main advantage is that this can be deployed at the connection level, and the app doesn't have to change much.
Extend USE syntax: Something like USE keyspace:shard@rdonly+streaming. One problem with this approach is that the USE statement gets too complicated. The other problem is that streaming is a protocol requests. So, trying to overload USE for it feels incorrect. Upside is that you don't have to make the extra SET STREAMING request.
Automatically use streaming if it's @rdonly: The idea is that rdonly should be the tablet type you should be streaming from. The master and replica types are for OLTP workloads. This feels very inflexible. Just suggesting this for completeness.
Query hints like SELECT /* streaming */ ...: The upside is that a query is usually written for streaming (or not). So, this makes sense in some way. Downside is that all call-sites will have to be hunted down and changed. This gets even harder if there's an ORM layer in the middle.

Feel free to share your thoughts. If there are no strong opinions, I'm likely to go with option 1.

if i can vote for two options, i'd vote for 1 and 4. i'm not sure which of those i prefer if i can only vote for one.

Fifth option: provide a separate port for streaming. However, VTGate already has three ports now: http, grpc and mysql.

I think we should make it compatible with the useCursorFetch option of the oracle/mysql jdbc -- in sure other MySQL clients have a similar setting. Not totally sure how that translates into protocol options, will need to check the code.

In JDBC specification, if we set fetchSize(int) on a statement object then
any select query through that object should stream results.
So, I think option 1 should be there to enable stream at any time.
On Jun 17, 2017 04:08, "Alex Charis"<EMAIL_ADDRESS>wrote:

if i can vote for two options, i'd vote for 1 and 4. i'm not sure which of
those i prefer if i can only vote for one.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/youtube/vitess/issues/2933#issuecomment-309151910,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALSUiiF-zEoRqVl1cFkuy66TusBYjnkrks5sEwPlgaJpZM4N7_gw
.

Note that this is only for the MySQL protocol. For grpc, the API is already split between Execute and StreamExecute. This is something we have to independently revisit. Maybe think about consolidating the two.
As for the wire protocol for MySQL, it always streams. So, the differences are the functional parts, which could be renamed to OLTP vs OLAP:

OLTP allows DMLs, has aggressive timeouts and limits rowcounts.
OLAP: Only selects, generous timeouts and indefinitely large result sets.

Short term

Add a field to ExecuteOptions called workload. It can be oltp or olap. The reason why it should be in ExecuteOptions is because the field will also be sent to vttablet. Justification below.
This field can be set through the SET WORKLOAD='OLTP' SQL command. For now, it will only be used by the mysql protocol. The difference in behavior will be as described in my previous post.

Longer term

We can change grpc to always use StreamExecute and deprecate Execute. StreamExecute is more memory efficient than Execute. The current downside is that it sends at least two packets per request, which can be easily fixed. Once this is done, the grpc clients can get the older Execute functionality by calling StreamExecute with workload set to oltp.
ExecuteBatch can also be taken over by StreamExecute by making it accept semi-colon separated statements and return multiple results. This will fulfill most existing use cases, and will also extend gracefully for the mysql protocol. We may still have to build special functions for the parallel execution use case.
The workload flag will need to be sent to vttablet because it also needs to behave differently when its StreamExecute is called. This is why the flag should be in ExecuteOptions.


Added Mish(), a skeleton of #362

Coverage decreased (-0.04%) to 63.61% when pulling 9de7b670bdc7450f508f8df8b461828928e8713b on feat-362 into a0c8e0bc68b53c677f0c988527083714484ebf84 on master.


unrecognized function

Hi!
Do you mean that "local" is not highlighted?

The Analyzer is disabled, so it won't update code info.
You need to enable it (you probably have wrong Java set, check out Analyzer settings)

WHere can i set the Java? and also how can i create a addon for that?

Java seems to be set correctly.
Please join our Discord server and report it there, because the Analyzer is just an addon, so it's not the best place for issues with it.


systemd-stage-1: Fix hibernate-resume with systemd v255

Description of changes
In 254, systemd-hibernate-resume.service was generated by systemd-hibernate-resume-generator. Since 255, the generator just enables a unit shipped with systemd. So we have to add systemd-hibernate-resume.service to the upstream units for initrd
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

@ofborg test hibernate-systemd-stage-1


Just a question

Hi, found your project my mistake.
I am new to nexus.If I may, I don't understand the benefits of your plugin.
I can define mutations with the nexus api - therefore how does your plugin help?
Thank you

Hi @orefalo
Following Designing GraphQL Mutations

The main points to consider when designing your GraphQL mutations are:

...
Input object. Use a single, required, unique, input object type as an argument for easier mutation execution on the client.
Unique payload type. Use a unique payload type for each mutation and add the mutation’s output as a field to that payload type.
...

So, If you have many mutations, you will manually define CreateInput, CreatePayload, UpdateProductInput, UpdateProductPayload and more.
The plugin will help you to define the XXXtInput and XXXPayload automatically. Your mutations are more consistent than ever.
Please check more examples in the readme.

Hi @orefalo
Following Designing GraphQL Mutations

The main points to consider when designing your GraphQL mutations are:

...
Input object. Use a single, required, unique, input object type as an argument for easier mutation execution on the client.
Unique payload type. Use a unique payload type for each mutation and add the mutation’s output as a field to that payload type.
...

So, If you have many mutations, you will manually define CreateInput, CreatePayload, UpdateProductInput, UpdateProductPayload and more.
The plugin will help you to define the XXXtInput and XXXPayload automatically. Your mutations are more consistent than ever.
Please check more examples in the readme.

Thank you -
I am just an old dude, pardon my ignorant - I didn't understand the examples and what they were bringing to the table
But your explanation helps,
If I may, I think you should explicitly highlight the benefits in the README.
Sincerely

Thank you -
I am just an old dude, pardon my ignorant - I didn't understand the examples and what they were bringing to the table
But your explanation helps,
If I may, I think you should explicitly highlight the benefits in the README.
Sincerely


CI enhancement, add logic when we need to test build logic

Currently our CI logic is just based on pr or push.
hence when we making changes on dockerfile or CI as base image github action, the test is not triggered, hence we need to modify the CI trigger in tmp....
Hence a between solution is that if the dockerfile, makefile or github action in related been modified, we trigger the image build without push to docker reg.

@SamYuan1990 Isn't this similar to #1078  ?

@SamYuan1990 Isn't this similar to #1078 ?

Yes


Adding wait_for_completion = True feature of TriggerDagRunOperator

Hi,
Is it possible to add wait_for_completion = True feature in this so that my controller dag waits for target dag execution completion and then proceeds with the next step?

Hi @VaibhavSingh98
there is a MultiDagRunSensor for such a purpose
https://github.com/mastak/airflow_multi_dagrun/blob/master/examples/trigger_with_multi_dagrun_sensor.py#L37


Add fix for CVE-2022-46176

r? @ehuss

@bors r=weihanglo

:pushpin: Commit d992ab4e9034930e7809749f04077045af8f4d79 has been approved by weihanglo
It is now in the queue for this repository.

:hourglass: Testing commit d992ab4e9034930e7809749f04077045af8f4d79 with merge 0849a28cc28ae88c31a6e47b7ed7e08595ffb433...

:sunny: Test successful - checks-actions
Approved by: weihanglo
Pushing 0849a28cc28ae88c31a6e47b7ed7e08595ffb433 to master...


use release-2.3 branch in clusterimageset subscription

https://github.com/open-cluster-management/backlog/issues/17849

/cc @qiujian16 @KevinFCormier @jnpacker

/lgtm


git-ignore: 1.2.2 -> 1.3.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/project/git-ignore/versions.
meta.description for git-ignore is: Quickly and easily fetch .gitignore templates from gitignore.io
meta.homepage for git-ignore is: https://github.com/sondr3/git-ignore
meta.changelog for git-ignore is: https://github.com/sondr3/git-ignore/blob/v1.3.1/CHANGELOG.md
Updates performed

Rust version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.3.1 with grep in /nix/store/2v2pa53x1g36nfa1adm4vy3brh6aaggj-git-ignore-1.3.1
found 1.3.1 in filename of file in /nix/store/2v2pa53x1g36nfa1adm4vy3brh6aaggj-git-ignore-1.3.1

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
git-ignore

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/2v2pa53x1g36nfa1adm4vy3brh6aaggj-git-ignore-1.3.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A git-ignore https://github.com/r-ryantm/nixpkgs/archive/d7ff5c623b349d1a0a3033817145be8d1fadcda1.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/d7ff5c623b349d1a0a3033817145be8d1fadcda1#git-ignore

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/2v2pa53x1g36nfa1adm4vy3brh6aaggj-git-ignore-1.3.1
ls -la /nix/store/2v2pa53x1g36nfa1adm4vy3brh6aaggj-git-ignore-1.3.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

git-ignore

Maintainer pings
cc @figsoda for testing.

Result of nixpkgs-review pr 289586 run on x86_64-linux 1

1 package built:

git-ignore


Review: generic noise("name",coords,...) and pnoise("name",coords,periods,...)

This allows for additional noise functions to be added in the future without needing to change APIs or add new functions, opens the door to individual renderer implementations being able to supply nonstandard noise functions as an extension, and makes it so that shaders can choose among many noise functions by just changing what name string they pass, rather than needing a big nested "if".  Initially, I have defined "perlin", "uperlin" (unsigned perlin), and "cell" to be the noises we already had.  A new variety, that was the impetus for this refactoring, will be coming soon...
If the name can be decoded at runtime (including if it's a shader parameter), the optimizer will make a direct call to the correct noise, so there is no string comparison at runtime.  Only if the optimizer can't reduce the name to a constant will it call the truly generic version that compares a few ustrings.  The implementation of that is most of the ugly verbiage of this review (though conceptually straightforward).
I decided that "perlin" is the name for what used to be snoise(), and "uperlin" is the unsigned variety that used to be noise().  I also accept the synonyms "snoise" and "noise", respectively.  And the old functions continue to work as well.  If people are not happy with this naming scheme, please speak up.

I should say, this is a work in progress.  The "pnoise" varieties are not fully working, but the regular noise("name",...) works fine.  But this seemed like a good first chunk to get feedback on.

Pushed a fix that addresses Chris' comments, also extends this work to completely subsume pnoise -- it all goes through the same IR-generating code path now.  And pnoise("cell",...) works now, the first true functionality (versus API) extension of this code.  I seek approval to merge what we have at this point.  I have a separate set of work (and reviews) that add more functionality, coming soon...

LGTM


cdn.jsdelivr.net links broken

The links in the readme for the tensor flow plugin and map plugin both don't exist.

I fixed the package config and re-published. The links work now!

@rsimon Thank you!


Training model failed. class not found: val

I have this directory structure:
DmitryCode
---   train
---   val
---   texture_nets
And when I launch:   "th train.lua -data ../ -style_image style.jpg"
I get this error :(
"
=> Generating list of images
| finding all validation images
find: ‘standard output’: Tubería rota
find: write error
/home/pixie/pixify/torch/install/bin/luajit: /home/pixie/pixify/trainlua/datasets/style-gen.lua:67: class not found: val
stack traceback:
[C]: in function 'assert'
/home/pixie/pixify/trainlua/datasets/style-gen.lua:67: in function 'findImages'
/home/pixie/pixify/trainlua/datasets/style-gen.lua:102: in function 'exec'
./datasets/init.lua:28: in function 'create'
./dataloader.lua:24: in function 'create'
train.lua:79: in main chunk
[C]: in function 'dofile'
...xify/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
[C]: at 0x004064d0
"
Somebody knows how could I fix this? Thanks

I had the same error without the dummy folders and now with them I don't have this error and training works..

what are the dummy folders??

Read the description please

Your folder structure needs to mimick fb.resnet.torch because Dmitry used some code from that repo for ingesting training data. This means -data <dataset> needs to point to a folder that contains both a train directory and a val directory, and each of those need to have a directory called dummy. This is because fb.resnet.torch is doing supervised learning on image classes where the directory of class "dummy" contains all the photos. Just put all photos in train/dummy and make val/dummy empty.
From the README: "The image dataset should be structured as in fb.resnet.torch having train and val folders and some folders corresponding to classes as you were doing classification. You can create a dummy folder train/dymmy/ and val/dummy/ and store all the images in them. Only images from train forlder will be used."

Thanks :)


Bot posts code snippets in Discord chat with nice formatting and highlighting

did you get the bot sending text to channels yet?

did you get the bot sending text to channels yet?

tracked in issue #3 , I have to finish the auth part (requires user interaction to get token from Twitch)

Can we close this issue now - as solved in issue #4

agreed. closing.


Discord4J unknown event type spam

See https://github.com/Discord4J/Discord4J/issues/1155
[00:57:17 WARN]: [discord4j.gateway.payload.JacksonPayloadReader] Error while decoding JSON (java.lang.IllegalArgumentException: Attempt to deserialize payload with unknown event type: APPLICATION_COMMAND_PERMISSIONS_UPDATE): {"t":"APPLICATION_COMMAND_PERMISSIONS_UPDATE","s":971,"op":0,"d":{"permissions":[{"type":3,"permission":false,"id":"429689904167911423"},{"type":3,"permission":true,"id":"958824056751288382"},{"type":3,"permission":true,"id":"882164398951628800"}],"id":"410138706490425344","guild_id":"429689904167911424","application_id":"410138706490425344"}}

Requires updating Discord4J to v3.3.0, currently pre-release

Fixed in 7fa43a5e833500debd52cccc89a29b31fbe4bd87


Mark code in troubleshooting.md

What kind of changes does this PR include?

New or updated content

Description

Uses our new code snippet inline marking feature to highlight important parts of the code blocks.

LGTM! ✅


registry token per service

Given an organization with many teams, each team owning one subgraph: it'd be great to emit a registry token per team so that each team can only update its own subgraph.
Currently, a registry token is not bound to a specific subgraph.

We will soon work on the new access layer in Hive, to enable more granular access, but also personal access tokens etc.


Hover with a light grey / offwhite

@caffo Are you happy with our current implementation?

Closing it as it seems to be :+1:


Question: Why Yoda-condition fixer does senseless replacements?

Yoda condition is needed to avoid mistake when there is variable on the left side of equality expression.
So, why do we need the replacements in inequality and in equalities without variables where impossible make mistake?
For example:
count($array) === 0
0 === count($array)

count($array) > 0
0 < count($array)

I apologize in advance , if this subject already rose, give me the link please.

Senseless or not - that's your opinion (personally I don't use Yoda conditions at all and the fixer helps me to do that). According to wiki:

Yoda conditions is a programming style where the two parts of an expression are reversed from the typical order in a conditional statement.

Thus yoda_style is to change all possible expressions to/from Yoda style.
The configuration of the fixer you can find in README allows you to specify types of statements to change, so you can easily achieve that only first example is in Yoda style.

@kubawerlos According to your link:

In programming languages that use a single equals sign (=) for assignment and not for comparison, a possible mistake is to assign a value unintentionally instead of writing a conditional statement.

I use Yoda condition and want the fixer fix it up:
$object === null to null === $object, $min === 10 to 10 === $min and so on.
But nothing more.

If you want to have more options on the Yoda fixer it would be a FR. For more background info about how the Yoda fixer came to be including the options and the case is does (not) fix you could read up on the PR https://github.com/FriendsOfPHP/PHP-CS-Fixer/pull/2446 , the PR's it replaced and the issues it closed. However this is not likely to provide more info than @kubawerlos stated.
Closing as @kubawerlos gave some good insights on this topic.

Yoda conditions are not just a convention - They exist because they turn potential runtime errors into compile time errors, but significantly, this benefit is only realised in some specific circumstances, for example:
if ($a == 5) {

is more safely expressed as:
if (5 == $a) {

because if you accidentally typed if (5 = $a) {, you'd get an immediate compile error, whereas  if ($a = 5) { would not. So that's all fine and understandable.
What is not understandable, and makes no particular sense is where this approach is applied for convention's sake, where there is no practical benefit, for example converting:
if (count($a) == 5) {

to
if (5 == count($a)) {

is pointless because in either case, typing a = instead of a == will result in a compile time error, so it doesn't matter which way around they are so there is no value in enforcing one over the other. Given that the conventional left-to-right format is generally considered easier to read (which is why many don't like yoda conditions), it would make sense for there to be an option in php-cs-fixer to only apply yoda conditions where it is likely to actually help, that is, where the left hand side of a comparison is assignable, otherwise by fixing the few cases where it does help, you reduce readability for the entire code base for no practical gain.

@Synchro Exactly!

I just found that the original PR for enforcing yoda conditions (#693) contained exactly the same point, and what's more pretty much everyone involved in it seemed to agree - so I'm not sure how it ended up here, doing exactly the opposite...

Applying Yoda style everytime including when it's not necessary according to the rule you explained makes sense too: people can get used to read Yoda style conditions and prefer avoid mixing both styles.
Though I think implementing a new option like e.g. only_when_left_operand_is_assignable makes sense. I invite you to open a new feature request or, if you have enough time, implement it directly and submit a pull request.

@Synchro I agree with @julienfalque : 0 === count($var) appears to be useless but instead makes sense.
Imagine the first revision of the code to be:
if (count($var) === 0) {
    // do stuff
}

One may think yoda condition only raises read complexity without any gains.
But then you need to refactor the code like:
$count = count($var);
if ($count === 0) {
    // do stuff
}

And the fixer must fix it:
$count = count($var);
if (0 === $count) {
    // do stuff
}

Another refactor (it happens), and you revert the logic:
if (0 === count($var)) {
    // do stuff
}

Maybe you have more condition like this around:
if (0 === count($var)) {
    if (strlen($text) === 1) {
        // do stuff
    }
}

And you end up in a mixed mess.
The solutions are:

The developer remember to switch back to non-yoda all useless yoda conditions
The fixer forces useless yoda condition to be non-yoda
You either choose yoda or non-yoda everywhere, without exceptions

Now apply the same logic to other comparison operators (which the fixer handles), and it invites disaster:
if (count($var) >= 5) {
    // do stuff
}

Needs to become (assuming we're doing exception-free Yoda):
if (5 < count($var)) {
    // do stuff
}

That kind of thing is extremely likely to introduce unexpected errors, represents a major drop in readability, and it doesn't do anything to convert a runtime error to a compile time one. In short, useless in every practical way other than to conform to some rigid dogmatic rule.
Also, if your'e using === rather than ==, you're much less likely to accidentally mistype it as = anyway - which is after all the entire point of the exercise.

Now apply the same logic to other comparison operators (which the fixer handles), and it invites disaster:

why would you apply it if you don't like, it is even off by default?

Uh, @slamdunk said "yoda or non-yoda everywhere, without exceptions" was the clearest and most understandable choice; I was just showing that that's not true.

rigid dogmatic rule

While l do like hearing suggestions from people about how we could improve something or what we might have done wrong, I've to say I'm done with comments like this.
What we provide here is both free and optional, no one forces you to use it.
Don't like it? Here is how things can be improved:

provide clear and simple examples, so no lengthy discussion going back and forth between multiple closed issues and PR's
open a PR and see it through till its merged


Replace macros to functions in SHA###

Notes:
Description
Replace macros in mbedtls SHA256 & SHA512 to function ro reduce code size
the tradeoff in replacing the macro is size over speed benchmark testing shows the following results:
Macro
SHA-256  515 Kb/s
SHA-512  338 Kb/s
Functions
SHA-256  456 Kb/s
SHA-512  309 Kb/s
SHA256 drops by 12% and SHA512 drops by 9%
Status
READY
Requires Backporting
NO
Migrations
NO
Additional comments
Tested on K64F
Todos

[ ] Tests
[ ] Documentation
[ ] Changelog updated
[ ] Backported

the tradeoff in replacing the macro is size over speed

Speed measurements showed how much was lost.
It would be interesting to know, what exactly was the gain in absolute and relative numbers?

I tried that for SHA-256 (on top of the existing configuration MBEDTLS_SHA256_SMALLER, which people will have enabled anyway if they care about code size), and building for Cortex-M0 with ARM-GCC 7 it saves 112 bytes (84 with armcc5). What's more interesting, making P a function only saves 16 bytes with ARM-GCC and 8 with armcc5 (which is no so surprising as it's only called in one place with the SMALLER option), so I think we could get most of the code-size savings by just making PUT_UINT32_BE() a function.

@Patater @Alonof I suspect now the PR should probably be re-raised to https://github.com/ARMmbed/mbedtls-restricted/pull/621?

Hi @kjbracey-arm
Do you have thoughts?

Actually I just ported the SHA-256 part of if to baremetal here: https://github.com/ARMmbed/mbedtls-restricted/pull/624 (which will be side-ported to Mbed Crypto at some point) and raised the SHA-512 part (plus some more) directly to Mbed Crypto here: https://github.com/ARMmbed/mbed-crypto/pull/178
So I think this PR can now be closed.


Fresh start - failed to build

building...
app/handlers/landing.go:4:2: package learn-superkit/app/views/landing is not in std<EMAIL_ADDRESS>app/routes.go:5:2: package learn-superkit/app/views/errors is not in std<EMAIL_ADDRESS>failed to build, error: exit status 1
running...
/bin/sh: tmp/bin/main: No such file or directory
Process Exit with Code: 127

go version go1.22.4 darwin/arm64
Model Name:	MacBook Pro
Model Identifier:	Mac14,9
Model Number:	MPHE3ZS/A
Chip:	Apple M2 Pro
Total Number of Cores:	10 (6 performance and 4 efficiency)
Memory:	16 GB
System Firmware Version:	10151.121.1
OS Loader Version:	10151.121.1
Serial Number (system):	FXL2C7YL21
Hardware UUID:	DEAF02ED-9A78-50C3-B6BF-D336FF0D66F0
Provisioning UDID:	00006020-000A3112366B401E
Activation Lock Status:	Enabled

Did you tried with a go mod tidy or go get -u ./...?

Yeah, here's the output:
❯ go get -u ./...
warning: ignoring symlink /Users/shah/Projects/learn-superkit/node_modules/esbuild
warning: ignoring symlink /Users/shah/Projects/learn-superkit/node_modules/tailwindcss
go: downloading github.com/go-chi/chi/v5 v5.0.14
go: downloading github.com/anthdm/superkit v0.0.0-20240622052611-30be5bb82e0d
go: learn-superkit/app imports
        learn-superkit/app/views/errors: package learn-superkit/app/views/errors is not in std<EMAIL_ADDRESS>go: learn-superkit/app/handlers imports
        learn-superkit/app/views/landing: package learn-superkit/app/views/landing is not in std<EMAIL_ADDRESS>

It could be due to your templ version. It should be the latest version. The latest version is 0.2.707 https://github.com/a-h/templ/releases/tag/v0.2.707 @omarshah0

You can also try with go clean -cachemod && go get -u ./.... I need to find a good way fix these dependency issues coming from bootstrap folder

Really would like to use this library, get the same output on Linux, Ubuntu 24.04
go: master/app imports 	master/app/views/errors: package master/app/views/errors is not in std<EMAIL_ADDRESS>go: master/app/handlers imports 	master/app/views/landing: package master/app/views/landing is not in std<EMAIL_ADDRESS>Are there any known solutions? I've tried upgrading templ, didn't seem to have an effect


Optionally set chat type in BorderClaimEvent

Currently theres no way for thirdparty addons how to display enter/exit messages of the claimed area in the actionbar
..english might be a little bit off in the javadocs.


Fix #696 complicated delete problem: continues #700

It fixes #696.
It is based on #700.
I have fixed test_delete.vim tests.
@k-takata : Can you test it in Windows?
It seems better #700.  But it may not work in Windows.
Any fix is appreciable.

This is actually a difficult problem.
After I wrote the comments in #700, I found that replacing [ with [[] doesn't always work.
If there is a file named foobar[.txt, :w! foobar[[].txt writes the current buffer to the file foobar[.txt.
However if there is no file named foobar[.txt, :w! foobar[[].txt writes to the file foobar[[].txt instead of foobar[.txt. The behavior depends on the existing files.
This means that there is no general way to escape a filename on Windows.
I think there are (at least) two ways to fix the problem.

Add a new flag to gen_expand_wildcards() (or create a new function) to list all files and directories under the specified directory.
Currently, we pass "pathname/*" to gen_expand_wildcards() but we don't need to expand the part of "pathname/".
Create a general specification to escape a filename on Windows, then use it for this.
I think this is difficult, and this could be cause some backward incompatibilities.

I don't know this patch is correct.
But, You should do the NULL check to the esc_name and esc_name2.
If NULL, call the vim_free() properly and return -1.

Thank you.  I have added the NULL check.
And I have fixed the tests error.

Add a new flag to gen_expand_wildcards() (or create a new function) to list all files and directories under the specified directory.

I think it is better.  But it seems difficult...

I think it is better. But it seems difficult...

Yeah, agree. But I don't think we can fix this without it.
https://ci.appveyor.com/project/chrisbra/vim/build/2031/job/lcfupaxg9gd50kfp#L2019

We need listdir() implementation.
I may implement it for Unix.
But it is difficult to implement for Windows.

Fixed by 8.1.1378.


Update Deploy on Fly.io section of readme

The FLy.io deploy process has changed and has a UI component now, so updates are required to the readme for this repo and the other example repos.
You'll want to test with fly launch --no-deploy to see the changes. That way you can still  add the secret before running fly deploy.

Also, the way to scale into other regions needs an update. See this PR: https://github.com/superfly/docs/pull/1235

Thanks suggestions. README has been updated.


Add BioStructures.jl

I think this is all I need to do to add the BioStructures.jl package to the main site?
The doc build is working, let me know if there is anything else.

That should be all that is required, thanks @jgreener64 :+1:


efficientnet feature extractor

[x] Implement a FeatureExtractor in spacer/extract_features.py
[ ] Add pytorch utils in torch_utils.py
[ ] Write tests of all utils and methods

@beijbom hey Oscar, all the unit tests are passed on my local but The Travis CI build failed b/c of the permission issue, do you know how to solve this?
FAIL: test_gray (spacer.tests.test_torch_utils.TestClassifyFromPatchList)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/beijbom/pyspacer/spacer/tests/test_torch_utils.py", line 42, in setUp
    'efficientnetb0_5eps_best.pt')
  File "/home/travis/build/beijbom/pyspacer/spacer/storage.py", line 186, in download_model
    assert config.HAS_S3_MODEL_ACCESS, "Need access to model bucket."
AssertionError: Need access to model bucket.
======================================================================
FAIL: test_rgb (spacer.tests.test_torch_utils.TestClassifyFromPatchList)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/beijbom/pyspacer/spacer/tests/test_torch_utils.py", line 42, in setUp
    'efficientnetb0_5eps_best.pt')
  File "/home/travis/build/beijbom/pyspacer/spacer/storage.py", line 186, in download_model
    assert config.HAS_S3_MODEL_ACCESS, "Need access to model bucket."
AssertionError: Need access to model bucket.

hey @qiminchen : I had disabled s3 access for Travis due to the SQS tests. But since those are skipped now, i re-enalbled it. But also, you should use the @unittest.skipUnless(config.HAS_S3_MODEL_ACCESS, 'No access to models') decorator for tests that require model access, since we can't assume all developers have that access. That will skip the test rather than fail out.
A few more notes looking from glancing at your code:

Please re-base to master now that I merged my big PR.
You don't have to follow the exact structure of the caffe feature extractor as long as you implement the interface. For example, you don't have to call your method classify_from_patchlist. Although, if it is convenient, by all means use it. But I do think it's pretty confusing for the reader to have two methods that have the same exact name but work on different libraries. Better then to make a new shared classify_from_patchlist method that is a shared util agnostic to the feature extractor. Or perhaps make a method im_to_patchlist that only does the cropping and pre-processing and then write new methods classify_from_patchlist that take in the patchlist. Sorry, this comment got a bit confusing. Perhaps take a stab at finishing this up after rebasing to new master and then I can add more specific comments.
If there are utility function in caffe_utils that you want to use, perhaps move to a shared extract_features_utils file?
All imports go at the beginning of the file. The only reason I broke this rule is that caffe isn't available in all builds (pip install and clone) so I had to protect those imports.

@qiminchen : CI pipeline permission updated.

One more suggestion @qiminchen : can you write a script (pls. put in in the scripts folder), that runs this new feature extractor together with the classifier training on the exported spacer trainingdata and confirms that it get similar (or higher) performance than production. That would function as a regression test basically.

Please re-base to master now that I merged my big PR.

Done

If there are utility functions in caffe_utils that you want to use, perhaps move to a shared extract_features_utils file?

I moved gray2rgb(), crop_patch() and crop_simple() to the shared extract_features_utils.py
torch_utils.py only serves as feature extraction

so in extract_features.py, it crops the image to patch list, calls extract_feature() from torch_utils.py and return ImageFeatures and ExtractFeaturesReturnMsg

All imports go at the beginning of the file. The only reason I broke this rule is that caffe isn't available in all builds (pip install and clone) so I had to protect those imports.

All imports go at the beginning.

One more suggestion @qiminchen : can you write a script (pls. put in in the scripts folder), that runs this new feature extractor together with the classifier training on the exported spacer trainingdata and confirms that it get similar (or higher) performance than production. That would function as a regression test basically.

Sure, I will work on it. By the way, I need to add unittest for EfficientNetExtractor in test_extract_features.py and more specifically I should generate features_legacy for test_regression right?

@beijbom hey Oscar, thanks for the review

Please re-base to master now that I merged my big PR.

Done

If there are utility functions in caffe_utils that you want to use, perhaps move to a shared extract_features_utils file?

I moved gray2rgb(), crop_patch() and crop_simple() to the shared extract_features_utils.py
torch_utils.py only serves as feature extraction

so in extract_features.py, it crops the image to patch list, calls extract_feature() from torch_utils.py and return ImageFeatures and ExtractFeaturesReturnMsg

All imports go at the beginning of the file. The only reason I broke this rule is that caffe isn't available in all builds (pip install and clone) so I had to protect those imports.

All imports go at the beginning.

One more suggestion @qiminchen : can you write a script (pls. put in in the scripts folder), that runs this new feature extractor together with the classifier training on the exported spacer trainingdata and confirms that it get similar (or higher) performance than production. That would function as a regression test basically.

Sure, I will work on it. By the way, I need to add unittest for EfficientNetExtractor in test_extract_features.py and more specifically I should generate features_legacy for test_regression right?

@qiminchen : sounds good. I added some comments, but let me know when you want me to take a close look again. It's not a bad idea to write a regression test. You can go ahead and upload any test fixtures you want to s3.
Re. the shared utils, please move them from caffe_utils so we don't duplicate code. Also move tests. And please feel free to refactor the caffe extractor also as you are cleaning up and reorganizing the code.

One more suggestion @qiminchen : can you write a script (pls. put in in the scripts folder), that runs this new feature extractor together with the classifier training on the exported spacer trainingdata and confirms that it get similar (or higher) performance than production. That would function as a regression test basically.

@beijbom Oscar, can you refer me to the files used for extracting features of a source, looks like scripts/classifier_training_regression.py downloads the features to local and trains the classifier. What about extracting the features and saving to the temporary directory that you used for VGG16?
For TestEfficientNetExtractor, I was able to pass all the cases in local but it fails at Travis CI, it might be because of the libjpeg version?
Traceback (most recent call last):
  File "/home/travis/build/beijbom/pyspacer/spacer/tests/test_extract_features.py", line 295, in test_regression
    self.assertEqual(pf_new.data, pf_legacy.data)
AssertionError: Lists differ: [1.6237620115280151, -0.1788397878408432, -0.18572[27272 chars]3397] != [1.6237621307373047, -0.17883972823619843, -0.1857[27283 chars]7278]
First differing element 0:
1.6237620115280151
1.6237621307373047

One more suggestion @qiminchen : can you write a script (pls. put in in the scripts folder), that runs this new feature extractor together with the classifier training on the exported spacer trainingdata and confirms that it get similar (or higher) performance than production. That would function as a regression test basically.

@beijbom Oscar, can you refer me to the files used for extracting features of a source, looks like scripts/classifier_training_regression.py downloads the features to local and trains the classifier.

Yeah, I download them b/c we run 5 epochs, so it's faster to download once and then run locally. You don't have to do that if you don't want to.

What about extracting the features and saving to the temporary directory that you used for VGG16?

Hmm. I'm not sure what you mean. For extracting features, just compose a msg  = ExtractFeaturesMsg() and then call tasks.extract_features(msg). The beauty of the new DataLocation design is that image_loc and feature_loc can point to the same storage system, but they don't have to do so. You can have the task load from the S3 bucket and write to local disk. What ever you prefer. So one way is to download also the image files, and then extract features for all images once they are local. Another way is to point image_loc to s3 and feature_loc to the local folder.

For TestEfficientNetExtractor, I was able to pass all the cases in local but it fails at Travis CI, it might be because of the libjpeg version?
Traceback (most recent call last):
  File "/home/travis/build/beijbom/pyspacer/spacer/tests/test_extract_features.py", line 295, in test_regression
    self.assertEqual(pf_new.data, pf_legacy.data)
AssertionError: Lists differ: [1.6237620115280151, -0.1788397878408432, -0.18572[27272 chars]3397] != [1.6237621307373047, -0.17883972823619843, -0.1857[27283 chars]7278]
First differing element 0:
1.6237620115280151
1.6237621307373047

I don't think this has to do with libjpeg. When you compare float32 vectors you can't expect identical results. Instead use a test that is robust to float precision errors. For example, you can borrow from what I did here:
https://github.com/beijbom/pyspacer/blob/master/spacer/tests/test_beta_regression.py#L141

One thing to note about Travis CI. Some unit-tests that involved fetching stuff from S3 do fail intermittently. So if you see fails related to that ping me and I can restart that job on Travis. In the long term we should probably move to using local test fixtures.

@beijbom hey Oscar, the script that runs the new feature extractor together with the classifier training on the exported spacer trainingdata is ready and I tested it on s294, the performance is better than the production as we're using EfficientNet-b0. The new repo is a legend!
-> Downloading data for source id: 294.
-> Downloading 201 metadata...
-> Assembling train and val data for source id: 294
-> Training...
-> Trainset: 157, valset: 18 images
-> Using 25 images per mini-batch and 7 mini-batches per epoch
-> Trainset: 20, valset: 9, common: 8 labels
-> Loading reference data.
-> Online training...
-> Epoch 0, acc: 0.8368991386496248
-> Epoch 1, acc: 0.8388441233676021
-> Epoch 2, acc: 0.8399555432064463
-> Epoch 3, acc: 0.8396776882467352
-> Epoch 4, acc: 0.8407891080855793
-> Epoch 5, acc: 0.8405112531258683
-> Epoch 6, acc: 0.8421783828841345
-> Epoch 7, acc: 0.8421783828841345
-> Epoch 8, acc: 0.8419005279244235
-> Epoch 9, acc: 0.8424562378438455
-> Calibrating.
-> Re-trained LTER Back Reef (294). Old acc: 80.0, new acc: 86.8

I tried to make the script more flexible by:
def main(extractor_name, source_id, n_epochs):
    extractor = ExtractFeatures(extractor_name)
    extractor(source_id)
    classifier = TrainClassifier()
    classifier(source_id, n_epochs)

if __name__ == '__main__':
    fire.Fire(main)

then I ran into some relative import issue and haven't found a better way to solve it, e.g.
Traceback (most recent call last):
  File "scripts/features_extracting_training.py", line 16, in <module>
    from spacer import config
ModuleNotFoundError: No module named 'spacer'

Do you have any ideas? Other than that, this PR should be ready for review.

@beijbom hey Oscar, here are some updates on the progress:

Can you go through and add Typing hints to all utils? I was a bit lazy previously, but we might as well do it.

I should've completed adding type hints to files that needed. Please let me know if I miss any

Code coverage for the files in /models folder is not 100%. Can you add test to cover those lines?

Added, should be coveraged now, already tested in local

For the scripts, I think there are some duplicated code between features_extracting_training.py and classifier_training_regression.py. Can you move the shared code to a new scripts.utils.py?

Added. I moved the build_traindata() and start_training() to the scripts_utils.py and cleaned up both features_extracting_training.py and classifier_training_regression.py.
The reason I didn't move _cache() to scripts_utils.py is that classifier_training_regression.py downloads all the files to local (features.json, anns.json and image.meta.json) while features_extracting_training.py only downloads anns.json and image.meta.json as features are downloaded when being extracted.

As per your question. Hmm. That should work fine. Can you push that change so I can see if I can replicate the problem? My guess is that you may have to add spacer to your PYTHONPATH.

Thanks for the help, I solved this issue by adding the spacer to $PYTHONPATH, so config.LOCAL_FEATURE_DIR goes to the CLI, you can use
$ python scripts/features_extracting_training.py efficientnet_b0_ver1 294 10 \ /path/to/local/path/for/saving/features
where efficientnet_b0_ver1 294 is extractor_name, 294 is source_id and 10 is the number of training epochs, to run the features_extracting_training.py

actually, it does not pass. Can you try using png?

Actually the images I used for generating the legacy features are already png, and the original image for production testing is also png so technically it should work. Weird... still debugging...

@qiminchen : Great job pushing this PR through. Feel free to merge this to master when you are ready.

Nice work on this!


Fixing SPARK-5744.

RDD.isEmpty fails when an RDD contains empty partitions.

FYI: The method was introduced in https://github.com/apache/spark/pull/4074

Can one of the admins verify this patch?

Thanks for doing this, but the title of this PR isn't sufficient.  It will become the commit log message, so please update the PR title to adequately describe what you did so that other developers don't have to look into the details of the commit or look up the JIRA issue just to get an idea of what this PR is about.

@tbertelsen Better, but you still should include SPARK-5744 and add [CORE] to the PR title.

Sorry. Is is good now?

perfect

This works, so it's not quite empty partitions:
sc.parallelize(Seq[Int](), 1).isEmpty()

This also creates an exception, so it's to do with Seq() (which is of Nothing, not Any, sorry):
sc.parallelize(Seq(), 1).take(1)

I think the problem roughly boils down to this behavior in Scala:
Seq(Seq().toArray,Seq().toArray).toArray
java.lang.ArrayStoreException: [Ljava.lang.Object;
...

The problem is the Nothing type, rather than emptiness, but I'm still scratching my head figuring out the cleanest way to deal with that.

I think I have a solution. A Seq[Nothing] must be empty since there are no instances of Nothing. So I think this can be fixed (and optimized) by returning an EmptyRDD from parallelize when given an empty argument.
  def parallelize[T: ClassTag](seq: Seq[T], numSlices: Int = defaultParallelism): RDD[T] = {
    assertNotStopped()
    if (seq.isEmpty) {
      new EmptyRDD[T](this)
    } else {
      new ParallelCollectionRDD[T](this, seq, numSlices, Map[Int, Seq[String]]())
    }
  }

Witness:
scala> sc.parallelize(Seq(), 1).isEmpty
res0: Boolean = true

scala> sc.parallelize(Seq(), 1).take(1)
res1: Array[Nothing] = Array()

It deserves a unit test for both of these and a run through all the tests. Want to try that?

How do we treat null values, e.g. sc.parallelize(Seq(new Array[Nothing](1))). I have no experience with TypeTages, but couldn't we use them in a smart way?
By the way sc.parallelize(Seq(new Array[Nothing](1))) on almost anything

Array[Nothing](1) won't compile by itself, nor will Seq[Nothing](1) for different reasons. new Array[Nothing](1) won't be accepted as a Seq[?] for reasons I don't 100% get.
But there is actually still the same problem with the Null type: sc.parallelize(Seq(null), 1).isEmpty still fails. The change above is good but doesn't fix this case. Back to the drawing boards...

Hmph. I think we might could work around this if runJob didn't have to return an Array, but it does. Even if it collects its results in another collection it has to get copied to an Array. I think that's where this runs into the mismatch between these Scala types and JVMs and arrays and covariance every way I can see.
All of these are fairly artificial cases. An empty RDD or partition of a normal type works fine. I think it's worth making the EmptyRDD change to take care of the Seq() case, and treating the rest as just how it is with Scala and arrays and these types and the current API.

It compiles if you include new. It just becomes a null filled array.
scala> new Array[Nothing](1)
res19: Array[Nothing] = Array(null)

I think your right about this being fairly artificial.  One solution could be, just not to accept Nothing as the type of an RDD, and fail fast and predictable like this:
    if (implicitly[ClassTag[T]].runtimeClass == classOf[Nothing]) {
      throw new RuntimeException("RDD[Nothing] is not allowed.")
    }

We must however be carefull. Right now sc.emptyRDD returns an RDD[Nothing], and there could easily be unforeseen consequences by prohibiting RDD[Nothing]
If we just want to provide a good error message, we could also just catch the ArrayStoreException and wrap it in an exception with a more descriptive message. Perhaps at the place where, it is already wrapped in the SparkDriverExecutionException.

The case of Nothing is actually fine to solve here; you don't have to reject it. It's Null I can't figure out. I think at best, just fix the Nothing case per above.

@tbertelsen Here's my proposed fix: https://github.com/srowen/spark/commit/2390a3f612eece1a2f68c7bd8edbb88647062854
I discovered along the way that histogram() doesn't support an RDD with 0 partitions, so included that fix.

It looks like it fixes the issue with Seq(), but not e.g. new Array(2) (have not run it though). But after our discussion I think that is perfectly fine.
If you manage to create a non-empty collection of Nothing, then practically nothing will work in spark, or in scala for that matter, e.g., (new Array(2)).toList won't compile.
Handling RDD[Nothing] seems like it will require quite some work, but I fail to see the use case where it brings real functionality.
In summary: your fix seems like a good solution. I'll close this PR.

OK, I can merge my PR. I would be fine with you adding this in to yours, with whatever additions you want to. That is, I didn't intend to hijack your change, and can credit you in JIRA.


Syntax Highlighting

The mermaid live editor is using the monaco library and contains a definition for the mermaid language: https://github.com/mermaid-js/mermaid-live-editor/tree/master/src/mermaid-language
Since it's based on the same editor engine, is it possible to incorporate this for syntax highlighting?

Hello. Thank you for really great request and information around monaco-editor. AFAIK, Monarch is not supported in VS so far. (https://github.com/microsoft/vscode/issues/216) But I'll try to support syntax highlighting.

Thank you for waiting the feature. v0.8.0 has supported the syntax highlight.


Dynamic smokable inhalation rate and smokable's max volume tweaks

About the PR

In my last PR's #30053 and #30101 I had increased many of the max volumes for smokable entities to align them with the base max volume for Dan's Soaked Smokes. Instead of increasing the base inhalation rate to counter their extended duration, I have set all smokables to last a set amount of time and dynamically change their inhalation rates to match the time to finish smoking. This PR is identical to #30101 except I have fixed (or at least attempted to) the serialization issues brought up by Sloth and Emogarbage. I've made a new PR since I thought I wouldn't have time to fix it and closed #30101 then not 4 hours later was I gas lit into fixing it by friends.
Why / Balance

Many chemicals and narcotics when added to cigarettes are either not abundant enough and/or do not metabolize fast enough to possess a significant threat or achieve a desired effect, allowing more room for chemicals to be added means that a higher amount of said chemicals will enter the blood stream allowing for longer or more potent effects with accumulation. This will expand upon gameplay opportunities for the entrepreneurial Chemist, Syndicate Agent with a chemical kit, or Botanist with a "Green" thumb. Due to these changes, the Syndicate Omnizine Smoke will be significantly buffed, making it last the set 6 minutes to smoke from the 12 minutes it used to take to completely inhale all reagents with a static inhalation amount.
Technical details

The Smoking System now calculates the inhalation amount instead of using the Smokable Component's fixed inhalation amount, it will change depending on the volume of chemicals contained within the smokable making sure that you inhale the correct amount to finish all reagents within the set time frame of 360 seconds as defined by SmokableDuration float. In practice a full cigarette with 40u of chemicals will inhale ~0.33u of the total volume per update/tick in the Smoking System.
Compared to the "vanilla" system, this is an increase from the minimum time to smoke something with a base 10u of chemicals like what you would expect from a normal cigarette of 4 minutes, and a decrease from the maximum time to smoke something with a full 40u of chemicals like the syndicate omnizine smoke of 12 minutes.
In addition to the dynamic inhalation amount, adjusted the max volume of the base cigarette/cigar entity to match the volume of "Dan's Soaked Smokes" or just the normal "Soaked Smoke" entity (40u max), tweaked the max volume of other smokables such as blunts and joints to make more room for any additional chemicals added from plant mutations, and doubled the amount of extra chemicals to "Dan's Soaked Smoke's" smokes (with the exception to the Dan's Omnizine smoke, it remains 30u).
Media

Requirements

[X] I have read and I am following the Pull Request Guidelines. I understand that not doing so may get my pr closed at maintainer’s discretion
[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes

N/A I hope
Changelog

:cl:

tweak: The time to finish a smokable (cigarettes) is now fixed to 360 seconds, allowing you to metabolize more chemicals faster.
tweak: Cigarettes, Joints, Cigars, and Blunts can now hold a maximum 40u of chemicals.
tweak: Increased the amount of chemicals in a majority of "Dan's Soaked Smokes"

40u is a looooooot tbh as a unbiased player I believe maybe 20u would be a good max

40u is a looooooot tbh as a unbiased player I believe maybe 20u would be a good max

20u used to be default on paper but all smokables have a mandatory 10u of nicotine or THC, meaning only 10u of chemicals can be added to them. Soaked smokes allow for up to 40u, matching the max volume of soaked cigarettes to normal cigarettes means players have access to something that can hold a larger abundance of chemicals that will metabolize in larger amounts for later consumption.
I intend to make it so if someone makes a healing or damage dealing smoke it will have a much more immediate and noticeable effect, or just load 10 different types of narcotics and have some fun like what ive seen many chemists and botanists do.

I might be reading it wrong but does this not only set the InitialVolume on the first update the cigarette is spawned? Does the system appropriately account for injecting more reagents into the cigarette, either before or during smoking?

It should, if I recall correctly it checks every update tick for what’s contained as the volume.
It’s been a while since I’ve tested it and I’m not in the position to set up my computer to check at the moment due to moving.
I’ll hopefully get back to you on this soon, I’ll probably get a buddy to test it.

tested and confirmed that the cig lasts for longer/shorter than the intended time of 6 mins if you insert reagents into the cig before smoking it
I love droppering in 10us of omni into my cig
anyways, you'll have to change it so it does the math every time instead of just that single instance when it's spawned in.

There are some minor stuff I've considered with this approach.
Let's say a 20u cigarette takes 6 minutes to smoke. According to this PR, adding 20u more reagent won't change the smoke duration, you just consume more reagent each tick. That feels okay; the cigarette is more saturated, makes sense you get more effect.
 If we look at the most extreme case and say I have a cigarette I've smoked for 5 minutes, 55 seconds. If I inject it with 20u reagent, will those 20u be ingested over the last 5 seconds?

There are some minor stuff I've considered with this approach.
Let's say a 20u cigarette takes 6 minutes to smoke. According to this PR, adding 20u more reagent won't change the smoke duration, you just consume more reagent each tick. That feels okay; the cigarette is more saturated, makes sense you get more effect.
 If we look at the most extreme case and say I have a cigarette I've smoked for 5 minutes, 55 seconds. If I inject it with 20u reagent, will those 20u be ingested over the last 5 seconds?

Injecting any reagent will extend the time to smoke, it calculates the amount inhaled from the current volume to match 6 minutes before its lit. There’s no timer governing how long it should last, just "this should last 6 minutes, you have 40u before lighting, every tick you will inhale X amount".
This is even an issue outside of this PR since smokes only go unlit when the amount of reagents in them hits 0, you could smoke all the nicotine in a smokable but keep adding new chemicals to smoke it forever.

I'm gonna draft this since it would take me at least a month before I'm able to fix these issues, I'll notify a maintainer and un-draft it when my work is complete.


fix CI + Update SDL2 crate v0.35

support GLES 3 on linux (on examples/basic.rs),
support create/update rgba8 user texture.
impl epi to Painter (Optional, can be enabled: features = ["use_epi"])  to support most demo lib functionality.
native sdl2 clipboard support.
fix reactive mode on demo_lib example.
fix clippy.

Thank you for the fixes and improvements!


Provision step fails if service not available in location

[X] Make sure you've installed the latest version using instructions in the wiki

Output from azd version
Run azd version and copy and paste the output here:
azd version 0.6.0-beta.2 (commit c4a201e55a13eba77f9037b296a90e0f81b00dbe)
Output from az version
Run az version and copy and paste the output here (minimum required version is 2.38.0):
{
  "azure-cli": "2.45.0",
  "azure-cli-core": "2.45.0",
  "azure-cli-telemetry": "1.0.8",
  "extensions": {}
}

Describe the bug
When deploying the standard monitoring section in westcentralus, the following error occurs:
{
    "status": "Failed",
    "error": {
        "code": "DeploymentFailed",
        "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.",
        "details": [
            {
                "code": "BadRequest",
                "message": "{\r\n  \"error\": {\r\n    \"code\": \"LocationNotAvailableForResourceType\",\r\n    \"message\": \"The provided location 'westcentralus' is not available for resource type 'microsoft.insights/components'. List of available regions for the resource type is 'eastus,southcentralus,northeurope,westeurope,southeastasia,westus2,uksouth,canadacentral,centralindia,japaneast,australiaeast,koreacentral,francecentral,centralus,eastus2,eastasia,westus,southafricanorth,northcentralus,brazilsouth,switzerlandnorth,norwayeast,norwaywest,australiasoutheast,australiacentral2,germanywestcentral,switzerlandwest,uaecentral,ukwest,japanwest,brazilsoutheast,uaenorth,australiacentral,francesouth,southindia,westus3,koreasouth,swedencentral,canadaeast,jioindiacentral,jioindiawest,qatarcentral'.\"\r\n  }\r\n}"
            }
        ]
    }

This is because App Insights is not available in westcentralus.
To Reproduce
Steps to reproduce the behavior...
Deploy one of the standard templates in westcentralus
Expected behavior
A clear and concise description of what you expected to happen.
While each core template should allow for all locations, if a location is not supported by the resource, it should be redirected to a region-specific location that is supported.  For example, in westcentralus, we might redirect App Insights to southcentralus.

Hi, I am having the same issue...

azd version 0.6.0-beta.2 (commit c4a201e55a13eba77f9037b296a90e0f81b00dbe)
{
  "azure-cli": "2.45.0",
  "azure-cli-core": "2.45.0",
  "azure-cli-telemetry": "1.0.8",
  "extensions": {}
}

Ideally we should analyze all descriptors prior to initializing the provisioning to check the required region is supported, and suggest changes. The only possible way to recover now is to destroy and start over.


hawtio could show ActiveMQ if installed

Since wildfly-camel patch distributes and installs hawtio it maight activate by default ActiveMQ management-plugin when installed in Wildfly with following way: http://stackoverflow.com/questions/24921640/how-to-set-up-messaging-subsystem-using-cli-in-wildfly

If I understand correctly, you're asking for this https://github.com/hawtio/hawtio/issues/2185, right?

Yes

Ok - I think it'd be better that this is tracked in the HawtIO project under https://github.com/hawtio/hawtio/issues/2185. Once the Artemis Plugin is part of the of the HawtIO build, WildFly-Camel can consume the new functionality once it's available in a new HawtIO release.
I'll close this issue and see if I can find a resolution for https://github.com/hawtio/hawtio/issues/2185 when I have some more time.


Print usage when no command is supplied.

Bin should print its usage info when executed without any command.
Expected behavior:
$ bin
Effortless binary manager

Usage:
  bin [command]

Available Commands:
  help        Help about any command
  install     Installs the specified project
  list        List binaries managed by bin
  prune       Prunes binaries that no longer exist in the system
  remove      Removes binaries managed by bin
  update      Updates one or multiple binaries managed by bin

Flags:
      --debug     Enable debug mode
  -h, --help      help for bin
  -v, --version   version for bin

Use "bin [command] --help" for more information about a command.

Observed Behavior:
$ bin

 Path   Version         URL     Status

OS/Arch: Linux/x86
Bin version:
bin version 0.2.0
commit: 1e4d59085ecfa8dd0558d237407382eda76c5d3c
built at: 2020-08-09T15:41:51Z
built by: goreleaser

Thx for the contribution. This decision was intentional, since I prefer it this way. https://github.com/marcosnils/bin/pull/35#issuecomment-695840033

Okay, but why then does bin help and bin --help behave differently?
$ bin --help
Effortless binary manager

Usage:
  bin [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  ensure      Ensures that all binaries listed in the configuration are present
  help        Help about any command
  install     Installs the specified project from a url
  list        List binaries managed by bin
  prune       Prunes binaries that no longer exist in the system
  remove      Removes binaries managed by bin
  update      Updates one or multiple binaries managed by bin

Flags:
      --debug     Enable debug mode
  -h, --help      help for bin
  -v, --version   version for bin

Use "bin [command] --help" for more information about a command.

$ bin help

 Path				Version		URL					Status
 /home/jcube/.local/bin/pants	v0.5.4		github.com/pantsbuild/scie-pants	OK

$ bin help install

 Path				Version		URL					Status
 /home/jcube/.local/bin/pants	v0.5.4		github.com/pantsbuild/scie-pants	OK

$ bin install --help
Installs the specified project from a url

Usage:
  bin install <url> [flags]

Aliases:
  install, i

Flags:
  -a, --all               Show all possible download options (skip scoring & filtering)
  -f, --force             Force the installation even if the file already exists
  -h, --help              help for install
  -p, --provider string   Forces to use a specific provider

Global Flags:
      --debug   Enable debug mode

help command (not option --help) feels useless.
$ bin --version
bin version 0.16.2
commit: d1213c11e63483174ed95308c9eb6372bb9b16e9
built at: 2023-02-21T13:11:19Z
built by: goreleaser\

Okay, but why then does bin help and bin --help behave differently?

it's a bug, just pushed a fix for that https://github.com/marcosnils/bin/commit/2d0b198e88058dedbbd721f5275c01827c9e7dc1


Add docs for random provider

The random provider was not part of the run_typedoc.sh script - this pull request rectifies that and generates an initial set of docs based on v0.4.0.

Also - this fixes #862.


FlatList docs (props) are incomplete

FlatList appears to accept the ListView props pageSize and initialNumToRender, but these props (and maybe others?) are missing in the documentation.

VirtualizedList is the base for both FlatList and SectionList.

Hi there! This issue is being closed because it has been inactive for a while.
But don't worry, it will live on with Canny! Check out its new home: https://react-native.canny.io/feature-requests/p/flatlist-docs-props-are-incomplete


connectivity: Label test namespaces

Sometimes the perf command fails to schedule pods because there are too many pods ^1. Add app.kubernetes.io/name label ^2 to connectivity test namespaces, and use the label to delete test namespaces before running the perf command.

✅ success

aks: https://github.com/cilium/cilium-cli/actions/runs/9735853145/job/26865594789?pr=2652
eks: https://github.com/cilium/cilium-cli/actions/runs/9735853149/job/26865594359?pr=2652
eks (tunnel): https://github.com/cilium/cilium-cli/actions/runs/9735853152/job/26865595170?pr=2652
external workloads: https://github.com/cilium/cilium-cli/actions/runs/9735853159/job/26865645799?pr=2652
gke: https://github.com/cilium/cilium-cli/actions/runs/9735853144/job/26865645824?pr=2652


BUG: Bug in clearing the cache on DataFrame.pop and a subsequent inplace op #10912

closes #10912

lgtm.


Add instance support:

Currently, in order to support things like:
sealed case class Foo(value: Int)

implicit val fooSemigroup = Semigroup.instance[Foo](
  append = (a, b) => Foo(a.value + b.value)
)

a lot of boilerplate is needed:
object Semigroup {
  def instance[T](append: (T, T) => T) = {
    val append0 = append
    new Semigroup[T] { def append(a: T, b: T): T = append0(a, b) }
  }
}

for type classes with 2 and more member functions, this gets worse:
object Monoid {
  def instance[T](zero: T, append: (T, T) => T) = {
    // all the aliases are needed so that the parameter names could be
    // the same as the function names, while avoiding infinite recursion
    val zero0 = zero
    val append0 = append
    new Monoid[T] {
      def zero = zero0
      def append(a: T, b: T): T = append0(a, b)
    }
  }
}

The default order would match the declaration order (inherited type class members first or last?); if the default order isn't good enough and named arguments are not desired, one could use the hypothetical instance = ... parameter to @typeclass:
@typeclass(instance = ('zero, 'append))
trait Monoid[T] extends Semigroup[T] {
  def zero: T
}

implicit val fooMonoid = Monoid.instance[Foo](Foo(0), (a, b) => Foo(a.value + b.value))
// or
implicit val fooMonoid = Monoid.instance[Foo](
  zero = Foo(0),
  append = (a, b) => Foo(a.value + b.value)
)

...or possibly better alternatives than instance, perhaps order or deriveOrder or something like that?

One thing to note with this syntax is that it wouldn't support multiple parameter lists designed to encourage type inference and syntax. For example, Monod.instance might be defined as:
def instance[A](id: A)(append: (A, A) => A): Monoid[A]

This definition works nicer with Scala's type inferencing rules. E.g., in Monoid.instance(0)(_ + _), A is correctly inferred as Int whereas in Monoid.instance(0, _ + _), Scala is not able to infer that the second param is a Function2[Int, Int, Int].
I don't think this is a blocker though -- folks can always define their own instance methods that are hand crafted to these types of situations. Just an observation.

To me, Monoid.instance[Int](...) is preferrable anyway rather than leaving off the type parameter. In Haskell you also declare it explicitly: instance Monoid Int where.

Found myself wanting something like this today. Makes sense when creating instances for typeclasses that have only one abstract method on them.
@typeclass trait UrlEncoded[T] {
  def urlEncoded(t: T): String
}

could helpfully generate
   def instance[T](body: T => String): UrlEncoded[T] = new UrlEncoded[T] {
     override def urlEncoded(t: T): String = body(t)
   }

on the companion.
// @SystemFw

I'm wondering if there is something even more radical we could do. Imagine "first class instances" on the companion like this (for a typeclass F with a single abstract method that returns B)
def instance(f: Foo): B = ...

that would get converted into
implicit val FFoo: F[Foo] = instance(f => ... )

(or the longer form without instance).
I'm not sure if the @typeclass on the trait would allow for messing with the companion, or if the overloading of methods named instance would get as far as the macro stage... but it'd be lovely if it worked!


SPARK-6861 [BUILD] [WIP] Scalastyle config prevents building Maven child modules alone

Move scalastyle-config.xml to dev/ (SBT config still doesn't work) to fix running mvn targets from subdirs; make scalastyle a verify stage target again in Maven; output results in target not project root; update to scalastyle 0.7.0

Test build #30077 has started for   PR 5471 at commit c4fb42c.

Test build #30077 has finished for   PR 5471 at commit c4fb42c.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.
This patch removes the following dependencies:

RoaringBitmap-0.4.5.jar
activation-1.1.jar
akka-actor_2.10-2.3.4-spark.jar
akka-remote_2.10-2.3.4-spark.jar
akka-slf4j_2.10-2.3.4-spark.jar
aopalliance-1.0.jar
arpack_combined_all-0.1.jar
avro-1.7.7.jar
breeze-macros_2.10-0.11.2.jar
breeze_2.10-0.11.2.jar
chill-java-0.5.0.jar
chill_2.10-0.5.0.jar
commons-beanutils-1.7.0.jar
commons-beanutils-core-1.8.0.jar
commons-cli-1.2.jar
commons-codec-1.10.jar
commons-collections-3.2.1.jar
commons-compress-1.4.1.jar
commons-configuration-1.6.jar
commons-digester-1.8.jar
commons-httpclient-3.1.jar
commons-io-2.1.jar
commons-lang-2.5.jar
commons-lang3-3.3.2.jar
commons-math-2.1.jar
commons-math3-3.1.1.jar
commons-net-2.2.jar
compress-lzf-1.0.0.jar
config-1.2.1.jar
core-1.1.2.jar
curator-client-2.4.0.jar
curator-framework-2.4.0.jar
curator-recipes-2.4.0.jar
gmbal-api-only-3.0.0-b023.jar
grizzly-framework-2.1.2.jar
grizzly-http-2.1.2.jar
grizzly-http-server-2.1.2.jar
grizzly-http-servlet-2.1.2.jar
grizzly-rcm-2.1.2.jar
groovy-all-2.3.7.jar
guava-14.0.1.jar
guice-3.0.jar
hadoop-annotations-2.2.0.jar
hadoop-auth-2.2.0.jar
hadoop-client-2.2.0.jar
hadoop-common-2.2.0.jar
hadoop-hdfs-2.2.0.jar
hadoop-mapreduce-client-app-2.2.0.jar
hadoop-mapreduce-client-common-2.2.0.jar
hadoop-mapreduce-client-core-2.2.0.jar
hadoop-mapreduce-client-jobclient-2.2.0.jar
hadoop-mapreduce-client-shuffle-2.2.0.jar
hadoop-yarn-api-2.2.0.jar
hadoop-yarn-client-2.2.0.jar
hadoop-yarn-common-2.2.0.jar
hadoop-yarn-server-common-2.2.0.jar
ivy-2.4.0.jar
jackson-annotations-2.4.0.jar
jackson-core-2.4.4.jar
jackson-core-asl-1.8.8.jar
jackson-databind-2.4.4.jar
jackson-jaxrs-1.8.8.jar
jackson-mapper-asl-1.8.8.jar
jackson-module-scala_2.10-2.4.4.jar
jackson-xc-1.8.8.jar
jansi-1.4.jar
javax.inject-1.jar
javax.servlet-3.0.0.v201112011016.jar
javax.servlet-3.1.jar
javax.servlet-api-3.0.1.jar
jaxb-api-2.2.2.jar
jaxb-impl-2.2.3-1.jar
jcl-over-slf4j-1.7.10.jar
jersey-client-1.9.jar
jersey-core-1.9.jar
jersey-grizzly2-1.9.jar
jersey-guice-1.9.jar
jersey-json-1.9.jar
jersey-server-1.9.jar
jersey-test-framework-core-1.9.jar
jersey-test-framework-grizzly2-1.9.jar
jets3t-0.7.1.jar
jettison-1.1.jar
jetty-util-6.1.26.jar
jline-0.9.94.jar
jline-2.10.4.jar
jodd-core-3.6.3.jar
json4s-ast_2.10-3.2.10.jar
json4s-core_2.10-3.2.10.jar
json4s-jackson_2.10-3.2.10.jar
jsr305-1.3.9.jar
jtransforms-2.4.0.jar
jul-to-slf4j-1.7.10.jar
kryo-2.21.jar
log4j-1.2.17.jar
lz4-1.2.0.jar
management-api-3.0.0-b012.jar
mesos-0.21.0-shaded-protobuf.jar
metrics-core-3.1.0.jar
metrics-graphite-3.1.0.jar
metrics-json-3.1.0.jar
metrics-jvm-3.1.0.jar
minlog-1.2.jar
netty-3.8.0.Final.jar
netty-all-4.0.23.Final.jar
objenesis-1.2.jar
opencsv-2.3.jar
oro-2.0.8.jar
paranamer-2.6.jar
parquet-column-1.6.0rc3.jar
parquet-common-1.6.0rc3.jar
parquet-encoding-1.6.0rc3.jar
parquet-format-2.2.0-rc1.jar
parquet-generator-1.6.0rc3.jar
parquet-hadoop-1.6.0rc3.jar
parquet-jackson-1.6.0rc3.jar
protobuf-java-2.4.1.jar
protobuf-java-2.5.0-spark.jar
py4j-<IP_ADDRESS>.jar
pyrolite-2.0.1.jar
quasiquotes_2.10-2.0.1.jar
reflectasm-1.07-shaded.jar
scala-compiler-2.10.4.jar
scala-library-2.10.4.jar
scala-reflect-2.10.4.jar
scalap-2.10.4.jar
scalatest_2.10-2.2.1.jar
slf4j-api-1.7.10.jar
slf4j-log4j12-1.7.10.jar
snappy-java-<IP_ADDRESS>.jar
spark-bagel_2.10-1.4.0-SNAPSHOT.jar
spark-catalyst_2.10-1.4.0-SNAPSHOT.jar
spark-core_2.10-1.4.0-SNAPSHOT.jar
spark-graphx_2.10-1.4.0-SNAPSHOT.jar
spark-launcher_2.10-1.4.0-SNAPSHOT.jar
spark-mllib_2.10-1.4.0-SNAPSHOT.jar
spark-network-common_2.10-1.4.0-SNAPSHOT.jar
spark-network-shuffle_2.10-1.4.0-SNAPSHOT.jar
spark-repl_2.10-1.4.0-SNAPSHOT.jar
spark-sql_2.10-1.4.0-SNAPSHOT.jar
spark-streaming_2.10-1.4.0-SNAPSHOT.jar
spire-macros_2.10-0.7.4.jar
spire_2.10-0.7.4.jar
stax-api-1.0.1.jar
stream-2.7.0.jar
tachyon-0.5.0.jar
tachyon-client-0.5.0.jar
uncommons-maths-1.2.2a.jar
unused-1.0.0.jar
xmlenc-0.52.jar
xz-1.0.jar
zookeeper-3.4.5.jar

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30077/
Test FAILed.

Does Maven, in general support building child modules independently without first doing a "mvn install" (which anyways requires doing the entire build)?

Yes, in general there's no special reason the children can't be built independently ... except of course that all of their dependencies must exist! and if they depend on SNAPSHOTs of other project modules, which aren't otherwise published, then you have to have install-ed them locally first.
I can see this being a legit use case if you were, say, iterating on a test / patch in a child module and just wanted to keep rebuilding it. You can do it from the top level with -pl project, but some expect to also be able to run from the module itself. At least, this was prompted by a tweet declaring Spark's Maven build so silly because this wasn't possible :)
Since this instance was fixable and appeared to be the only barrier to building (most) child modules, and I saw some other things to update, I figured I'd go for it. Individually none of these changes are that important. I may have to back out the file move here if SBT can't be updated to see it.

Thanks for fixing this @srowen. I am not familiar with the SBT system and the change looks good w.r.t maven (except that the file needs to be moved to the dev folder).
On a side note: My apologies for this silly mistake. Could you point me to the said tweet?

@rahulsinghaliitd no problem, it's not a usage pattern that is common in Spark. https://twitter.com/abayer/status/586660131045122049 but don't worry about that, I'm sure abayer isn't actually worked up about it.

I give up on moving scalastyle-config.xml. This sticks to updating the version, putting the output in target, and moving scalastyle to its default verify phase which addresses the original concern.

Test build #30241 has started for   PR 5471 at commit 35a4fd2.

Test build #30241 has finished for   PR 5471 at commit 35a4fd2.

This patch passes all tests.

This patch merges cleanly.

This patch adds no public classes.

This patch adds the following new dependencies:

commons-math3-3.4.1.jar

This patch removes the following dependencies:

commons-math3-3.1.1.jar

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30241/
Test PASSed.

Test build #30336 has started for   PR 5471 at commit acac637.

Test build #30336 has finished for   PR 5471 at commit acac637.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.
This patch does not change any dependencies.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30336/
Test PASSed.


Updated ME16 more like ME32 than ME1 with lots of tests. Passes all p…

Updated ME16 more like ME32 than ME1 with lots of tests. Passes all previous defined tests and a chunk of new ones stolen from ME32.
TP2000-389 - Update ME16 to look at hierarchy and history

TP2000-389
The ME16 rule definition is poorly worded and vague, it was interpreted in its most basic way initially, but after scrutiny by Steve, Matt Lavis and myself, we have concluded that this was incorrect (and probably a friday night job) as far as the TARIC3 spec is concerned. The initial implementation respected the wording but not the underlying intent. This update addresses the underlying intent of the rule, very similar to ME32 but checking for the inverted presence of an additional code.

Why
Why is this change happening, e.g. goals, use cases, stories, etc.?

Update ME16 to correctly check for matching measures in in the hierarchy but for one difference, if the additional code for the measure being verified is null, that a measure clashes but with an additional code (and the reverse),
More testing added around the rule
Existing tests still pass

What
What is this PR doing, e.g. implementations, algorithms, etc.?

Update ME16 to be like ME32 but invert the check for the presence of additional code for the measure being validated

Checklist

Requires migrations? No
Requires dependency updates? No

Links to relevant material
See: Jira Ticket

Codecov Report
Patch coverage: 94.39% and project coverage change: +0.02 :tada:

Comparison is base (677be35) 92.26% compared to head (a52e279) 92.28%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #863      +/-   ##
==========================================
+ Coverage   92.26%   92.28%   +0.02%
==========================================
  Files         412      413       +1
  Lines       27335    27422      +87
  Branches     2051     2058       +7
==========================================
+ Hits        25221    25307      +86
  Misses       1725     1725
- Partials      389      390       +1

Impacted Files
Coverage Δ

measures/tests/test_business_rules.py
95.77% <ø> (-0.06%)
:arrow_down:

measures/business_rules.py
89.07% <84.61%> (+0.34%)
:arrow_up:

measures/tests/test_buinsess_rules/test_ME16.py
97.53% <97.53%> (ø)

... and 1 file with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


(Groups) Kicking a requested user returns an error

Description
(Groups) Kicking a requested user returns an error "Could not kick user from group"
and on server logs:
"Could not kick user from group uid=5c96570d-726e-42e7-966c-ac7da2a4ac1a sid=32c36d15-b34c-4876-8283-ed779c5118c8 cid=d5705895-ba36-473d-b636-78f4ee77f409 group_id=57b38f3b-deb1-4ae5-ad02-ada89bc2ea9a user_id=fa6226f2-2a71-403a-ae67-9ce20d44769e error=pq: failed to satisfy CHECK constraint (count >= 0) stacktrace=goroutine 7397"
Steps to Reproduce
Kick a user that has requested to join the group but isn't a member yet
Expected Result
remove the "join" user from the list of users
Actual Result
the requested user remains on the list
Context

[ ] Unity

Your Environment
name=nakama-a751 version=0.12.1+aab3e2e
cockroach version {"Version":1}

Looks like an issue in group kick where we incorrectly attempt to adjust the group member count when we remove a join request, which is not counted as part of the group count field.
This has also highlighted a separate issue where groups are incorrectly created with an initial count of 0, instead of 1 to reflect the group creator as the first member. This does not affect core group functionality but should be fixed to show an accurate member count.

@GPK42 This is fixed in c6e3521a8590223be16cf08a0eaf2d1114be9afb and will be part of the next release of Nakama.


sudo dnf install crystal results in error

sudo dnf install crystal results in error
sudo dnf install crystal
[sudo] password for heinrich:
Last metadata expiration check: 0:06:58 ago on Fri 12 Jul 2024 12:48:04 PM EDT.
Error:
 _Problem: package crystal-1.13.0-1.1.x86_64 from devel_languages_crystal requires crystal1.13, but none of the providers can be installed
  - package crystal-1.13.0-1.1.x86_64 from devel_languages_crystal obsoletes crystal1.13 provided by crystal1.13-1.13.1-1.1.x86_64 from devel_languages_crystal
  - conflicting requests_
(try to add '--skip-broken' to skip uninstallable packages)

I posted this issue to the AskFedora group and it was suggested this is an issue on the Crystal Repo Maintainers side.

Can you post what distro and version you are using?

Sorry, I should have posted this:
Fedora 40
Kernel: 6.9.8-200.fc40.x86_64
GNOME 46.3.1
Intel i9-12900KF
NVIDIA GeForce RTX 3080 Ti

Looks like it does work, although there appears to be a warning, if you do a dnf install crystal1.13 and is functional.

That is correct, if I do dnf install crystal1.13 it works. Is this the expected method to install Crystal? If yes, perhaps update the documentation.
Shall I mark this issue closed?

Looks like the version for crystal was wrong: 1.13.0 instead of 1.13.1 which causes the install failure. Should be fixed in a moment.

I don't think it's been fixed yet, but I will test it again tomorrow.
sudo dnf install crystal
[sudo] password for heinrich:
\\Last metadata expiration check: 0:59:07 ago on Sat 13 Jul 2024 05:03:14 PM EDT.
Error:
 Problem: package crystal-1.13.1-1.2.x86_64 from devel_languages_crystal requires crystal1.13, but none of the providers can be installed
  - package crystal-1.13.1-1.2.x86_64 from devel_languages_crystal obsoletes crystal1.13 provided by crystal1.13-1.13.1-1.1.x86_64 from @System
  - package crystal-1.13.1-1.2.x86_64 from devel_languages_crystal obsoletes crystal1.13 provided by crystal1.13-1.13.1-1.1.x86_64 from devel_languages_crystal
  - conflicting requests
(try to add '--skip-broken' to skip uninstallable packages)

sudo dnf install crystal
Last metadata expiration check: 3:09:26 ago on Tue 16 Jul 2024 06:08:12 AM EDT.
Dependencies resolved.
==========================================================================================================================================================
 Package                         Architecture                   Version                             Repository                                       Size
==========================================================================================================================================================
Installing:
 crystal                         x86_64                         1.13.1-2.1                          devel_languages_crystal                         9.0 k

Transaction Summary
==========================================================================================================================================================
Install  1 Package

Total download size: 9.0 k
Installed size: 82
Is this ok [y/N]: y
Downloading Packages:
crystal-1.13.1-2.1.x86_64.rpm                                                                                28 kB/s | 9.0 kB     00:00
----------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                28 kB/s | 9.0 kB     00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                1/1
  Installing       : crystal-1.13.1-2.1.x86_64                                                                                1/1

Installed:
  crystal-1.13.1-2.1.x86_64

Complete!

Works like a champ!

Test on centos 9 stream, but not ok.
[root@VM-0-10-centos yum.repos.d]# curl -fsSL https://crystal-lang.org/install.sh | sudo bash
Repository baseos is listed more than once in the configuration
Repository baseos-source is listed more than once in the configuration
Repository appstream is listed more than once in the configuration
Repository appstream-source is listed more than once in the configuration
Repository rt is listed more than once in the configuration
Repository rt-source is listed more than once in the configuration
Repository resilientstorage is listed more than once in the configuration
Repository resilientstorage-source is listed more than once in the configuration
Crystal (CentOS_9)                                                               598  B/s | 1.1 kB     00:01
Errors during downloading metadata for repository 'crystal':
  - Status code: 404 for https://download.opensuse.org/repositories/devel:/languages:/crystal/CentOS_9/repodata/repomd.xml (IP: <IP_ADDRESS>)
Error: Failed to download metadata for repo 'crystal': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

It is broken by new created /etc/yum.repos.d/crystal.repo.
[root@VM-0-10-centos yum.repos.d]# cat crystal.repo
[crystal]
name=Crystal (CentOS_9)
type=rpm-md
baseurl=https://download.opensuse.org/repositories/devel:/languages:/crystal/CentOS_9/
gpgcheck=1
gpgkey=https://download.opensuse.org/repositories/devel:/languages:/crystal/CentOS_9/repodata/repomd.xml.key
enabled=1

I think that's just because there isn't a Cent OS 9 binary available, rather than the issue described in the OP. Possibly just need to enable it?
\cc @straight-shoota

Still not available in CentOS 9 stream, should we enable this issue?

I poked around OBS and found what I think is the right place and there is no Cent OS 9 Stream option. So either I'm looking in the wrong place, or it's not supported at the moment. Johannes is currently on holiday, so will have to wait to see what he says when he gets back.

I never use package manager for install Crystal, because i always compile it myself.
But, i consider this is a very very bad experience if i following the official document and run curl -fsSL https://crystal-lang.org/install.sh | sudo bash, then, it broken my package manager unexpected!  It broken the user's centos and even prevented the user from installing any other packages, for newbies, probably don't know how to fix this.

CentOS Stream 9 is not a supported build target for OBS (https://en.opensuse.org/openSUSE:Build_Service_supported_build_targets). So unfortunately, there's nothing we can do except from setting up a separate build pipeline for this system. This is certainly not a priority for the project at this point. We offer readly-usable packages for various platforms on a best-effort basis. We cannot cover all possible targets where Crystal can run.
We could however consider tweaking the installer script to recognize CentOS Stream 9 and issue an error message.


Only count pages with 'new' id type when returning pageviewer page count

The format of the id used for pages in ES and S3 changed in https://github.com/guardian/pfi/pull/884 and https://github.com/guardian/pfi/pull/886 to be
{documentHash}-{pageNumber}

We use the pageCount endpoint to determine whether to use the page viewer or fall back to the old text/OCR/preview viewer. This means that documents with the old id would have a non-zero page count, but fetching the page preview or data by id would fail because of the different id format.
This PR changes the pageCount endpoint to return 0 pages for documents with the old id, which will tell the frontend to fall back to the old viewer.
Examples of old-style and new-style ES pages docs:
Old id
{
  "_index": "pfi-pages-text",
  "_type": "_doc",
  "_id": "_z1TYXIBKUVvS4VMPvmz",
  "_version": 1,
  "_score": 0,
  "_source": {
    "resourceId": "TbzD1ZdaGDFcVduonjP_3BWM-lpMrU1H4xKS61qqr5QO4giJYZxZDNqJEs7O-qTzbIa1GTtYJ_rDfW1WKKjrEA",
    "dimensions.width": 1190.52001953125,
    "dimensions.height":<PHONE_NUMBER>101562,
    "dimensions.bottom":<PHONE_NUMBER>101562,
    "page": 1,
    "value": {
      "english": "OFFICIAL: Potential impact of behavioural and social interventions on a Covid-19 epidemic in the UK – 9 March 2020 \n \nOFFICIAL \n \nPotential impact of behavioural and social interventions on an epidemic of Covid-19 in the UK \n \nPurpose: \n1. This paper outlines the available scientific evidence base around behavioural and social interventions (previously referred to as non-pharmaceutical interventions) that could be applied as part \nof the HMG response to a UK epidemic of Covid-19, including the expected impacts on the spread of the virus and public behaviours. The note does not cover economic, operational or policy \nconsiderations. \n \n2. SAGE advises that a combination of individual home isolation of symptomatic cases, household isolation and social distancing of the over 70s1 could have a positive effect on: delaying the \nonset of the peak; reducing the number of cases during the peak; and reducing the total number of cases.  Any decision must consider the impacts these interventions may have on society, on \nindividuals, the workforce and businesses, and the operation of Government and public services.  \n \nBackground: \n3. In the event of a severe epidemic, the NHS will be unable to meet all demands placed on it. In the reasonable worst-case scenario, demand on beds is likely to overtake supply well before the \npeak is reached.  \n \n4. There are a range of behavioural and social interventions which are evidenced as having been effective in responding to historic epidemics. These interventions are well understood by the \npublic and have been enacted in other countries.  Modelling suggests as compliance drops so does impact, but there is no major inflexion point at which a drop in compliance leads to a \ndisproportionate drop in effect. \n \n5. Applying these interventions could be helpful in containing an epidemic to some degree or changing the shape of the epidemiological curve, see figure 1, potentially making the response of the \nNHS and other sectors more sustainable. The objectives of these interventions could be to:  \n1. Contain the outbreak so that it does not become an epidemic; \n2. Delaying the peak so it occurs when the NHS is out of Winter pressures;  \n3. Reducing the size of and/or extending the peak so that the response by the NHS and other sectors can be maintained more sustainably; and  \n4. Reducing the total number of deaths by limiting the number of cases in vulnerable groups. \n \n6. Any intervention would need to be Government policy for a significant duration (2-3 months) in order to see the benefit, as removing and/or relaxing the intervention too early could result in a \nnew outbreak and potentially extend transmission of the virus into Winter 2020.  However, the timescale for this are uncertain and would need to be kept under review to provide confidence \nthat these are in place to sufficiently cover the peak of the outbreak. \n \n7. SAGE advises that measures relating to individual and household isolation will likely need to be enacted within the next two weeks to be fully effective, and those concerning social distancing \nof the elderly and vulnerable 2-3 weeks after this.  However, the triggers for individual and household isolation could be met earlier depending on the progress of the outbreak in the UK.  CMO, \nGCSA and PHE will review case numbers daily to advise further on the meeting of any trigger points.  \n \n \n \n \n \n1 To be discussed and agreed by SAGE on 10 March as a change from over 65’s \n"
    },
    "dimensions.top": 0
  },
  "highlight": {
    "resourceId": [
      "@kibana-highlighted-field@TbzD1ZdaGDFcVduonjP_3BWM-lpMrU1H4xKS61qqr5QO4giJYZxZDNqJEs7O-qTzbIa1GTtYJ_rDfW1WKKjrEA@/kibana-highlighted-field@"
    ]
  }
}

New id
{
  "_index": "pfi-pages-text",
  "_type": "_doc",
  "_id": "R78l_QnuESe0S0aGZ_N2ki7eSJyHeiGo-EKYxPUBlPgyYf6rE8Up2i9osL8-ZmAjppBv7QnA8ZgbYFq5Pd_J_w-1",
  "_version": 2,
  "_score": 0,
  "_source": {
    "resourceId": "R78l_QnuESe0S0aGZ_N2ki7eSJyHeiGo-EKYxPUBlPgyYf6rE8Up2i9osL8-ZmAjppBv7QnA8ZgbYFq5Pd_J_w",
    "dimensions.width":<PHONE_NUMBER>445312,
    "page": 1,
    "dimensions.top": 0,
    "dimensions.bottom":<PHONE_NUMBER>140625,
    "dimensions.height":<PHONE_NUMBER>140625,
    "value": {
      "english": "C/) \nPaul Chiusano \nRunar Bjarnason \nIga MANNING Foreword by Martin Odersky\n"
    }
  },
  "highlight": {
    "resourceId": [
      "@kibana-highlighted-field@R78l_QnuESe0S0aGZ_N2ki7eSJyHeiGo-EKYxPUBlPgyYf6rE8Up2i9osL8-ZmAjppBv7QnA8ZgbYFq5Pd_J_w@/kibana-highlighted-field@"
    ]
  }
}

Have tested on Playground

Seems like a reasonable approach for the time being - if possible it would be nice to clear out old stuff or something so we can have a single query in the future!

Agreed - once we re-process all old docs, we won't need this check