Symfony 4 support

Symfony 4 supports bundle-less applications (and it's considered best-practice).
The configure command from this bundle doesn't work with this at the moment so I don't know how to initialize graphql in my project.

Also the explorer doesn't work. It throws this error:
Unable to find template "GraphQLBundle:Feature:explorer.html.twig" (looked into: <projectPath>/templates, <projectPath>/vendor/symfony/twig-bridge/Resources/views/Form).

Having exactly the same issue, followed the instructions on the readme.

I got it working by adding the following to my config.yml:
framework:
templating:
engines: ['twig']

r.e. the configure command not working, it looks like the bundle is using functionality which was deprecated in Symfony 3.4 and removed in 4.0:

[2018-02-14 10:01:47] php.INFO: User Deprecated: Auto-registration of the command "Youshido\GraphQLBundle\Command\GraphQLConfigureCommand" is deprecated since Symfony 3.4 and won't be supported in 4.0. Use PSR-4 based service discovery instead. {"exception":"[object] (ErrorException(code: 0): User Deprecated: Auto-registration of the command "Youshido\GraphQLBundle\Command\GraphQLConfigureCommand" is deprecated since Symfony 3.4 and won't be supported in 4.0. Use PSR-4 based service discovery instead. at /Users/gareth.jones/Github/driveplan/driveplan-api/vendor/symfony/symfony/src/Symfony/Component/HttpKernel/Bundle/Bundle.php:190)"} []

I'm also seeing the following deprecation warnings on Symfony 3.4 related to the GraphQLController when using service config:
graphql:
    schema_class: "AppBundle\\GraphQL\\Schema"
    logger: "@logger"

and parameters
graphql.execution_context.class: AppBundle\GraphQL\ExecutionContext

1x: Autowiring services based on the types they implement is deprecated since Symfony 3.3 and won't be supported in version 4.0. You should alias the "AppBundle\GraphQL\Schema" service to "Youshido\GraphQL\Schema\AbstractSchema" instead.

1x: Checking for the initialization of the "graphql.schema" private service is deprecated since Symfony 3.4 and won't be supported anymore in Symfony 4.0.

1x: The "graphql.schema" service is private, replacing it is deprecated since Symfony 3.2 and will fail in 4.0.

1x: The "graphql.processor" service is private, getting it from the container is deprecated since Symfony 3.2 and will fail in 4.0. You should either make the service public, or stop using the container directly and use dependency injection instead.

In order to make this work in sf4 you need to:

Install the templating component
 composer require templating

Configure the templating in your framework.yml as mentioned in https://github.com/Youshido/GraphQLBundle/issues/68#issuecomment-362233618

Manually create a schema file in src/GraphQL/Schema.yml with the following content:

<?php

declare (strict_types=1);

namespace App\GraphQL;

use Youshido\GraphQL\Schema\AbstractSchema;
use Youshido\GraphQL\Config\Schema\SchemaConfig;
use Youshido\GraphQL\Type\Scalar\StringType;

final class Schema extends AbstractSchema
{
    public function build(SchemaConfig $config)
    {
        $config->getQuery()->addFields([
            'hello' => [
                'type'    => new StringType(),
                'args'    => [
                    'name' => [
                        'type' => new StringType(),
                        'defaultValue' => 'Stranger'
                    ]
                ],
                'resolve' => function ($context, $args) {
                    return 'Hello ' . $args['name'];
                }
            ]
        ]);
    }
}

create a file graphql.yaml in your config folder with the following contents:

graphql:
  schema_class: 'App\GraphQL\Schema'

fix the service related issues by re-declaring the services in my services.yml as following:

    graphql.schema:
      public: true
      synthetic: true

    graphql.processor:
      public: true
      class: Youshido\GraphQLBundle\Execution\Processor
      arguments:
        $executionContext<EMAIL_ADDRESS>
Perhaps I'll run into some other issues further on, but I'll try to document them here

In case someone else encounters the same:
After following the steps layed out by mvriel i got a "Schema class  does not exist" error message.
The fix is to create the file at point 4 in the config/packages folder instead of the config folder.

Also it would be great if you create recipe for automate installation

Thanks to these comments I could build GraphQL Demo for Symfony 4 which is a minimum set up to show how to use GraphQLBundle in Symfony 4 projects.


How to enable rules in a disabled group?

For example, I'd like to enable
- group:
    name: extra
    enabled: false
    rules:
    - warn: {lhs: fmap concat (forM a b), rhs: concatForM a b}

Adding - warn: {name: "Use concatForM"} to my .hlint.yaml doesn't seem to work.
Also is there a way to enable the whole group?

--with-group=extra works. I run --with-group=extra --with-group=future on all my hints.
I'm not a massive fan of the UI around groups. I'd really like to have things like extra be conditional on use of the extra library - so if you show you are using the extra library in a module by importing something, it gives you the hints for free. That requires getting a handle on the scoping data type.

That does sound like a desirable feature - I'll leave the ticket open for it then.

(Unrelated to the desirable feature you mention, for future people looking at this "How to enable rules in a disabled group?" issue)

Also is there a way to enable the whole group?

--with-group=extra works

One can also add - group: {name: extra, enabled: true} to .hlint.yaml

@brprice thanks! I hadn't remembered that, so have switched HLint itself to do that: 3111617d64c20ce0701cd14f69d60c32f8a7326c
I've documented this feature in the FAQ as part of 93b3d33e10996596b0a5b5b4155cdc5f45107a70


High-memory consumption in AMG

The AMG needs a lot of memory, which can lead to severe drops in performance, especially when using MPS. See #183 for details.
The underlying reason is that we store all the masks that are computed from the point grid in initialize, and then only filter out low quality masks (measured by the predicted IOU score and the stability score) in generate.
While the masks are stored in RLE after initialize (and so don't need much storage), they are represented as binary masks in the intermediate computation steps. So by default there will be 32 x 32 x 3 masks. (32 is the default number of points per dimension, and for each points 3 masks will be predicted). Hence they take up some memory, and the computations run on top (computing the stability scores, RLEs etc.) will take up some more.
There are two possible ways to decrease this memory requirement:

See which of the intermediate computation takes up most memory and look into more memory efficient implementations (for this we would need to run memory profiling; I am not so familiar with the best practices for this in python)
Discard masks of low quality early. I.e. already include thresholds for the iou_score and stability_score (set to a fairly conservative value that keeps all relevant masks) in initialize and use it to pre-filter.

cc @GenevieveBuckley

Do you know any tool for memory line profiling?

I have done a little bit of work with memory profiling before, but not lots.

memray - this is a new project I haven't used before, and it looks really interesting.

I also think it's cool that there is a pytest-memray plugin. I'm not sure if it's a good fit for micro-sam, but I haven't seem memory profilers hooked into pytest in this way before, and it seems like a useful thing to have in some circumstances.

Fil - I have used this before, but don't remember a lot about it. Here is an example, and the Fil blogpost.
memory-profiler - I've used this before, although a note was added last year to the README to say it is no longer actively maintained. So it might be better to use a newer option like memray instead if we can.
guppy3 - might also be an option, I've never used it before.

I'll take a quick look at it today or tomrrow

I looked into this a bit more and tried 3 things:

implementing the RLE computation in numba in #190. This is slower than the current implementation, but may save memory (I haven't profiled memory yet).
filtering unlikely masks in initialize in #191. This does not affect runtime signifcantly, but may save memory (again not yet mem profiled).
implementing the RLE computation in C++. This does not negatively affect runtime and may also save memory (I would assume that the memory saving is the same as the numba implementation).

@GenevieveBuckley could you check if options 1 or 2 actually save memory? Depending on this we can see how to continue. Note that 3 cannot be tested yet because the C++ code is unreleased, but I think we can just extrapolate the results from 3 to it (and I have put the usage of it in comments in #190).
Thanks!

I ran memray on a single iteration of the amg benchmark from the benchmark.py file.
I emailed Constantin a copy of the output .bin files yesterday. Looking at the flamegraphs, I think the biggest memory consumer is loading the model itself, and the next biggest is the mask_to_rle computations.
Here are the memory profiler summary stats from the current dev branch:

vit_b (click to expand)
memray stats memray-output-detailed-amg-vit_b.bin
📏 Total allocations:
	54383659

📦 Total memory allocated:
	8.280GB

📊 Histogram of allocation size:
	min: 0.000B
	-----------------------------------------------
	< 6.000B   :    75669 ▇
	< 37.000B  : 17694397 ▇▇▇▇▇▇▇▇▇▇▇▇▇
	< 225.000B : 35440641 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇
	< 1.344KB  :   933491 ▇
	< 8.187KB  :   180202 ▇
	< 49.868KB :    49047 ▇
	< 303.748KB:     6924 ▇
	< 1.807MB  :     3151 ▇
	< 11.005MB :      116 ▇
	<=67.031MB :       21 ▇
	-----------------------------------------------
	max: 67.031MB

📂 Allocator type distribution:
	 PYMALLOC_MALLOC: 49356617
	 MALLOC: 2661871
	 PYMALLOC_REALLOC: 928414
	 CALLOC: 634006
	 POSIX_MEMALIGN: 464982
	 PYMALLOC_CALLOC: 321287
	 REALLOC: 16472
	 MMAP: 10

🥇 Top 5 largest allocating locations (by size):
	- mask_to_rle_pytorch:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:91 -> 1.048GB
	- mask_to_rle_pytorch:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:84 -> 1.047GB
	- _deepcopy_list:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/copy.py:206 -> 878.503MB
	- deepcopy:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/copy.py:138 -> 877.428MB
	- deepcopy:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/copy.py:144 -> 877.428MB

🥇 Top 5 largest allocating locations (by number of allocations):
	- deepcopy:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/copy.py:138 -> 12778469
	- deepcopy:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/copy.py:144 -> 12778467
	- deepcopy:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/copy.py:137 -> 12778467
	- _compile_bytecode:<frozen importlib._bootstrap_external>:672 -> 1352323
	- mask_to_rle_pytorch:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:96 -> 971485

vit_l (click to expand)
memray stats memray-output-amg-vit_l.bin
📏 Total allocations:
	4365278

📦 Total memory allocated:
	7.470GB

📊 Histogram of allocation size:
	min: 0.000B
	----------------------------------------------
	< 6.000B   :   76036 ▇
	< 37.000B  : 1903764 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇
	< 225.000B : 1414510 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇
	< 1.344KB  :  720471 ▇▇▇▇▇▇▇▇▇▇
	< 8.187KB  :  184257 ▇▇▇
	< 49.868KB :   55143 ▇
	< 303.748KB:    7500 ▇
	< 1.807MB  :    3364 ▇
	< 11.005MB :      68 ▇
	<=67.031MB :     165 ▇
	----------------------------------------------
	max: 67.031MB

📂 Allocator type distribution:
	 MALLOC: 2989224
	 CALLOC: 736168
	 POSIX_MEMALIGN: 466225
	 REALLOC: 173474
	 MMAP: 187

🥇 Top 5 largest allocating locations (by size):
	- load_tensor:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/serialization.py:1357 -> 1.164GB
	- __init__:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/modules/linear.py:96 -> 1.140GB
	- mask_to_rle_pytorch:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:84 -> 1.047GB
	- mask_to_rle_pytorch:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:91 -> 1.047GB
	- _deepcopy_list:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/copy.py:206 -> 1.013GB

🥇 Top 5 largest allocating locations (by number of allocations):
	- <stack trace unavailable> -> 358211
	- forward:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/modules/linear.py:114 -> 311930
	- cat:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/segment_anything/utils/amg.py:64 -> 254343
	- _call_with_frames_removed:<frozen importlib._bootstrap>:241 -> 206986
	- layer_norm:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/functional.py:2543 -> 160940

vit_h (click to expand)
```
memray stats memray-output-amg-vit_h.bin
📏 Total allocations:
	4439043
📦 Total memory allocated:
10.015GB
📊 Histogram of allocation size:
min: 0.000B
----------------------------------------------
< 6.000B   :   78229 ▇▇
< 37.000B  : 1938693 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇
< 225.000B : 1445049 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇
< 1.344KB  :  733839 ▇▇▇▇▇▇▇▇▇▇
< 8.187KB  :  180472 ▇▇▇
< 49.868KB :   48912 ▇
< 303.748KB:   10168 ▇
< 1.807MB  :    3384 ▇
< 11.005MB :      84 ▇
<=67.031MB :     213 ▇
----------------------------------------------
max: 67.031MB
📂 Allocator type distribution:
MALLOC: 3024885
CALLOC: 781845
POSIX_MEMALIGN: 466965
REALLOC: 165160
MMAP: 188
🥇 Top 5 largest allocating locations (by size):
- load_tensor:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/serialization.py:1357 -> 2.388GB
- init:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/modules/linear.py:96 -> 2.359GB
- _deepcopy_list:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/copy.py:206 -> 1.065GB
- mask_to_rle_pytorch:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:84 -> 1.047GB
- mask_to_rle_pytorch:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:91 -> 1.047GB
🥇 Top 5 largest allocating locations (by number of allocations):
-  -> 365821
- forward:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/modules/linear.py:114 -> 320654
- cat:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/segment_anything/utils/amg.py:64 -> 254397
- _call_with_frames_removed::241 -> 208003
- layer_norm:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/functional.py:2543 -> 164208

</details>

Thanks for posting it here too @GenevieveBuckley. If I have some time later I will see if I get to run the mem profiling to run myself to also compare this for #190 and #191. (But it would still be great if you can run the profilers for those as well at some point :))

I left this comment on the rle-numba PR, but since it refers to all the current branches involving memory use, I'm copying the contents here too.

Unfortunately the current attempt is slower than the previous implementation:

That's interesting, I don't see the same results.
On my machine, the rle-numba branch is slightly faster than the current dev branch. But the prefilter amg branch with the vit_h model did exceed the available 16GB memory, which is why I think we see a slowdown with the mps backend there. I imagine if more memory was available it wouldn't be slower.
3 iterations of the amg benchmark function, average runtime reported:

model
device
branch
benchmark
runtime

vit_h
cpu
dev
amg
37.7422

vit_h
mps
dev
amg
33.8933

vit_h
cpu
rle-numba
amg
35.9172

vit_h
mps
rle-numba
amg
33.4212

vit_h
cpu
prefilter-amg
amg
33.2118

vit_h
mps
prefilter-amg
amg
43.8554

vit_h
cpu
numba+prefilter
amg
34.6391

vit_h
mps
numba+prefilter
amg
42.9275

*The numba+prefilter branch here is me merging PR Filter unlikely masks in AMG #191  and Implement RLE computation with numba #190 just to see what the combination would do.

Here are the memray summary statistics for the amg memory benchmarks.
Details: dev branch

Details: memray stats memray-output-amg-vit_b.bin

📏 Total allocations:
	4218946

📦 Total memory allocated:
	5.585GB

📊 Histogram of allocation size:
	min: 0.000B
	----------------------------------------------
	< 6.000B   :   72175 ▇
	< 37.000B  : 1846375 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇
	< 225.000B : 1362981 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇
	< 1.344KB  :  697829 ▇▇▇▇▇▇▇▇▇▇
	< 8.187KB  :  180139 ▇▇▇
	< 49.868KB :   49049 ▇
	< 303.748KB:    6924 ▇
	< 1.807MB  :    3337 ▇
	< 11.005MB :     116 ▇
	<=67.031MB :      21 ▇
	----------------------------------------------
	max: 67.031MB

📂 Allocator type distribution:
	 MALLOC: 2929856
	 CALLOC: 659165
	 POSIX_MEMALIGN: 465115
	 REALLOC: 164614
	 MMAP: 196

🥇 Top 5 largest allocating locations (by size):
	- mask_to_rle_pytorch:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:84 -> 1.047GB
	- mask_to_rle_pytorch:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:91 -> 1.047GB
	- _deepcopy_list:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/copy.py:206 -> 867.996MB
	- load_tensor:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/serialization.py:1357 -> 357.687MB
	- __init__:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/modules/linear.py:96 -> 339.159MB

🥇 Top 5 largest allocating locations (by number of allocations):
	- <stack trace unavailable> -> 341937
	- forward:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/modules/linear.py:114 -> 302202
	- cat:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/segment_anything/utils/amg.py:64 -> 248369
	- _call_with_frames_removed:<frozen importlib._bootstrap>:241 -> 206988
	- layer_norm:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/functional.py:2543 -> 152427

Details: memray stats memray-output-amg-vit_l.bin

📏 Total allocations:
	4365278

📦 Total memory allocated:
	7.470GB

📊 Histogram of allocation size:
	min: 0.000B
	----------------------------------------------
	< 6.000B   :   76036 ▇
	< 37.000B  : 1903764 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇
	< 225.000B : 1414510 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇
	< 1.344KB  :  720471 ▇▇▇▇▇▇▇▇▇▇
	< 8.187KB  :  184257 ▇▇▇
	< 49.868KB :   55143 ▇
	< 303.748KB:    7500 ▇
	< 1.807MB  :    3364 ▇
	< 11.005MB :      68 ▇
	<=67.031MB :     165 ▇
	----------------------------------------------
	max: 67.031MB

📂 Allocator type distribution:
	 MALLOC: 2989224
	 CALLOC: 736168
	 POSIX_MEMALIGN: 466225
	 REALLOC: 173474
	 MMAP: 187

🥇 Top 5 largest allocating locations (by size):
	- load_tensor:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/serialization.py:1357 -> 1.164GB
	- __init__:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/modules/linear.py:96 -> 1.140GB
	- mask_to_rle_pytorch:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:84 -> 1.047GB
	- mask_to_rle_pytorch:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:91 -> 1.047GB
	- _deepcopy_list:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/copy.py:206 -> 1.013GB

🥇 Top 5 largest allocating locations (by number of allocations):
	- <stack trace unavailable> -> 358211
	- forward:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/modules/linear.py:114 -> 311930
	- cat:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/segment_anything/utils/amg.py:64 -> 254343
	- _call_with_frames_removed:<frozen importlib._bootstrap>:241 -> 206986
	- layer_norm:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/functional.py:2543 -> 160940

Details: memray stats memray-output-amg-vit_h.bin

📏 Total allocations:
	4439043

📦 Total memory allocated:
	10.015GB

📊 Histogram of allocation size:
	min: 0.000B
	----------------------------------------------
	< 6.000B   :   78229 ▇▇
	< 37.000B  : 1938693 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇
	< 225.000B : 1445049 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇
	< 1.344KB  :  733839 ▇▇▇▇▇▇▇▇▇▇
	< 8.187KB  :  180472 ▇▇▇
	< 49.868KB :   48912 ▇
	< 303.748KB:   10168 ▇
	< 1.807MB  :    3384 ▇
	< 11.005MB :      84 ▇
	<=67.031MB :     213 ▇
	----------------------------------------------
	max: 67.031MB

📂 Allocator type distribution:
	 MALLOC: 3024885
	 CALLOC: 781845
	 POSIX_MEMALIGN: 466965
	 REALLOC: 165160
	 MMAP: 188

🥇 Top 5 largest allocating locations (by size):
	- load_tensor:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/serialization.py:1357 -> 2.388GB
	- __init__:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/modules/linear.py:96 -> 2.359GB
	- _deepcopy_list:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/copy.py:206 -> 1.065GB
	- mask_to_rle_pytorch:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:84 -> 1.047GB
	- mask_to_rle_pytorch:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:91 -> 1.047GB

🥇 Top 5 largest allocating locations (by number of allocations):
	- <stack trace unavailable> -> 365821
	- forward:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/modules/linear.py:114 -> 320654
	- cat:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/segment_anything/utils/amg.py:64 -> 254397
	- _call_with_frames_removed:<frozen importlib._bootstrap>:241 -> 208003
	- layer_norm:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/functional.py:2543 -> 164208
(test-micro-sam-mps-nightly) genevieb@192-168-1-102 development %

Details: prefilter-amg branch

Details: memray stats memray-output-prefilter-amg-vit_b.bin

📏 Total allocations:
	6686407

📦 Total memory allocated:
	4.289GB

📊 Histogram of allocation size:
	min: 0.000B
	----------------------------------------------
	< 5.000B   :   73906 ▇
	< 34.000B  : 3118725 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇
	< 203.000B : 2154728 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇
	< 1.167KB  : 1085524 ▇▇▇▇▇▇▇▇▇
	< 6.864KB  :  211312 ▇▇
	< 40.368KB :   37664 ▇
	< 237.376KB:    2179 ▇
	< 1.363MB  :    2231 ▇
	< 8.015MB  :      69 ▇
	<=47.131MB :      69 ▇
	----------------------------------------------
	max: 47.131MB

📂 Allocator type distribution:
	 MALLOC: 4825088
	 POSIX_MEMALIGN: 1029064
	 CALLOC: 715375
	 REALLOC: 116697
	 MMAP: 183

🥇 Top 5 largest allocating locations (by size):
	- mask_to_rle_pytorch:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:84 -> 662.692MB
	- mask_to_rle_pytorch:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:91 -> 662.630MB
	- load_tensor:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/serialization.py:1357 -> 358.687MB
	- __init__:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/modules/linear.py:96 -> 339.159MB
	- <stack trace unavailable> -> 180.832MB

🥇 Top 5 largest allocating locations (by number of allocations):
	- <stack trace unavailable> -> 936383
	- forward:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/modules/linear.py:114 -> 300182
	- cat:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/segment_anything/utils/amg.py:64 -> 256355
	- batched_mask_to_box:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:51 -> 256253
	- batched_mask_to_box:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:43 -> 250693

Details: memray stats memray-output-prefilter-amg-vit_l.bin

📏 Total allocations:
	7132819

📦 Total memory allocated:
	6.366GB

📊 Histogram of allocation size:
	min: 0.000B
	----------------------------------------------
	< 5.000B   :   78073 ▇
	< 35.000B  : 3339967 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇
	< 210.000B : 2305414 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇
	< 1.221KB  : 1132402 ▇▇▇▇▇▇▇▇▇
	< 7.260KB  :  226257 ▇▇
	< 43.175KB :   44831 ▇
	< 256.735KB:    3280 ▇
	< 1.491MB  :    2362 ▇
	< 8.865MB  :      68 ▇
	<=52.717MB :     165 ▇
	----------------------------------------------
	max: 52.717MB

📂 Allocator type distribution:
	 MALLOC: 5104480
	 POSIX_MEMALIGN: 1094798
	 CALLOC: 800617
	 REALLOC: 132744
	 MMAP: 180

🥇 Top 5 largest allocating locations (by size):
	- load_tensor:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/serialization.py:1357 -> 1.164GB
	- __init__:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/modules/linear.py:96 -> 1.140GB
	- mask_to_rle_pytorch:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:84 -> 710.521MB
	- mask_to_rle_pytorch:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:91 -> 710.460MB
	- _deepcopy_list:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/copy.py:206 -> 355.178MB

🥇 Top 5 largest allocating locations (by number of allocations):
	- <stack trace unavailable> -> 969540
	- forward:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/modules/linear.py:114 -> 308593
	- batched_mask_to_box:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:51 -> 297101
	- batched_mask_to_box:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:43 -> 284254
	- filter:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/segment_anything/utils/amg.py:49 -> 263420

Details: memray stats memray-output-prefilter-amg-vit_h.bin

📏 Total allocations:
	7838182

📦 Total memory allocated:
	8.850GB

📊 Histogram of allocation size:
	min: 0.000B
	----------------------------------------------
	< 5.000B   :   81028 ▇
	< 35.000B  : 3697177 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇
	< 211.000B : 2532510 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇
	< 1.233KB  : 1241268 ▇▇▇▇▇▇▇▇▇
	< 7.355KB  :  240018 ▇▇
	< 43.857KB :   40360 ▇
	< 261.478KB:    3139 ▇
	< 1.522MB  :    2385 ▇
	< 9.076MB  :      84 ▇
	<=54.114MB :     213 ▇
	----------------------------------------------
	max: 54.114MB

📂 Allocator type distribution:
	 MALLOC: 5610353
	 POSIX_MEMALIGN: 1236785
	 CALLOC: 862862
	 REALLOC: 128001
	 MMAP: 181

🥇 Top 5 largest allocating locations (by size):
	- load_tensor:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/serialization.py:1357 -> 2.388GB
	- __init__:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/modules/linear.py:96 -> 2.359GB
	- mask_to_rle_pytorch:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:84 -> 723.089MB
	- mask_to_rle_pytorch:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:91 -> 723.028MB
	- _deepcopy_list:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/copy.py:206 -> 253.180MB

🥇 Top 5 largest allocating locations (by number of allocations):
	- <stack trace unavailable> -> 1204709
	- batched_mask_to_box:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:51 -> 332085
	- batched_mask_to_box:/Users/genevieb/Documents/GitHub/micro-sam/micro_sam/_vendored.py:43 -> 320806
	- filter:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/segment_anything/utils/amg.py:49 -> 318677
	- forward:/Users/genevieb/mambaforge/envs/test-micro-sam-mps-nightly/lib/python3.10/site-packages/torch/nn/modules/linear.py:114 -> 318255

Hi @GenevieveBuckley,
I think this is resolved now after merging #190. And I think we have a consistent speed up with MPS now!
(If there is anything else here from your side, then feel free to reopen!)


"Title not Present"

Module Version: v1.0.9
Describe the bug
When chaining macros whether I use the id of the macro or the title it will not chain the macro's and will bring up this error.
Expected behavior
It should chain the macro's
Screenshots

Browser:
Foundry Desktop App
**Foundry Version: V11.315
**Game System: D&D 5E  V3.1.2
Additional context
Add any other context (like other modules installed) about the problem here.

Sorry for the delay i will checkout this.

Sadly it seem to work fine here the macro i tested:
game.modules.get("choices-plus").api.showChoices({
    title: "A dream...or something else?",
    text: "<p>Something has been haunting your sleep. Flickers of dreams..or visions, perhaps? A porcelain mask of an eerie face, one that looked familiar and strange, with a dueling scar bisecting one eye hole. Long corridors of twisting stones: solid one moment, then thin and gossamer as rising smoke the next. A room with five pedestals around a rotating dais. A tangled forest of thorny brush, twitching and pulsing like the veins in your neck.


lb ip is deleted accidentally in statedb when l2announce, requestip and shardingkeys are used together

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?

enable l2announce
create multiple services with same annotations io.cilium/lb-ipam-ips and io.cilium/lb-ipam-sharing-key
delete some services
execute cilium-dbg statedb l2-announce to see the ip is missing

Cilium Version
1.15.3
Kernel Version
5.15.0-102-generic
Kubernetes Version
v1.28.7+k3s1
Regression
Bug
Sysdump
https://drive.google.com/file/d/1C3Y-glpgfJ0efHDFfr5TqO5hI4KOkWar/view?usp=drive_link
Relevant log output
No response
Anything else?
When creating multiple services using the same IP, you can still see the IP listed multiple times in the statedb on multiple nodes.
Cilium Users Document

[X] Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

[X] I agree to follow this project's Code of Conduct

+1


feat(model): Add optional message field to auditstamp

Checklist

[ ] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
[ ] Links to related issues (if applicable)
[ ] Tests for the changes have been added/updated (if applicable)
[ ] Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
[ ] For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

Let's wait til we release new CLI version - this has potential to break the old actions framework because its not forwards compatible

closing in favor of https://github.com/datahub-project/datahub/pull/5611


Perpetually "Windows is loading files"

Hi,
I followed the instructions to start a IE10/Win7 VM on Ubuntu. I installed VirtualBox 5.0.24.
Windows is not starting. A black screen with a progress bar appears "Windows is loading files...", then blinks, and starts again. Any thoughts as to what is going wrong?
regards,
Paul

Hello @pwiegers
Looking at this steps? https://github.com/magnetikonline/linuxmicrosoftievirtualmachines#ie10---windows-7
Yeah I have seen this, but it's typically because I didn't setup the VM using the VirtualBox OVA export for the import process exactly as specified, since often Windows can get fussy with hardware types/etc.
Can you try another import of the OVA - not touching any hardware settings proposed? If you still get issues - let me know, I'll try on my end.
Pete

Hi Pete,
I tried several ones, but the error is always the same. I tried some different HW-settings, starting of course with the recommended settings... the problem appears to be the same any way I try...
regards,
Paul

Thanks @pwiegers - leave it with me, I'll give it a download and try out over the weekend.
Pete

Hey @pwiegers,
just gave this a go - as per the steps here: https://github.com/magnetikonline/linuxmicrosoftievirtualmachines#ie10---windows-7
Worked perfectly - see screenshot:

VirtualBox version: 5.0.24 r108355 (latest at time of writing)
Host OS version: Ubuntu 16.04LTS Linux magnetik.dev 4.4.0-28-generic #47-Ubuntu SMP Fri Jun 24 10:09:13 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
Just installed the OVA via import - the only settings I changed was the RAM to 1024MB, and the VM name to Windows 7 - IE10 to meet my pedantic naming conventions for VMs :smiley:

Hey @pwiegers - going to close this now.
If you keep having issues, let me know - but the old "it works on my machine" is at play here for me.


Add a range-lookup() XSLT function

Currently the lookup() function will assume the passed key is a string value to lookup in the key/value store.  If it can't find it it then  tries to convert the key into a long and if that works does a lookup in the range store.
For use cases where the person writing the XSLT (that uses the lookup) knows the data is all range keys it would be more performant to have a range-lookup() function that went directly to the range store.
lookup() could still be used for cased where the type of key in the store is unknown.

It could be argued that lookup should only lookup in the kv store to avoid the additional lookup in the range store (if the key is a long), but changing the behaviour would impact existing uses of the lookup function.
Another option would be to add a new overloaded arg to specify the mode, auto|key|range, auto behaves as currently, key just does and key lookup and range converts to long and does a range lookup.


v2.0 status?

Hi @dns2utf8,
I noticed your Release 2.0.0-alpha.1 commit on the 2.0 branch.
Are you planning to publish it on crates.io?  Do you have a timetable for 2.0?
Thanks!

Any update on this? :)

Hi @dns2utf8 ,
Sorry I didn't get back to you last year.  In my experience, having an alpha/beta suffix prevents auto-update when you publish it to crates.io.
So yeah, it'd be great if you can publish the alpha/beta, so the community can try it out and give you some feedback.


Support for Mastodon custom emojis?

Is there planned integration to show masto instances' local emojis inline and be able to post with them?

I didn't think about this yet... Just looked up https://emojos.in/mastodon.social
Want to implement this?

Looks like that's only the set from mastodon.social. Is there a way to grab custom emojis from each instance? Or even as a start, the instance that a user has their account on?

As I understand, if you manage to configure custom emojis in AndStatus somehow, only you will see them this way. Other users, including your friends will see other emojis in the same messages (or no emojis at all, like in AndStatus now).
?!

As documented here, Mastodon instances can include their own list of custom emoji. When AndStatus receives a Mastodon post or profile text with a custom emoji, it shows as the emoji's slug as defined by the instance owners. AndStatus does not currently support showing these emoji. For example:

These slugs can be translated to the proper image URL using the Mastodon API GET /api/v1/custom_emojis. Here's how my profile renders on the official Mastodon client:

Custom emoji support would be a great addition to AndStatus -- considering reopening this ticket and implementing custom emote support. I am considering implementing but I am not super well-versed in Kotlin.

Hi @kobitate!
Is absence of custom emojis the most annoying AndStatus problem for you?

Yes, I'm on an instance with a lot of custom emoji. When posts come across my feed they're often missing the emoji for users on the same instance.

I am considering implementing but I am not super well-versed in Kotlin.

Reading this thread again I see that turning this feature on will mean that:

AndStatus will have to download a separate set of images plus mappings from emoji codes to these images for each Mastodon instance, actors (users) or notes (posts, actions...) from which are present in the local app cache (present in some of timelines...). We need new data tables etc. to store them, ability to view and choose them in some screen...
For viewing each note should be transformed (emoji codes replaced with images) according to this note's origin's (instance, where it was created) set of emojis. So notes having the same emoji codes but created at different servers will look different in the same AndStatus timeline.
Currently text of a note cannot show images inside it. We need to learn how to start showing images inlined in text. Probably this is the easiest part...
We may expect performance degradation, not so smooth timeline scrolling due to above complexities.

@kobitate Regarding Kotlin in AndStatus.
AndStatus is being developed since 2009, and only in 2021 it was converted to Kotlin. So most of its code still looks very like Java :-)
I want to encourage you to implement this feature, but note that the main difficulty is not in coding, but in figuring out, where to make changes in existing code for each part of the change.
And above 4 points are only a part of what really needs to be changed/added to make this feature live.


Android - Concierge- Composed box looks expended and cursor is overlapping the text

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Launch the app
Log in with expensifail account
Search for Concierge
Check composed box and tap on it

Expected Result:
Composed box should be normal and cursor should not overlap with text
Actual Result:
Composed box looks expended and cursor is overlapping the text
Workaround:
Unknown
Platform:

Where is this issue occurring?

Android

Version Number: <IP_ADDRESS>
Reproducible in staging?: Yes
Reproducible in production?: Yes
Email or phone of affected tester (no customers): any
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation

https://user-images.githubusercontent.com/93399543/192596594-a1adc5ed-bac8-4089-bc15-f5a5fe54310b.mp4
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
View all open jobs on GitHub

hmmm, I would believe this is related to the screen size being too small?
cc @Expensify/design what do you think of this? What should happen in this case if the phone is too narrow?

Hmm maybe we just need to make these suggested hints shorter? Is there discussion somewhere from where this was implemented?

I have no idea...

Yeah, even without the cursor error it's not ideal to have two lines of suggested text.

Not overdue

I think the easiest thing to do here would just be to shorten the hint text - thoughts?

I agree with that.

So we could rephrase the above hint text to the following:

"Ask how to scan receipts!"
"Ask for free corporate cards!"
"Ask how to send invoices!"

@iwiznia what do you think about this?

Not overdue

🤷 let me assign someone from marketing

Do we need multiple inputs here? Is there a character limit or number we're aiming for when we say its too long/short?
Maybe a simple "How can I help?"

This is occurring from the concierge placeholder options we have here.
We will need to update both the en.js and es.js files.

It is the last 3 entries that are creating issues. So we can just rephrase those ones?

Issue not reproducible during KI retests. (Second week)

Yes, that makes sense to me! Either replace them or don't show them on smaller screens.

Ask me how to scan a receipt - 28 characters, so we should do less than that.
Ask me how to pay people - 24 characters, so that is ok.
I don't think they all have to start with Ask me - seems arbitrary so suggesting the below using those character parameters:

Help me get reimbursed (22 characters)
How can I help? (16 characters)
Questions? I can help! (24 characters)

or agree with shawn to cut the last 3.

still waiting for feedback on my suggestions

Issue not reproducible during KI retests. (Third week) Are we ok to close this one?

I think the best solution will be to ignore the last 3 options on smaller screens. I am going to push a PR today to update it.

Issue not reproducible during KI retests. (Fourth week)

Issue not reproducible during KI retests. (Fifth week)

The PR is on production, we can issue the payment once the regression period is over.

Heads up, there is no Bug Zero member assigned to this issue.

The C+ assigned to the PR is asking about payment, I guess for both the C+ work and the 50% Bonus

Cool. Interesting that C+ wasn't assigned to this issue. Regardless, I don't think it's eligible for 50% bonus as the PR wasn't merged in 3 business days?

@JmillsExpensify This was merged within 3 business days. So we will need to pay @mananjadhav a 50% bonus on this.

@JmillsExpensify A little confusion, should this be treated as an external issue or an internal PR? Because the payouts are different for this one.

Ah yeah, that's the confusion. The bonus only applies to external PRs, per our Contributing guidelines. Regardless, I will quickly apply/remove the external label so that an Upwork job will automatically be created.

@mananjadhav Upwork job is linked above! 🙌🏼

Applied @JmillsExpensify

Awesome, offer sent! I'm going to wait to pay until Expensify agrees internally regarding the 50% bonus announcement we've documented in the Contributing.md file. I'll circle back on Monday when people are back and chime in.

@JmillsExpensify Did you get any updates on this one?

Yes, the bonus only applies externally.

 Closing this issue.


Projects with an element having an id="global" causes failure in testing script

Describe your problem and - if possible - how to reproduce it
I discovered this problem when a camper noticed the test menu was not appearing in the camper's Technical Documentation project even though it was properly added. After dissecting the camper's HTML code, I discovered the camper had used an id="global" on one of their div, which seems to be the only thing causing the test script to fail.  Once I changed this element id to something other than "global", everything worked fine.
The test script file used for the project was https://cdn.freecodecamp.org/testable-projects-fcc/v1/bundle.js which is the current version being used by all the projects.
Add a Link to the page with the problem
I created a simple example which shows what happens when an element has an id="global" in the following JSFiddle link.
https://jsfiddle.net/hy3t8po4/2/
Tell us about your browser and operating system

Browser Name: Chrome
Browser Version: 68.0.3440.75 (Official Build) (64-bit)
Operating System: Windows 8.1

If possible, add a screenshot here
HTML code along with error messages in the console

@RandellDawson I think it's fixed now, it's working for me:

So maybe you can close the issue?

This has been fixed, so I am closing this issue.


Beach Repair

The right hand tooth of the beach has structural failure in 2 pieces of the fiberglass angle framework. Beach has been lifted and the tooth pulled for repair. Beach is to remain out at this time until repair complete. Additionally, 3 6x6 beams must be purchased for supporting beach when stowed in the lifted position.

@amcleodUNH, when do you guys think this will be done? I need to have the beach in for some experiments starting Feb 1.

Done. Closing.


Pass parameters to parent class in SnowflakeSensorAsync

Currently the SnowflakeSensorAsync does not propagate parameters to its parent class, SqlSensor. Because of this, parameters are not passed for execution and can cause task failures when attempting doing so.

@Lee-W perhaps we need to make a release for astronomer-providers after merging this?


Resizing png 16-bit colorspace stopped working after 0.29.1 update

After updating to 0.29.1 from 0.29.0 my tests start failing when parsing buffers after resize
What are the steps to reproduce?
I have a resize function that uses sharp and where the data is sent over a buffer where sharp fails to reconstruct the 16-bit data
To reproduce using sharps own unit tests:
  it("16-bit PNG after resize should retain 16-bit space", async function () {
    const img = sharp(fixtures.inputPngWithTransparency16bit);
    const orgSpace = (await img.metadata()).space;

    const space = await new Promise((resolve, reject) => {
      img
        .resize(32, 16, { fit: "contain" })
        .toBuffer(async function (err, data, info) {
          if (err) throw err;

          resolve((await sharp(data).metadata()).space);
        });
    });
    assert.equal(space, orgSpace);
  });

Hi, v0.29.0 and v0.29.1 (and v0.28.3) produce the same result for me, namely that orgSpace is "rgb16" and space is "srgb", which is expected.
For "rgb16" output use toColourspace(), there's an example in the docs at https://sharp.pixelplumbing.com/api-colour#tocolourspace

Wow, thanks for the quick reply. Maybe I missed something when writing up my test. My original problem is with "grey16", the  debugging code in my app looks like this:
  const orgImg = imageCache(path);
  const resized = await orgImg.resize({
    width: maxWidthOrHeight,
    height: maxWidthOrHeight,
    fit: 'inside',
    withoutEnlargement: true,
  });
  const reverted = await new Promise((resolve) =>
    resized.toBuffer(async (err, data) => {
      resolve(await (await sharp(data).metadata()).space);
    }),
  );
  console.log({ org: (await orgImg.metadata()).space, resized: (await resized.metadata()).space, reverted });

In v0.29.0 I get the output:
{ org: 'grey16', resized: 'grey16', reverted: 'grey16' }
while if I upgrade to v0.29.1 I get:
{ org: 'grey16', resized: 'grey16', reverted: 'b-w' }
is this just as expected?

So I've been trying to dig a little deeper into this rabbit hole... Are these assumptions correct?

A sharp object that is called toBuffer() on will be converted to 8-bit format
Calling .raw() will inhibit the compression and allow a per-pixel output but in 8-bit (i.e. the length = width * height)
Calling .raw({ depth: 'ushort' }) will retain the full 16-bit resolution in the format of a Uint8Array with lower and upper byte separation (i.e. the length = width * height * 2)
Recreating from the raw buffers is impossible as meta-data isn't available

The task that I'm trying to solve is sending a 16-bit image to the front-end and sometimes resizing the image but I don't want to drop the bit depth in the process. Currently I use myResizedImage.pipe(expressRespons) but maybe I've misunderstood how sharp works?

So I think everything stems from the issue that you set the pngBitDepth to 8 without allowing the user to modify this and it doesn't respond to the colorspace. So I get this test to pass:
  it("16-bit PNG after resize should retain 16-bit space", async function () {
    const img = sharp(fixtures.inputPng16BitGrey16).toColourspace("grey16");
    const meta = await img.metadata();

    const space = await new Promise((resolve, reject) => {
      img
        .resize(32, 16, { fit: "contain" })
        .png({ pngBitdepth: 16 })
        .toBuffer(async function (err, data, info) {
          if (err) throw err;

          resolve((await sharp(data).metadata()).space);
        });
    });
    assert.equal(space, meta.space);
  });

after adding this section to the output > png():
    if (is.defined(options.pngBitdepth)) {
      if ([8, 16].includes(options.pngBitdepth)) {
        this.options.pngBitdepth = options.pngBitdepth;
      } else {
        throw is.invalidParameterError('pngBitdepth', 'integer of 8 or 16', options.png);
      }
    }

There should probably be a section in the toColourspace() that sets the bit depth depending on the colors-space, e.g.
if (colourspace.match(/16$/)) {
   this.options.pngBitdepth = 16
} else {
   this.options.pngBitdepth = 8
}

I'm not working with the other formats but I guess an option worth considering is having a general bit depth setting makes more sense, a quick search indicates that you have both pngBitDepth and tiffBitDepth that most likely serve the same purpose.

Sorry for being slow, you are of course right in how this works. I've run a few more tests and it seems that no information is lost and the only problem is that it defaults to the 'b-w' colorspace. The following test code runs without any issues (the inputPng16BitGrey16 is my own test image, a simple 50 x 50 grey16 image created in Gimp):
  it("16-bit PNG to and from buffer should retain 16-bit space", async function () {
    const img = sharp(fixtures.inputPng16BitGrey16);
    const meta = await img.metadata();

    const { newMeta, newImg } = await new Promise((resolve, reject) => {
      img
        .resize(meta.width / 2, meta.height / 2)
        .toBuffer(async function (err, data, info) {
          if (err) throw err;

          const imgFromBuffer = sharp(data);
          resolve({
            newMeta: await imgFromBuffer.metadata(),
            newImg: imgFromBuffer,
          });
        });
    });

    const bufferOrg = await sharp(fixtures.inputPng16BitGrey16)
      .toColourspace(meta.space)
      .raw({ depth: "ushort" })
      .toBuffer();
    const bufferNew = await newImg
      .toColourspace(meta.space)
      .raw({ depth: "ushort" })
      .toBuffer();

    assert.equal(bufferOrg.length / 4, bufferNew.length);
    assert.equal(bufferNew.length, newMeta.width * newMeta.height * 2);
  });

It would be nice if there was a little more behind-the-scenes in the docs but I guess that wouldn't have saved me from making this mistake. For me the most important thing is that nothing is lost when sending the image through the pipe command.
It would be nice if the space was retained after running the image through a buffer but if this isn't possible then feel free to close this issue and once again, thank you for taking the time in answering my questions and building this awesome library!

Please can you create a standalone repo with code and images that allows someone else to reproduce this. I ask this as the appearance of .png({ pngBitdepth: 16 }) suggests you are using a modified version of sharp; that option is not part of sharp's API.

Sure, I've pushed my suggested fix and linked a PR so that hopefully is clearer what I've done. It is the same fix that I mentioned in one of my above comments but this issue is becoming quite the read...

Thanks for the extra info. I've opened an upstream PR for this at https://github.com/libvips/libvips/pull/2515
Once that's landed, the fix to sharp should be a one-liner, something like:
--- a/src/pipeline.cc
+++ b/src/pipeline.cc
@@ -994,7 +994,7 @@ class PipelineWorker : public Napi::AsyncWorker {
             ->set("filter", baton->pngAdaptiveFiltering ? VIPS_FOREIGN_PNG_FILTER_ALL : VIPS_FOREIGN_PNG_FILTER_NONE)
             ->set("palette", baton->pngPalette)
             ->set("Q", baton->pngQuality)
-            ->set("bitdepth", baton->pngBitdepth)
+            ->set("bitdepth", sharp::Is16Bit(image.interpretation()) ? 16 : baton->pngBitdepth)
             ->set("dither", baton->pngDither));
           baton->formatOut = "png";
         } else if (baton->formatOut == "webp" || (mightMatchInput && isWebp) ||

Awesome, great to know that this isn't due to my lack of understanding.

v0.30.0 now available, thanks again for reporting.


move_base_flex: 0.3.4-1 in 'melodic/distribution.yaml' [bloom]

Increasing version of package(s) in repository move_base_flex to 0.3.4-1:

upstream repository: https://github.com/magazino/move_base_flex.git
release repository: https://github.com/uos-gbp/move_base_flex-release.git
distro file: melodic/distribution.yaml
bloom version: 0.10.0
previous version for package: 0.3.3-1

mbf_abstract_core

No changes

mbf_abstract_nav

No changes

mbf_costmap_core

No changes

mbf_costmap_nav
* fix blind driving (#243 <https://github.com/magazino/move_base_flex/issues/243>)

mbf_msgs

No changes

mbf_simple_nav

No changes

mbf_utility

No changes

move_base_flex

No changes

@clalancette @tfoote This fix is urgent, is there a way to get it into releases faster?


New version: MPI v0.12.0

Registering package: MPI
Repository: https://github.com/JuliaParallel/MPI.jl
Created by: @simonbyrne
Version: v0.12.0
Commit: 7536834fd4e36daaa9efb4b0b54eeb39a6b8deee
Reviewed by: @simonbyrne
Reference: https://github.com/JuliaParallel/MPI.jl/commit/7536834fd4e36daaa9efb4b0b54eeb39a6b8deee#commitcomment-37410064

This won't load without MPI installed.


Added Docker support

Also you can specify host, port and debug using environment variables.

awesome!
I am in the middle of a huge refactor, but I will surely take advantage of it :-)


feat: persist state locally in dev

Currently, .storage is only held in memory. This is fine for most usecases, but it means if you shutdown the dev process, we lose any state you may have been holding. This PR adds a config/flag persist to store this locally. We also turn it on by default. You can pass --persist false in the cli, or persist: false in config to turn it off (or true for default path). You can also pass --persist some/custom/path in the cli, or `persist: "some/custom/path" in the config, to use a custom path for the state.
Fixes https://github.com/partykit/partykit/issues/161

landing this, need it locally ha.


Full width image no text top banner

Figma: https://www.figma.com/file/F2OnKFEoMwFaANMgeGRGqE/CMS?node-id=1511%3A0

![50-50 example.png](https://ima![50-50 mobile.png](https://ima[Uploading... wide screen 50-50.png]ges.zenhubusercontent.com/62e8c66dc59ea7c87816be13/ff1da6b3-5434-4407-9cb9-1d607666e627)ges.zenhubusercontent.com/62e8c66dc59ea7c87816be13/7e7f7b7b-8cd3-4d31-b2e5-7e52a02eba2d)

Sketches can be found in Figma on TOP BANNERS tab

@NickHaggerty1 the 2:1 and full screen looks almost the same, is there any other preset you want rather than the ones listed?

@fernandolucchesi On the sketch you mean right? Bit hard to sketch full screen for all screen sizes. I assume thats something that will be set in the code? to be responsive to screen sizes? The priority would be the 10:3 (like the full width image component) and a full width 2:1.  Full width is less priority as this would be used rarely I think. Be great to see how a full screen could look if thats something you could demo though? :)

Yup @fernandolucchesi  Topic and Magazine. Obrigado!

LGTM

@NickHaggerty1 can you please check this one? :)

Available at https://studio-global-development-equinor-web-sites-dev.c2.radix.equinor.com/desk

If "designer" role is required to modify these banners more work has to be done (design permissions conversation we had a while back)

[Uploading... banner - title.png]

@fernandolucchesi @meols hey, as discussed on Friday I have added title / image text to the banner sketch. The spacing would follow the logic/grid for the default banner banner - title.png

@padms, can you review the latest changes?

@fernandolucchesi  The full screen image is not working as expected. The image is not taking full height of the screen.

Hi @padms, thanks :) can you please check again?

@fernandolucchesi  Left a comment here https://github.com/equinor/energyvision/commit/2895d894881301cc6ab31e49c2329ee1a95e8b5d#r86410264

@fernandolucchesi Left a comment here 2895d89#r86410264

Fixed the image src and left a reply about the width, please check

@fernandolucchesi Sorry for not adding all the comments at once.. Full screen image is not respecting hotspot. Not sure if this is intended.

@padms @fernandolucchesi the screengrab attached highlights the issue with the title coming after the banner but before the horizontal bar

@fernandolucchesi Sorry for not adding all the comments at once.. Full screen image is not respecting hotspot. Not sure if this is intended. Otherwise LGTM

Hi @padms, I've just tested and it looks correct to me

@NickHaggerty1 - can you please check if your comments are captured


Azdevify eShopOnContainers

https://github.com/dotnet-architecture/eShopOnContainers
Let's convert this to aca and azdevify it.

@jongio, @rajeshkamal5050 - This repo is not an Azure Sample. Do we still want to azdevify it?

Yes, please do a PR to the repo for this one.  Thanks

@jongio, @rajeshkamal5050 - The repo has been achieved. And eShop has moved to here. Do we need to azdevify the new repo now?

@zedy-wj @v-xuto - The .NET team has decided to only publish the Aspire version of eShop.  So the whole effort to azdevify eShop can stop.  Thank you for the effort, but we can close this now.
cc @rajeshkamal5050


Setting IFS in exec-env breaks asdf-direnv logging

Hi,
I am using ASDF with both the asdf-direnv plugin and this asdf-kotlin plugin.
I recently encountered an issue where the default logging of direnv was being broken up by many newlines after sourcing the asdf-kotlin environment.
After a short investigation, I figured out that the following line was the culprit for this behaviour:
https://github.com/asdf-community/asdf-kotlin/blob/ce20398cb731f755ffff16e6188ae49b2807abb5/bin/exec-env#L3
This ends up being problematic, because the exec-env script is being sourced by direnv in a loop, when using the asdf-direnv integration.
You can see that when looking at an environment definition example:
log_status using asdf kotlin 1.6.10
PATH_add /Users/<user>/.asdf/installs/kotlin/1.6.10/kotlin-native/bin
PATH_add /Users/<user>/.asdf/installs/kotlin/1.6.10/kotlinc/bin
ASDF_INSTALL_TYPE='version' ASDF_INSTALL_VERSION='1.6.10' \
    ASDF_INSTALL_PATH='/Users/<user>/.asdf/installs/kotlin/1.6.10' \
    source_env /Users/<user>/.asdf/plugins/kotlin/bin/exec-env
...

This script is being sourced by the source_env function, after which it breaks all invocations of the log_status and log_error functions.
That is because they both use msg=$* to concatenate their arguments into a single string for printing, and removing the space-character from IFS breaks that expected functionality.
I understand migrating all scripts to Strict Bash conventions, but I would propose one of the following steps:

Remove this line, as it is not effectively being used in this script
Reset IFS back to its original value after the script has executed
Wrap everything in a function and manage the modifications to IFS as a local variable

An example of how 3 would look:
#!/usr/bin/env bash
set -eo pipefail

export_kotlin_home() {
    local IFS=$'\n\t'
    if [[ -z ${KOTLIN_HOME:-""} ]]; then
      export KOTLIN_HOME=${ASDF_INSTALL_PATH}/kotlinc
    fi
}

export_kotlin_home

Yeah, I'm ok with this compromise, would you be ok testing the change from this branch? https://github.com/asdf-community/asdf-kotlin/pull/23


Allow the user the get the current number of bytes written to the wrapped writer.

In a project I'm working on, I need to know an estimation of how much bytes have already been written to decide when to call Encoder.finish().
I need that to be able to create evenly sized compressed blocks in a bigtable like implementation.

I do not like this change:

Adding Encoder.get_written_size also involves adding Decoder.get_readed_size;
This functionality can be added without lz4-rs modification by code like:

struct Wrapper<W: Write> {
	w: W,
	written_size: usize,
}

impl<W: Write> Wrapper<W> {
	pub fn get_written_size(&self) -> usize {
		self.written_size
	}
}

impl<W: Write> Write for Wrapper<W> {
	fn write(&mut self, buffer: &[u8]) -> Result<usize> {
		self.written_size += buffer.len();
		self.w.write(buffer)
	}

	fn flush(&mut self) -> Result<()> {
		self.w.flush()
	}
}

I didn't thought of that but I think it won't work.
When you build an Encoder, the Writer is moved inside it and is therefore inaccessible until you get it back from finish().
As a matter of fact, the Writter I'm using already implements the method len() which I could have used if it wasn't moved...

Sounds reasonable.
I will add this feature fore Encoder/Decoder in few days.

Great, thank you! I didn't felt like using the C bindings directly :-)


lint will not build

Upon attempting to build and install lint I am getting the following errors:
.\lint.go:1480: too many arguments in call to types.Eval
.\lint.go:1480: assignment count mismatch: 2 = 3

Run go get -u github.com/golang/lint/golint, as per the README.


Overflow in vm/cc/PCRelativeCallPatterns

The tests
vm/cc/PCRelativeCallPatterns

are failing on configurations
vm-ubsan-linux-release-x64

--- Command "run_vm_unittest" (took 01.000196s):
DART_CONFIGURATION=ReleaseUBSANX64 out/ReleaseUBSANX64/run_vm_tests --dfe=/b/s/w/ir/cache/builder/sdk/out/ReleaseUBSANX64/gen/kernel-service.dart.snapshot --enable-experiment=records --sound-null-safety --ignore-unrecognized-flags PCRelativeCallPatterns

exit code:
1

stdout:
Running raw test: PCRelativeCallPatterns

stderr:
../../runtime/vm/instructions_x64.h:127:29: runtime error: signed integer overflow: -2147483648 - 5 cannot be represented in type 'int'
    #0 0x55857017c832 in dart::PcRelativeCallPattern::set_distance(int) ../../out/ReleaseUBSANX64/../../runtime/vm/instructions_x64.h:127:29
    #1 0x55857017c648 in dart::Dart_TestPCRelativeCallPatterns() ../../out/ReleaseUBSANX64/../../runtime/vm/compiler/relocation_test.cc:449:13
    #2 0x55856fedec6f in dart::RawTestCase::Run() ../../out/ReleaseUBSANX64/../../runtime/bin/run_vm_tests.cc:58:3
    #3 0x55856feded18 in dart::TestCaseBase::RunTest() ../../out/ReleaseUBSANX64/../../runtime/bin/run_vm_tests.cc:64:11
    #4 0x5585700a56cc in dart::TestCaseBase::RunAllRaw() ../../out/ReleaseUBSANX64/../../runtime/vm/unit_test.cc:75:13
    #5 0x55856fedf6d3 in dart::Main(int, char const**) ../../out/ReleaseUBSANX64/../../runtime/bin/run_vm_tests.cc:409:3
    #6 0x55856fedf210 in main ../../out/ReleaseUBSANX64/../../runtime/bin/run_vm_tests.cc:428:29
    #7 0x7f0ed6bd8c86 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21c86) (BuildId: f7307432a8b162377e77a182b6cc2e53d771ec4b)
    #8 0x55856fec4229 in _start (/b/s/w/ir/cache/builder/sdk/out/ReleaseUBSANX64/run_vm_tests+0x30ec229)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../runtime/vm/instructions_x64.h:127:29 in

--- Re-run this test:
python3 tools/test.py -n vm-ubsan-linux-release-x64 vm/cc/PCRelativeCallPatterns

@mkustermann

Thanks, @rmacnak-google. Uploaded cl/280322


[WIP][HIVEMALL-138] to_ordered_map UDAF with size limit

What changes were proposed in this pull request?
Support integer 3rd argument on to_ordered_map UDAF to set limit of map size.
It can be used as an alternative of each_top_k UDTF. The main difference is that the former actively utilizes mapper-side aggregation.
What type of PR is it?
Feature
What is the Jira issue?
https://issues.apache.org/jira/browse/HIVEMALL-138
How was this patch tested?

Manual test on local and EMR
Unit test

How to use this feature?
to_ordered_map(key, value, size)

with t as (
    select 10 as key, 'apple' as value
    union all
    select 3 as key, 'banana' as value
    union all
    select 4 as key, 'candy' as value
)
select
    to_ordered_map(key, value, true),   -- {10:"apple",4:"candy",3:"banana"} (reverse)
    to_ordered_map(key, value, 1),      -- {10:"apple"} (top-1)
    to_ordered_map(key, value, 2),      -- {10:"apple",4:"candy"} (top-2)
    to_ordered_map(key, value, 3),      -- {10:"apple",4:"candy",3:"banana"} (top-3)
    to_ordered_map(key, value, 100),    -- {10:"apple",4:"candy",3:"banana"} (top-100)
    to_ordered_map(key, value),         -- {3:"banana",4:"candy",10:"apple"} (natural)
    to_ordered_map(key, value, -1),     -- {3:"banana"} (tail-1)
    to_ordered_map(key, value, -2),     -- {3:"banana",4:"candy"} (tail-2)
    to_ordered_map(key, value, -3),     -- {3:"banana",4:"candy",10:"apple"} (tail-3)
    to_ordered_map(key, value, -100)    -- {3:"banana",4:"candy",10:"apple"} (tail-100)
from t
;

Checklist

[x] Did you apply source code formatter, i.e., mvn formatter:format, for your commit?

@myui This is PoC implementation. Could you give me feedback, esp. in terms of interface? I will update documentation after interface has been fixed.

Coverage increased (+0.3%) to 41.179% when pulling 3c029f9bd71adb70db8dfc48f6452362dacc164c on takuti:topk-ordered-map into 7205de1e959f0d9b96ac756e415d8a8ada7e92af on apache:master.

How about to_ordered_map accepts int k for the third argument. Then, use BoundedOrderedMapEvaluator.

Do you mean something like:

3rd argument is boolean => current to_ordered_map
3rd argument is integer

positive => top-k ordered map (i.e., reverse ordered bounded map)
negative => tail-k ordered map (i.e., natural ordered bounded map)

?
Okay, it sounds reasonable for me.

Coverage increased (+0.06%) to 40.888% when pulling d68e5e9493cb78c4321169c15d7d04fedbbca691 on takuti:topk-ordered-map into 7205de1e959f0d9b96ac756e415d8a8ada7e92af on apache:master.

@takuti Yes. That's my intention.

Updated to_ordered_map as you suggested.
From here, I will implement top_k_queue to work around duplicated keys.

to_ordered_queue(NUMBER key, ANY value, const string options)
options:
-k -- bounded queue
-reverse  -- to reverse order queue

Coverage increased (+0.2%) to 41.003% when pulling c448099c4f35870dd75cf0a5166acf9e6fdf4d17 on takuti:topk-ordered-map into 7205de1e959f0d9b96ac756e415d8a8ada7e92af on apache:master.

@takuti sorry, I'm considering to rename it to to_ordered_list .

@myui Ah, okay. But I imagine that "ordered list" indicates a little bit different function like to_ordered_list(ANY value)... Do you have any other ideas?

to_ordered_list(PRIMITIVE key, ANY value) is also acceptable for me, btw ;)

@takuti Do you have any other ideas?
No corresponding data structure. Then, not to use Data structure name such as aggr_top_k(cmpkey, value)::values

I tested each_top_k, to_ordered_map and to_ordered_list on the same MovieLens 1M data. As we expected, to_ordered_map collects duplicated keys, and the number of ratings is 3 while we launched top-10 aggregation.
with topk as (
    select
        each_top_k(
            10, userid, rating,
            userid, movieid
        ) as (rank, rating, userid, movieid)
    from (
        select
            userid, movieid, rating
        from ratings
        cluster by userid
    ) t
)
select
    count(1), collect_list(array(movieid, rating))
from
    topk
where
    userid = 1
;

10      [[527.0,5.0],[3105.0,5.0],[1270.0,5.0],[48.0,5.0],[1035.0,5.0],[1193.0,5.0],[1287.0,5.0],[2355.0,5.0],[595.0,5.0],[2804.0,5.0]]

with topk as (
    select
        userid,
        to_ordered_map(rating, movieid, 10) as movies
    from
        ratings
    group by
        userid
)
select
    count(1), collect_list(array(movieid, rating))
from
    topk
lateral view explode(movies) t as rating, movieid
where
    userid = 1
;

3       [[2028,5],[1246,4],[745,3]]

with topk as (
    select
        userid,
        to_ordered_list(movieid, rating, '-k 10') as movies
    from
        ratings
    group by
        userid
)
select
    size(movies), movies
from
    topk
where
    userid = 1
;

10      [595,1035,3105,2355,1287,2804,1193,2028,1029,1270]

that's bad...

what's bad...?

lots of collisions in to_ordered_map.

Oh, that's expected behavior. I just showed how each top k, map and list are different. Using map is obviously a wrong choice in the MoviLens case.

Failed also for to_ordered_map in Hive v2.3.0.
IllegalArgumentException Size requested for unknown type: java.util.Map

Merged. Thanks!


Activating Open Collective

Hi, I'm making updates for Open Collective. Either you or another core contributor signed this repository up for Open Collective. This pull request adds financial contributors from your Open Collective https://opencollective.com/color-picker ❤
What it does:

adds a badge to show the latest number of financial contributors
adds a banner displaying contributors to the project on GitHub
adds a banner displaying all individuals contributing financially on Open Collective
adds a section displaying all organizations contributing financially on Open Collective, with their logo and a link to their website

We have also added a postinstall script to let people know after npm|yarn install that you are welcoming donations. Feel free to remove it if you don't want it. [More info]
P.S: As with any pull request, feel free to comment or suggest changes.
Thank you for your great contribution to the Open Source community. You are awesome! 🙌
And welcome to the Open Collective community! 😊
Come chat with us in the #opensource channel on https://slack.opencollective.com - great place to ask questions and share best practices with other Open Source sustainers!

Thanks. Just one thing that still makes me doubtful. How about my ethics for the people who contribute? I mean, if I get income from this project, then what about the fee for contributors?
I am into this:

Is it even ethical to accept money for a project to which others have contributed? How could money be shared with contributors? How to fairly decide who gets how much?

https://hackaday.com/2019/07/05/how-not-to-get-paid-for-open-source-work/

What do you think?

If this project is going Open Collective... I would love to see touchscreen device support.  For me, I have plans to swap this package out for a touchscreen-compatible package as soon as I get the chance.

This already has touch-screen support from the start.

Doh... confused this for another package... sorry about that. I stand corrected!


proposal: all: broader assignability to send/receive channel types

What did you do?
https://play.golang.org/p/ORxhPRy1cl
What did you expect to see?
7
42

Since ProduceTo declares its argument as a chan<- int, it can only ever write ints (or close) to the channel. ints can be assigned to variables of type interface{}, and ints can be sent to a chan interface{}, so it'd be nice to let ProduceTo write to a chan interface{} even though its argument is chan<- int.
Similarly, since ConsumeFrom takes a <-chan interface{}, so it must be able to handle any interface{} value read from the channel. By passing a chan int value, the programmer knows that in reality ConsumeFrom will only receive int values, but ConsumeFrom is ready. ConsumeFrom can handle anything! Believe in ConsumeFrom!
What did you see instead?
tmp/sandbox755940094/main.go:17: cannot use c1 (type chan interface {}) as type chan<- int in argument to ProduceTo
tmp/sandbox755940094/main.go:22: cannot use c2 (type chan int) as type <-chan interface {} in argument to ConsumeFrom

Working around the non-assignability of appropriately covariant / contravariant channels usually needs another channel and goroutine: https://play.golang.org/p/b2JsBdvLy1.

This is too big of a language change at this point. See previous discussion in https://github.com/golang/go/issues?utf8=✓&q=covariant
These sorts of issues will be reconsidered if/when there's a Go2.

Thanks for taking a look anyway. I did a number of similar searches and found nothing specific to channel send/receive types, so figured it was worth a shot.


OCPBUGS-43700: Fix reporting of diff when using external diff 'diff -y'

Beacuse 'diff -y' returns the entirity of both manifest even when no diffs are present, had to switch to using attempting to use the leaf count and not the diff output to determing if a diff exists in the output

Should we add a compare_test scenario for this bad boy?

/approve

/lgtm

/jira refresh


Suenglab's TEASAR

Starts with point cloud?
seung-lab/skeletonization "Python TEASAR Skeletonization for Sparse Labels"

A Connectome of the Adult Drosophila Central Brain

Neuron skeletons are computed from the segmentation[40], and not entered
or edited directly. A skeleton representation describes each neuron with (branching) centerlines and diameters, typically in the SWC format popularized by the
simulator Neuron.
[40] Zhao and Plaza. 2014

Ting Zhao and Stephen M Plaza. Automatic neuron type identification by neurite localization in the Drosophila medulla. arXiv:1409.1892, 2014.

Skeletonization is the process of converting a binary image into a skeleton model.
There are multiple ways of skeletonizing a binary object and we found the
method called TEASAR [15] to be most appropriate to our problem. In our
implementation, we adopted TEASAR by applying the following steps:


Instance: Unify profile validation and loading for instance create endpoint

Part of the requirements in order to provide an expanded request to the instance placement scriptlet (https://github.com/lxc/lxd/pull/11180).
Also reduces transactions and queries used.
Merge https://github.com/lxc/lxd/pull/11222 first.

Testsuite passed

Looks like I just broke it.

jenkins looking a bit mad at this one :)

@tomponline did you mean to push a fix for this one?

All fixed now thanks

@stgraber if you take a look at https://github.com/lxc/lxd/pull/11222 first, then I can rebase this one once its merged to make review easier.

@tomponline ready for rebase

Thanks. I've got an issue that isn't detected by the tests to fix on this branch, that I just noticed now as part of my testing of the instance placement scriptlet.

@stgraber ok this is ready for review now.


Only first task in sequence runs

Note: we first noticed this issue after upgrading to node 4.4.4. Unsure whether or not that's related.
gulpfile.js:
var gulp = require('gulp');
var sequence = require('run-sequence');

gulp.task('foo', function (cb) {
    console.log('foo');
});
gulp.task('bar', function (cb) {
    console.log('bar');
});

gulp.task('default', function (cb) {
    sequence('foo', 'bar', cb);
});

package.json:
{
  "name": "bug-test",
  "version": "1.0.0",
  "engines": {
    "node": "4.4.4"
  },
  "devDependencies": {
    "gulp": "^3.9.0",
    "run-sequence": "^1.2"
  }
}

gulp output:
 ~/dev/run-sequence-bug# gulp
[18:00:19] Using gulpfile ~/dev/run-sequence-bug/gulpfile.js
[18:00:19] Starting 'default'...
[18:00:19] Starting 'foo'...
foo
 ~/dev/run-sequence-bug# echo $?
0

Your tasks aren't finishing, they aren't configured correctly.  If you accept the callback in your task, you have to call it at some point.
Please read the Usage section.
I'm using run-sequence on Node 4, 5, and 6 without issue.

@OverZealous got it, thanks!


updated three.js to r75

updated library file to r75, no errors popped up.

thanks for the contributions, i ported the examples too


python312Packages.ecpy: normalize pname

Description of changes
part of #245383
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 296514 run on x86_64-linux 1

2 packages failed to build:

python311Packages.pycardano
python311Packages.pycardano.dist

10 packages built:

ledger-agent (python311Packages.ledger-agent)
ledger-agent.dist (python311Packages.ledger-agent.dist)
python311Packages.ecpy
python311Packages.ecpy.dist
python311Packages.ledgerblue
python311Packages.ledgerblue.dist
python312Packages.ecpy
python312Packages.ecpy.dist
python312Packages.ledgerblue
python312Packages.ledgerblue.dist


Default key behaviour should be prevented for examples

Some of the examples use the space or arrow keys which by default will cause the window to scroll making the demo hard to use.

Examples updated! It should work as expected now! :D

I did look into this a bit, but got distracted by other things so I never followed through.
Problem with this is that you can no longer type a space character into text boxes.
So I looked into text_input_box.js and found that behind the scenes GetCharPressed is mapped javascript's  keypress event (supposedly deprecated: https://developer.mozilla.org/en-US/docs/Web/API/Document/keypress_event), and unfortunately calling preventDefault in keydown also prevents keypress from firing.
The only solution I think of is to change this to use keydown and use the charCode (also deprecated apparently) or key property and filtering out non-printable characters.
I'm not sure if you have control over this though because it looks like it's part of the compile GLFW javascript.

@cmann1 ooh... I see... unfortunately I don't have control over it, emscripten provides glfw implementation. I've I had more time I'd consider replacing it by my custom implementation...


Rewrite printing for better terminal support and looks

The current highlighting requires a terminal with 24bit color support. Additionally the current printing doesn't wrap in a good way, so A rewrite is due that solves this.

The plan is to write some custom printing logic that handles the length calculations and wraps nicely where needed.
While at it we add printing of separation characters after the line numbering and as a horizontal line before the new command line.

The highlighting is to be revamped in the same vein, using a basic regex engine based on config files mostly following Sublime's format.
Theming will be done through the main application config by giving any string to start and another to end the highlighting of each type. This results in theoretically complete terminal support.

Using the same hack as 'bat' a working solution now exists.


Remove AsciidoctorUtils class

Kind of change

[ ] Bug fix
[ ] New non-breaking feature
[x] New breaking feature
[ ] Documentation update
[ ] Build improvement

Description
Remove AsciidoctorUtils class.
What is the goal of this pull request?
This class contained a method to convert options into asciidoctor cli. Used only for logging and tests it's considered not useful.
Other changes:

Add AssertJ dependency to Java projects by default
Refactored AsciidoctorUtils.isOptionWithAttribute into JRubyAsciidoctor.containsAttributeWithValue

How does it achieve that?
Removes class and extracts a single method that is still used once as private method in the calling class (JRubyAsciidoctor).
Are there any alternative ways to implement this?
If there's any concern about users impacted. We could keep the class and remove its usage, but it's already in an internal package and we can. It should not be a surprise.
Are there any implications of this pull request? Anything a user must know?
Only if they are using the class.
Issue
If this PR fixes an open issue, please add a line of the form:
Fixes #1169
Release notes
Please add a corresponding entry to the file CHANGELOG.adoc

I like those

I had a look at the "test upstream" and I don't understand what it's doing.
It installs a local jar containing the asciidoctor gem? and then the ruby gradle plugin uses it instead of the gem?

Yes, the test upstrea is testing against the unreleased version of Asciidoctor. Since it's not distributed, it must be installed locally and used as though it had been retrieved from a repository.

Yes, the test upstrea is testing against the unreleased version of Asciidoctor. Since it's not distributed, it must be installed locally and used as though it had been retrieved from a repository.

Thanks! My doubt is more about the "how" though. In a normal gradle build com.github.jruby-gradle.base downloads the gem directly into ./build/gems, but seeing that we install the gem in test upstream with the maven gem-maven-plugin, that means that the gem dependency we define can also process local jars. Not being able to generate the same jar now, makes it's hard to know what the plugin expects (yes, I haven't read their docs fully yet :sweat_smile: ).
I'll do some testing today, maybe we can explore another option. I just saw we don't pull transitive dependencies for asciidoctor which was one of my concerns...I wonder if we can just unpack the asciidoctor zip from gh in ./build/gems.

I'm testing my memory, but I think the how predated the switch to Gradle. So it's very possible that it is an outdated approach and worth rethinking. But without looking into it deeper, I wouldn't be able to say more just from memory alone.

Nice, loosing some weight :)


⚠ Bofrost.es has degraded performance

In 7eaaefd, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1698 ms

Resolved: Bofrost.es performance has improved in ef57d31 after 8 minutes.


Adding Industrial Ore Processor premium recipe

About the PR

Adding Plasteel, Uranium Glass, Reinforced Uranium Glass recipe for Industrial Ore Processor only.
Fix Reinforced Glass and Plasma Glass recipe not working in Ore Processor.

Why / Balance

Give Science another reason to upgrade starting ore processor and adding more value to industrial ore processor instead of being cheaper material to process stuff

Technical details

YML changes only
Media

[X ] I have added screenshots/videos to this PR showcasing its changes ingame

Breaking changes

changes few recipes in sheet.yml :
Steel to rawIron
Glass to rawQuartz

Changelog

:cl:

fix: Fixed Reinforced Glass Recipe in Ore Processor!
add: Added Premium Material that can only be processed by Industrial Ore Processor!

changelog is missing :cl:

changelog is missing 🆑

alright fixed, sorry

@EmoGarbage404 emocabbage


How to manage permissions to run azure devops pipelines and change variable groups pragmatically?

I am reading through this documentation - https://docs.microsoft.com/en-us/rest/api/azure/devops, I can't see a clear answer on how one can manage permissions to run pipelines and edit variable groups through the Rest API.

@melezhik  Thank you for the feedback.
Given the extent of available options in Azure DevOps it's hard to reference every scenario.  Perhaps try the API search feature?

For pipelines, please refer to Build or Release and for variable groups see Variable groups.

Hi @msebolt , I refined the issue title I meant only permissions. I am already though https://docs.microsoft.com/en-us/rest/api/azure/devops/security , but IMHO the documentation is not that clear and does not cover many aspects of managing permissions through the RestAPI

@melezhik Thank you for the clarification, assigning this to the author for review.
For now, please refer to Pipeline permissions and Security roles in determining which APIs are needed.

@msebolt  I've not found RestAPI to manage variable groups permissions, there is no even security-namespace for that, is it correct?

Ah, I found, it's called 'library'

@msebolt  how do I define token for git branch?
az.cmd devops security permission list --id  2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87   --subject<EMAIL_ADDRESS>gives me a guess that it should be something:
repoV2/$project_id/$repo_id/refs/heads/$unknown_chunk
But I can't understand how to calculate the last bit of it  - $unknown_chunk, logically it should be something that define branch.

looks like it's
for every letter of branch name:
oct($i)00
Why not document it?

@msebolt , any updates here? thanks


IML <IP_ADDRESS> Manual Testing

OS Specs:
CentOS Linux release 7.6.1810 (Core)
kernel.x86_64                         3.10.0-957.1.3.el7
kernel-devel.x86_64              3.10.0-957.1.3.el7
kernel-headers.x86_64          3.10.0-957.1.3.el7
Managed Ldiskfs
Installation output:
[root@adm iml-<IP_ADDRESS>]# ./install --no-dbspace-check --no-platform-check
Testing YUM
Loaded plugins: fastestmirror, langpacks, versionlock
Repository extras is listed more than once in the configuration
Starting Integrated Manager for Lustre software installation
Unpacking installation package el7/iml-manager-<IP_ADDRESS>.tar.gz
Installing Integrated Manager for Lustre
-
Starting setup...

Setting up PostgreSQL service...
Creating database owner 'chroma'...

Creating database 'chroma'...

Creating database tables...
Loaded 13 default power device types.
Creating groups...
An administrative user account will now be created using the credentials which you provide.
Username: admin
Password:
Confirm password:
Email:
User 'admin' successfully created.
API user created
NTP Server [localhost]:
Writing ntp configuration: localhost
Restarting ntp
Starting RabbitMQ...
Creating RabbitMQ user...
Creating RabbitMQ vhost...
Enabling daemons
Starting daemons
Checking service configuration...

Setup complete.
Registering profiles
chroma-manager <IP_ADDRESS>-5039.el7 is currently installed

Integrated Manager for Lustre software installation completed successfully

Deploying agents:

Installing Packages Step Failed:
https://gist.github.com/johnsonw/82c7722b34ebc8652d3c6e8c40648b51

This error had to do with the kernel version. I've updated the vagrant box to use the downgraded kernel version.

After updating the box to use the correct kernel, we ran the install a second time. It looks like it still failed, but not due to the original error: https://gist.github.com/johnsonw/3c1674203e9732b327ccd8eb3babb122
Cause of Package Installation Failure:

The output of repoquery has changed, which prevented chroma from parsing the output correctly and thus an exception was raised. To fix this, we added the quiet flag to the dnf calls: https://github.com/whamcloud/integrated-manager-for-lustre/commit/99c59dc0a70cd08e3e1832cd3cc5669e6c88cef4

Adding the quiet flags resolved the package installation issue.
Corosync Configuration Issue:
The Configure Corosync step was spinning consistently and never finished. This may be an existing error that we discovered as running the command a second time seems to resolve the issue. The problem appears to be related to the following line: https://github.com/whamcloud/integrated-manager-for-lustre/blob/iml-4.0.9/chroma-agent/chroma_agent/lib/networking.py#L59. It looks like we are filtering network packets and looking for a particular pattern that never seems to show up. I wrote a short script that fetched all available interfaces and read packet data from a selected interface. When running this script, it appears to have triggered something that allowed the Configure Corosync job to complete.
Power Control:
Power control would not configure properly due to a stack trace that was occurring. This happened because the version of fence-agents-all, which is a dependency of fence-agents-vbox, changed in el7.6. We made the necessary changes and published a new version of fence-agents-vbox. Power control now works as expected.
High Availability:
After power control was setup, I halted OSS1 and verified that the targets on OSS1 automatically moved over to OSS2. I then brought up OSS1 and was able to successfully move the targets back to OSS1.
Charts:
After mounting the latest client, I was able to generate data and see the data on the charts:

Of note, we are fixing these issues in #766, and everything except pcappy hanging (which needs to be reproduced) should be addressed.

Managed Ldiskfs:
Finally passed, but hit and resolved the following issues during the test

Since the versions of fence-agents-common on the nodes were not compatible, power control can't be configured correctly. This issue will be fixed by https://github.com/whamcloud/fence-agents-vbox/pull/2.
lustre client can't be mounted after installing lustre client rpm packages, due to the following error:

# mount -t lustre <IP_ADDRESS>@tcp0:<IP_ADDRESS>@tcp0:/fs /mnt/fs
mount.lustre: mount <IP_ADDRESS>@tcp0:<IP_ADDRESS>@tcp0:/fs at /mnt/fs failed: No such device
Are the lustre modules loaded?
Check /etc/modprobe.conf and /proc/filesystems

This issue can be resolved by rebooting client node.

Managed ZFS: passed without any issue

Monitored ldiskfs: passed, with the following message when configuring lnet
mds1::
del:
    - net:
          errno: -2
          descr: "cannot del network: No such file or directory"
add:
    - net:
          errno: -22
          descr: "cannot add network: Invalid argument"

Monitored zfs:
Finally passed, but hit and resolved the following issues during the test

Deploying agent to oss1.lfs.local failed, then tried to deploy again, succeed
LNet can't be configured correctly on some nodes, as shown in the figure. This issue was fixed by restarting the agent service on these nodes, by the command "systemctl restart chroma-agent.service"


Budget Allocator 1 and 2

Hi I am trying to do budget allocation, but I have repeatedly encounter issues; I have used the below methodologies to fix. I would like to report here in case other people also encounter the same error

Buddget Allocator 1: when I used default code,

AllocatorCollect1 <- robyn_allocator(
InputCollect = InputCollect,
OutputCollect = OutputCollect,
select_model = select_model,
date_range = NULL, # Default last month as initial period
total_budget = NULL, # When NULL, default is total spend in date_range
channel_constr_low = 0.7,
channel_constr_up = c(1.2, 1.5, 1.5, 1.5, 1.5),
channel_constr_multiplier = 3,
scenario = "max_response",
export = create_files
)
The system gave me the below error:

Running budget allocator for model ID 3_175_3 ...
check_allocator_constrains(channel_constr_low, channel_constr_up) でエラー:
Inputs 'channel_constr_up' and 'channel_constr_low' must have the same length or length 1

Therefore, I changed code of channel  channel_constr_up = c(1.2, 1.5, 1.5, 1.5, 1.5) -> 1.3
And then I was able to get the result! :)

Strange, just tested and can't reproduce this error. Please reopen if it happens again

Hi @gufengzhou , I understand the data structure now, now it works. thanks


Scrollable sidebar and dashboard

makes the overall "App" component not scrollable and with a height exactly the size of the viewport.

makes the "sidebar" and "dashboard" components vertically scrollable by giving them overflow-y: auto.

removes a random unused import. hehe

wasn't really sure if unit tests are needed here as it's only css changes

happy hacktober! 🥳  🎃

@JimLynchCodes Looking good! Thank you 👻🎉


Avoid nullify map if user cancelled Log in

Avoid nullify map if user cancelled Log in OAuth process. Otherwise user need to kill the app and launch it again in order to be able to continue using it.

@Devepre Do you have the ability to merge this PR?  If not, I can do it.  Thank you for your contribution!

@mhdostal that's perfect code suggestion, thank you for the feedback. I beliive I don't have write access for this repository and unable to merge this PR.

@Devepre - Thanks for finding and fixing this!  Merged into master.


Cleaned up session closing

For getlantern/lantern-internal#2373
Depends on https://github.com/getlantern/idletiming/pull/14
The basic idea is that instead of relying on IdleTiming to close our TCP connections like we have in the past, lampshade now closes them when they're no longer needed.

Great catch @oxtoacart! Done my round and only two comments!

Thanks for the review @joesis!


repository wildcard cert has expired

FYI the wildcard cert on https://repo.craftyn.com/repository/snapshots/ is invalid (cert expired)
Common Name (CN)	*.craftyn.com
Organisation (O)
Organisational Unit (OU)	Domain Control Validated
Common Name (CN)	COMODO RSA Domain Validation Secure Server CA
Organisation (O)	COMODO CA Limited
Organisational Unit (OU)
Issued On	Tuesday, 20 January 2015 at 10:00:00
Expires On	Wednesday, 21 March 2018 at 09:59:59
Simple solution might be to use http:// instead of https:// but the server always redirects to SSL.
Various "dont check ssl" maven options dont seem to work in maven 3.5.3

The ssl has been ordered, I'm just needing to finish the process. My apologies

no big. i thought i should at least mention it as sometimes people dont :)

Closing as I finally was able to get things going again. :)


JDL2 tooling setup

Each component (package) should define own dependencies versions for React, ReactDom - we need to control that per package, e.g. if something requires minimum React 16+. Also, a suggestion, common tooling (e.g.  jdl-component-creator.js, storybook, styleguide, ... ) can be available as a shared package (e.g. /utils/shared), again so that we can introduce breaking changes without affecting every component.

It would be nice to specify compatible versions but components can be designed for react 15 & 16 (and presumably higher) and work fine so the version would have to be: ^15 which is almost effectively *. What I don't want to do is: a) have something that results in multiple copies of React in the user's bundles or b) prevent users from upgrading React needlessly. By specifying a wildcard version, at least it lets the consumer try to use these without having to continually upgrade.
As for the other comment about storybook, style guide and such, these are not intended to be consumed downstream.


Add text offset

Currently the text height is always calculated automatically. This is a good idea and supporting bottom/top alignment is probably not needed.
However when setting an underline with the beckground image, it might be nice to offset the text by a bit, so it is centered when ignoring the line.
Something like this:
let text = Text::new("Hello, World").y_offset(-3)

Fixed with 78fbcd3b6cd6dbcf8668949ec072458c84365265


bug: E127 Cannot redefine function

Did you check docs and existing issues?

[X] I have read all the lazy docs
[X] I have searched the existing issues of lazy
[X] I have searched the exsiting issues of the plugin I have a problem with

Neovim version (nvim -v)
NVIM v0.9.0-dev-580+g4703e561d
Operating system/version
Ubuntu 22.0.4
Describe the bug
When using fzf-lua's previewer to search for text that appears in a rust file:
Failed to source `/home/dch/.local/share/nvim/lazy/rust.vim/ftdetect/rust.vim`
...local/share/nvim/lazy/lazy.nvim/lua/lazy/core/loader.lua:303: Vim(function):E127: Cannot redefine function <SNR>5_set_rust_filetype: It is in use
# stacktrace:
  - /fzf-lua/lua/fzf-lua/previewer/builtin.lua:589
  - /fzf-lua/lua/fzf-lua/previewer/builtin.lua:588 _in_ **do_syntax**
  - /fzf-lua/lua/fzf-lua/previewer/builtin.lua:661 _in_ **preview_buf_post**
  - /fzf-lua/lua/fzf-lua/previewer/builtin.lua:538 _in_ ****
  - vim/_editor.lua:0

which seems surprising, rust.vim goes function!, I thought I understood that the meaning of the exclamation mark was exactly to make this not happen
I'm experimenting with the switch from packer to lazy.nvim, and still have a computer with the packer setup - and can confirm that the problem does not reproduce there.  So while I could imagine this turning out to be a bug elsewhere, I'm starting with lazy.nvim.
Steps To Reproduce

Have a rust project eg https://github.com/dimbleby/rust-c-ares
Open a file eg Cargo.toml
fzf-lua live_grep_native() for something eg "metadep".  (I've mapped the fzf-lua function to <Leader>r in the repro.lua below)
scroll down the preview until you hit a rust file containing that text

Expected Behavior
No errors
Repro
-- DO NOT change the paths and don't remove the colorscheme
local root = vim.fn.fnamemodify("./.repro", ":p")

-- set stdpaths to use .repro
for _, name in ipairs({ "config", "data", "state", "cache" }) do
  vim.env[("XDG_%s_HOME"):format(name:upper())] = root .. "/" .. name
end

-- bootstrap lazy
local lazypath = root .. "/plugins/lazy.nvim"
if not vim.loop.fs_stat(lazypath) then
  vim.fn.system({
    "git",
    "clone",
    "--filter=blob:none",
    "--single-branch",
    "https://github.com/folke/lazy.nvim.git",
    lazypath,
  })
end
vim.opt.runtimepath:prepend(lazypath)

-- install plugins
local plugins = {
  "folke/tokyonight.nvim",
  { "rust-lang/rust.vim", ft = "rust" },
  {
    "ibhagwan/fzf-lua",
    lazy = true,
    config = function()
      require("fzf-lua").setup({})
    end,
  },
}
require("lazy").setup(plugins, {
  root = root .. "/plugins",
})

vim.cmd.colorscheme("tokyonight")

vim.keymap.set("n", "<Leader>r", function()
  require("fzf-lua").live_grep_native()
end, { noremap = true })

oh actually I seem to be able to reproduce this error without involving fzf-lua at all: just opening a rust file is enough.
so that simplifies the repro, you can remove all fzf-lua related bits
Failed to source `/home/dch/rust-c-ares/.repro//plugins/rust.vim/ftdetect/rust.vim`
...c-ares/.repro/plugins/lazy.nvim/lua/lazy/core/loader.lua:303: Vim(function):E127: Cannot redefine function <SNR>4_set_rust_filetype: It is in use


Error Calling RCTEventEmitter.receiveEvent

Issue Description
Once Camera detected the Qr Code i m getting this error "Error calling RCTEventEmitter.receiveEvent"
Steps to Reproduce / Code Snippets / Screenshots
[FILL THIS OUT - It will be extremely helpful]

Environment

React Native Navigation version: "1.0.0-beta.22",
React Native version: "0.55.3"
Platform(s) (iOS, Android, or both?): Android
Device info (Simulator/Device? OS version? Debug/Release?): Android 8 , it happened in debug mode.

simulator? read device?

same issue.

Same issue in Simulator