Improve ColorExt implementation

After working looking a couple times at the ColorExt trait, I realized that there are a couple of improvements that could be done to the trait. I have been playing and working in this a bit now, and would be glad to make a pull request them as soon as you approve them, I just want to check if everybody likes the look of the changes! :smile:
Take ownership instead of borrow self
As of now, the ColorExt trait takes &self, and returns Self. This makes the implementation a bit wordy, when it could be simplified. The fact that the function would now consume the value is not a problem, as it would return it right away as is or return a new owned value based on the original. Right now, this is the triat:
pub trait ColorExt {
  fn reset(&self, flags: &args::Flags) -> Self;
  /* ... */
}

#[cfg(feature = "color")]
impl ColorExt for String {
  fn reset(&self, flags: &args::Flags) -> Self {
    if flags.no_color {
      self.to_string()
    } else {
      format!("{}{}", self, String::from(RESET))
    }
  }
  /* ... */
}

Now, if the function was changed to take ownership of the value, the code could be simplified, without having any trouble with consuming the value, as we are returning the owned value back again (modified or as is). You can see below how this could be improved:
pub trait ColorExt {
  fn reset(self, flags: &args::Flags) -> Self;
  /* ... */
}

#[cfg(feature = "color")]
impl ColorExt for String {
  fn reset(self, flags: &args::Flags) -> Self {
    if flags.no_color {
      self
    } else {
      format!("{}{}", &self, String::from(RESET))
    }
  }
  /* ... */
}

It is not much of a change, but it can highly simplify the code, specially when taking a look at the next section of the issue.
Default implementations
Were changing the borrow to an owned for self really shines is with default implementations. If the function takes self instead of &self, there can be a default implementation like the following:
pub trait ColorExt {
  fn reset(self, flags: &args::Flags) -> Self {
    self
  }
  /* ... */
}

By doing this, we can remove roughly 40 lines of code, which are the implementation for String without the color feature. This change is also beneficial for the following section.
Apply to all types that implement Display (or Debug)
Both the previous changes could help in making the ColorExt trait more available and also saving us from many "...".to_owned() calls. The following implementation would work on &str, String, u8 or any other type that implements the Display trait. This could also be done for types that implement the Debug trait.
Here, taking self instead of &self, helps in the fact that not all types that implement Display have a to_owned() or similar method, so we can't implement it if we rely on that. The default implementation, again, just saves up so many lines of code. Here is what the implementation would look like (for the color feature, without it the code would be eve simpler):
#[cfg(feature = "color")]
impl<T> ColorExt for T where T: std::fmt::Display {
  fn reset(self, flags: &args::Flags) -> Self {
    if flags.no_color {
      self
    } else {
      format!("{}{}", &self, String::from(RESET))
    }
  }
  /* ... */
}

Trying to implement it for any type that implements Display, I came across a little error. The function signature used Self as return, which was completely perfect since the trait was only implemented to String, so returning self and format!() would return the same type. Now that has changed, and it is nothing difficult, but I will have to make more changes. The signature would change to:
pub trait ColorExt {
  fn reset(self, _flags: &args::Flags) -> String
  where
    Self: Sized + std::fmt::Display,
  {
    format!("{}", self)
  }
  /* ... */
}

Should not give any problems with its use, just changes (and complicates) a little bit the trait itself. I could get rid of the `where Self: ...` part, but that would make me not able to create default implementation, what do you think of that? Should I get rid of that?

I think making it where it works on anything that impliments Display is great, even if it turns it into a String. This is really awesome, thanks for working on this!


Update Policy Report UI

Signed-off-by: Frank Jogeleit<EMAIL_ADDRESS>

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 93.778%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
2110

Relevant Lines:
2250

💛  - Coveralls


Cleanse AGOL tokens from logs

It's probably a good idea to get these out of the logs. There's no reason to have them.

I don't


Configuration not created at install

Hello !
I just installed the gateway on my pi, but it seems I have an little issue : the configuration file has not been created, so I created the file using your example.
But I'm not sure what to use on the "server :" parameter for exampe.
It would be really helpfull to succeed with your tool !!
Thanks !

hello bro i had same problem.
but i solved.
the problem is tab space :)
you can edit as in image :)


Pelican object ls command implementation

New command added pelican object ls that lists the objects of the specified dir in a namespace (or on the namespace itself). This PR also comes with tests for this new command as well as tests for a helper function to obtain 'dirlisthost'. The new command with a few sets of flags that I felt were the most important from the standard 'ls' command.
This still needs a little more polishing, but I thought to go ahead and make the PR if anyone wants to take a look and to push my current progress on this new command

This also comes with a rough design doc I used for my thought process: https://docs.google.com/document/d/1bCaFpQxbmySAnv2PT4jB6SP_45I2DuMfjstV2iE0HaM/edit?usp=sharing

Addressed most of the PR comments with this latest commit. Again, still needs a little polishing but wanted to keep track of current changes.


Add SSH agent client with an example

I'm not happy with the API (especially the Client requiring type for the ListeningSocket) but overall this works so I'm sharing for any potential feedback :)

Looks great! I've added a few comments around the error handling (it may be good to expose a clearer error to the consumer, rather than returning UnexpectedResponse), but otherwise this looks great! The example is super clear as well, nice work!

Great comments about the errors! As I mentioned in the other comment I think this is a bigger problem and we should be migrating away from the Box into maybe a more specific error type... I'll check it out in another PR but I consider it mostly orthogonal to this PR (as it applies to all others too :( ).

Looks great to me! The API here (mirroring the agent-side Session) is very clean!

Looks great to me! The API here (mirroring the agent-side Session) is very clean!

Haha, thanks! :blush: It occurred to me that thanks to the trait being the same one can build and agent that simply forwards requests to another agent with relative ease :)
I'm going to merge it now to avoid API drift even though I had a couple of tweaks in mind.


[Windows] Fix more GCC MinGW warnings.

Fixes the following MinGW / GCC 12.2.0 warnings added by a recent changes:
platform/windows/os_windows.cpp: In member function 'virtual Vector<String> OS_Windows::get_video_adapter_driver_info() const':
platform/windows/os_windows.cpp:335:41: warning: ISO C++ forbids converting a string constant to 'BSTR' {aka 'wchar_t*'} [-Wwrite-strings]
  335 |         hr = wbemLocator->ConnectServer(L"root\\CIMV2", NULL, NULL, 0, NULL, 0, 0, &wbemServices);
      |                                         ^~~~~~~~~~~~~~
platform/windows/os_windows.cpp:335:72: warning: passing NULL to non-pointer argument 5 of 'virtual HRESULT IWbemLocator::ConnectServer(BSTR, BSTR, BSTR, BSTR, LONG, BSTR, IWbemContext*, IWbemServices**)' [-Wconversion-null]
  335 |         hr = wbemLocator->ConnectServer(L"root\\CIMV2", NULL, NULL, 0, NULL, 0, 0, &wbemServices);
      |                                                                        ^~~~
In file included from platform/windows/os_windows.cpp:56:
/opt/homebrew/Cellar/mingw-w64/10.0.0_3/toolchain-x86_64/x86_64-w64-mingw32/include/wbemcli.h:1538:14: note:   declared here
 1538 |         LONG lSecurityFlags,
      |         ~~~~~^~~~~~~~~~~~~~

Thanks!


Fixed indentation to follow PEP8 guidelines.

https://www.python.org/dev/peps/pep-0008/#indentation

Use 4 spaces per indentation level.

openpilot uses 2 spaces like Google. We decided not to follow PEP8 here.

If you want you can contact me on our discord (Willem Melching#9790) if you want to discuss some style changes before you spent the time making a PR.

Would it be better to just open issues for all files that violate pep8?


None of the examples run. It is wonderful that some hyper advanced flutter guy can run this. But for noobs it is frustrating and useless

Error: Couldn't resolve the package 'ffi' in 'package:ffi/ffi.dart'.
Error: Couldn't resolve the package 'win32' in 'package:win32/win32.dart'.
example/hello.dart:8:8: Error: Not found: 'package:ffi/ffi.dart'
import 'package:ffi/ffi.dart';
^
example/hello.dart:10:8: Error: Not found: 'package:win32/win32.dart'
import 'package:win32/win32.dart';
^
example/hello.dart:14:10: Error: Getter not found: 'WM_DESTROY'.
case WM_DESTROY:
^^^^^^^^^^
example/hello.dart:15:7: Error: Method not found: 'PostQuitMessage'.
PostQuitMessage(0);
^^^^^^^^^^^^^^^
example/hello.dart:18:10: Error: Getter not found: 'WM_PAINT'.
case WM_PAINT:
^^^^^^^^
example/hello.dart:19:25: Error: 'PAINTSTRUCT' isn't a type.
final ps = calloc();
^^^^^^^^^^^
example/hello.dart:19:18: Error: Method not found: 'calloc'.
final ps = calloc();
^^^^^^
example/hello.dart:20:19: Error: Method not found: 'BeginPaint'.
final hdc = BeginPaint(hWnd, ps);
^^^^^^^^^^

Guides on how to use packages is can be found here:
https://dart.dev/guides/packages
If you're in the folder root, you'll run dart pub get to download its dependencies, and then you can run:
dart example\hello.dart

to run an example.


how to unload model

hi
how am i unload model,
i cant found
thx

Hi.
Good idea, I'll add a button.

thanks i love the way you make this extension but after genearate prompts my a1111 is very slow I try use unload in other extension its not work lol
and sorry for no good in English

Done.
Please check a ❌ button right of model selector.
It should unload model (if it was loaded before) and write a result in WebUI console.
Smth like:
Model unloaded, GPU memory freed: 3827.61 MB

by the way do you interest about ComfyUI

No, I'm not using and have no plans to use ComfyUI near future.


Bug: fix columns in wizard forms

Columns not aligned properly and coming in a new line even though it is using col-6 columns

Resolved.


[VL] support array_join

What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
(Fixes: #ISSUE-ID)
How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

@PHILO-HE Could you review this PR？


Not producing correct result

After successfully compiled the program with tensorflow 1.3.0, I tried running the sample using
python -m src.flownet2.test --input_a data/samples/0img0.ppm --input_b data/samples/0img1.ppm --out ./
However， the program is not producing expected flow result, instead, it produces something as attached, and the result is different for each run.
Has anyone encountered this? Thanks in advance!
Produced results (all running on the same sample images):

`(tensorflow_env_py2_test1) zhouchen@pku-PowerEdge-R730:/nfs-pku/zhouchen/code/flownet2-tf/flownet2-tf$ python -m src.flownet2.test --input_a data/samples/0img0.ppm --input_b data/samples/0img1.ppm --out ./
2017-09-20 14:07:17.244554: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-20 14:07:17.244621: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-20 14:07:17.244638: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-09-20 14:07:17.244650: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-20 14:07:17.244662: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2017-09-20 14:07:18.136647: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 0 with properties:
name: Tesla K40m
major: 3 minor: 5 memoryClockRate (GHz) 0.745
pciBusID 0000:83:00.0
Total memory: 11.92GiB
Free memory: 11.85GiB
2017-09-20 14:07:18.136753: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0
2017-09-20 14:07:18.136772: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0:   Y
2017-09-20 14:07:18.136791: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla K40m, pci bus id: 0000:83:00.0)
restored parameters from ./checkpoints/FlowNet2/flownet-2.ckpt-0
max flow: 20.7357
flow range:
u = -16.198 .. 7.102
v = -13.953 .. 7.434
(tensorflow_env_py2_test1) zhouchen@pku-PowerEdge-R730:/nfs-pku/zhouchen/code/flownet2-tf/flownet2-tf$ python -m src.flownet2.test --input_a data/samples/0img0.ppm --input_b data/samples/0img1.ppm --out ./
2017-09-20 14:07:47.206915: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-20 14:07:47.206981: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-20 14:07:47.206997: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-09-20 14:07:47.207009: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-20 14:07:47.207022: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2017-09-20 14:07:48.100539: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 0 with properties:
name: Tesla K40m
major: 3 minor: 5 memoryClockRate (GHz) 0.745
pciBusID 0000:83:00.0
Total memory: 11.92GiB
Free memory: 11.85GiB
2017-09-20 14:07:48.100631: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0
2017-09-20 14:07:48.100648: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0:   Y
2017-09-20 14:07:48.100665: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla K40m, pci bus id: 0000:83:00.0)
restored parameters from ./checkpoints/FlowNet2/flownet-2.ckpt-0
max flow: 34.9598
flow range:
u = -24.255 .. 18.996
v = -25.298 .. 14.050
`

Hi!
I had a similar error when I built against the wrong CUDA capabilities version (see this commit: https://github.com/sampepose/flownet2-tf/commit/c74bc8e678734c7db7bef346c41e1f9debd23b39).
You may need to hardcode the correct --gpu-name to match your GPUs Compute Capabilities. By default, it is sm_30. It's annoying because compiling for the wrong GPU doesn't yield any compile or runtime errors...it just outputs incorrect results as you've seen here.
Hope this helps.

Thank you very much for the reply!
I managed to successfully run the code on another machine but the problem is not solved. I will try out your suggestion!

Any update on this?

Your suggestion just works, thanks!
When explicitly specifying the architecture as
-GPUCFLAGS = -c -gencode=arch=compute_30,code=sm_30
the program works.


Pk/stream warnings to terminal v1

Description:

Is this change user facing?
YES/NO

References (if applicable):

This gives me an ability to stream warning from starlark to cli on demand.
In next PRs, I can add formatting so that the warning is yellow in colour and also to make sure only one warning is shown irrespective how many times an instruction is called. At the moment; if I am printing a warning in say add_service and calling add_services 10 times. The warning will appear 10 times ( which is not good)


Build error

I'll get this build errors on android after adding this plugin:
/node_modules/capacitor-plugin-webview-cache/android/src/main/java/com/fs/htmlclient/plugins/webviewcache/ClearCache.java:3: error: package android.webkit does not exist import android.webkit.WebView;
/node_modules/capacitor-plugin-webview-cache/android/src/main/java/com/fs/htmlclient/plugins/webviewcache/ClearCache.java:7: error: cannot find symbol     private WebView webView;             ^   symbol:   class WebView   location: class ClearCache
The compile and target SDK are set to 31, using capacitor 3.4.3

If android.webkit cannot be imported you probably are missing a project/sdk reference. The error is not introduced by this plugin.


Wire-Screens inaccurate with decimal numbers when all settings are disabled

Describe the bug
WireMod screens showing inaccurate values when Text-Screens show normally
To Reproduce
Steps to reproduce the behavior:

Spawn Screen with all settings off except "Only one value" turn that on
Create Constant Value with a number with decimals. I used:<PHONE_NUMBER>
Spawn Constant Value
Wire Screen to Constant Value
See horribly inaccurate data on screen

Expected behavior
Screen should just output what was wired in when all format settings are off.
As u can see it displays perfectly on the Text-Screen when Wired to a "Number to String" gate
Screenshots

This is just floating point imprecision because NWVars use floats instead of doubles. Could try sending a string representation of the number and having it tonumbered if we want double precision.

Anything to have it sync better because with any decimal it fails especially a small number like 4.7 or 57.81 it spits out a bunch of gook

could split it into two floats, like a SetNWDouble, but Idk if I'd trust the two float to always sync at the same time

could split it into two floats, like a SetNWDouble

How would this work?

I was working on a feature for the Round Gate to add the functionality to set the amount of decimal places to round by and I found out it is useless until the screen inaccuracy is fixed in some way.
https://i.imgur.com/wSCd04b.png

@Denneisk Stringstream from starfall can do this
local ss = bit.stringstream()
ss:writeDouble(d)
ss:seek(1)
local f1, f2 = ss:readFloat(), ss:readFloat()

SetNWString would be the better solution though I think.

If u do get this updated and working would u mind adding this to wire/lua/wire/gates/arithmetic.lua Line 145:

It adds the functionability to insert a decimal place amount to the rounding and if nothing is put in it rounds it normally
I never made a Commit before soooo

You could PR that yourself if you wanted to.

Sent out a Request

Any updates on this issue?


Ability to configure a warning on SQL execution number #8

I ve tested it manually with the following properties :
quickperf.database.operations.sqlExecutionDetected=true
quickperf.database.operations.sqlExecutionThreshold=1
and it gave this
2022-10-19 15:09:16.100  INFO 68225 --- [  restartedMain] o.q.s.s.ASpringBootApplication           : Started ASpringBootApplication in 7.06 seconds (JVM running for 9.127)
Hibernate: select player0_.id as id1_0_, player0_.first_name as first_na2_0_, player0_.last_name as last_nam3_0_, player0_.team_id as team_id4_0_ from player player0_
Hibernate: select player0_.id as id1_0_, player0_.first_name as first_na2_0_, player0_.last_name as last_nam3_0_, player0_.team_id as team_id4_0_ from player player0_
Hibernate: select team0_.id as id1_1_0_, team0_.name as name2_1_0_ from team team0_ where team0_.id=?
2022-10-19 15:10:38.899  WARN 68225 --- [tp1496937262-36] s.QuickPerfHttpCallHttpCallWarningLogger :
GET 200 http://localhost:8080/json-with-n-plus-one-select
* [WARNING] You have reached your SQL executions threshold - 2 > 1
do not hesitate to give me any hints to improve the code or test coverage..

It seems great! Thanks Walid! I will look at this in more detail tonight.
For the test, you could use TestQuickPerfMessages. An example here.

just added a test

Great! Thanks! One last thing, could you please add a bit of documentation in the SQL query part?

I have just enabled the CI for you (since it's your first contribution)

I have just enabled the CI for you (since it's your first contribution)

The CI is failing because a license header is missing in the test class. Could you please add it?

I've just added the license

it should be ok now
I ve added something on the before filter to make the recorder work for this case

Great! Thanks! One last thing, could you please add a bit of documentation in the SQL query part?

yes of course, I will work on it tomorrow :)

Great! Thanks! One last thing, could you please add a bit of documentation in the SQL query part?

yes of course, I will work on it tomorrow :)

It would just remain to add a bit of documentation. It would be great if you would have a bit of time to do it.

All checks have passed

am on it :)

Great! Thanks! One last thing, could you please add a bit of documentation in the SQL query part?

yes of course, I will work on it tomorrow :)

It would just remain to add a bit of documentation. It would be great if you would have a bit of time to do it.

am on it :)

Thank you very much Walid!!!

Fix #8

Thank you very much Walid!!!

this has been very exciting
thank you jean !


Custom adapters converting between JOOQ and R2DBC types

For example between org.jooq.JSONB and io.r2dbc.postgresql.codec.Json. There should be a general mechanism to add further adapters.

Solved in  947868b


Wrong version in JS files

/dist js files having version: "3.1.3" while currently expected to be 3.5.6
https://github.com/sampotts/plyr/blob/74e3990604698b1f2af3ed51e65b0ca820c1d624/dist/plyr.polyfilled.mjs#L250

That version is unrelated to Plyr, it's from a polyfill. Thanks for checking though.

@sampotts, are you saying Plyr code doesn't mention its version?


Added toggle-children option to docs

Per the discussion here:
https://github.com/alexandernst/angular-multi-select/pull/57

Perfect!


Color boxes not showing for LESS variables specifically

Issue Type: Bug
In a document formatted for LESS, the following does not show color boxes like it does if the hex codes are listed somewhere else. This is different before today. There was an update that was just installed for VSCode that may have broke this.
@1: #F1FFDB;
@2: #B5E8A8;
@3: #DEE7A8;
@4: #F9BB4F;
@5: #F58513;
VS Code version: Code 1.43.0 (78a4c91400152c0f27ba4d363eb56d2835f9903a, 2020-03-09T19:34:44.548Z)
OS version: Darwin x64 19.3.0

System Info

Item
Value

CPUs
Intel(R) Core(TM) i5-7500 CPU @ 3.40GHz (4 x 3400)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmetal: disabled_offmultiple_raster_threads: enabled_onoop_rasterization: disabled_offprotected_video_decode: unavailable_offrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledviz_display_compositor: enabled_onviz_hit_test_surface_layer: disabled_off_okwebgl: enabledwebgl2: enabled

Load (avg)
2, 2, 2

Memory (System)
24.00GB (3.90GB free)

Process Argv

Screen Reader
no

VM
0%

Extensions (10)

Extension
Author (truncated)
Version

bracket-pair-colorizer-2
Coe
0.0.29

githistory
don
0.6.2

prettier-vscode
esb
3.20.0

minify
Hoo
0.4.3

applescript
idl
0.18.1

vscode-edit-csv
jan
0.2.6

json2csv
kha
0.0.1

es6-css-minify
olb
3.3.2

json-template
sto
0.3.1

vscode-icons
vsc
10.0.0

(1 theme extensions excluded)

Here is another example that shows hex colors are being treated differently.

Duplicate of #92417


Is Field or Control triggers?

The description is now ambiguous as both is used. In IMHO it should be called "control trigger".

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: df0e36d7-395e-ed94-2e77-f7edb74e7757
Version Independent ID: 1f838a73-fefc-8fbd-bcb9-cc9ad899dbdd
Content: Page and Action Triggers - Business Central
Content Source: dev-itpro/developer/triggers/devenv-page-and-action-triggers.md
Service: dynamics365-business-central
GitHub Login: @SusanneWindfeldPedersen
Microsoft Alias: solsen

Thanks for the feedback. The triggers are at this point auto-generated from source code, sorted, and searchable in the table of contents. They are sorted by Field triggers (table fields) and Page Field triggers (control triggers). Thanks.


Support analyzer options

Support analyzer options, if only initially to include the analyzer pedantic package options. IMHO, the pedantic analyzer options offer good feedback for folks (esp. beginners) working on (embedded) DartPad exercises. /cc @kwalrath @legalcodes

package:lints and package:flutter_lints are now used by default! @chalin did you have a specific request for any other analysis option?

package:lints and package:flutter_lints are now used by default! @chalin did you have a specific request for any other analysis option?

Woohoo! That's great! No, not that I can recall, thanks.


Transaction filter

Description
Created a transaction filter. The structure is largely similar to a block filter since it was referenced, but it is designed to enable search for each data using TxResult.
Additionally, I utilized the method chaining to implement a query-like search. This allows the application of complex filtering criteria in a simplified manner.
Example Usage
filteredTxResults := NewTxFilter().
    Height(100).
    GasUsed(500, 1000).
    Apply()

f.ClearConditions()

Hey @notJoon,
Thank you for the PR 🙏
I apologize for the late reply on this, it slipped through my notifications.
I'll take a look at the PR soon and give feedback, I have a feeling we can improve it just by skimming through

I think we can avoid applying filters with elements on memory and do it directly using pebbleDB and their properties to iterate through keys in order.
Have a look at #12 . This approach can evolve even more adding secondary indexes. Not needed right now because queries take less than 20ms.

I think we can avoid applying filters with elements on memory and do it directly using pebbleDB and their properties to iterate through keys in order.

yeah, I think that would be much efficient than this filtering things.

Hey @notJoon, what's the status on this PR? Did you get a chance to check the comments?

Hey @notJoon, what's the status on this PR? Did you get a chance to check the comments?

I apologize for the late confirmation. I should be able to get it all done by in this weekend.


New dockerfile for running pre-commit checks in the CI pipeline

Adds a new dockerfile with pre-commit installed which can be used to run pre-commit in the openshift CI/CD.
Signed-off-by: ashish<EMAIL_ADDRESS>

/lgtm

/approve

/retest

/retest

/retest


Yelp API is failing in Postman

{
"error": {
"text": "Signature was invalid",
"id": "INVALID_SIGNATURE",
"description": "Invalid signature. Expected signature base string: GET&https%3A%2F%2Fapi.yelp.com%2Fv2%2Fsearch&location%3DSan%2520Francisco%26oauth_consumer_key%3DHJ0FdPSYP0GlbIAKq-Wdcw%26oauth_nonce%3DKyv7x3%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1467023743%26oauth_token%3DpO9EzdIt_C8z5obGMmqtFJ9mFepDA8vE%26oauth_version%3D1.0%26term%3Dfood"
}
}
http://my.jetscreenshot.com/demo/20160627-8rjq-18kb.jpg
http://my.jetscreenshot.com/demo/20160627-f1i0-32kb.jpg
Also i try to clear timestamp, Nonce and oauth_signature  - PostMan generate it, but i have the same error

Hi prsmax,
Your screenshots have expired :( Are you still having this problem? If so, can you give me some more info about how you made your request? (Maybe this was in your screenshots.)

"%2520" looks double-escaped to me.
How are you entering the location?
Does location "Boston" (no spaces) work?
Again, the answers may have been in your screenshots, but they've expired.


Issue with automatic refueling

So here's the thing.
I have 6+ stacks of graphite in my inventory, and my mecha is fueled by 4 stacks of it.
I also have 5 stray coal pieces, and I decide I want to burn them, so I swap the last slot of my mecha's fuel chamber for these few pieces of coal, and put the stack of graphite in my inventory.
A few seconds later, the mod simply deleted all the graphite in my inventory. The 3 stacks left in my mecha are still there.
EDIT : Looks like it deleted all other types of fuel as well ... Tsk. i'm disabling auto refuel for now.

I definitely changed this code when I was adding support for sprayed items. I'll check it out

Thanks !

The fix should be out there now. I've been using sprayed AM rods so I didn't notice it because they burn so slow.
But geez, great bug report, thank you very much. I can't believe I let it past.
To make up for it, what do you think about a button that would players turn on a mode where the next logistics vessel worth of fuel items (for each type) is free?

What do you mean, make up for it ? Also free vessel worth of fuel ? I don't get it.
While we're talking about logistics, i have a few questions.
Mostly, what happens when things are removed from my inventory (because recycling them manually, they exceed my quota, or I drop them on the ground), but there is no station set to the corresponding item, or they're completely full ?

What do you mean, make up for it ? Also free vessel worth of fuel ? I don't get it.

Was just saying I could try to recover what was lost by giving away free stuff.

what happens when things are removed from my inventory [...] but there is no station set to the corresponding item, or they're completely full

If there's no station anywhere for the item they shouldn't be picked up from the ground at all (or removed from recycle area). If there is a station it will first go to the buffer and then if the item is not currently requested it stays until a free station can be found for it.

Was just saying I could try to recover what was lost by giving away free stuff.
Nah, I guess it's fine, people should know (myself included) that installing mods could lead to some issues. "Remember guys, always backup" ;)
If there's no station anywhere for the item they shouldn't be picked up from the ground at all (or removed from recycle area). If there is a station it will first go to the buffer and then if the item is not currently requested it stays until a free station can be found for it.
And if I then remove the only station, while the item is still in the buffer ? Also is there a way to get items from the buffer manually ? (For example, if I recycled something that no station is configured to accept)

And if I then remove the only station, while the item is still in the buffer ?

It gets stuck. The buffer is persisted in a file next to your game save so it shouldn't ever be completely lost, though (unless the game is saved with the mod disabled)

Also is there a way to get items from the buffer manually ? (For example, if I recycled something that no station is configured to accept)

Yeah, if you click the Settings icon it will open the legacy UI window. There is a Buffered Items tab. You should be able to use "Add to inventory" from there.

Well then I think it's all good. I'll re-enable refueling then, and will post a new bug report if I notice something odd again. Thanks !
I'm closing this.


listing of large directories in gcs

Listing of large directories in gcs bucket is slow since the maxListItemsPerCall in GoogleCloudStorageOptions is not configurable and default 1024 is too low for large buckets/directories resulting in large number of round-trips to the server

Thanks for the suggestion! At the moment, since GCS doesn't promise returning the full maxResults number of items per page, it does appear currently to still clip to 1000 (less than the requested 1024) items per round trip; you can play around with the real GCS behavior using the apis-explorer to see if your experiences match this observation.
That said, you're right that we should at least expose it as a Hadoop configuration for ease of use when/if GCS supports larger pages on the backend in the future. That was the original intent with wiring it at least partially into GoogleCloudStorageOptions. I've added tracking for this request on our internal issue tracker as well.

By the way, AWS S3A code in hadoop requests  5000 entries for directory listing per round trip


and getter and setter for response cache

Gonna close this PR as I'm working on v4 which will have a solution for this issue ✌

Actually gonna base the implementation off of this work, so...reopening time!

well that was short-lived. I rebased it and am finishing this up over in #47 . Closing 😆


Machine win-7 went dead this morning from monitoring website.

Machine win-7 went dead this morning from monitoring website.

Network connection lost connection from the cable, I rebooted machine win-7 to restore the connection.


Kivycatalog fix

Fixes an issue with how KivyCatalog reloads when changing screens, which I think may have caused #2791. Also prevents Spinner from trying to reopen itself when selecting an item (on select changes is_open to False which calls dismiss which triggers on_dismiss which calls toggle_dropdown setting is_open back to True).

@kived :+1:


4个方向扫描的Mamba Block是否可以共享参数

作者您好！

config中注释似乎表明 4用于不同方向的Mamba Block如果是同一组参数会无法识别？ 如果共享一组参数的话，性能会下降多少呢？
用同一组参数不work的原因可能是不同顺序的patch序列输入扰乱的学习么？

The reason that we do not use the shared parameters for four-way scan is mainly that it had no speed up. The bottleneck of SSM is the selective scan operation (or the prefix sum), but not the procedure of preparing the parameters. Even when we share parameters, we need to calculate prefix sum four times (parallelly or serially), as the prefix sum in cuda library needs to scan from the very first thread to the last one each time, without alternative order. see cub::blockscan for details.
We did not finish running the version of share_ssm, the performance is about 1 point below the normal forwardv0 in around epoch 30, as I remember.
we may release the code of share_ssm and share_a and do this ablation later when we are about to write the paper.


Features/beatmap difficulty metadata

Proposed Changes
Addresses #6
Stores metadata of each difficulty (as an object instead of true) when parsing a map, including:

Number of notes
Number of obstacles
Duration (time of last note)

Platforms
This pull request modifies: (select all that apply)

[ ] Client
[x] Server

Types of Changes
This pull request is contains: (select all that apply)

[ ] Bug fixes (non-breaking change which fixes an issue)
[ ] New features (non-breaking change which adds functionality)
[x] Breaking changes (fix or feature that would cause existing functionality to not work as expected)

Checklist

[x] I have read the contribution guidelines
[x] I have checked the changes adhere to the project's style guide and all tests pass
[x] I have (to the best of my ability) checked for vulnerabilities, or any ways that my code could be abused and concluded that it is safe to run in production

Further Comments
This changes the booleans in the difficulties metadata to either be an object or null. While keeping the values correctly truthy / falsy, this could break some code checking the equality with true / false more strictly.
I guess a script would also need to be ran if we wanted to get this data on existing maps.
If you think this would break stuff, I am willing to either go for an alternative solution or wait for a better time to implement this.

Just saw the build fail, will update

Sorry for the delay. Passing the tests now. Let me know if you think it needs to be done differently when you have time to get to it.

Will review in the morning

Additionally, I think it'd be best to put the beatmap data in the characteristics array, and keep the difficulties array as-is.
Motivation

difficulties is good for quick searches
Beatmaps can have more than one of any difficulty using characteristics

Sounds good, will try to update later along with your other comments.
As far as I can see, characteristics is currently an array of strings. What do you think would be the best way to put data like the note counts in there (and still associate all the it with the correct difficulty)?

I was thinking something like this:
{
  // inside metadata
  "characteristics": [
    {
      "name": "Standard",
      "difficulties": {
        "easy": null,
        "normal": null,
        "hard": {
          "notes": 1000,
          "bombs": 50,
          "obstacles": 10,
          "duration": 120,
        },
        "expert": {
          "notes": 1000,
          "bombs": 50,
          "obstacles": 10,
          "duration": 120,
        },
        "expertPlus": {
          "notes": 1000,
          "bombs": 50,
          "obstacles": 10,
          "duration": 120,
        },
      }
    },
    {
      "name": "OneSaber",
      "difficulties": {
        "easy": null,
        "normal": null,
        "hard": null,
        "expert": {
          "notes": 1000,
          "bombs": 50,
          "obstacles": 10,
          "duration": 120,
        },
        "expertPlus": null
      }
    }
  ]
}

Sorry for the delay. Updated a bit to add things to the characteristics instead.
Still didn't take the time to add the beatmap file validation that you suggested. What would you like for that? Make sure each property exists?
I can look into making a script parsing existing beatmaps once everything is approved, just not sure when I'll be able to get to it.

For the time being, don't worry about validation. I'll open that as a separate issue later, as I have a few ideas of what I'd like to implement.

Wish I had thought to mention this sooner, but what about including NoteJumpSpeed too?

I haven't forgotten about this, I've just been busy recently.
Merge is waiting on a script to scan old characteristics, but I haven't had enough time to write one.
I have a few free days coming up so I can do it then.
If you wish to tackle it yourself, the songs are stored in zip files in subfolders.
EG:
cdn
├── 52d2
│   ├── 7c7f38d467bb43fe11a142581e63e324622ecc71.jpg
│   └── 7c7f38d467bb43fe11a142581e63e324622ecc71.zip
└── 52d3
    ├── 14c873851df7c4c7ef4a12d3817f6dd581bdb605.jpg
    └── 14c873851df7c4c7ef4a12d3817f6dd581bdb605.zip

Script written, awaiting results.
Will merge once I know it all worked!

Thanks for your hard work on this!

Sorry for the lack of response on the script part! I wasn't available much lately either. Thank you


Adding FSharp.CosmosDB link

@aaronpowell thank you! Could you please follow alphabet order?

Sorry @Neftedollar, fixed!

@aaronpowell thank you!


feat: Added reject aggregated measure request to outgoing messages

Description

References

#0000

Codecov Report

Merging #387 (e11a863) into main (21978a1) will decrease coverage by 68.18%.
Report is 90 commits behind head on main.
The diff coverage is 12.06%.

@@             Coverage Diff             @@
##             main     #387       +/-   ##
===========================================
- Coverage   83.07%   14.90%   -68.18%
===========================================
  Files         456      411       -45
  Lines       10418     7581     -2837
===========================================
- Hits         8655     1130     -7525
- Misses       1438     6433     +4995
+ Partials      325       18      -307

Files Changed
Coverage Δ

...ivedMessages/ArchivedMessageGetDocumentListener.cs
0.00% <ø> (ø)

.../ArchivedMessages/ArchivedMessageSearchListener.cs
0.00% <0.00%> (ø)

source/Api/Configuration/InboxMessageMetadata.cs
0.00% <0.00%> (ø)

...ation/Middleware/Correlation/CorrelationContext.cs
66.66% <ø> (ø)

...uration/Middleware/UnHandledExceptionMiddleware.cs
0.00% <0.00%> (ø)

source/Api/EventListeners/InboxEventListener.cs
0.00% <0.00%> (ø)

...essages/RequestAggregatedMeasureMessageReceiver.cs
0.00% <0.00%> (ø)

...equestAggregatedMeasureTransactionQueueListener.cs
0.00% <0.00%> (ø)

...rce/Api/OutgoingMessages/DequeueRequestListener.cs
0.00% <0.00%> (ø)

source/Api/OutgoingMessages/PeekRequestListener.cs
0.00% <0.00%> (ø)

... and 177 more

... and 289 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


update code quality services (for Python)

This PR fixes links to the Turing Way sections about online code quality services, which were all broken due to some updates there.
For Python specifically:

landscape.io doesn't exist anymore (at least over https)
add Sonarcloud, which is the default choice in the python template

Note to reviewer: the link checker is failing because of dead links in master. We should make a separate PR to fix them. In this PR, landscape.io is removed, which at least removes two of the dead links.

As discussed, I replaced the specific suggestions with a link to The Turing Way. I expanded a bit on our default Python template choice, including @sverhoeven's suggestion to use SonarLint locally.


Could not install package in VS 2015

Could not install package 'StockSharp 4.3.7'. You are trying to install this package into a project that targets '.NETFramework,Version=v4.6', but the package does not contain
any assembly references or content files that are compatible with that framework. For more information, contact the package author.
Works ok in VS 2013. In VS 2015 the package https://www.nuget.org/packages/StockSharp cannot be installed.

This issue was moved to NuGet/Home#1542


dicomParser is including node.js buffer code

While inspecting the source for https://rawgit.com/cornerstonejs/dicomParser/master/examples/dumpWithDataDictionary/index.html, I noticed in the network debug window that dicomParser.min.js was 159K, much larger than I expected.  I looked at the resonse tab and noticed a copyright notice for a node.js buffer module which is not needed for the browser version at least, since it uses TypedArrays.  This likely got added as part of the change to WebPack.  @dannyrb

related:

https://github.com/webpack/webpack/issues/4872
https://stackoverflow.com/questions/36439872/webpack-includes-unused-library-in-bundle-js/42744989


Provide *.tar.gz on PyPI

Could you provide *.tar.gz on PyPI as it was made earlier? The recipes in Buildroot are tailored to this format and handling *.zip files requires extra actions with folder renaming etc.

We can't easily. PyPI now restricts install of sdists to only one form, and the canonical form for historical reasons is .zip. To distribute with a tarball, we would have to drop zip files.


Add support for Slack/Discord unfurling

https://medium.com/slack-developer-blog/everything-you-ever-wanted-to-know-about-unfurling-but-were-afraid-to-ask-or-how-to-make-your-e64b4bb9254
Links like https://phish.in/1995-10-31 don't expand in Slack/Discord currently.

Implemented https://github.com/jcraigk/phishin/commit/edcaa7c3f8a4e0017d7c13ca1f5e037cc24bebe0


Add action, possession info to Permission instance

Hi,
In the IAccessInfo I can only see denied. Is "granted" missing ?
Thank you
Ismail

Hi. IAccessInfo only has a denied flag. See constructor of Access class for details.
Closing this. Pls continue thread if you need to.

Hi,
Sorry, my question was more about naming variables and functions which deal with the function granted.
I mean, in my application, I only check if my permissions are granted, for this I declare my permissions like this:
const queryInfo: IQueryInfo =  {     resource: 'user',     action: 'create',     possession: 'any'   }
Then I get the "granted" value with accessControl.permission(...).granted, and I want my accessInfo to be like:
const accessInfo: IAccessInfo = [   {     resource: 'user',     action: 'create',     possession: 'any',     granted: true // result of accessControl.permission(...).granted   };
As I check if my permission is granted, I want to deal with a value called "granted" which is not present in the interface IAccessInfo. There is only denied. Of course "denied" means "not granted", but I want my roles system to drive in one way: isGranted = granted, and not isGranted = !denied
Thanks,
Ismail

First of all, both your IQueryInfo and IAccessInfo objects should have a role property.
Second,  as mentioned before, IAccessInfo does not have it. granted is a property of AccessControl~Permission instance.
An access is not granted simply by setting some value to true. Internally, it's granted or denied by resource attributes. So if a role is allowed even only a single attribute of a resource, the permission query will return granted as true. If no attributes are allowed, it will return granted as false. But you should take attributes into account to see what (or how much of it) is actually granted.
That denied property of Access inner class only indicates the initial state. It's actually an internal flag, you shouldn't really depend on.
An Access instance or IAccessInfo interface defines a privilege. For checking whether that privilege is granted or denied, you should make use of the Permission inner class (and IQueryInfo interface).
Granting Access
// granting "video" resource for the "user" role, with all attributes:
ac.grant('user').create('video');
// is equivalent to
ac.grant('user').create('video', ['*']);
// is equivalent to
const access: IAccessInfo = {
    role: 'user',
    resource: 'video',
    action: 'create',
    possession: 'any',   // defaults to "any" if omitted
    attributes: ['*']
};
ac.grant(access);

Denying Access
ac.deny('user').create('video');
// is equivalent to
ac.grant('user').create('video', []); // grant no attributes
// is equivalent to
const access: IAccessInfo = {
    role: 'user',
    resource: 'video',
    action: 'create:any',
    attributes: []   // <—— no attributes
};
ac.grant(access);

Querying for Permission
const permission = ac.can('user').create('video');
console.log(permission.granted);

or you can:
const query: IQueryInfo = {
     role: 'user',
     resource: 'video',
     action: 'create',
     possession: 'any'
 };
const permission = ac.permission(query);
console.log(permission.granted);

Hello,
Yeah of course the role was missing in my code above but I add it in another step in my code.
To check if I have the permission I do the last example you said:
const query: IQueryInfo = {
     role: 'user',
     resource: 'video',
     action: 'create',
     possession: 'any'
 };

const permission = ac.permission(query);

Then I do this:
const response: -_TYPE_- = {
     role: 'user',
     resource: 'video',
     action: 'create',
     possession: 'any',
     granted: permission.granted
 };

I use the Permission inner class. The query has type IQueryInfo, but what is the type of this object 'response' above ? I return this to the front, then in front I authorize access to my components depending on the value "granted" in the object response (of course which depends on attributes)
Currently I created an interface IAccessInfo which extends IQueryInfo and I assign this type to my object response:
interface IAccessInfo extends IQueryInfo {
  granted: boolean;
}

I called it IAccessInfo because it is looking to be the closest name type for my case. But I hope there is a better solution
Thank you for the answer !
Ismail

Don't do that. IAccessInfo is a core interface already.
You have the Permission class.
const permission: Permission = ac.permission(query);

This has granted: Boolean, roles: string[], resource: String, attributes: string[] properties you need (and more)...
I see you also need action and possession in the Permission instance.
I can consider this as a feature request. Until implemented, you can do this:
interface IPermission extends IQueryInfo {
  granted: boolean;
}

Oki, I hope we'll have this feature very soon !
Thank you very much ! See you !

By the way, this issue is labelled "invalid", should it be changed ?

Yes. Now that we have an understanding, I'm re-opening this issue as a feature-request.

Oki thank you !
Actually it is my first issue on Github, I'm very happy we have an understanding, I'll try to be more explicit with more details for my future issues on Github :)

@onury is there any other medium to get in touch with you.

Is there any way we can check do something like as updateOwn based on user belong to team A or team B
ac.grant('member')
  .createOwn('post', ['*'])
  .readOwn('post', ['*'])
  .updateOwn('post', ['*'], {
    // Condition: Only allow update if the user has the 'canUpdatePosts' property
    condition: (user, resource) => user.team == 'A',
  })
  .deleteOwn('post', ['*'], {
    // Condition: Only allow delete if the user has the 'canDeletePosts' property
    condition: (user, resource) => user.team == 'A',
  });


ec2_instance_type is not respecting AWS_MAX_ATTEMPTS env var

Describe the bug
Queries to the ec2_instance_type table do not appear to be respecting the max attempts set through the AWS_MAX_ATTEMPTS env variable
Steampipe version (steampipe -v)
v0.19.0-rc.2
Plugin version (steampipe plugin list)
v0.93.0
To reproduce
Query the ec2_instance_type with caching off repeatedly
Expected behavior
The API requests should be retried according to the env var
Additional context
Related SPC issue: https://github.com/turbotio/steampipe-cloud/issues/1770

@cbruno10, I could not replicate the issue. I experienced the following observation.

Set the env variable AWS_MAX_ATTEMPTS with value 2 (export AWS_MAX_ATTEMPTS=2)
Removed max_error_retry_attempts config from aws.spc file.
Ran the steampipe query select * from aws_ec2_instance_type

With the above set of configuration the current retry and exponential back-off logic is seems to be working fine with maximum retry attempt 2.
If we are not setting any max attempt value in either wayenvironment variableor in spc file, then the default value is 9, but after setting up the AWS_MAX_ATTEMPTS env value to 2 it is retrying the only 2 times. So I concluded that if we are setting up the  AWS_MAX_ATTEMPTS env variable then it is picking up the value from env variable.
I am getting the following error after attempting 2 times.
> select * from aws_ec2_instance_type

Error: table 'aws_ec2_instance_type' column 'auto_recovery_supported' requires hydrate data from describeInstanceType, which failed with error operation error EC2: DescribeInstanceTypes, exceeded maximum number of attempts, 2, https response error StatusCode: 503, RequestID: 08f9f38f-6702-4e03-be4c-7dabbbb5c915, api error RequestLimitExceeded: Request limit exceeded..

Steampipe plugin version:  aws@latest v0.96.0
Steampipe SDK version: v5.2.0
Steampipe CLI version: v0.19.0-rc.5
I am not sure if I am doing anything wrong here?  Should i try this out with any other set of configuration?

@cbruno10 could you please provide some insights if I am doing anything wrong here?

@ParthaI Thanks for testing, I'll close this one for now since we can't reproduce, but will re-open if we see it again.


Update structure.rst

changed omega_b to omega_m for throughout consistency

The number in that table is Omega_b, not Omega_m. Omega_m varies across simulations

sigma8 is also mentioned in this table and it varies across simulations too. Does that need change?

good point! I have removed sigma_8 from the table. thanks!


What is recommended "concurent_request_per_domain" ?

It looks like it easy to overflow queue. Entire system down.  This happen when I set concurrency more than ~20. What is the recommended setting of concurrency for Crawly? What is performance can you achieve with it?
22:27:10.427 [error] GenServer #PID<0.527.0> terminating
** (MatchError) no match of right hand side value: {:empty, {[], []}}
    (hackney 1.15.2) /Users/mycomputer/Documents/Projects/Playgraound/homebase/deps/hackney/src/hackney_pool.erl:509: :hackney_pool.queue_out/2
    (hackney 1.15.2) /Users/mycomputer/Documents/Projects/Playgraound/homebase/deps/hackney/src/hackney_pool.erl:376: :hackney_pool.dequeue/3
    (hackney 1.15.2) /Users/mycomputer/Documents/Projects/Playgraound/homebase/deps/hackney/src/hackney_pool.erl:349: :hackney_pool.handle_info/2
    (stdlib 3.11.2) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib 3.11.2) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib 3.11.2) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message: {:DOWN, #Reference<0.4051887452.3045064705.41660>, :request, #PID<0.455.0>, :shutdown}
State: {:state, :default, {:metrics_ng, :metrics_dummy}, 50, 150000, {:dict, 50, 16, 16, 8, 80, 48, {[], [], [], [], [], [], [], [], [], [], [], [], [], [], [], []}, {{[[#Reference<0.4051887452.3045064705.22552> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064712.10994> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064710.25098> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064705.39221> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064708.41389> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064708.41530> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064708.49674> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064712.31884> | {'www.homebase.co.uk', 443, :hackney_ssl}]], [[#Reference<0.4051887452.3045064705.34134> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064711.22288> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064708.35098> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064709.19015> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064705.39210> | {'www.homebase.co.uk', 443, :hackney_ssl}]], [[#Reference<0.4051887452.3045064706.63253> | {'www.homebase.co.uk', 443, :hackney_ssl}]], [[#Reference<0.4051887452.3045064707.42573> | {'www.homebase.co.uk', 443, :hackney_ssl}]], [], [[#Reference<0.4051887452.3045064708.30170> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064709.13738> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064710.33158> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064705.41238> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064708.41539> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064706.61055> | {'www.homebase.co.uk', 443, :hackney_ssl}]], [[#Reference<0.4051887452.3045064709.21269> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064708.42098> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064706.61233> | {'www.homebase.co.uk', 443, :hackney_ssl}]], [[#Reference<0.4051887452.3045064712.11785> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064707.34068> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064712.26414> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064711.38039> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064712.34782> | {'www.homebase.co.uk', 443, :hackney_ssl}]], [], [[#Reference<0.4051887452.3045064712.8112> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064712.24733> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064707.43205> | {'www.homebase.co.uk', 443, :hackney_ssl}]], [[#Reference<0.4051887452.3045064708.22048> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064705.41660> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064707.35050> | {'www.homebase.co.uk', 443, :hackney_ssl}]], [[#Reference<0.4051887452.3045064705.24426> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064709.13941> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064709.18037> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064706.55708> | {'www.homebase.co.uk', 443, :hackney_ssl}]], [[#Reference<0.4051887452.3045064709.13898> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064712.23688> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064706.52408> | {'www.homebase.co.uk', 443, :hackney_ssl}]], [[#Reference<0.4051887452.3045064708.32218> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064707.33419> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064708.44346> | {'www.homebase.co.uk', 443, :hackney_ssl}]], [[#Reference<0.4051887452.3045064709.13751> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064708.41062> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064709.34340> | {'www.homebase.co.uk', 443, :hackney_ssl}]], [[#Reference<0.4051887452.3045064707.20603> | {'www.homebase.co.uk', 443, :hackney_ssl}], [#Reference<0.4051887452.3045064712.26880> | {'www.homebase.co.uk', 443, :hackney_ssl}]]}}}, {:dict, 1, 16, 16, 8, 80, 48, {[], [], [], [], [], [], [], [], [], [], [], [], [], [], [], []}, {{[], [], [], [[{'www.homebase.co.uk', 443, :hackney_ssl} | {[], []}]], [], [], [], [], [], [], [], [], [], [], [], []}}}, {:dict, 20, 16, 16, 8, 80, 48, {[], [], [], [], [], [], [], [], [], [], [], [], [], [], [], []}, {{[[#Reference<0.4051887452.3045064712.10994> | {{#PID<0.436.0>, #Reference<0.4051887452.3045064712.11107>}, {'www.homebase.co.uk', 443, :hackney_ssl}}], [#Reference<0.4051887452.3045064710.25098> | {{#PID<0.491.0>, #Reference<0.4051887452.3045064710.25100>}, {'www.homebase.co.uk', 443, :hackney_ssl}}], [#Reference<0.4051887452.3045064708.49674> | {{#PID<0.435.0>, #Reference<0.4051887452.3045064708.49679>}, {'www.homebase.co.uk', 443, :hackney_ssl}}], [#Reference<0.4051887452.3045064712.31884> | {{#PID<0.503.0>, #Reference<0.4051887452.3045064712.31886>}, {'www.homebase.co.uk', 443, :hackney_ssl}}]], [[#Reference<0.4051887452.3044802561.184150> | {{#PID<0.428.0>, #Reference<0.4051887452.3044802561.184156>}, {'www.homebase.co.uk', 443, :hackney_ssl}}]], [[#Reference<0.4051887452.3045064706.63253> | {{#PID<0.521.0>, #Reference<0.4051887452.3045064706.63260>}, {'www.homebase.co.uk', 443, :hackney_ssl}}]], [[#Reference<0.4051887452.3045064707.42573> | {{#PID<0.516.0>, #Reference<0.4051887452.3045064707.42615>}, {'www.homebase.co.uk', 443, :hackney_ssl}}]], [], [[#Reference<0.4051887452.3045064708.30170> | {{#PID<0.519.0>, #Reference<0.4051887452.3045064708.30172>}, {'www.homebase.co.uk', 443, :hackney_ssl}}], [#Reference<0.4051887452.3045064705.41238> | {{#PID<0.457.0>, #Reference<0.4051887452.3045064705.41240>}, {'www.homebase.co.uk', 443, :hackney_ssl}}], [#Reference<0.4051887452.3045064706.61055> | {{#PID<0.501.0>, #Reference<0.4051887452.3045064706.61080>}, {'www.homebase.co.uk', 443, :hackney_ssl}}]], [[#Reference<0.4051887452.3045064706.61233> | {{#PID<0.448.0>, #Reference<0.4051887452.3045064706.61248>}, {'www.homebase.co.uk', 443, :hackney_ssl}}]], [[#Reference<0.4051887452.3045064712.11785> | {{#PID<0.495.0>, #Reference<0.4051887452.3045064712.11789>}, {'www.homebase.co.uk', 443, :hackney_ssl}}], [#Reference<0.4051887452.3045064711.38039> | {{#PID<0.514.0>, #Reference<0.4051887452.3045064711.38065>}, {'www.homebase.co.uk', 443, :hackney_ssl}}], [#Reference<0.4051887452.3045064712.34782> | {{#PID<0.450.0>, #Reference<0.4051887452.3045064712.34787>}, {'www.homebase.co.uk', 443, :hackney_ssl}}]], [], [[#Reference<0.4051887452.3045064707.43205> | {{#PID<0.446.0>, #Reference<0.4051887452.3045064707.43464>}, {'www.homebase.co.uk', 443, :hackney_ssl (truncated)

@Unumus Normally I am using 5-10 connections to get ~200 - 300 items per minute on a local machine. The system is not supposed to shut down like this in any case. So let's explore the case and maybe improve Crawly's stability.
I would ask for the following information:

Crawly version
Spider code (if possible)
Your channel information
Some runtime information: how long the spider run, how many items, errors did you get. maybe if you could attach logs it could be useful as well.

After several tries, I got not identical behaviour.
12:47:14.629 [error] GenServer Crawly.RequestsStorage terminating
** (stop) exited in: GenServer.call(#PID<0.425.0>, {:store, %Crawly.Request{headers: [], middlewares: [Crawly.Middlewares.DomainFilter, Crawly.Middlewares.UniqueRequest, Crawly.Middlewares.UserAgent], options: [], prev_response: %HTTPoison.Response{body: "\r\n\r\n<!doctype html>\r\n<!--[if lt IE 7]> <html class=\"no-js lt-ie9 lt-ie8 lt-ie7 ie\" lang=\"en\"> <![endif]-->\r\n<!--[if IE 7]>    <html class=\"no-js lt-ie9 lt-ie8 ie\" lang=\"en\"> <![endif]-->\r\n<!--[if IE 8]>    <html class=\"no-js lt-ie9 ie\" lang=\"en\"> <![endif]-->\r\n<!--[if gte IE 9]> <html class=\"ie no-js\" lang=\"en\"> <![endif]-->\r\n<!--[if !IE]><!-->\r\n<html class=\"no-js\" lang=\"en\">\r\n<!-- <![endif]-->\r\n<head><meta http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\" /><meta charset=\"utf-8\" /><meta http-equiv=\"X-UA-Compatible\" content=\"IE=edge,chrome=1\" /><script type=\"text/javascript\">window.NREUM||(NREUM={});NREUM.info = {\"beacon\":\"bam.nr-data.net\",\"errorBeacon\":\"bam.nr-data.net\",\"licenseKey\":\"cb2fc5a887\",\"applicationID\":\"107236984\",\"transactionName\":\"NFEHZhYDDEpUB0FcXA0bN1cVFwdKQUtBUF4TWARGAU0yS1oAQFZHIFURVwMNEEA=\",\"queueTime\":0,\"applicationTime\":886,\"agent\":\"\",\"atts\":\"\"}</script><script type=\"text/javascript\">(window.NREUM||(NREUM={})).loader_config={licenseKey:\"cb2fc5a887\",applicationID:\"107236984\"};window.NREUM||(NREUM={}),__nr_require=function(e,n,t){function r(t){if(!n[t]){var i=n[t]={exports:{}};e[t][0].call(i.exports,function(n){var i=e[t][1][n];return r(i||n)},i,i.exports)}return n[t].exports}if(\"function\"==typeof __nr_require)return __nr_require;for(var i=0;i<t.length;i++)r(t[i]);return r}({1:[function(e,n,t){function r(){}function i(e,n,t){return function(){return o(e,[u.now()].concat(f(arguments)),n?null:this,t),n?void 0:this}}var o=e(\"handle\"),a=e(4),f=e(5),c=e(\"ee\").get(\"tracer\"),u=e(\"loader\"),s=NREUM;\"undefined\"==typeof window.newrelic&&(newrelic=s);var p=[\"setPageViewName\",\"setCustomAttribute\",\"setErrorHandler\",\"finished\",\"addToTrace\",\"inlineHit\",\"addRelease\"],l=\"api-\",d=l+\"ixn-\";a(p,function(e,n){s[n]=i(l+n,!0,\"api\")}),s.addPageAction=i(l+\"addPageAction\",!0),s.setCurrentRouteName=i(l+\"routeName\",!0),n.exports=newrelic,s.interaction=function(){return(new r).get()};var m=r.prototype={createTracer:function(e,n){var t={},r=this,i=\"function\"==typeof n;return o(d+\"tracer\",[u.now(),e,t],r),function(){if(c.emit((i?\"\":\"no-\")+\"fn-start\",[u.now(),r,i],t),i)try{return n.apply(this,arguments)}catch(e){throw c.emit(\"fn-err\",[arguments,this,e],t),e}finally{c.emit(\"fn-end\",[u.now()],t)}}}};a(\"actionText,setName,setAttribute,save,ignore,onEnd,getContext,end,get\".split(\",\"),function(e,n){m[n]=i(d+n)}),newrelic.noticeError=function(e,n){\"string\"==typeof e&&(e=new Error(e)),o(\"err\",[e,u.now(),!1,n])}},{}],2:[function(e,n,t){function r(e,n){var t=e.getEntries();t.forEach(function(e){\"first-paint\"===e.name?c(\"timing\",[\"fp\",Math.floor(e.startTime)]):\"first-contentful-paint\"===e.name&&c(\"timing\",[\"fcp\",Math.floor(e.startTime)])})}function i(e,n){var t=e.getEntries();t.length>0&&c(\"lcp\",[t[t.length-1]])}function o(e){if(e instanceof s&&!l){var n,t=Math.round(e.timeStamp);n=t>1e12?Date.now()-t:u.now()-t,l=!0,c(\"timing\",[\"fi\",t,{type:e.type,fid:n}])}}if(!(\"init\"in NREUM&&\"page_view_timing\"in NREUM.init&&\"enabled\"in NREUM.init.page_view_timing&&NREUM.init.page_view_timing.enabled===!1)){var a,f,c=e(\"handle\"),u=e(\"loader\"),s=NREUM.o.EV;if(\"PerformanceObserver\"in window&&\"function\"==typeof window.PerformanceObserver){a=new PerformanceObserver(r),f=new PerformanceObserver(i);try{a.observe({entryTypes:[\"paint\"]}),f.observe({entryTypes:[\"largest-contentful-paint\"]})}catch(p){}}if(\"addEventListener\"in document){var l=!1,d=[\"click\",\"keydown\",\"mousedown\",\"pointerdown\",\"touchstart\"];d.forEach(function(e){document.addEventListener(e,o,!1)})}}},{}],3:[function(e,n,t){function r(e,n){if(!i)return!1;if(e!==i)return!1;if(!n)return!0;if(!o)return!1;for(var t=o.split(\".\"),r=n.split(\".\"),a=0;a<r.length;a++)if(r[a]!==t[a])return!1;return!0}var i=null,o=null,a=/Version\\/(\\S+)\\s+Safari/;if(navigator.userAgent){var f=navigator.userAgent,c=f.match(a);c&&f.indexOf(\"Chrome\")===-1&&f.indexOf(\"Chromium\")===-1&&(i=\"Safari\",o=c[1])}n.exports={agent:i,version:o,match:r}},{}],4:[function(e,n,t){function r(e,n){var t=[],r=\"\",o=0;for(r in e)i.call(e,r)&&(t[o]=n(r,e[r]),o+=1);return t}var i=Object.prototype.hasOwnProperty;n.exports=r},{}],5:[function(e,n,t){func" <> ..., headers: [{"Date", "Thu, 19 Mar 2020 10:47:10 GMT"}, {"Content-Type", "text/html; charset=utf-8"}, {"Transfer-Encoding", "chunked"}, {"Connection", "keep-alive"}, {"Set-Cookie", "__cfduid=d53fbe3c8646bf32a8864ce2e207e0b101584614829; expires=Sat, 18-Apr-20 10:47:09 GMT; path=/; domain=.homebase.co.uk; HttpOnly; SameSite=Lax; Secure"}, {"Cache-Control", "no-cache, no-store"}, {"Pragma", "no-cache"}, {"Expires", "-1"}, {"Vary", "Accept-Encoding"}, {"Set-Cookie", "Bunnings.Device=default; path=/"}, {"Set-Cookie", "ASP.NET_SessionId=erdggz52redh5gzpjc4qsxeo; path=/; HttpOnly; SameSite=Lax"}, {"Set-Cookie", "Bunnings.Device=default; path=/"}, {"Set-Cookie", "ASP.NET_SessionId=erdggz52redh5gzpjc4qsxeo; path=/; HttpOnly; SameSite=Lax"}, {"Set-Cookie", "Bunnings.UserType=RetailUser; path=/"}, {"Set-Cookie", "Bunnings.NearestStoreID=335; expires=Fri, 19-Mar-2021 10:47:09 GMT; path=/"}, {"Set-Cookie", "Bunnings.NearestStoreName=Milton Keynes; expires=Fri, 19-Mar-2021 10:47:09 GMT; path=/"}, {"Set-Cookie", "Bunnings.NearestStoreNumber=0335; expires=Fri, 19-Mar-2021 10:47:09 GMT; path=/"}, {"Set-Cookie", "Bunnings.Region=Homebase; expires=Fri, 19-Mar-2021 10:47:09 GMT; path=/"}, {"Set-Cookie", "Bunnings.NearestStoreDisableOnlineOrderFulfilment=False; expires=Fri, 19-Mar-2021 10:47:10 GMT; path=/"}, {"Set-Cookie", "Bunnings.NearestStoreIsRemote=False; expires=Fri, 19-Mar-2021 10:47:09 GMT; path=/"}, {"Set-Cookie", "Bunnings.SelectedSuburbID=; expires=Mon, 09-Mar-2020 10:47:09 GMT; path=/"}, {"Set-Cookie", "Bunnings.SelectedPostcode=; expires=Mon, 09-Mar-2020 10:47:09 GMT; path=/"}, {"Set-Cookie", "__AntiXsrfToken=93447a2d6de14d08ba0333baa9bf33bd; path=/; HttpOnly"}, {"Set-Cookie", "__RequestVerificationToken=UaiwRUprXoSpPJl3guwJCOltQvtR8aOHTSnW4wrSw3AISghNpd5m4BloNGeXpez7SnwKHyPUXU_qxEmxKjHda0oKlyo1; path=/; HttpOnly"}, {"Set-Cookie", "CookieBannerDisplay=false; domain=www.homebase.co.uk; expires=Thu, 16-Apr-2020 09:47:09 GMT; path=/; HttpOnly"}, {"COMMERCE-SERVER-SOFTWARE", "Commerce Server, Enterprise Edition"}, {"X-Server-Region", "Homebase"}, {"X-Frame-Options", "SAMEORIGIN"}, {"Set-Cookie", "ApplicationGatewayAffinity=d5f42a2203cf101bdb79095de889e7ef00167d78417b6877d252c776b358cba7;Path=/;Domain=www.homebase.co.uk"}, {"Set-Cookie", "ApplicationGatewayAffinityCORS=d5f42a2203cf101bdb79095de889e7ef00167d78417b6877d252c776b358cba7;Path=/;Domain=www.homebase.co.uk;SameSite=None;Secure"}, {"CF-Cache-Status", "DYNAMIC"}, {"Expect-CT", "max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\""}, {"Server", "cloudflare"}, {"CF-RAY", "57669d1d1f1ed240-SOF"}], request: %HTTPoison.Request{body: "", headers: [{"User-Agent", "Crawly Bot 1.0"}], method: :get, options: [], params: %{}, url: "https://www.homebase.co.uk/our-range/storage-and-home/furniture/kids-furniture/kids-chest-of-drawers"}, request_url: "https://www.homebase.co.uk/our-range/storage-and-home/furniture/kids-furniture/kids-chest-of-drawers", status_code: 200}, retries: 0, url: "https://www.homebase.co.uk/softline-2-4-chest-of-drawers_p529432"}}, 5000)
    ** (EXIT) no process: the process is not alive or there's no process currently associated with the given name, possibly because its application isn't started
    (elixir 1.10.1) lib/gen_server.ex:1023: GenServer.call/3
    (elixir 1.10.1) lib/enum.ex:783: Enum."-each/2-lists^foreach/1-0-"/2
    (elixir 1.10.1) lib/enum.ex:783: Enum.each/2
    (crawly 0.8.0) lib/crawly/requests_storage/requ (truncated)

My config file

use Mix.Config
# in config.exs
config :crawly,
  closespider_timeout: 10,
  concurrent_requests_per_domain: 100,
  closespider_itemcount: 1000,
  middlewares: [
    Crawly.Middlewares.DomainFilter,
    Crawly.Middlewares.UniqueRequest,
    Crawly.Middlewares.UserAgent
  ],
  pipelines: [
    {Crawly.Pipelines.Validate, fields: [:title, :sku, :price]},
    {Crawly.Pipelines.DuplicatesFilter, item_id: :title},
    Crawly.Pipelines.JSONEncoder,
    {Crawly.Pipelines.WriteToFile, extension: "jl", folder: "/tmp"} # NEW IN 0.7.0
   ],
   port: 4001

The code here I played around the Homebase crawler.
defmodule Tools do
  @behaviour Crawly.Spider

  @impl Crawly.Spider
  def base_url(), do: "https://www.homebase.co.uk"

  @impl Crawly.Spider
  def init() do
    [
      start_urls: [
        "https://www.homebase.co.uk/"
      ]
    ]
  end

  @impl Crawly.Spider
  def parse_item(response) do
    product_categories = extract_href(response, ".article-tiles a")
    product_pages = extract_href(response, "a.product-tile  ")

    urls = product_pages ++ product_categories

    requests  =
      urls
      |> Enum.uniq()
      |> Enum.map(&build_absolute_url/1)
      |> Enum.map(&Crawly.Utils.request_from_url/1)

    item = %{
      title: extract_text(response, ".page-title h1"),
      sku: extract_text(response, ".product-header-heading span"),
      price: extract_text(response, ".price-value [itemprop=priceCurrency]")
    }

    %Crawly.ParsedItem{:items => [item], :requests => requests}
  end

  defp extract_text(response, selector) do
    response.body
    |> Floki.find(selector)
    |> Floki.text()
  end

  defp extract_href(response, selector) do
    response.body
    |> Floki.find(selector)
    |> Floki.attribute("href")
  end

  defp build_absolute_url(url), do: URI.merge(base_url(), url) |> to_string()
end

Yes, this is the death of request storage, where the first thing looked like an error in hackney (e.g. hackney did not handle the empty queue outs). In any case I will try to reproduce the error locally to come with some improvements for the case. Otherwise, can you downgrade the concurrency to 10 to see how stable it is?

Before I reduce concurrency to 10. I should notice that I re-run the same spider with concurrency 100. This time it was finished normally.
13:00:35.075 [info]  Current crawl speed is: 629 items/min

13:00:35.075 [info]  Stopping Tools, closespider_itemcount achieved

But in the Tools.jl file have only 1112 scraped items.

My current issue is that due to the COVID 19 I am at home with very basic internet connection. So it's hard to reproduce it. Could I ask you to commit it somewhere on github, so it's easy to just clone & run it.

Yes sure.
https://github.com/Unumus/for_oltarasenko

But in the Tools.jl file has only 1112 scraped items.

You need to tweak the closespider_itemcount: 1000,
E.g. try to have something like 10_000_000 there to see how it works..
Yeah, locally I can't get even a half of your speed, to see the error. I will be able to test once the COVID disaster ends... in the ideal scenario, luckily for us. However, I will try to invent a proper benchmarking system for Crawly.

I could scrape 7096 items, with concurrency of 100.
I will report new issue if they appear here.

But in the Tools.jl file has only 1112 scraped items.

You need to tweak the closespider_itemcount: 1000,
E.g. try to have something like 10_000_000 there to see how it works..
Yeah, locally I can't get even a half of your speed, to see the error. I will be able to test once the COVID disaster ends... in the ideal scenario, luckily for us. However, I will try to invent a proper benchmarking system for Crawly.

In my own usage, this was also a mystery for me when I was setting up my crawler, as the close count is set by default. I think adding some logging to advise users to tweak this setting when the itemcount is reached would help

@Unumus @Ziinc I kind of agree that 1000 is kind of strange and not intuitive at all. Shall we make it  disabled by default?

Sounds good to disable. Maybe can replace it with reporting the total items scraped count, which, if I'm not wrong, is already stored in the worker state.

@oltarasenko  Sorry for long reply, was busy with an other project.
Disabled it by default, it seems like a good idea!

Ok, done!


Errors on parsing elixir matches with maps

Environment
On neovim 0.8 and latest heirline
When looking at at file that contains some elixir map in a match for a  function declaration:
def test(%{"test" => test}) do
	test
end

I get an error:
E15: Invalid expression: "test" => test

This also happens with something like
def test(%{test: test}) do
	test
end

where I get:
E121: Undefined variable: test

and when I hammer it enough it comes out as this:

(excuse the screenshot, it's all I could grab of the issue
I originally thought this was an elixir-ls problem - but now it seems with that error that it stems from the _eval call from this plugin. I am still not sure it's not which-key? but it seems like the trace just shows that as being it's re-render function.
Thanks so much for the help!

That syntax is potentially able to be interpreted as a statusline format string and vim would try to execute whatever is inside %{ } as a vim expression. This is a problem if you had any provider returning the content of the current line. To prevent it, you need to make the output of such provider safe by substituting % with %%

Thank you for the quick response! it seems to be an issue with the AstroNvim heirline config then. I can try to track down the issue and open a PR/issue there

Glad to help, also thanks for bringing to my attention that AstroNVim uses heirline ;)

No problem! I think I narrowed it down to this provider, though I can't figure out why.
I think I will just open an issue there, and disable it locally for myself for now. Thanks again!

--- { provider = d.name }, -- add symbol name
+++ { provider = string.gsub(d.name, "%%", "%%%%")

Thanks so much for looking at this @rebelot ! I will test out the change you have shared !
Yeah AstroNvim has been getting ready to migrate to using Heirline as the statusline. It will be in the stable release we are making on Friday for AstroNvim v2.0. Great work on this plugin it is super great and very fast and has allowed us to build a great API for letting users easily customize the statusline in Heirline quite nicely. If you are curious, here are our new docs for customizing the statusline in AstroNvim with heirline and our status module: https://astronvim.github.io/nightly/Recipes/status

great project! keep an eye on the cookbook for updates ;)

@mehalter You might want to check out updates on navic component for multi-window support and better logic for clickable elements (does not pollute global namespace with redundant functions)
Converting it to work with aerial should be straightforward.

Thanks for the recommendation @rebelot ! Definitely super interested in improving the clickable implementation. I'll check out their implementation to help improve ours. I do want to continue using Aerial as the location engine mainly because it supports getting the location from more sources from just the LSP like if I just have treesitter running for example I can still get breadcrumbs for that. Also we are using Aerial already for having a symbol browser so not needing to rely on another plugin to get location when we have another plugin already calculating symbols as well. Hopefully that explanation is clear 😂 but will definitely check out the navic codebase to see how they are building the clickable elements

Checked out the code in the codebook and it looks great! Thanks so much for this, will definitely update our component in AstroNvim with this implementation!


Delete models installed from Ollama in my Mac to free the space

HI,
I installed two Llama models using "Ollama run" in the terminal. Those occupy a significant space in disk and I need to free space to install a different model.
I tried Ollama rm command, but it only deletes the file in the manifests folder which is KBs. I also tried to delete those files manually, but again those are KBs in size not GB as the real models.
I need a solution to delete the big files out o my system.
Any clues?
Any help will be appreciated

Hi,
Already solved.  ;-)

There are several utilities that exist to help manage your cache. It is not generally recommended to manually delete files from your cache, as you could irreversibly corrupt it.
From the command line, you can remove a model one at a time using:
ollama rm <model name>

For example:
> ollama list
NAME                           	ID          	SIZE  	MODIFIED
codegemma:7b-code-fp16<PHONE_NUMBER>85	17 GB 	2 days ago
codegemma:7b-instruct-fp16     	27f776c137a0	17 GB 	2 days ago
codellama:70b-code-q2_K        	a971fcfd33e2	25 GB 	2 days ago
codellama:latest               	8fdf8f752f6e	3.8 GB	10 days ago
command-r:latest               	b8cdfff0263c	20 GB 	4 weeks ago

> ollama rm codellama
deleted 'codellama'
> ollama rm codellama:70b-code-q2_K
deleted 'codellama:70b-code-q2_K'

In Python, if you have installed ollama-python:
import ollama
status = ollama.delete('codellama').       # status is either of: 'success' or 'error'

I've written a utility to help manage the cache, called ollamautil. Feel free to use that to manage your cache, if you find it useful!

@sealad886
Thanks a lot for the feedback. I really had no option but to delete the files from the Cache. The problem as fundamentally, that Ollama rm commend didn't work. I will test your solution. Thanks a lot.

The cache tries to intelligently reduce disk space by storing a single blob file that is then shared among two or more models. If the blob file wasn't deleted with ollama rm <model> then it's probable that it was being used by one or more other models.
The way Ollama has implemented symlinking is actually essentially agnostic to the OS (i.e. I'm assuming their method allows this pseudo-symlink to work on Windows).: each <quant> file in /models/manifests/registry.ollama.ai/library/<model>/<quant> is actually a text file that just has the blob's sha256 hash stored, which is also the name of the blob file itself.

The cache tries to intelligently reduce disk space by storing a single blob file that is then shared among two or more models. If the blob file wasn't deleted with ollama rm <model> then it's probable that it was being used by one or more other models.
The way Ollama has implemented symlinking is actually essentially agnostic to the OS (i.e. I'm assuming their method allows this pseudo-symlink to work on Windows).: each <quant> file in /models/manifests/registry.ollama.ai/library/<model>/<quant> is actually a text file that just has the blob's sha256 hash stored, which is also the name of the blob file itself.

I am not a programmer. So no idea about this. However, in your repo the ollamautil.py in line 578 and 580 is asking for the path to external and internal DIR. How do I know where to find those?

I tried Ollama rm command, but it only deletes the file in the manifests folder which is KBs. I also tried to delete those files manually, but again those are KBs in size not GB as the real models.

Different models can share files. These files are not removed using ollama rm if there are other models that use the same files. For example, if model A uses blob A, B and model B uses blob A, C, removing model A will only remove blob B. This is likely the main source of the behaviour you're seeing.

In my experience, we can just restart ollama after doing "ollama rm model". The related blobs and cache will be deleted and we have the free space again.

I am not a programmer. So no idea about this. However, in your repo the ollamautil.py in line 578 and 580 is asking for the path to external and internal DIR. How do I know where to find those?

@ISK-VAGR  I realize this question was posed a while ago, but for what it's worth, you have to create those directories and delete the real directory.  See: https://github.com/sealad886/ollamautil/blob/master/README.md#before-you-start
Personally, I did not feel comfortable doing that, so I chose not to.  Therefore, I can't use the script.


Spark File Path Issue

I noticed when using the spark view engine it was looking for files in the following locations
/Views_{Module}/{View}.spark
I fixed this to be more like the conventional layout of
/Views/{Module}/{View}.spark

This was only supposed to be the first of those 2 commits. Ooops

Thanks! I just built from your fork and it worked perfectly.

Thank you @garethrhughes for sending this in and to @ryanohs for confirming the fix works. We'll be pulling this straight into 0.11 which we'll start baking sometime around this weekend and should be out within a couple of weeks after that

A small heads up. Please avoid including additional functionality into an open pull request. I just noticed that this now contains additional commits by @TimWilde which were not in the original pull request which widens the scope and renders the description invalid.
The additional commits raises a question for me. Nancy already supports conventions based view locations and Tim's commits raises the question on whether or not these should be made more first-class in the Spark engine or if Spark should layer another set of conventions on top of those.
I take is that the locations that Tim added
Path.Combine("Shared", masterName + ".spark"),
Path.Combine("Views", masterName + ".spark"),
Path.Combine("Views", "Layouts", masterName + ".spark"),
Path.Combine("Views", "Shared", masterName + ".spark")

are the default conventions you would see Spark use if you were using it from, for example, MVC?
Thanks

Sorry about that.
Yes, those are the conventions from spark in mvc.

No worries, just a heads up :)
Going to be a bit lazy because I am at the office atm, but how are these used by Spark and Nancy. Are they appended on existing conventions are relative to the root (using IRootProvider?)

I had another look and I think it's alright. The SparkViewFolder wrapps the ViewLocationResults that we have discovered, at application startup, by using our view location conventions. The spark stuff is then a second level of conventions for trying to locate views among the ViewLocationResults that are for spark files

Hi Andreas.
I didn't add all of those paths that you mention above, just the last two which cause spark to look in the shared and layout folders under the views folder which brings it in line with the conventions when used with MVC.
Sorry for the confusion with the pull request - I didn't realise submitting a pull request to Gareth would mean the changes would be included in his pull request; I thought that would be an atomic entity defined when the pull request is submitted. I'm quite new to Git :)

@TimWilde No worries. The only reason it appeared in the pull request is because he accepted your pull request into his branch :) All good, all pulled in and all tests green!


10 footer

Please always provide the GitHub issue(s) your PR is for, as well as test URLs where your change can be observed (before and after):
Fix #10
Test URLs:

Before: https://main--servicenow--hlxsites.hlx.live/
After: https://10-footer--servicenow--hlxsites.hlx.page/footer

Closing in favour of https://github.com/hlxsites/servicenow/pull/56


14520: new message green line appears

Details

I'm checking if there is any unread action before cleaning the new green marker flag
Fixed Issues
https://github.com/Expensify/App/issues/14520
PROPOSAL: https://github.com/Expensify/App/issues/14520#issuecomment-1425676445
Tests

Android/iOS

[ ] Open a chat with messages and mark a message as unread, a green line should show up
[ ] with the green line, put the app in background, and open the app again, the green line should be there
[ ] with the green line, put the app in background, open the app again, put back the app in BG and open again, the green line should show for a sec (or not show at all) and then hide.

Web

[ ] Open the chat with messages and mark a message as unread, a green line should show up
[ ] with the green line, navigate to another screen and then go back, no line should be there

Offline tests
This feature doesn't change or is impacted by offline mode.
QA Steps
This feature doesn't change or is impacted by offline mode.

[ ] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[ ] I added steps for local testing in the Tests section
[ ] I added steps for the expected offline behavior in the Offline steps section
[ ] I added steps for Staging and/or Production testing in the QA steps section
[ ] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[ ] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[ ] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[ ] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[ ] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[ ] I verified any copy / text that was added to the app is correct English and approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[ ] If any new file was added I verified that:

[ ] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn't already exist
[ ] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[ ] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Android
https://user-images.githubusercontent.com/1676818/218546477-c73d1743-1a5f-434d-b15d-dc0cbe2a0639.mp4

iOS
https://user-images.githubusercontent.com/1676818/218546713-d0702197-cb79-40f0-a28e-9c231aa36ca7.mp4

@monilbhavsar Please copy/paste the Reviewer Checklist from here into a new comment on this PR and complete it. If you have the K2 extension, you can simply click: [this button]

I have read the CLA Document and I hereby sign the CLA

@gedu looks like you have a unit test failing.

@gedu could you also have a look into the mWeb platforms (Chrome and Safari on mobile) thats missing in the PR body.
The tests failing might be actually related to the changes in this PR

The tests failing might be actually related to the changes in this PR

@mountiny  I started a chat about it Slack, is failing even on main branch, without this changes.

@mountiny Videos uploaded

@gedu, there's still videos missing for Desktop and Web
I think we can proceed with the test failing

Btw, is there supposed to be a green checkmark if one of the jobs failed?

Reviewer Checklist

[ ] I have verified the author checklist is complete (all boxes are checked off).
[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[ ] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[ ] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[ ] I verified any copy / text that was added to the app is correct English and approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[ ] If any new file was added I verified that:

[ ] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn't already exist
[ ] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[ ] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

Mobile Web - Chrome

Mobile Web - Safari

Desktop

iOS

Android

@puneetlath missing videos, should be up

🚀 Deployed to staging by https://github.com/puneetlath in version: 1.2.73-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/francoisl in version: 1.2.73-1 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/francoisl in version: 1.2.73-1 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅


Tutorial remains in the game

When you return to this zone, the tutorial remains in the game.
if in the future you need to return to this room, you will see the tutorial again

postscript: the demo is very good, I loved it

I know about that problem, this will be fixed in a final version of level tutorial, thanks for your report!!!!