Rust connection pool implementation

Note: this is not a Python integration -- that part of the code was pulled out of this PR and will happen in https://github.com/edgedb/edgedb/pull/7544
This PR migrates the connection pooling algorithm from Python to Rust. It attempts to maintain as much of the algorithm as possible, though there will be some differences as a result of how the code is structured.
We maintain the same terminology of "blocks" from the existing implementation and the spirit of the overall algorithm is the same as before. There are some unavoidable differences due to code structure and language changes, however the QoS results show the similar behaviour in all cases.
The code tries to maintain separation between the following components to make it easier to reason about:

The low-level blocks/block, connections, and metrics code. This code
creates, destroys and transfers connections without understanding of
policies, quotas or any sort of algorithm. We ensure that the blocks and
metrics are reliable, and use this as a building block for our pool.
The algorithm. This performs planning operations for acquisition, release
and rebalancing of the pool. The algorithm does not perform operations, but
rather informs that caller what it should do.
The pool itself. This drives the blocks and the connector interface, and
polls the algorithm to plan next steps during acquisition, release and
during the timer-based planning callback.
The Python integration code. This is behind an optional feature, and exposes
PyO3-based interface that allows a connection factory to be implemented in
Python.

QoS:
2024-07-15T19:05:01.550982Z  INFO conn_pool::pool::tests: QoS[test_connpool_1] = [94.19]
2024-07-15T19:05:01.550987Z  INFO conn_pool::pool::tests: QoS[test_connpool_10] = [100.00]
2024-07-15T19:05:01.550992Z  INFO conn_pool::pool::tests: QoS[test_connpool_2] = [78.73]
2024-07-15T19:05:01.550996Z  INFO conn_pool::pool::tests: QoS[test_connpool_3] = [99.67]
2024-07-15T19:05:01.551000Z  INFO conn_pool::pool::tests: QoS[test_connpool_4] = [91.84]
2024-07-15T19:05:01.551004Z  INFO conn_pool::pool::tests: QoS[test_connpool_5] = [96.70]
2024-07-15T19:05:01.551009Z  INFO conn_pool::pool::tests: QoS[test_connpool_6] = [100.00]
2024-07-15T19:05:01.551013Z  INFO conn_pool::pool::tests: QoS[test_connpool_7] = [99.53]
2024-07-15T19:05:01.551017Z  INFO conn_pool::pool::tests: QoS[test_connpool_8] = [100.00]
2024-07-15T19:05:01.551022Z  INFO conn_pool::pool::tests: QoS[test_connpool_9] = [84.71]
2024-07-15T19:05:01.551029Z  INFO conn_pool::pool::tests: QoS = [94.54] (rms=91.07)

The following items are left for a future, integration TODO:

[ ] Error states for connections
[ ] Python/Rust integration
[ ] Generate compatible JSON for previous reports
[ ] Hook up old connection tests
[ ] iterate_connections API
[ ] _build_snapshot (metrics)
[ ] Garbage collection may be too aggressive
[ ] Try LRU/MRU for connection acquisition in blocks

Review TODO:

[ ] Confirm that all-time Closed connections are correct
[ ] Make sure that queuing is fair (don't skip the line)

This PR migrates the connection pooling algorithm from Python to Rust. It attempts to maintain as much of the algorithm as possible, though there will be some differences as a result of how the code is structured.
We maintain the same terminology of "blocks" from the existing implementation and the spirit of the overall algorithm is the same as before. There are some unavoidable differences due to code structure and language changes, however the QoS results show the similar behaviour in all cases.


Update outdated hwlib to support newest firmware and central (hwlib 3.10)

Is there a reason why this repository does not contain the newest chhwlib.h (v3.10) that comes with the Cerebus Windows Suite?
These changes seem to be missing, some even signed of by @dashesy and @jsoutherland:
// 3.10 - 23 Oct 2014 hls - Added reboot capability for extension loader
//        26 Jul 2014 js  - Added analog output to extension
//        18 Mar 2014 sa  - Add triggered digital output
//         3 Mar 2013 eaz - Adjust MTU

Or is this not the official development repo of the cbsdk?

@FloFra as soon as I get the latest cbhwlib from Hyrum I can merge and update to the latest, you as a customer can ask Blackrock for the file, or even direct contribution :)

Sorry, I am somewhat confused. So far I have only received the precompiled Windows version of the Cerebus SDK Library and I was rather surprised to also find the source code licensed under GPL on github.
Since it is released under GPL I assumed there must be an official public repository but based on your comment this repo is apparently not it. Sorry for the confusion.
I will talk to Blackrock about this and see what come out of it.

@FloFra This is GPLv2 public release that Blackrock holds the copyright, I do not work for Blackrock. Blackrock sends me the latest protocol header file (cbhwlib) and I merge it, usually that would be the only change needed. I will email Hyrum to get the latest.

With the hwlib of CereLink at v3.9 accessing any of the channel info on a v3.10 instrument will return incorrect results (most often it complains that the channel number is invalid, because chid is zero). This is due to the fact that v3.10 uses larger cbPKT_CHANINFO packages and buffers resulting in data being misinterpreted.
I am in contact with Blackrock to properly resolve this issue and I will sent a pull request as soon as the supply me with the new version.
To avoid any issues in the future, I also submitted #50 (and #49).

Yes, new version requires newer cbhwlib.h and cbhwlib.cpp. I will tag the current version as 6.04 then should merge the new protocol. The protocols ABI is not backward compatible.

If that has always been the case, then I don't understand why nobody ever thought to check the hwlib version in the code on connection.
This SDK has been outdated for almost a year now with no note or warning anywhere.
If you want I can at least post a patch that makes the current code compatible with 3.10 (without adding any of the new functionality but preserving old functionality).

Yes, unless one installs the compatible version with the library, many people do not update that often. The way I have done it from 6.03 to 6.04 was to tag it as v6.03 and then merge 6.04.
If someone wants an older version (but with fixes) there is always an option to backport and call it v6.03.02. Unfortunately there is no easy option. I always hoped Blackrock would step up and at least do some protocol updates more often.

Just in case you have already updated, you can just apply the following simple patch to get the current version of CereLink to work with 3.10 hardware: FloFra@a47610c

This will only change the version, they could have different protocols that may result in unknown or unexpected results

No, this patch indeed contains all the changes that are necessary to make the binary protocol compatible again.
Since I do not have access to the source code of the latest version, but was only able to extract these protocol changes from the latest hwlib headers, It does of course not contain any of the new functionality.

FYI, Nick Halper at Blackrock informed me that he recently shared the source for 3.10 with @dashesy .

The newer version is now merged


fix: Rm old counter shell script that is no longer in use

This was previously added as a way to demonstrate how to build, deploy and solve contracts without using any Rust, but since then the direction of the repo has changed significantly. The test is now outdated now that the essential-rest-server and essential-cli have been removed.
We can re-add this or something similar once we have a clearer story on how to cleanly build, deploy and solve entirely from the CLI. In the meantime, the presence of this script is just confusing curious devs who are exploring the repo.

Should we open an issue to reintroduce a script in the future?


Fix minor typo in DO droplet message

@mrxinu this is the best PR - it was legit bugging me - thank you so much! Are you running kubicorn?!

LGTM

Not yet, but I'm excited to try it out this weekend!!


FIX unsaved script doesn´t actualize #28693

This fix issue #28693
https://github.com/godotengine/godot/issues/28693
Script editor doesn´t send an unsaved script in signal "editor_script_changed" ,it sends the last "disk stored" script unless you close other script. This commit fix that.

For a more detailed explanation:
Unique way to get current TextEdit opened in ScriptEditor from code is compare get_editor_interface().get_script_editor().get_current_script().source_code with all the TextEdit.text. But if you do that, you only find the unmodified scripts unless you close one script (because closing an script do apply_scripts() )
I test the commit and works with no problem.

Well, didn´t work. I tested past version. I will pull other solution.


Errors in tiled example

Description
Logs:
2024-04-12T20:42:11.878635Z  INFO bevy_winit::system: Creating new window "App" (0v1)
2024-04-12T20:42:11.879108Z  INFO log: Guessed window scale factor: 2
2024-04-12T20:42:11.987808Z  INFO bevy_render::renderer: AdapterInfo { name: "AMD Radeon RX 6500 XT (RADV NAVI24)", vendor: 4098, device: 29759, device_type: DiscreteGpu, driver: "radv", driver_info: "Mesa 23.3.6", backend: Vulkan }
2024-04-12T20:42:12.016972Z  WARN bevy_inspector_egui::inspector_options::default_options: Attempting to set default inspector options for bevy_pbr::light::AmbientLight, but it wasn't registered in the type registry.
2024-04-12T20:42:12.016989Z  WARN bevy_inspector_egui::inspector_options::default_options: Attempting to set default inspector options for bevy_pbr::light::PointLight, but it wasn't registered in the type registry.
2024-04-12T20:42:12.016993Z  WARN bevy_inspector_egui::inspector_options::default_options: Attempting to set default inspector options for bevy_pbr::light::DirectionalLight, but it wasn't registered in the type registry.
2024-04-12T20:42:12.016997Z  WARN bevy_inspector_egui::inspector_options::default_options: Attempting to set default inspector options for bevy_pbr::pbr_material::StandardMaterial, but it wasn't registered in the type registry.
2024-04-12T20:42:12.017005Z  WARN bevy_inspector_egui::inspector_options::default_options: Attempting to set default inspector options for bevy_pbr::light::ClusterConfig, but it wasn't registered in the type registry.
2024-04-12T20:42:12.042817Z  INFO system{name="bevy_diagnostic::system_information_diagnostics_plugin::internal::log_system_info"}: bevy_diagnostic::system_information_diagnostics_plugin::internal: SystemInfo { os: "Linux 39 Fedora Linux", kernel: "6.8.4-200.fc39.x86_64", cpu: "Intel(R) Xeon(R) W-2195 CPU @ 2.30GHz", core_count: "18", memory: "251.4 GiB" }
2024-04-12T20:42:15.371670Z  WARN system{name="bevy_entitiles::tiled::load_tiled_xml"}: bevy_entitiles::tiled::resources: Object rotation is not recommended! It will results in wrong collision shapes. But if you just want to use this object as sort of static image it will be ok.
2024-04-12T20:42:15.390546Z ERROR system{name="bevy_render::renderer::render_system"}:run_graph{name="main_graph"}:run_graph{name="Core2d"}:node{name="bevy_core_pipeline::core_2d::main_pass_2d_node::MainPass2dNode"}:main_pass_2d: bevy_entitiles::render::draw: Failed to get material bind group!
2024-04-12T20:42:16.935978Z ERROR system{name="bevy_render::renderer::render_system"}:run_graph{name="main_graph"}:run_graph{name="Core2d"}:node{name="bevy_core_pipeline::core_2d::main_pass_2d_node::MainPass2dNode"}:main_pass_2d: bevy_entitiles::render::draw: Failed to get material bind group!
2024-04-12T20:42:16.955157Z ERROR system{name="bevy_render::renderer::render_system"}:run_graph{name="main_graph"}:run_graph{name="Core2d"}:node{name="bevy_core_pipeline::core_2d::main_pass_2d_node::MainPass2dNode"}:main_pass_2d: bevy_entitiles::render::draw: Failed to get tilemap instance!
2024-04-12T20:42:19.867773Z ERROR system{name="bevy_render::renderer::render_system"}:run_graph{name="main_graph"}:run_graph{name="Core2d"}:node{name="bevy_core_pipeline::core_2d::main_pass_2d_node::MainPass2dNode"}:main_pass_2d: bevy_entitiles::render::draw: Failed to get material bind group!
2024-04-12T20:42:19.888908Z ERROR system{name="bevy_render::renderer::render_system"}:run_graph{name="main_graph"}:run_graph{name="Core2d"}:node{name="bevy_core_pipeline::core_2d::main_pass_2d_node::MainPass2dNode"}:main_pass_2d: bevy_entitiles::render::draw: Failed to get tilemap instance!
2024-04-12T20:42:23.300433Z ERROR system{name="bevy_render::renderer::render_system"}:run_graph{name="main_graph"}:run_graph{name="Core2d"}:node{name="bevy_core_pipeline::core_2d::main_pass_2d_node::MainPass2dNode"}:main_pass_2d: bevy_entitiles::render::draw: Failed to get material bind group!
2024-04-12T20:42:23.317788Z ERROR system{name="bevy_render::renderer::render_system"}:run_graph{name="main_graph"}:run_graph{name="Core2d"}:node{name="bevy_core_pipeline::core_2d::main_pass_2d_node::MainPass2dNode"}:main_pass_2d: bevy_entitiles::render::draw: Failed to get tilemap instance!
2024-04-12T20:42:23.734168Z ERROR system{name="bevy_render::renderer::render_system"}:run_graph{name="main_graph"}:run_graph{name="Core2d"}:node{name="bevy_core_pipeline::core_2d::main_pass_2d_node::MainPass2dNode"}:main_pass_2d: bevy_entitiles::render::draw: Failed to get material bind group!
2024-04-12T20:42:23.751098Z ERROR system{name="bevy_render::renderer::render_system"}:run_graph{name="main_graph"}:run_graph{name="Core2d"}:node{name="bevy_core_pipeline::core_2d::main_pass_2d_node::MainPass2dNode"}:main_pass_2d: bevy_entitiles::render::draw: Failed to get tilemap instance!
2024-04-12T20:42:27.512215Z  INFO system{name="bevy_window::system::exit_on_all_closed"}: bevy_window::system: No windows are open, exiting
2024-04-12T20:42:27.519069Z  INFO system{name="bevy_winit::system::despawn_windows"}: bevy_winit::system: Closing window 0v1

Reproduction
git checkout 0.7.0
cargo run --example tiled --features=tiled,physics

Versions

OS: Linux
Crate and other dependencies: 0.7.0

If the error doesn't continuously pop up, it's expected. Bind groups are prepared asynchronously, so we are not sure if it's ready when the tilemap is been drawn.

I'm sorry, but I'm not sure what you mean here. Are you saying that this is not a problem with this library and won't be fixed?

Yeah, it's expected and it won't affect anything except you will see these ugly error outputs, so please just ignore them.
But that doesn't mean they're totally meaningless. If there's really any problem with bind groups, we can know that instead of seeing an empty scene and have no idea what's wrong.


Move Null to the Bottom - spec

This is the spec issue for #28024

The spec has been updated (rev. 10abaf89fd9378ff). Please take a look and see if anything doesn't make sense.


<EMAIL_ADDRESS>hash check failed

Hash check fails. See below:
Checking hash of LunacySetup_6.0.0.exe ... ERROR Hash check failed!
App:         extras/lunacy
URL:         https://desktop.icons8.com/lunacy/LunacySetup_6.0.0.exe
First bytes: 4D 5A 50 00 02 00 00 00
Expected:    acb938293b27f1f2823ed14fb9249bb0077ff8d047933de9eea0f28ecf43baec
Actual:      090f1ca701dbb6b99a40a5ad0ddc63362f7b3a86e6f5e8415745a2bbf68b4b03
Either something is wrong with the manifest or with the actual installer. Please relay this to the maintainers of the scoop manifest for Lunacy.
Thank you!

BTW, it would be good that scoop does not interrupt upgrading other apps whenever one of them fails. Currently, if there are more than one app that is eligible for upgrade and one of them fails, then the whole process gets interrupted.
Thank you!


Live execution output

It appears that currently the script is executed, and nothing happens until the execution is complete, after which the entire output is displayed at once. Could this be changed, or a feature added, so that it would show output as it happens? If there's a long-running script, I would like to be able to see the output live.

That was the original intention for this addin but I gave up on it after a few unfruitful days 😞 More than happy to accept a PR for this though if your up for the challenge 😉

I believe I have a working solution for this. I will submit a PR for your review.

Closing this now as @yanigisawa solved this with #23 👍


make install fails due to missing files in doc/

I have built libmseed v3.0.2 successfully using make shared.
make install fails with the following message:
cp: cannot stat 'doc/libmseed-UsersGuide': No such file or directory

In fact doc/libmseed-UsersGuide is missing from the source tree and there is no rule in the Makefile to create it.
The install target also contains this command, which will fail because the files doc/ms*.3 do not exist:
@cp -a doc/ms*.3 $(DESTDIR)$(MAN3DIR)/

Thanks @senshu!  Fixed in ec7a205f10ab6a39fb25a2ac4d3c80f9a7b9fb49


FormBuilderChipsInput validator error

Hi Guys,
I just discovered that I cannot use required validator in FormBuilderChipsInput that it starts to give a flutter framework error.

Thanks!

This is the same issue for FilterChip too.

Hi @danielweil / @andrewzakhartchouk
Thanks for bringing the issue to my attention.
For now, consider declaring the generic type on the widget like so: FormBuilderChipsInput<String> or FormBuilderFilterChip<int>.

Hi @awhitford,
This issue seems to originate from this line where, in the effort to have our code well-typed, we are declaring the validator to be of type FormFieldValidator<List<T>>.
The problem with this is that if the generic type T is not explicitly declared when creating the widget, the type T is not known, and for some reason, it's being replaced by Null. To solve this issue I'm considering  having  it  as FormFieldValidator<List> validator,

I do not like the idea of trying to defeat the type system.  I usually see issues where people aren't being consistently clear on type, so Dart needs to fall back to something abstract like dynamic.  We may instead be missing something trivial, like saying <T>[] -- but I didn't think that is necessary when the assignment type is List<T>.  (I fixed something minor in #611 by adding a missing type declaration, not relaxing the declaration.)  Show me an example code snippet that I can analyze.

In the example project, if you add a validator, you'll be able to reproduce.

The VS Code hover message clearly says that Dart is unable to infer the type.  This actually makes sense -- what is going to tell the type when you don't specify a <T>?  A natural thing is initialValue -- but if you exclude that, then what else is used?  Well, maybe the validators, but if you omit the <T> from those, then again Dart has no information.
Consider this code:
                  FormBuilderFilterChip(
                    name: 'filter_chip',
                    enabled: false,
                    decoration: const InputDecoration(
                      labelText: 'Select many options',
                    ),
                    initialValue: ['Test', 'Test 1', 'Test 2'],
                    maxChips: 3,
                    options: [
                      FormBuilderFieldOption(
                          value: 'Test', child: Text('Test')),
                      FormBuilderFieldOption(
                          value: 'Test 1', child: Text('Test 1')),
                      FormBuilderFieldOption(
                          value: 'Test 2', child: Text('Test 2')),
                      FormBuilderFieldOption(
                          value: 'Test 3', child: Text('Test 3')),
                      FormBuilderFieldOption(
                          value: 'Test 4', child: Text('Test 4')),
                    ],
                  ),

The type is not mentioned in the FormBuilderFilterChip declaration. However, Dart is able to infer the type from the initialValue property, which is a List<String>, so T is String.
The type is also not mentioned in the FormBuilderFieldOption declaration.  However, Dart is able to infer the type from the value property, which is a String, so T is String.  (And options is a List<FormBuilderFieldOption<String>> options.)
The easiest thing to do is to specify the type when declaring FormBuilderFilterChip.  Dart doesn't know, but the developer should know.  Help out Dart so that it can generate the best code:  FormBuilderFilterChip<String> or FormBuilderFilterChip<int> -- whatever makes sense.
In general, dynamic is a code smell; usually there is a more appropriate type declaration to specify -- and then the compiler can do much better static code analysis and may identify bugs at compile-time.

For the  FormBuilderChipsInput<Contact> example, I added this:
                    validator: FormBuilderValidators.compose([
                      FormBuilderValidators.required(context),
                    ]),

and it behaved fine because the type is clear.
Similarly, I added the validator to the FormBuilderChoiceChip example:
                  FormBuilderChoiceChip(
                    name: 'choice_chip',
                    decoration: const InputDecoration(
                      labelText: 'Select an option',
                    ),
                    options: [
                      FormBuilderFieldOption(
                          value: 'Test', child: Text('Test')),
                      FormBuilderFieldOption(
                          value: 'Test 1', child: Text('Test 1')),
                      FormBuilderFieldOption(
                          value: 'Test 2', child: Text('Test 2')),
                      FormBuilderFieldOption(
                          value: 'Test 3', child: Text('Test 3')),
                      FormBuilderFieldOption(
                          value: 'Test 4', child: Text('Test 4')),
                    ],
                    validator: FormBuilderValidators.compose([
                      FormBuilderValidators.required(context),
                    ]),
                  ),

and it also behaved correctly.  Note that the FormBuilderChoiceChip example correctly infers T as String because the options are List<FormBuilderFieldOption<String>>.
In summary, I do not advocate for any change.  Instead, the developer needs to ensure that the type is being properly declared or inferred. (This is just how Dart works.)
Projects can proactively identify missing type information by activating analyzer rules -- just like how FFB does this:

https://github.com/danvick/flutter_form_builder/blob/master/analysis_options.yaml

This is the same issue for FilterChip too.

It is pulling from an array mapped to FormBuilderFieldOptions. Using a regular list has the same issue.

@andrewzakhartchouk Based on this message, your validator may be faulty.  What exactly do you have declared?  I'd like to see the whole widget declaration if you can.

@andrewzakhartchouk Based on this message, your validator may be faulty. What exactly do you have declared? I'd like to see the whole widget declaration if you can.

@awhitford Here we are. I'm not in need of a validator for this particular portion, but am I correct in guessing it might have to do with my options (array being mapped and then toList)?

@andrewzakhartchouk Based on this message, your validator may be faulty. What exactly do you have declared? I'd like to see the whole widget declaration if you can.

@awhitford Here we are. I'm not in need of a validator for this particular portion, but am I correct in guessing it might have to do with my options (array being mapped and then toList)?

@andrewzakhartchouk Generating options from _symptoms is not really a problem.  However, with the above code, Dart doesn't have enough information to infer the type, and is falling back to dynamic.  To address this, help Dart out by being explicit.
The easiest thing to do is to specify the type when declaring FormBuilderFilterChip.  For example, if this is for String values, then the declaration should be, FormBuilderFilterChip<String> (line 1).  That is enough to add type information to validator and options.  When specifying a type at the top, that also specifies a type for validator and options.
If you are using VS Code, you can hover over items to see the type information.

@andrewzakhartchouk Generating options from _symptoms is not really a problem.  However, with the above code, Dart doesn't have enough information to infer the type, and is falling back to dynamic.  To address this, help Dart out by being explicit.
The easiest thing to do is to specify the type when declaring FormBuilderFilterChip.  For example, if this is for String values, then the declaration should be, FormBuilderFilterChip<String> (line 1).  That is enough to add type information to validator and options.  When specifying a type at the top, that also specifies a type for validator and options.
If you are using VS Code, you can hover over items to see the type information.

That makes sense. Tested and it works. Thanks for the advice.

That makes sense. Tested and it works. Thanks for the advice.

@andrewzakhartchouk Great to hear!
Closing this issue because it seems invalid.  If people run into this issue, they need to ensure that the type is specified.

@andrewzakhartchouk Great to hear!
Closing this issue because it seems invalid.  If people run into this issue, they need to ensure that the type is specified.


Miss tickers when mass scraping

There are more than 3,000 tickers in file bizType_ind_tickers.csv. However, there are only 600-1000 tickers dowloaded when I mass scrap (it took 5 hours everytime to scrap those tickers with my internet connection). It missed many bluechips, whose information are certainly available on VietStock such as VIC, GAS.
I have tried several times by both WiFi and Lan connection. The number of tickers and number json files vary from one time to another. One time has 657 tickers, another time has 1025 tickers. None of those times are closed to the total 3129 tickers as in bizType_ind_tickers.csv.
The executions were all stopped by itself and stated clearly that it finished (as terminal shown here at the end).
Following are count summary of total tickers and downloaded tickers for one time I scrap mass:

biztype_id
ind_id
ticker
ticker_download

TOTAL

3129
657

0
1
100
136
110

1
1
200
81
30

2
1
300
171
49

3
1
400
598
49

4
1
500
903
50

5
1
600
192
51

6
1
700
67
11

7
1
800
221
50

8
1
900
84
24

9
1
1000
22
0

10
1
1100
3
0

11
1
1200
66
12

12
1
1300
5
0

13
1
1400
66
7

14
1
1500
4
0

15
1
1600
5
0

16
1
1700
7
0

17
1
1800
43
0

18
1
1900
6
0

19
1
2000
4
0

20
2
1000
105
103

21
3
1000
75
60

22
4
900
1
0

23
4
1000
39
24

24
4
1600
1
0

25
5
1000
32
19

26
6
1000
2
1

27
6
2000
2
1

28
7
1000
7
6

29
8
1200
181
0

Following are terminal record:
Scrape Finance Data - version 2

Do you wish to mass scrape? [y/n] y
Do you wish clear ALL scraped files and kill ALL running Celery workers? [y/n] y
Clearing scraped files and all running workers, please wait...

OK
rm: cannot remove './run/celery/*': No such file or directory
removed './run/scrapy/financeInfo.scrapy' removed './logs/corporateAZExpress_log_verbose.log' removed './logs/corporateAZOverview_log_verbose.log' removed './logs/financeInfo_log_verbose.log'
Do you wish to start mass scraping now? Process will automatically exit when finished. [y] y
Creating Celery workers...
Running Celery tasks for mass scrape...
Scrapy is still running…
Scrapy is still running...
Scrapy is still running...
…
Scrapy is still running...
Scrapy has finished
Killing Celery workers, flushing Redis queues, deleting Celery run files...
OK
removed './run/celery/workercorpAZ.pid'
removed './run/celery/workerfinance.pid'
Exiting...

Can you post the summary at the end of the financeInfo log file here. It looks something like this:
2021-08-29 14:28:48 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'bans/status/403': 1,
 'elapsed_time_seconds': 1442.329607,
 'finish_reason': 'CorpAZ is closed; CorpAZ queue is empty; Spider is idling',
 'finish_time': datetime.datetime(2021, 8, 29, 14, 28, 48, 534389),
 'httpcompression/response_bytes': 84604811,
 'httpcompression/response_count': 2330,
 'log_count/INFO': 3533,
 'memusage/max': 64888832,
 'memusage/startup': 61087744,
 'response_received_count': 2331,
 'robotstxt/request_count': 1,
 'robotstxt/response_count': 1,
 'robotstxt/response_status_count/403': 1,
 'scheduler/dequeued/redis': 2330,
 'scheduler/enqueued/redis': 6500,
 'start_time': datetime.datetime(2021, 8, 29, 14, 4, 46, 204782)}
2021-08-29 14:28:48 [scrapy.core.engine] INFO: Spider closed (CorpAZ is closed; CorpAZ queue is empty; Spider is idling)
2021-08-29 14:28:48 [financeInfo] INFO: Deleted status file at run/scrapy/financeInfo.scrapy

Here we go:
2021-08-29 22:18:53 [financeInfo] INFO: === IDLING... ===
2021-08-29 22:18:53 [financeInfo] INFO: corpAZ closed key: 1
2021-08-29 22:18:53 [financeInfo] INFO: corpAZ key financeInfo:corpAZtickers contains: []
2021-08-29 22:18:53 [financeInfo] INFO: set()
2021-08-29 22:18:53 [scrapy.core.engine] INFO: Closing spider (CorpAZ is closed; CorpAZ queue is empty; Spider is idling)
2021-08-29 22:18:53 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'bans/error/twisted.internet.error.TimeoutError': 1,
 'bans/status/403': 1,
 'elapsed_time_seconds': 21089.140718,
 'finish_reason': 'CorpAZ is closed; CorpAZ queue is empty; Spider is idling',
 'finish_time': datetime.datetime(2021, 8, 29, 22, 18, 53, 434348),
 'httpcompression/response_bytes':<PHONE_NUMBER>,
 'httpcompression/response_count': 33926,
 'log_count/INFO': 44694,
 'log_count/WARNING': 1,
 'memusage/max': 71532544,
 'memusage/startup': 71348224,
 'response_received_count': 33927,
 'retry/count': 1,
 'retry/reason_count/twisted.internet.error.TimeoutError': 1,
 'robotstxt/request_count': 1,
 'robotstxt/response_count': 1,
 'robotstxt/response_status_count/403': 1,
 'scheduler/dequeued/redis': 33927,
 'scheduler/enqueued/redis': 33927,
 'start_time': datetime.datetime(2021, 8, 29, 16, 27, 24, 293630)}
2021-08-29 22:18:53 [scrapy.core.engine] INFO: Spider closed (CorpAZ is closed; CorpAZ queue is empty; Spider is idling)
2021-08-29 22:18:53 [financeInfo] INFO: Deleted status file at run/scrapy/financeInfo.scrapy

@vincetran96 although the mass scraping doesn't work perfectly with me as being described, it runs much faster than the business-industry options. It takes about 5 hours for mass options download ~600 tickers before stopping. While it takes... 20 hours to scrap biz-ind option 1;400, in which there are also ~600 tickers.

That's quite a big difference. There could still be some logic error in the implementation of the biz-ind scraping option. Other factors like the quality of connection to Vietstock API at the time of scraping may also be involved. Unfortunately, I currently do not have the time to go through the code.
If you're really interested and have the time, you're more than welcome to inspect the code and propose what needs to be changed to make it better 😄

@truongphanduykhanh are you satisfied with what you got from the program so far? Would you still like me to look into this? I'm suspecting network problems (pertaining to connection quality) that are not caught during the scraping process, as this tends to happen a lot when downloading data for extended periods of time.
If you're OK with what you got, I will close this issue.

@vincetran96 I haven't figured out how to scrape mass option (without interupting). I have tried to scrape mass 5 times and all of them were interrupted after finishing 600-1000 tickers.
If it doesn't take you too much time, can you look into it? Otherwise, the biz-ind option can download all tickers even though it is slow.

Sorry for the delayed comment. I was trying to look into this yesterday. However, it seems like Vietstock has implemented a new security measure to prevent repeated/duplicate request to their financials API endpoints (see issue #8). Now I have to find out a workaround that before addressing this issue.

By default, Scrapy only handles response codes in the 200-300 range (source). While I suspect there were some responses outside that range when you mass-scraped, there doesn't seem to be any in the scrape log summary you posted.
I will have to do some further testing to see how Scrapy behaves with HTTP exceptions. For now, please just scrape whatever tickers are missing. Thanks!

@truongphanduykhanh I'm closing this issue since it's been inactive for more than a week. If there's anything, please comment :)


scroll to index functionality for flutter

Use case
Have a scrollable row with textfields and so to change focus from current to next textfield it should be visible on screen . In order to do so I need to scroll to that widget .

Proposal
Request for official support for scrollable_positioned_lists package

@radha1990kumari
Not sure if I understand your proposal. The package scrollable_positioned_lists seems to be developed by google itself. Does that plugin not work for you ?
Also I see a similar community plugin https://pub.dev/packages/scroll_to_index which seems to achieve what you want already.  Check that and see if it helps.

@darshankawar , the package says "This is not an officially supported Google product" , the package works but it has certain bugs which I believe are not actively worked on.  What I propose is add support for this.

@radha1990kumari
Did you also check the other community plugin scroll_to_index, does that work for you ?
I doubt if the other plugin's functionality will be integrated in the framework itself and may not be feasible to do so.
Your best bet is to nudge the plugin owner and see if they acknowledge and work on those issues.

@darshankawar , I tried scroll_to_index but it doesn't work for me. I will for the time being use the buggy one ,hopefully it will integrated with flutter as I personally believe that it is somewhat an essential feature.


[Bug] Documentation missing links and change text

The following links are missing:

MIT Licence to licence
C# to Wikipedia and "C#" not "c#"
last commit maybe by date format dd.mm.yyyy
Split issues / code changes / feature request
link download correct

Change text:

Buffersize (with release 1.6)

Fixed:

MIT Licence to licence
C# to wikipedia
link download correct
Buffersize
Standard install is AppData
Why clear and textbox can't be used while runnin

Can't fix:

"C#" not "c#"
Split issues / code changes / feature request


false front- map flags get shy

in imperial pale camp

Hmmm... I will take a look but CWO does not do anything to these...

Could be a vanilla thing

Thank you! I saw arthmoor's comment. That should be an easy fix.

Fixed in 1.0.17


fix: indeterminate checkbox in nested lists with selection-disabled [GAUD-5361]

JIRA Ticket
SelectionInfo now carries data on the number of disabledKeys within a list

numDisabledKeys is evaluated during render to decide if checkbox should be disabled as indeterminate

This is very similar to Margaree's original approach back in December DE55986 but I was unable to glean what prevented this fix from being rolled out then. Maybe it just got stale?
Tweaked a couple tests cases to cover this but we could potentially be more explicit and actually verify the state of the input-checkbox, just didn't seem particularly necessary - let me know if desired.

New approach adds allEnabledSelected to getSelectionInfo which is used to track the AND gate of all enabled child items.

This is then used to toggle the state of all children on parent interaction

There is further discussion here that the selection-disabled parent should not be able to enter a selected state even if all its child are selected - as in, the selection-disabled attribute should always win out

I'm not sure how I feel about supporting a scenario where a parent's selection is disabled while its childrens' selection is enabled. Currently, the parent's selection state is purely a function of it's childrens' collective selection state. If we do this, we're starting to move towards breaking that tie since a user could toggle child selection and the parent's selection would not change. In effect, parent selection state would no longer be defined by child selection state, and I think that is a big change, not just in the code, but also in terms of what parent selection means to the consumer and user.
I could be convinced otherwise, but I tend to lean away from solutions where the parent's selection is sort of defined by the children. I feel like it starts to get really messy, and so I favour one way or the other. An operating model where all parent and child selection states are independent would require more discussion imo. 😄

@dbatiste Nothing else in particular - it would definitely break the current list pattern to de-couple parent and child selection state (though I agree we shouldn't really be supporting a scenario where the parent is disabled but children are selectable; that gets quite complicated). Jeff did seem to have scenario(s) that had come up where we'd want parent and child selection to be decoupled, but I don't think there's anything there that would need support at this point and that's something that can be discussed outside of this PR.


Feature/fdk 605 datakatalog side

Coverage remained the same at 64.467% when pulling d95a708728c225a16b52d2b0257dd6c3167713b2 on feature/FDK-605_datakatalog_side into 547a52dceeb4c61f6b1c956ad4db7b0060c04ea3 on develop.


feat(workflow-dev): annotate "keeper" manifests

The helm-keep: "true" annotation will be used by newer versions of Helm Classic to leave the Namespace and deis-router and deis-registry Services intact when installing or uninstalling the Chart.
TODO:

[ ] create a documentation PR describing upgrade with helm-keep
[ ] create a jenkins-jobs (or chart-mate?) PR that ensures cluster teardown doesn't break
[ ] get broader consensus on whether these are the right manifests to helm-keep

Depends on helm/helm-classic#474.
Questions:

Are these the only two manifests that are "keepers"? (Maybe minio?) Also registry-service.
Is it possible to add this also to workflow-beta4? I know that's been cut already, but without the annotations existing, it won't help for those upgrading from workflow-beta4 -> workflow-rc1. 😞  I'd really like to make that happen but don't want to anger the semver gods.
Is this a supportable stop-gap solution for Helm Classic, or an unforgivable hack? Make your voice heard at helm/helm-classic#474.

The helmc install behavior is invisible (I didn't see a good place to modify the output but would love suggestions). But on helmc uninstall here is the user-visible difference with the proposed change to Helm Classic:
$ helmc uninstall -n deis -y workflow-dev
Service/deis-builder
Service/deis-controller
...
ServiceAccount/deis-workflow-manager
Namespace/deis
---> Running `kubectl delete` ...
---> service "deis-builder" deleted
...
---> service "deis-registry" deleted

[WARN] Not uninstalling Service deis-router because of "helm-keep" annotation.
---> Use kubectl to uninstall keeper manifests.

---> service "deis-workflow-manager" deleted

---> replicationcontroller "deis-builder" deleted
...
---> serviceaccount "deis-workflow-manager" deleted

[WARN] Not uninstalling Namespace deis because of "helm-keep" annotation.
---> Use kubectl to uninstall keeper manifests.

---> Done
$ kubectl --namespace=deis get svc
NAME          CLUSTER-IP       EXTERNAL-IP   PORT(S)                            AGE
deis-router   <IP_ADDRESS>                 80/TCP,443/TCP,2222/TCP,9090/TCP   2m

Should we add Registry SVC to this? if you update Workflow then you will be getting a new registry IP at this point, generally not a problem if you can re-deploy your apps or you are re-seeding your k8s cluster with the apps but if your apps are still running and then you go to scale it then the RC / Deployment / etc will be referencing the old IP of the Registry... Or we could use DNS but sky-dns isn't a requirement right now :)

Should we add Registry SVC to this?

Good point. A successful upgrade is basically "apps are still running," and if deis scale breaks, that's not acceptable. I'll add the annotation to deis-registry-service.yaml too.

Minio's service might be a candidate too (if minio is installed, that is).

honestly i wonder if all services should be made keepers. this would keep most env vars good through reinstalls of deis to upgrade components and stuff like that

I had that thought as well @jchauncey. The only thing it leaves me wondering about is what if we want to change any services in some way with a release. I'm not sure what the right thing is to do here.

what if we want to change any services in some way with a release?

That is the essential feature / limitation of the helm-keep mechanism: drastic changes to "keeper" manifests such as removal or filename changes won't be picked up. Effectively you're defining a subset of k8s infrastructure that can't change without requiring the user to kubectl delete them before helmc install.
On helmc install the keeper manifests are updated however with kubectl apply. In the case of deis-workflow, that should mean the namespace and these two service definitions are updated before their dependent resources are installed with kubectl create. So everything should be hunky-dory unless we need to remove or rename one of these items.

i wonder if all services should be made keepers

I would prefer to err on the side of pinning the fewest items possible to allow a helmc uninstall && helmc install upgrade path to work. I think this meets that criterion, but if we can find ways in which recreating other services is disruptive, let's helm-keep them as well.

won't we need to helm-keep the router replication controller as well as the service so they'll continue routing application traffic? :)
I see where helgi is coming from though by keeping the registry alive during an upgrade. From what I understand, there are several components required to boot up an application pod on failure:
buildpack -> S3 + quay (for slugrunner)
dockerfile -> registry + S3
deis pull -> registry + S3
So should a Dockerile or deis pull application go down during an upgrade, it will fail to boot.

btw just to be clear by services i meant the svc definitions. This way we retain all the connection information which could be important if people are sending data from off cluster to something like influx. but maybe its not a big deal.
where it becomes more useful for something like this is if we have an upgrade path for single components.

won't we need to helm-keep the router replication controller as well

@bacongobbler well the core requirement was just not to cause the LoadBalancer to be torn down, requiring DNS to be set up again. So downtime is considered acceptable.
If we did keep the other router pieces it could be a zero-downtime upgrade for apps, but Workflow would lose a lot of flexibility to change router. We could prefer that approach, but that's not how I read the rc1 requirements.

gotcha. Okay then I'm happy with what we got!

I didn't mean to sound dismissive of either idea: keeping all services or keeping the router RC. We should continue to think about this and whether those changes would lead to a better UX.
I'm not going to merge this until it has a sister documentation PR, and a sister CI PR that ensures this won't break the way the Jenkins setup expects to tear down a cluster.

I'm starting a doc PR to accompany this helm-keep change, but I would like to merge this first since RC1 looms and this should be in the workflow-rc1 Chart too.
I'm unclear on how cleanup works currently in CI jobs. Will leaving a few pieces around cause our current setup to fail? And if so, where might I make a change to invoke kubectl for final cleanup? ping @vdice @jchauncey.

I hadn't even thought about that. Is there no way to tell helm to remove these?

Is there no way to tell helm to remove these?

No, basically by design. For keepers, helmc prints out a warning saying "use kubectl to remove this." (The --force flag already has specific meaning that can't be hijacked to say "no, really remove this.")
So I think the current CI setup will be fine with this change (right?), and for the k8s-claimer future a simple kubectl delete namespace deis does it.

@mboersma as of this writing (pre-e2e-runner), there should be no issues as each run gets a fresh k8s cluster.


Improve error message when non-existent user signs in with auto_create_users disabled

https://discourse.firez.one/t/azure-ad-sso-error-signing-in-not-found/352/2

Fixed in #1155


upgrades from catalog ignore upgrade_strategy

Rancher Version: 1.12
Setup Details: (single node rancher vs. HA rancher, internal DB vs. external DB)
single node, external DB
Environment Type: (Cattle/Kubernetes/Swarm/Mesos)
cattle
Steps to Reproduce:
excerpt from rancher-compose.yml of a catalog entry:
haproxy:
scale: 1
upgrade_strategy:
start_first: true
batch_size: 20
Results:
the upgrade ignores any setting in upgarde_strategy
Expected:
should behave like rancher-compose CLI or at least the stack should remember how the last upgrade was done and run the next upgrade the same.

are you using rancher-compose to upgrade the catalog item? if so whats the command youre using?

@pschrammel Thanks for reporting. Could you please try to reproduce and if so, report a new issue with used Rancher and rancher-compose versions?


Update make:publicationTags command to not add empty tags

It's easy enough to accidentally add some extra newlines when running the command. For exmaple consider the following input
foo

bar
baz

This creates four tags, foo, '', bar, and baz.
It should be easy enough to filter out empty entries before proceeding. Because I don't think the following generated Yaml is too helpful:
test:
    - foo
    - ''
    - bar
    - baz

Resolved in https://github.com/hydephp/develop/pull/1367


armeabi-v7a build is missing on f-droid

Hi,
I wanted to do the upgrade to v0.9.4, but I have a phone with the older ARMv7 architecture.
Digging in your build.gradle it looks like you basically support it, but on f-droid there are only three different apks for "arm64-v8a", "x86" and "x86_64" but "armeabi-v7a" is missing.
Is there a reason?
Can I get the apk somewhere else?
Thanks!

There is no reason, we will check with Markus, who handle the f-droid delivery.
You can get the apk here: https://buildkite.com/matrix-dot-org/riot-android, but sadly it's broken now, we are working on it

For everyone who is also confused: If you enable the archives repo in F-Droid then you can get it.

Ok so there is no issue then?
I've check in the script and the APK for armeabi-v7a should be generated like the 3 others.

@aiomaster can you please explain what is the archives repo and why the APKs went there?

I guess only the 3 most recent versions are in the f-droid repo, and armeabi-v7a was built first...

Sounds plausible, but I don't know if it is a bug in F-Droid or if the apks are not tagged correctly.
Last three versions of the apk should just be differentiated by the version number and not by archs.
@DamienCassou The archives are for older apks and can be activated in the package sources menu of the f-droid app.
You find them here: https://f-droid.org/archive

@Bubu any idea?

As far as I can see it's available here: https://f-droid.org/repo/im.vector.alpha_10090400.apk


Replace Native Welch t-test with statsmodels

As discussed in issue #1948, this PR represents the initial step towards differential abundance analysis. This update replaces our previous implementation of the Welch's t-test, which was based on a method derived from this Stack Exchange post, with the functionality provided by the statsmodels package. Primarily, I utilized the CompareMeans class from statsmodels to perform the necessary computations. The results are then stored in the same DataFrame format as our previous implementation, ensuring the output remains consistent.
Please complete the following checklist:

[x] I have read the contribution guidelines.

[x] I have documented all public-facing changes in the changelog.

[ ] This pull request includes code, documentation, or other content derived from external source(s). If this is the case, ensure the external source's license is compatible with scikit-bio's license. Include the license in the licenses directory and add a comment in the code giving proper attribution. Ensure any other requirements set forth by the license and/or author are satisfied.

It is your responsibility to disclose code, documentation, or other content derived from external source(s). If you have questions about whether something can be included in the project or how to give proper attribution, include those questions in your pull request and a reviewer will assist you.

[x] This pull request does not include code, documentation, or other content derived from external source(s).

Note: This document may also be helpful to see some of the things code reviewers will be verifying when reviewing your pull request.

Update: I have temporarily submitted this as a draft PR to assess it's impact during the CI tests. Particularly, I want to ensure I have updated the requirements accurately since this utilizes a new package.

It looks like stats models is not detected in some of these tests. It may require investigating the GitHub workflows, starting here.

Yes, I looked into the failed workflows and identified two errors:

The latest version of statsmodels (0.14.2) isn't supported on some of our older CI workflows so I replaced it with a supported version (0.14.0)
I overlooked updating conda_requirements.txt so I made that change as well

Additionally, should I add the license of statsmodels in the license folder?

Hello @aryankeluskar , if you only use statsmodels as a dependency of scikit-bio, but don't directly copy code from statsmodels into scikit-bio, you don't need to include statsmodels' license in the scikit-bio repository.

Thanks @aryankeluskar for your contribution. I think this is good to merge. @qiyunzhu ?

@aryankeluskar @mortonjt I think this is good. Let's merge after 0.6.1's release. It will last some 24-48 hr. We also need to think what other homebrew code in scikit-bio can be replaced by statsmodels (e.g., FDR correction).

Hello @aryankeluskar Let's merge this PR. The only question I have is the statsmodel version 0.14.0. It probably doesn't have to be this new. I found that as early as verison 0.10.2, that CompareMeans class was available and provides the same interface. But we probably don't need to be picky for now. scikit-bio's dependency list has lots of old and new packages but the CI workflow does not explicitly test the oldest possible versions of the dependencies. This can be fixed in a new PR.


Ama website scans is down displaying error net::ERR_CONNECTION_TIMED_OUT

Source name
Ama scan
Source language
Portuguese Brazil
Source link
https://amascan.com
Other details

Acknowledgements

[X] I have updated all installed extensions.
[X] I have opened WebView and checked that the source website is down.
[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a meaningful title with the source name.
[X] I will fill out all of the requested information in this form.

Website down


[idea] Support parent deployments

Implementation:

generate deployment manifest for parent deployments
spiff merge [current deployment stub] [parent deployment manifest]
normal spiff merge with result from above

Would be solved by deployment links


Builder/Space Chamber Issues

I was doing some testing on a server with a friend of mine. We found that:

You can move bedrock with the builder. I don't know if this was intended.
You can move the End Crystals that heal the Ender Dragon. Not really an issue I suppose but I don't know if it's intended.
You CANNOT move the End Portal inside of The End. If you try, it will crash the server. It moves the bedrock blocks but I believe in it's attempt to move the portal blocks it causes the crash.

I need to protect this. Thanks for reminding me :-)

Fixed in next release


[Features request]  use bits

Hi
Great work on the project I was wonder why use web client request instead of BITS which is faster and you can queue both downloads and resume them if the connection gets dropped.
https://docs.microsoft.com/en-us/windows/win32/bits/about-bits?redirectedfrom=MSDN

Hi @Darkcast - unstable connections weren't really considered when designing DetectionLab, but using BITS inside of scripts is a pretty interesting idea and does seem to have some advantages over IWR. This isn't quite a high-pri fix for me at the moment, but if you wanted to a submit a PR that modified all the web request calls to BITS calls, I'd be happy to review it!

Thanks for your msg ill see if I have the time.

Closing as this is not a priority at the moment.


[AA-1206] Apply the ReSharper "Remove Unused Field" refactoring to address compiler warnings

A few fields became no longer used in recent updates to the BulkRegisterOdsInstancesModelValidator. This simply resolves the warnings by applying the ReSharper "Remove Unused Field" refactoring for each offender.
AA-1206 is focused on these simple warnings. The last remaining warning as of this commit will be covered by a separate ticket, AA-1215.

Developer Testing

Admin App builds
All tests pass
The concerned warnings no longer appear while building the app
Smoke test of the admin app shows the related feature is working correctly

Developer Testing

Admin App builds
All tests pass
The concerned warnings no longer appear while building the app
Smoke test of the admin app shows the related feature is working correctly


Aligned Footer and changed QUICK LINKS size

📝 Description
I aligned Heel2Toe to Quick Links in order to increase aesthetics. I also changed the Quick Links font size in order to increase readability and understanding.
🔧 Fixes #(issue)
Unaligned elements in footer and "quick links" looks like a link #80

📷 Provide related Screenshot/Gifs if any.

[x] I have performed a self-review of my own code.
[x] I'm a contributor in Hacktoberfest'21.

Thank you @InternetRamen


Auto-update for packages related to 'healthcare-components-1.0.0-master-20200922-2'

Updates package 'Microsoft.Health.Abstractions' to version '1.0.0-master-20200922-2'
Updates package 'Microsoft.Health.Api' to version '1.0.0-master-20200922-2'
Updates package 'Microsoft.Health.Extensions.DependencyInjection' to version '1.0.0-master-20200922-2'
Updates package 'Microsoft.Health.Core' to version '1.0.0-master-20200922-2'
Updates package 'Microsoft.Health.Test.Utilities' to version '1.0.0-master-20200922-2'
Updates package 'Microsoft.Health.Client' to version '1.0.0-master-20200922-2'
Updates package 'Microsoft.Health.SqlServer' to version '1.0.0-master-20200922-2'
Updates package 'Microsoft.Health.SqlServer.Api' to version '1.0.0-master-20200922-2'
Updates package 'Microsoft.Health.Extensions.BuildTimeCodeGenerator' to version '1.0.0-master-20200922-2'

This upgrade fixes the schema test and will be merged in post the publish notification changes are merged in.


Add social event

That's lovely! Thank you @hypha for putting it together.
We also have a picture of the band if we need one, we can add it to the page

Oh a picture might be nice, @Kislovskiy! Send them to me in any platform of your choice and I'll see if I can make it nice. Also, I was hoping to link them but couldn't find a website apart from the Facebook one.  Should I link that or is there something better?

That's lovely! Thank you @hypha for putting it together.
We also have a picture of the band if we need one, we can add it to the page

Oh a picture might be nice, @Kislovskiy! Send them to me in any platform of your choice and I'll see if I can make it nice. Also, I was hoping to link them but couldn't find a website apart from the Facebook one. Should I link that or is there something better?

Yeah, I think Facebook group is fine. I'll add a commit with the image to img directory

@hypha I put the image here: https://github.com/EuroPython/website/pull/350/commits/dd58ea2d36f9421170c1e9bf0d30d9db0eaa5a8c


modisco meme command does not return trimmed motifs and no negative patterns but only positive ones

This worked for me to get all motifs out:
import click
import h5py
import numpy as np

@click.command()
@click.option(
    "--reportFile",
    "report_file",
    required=True,
    multiple=False,
    type=str,
    default="modisco_resultshypothetical_contribution_scores_mean_diffTeloHEAC_CTRL_vs_6h.npz.h5",
    help="e.g. modico output",
)
@click.option(
    "--outputPWM",
    "out_pwm",
    required=True,
    multiple=False,
    type=str,
    help="output PWM file",
)
@click.option(
    "--outputHCWM",
    "out_hcwm",
    required=True,
    multiple=False,
    type=str,
    help="output PWM file",
)
def cli(report_file, out_pwm, out_hcwm):

    # read data base entries
    report=h5py.File(report_file , 'r')

    # trim pos patterns
    trimmed_ppms=[]
    trimmed_hcwms=[]
    for key1 in report.keys(): #pos patterns and neg patterns
        for key2 in report[key1].keys():
            pwm=report[key1][key2]["sequence"][:]
            cwm=(report[key1][key2]["hypothetical_contribs"][:])

            score = np.sum(np.abs(cwm), axis=1)
            trim_thresh = np.max(score) * 0.2  # Cut off anything less than 30% of max score
            pass_inds = np.where(score >= trim_thresh)[0]
            trimmed = pwm[np.min(pass_inds): np.max(pass_inds) + 1]

            trimmed_ppms.append(trimmed)
    report.close()

    #write MEME file with trimmed PWM motifs
    background=[0.25, 0.25, 0.25, 0.25]
    f = open(out_pwm, 'w')
    f.write('MEME version 4\n\n')
    f.write('ALPHABET= ACGT\n\n')
    f.write('strands: + -\n\n')
    f.write('Background letter frequencies (from unknown source):\n')
    f.write('A %.3f C %.3f G %.3f T %.3f\n\n' % tuple(list(background)))

    for i in range (0, len(trimmed_ppms), 1):
        f.write('\nMOTIF '+ str(i) +'\n')
        f.write('letter-probability matrix: alength= 4 w= %d nsites= 1 E= 0e+0\n' % trimmed_ppms[i].shape[0])
        for s in trimmed_ppms[i]:
            f.write('%.5f %.5f %.5f %.5f\n' % tuple(s))
    f.close()

if __name__ == "__main__":
    cli()

@ivyraine would you mind taking a look when you have a chance?

https://github.com/jmschrei/tfmodisco-lite/blob/d98aeb17a9c79398ded3e21d0a30d004f39fdcd8/modiscolite/io.py#L243C2-L243C2
I think here the for loop would need to be changed to also include negative patterns.
And within this for loop one could consider adding trimming of the motif :-). Because if you want to run fimo or tomtom with the untrimmed PWMs the results are very different (and starnge) compared to the trimmed ones. And if I understand modisco correctly, when tomtom is used internally, modisco uses the trimmed motifs (probably for that reason).

I'll take a look soon, thanks for waiting


Add hardware frame context for hardware video encoding

With the addition of hardware frame context, hardware video encoding can be achieved. The official vaapi_encode.c example can now be implemented in Go. I personally tried with CUDA (h264_nvenc to be exact) and it works similar to h264_vaapi, with a very slight field change.
This PR does not have UTs yet. Please help review the changes before I can proceed to write some UTs.

Thanks for such a detailed review! I will make the relevant changes once I get back.

Thanks for such a detailed review! I will make the relevant changes once I get back.

No problem. Don't forget to add a comment here once you're done as I won't get notified by Github otherwise 👍

@asticode All above comments were fixed. Please review when you have time.

Is the vaapi_encode.c Go example required for this pr to get merged? Or should we do it in a separate one?

Did you have the chance to finalize this PR by any chance?

Did you have the chance to finalize this PR by any chance?

I will have to find some time next week for this.

The hardware video encoding example is now added.
The vaapi_encode.c example in the ffmpeg repo has only hardcoded support for vaapi. I do not have a device that supports vaapi. Or maybe I do, but I'm not familiar with the ecosystem. Therefore, I took the liberty to remove the hardcoded parts to make the example support other hardware devices like cuda. I think this example will be more helpful for most people.

@l0rem1psum we can also remove the example from the PR and merge it like this, and I can add the example later 👍

FYI I've picked up the PR and merged it 👍


Update Byobu URL.

Checklist for submitting a pull request to Terminals Are Sexy:

[x] I have carefully read and comply with the Contributing Guidelines of this repo.
[x] I have searched the PRs of this repo and believe that this is not a duplicate.

Title: Byobu
Link: https://byobu.org
Category: Tools and Plugins
Description: Byobu is an open source text-based window manager and terminal multiplexer.

I'm just updating the Byobu URL since I noticed they switched to another one.

Hi there @Timecrash. Please alter your pull request so that there is only one line change. Thank you in advance.


Reactnative/release 2.2.3

The PR has below changes
✅   Accessibility for adaptive card elements
✅   Fix for input box max length - #5122
✅   Added action title to onExecuteAction callback for the element Action.Submit.
Microsoft Reviewers: Open in CodeFlow

LGTM

LGTM


Does "rethrow" really need in in_memory_io.dart?

Describe the bug
Unhandled Exception: FileSystemException: Cannot open file, path = 'cache.txt' (OS Error: Read-only file system, errno = 30)

To Reproduce

launch iOS simulator
flutter run or VSCode debug start
when app is running, click home button for app being background.
re-open app
app is freezing because vscode debug catched exception.

error log in console.
[VERBOSE-2:ui_dart_state.cc(144)] Unhandled Exception: FileSystemException: Cannot open file, path = 'cache.txt' (OS Error: Read-only file system, errno = 30)
#0      _File.open.<anonymous closure> (dart:io/file_impl.dart:364:9)
#1      _rootRunUnary (dart:async/zone.dart:1132:38)
#2      _CustomZone.runUnary (dart:async/zone.dart:1029:19)
#3      _FutureListener.handleValue (dart:async/future_impl.dart:137:18)
#4      Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:678:45)
#5      Future._propagateToListeners (dart:async/future_impl.dart:707:32)
#6      Future._completeWithValue (dart:async/future_impl.dart:522:5)
#7      Future._asyncComplete.<anonymous closure> (dart:async/future_impl.dart:552:7)
#8      _rootRun (dart:async/zone.dart:1124:13)
#9      _CustomZone.run (dart:async/zone.dart:1021:19)
#10     _CustomZone.runGuarded (dart:async/zone.dart:923:7)
#11     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:963:23)
#12     _microtaskLoop <…>

https://github.com/zino-app/graphql-flutter/blob/519381cd1c57515fd26cedb33c68a68d563a6eac/packages/graphql/lib/src/cache/in_memory_io.dart#L88-L105
Expected behavior
when I run app through flutter run, not vscode debug, and when app is in background,
it just shows verbose exception error. error doesn't affect app's running.
but, I wonder it is okay to ignore.
Desktop (please complete the following information):

OS: macOS Catalina
Version [e.g. 22]

Smartphone (please complete the following information):

Device: iOS Simulator - iphone 8
OS: iOS
Version: 8

outdated / solved implicitly by 4.0.0-beta.1 as we've decoupled persistence from the cache and HiveStore does not have this problem


Update "toggle ndview" text

References and relevant issues
Fixes https://github.com/napari/napari/issues/5390
Description
Makes the tooltip shown for the switching between 2D/3D views clearer.

why it is draft?

Because I like to keep my PRs in draft until I see that the CI has passed.

Could you also update the docs: https://napari.org/stable/tutorials/fundamentals/viewer.html?highlight=toggle ndisplay#toggle-ndisplay-2d-3d-button Or if you prefer make an issue over at the docs repo and tag me in it with a link to this PR to take care of. Thanks!

Which bit of those docs needs updating? I'm not seeing anything obvious, but maybe I missed something 😄

For example this section title https://napari.org/stable/tutorials/fundamentals/viewer.html?highlight=ndisplay


avro_schema_enum_number_of_symbols

When trying to import pyavroc I keep getting
.._pyavroc.so: undefined symbol: avro_schema_enum_number_of_symols
if I look at the .so's functions using nm, I see the enumber_number_of symbols there.
Anything else I should be doing?

How exactly are you building the code? Are you using clone_avro_and_build.sh ? Please could you try:
rm -r local_avro
./clone_avro_and_build.sh --static


Stop-n-go hack

Stop-n-go hack

Stop-n-go hack

Nice try!

Which car? You should add it to the list of cars with stop and go capability. See values.py

The problem with disabling this is if you are on a hill, then the car will ignore acceleration commands. Better to be reliability annoying than unreliably good.

I like it to be PREDICTABLY annoying occasionally (uphill) than CONSISTENTLY annoying.
I prefer to OCCASIONALLY push the gas on uphill when the car does not move (which is WAY less than always) than ALWAYS keep hitting the cruise stick in the rush hour.

Nice try!

Thanks :-D - would you unblock me now ;-)

Just make your own fork, or edit the file directly on your EON.

Just make your own fork, or edit the file directly on your EON.

exactly what i did: PATCH-3 on my fork and i push gas when uphill. I know you like this hack!

That's not really how to do it.

That's not really how to do it.

suggest a better way please! thanks in advance.

The main problem is you're making PRs directly against Comma's code... which wastes a lot of our time.  Make changes to your own fork, not the origin. Plenty of videos on YouTube for GitHub 101.

Apologies :-) and thanks for helping me help you save time! A rationalizing comment next to SnG in the code similar to George's response would've prevented it. Sorry for being redundant here and please let me know what i missed. Peace!


Ensure all embedder threads initialize their Thread before using VM

Embedder threads that call VM functionality without (or before) entering any isolate will currently not have their Thread structure initialized, thus Thread::Current() is NULL. For now, this is manageable, but it leads to some awkward special casing in our code, and potentially more serious design problems in the future.
We should either:
(a) add an initilization API call that each embedder thread must call before any other VM functionality (possibly excepting the main one that initializes the entire Dart VM), or
(b) always do check+lazy init in Thread::Current
If we opt for (a), we should do it sooner rather than later (since the number of embedders will hopefully only increase). For (b), we need to evaluate the performance impact.
@a-siva

I believe this was addressed by 9e19d236ca83136d49cf54f38a8b551927be70bf


ARROW-16823: [C++] Arrow Substrait enhancements for UDF

See https://issues.apache.org/jira/browse/ARROW-16823

@rtpsw I did skim through the PR, interesting!.
What is required to get a functional test case to evaluate registering a UDF? I went through the JIRA, but it is not clear what is meant by

registering a function (with an Id) external to the plan

@vibhatha, I think this PR is ready for review. Are you the one to review it?

@vibhatha, I think this PR is ready for review. Are you the one to review it?

@rtpsw I was reading it now. But I won't be a major reviewer. I will be closely reading and co-review certain parts.
cc @westonpace @lidavidm could you please take a look.

Right now I see only a minor change waiting for me to make. Let me know if you're still reviewing and I'll hold for your notification.

@vibhatha, I'm not up to date on Acero/Substrait progress anymore. Are the changes here reasonable?

@vibhatha, I'm not up to date on Acero/Substrait progress anymore. Are the changes here reasonable?

I have some explanation here, in case it helps. The TBD parts are expected in an upcoming PR (or two) I'll prepare.

For background on nested registries, see:

a completed PR on nested extension-id-registries
a pending PR on nested function-registries, which I ended up merging here.

@vibhatha, I'm not up to date on Acero/Substrait progress anymore. Are the changes here reasonable?

@lidavidm I am going to go through again. I will check with my knowlege on ACERO/substrait. But it would be better to have another review from @westonpace on this.

cc @westonpace could please take a look.

@vibhatha, I'm not up to date on Acero/Substrait progress anymore. Are the changes here reasonable?

I have some explanation here, in case it helps. The TBD parts are expected in an upcoming PR (or two) I'll prepare.

@rtpsw I added a comment to the JIRA. Appreciate your feedback to clarify the design and usage.

I think we have two important pieces discussed here. One is how Substrait-UDF usage is benefitted and the second is how the function registry usage must be modified. Since the function registry usage is an important piece for the first task, should we address it first and move for the second. Just a thought. We could test the usage of the temporary FR further.

Sorry, I'm still catching up from being out earlier this week.  I'll take a look at this tomorrow.

I skipped over the changes to nested function registries since I already reviewed those (I think) in #13252 .

I think there are a few additions here, so I'll try to rebase to make the diff clear.

This will enable custom non-embedded functions to be used in Substrait plans although I'd prefer it to be a bit more automatic (e.g. not requiring a second register call).

I'll look into this.

Rebase done and pushed using git push --force-with-lease origin HEAD.

@westonpace, is this good to go?

This is good to go from my point of view. I have an upcoming PyArrow UDF PR that will use the changes in this one.


Error when running the example in README

Hi! I run the sample in README using Option2 (the local context), with Transformers 4.16.2.
I encountered an Error:  TypeError: TextInputSequence must be str
It happened in line candidates = bert_reader.predict(question, contexts). Could someone help with it? Thank you a lot in advance.

Just updated the development branch.
Please roll back to transformers==3.4.0 (for my case it works but haven't verified for other versions).


Allowing specifying HTTP client instance used

I'm currently trying to implement some tests in my code base which uses the contentful-go client but due to not being able to specify the HTTP client it makes testing very difficult. I'm opening a PR that exposes the HTTP client so consumers can set it.

@giulianob should this be closed?


Validating webhook for policy.openservicemesh.io resources

Please describe the Improvement and/or Feature Request
Add a validating webhook to validate resources (currently Egress) in the policy.openservicemesh.io API group.
The webhook will be responsible for validating the resource when it is created/updated.
Scope (please mark with X where applicable)

New Functionality          [X]
Install                    [ ]
SMI Traffic Access Policy  [ ]
SMI Traffic Specs Policy   [ ]
SMI Traffic Split Policy   [ ]
Permissive Traffic Policy  [ ]
Ingress                    [ ]
Egress                     [ ]
Envoy Control Plane        [ ]
CLI Tool                   [ ]
Metrics                    [ ]
Certificate Management     [ ]
Sidecar Injection          [ ]
Logging                    [ ]
Debugging                  [ ]
Tests                      [ ]
CI System                  [ ]
Demo                       [ ]
Project Release            [ ]

Possible use cases
To prevent user misconfigurations when using the custom resource.

Shashank, can you comment on some of the validations we will be adding?


Delete visualisation feature

Add delete visualisation feature

+1 :)

Hi, please tell me why you need this feature?

I would like the user to choose which secondary visualizations (if any) to display next to the main one, for instance with a multi-selection dropdown in the header of my page.
If he first selects 3 visualizations and then removes one, I need a way to make it disapear.
Does it make sense?

Delete visualisation meaning delete visualisation from .kibana index, I think you need to disappear the visualisation from the dashboard, look at https://github.com/Webiks/kibana-API/issues/20
I think it's will help you, if not let me know.

👍yes this is exactly what I want to acheive
it doesn't seem to be working though.
I replied on the other thread


Can’t log in to perform tests

Environment

Ruby 2.4.1
Rails 5.1.2
Devise 4.4.3

Current behavior
I would like to test features inside the app with MiniTest.
Do to so, I need to be able to log in the app.
After creating and confirming a user account, I tried test helpers, How To: Testcontrollers, and Warden helpers, and my own solution:
def login(email = "em@il", password = "password")
    post "/users/sign_in", :params => { :user => { :email => email, :password => password } }
end

but, none of them works.
If I try to get pages from within the app, I always get redirected to /users/sign_in.
Expected behavior

Be able to log in the app from tests.

What is the superclass of your controller?

This is  ActionDispatch::IntegrationTest.

Have you tried https://github.com/plataformatec/devise#integration-tests?

Yes, but I have an NoMethodError: undefined method 'users'.

Ok, I have been able to log in with my solution.
I had an issue with something else. I don't know what it is.
Thanks anyway.


Downloads not working in Windows

Something in e726ac65a765ecf846e9082f99891663d8d4759d or 1a7712c12fd0743a8bc1317b98aae6b47c31ee45 broke downloads in Windows. Linux (Ubuntu) still works without issue.
aria is downloading the chunks successfully and writing their gids to the gids file, but twitch_vod_fetch thinks they failed and keeps trying to download them until it reaches the maximum number of retries.

Hey,
As you seem to be maintaining script version here, wanted to mention that I completely rewritten it in mk-fg/fgtk repo... which probably sounds bad, I know.
But it's much faster and simplier now (even though longer because of boilerplate like AsyncExitStack implementation), especially wrt how it works.
Instead of crazy nested polling loops, million of clutter chunk-files and .sh/.bat hook-scripts, listens for onDownloadComplete (and -Error) events on websocket and concatenates downloaded chuks to dst file in sequence, removing them right after that and only tracking last concatenated gid (instead of all downloaded gids along with the files).
I.e. whole "main loop" is in these ~50 lines: https://github.com/mk-fg/fgtk/blob/master/desktop/media/twitch_vod_fetch#L684-L743
It's also python-3.6+, using aiohttp module for http instead of requests.
As a bonus, pretty sure it might "just work" on windows, as there's no .sh hooks anymore or any subprocesses except aria2c, and that's started without any special params (close_fds is handled implicitly by python-3.x), so maybe don't need any extra patches now.
No idea if it's relevant to this fork though, as older script probably works just as well, and only big feature here is that there's less tempfiles around (and few that are left are kept in directory by default), plus maybe py3 (if keeping py2 around is a burden by now).
Apologies for hijacking the bug, but it might actually address it as well, if it's still there.

Thanks for letting me know. I'll take a look at the new version later.
If it "just works" on Windows, that would be a lot easier for me.

@mk-fg I've made a few small changes to the new version to allow it to run on Windows, but I'm running into an issue where aria2c is being stopped after 3 seconds and deleting the chunk files fails because they're in use by another process.
Do you know why this would happen or how to fix it?
You can see the code I'm running here and the output here.
I'm running the script with this command:
py C:\Users\USER\Documents\GitHub\fgtk\desktop\media\twitch_vod_fetch --debug https://www.twitch.tv/videos/148136907 "../download_temp/Dave" > log.txt 2>&1

Let me know if you want me to open a proper issue for this.

I'm running into an issue where aria2c is being stopped after 3 seconds and deleting the chunk files fails because they're in use by another process.

This is one issue, as that's how exit from aria2c works for some reason, i.e. this is what I think happens there:

After chunk<PHONE_NUMBER>000000 was appended to dst file (along with bunch of others) and new "file position + latest chunk N" mark was stored to file_dst_mark, it gets os.unlink().
os.unlink() fails presumably because aria2c didn't close the file yet, raising OSError.
OSError unwinds all the contexts, uncluding TVFAria2Proc, which sends "aria2.shutdown" command to aria2c.
aria2c waits for 3 seconds, then exists, and that exception gets printed.

Not sure why exactly aria2c needs to wait these 3 seconds or how to disable it, just how it works, and I've tried "aria2.forceShutdown" to same effect ¯\(ツ)/¯
It doesn't wait when receiving SIGTERM, but then it also warns that it's an emergency exit, so looks like a bad alternative to that relatively harmless delay.
Note on fsync/fdatasync - not actually sure why I've even put it there, shouldn't be needed, will remove.
(using any previous "pos + seq" mark is fine if latest one will get lost - thing will just re-download few more chunks)
And the actual issue - os.unlink() failing on windows for opened files, I think is totally windows-specific, as you iirc can't remove opened files there, while on linux it's totally fine and is used as a "good practice" for tmp files and such (file is always accessible through fd, name on VFS is just a "link" to it).
Will need to google wrt how to fix it - iirc there are tools on windows like "unlocker", maybe it's just some trivial os.release_this_file() syscall wrapper...
Also, EX_OK is always 0 so no reason to bother with constant (kinda weird that py doesn't provide fallback value on win32 though) and easy to put an if around signal handlers indeed.
Didn't know about ProactorEventLoop either, wonder why it's not picked on windows by default, though guess there's gotta be some good reason, will copy a check for that as well.

Thanks for testing btw!
Surprised by some of unixy quirks in asyncio, but seem to be quite few of them, all things considered.

Hm, nope, looks like it's more complicated on windows - opened file can't be removed, and these "unlocker" tools apparently look for procs that have the handle(s) and close it there via something like ptrace()... oh well, guess I'll add some asyncio coroutine that'll try removing these later, up to and including when aria2 itself exits- will be removable for sure then.

Thanks for the response. I'll be glad to get this working on Windows properly.

I think c28dccd should fix it (and other win-compat things are in 6547827 before that).
Now there's coro that basically retries unlinking any chunks it failed to remove instantly every 10s (conf.file_unlink_retry_delay) and when cancelled (i.e. on exit).
Should work if aria2 eventually releases these things, as it should, and does with other chunks, I think - just a race between dl-complete event and close() on the file.
Maybe try and see how many chunks you have around while download is in progress?
Should ideally be around 20 (conf.file_append_batch + any non-sequential ones), and definitely shouldn't swell into hundreds, as that'd indicate aria2 releasing files very poorly, and maybe need for some other fix.

Oh, right, gh doesn't linkify commits in other repos:

c28dccd: https://github.com/mk-fg/fgtk/commit/c28dccd
6547827 : https://github.com/mk-fg/fgtk/commit/6547827

The video now downloads, there are around 10-55 chunk files while the download is in progress.
When the download completes, there's one chunk file left that couldn't be deleted. New output is here.

Oh shoot, indeed that final rmtree runs before aria2c exits, easy to fix...

Should be fixed in https://github.com/mk-fg/fgtk/commit/277c0ed

there are around 10-55 chunk files while the download is in progress

Doesn't sound too bad, and can probably be reduced by lowering those batch/retry_delay opts above, if necessary.

Ugh, randomly spotted that script wasn't closing chunk files it opened itself!
And that probably explains all the issues here - it was python GC runs that closed them at random times, and not aria2c's fault in any way whatsoever.
Can you try this version of the script:
https://gist.github.com/mk-fg/396745b36f77a9fe84c0b27d92349fe7
I've reverted removal back to simple os.unlink there, but with proper closing of chunks, I strongly suspect it should work just fine, no need for that extra cleanup thing at all.

Also pushed that version to github now, as it really shouldn't be aria2's fault, just me not checking other possible (and more obvious) causes for that issue from the start...

It looks like it's working now, the video is downloaded and the temporary directory is completely removed. Thanks for fixing it.

@mk-fg If I created a PR to add the --output-format argument from the previous Windows version of the script to the new version, would that be welcome?
The MP4 videos produced by twitch_vod_fetch have a malformed AAC stream, which prevents Windows from showing their length and thumbnail (even though they can still be played in an MP4 player). I run a Lua script to detect and fix this after downloading.
The Lua script runs this ffmpeg command and scans the output for this line and the line after it. If it finds this message, it runs this ffmpeg command to fix the malformed stream and then overwrites the original file with the fixed one.
Would you consider adding a feature like this to twitch_vod_fetch, or should I just keep it in the Lua script?

If I created a PR to add the --output-format argument from the previous Windows version of the script to the new version, would that be welcome?

I think I've dropped it because it's redundant with --ytdl-opts (which apparently I forgot to use too), as it should be simple enough to specify -y ' -f 480p' or something, but if not, should definitely not be a problem to have that option back.
Will copy it from there in a moment.

MP4 videos produced by twitch_vod_fetch have a malformed AAC stream, which prevents Windows from showing their length and thumbnail (even though they can still be played in an MP4 player)

I've tried concatenating stuff with ffmpeg like youtube-dl does for twitch in that script initially, but found that either didn't work (produced broken files with "-f concat") or was terribly slow, which is why ended up using simple concatenation instead (fast and works).

Would you consider adding a feature like this to twitch_vod_fetch, or should I just keep it in the Lua script?

Should be easy to add ffmpeg to file_dst running alongside aria2 and just feed it instead of appending chunks to file, but I don't know how to handle interrupt/resume thing with it - can ffmpeg pick up half-finished file and start appending more stuff to it?
It probably can, just need to find options-magic for it.
That might still have all kinds of issues with sudden interrupts though, and probably add a fragile code-path which I'll never use (and hence might break randomly and never notice, similar to windows things), so unless there's need to speed it up like that, you'd probably be much better off using custom post-processing script that's fully under your control and much simplier than something coded into that downloader thing.
Would it be useful to have ffmpeg assembling video on-the-fly like that?

I think I've dropped it because it's redundant with --ytdl-opts (which apparently I forgot to use too), as it should be simple enough to specify -y ' -f 480p' or something

Oh, no, apparently I was thinking about some other (maybe non-existent) option here entirely, and you mean very different thing wrt output filename templating.
Will add the latter, shouldn't be difficult as such rename-after-download thing is already done for .part.mp4 file with --use-part-suffix.

Would it be useful to have ffmpeg assembling video on-the-fly like that?

I only care about the final video being fixed, whether that's done on-the-fly or after the download has completed doesn't matter to me.
I'll probably just keep the feature in my post-processing script.

Will add the latter, shouldn't be difficult as such rename-after-download thing is already done for .part.mp4 file with --use-part-suffix.

Thanks, that would be useful to have.

Added in https://github.com/mk-fg/fgtk/commit/49c70b0
It's -o/--ytdl-output-format option, and it can have optional argument (format), but still need to be specified otherwise to use format from youtube-dl defaults/configuration, e.g. twitch_vod_fetch -o -- vod1-url vod1-tmp vod2-url vod2-tmp.
That standalone "--" is how argparse (and most unixy tools) allows to separate explicit arguments from anything else, so that "vod1-url" won't be used as youtube-dl --output template (arg to -o option instead of arg to whole command).
Not needed if there's no ambiguity, e.g. twitch_vod_fetch -o name-tpl vod-url vod-tmp or twitch_vod_fetch -o --debug -a console-log-level=warn ....
Again, thanks for testing, was able to fix quite a few bugs in there as a result.
Also, might still be a nice to keep a separate repo for "known to be working" windows version, which someone else can use without risk of stumbling into some random unixism that I add later on and didn't test.
I.e. something that can be linked at https://github.com/mk-fg/fgtk/#twitch-vod-fetch as "tested and working windows version" in case anyone else was using that.

Added in mk-fg/fgtk@49c70b0

Thanks.

Again, thanks for testing, was able to fix quite a few bugs in there as a result.

No problem.

Also, might still be nice to keep a separate repo for "known to be working" windows version, which someone else can use without risk of stumbling into some random unixism that I added later on and didn't test.
I.e. something that can be linked at https://github.com/mk-fg/fgtk/#twitch-vod-fetch as "tested and working windows version" in case anyone else was using it.

That's a good idea.
I've decided to discontinue this repository now that I'm not maintaining a completely separate version of the script. You can link to Choonster/fgtk instead.


setProxy phantom function is missing

Could you implement this method please?
https://github.com/ariya/phantomjs/blob/master/examples/openurlwithproxy.js

PR welcome.

closed via #80


[Bug]: mouse hover generates too much traffic

Summary
When the mouse hovers on something, a download is performed. The cache is not used at all (max-age=0 with must-revalidate). This increase the network usage a lot.
Page
https://react.dev/
Details
https://github.com/reactjs/react.dev/assets/1054125/0d3868ac-10b0-499a-8368-baa71baaaacb

Hello @allevo!
I can no longer reproduce this issue (Mozilla's Firefox). Can you check on your side or specify which browser you see this on (if)?

I can reproduce it with Chrome, Safari, and Arc browser.

Exactly, this will be reproducible with any browser, as this seems to be default behavior of Next.js and the prefetching behavior that can't be simply disabled for hover: https://nextjs.org/docs/pages/api-reference/components/link#prefetch


I'm having the same issue. A (bad) workaround is to hit ctrl-k ctrl-g. The issue seems to be atomic waiting for a secondary key to be hit since there are other default keybindings starting with ctrl-k. When a timeout is reached, it performs the kill line command.
Some input from atomic-emacs devs would be great since kill line is such a fundamental feature.

@stufkan Install the package disable-keybindings. Once installed, go to settings for that plugin, Enable All Community Packages, add "atomic-emacs" into Except Community Packages, and finally "ctrl-k" into Prefix Keys.
ctrl-k works without the delay after that.

I solved this issue by adding this to the keymap.cson file:
'body':
 'ctrl-x': 'unset!'  # No cut
 'ctrl-k up': 'unset!'
 'ctrl-k down': 'unset!'
 'ctrl-k left': 'unset!'
 'ctrl-k right': 'unset!'
 'ctrl-k ctrl-w': 'unset!'
 'ctrl-k ctrl-alt-w': 'unset!'
 'ctrl-k ctrl-p': 'unset!'
 'ctrl-k ctrl-n': 'unset!'
 'ctrl-k ctrl-up': 'unset!'
 'ctrl-k ctrl-down': 'unset!'
 'ctrl-k ctrl-left': 'unset!'
 'ctrl-k ctrl-right': 'unset!'
'atom-workspace atom-text-editor':
 'ctrl-k ctrl-u': 'unset!'
 'ctrl-k ctrl-l': 'unset!'
'atom-workspace atom-text-editor:not([mini])':
 'ctrl-k ctrl-0': 'unset!'
 'ctrl-k ctrl-1': 'unset!'
 'ctrl-k ctrl-2': 'unset!'
 'ctrl-k ctrl-3': 'unset!'
 'ctrl-k ctrl-4': 'unset!'
 'ctrl-k ctrl-5': 'unset!'
 'ctrl-k ctrl-6': 'unset!'
 'ctrl-k ctrl-7': 'unset!'
 'ctrl-k ctrl-8': 'unset!'
 'ctrl-k ctrl-9': 'unset!'
'.platform-darwin, .platform-linux': 'ctrl-k ctrl-b': 'unset!'
'.platform-darwin .tree-view, .platform-linux .tree-view':
 'ctrl-k right': 'unset!'
 'ctrl-k l': 'unset!'
 'ctrl-k left': 'unset!'
 'ctrl-k h': 'unset!'
 'ctrl-k up': 'unset!'
 'ctrl-k k': 'unset!'
 'ctrl-k down': 'unset!'
 'ctrl-k j': 'unset!'
'.platform-darwin atom-text-editor, .platform-linux atom-text-editor':
 'ctrl-k ctrl-d': 'unset!'

Thanks for the solutions @bostrt and @crispamares .
This has come up before, and I think the general issue of key binding collisions between packages, and between package & core is something of an unsolved problem.
In the meantime, I've added a note to the README about this.


Publish taglish-profanity-check package

taglish-profanity-check is a version of profanity-check that detect Tagalog and English profanity
profanity-check is Authored by Victor Zhou
this is the taglish-profanity-check repo : https://github.com/reyniel26/profanity-check
The publish profanity-check version will be
bleepy-profanity-check

https://github.com/reyniel26/bleepy-profanity-check


Unable to enable hidden files

I wanted to enable the hidden files,folders and drive option in the settings but im not able to... im sure its not a bug but im not sure how to fix it. Is this option implemented in this preview? Thanks
Steps to reproduce the behavior:

Go to the app
Click on settings
Scroll down to files and folders
See error

Screenshots

Desktop

Windows 10 1909
App version: [v<IP_ADDRESS>]

Duplicate of #177


Tabs per channel/room

My Setup

Windows 10 and 2012R2:
Current App: 1.3.1:

Description
Hi
We would like the ability to have multiple tabs within the app. Each tab would essentially be a channel/room. and each tab would "alert" separately. Ie if the update comes from channel X and we have channel Y and Z open as well the tab hosting channel X should flash/have an update icon.
Thanks

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Thanks for the suggestion @rhysxevans but unfortunately this is too specific for your use case. We have not received anyone else requesting such a feature so we can't afford to spend time building this. If you develop a this feature yourself, please submit a PR and I'll be happy to review and merge it. Cheers, Gab.

I was about to post a new issue when I came upon this.  This would be a nice feature to have.  I could see it as an optional feature though.  I was looking for some way to switch between conversations quickly rather than to have to use the search or scroll through the long list of people/channels to chat with.  If I'm already chatting thing them I'd like to use Ctrl+Tab & Ctrl+Shift+Tab to navigate through the tabs like in a browser.  Conversation switching between channels and individuals can be somewhat of a pain at times and it would be nice to be able to just use the keyboard to navigate to them.
I could see how tabs may be much, but maybe if I hold Ctrl+Tab it could show a list of recent conversations and allow me to navigate through it by consecutively pressing Tab or Shift+Tab - like the dialog in Visual Studio and VS Code to switch between different active documents.


feat: Add styling and refactor some code.

Co-authored-by: Vlaidkolev0321<EMAIL_ADDRESS>

Very good practices. The code style satisfies the eslint convention.


Make unit tests run in parallel in GitHub Actions

https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs
https://stackoverflow.com/questions/66025220/paired-values-in-github-actions-matrix

on: push
jobs:
  test:
    strategy:
      matrix:
        platform: [ubuntu-latest, macos-latest, windows-latest]
    runs-on: ${{ matrix.platform }}
    steps:
    - uses: actions/checkout@v3
    - uses: actions/setup-node@v3
      with:
        node-version: 16
    - run: npm install-ci-test
    - uses:
  publish:
    needs: [test]
    steps:
    - uses: actions/checkout@v3


feature: support route insert sql to shadow table

ref: https://github.com/cectc/dbpack/issues/258
Ⅰ. Describe what this PR did

Add shadow table rule
If we add  /*+ Shadow() */ hint to insert sql, will route this sql to shadow table;
We also defined shadow expr, when the expr matched, the insert sql will route to shadow table.

Ⅱ. Does this pull request fix one issue?
Ⅲ. Why don't you add test cases (unit test/integration test)?
Ⅳ. Describe how to verify it
check integration test result

Ⅴ. Special notes for reviews

看晕了, anyways LGTM 🤣

空了聊下这个功能


Fix Materials breaks my materials

When I press "Fix model" it messes up the textures. I figured out that if I turn off "Fix textures" it solves the problem but then I can't extract textures in unity.  What happens is the empty parts of the textures (Like around the eyebrows) are appearing as black instead of transparent.

When I press "Fix model" it messes up the textures. I figured out that if I turn off "Fix textures" it solves the problem but then I can't extract textures in unity. What happens is the empty parts of the textures (Like around the eyebrows) are appearing as black instead of transparent.

Which version of Blender do you have? because if you have the newest one you should downgrade to the 2.93.8 or less

I was having the same issue. Turns out I wasn't importing the .vrm with the right tool. I recommend VRM Add-on for Blender. Once I used that, the model fix function worked perfectly


Update utils API ref for v0.3.0

Context
What is the purpose of this PR? Is it to

[ ] add a new feature
[ ] fix a bug
[x] update tests and/or documentation
[ ] other (please add here)

Please link to any issues this PR addresses.
Changelog

Reviewing utils docs for v0.3.0 release

Test plan
Please make sure to do each of the following if applicable to your PR. (If you're not sure about any one of these just ask and we will happily help. We also have a contributing page for some guidance on contributing.)

[ ] run pre-commit hooks and linters (make sure you've first installed via pre-commit install)
[ ] add unit tests for any new functionality
[x] update docstrings for any new or updated methods or classes
[ ] run unit tests via pytest tests
[ ] run recipe tests via pytest tests -m integration_test
[ ] manually run any new or modified recipes with sufficient proof of correctness
[ ] include relevant commands and any other artifacts in this summary (pastes of loss curves, eval results, etc.)

UX
If your function changed a public API, please add a dummy example of what the user experience will look like when calling it.
Example of docstring: https://github.com/pytorch/torchtune/blob/6a7951f1cdd0b56a9746ef5935106989415f50e3/torchtune/modules/vision_transformer.py#L285
Example in our docs: https://pytorch.org/torchtune/main/tutorials/qat_finetune.html#applying-qat-to-llama3-models

[ ] I did not change any public API;
[x] I have added an example to docs or docstrings;

CI is red

CI is red

LIES


Add disqus comments to Halve

I would not be in favour of this. Something like this would be much better: http://donw.io/post/github-comments/
There's also some statistics about load times and trackers on the link.

I agree with @libeclipse. Even though Disqus is very easy to set up, the loading time is quite big and I encountered trouble with CSP using it.
I've been thinking about setting up a comment system with Staticman following this guide, but I haven't got the time to design the comments layout in a page :smiley:.