feature: Missing language specific fixes/tests [NP-1480]

Adds in some final language specific tests
Makes a slight optimisation on a haml render

This fails backstop for 4 tests that show in some situations the position of the Search Bar is slightly different. I guess we'll need to confirm this change is ok (This change is likely caused by the non-rendering of a blank section now, causing undue padding)

For this PR what would we like to do now?
Approve and Merge or attempt to fix up the failing labels and then merge?
Thoughts @davidsauntson / @baseonmars

@luke-hill what is failing?  The tickets for the @failing scenarios in this PR are NP-1480, which is also what this PR is fixing, so I'm a bit confused, sorry!

Ah sorry. Yes that's poorly explained.
This ticket was to remedy several things. Primarily around labelling / missing tests.
In doing so (And re-adding all the missing tests e.t.c.) I've found one of the newly added tests doesn't work. I.e. one of the bits mentioned in the original ticket didn't get fixed (Or is invalid). So essentially it's a circular reference 😄
So the two scenarios here that are failing I "believe" might have some missing logic somewhere. The issue was around labelling.

OK - if the failing ones are marked as such, happy for you to merge if there is a clear Jira ticket for each failing feature.

Can we approve this then tackle the remaining issues in a follow-up ticket?


house-of-councillor: suppress "literal string will be frozen in the future" warning

Before change:
$ ruby test/run-test.rb -t HouseOfCouncillorTest 2>&1 | grep red-datasets
/Users/zzz/src/github.com/red-data-tools/red-datasets/lib/datasets/house-of-councillor.rb:108: warning: literal string will be frozen in the future
/Users/zzz/src/github.com/red-data-tools/red-datasets/lib/datasets/house-of-councillor.rb:149: warning: literal string will be frozen in the future
/Users/zzz/src/github.com/red-data-tools/red-datasets/lib/datasets/house-of-councillor.rb:151: warning: literal string will be frozen in the future
/Users/zzz/src/github.com/red-data-tools/red-datasets/lib/datasets/house-of-councillor.rb:153: warning: literal string will be frozen in the future
/Users/zzz/src/github.com/red-data-tools/red-datasets/lib/datasets/house-of-councillor.rb:155: warning: literal string will be frozen in the future

After change:
$ ruby test/run-test.rb -t HouseOfCouncillorTest 2>&1 | grep red-datasets

Thanks.

Thanks for your review!


Print best results for the best iteration round at the end

The Windows executable I mostly work with
does print all the result metric, but not the best result metric.
With a parameters like  early_stopping_round = 800 it will not print
the best result metric at the end and the command window buffer
would not hold too many data (unless set very large).
It would be useful for bookkeeping purposes to record the best metric
round RMSE or AUC at the very end.
The advantage is that one can quickly see the best result at the
end especially for manual parameter tuning, without piping
the text to a file or falling back to another application like R or Python.
[LightGBM] [Info] 2.057539 seconds elapsed, finished iteration 1053
[LightGBM] [Info] Iteration:1054, training l2 : 0.135771
[LightGBM] [Info] Iteration:1054, valid_1 l2 : 0.420244
[LightGBM] [Info] 2.061329 seconds elapsed, finished iteration 1054
[LightGBM] [Info] Iteration:1055, training l2 : 0.135649
[LightGBM] [Info] Iteration:1055, valid_1 l2 : 0.420301
[LightGBM] [Info] Early stopping at iteration 1055, the best iteration round is 255
[LightGBM] [Info] 2.063763 seconds elapsed, finished iteration 1055
[LightGBM] [Info] Finished training

so the final text output should be (as regression example metric)
[LightGBM] [Info] **Best iteration round 255** (0.508083 seconds elapsed)
[LightGBM] [Info] Iteration:255, training l2 : 0.307951
[LightGBM] [Info] Iteration:255, valid_1 l2 : 0.409929

Tnx

That was quick,
Thanks.


feat: adds the ability to configure headers to the 0x api

Description
Adds the ability to configure headers for instances of ZeroExApi.
This way we can configure e.g. authorization headers if needed for the 0x API calls.
Tests
Should still work. ✅

:tada: This PR is included in version 1.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Add WASI support by tokio_wasi

Description
I've been trying to use yew to render the page into the static HTML string on WASI. However, prokio cannot distinguish the browser WASM target (wasm32-unknown-unknown with wasm-bindgen) and WASI target (wasm32-wasi), and it would choose wrong modules for wasm32-*.
This pull request consists of the following changes:

Add the new runtime module which supported by tokio_wasi for wasm32-wasi target.

Checklist

[x] I have self-reviewed and tested this pull request.
[x] I have added tests for my changes.
[x] I have updated docs to reflect any new features / changes in this pull request.
[x] I have added my changes to CHANGELOG.md.

@futursolo Can you give this a review?

I did some research...Although I don’t quite understand your intention, we can indeed use run_until directly in a WASI program that already uses #[tokio::main(flavor = “current_thread”)].
However, I created this PR to support yew being able to do SSR in a WASI environment. In order to do this, I modified many upstream libraries, including this repository.
When yew implements SSR rendering internally, it will create additional threads through prokio for node parsing. This seems to be a good way to speed up rendering, but doing so directly makes the original rendering logic unable to run in the WASI environment...After all, WASI has not yet incorporated multi-threading into the standard.
For implement the SSR rendering capability in WASI first, I currently added the function render_async() to another PR associated with this PR in yew. This function will render nodes serially without creating any additional asynchronous tasks.
The above function has achieved my goal and is already running in my current private business code. Maybe this PR needs to be modified to only ensure that wasm_bindgen is not introduced when compiling with WASI to avoid global symbol export pollution. I may only be able to avoid single-threaded rendering of asynchronous tasks through my new render_async() method before WASI officially supports multi-threading.
cc @futursolo

When yew implements SSR rendering internally, it will create additional threads through prokio for node parsing. This seems to be a good way to speed up rendering, but doing so directly makes the original rendering logic unable to run in the WASI environment...After all, WASI has not yet incorporated multi-threading into the standard.

The reason why it is unable to do so is not because of lack of multi-threading but it assumes wasm-bindgen-futures instead of tokio for wasm32 targets.

This seems to be a good way to speed up rendering, but doing so directly makes the original rendering logic unable to run in the WASI environment...After all, WASI has not yet incorporated multi-threading into the standard.
For implement the SSR rendering capability in WASI first, I currently added the function render_async() to https://github.com/yewstack/yew/pull/3534 associated with this PR in yew. This function will render nodes serially without creating any additional asynchronous tasks.

I intended to address render_async in the Yew pull request.
The purpose of the task spawning in ServerRenderer is not to speed up rendering but to convert a !Send Future to a Send one. You can see below what happens if you try to use the _async functions.

Instead of adding _async flavour, you should use LocalServerRenderer if you do not need Send future and ServerRender if you need Send Future.

I may only be able to avoid single-threaded rendering of asynchronous tasks through my new render_async() method before WASI officially supports multi-threading.

The ServerRenderer avoids single-threading by pinning the renderer to a worker thread that is least busy via spawn_pinned, which achieves multi threading when there is multiple worker threads in the prokio runtime.

When yew implements SSR rendering internally, it will create additional threads through prokio for node parsing. This seems to be a good way to speed up rendering, but doing so directly makes the original rendering logic unable to run in the WASI environment...After all, WASI has not yet incorporated multi-threading into the standard.

The reason why it is unable to do so is not because of lack of multi-threading but it assumes wasm-bindgen-futures instead of tokio for wasm32 targets.

This seems to be a good way to speed up rendering, but doing so directly makes the original rendering logic unable to run in the WASI environment...After all, WASI has not yet incorporated multi-threading into the standard.
For implement the SSR rendering capability in WASI first, I currently added the function render_async() to yewstack/yew#3534 associated with this PR in yew. This function will render nodes serially without creating any additional asynchronous tasks.

I intended to address render_async in the Yew pull request. The purpose of the task spawning in ServerRenderer is not to speed up rendering but to convert a !Send Future to a Send one. Most tokio web frameworks require their futures to be Send and wasm_bindgen generated types are !Send, which makes Yew !Send.
You can see below what happens if you try to use the _async functions.

Instead of adding `_async` flavour, you should use `LocalServerRenderer` if you do not need `Send` futures and `ServerRender` if you need `Send` futures.

I may only be able to avoid single-threaded rendering of asynchronous tasks through my new render_async() method before WASI officially supports multi-threading.

The ServerRenderer avoids single-threading by pinning the renderer to a worker thread that is least busy via spawn_pinned, which achieves multi threading when there is multiple worker threads in the prokio runtime.

Thank you for your reminder. I didn't realize there was an interface called LocalServerRenderer until you gave me specific instructions.
I made some modifications based on your instructions. Now there are no new interfaces, and the original WASI rendering example has been changed to use LocalServerRenderer for rendering.
Perhaps this interface should be specifically pointed out in the offical guide to remind latecomers... I've add some description on the guide.

As it cannot be implemented for a while, could you release a patch version now to support yew to run normally in the WASI environment?
cc #12
@futursolo


Replace clojure.data.json by Chesire in the response marshalling

restQL-core is already using Chesire, however restQL-server is using data.json.
Chesire is twice as fast as data.json.

Pull request submitted
=)


OpenAI-DotNet 8.4.0

Add realtime support
Added o1, o1-mini, gpt-4o-mini, and gpt-4o-realtime model convenience properties

When will OpenAI-DotNet 8.4.0 be available for download?

When this pull request is merged. Still has a long way to go. Need to port the changes over from my unity package.

When will OpenAI-DotNet 8.4.0 be available for download?

When this pull request is merged. Still has a long way to go. Need to port the changes over from my unity package.

Thank you for your work, very cool! I tried to build the project with changes, it seems I got a .dll file, but after replacing the files I did not notice any changes. Can I help with something? Thank you!

No changes in this repo yet. I still need to port them from https://github.com/RageAgainstThePixel/com.openai.unity/pull/303


Patched: Error reading value

Does anyone have a clue how to solve this?
/usr/bin/python3 /home/pi/domoticz/scripts/python/miflora/domoticz.py

1: Palm
Patched: Error reading value retry after 5 seconds...

Patched: Error reading value retry after 5 seconds...

Patched: Error reading value

Not really. I've never seen this error message. Can you provide a stack trace of this error?

What info do you need? The lines in the post above are the only ones i see on screen.

Which source code are you running? Where does this domoticz.py come from?
Can you increase the log level of the output somehow? (I'm not familiar with domoticz)

I followed a tutorial which is located here: http://www.fam-oldenburger.nl/2017/09/06/plantensensor/
It's in Dutch but google translate can translate it to English :)
It does a git clone from https://github.com/Tristan79/miflora.git

looking at the code there: This seems to be a branch of a very old miflora implementation and is not supported any more. Please ask the owner of the repository to upgrade to the current version and use the current version from pypi instead of creating a copy..

@ChristianKuehnel thanks for the info.
I've now downloaded the MiFlora 0.4 from this github page and placed it in my Domoticz/scripts/python/ folder
Copied the Domoticz.py from the repo above and changed the following:
from miflora.miflora_poller import MiFloraPoller,
MI_CONDUCTIVITY, MI_MOISTURE, MI_LIGHT, MI_TEMPERATURE, MI_BATTERY
to:
from miflora.miflora_poller import MiFloraPoller
from btlewrap.gatttool import GatttoolBackend
and 2x:
poller = MiFloraPoller(address)
to
poller = MiFloraPoller(address, GatttoolBackend)
still receiving errors:
pi@raspberrypi: /usr/bin/python3 /home/pi/domoticz/scripts/python/miflora/domoticz.py
WARNING: No module named 'pexpect'
1: Palm
Patched: Error reading value retry after 5 seconds...
Patched: Error reading value retry after 5 seconds...


Registry in Institution and Manufacturer classes

It doesn't make sense to separate registry and registry identifier in this way. The field should be something like RORID or PID and use the PIDName class.

I misread the schema. This is already how it is


内存突然猛增，宕机

版本 3.3.5
今天2020-10-14发生故障，pika服务在09:41:36的时候挂了，客户端连接不上，猜测可能是被系统oom kill了；然后检查监控发现在09:33之前内存占用是1.7G，然后一直到09:39一直猛增到63G了，当然其中包括虚拟内存；在今天09:42的时候立即启动了pika
然后就是想起在昨天2020-10-13我修改了日志配置：
expire-logs-days: 1
expire-logs-nums: 3
修改配置是直接用config set操作的，没有重启
下面是09:41:36左右的日志部分内容
这是info日志：
[pika.INFO]
......
I1013 17:01:20.770723  9640 pika_stable_log.cc:151] ./log/log_db0/ Success purge 1 binlog file
I1013 17:08:15.799190  9640 pika_stable_log.cc:151] ./log/log_db0/ Success purge 1 binlog file
I1013 17:15:15.838668  9640 pika_stable_log.cc:151] ./log/log_db0/ Success purge 1 binlog file
I1013 17:22:00.856263  9640 pika_stable_log.cc:151] ./log/log_db0/ Success purge 1 binlog file
I1013 17:28:15.881863  9640 pika_stable_log.cc:151] ./log/log_db0/ Success purge 1 binlog file
I1013 17:33:50.904104  9640 pika_stable_log.cc:151] ./log/log_db0/ Success purge 1 binlog file
I1013 17:37:55.919528  9640 pika_stable_log.cc:151] ./log/log_db0/ Success purge 1 binlog file
Log file created at: 2020/10/14 09:41:06
Running on machine: kaluka
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I1014 09:41:06.249642 20533 pika.cc:187] Server at: conf/pika.conf
I1014 09:41:06.252269 20533 pika_server.cc:170] Using Networker Interface: eth0
I1014 09:41:06.252386 20533 pika_server.cc:213] host: <IP_ADDRESS> port: 6479
I1014 09:41:06.252404 20533 pika_server.cc:89] Worker queue limit is 20100
I1014 09:41:06.252660 20533 pika_binlog.cc:111] Binlog: Find the exist file.
I1014 09:41:06.253278 20533 pika_binlog.cc:111] Binlog: Find the exist file.
I1014 09:41:06.255131 20533 pika_binlog.cc:111] Binlog: Find the exist file.
I1014 09:41:06.255973 20533 pika_binlog.cc:111] Binlog: Find the exist file.
I1014 09:41:06.256211 20533 pika_binlog.cc:111] Binlog: Find the exist file.
I1014 09:41:06.257148 20533 pika_binlog.cc:111] Binlog: Find the exist file.
I1014 09:41:06.257423 20533 pika_binlog.cc:111] Binlog: Find the exist file.
I1014 09:41:06.257664 20533 pika_binlog.cc:111] Binlog: Find the exist file.
Log file created at: 2020/10/14 09:41:36
Running on machine: kaluka
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I1014 09:41:36.166808 20617 pika.cc:187] Server at: conf/pika.conf
I1014 09:41:36.167258 20617 pika_server.cc:170] Using Networker Interface: eth0
I1014 09:41:36.167330 20617 pika_server.cc:213] host: <IP_ADDRESS> port: 6479
I1014 09:41:36.167342 20617 pika_server.cc:89] Worker queue limit is 20100
I1014 09:41:36.167548 20617 pika_binlog.cc:111] Binlog: Find the exist file.
I1014 09:41:36.167660 20617 pika_binlog.cc:111] Binlog: Find the exist file.
I1014 09:41:36.167701 20617 pika_binlog.cc:111] Binlog: Find the exist file.
I1014 09:41:36.167753 20617 pika_binlog.cc:111] Binlog: Find the exist file.
I1014 09:41:36.167790 20617 pika_binlog.cc:111] Binlog: Find the exist file.
I1014 09:41:36.167829 20617 pika_binlog.cc:111] Binlog: Find the exist file.
I1014 09:41:36.167858 20617 pika_binlog.cc:111] Binlog: Find the exist file.
I1014 09:41:36.167887 20617 pika_binlog.cc:111] Binlog: Find the exist file.
I1014 09:42:02.445022 20533 pika_partition.cc:87] db0 DB Success
......
这是warn日志：
[pika.WARNING]
......
W1014 09:38:20.764549  9640 pika_stable_log.cc:120] ./log/log_db1/ Could not purge 28, since it is already be used
W1014 09:38:20.764569  9640 pika_stable_log.cc:120] ./log/log_db2/ Could not purge 72, since it is already be used
W1014 09:38:20.764591  9640 pika_stable_log.cc:120] ./log/log_db4/ Could not purge 57, since it is already be used
W1014 09:38:25.764804  9640 pika_stable_log.cc:120] ./log/log_db0/ Could not purge 5700, since it is already be used
W1014 09:38:25.765781  9640 pika_stable_log.cc:120] ./log/log_db1/ Could not purge 28, since it is already be used
W1014 09:38:25.765813  9640 pika_stable_log.cc:120] ./log/log_db2/ Could not purge 72, since it is already be used
W1014 09:38:25.765836  9640 pika_stable_log.cc:120] ./log/log_db4/ Could not purge 57, since it is already be used
Log file created at: 2020/10/14 09:42:11
Running on machine: kaluka
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
W1014 09:42:11.437595 20533 pika_server.cc:1580] The Rsync service is down, Try to restart
W1014 09:42:12.465382 20533 pika_rsync_service.cc:61] Rsync service is no live, path:./dbsync/
W1014 09:42:17.465766 20533 pika_server.cc:1580] The Rsync service is down, Try to restart
W1014 09:42:18.508173 20533 pika_rsync_service.cc:61] Rsync service is no live, path:./dbsync/
W1014 09:42:23.508544 20533 pika_server.cc:1580] The Rsync service is down, Try to restart
W1014 09:42:24.544127 20533 pika_rsync_service.cc:61] Rsync service is no live, path:./dbsync/
W1014 09:42:29.544494 20533 pika_server.cc:1580] The Rsync service is down, Try to restart
W1014 09:42:30.581077 20533 pika_rsync_service.cc:61] Rsync service is no live, path:./dbsync/
W1014 09:42:35.581465 20533 pika_server.cc:1580] The Rsync service is down, Try to restart
......
这是错误日志：
[pika.FATAL]
......
Log file created at: 2020/10/14 09:41:36
Running on machine: kaluka
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
F1014 09:41:36.168207 20617 pika_repl_server.cc:37] Start Pika Repl Server Thread Error: 1: bind port 8479 conflict, Listen on this port to handle the request sent by the Slave
......
重启之后一直报警，查看日志，一直是重复下面的警告：
......
W1014 10:37:13.112761 20533 pika_server.cc:1580] The Rsync service is down, Try to restart
W1014 10:37:13.112788 20761 pika_stable_log.cc:120] ./log/log_db0/ Could not purge 5708, since it is already be used
W1014 10:37:13.112857 20761 pika_stable_log.cc:120] ./log/log_db1/ Could not purge 28, since it is already be used
W1014 10:37:13.112877 20761 pika_stable_log.cc:120] ./log/log_db2/ Could not purge 72, since it is already be used
W1014 10:37:13.112902 20761 pika_stable_log.cc:120] ./log/log_db4/ Could not purge 57, since it is already be used
W1014 10:37:14.199055 20533 pika_rsync_service.cc:61] Rsync service is no live, path:./dbsync/
W1014 10:37:19.200078 20533 pika_server.cc:1580] The Rsync service is down, Try to restart
......

从日志看，rsync服务异常。
1首先kill掉rsync进程，然后重启pika来恢复服务。
2 查看dbsync中rsync异常的日志。

嗯，多谢

嗯，多谢


fix_dub

update to allow compiling with recent versions of ggplotd

@ljubobratovicrelja current Mir version is 0.6.5

current Mir version is 0.6.5

I dont' understand, could you be more precise? you mean mir-path or mir-algorithm?
the only mir entries in the dub.json were these (before this PR):
"mir-math": "~>0.0.1",
"mir-algorithm": "~>0.4.14"

I dont' understand, could you be more precise? you mean mir-path or mir-algorithm?

mir-algorithm

BTW, mir-math is a part of mir-algorithm now

ok i see, thanks, will update this PR.
any way you could mark DEPRECATED for deprecate packages in dub descriptions?
dub search mir
==== registry at http://code.dlang.org/ ====
mir-internal (0.0.5)    Generic utilities for Mir projects
numir (0.0.3)           numpy-like API wrappers of mir
mir (1.1.0)             LLVM-accelerated Generic Numerical Library for Science and Machine Learning
mir-math (0.0.1)        Dlang Math Library
mir-blas (0.0.2)        NDSLICE wrapper for BLAS
mir-cpuid (0.5.2)       CPU characteristics identification
mir-random (0.2.5)      Dlang Random Number Generators
mir-lapack (0.0.3)      NDSLICE wrapper for LAPACK
mir-algorithm (0.6.5)   Multidimensional arrays (ndslice), ranges, iterators, and algorithms for Dlang
mir-glas (0.2.3)        Generic Linear Algebra Subprograms

i see mir-math indeed marked as deprecated in https://github.com/libmir but not in the above description.
also, is mir still relevant or deprecated? I'm guessing it's still relevant? The description:
Sandbox for some mir packages: sparse tensors, Hoffman and others. The main repository is https://github.com/libmir/mir-algorithm is ambiguous, it could mean that since main repo is mir-algorithm and this is a sandbox, it's not meant to be used?

mir is relevant

I will  remove mir-marh and mir-internal after this PR is merged

BTW, compiler versions in Travis should be updated too

@timotheecour thanks for taking interest in dcv. :)
Could you please just update mir-algorithm to 0.6.5 then, and remove mir-math from dependencies? This change will be followed with some errors that would need fixing before merge (e.g mir.math.internal should be replaced with mir.math.common).
But anyhow, feel free to fix dcv enough so you can compile your project, and don't worry if travis is falling. Just say when you're done with your fixes, we'll merge, and I will see other details fixed in the following week. Hopefully until then you'll be able to use dcv master version in your project.

BTW, compiler versions in Travis should be updated too

I'll fix this also.

@9il Regarding mir-math
I think we need to replace most instances of import mir.math.internal with import mir.math.common as well

Could you please just update mir-algorithm to 0.6.5

updated to latest 0.6.6

then, and remove mir-math from dependencies? This change will be followed with some errors that would need fixing before merge (e.g mir.math.internal should be replaced with mir.math.common).

done
but now getting new link errors even for programs that were not giving link errors before this PR:
_D3mir7ndslice8topology114__T6unpackVE3mir7ndslice5slice9SliceKindi2VAmA1i1TS3mir7ndslice8iterator26__T11ZipIteratorTPfTPfTPfZ11ZipIteratorZ6unpackFNaNbNiNfS3mir7ndslice5slice113__T5SliceVE3mir7ndslice5slice9SliceKindi2VAmA1i1TS3mir7ndslice8iterator26__T11ZipIteratorTPfTPfTPfZ11ZipIteratorZ5SliceZS3mir7ndslice5slice113__T5SliceVE3mir7ndslice5slice9SliceKindi2VAmA1i1TS3mir7ndslice8iterator26__T11ZipIteratorTPfTPfTPfZ11ZipIteratorZ5Slice
start:time=D20170611T130038U704959 hnsecs=636328080387053170 arg:drun -demangle _D3mir7ndslice8topology114__T6unpackVE3mir7ndslice5slice9SliceKindi2VAmA1i1TS3mir7ndslice8iterator26__T11ZipIteratorTPfTPfTPfZ11ZipIteratorZ6unpackFNaNbNiNfS3mir7ndslice5slice113__T5SliceVE3mir7ndslice5slice9SliceKindi2VAmA1i1TS3mir7ndslice8iterator26__T11ZipIteratorTPfTPfTPfZ11ZipIteratorZ5SliceZS3mir7ndslice5slice113__T5SliceVE3mir7ndslice5slice9SliceKindi2VAmA1i1TS3mir7ndslice8iterator26__T11ZipIteratorTPfTPfTPfZ11ZipIteratorZ5Slice
pure nothrow @nogc @safe mir.ndslice.slice.Slice!(2, [1], mir.ndslice.iterator.ZipIterator!(float*, float*, float*).ZipIterator).Slice mir.ndslice.topology.unpack!(2, [1], mir.ndslice.iterator.ZipIterator!(float*, float*, float*).ZipIterator).unpack(mir.ndslice.slice.Slice!(2, [1], mir.ndslice.iterator.ZipIterator!(float*, float*, float*).ZipIterator).Slice)

done

Awesome. Let me know if I can merge this.
Not directly related to this PR - there's one issue (that I know of) in dcv.features.utils.extractCorners occurring in 2.074.0 - topN seems to be no longer pure. I'll fix this asap. After that I'll update compilers in travis, and after all passes with green flags, we'll make a new tag. @timotheecour you should be OK if you're using dmd 2.072.2.

@9il could you take a look at travis dmd build? What is causing those linking errors?

Ok to merge, yes. Btw would love to chat via hangout if you have time, about DCV apis. Would be more efficient than cia github

Btw would love to chat via hangout if you have time, about DCV apis

No problem, I was about to offer any help you need. Feel free to contact me via hangout.


Can't run certain apps in terminal

Does this issue occur when all extensions are disabled?: Yes

Version: 1.91.1
Commit: f1e16e1e6214d7c44d078b1f0607b2388f29d729
Date: 2024-07-09T22:08:12.169Z
Electron: 29.4.0
ElectronBuildId: 9728852
Chromium: 122.0.6261.156
Node.js: 20.9.0
V8: 12.2.281.27-electron.0
OS: Linux x64 6.8.0-38-generic snap

Installed the app as a classic Snap. It looks like that some apps can't be run from VSCode's terminal. CPU goes 100% and ram gets filled up and crashes the system:
NO extensions BTW
Steps to Reproduce:

open a terminal in code (snap)
run dolphin as an example and CPU goes 100%

Screencast: ignore the flashing
Ignore the flashing
https://github.com/user-attachments/assets/d9429512-9256-44bf-96fe-d9e72c4861a8

does this happen if you run that in an external terminal?

No, It only happens in vscode terminal. I believe this occurs due to snap sandbox blocking something.

Merging to https://github.com/microsoft/vscode/issues/179086
The library variables we set like GTK_PATH, GIO_MODULE_DIR interfere with launching external applications.

The library variables we set like GTK_PATH, GIO_MODULE_DIR interfere with launching external applications.

The app I ran here was dolphin which is a Qt app not GTK. Are you sure that this is related to my problem?
I dont have many GTK apps installed on my system but gnome-disks and lutris work fine from VSCode(snap) terminal.
Tried with more qt apps some of them just won't do anything, Halt the termianl and use 100% CPU and some of them will open up without issues. Note that all of them work fine on my normal Konsole its just vscode's terminal.

dolphin❌
kdialog --msgbox "Hello world"✅
kdialog --getsavefilename❌
kmines✅
kate❌
systemsettings❌
qbittorrent✅
ark opens but clicking on "Open Archive..." which opens a file dialog hangs the app

I suggest dumping the environment variables in your default terminal into a file.
Then dump the environment variables in the VSCode terminal to another file.
Compare the 2 files, and observe if any new environment variable related to Qt/Gtk by either unsettling them in the VSCode terminal to see if any fixes the issue.


Week1 day3

We checked out your repository. We're thrilled to say your solution is now listed in our Hall of Fame
Thank you for your contributions!
This PR is now closed! 🎉


Format on Save moves cursor to bottom of code file in 1.28.1

VSCode Version: 1.28.1
OS Version: Windows 10 1809

Steps to Reproduce:

Edit .ps1 file
Press Save (make sure format on save is enabled)
Cursor will be moved to bottom of the file instead of keeping it in the same place

I've created issue with "movie" on VSCode-PowerShell - https://github.com/PowerShell/vscode-powershell/issues/1576
This issue didn't exists in earlier versions.

Does this issue occur when all extensions are disabled?: Yes/No

This is typically caused by an extension that does not implement format correctly.
Can you reproduce this behaviour when you have this extension disabled<EMAIL_ADDRESS>Can you reproduce this when you have all extensions disabled? Can you reproduce it when you have only the Powershell extension enabled and all the other extensions disabled?

None of these extensions look like they participate on format on save.
Please reopen the issue when it happens again when you have all extensions disabled.


proxy: fix an issue about half-closing net.TCPConn after io.Copy()

Related to docker/docker#27539
After io.Copy(to, from), we should call to.CloseWrite(), not to.CloseRead().
Without this fix, TestTCP4ProxyHalfClose (newly added in this commit) fails as
follows:
--- FAIL: TestTCP4ProxyHalfClose (0.00s)
            network_proxy_test.go:135: EOF

In this test, the client calls Write(), CloseWrite(), then Read(), and the server calls ReadAll(), then Write().
Without the fix, the client fails to Read() because of EOF.
Signed-off-by: Akihiro Suda<EMAIL_ADDRESS>

Thanks @AkihiroSuda
LGTM

pls rebase

done @aboch

Test failure seems unrelated
--- FAIL: TestLinkContainers (0.02s)
	bridge_test.go:722: Failed to setup driver config: unable to add return rule in DOCKER-ISOLATION chain:  (iptables failed: iptables --wait -I DOCKER-ISOLATION -j RETURN: iptables: No chain/target/match by that name.
		 (exit status 1))


File preview tool truncates large files

Dear NOMAD developers,
the file preview tool seems to randomly cut the printout of large files.
Consider the attached Turbomole output file (rename to job.last for importing), when imported as new project and visualized with "preview", its content is first shown correctly, but at  certain point, no content is loaded and shown anymore (see screenshot, printing of atomic coordinates just stops at a certain point).
Data extraction seems to work reasonably well, OTOH.
There is no useful info in the docker logs.
NOMAD version: 1.2.2.dev357+g15b7cd2e1 on Docker 25.0.3.
job.last.txt

Hi @behnle!
Thanks for the report. I would suspect that there is some issue in our GUI code that might cut the preview too short. We will try to reproduce the problem with the data you provided.

There was indeed a problem with determining when the preview should load more data. This is now fixed in the develop branch, but will take some time to be a part of e.g. our central deployments and to our main docker images. If you want to try it right away locally with docker, you could use the gitlab-registry.mpcdf.mpg.de/nomad-lab/nomad-fair:develop image, but this version may not be as stable.


Outdated description of validation_data arg in model.fit method's docs

The last sentence in description of validation_data  argument says: "Note that validation_data does not support all the data types that are supported in x, eg, dict, generator or keras.utils.Sequence."
This seems to be an outdated info and must be removed.

@EvsanDlg are you able to confirm that validation_data does work with generators. (i.e. this is only a documentation update issue)?

I can confirm that validation_data supports tf.keras.utils.Sequence.

I can confirm that validation_data supports tf.keras.utils.Sequence.

I can also confirm that validation_data supports a keras.utils.Sequence much like the fit_generator() functions did.  Having that line in the instruction set is misleading and can send people down an unnecessary 'rabbit hole'.

Hi @EvsanDlg ,
We are in the process of going through backlogs.I have gone through this issue and it's not persists now. The description of validation_data changed already. I am attaching a screenshot of current documentation for reference.

Please cross check and feel free to close the issue if found OK. Thanks.

Checked. It is as stated now. Thanks!


MacVim occasionally stops rendering text after sleep

I've been experiencing issue #1068 (where syntax highlighting turns off in existing MacVim windows when computer wakes from sleep).  I also recently noticed a new issue where all text occasionally disappears in all open MacVim windows.  I think the trigger is either waking the laptop from sleep or attaching / detaching external displays.
The problem occurs every two days or so, and impacts both windows that were already open and any new windows opened after.  I can resolve the problem by quitting and restarting MacVim, but I often have many open windows so prefer to find a fix.
I'm currently using MacVim 8.2.2164 (169), running on OS X 10.15.7.  I think this issue first appeared after the previous MacVim update  (I tried updating when I first noticed the problem to see if it would resolve the issue, but no luck).
Attached image shows a new MacVim window opened up after the issue starts. The only visible text is up on the title bar, the VIM splash screen doesn't show up.

Can you try running with a clean vimrc (by doing mvim --clean), and show me the settings you have by doing defaults read org.vim.MacVim | grep MM?
Also, if you say resize the window, would it have fixed the problem?

Can you try running with a clean vimrc (by doing mvim --clean), and show me the settings you have by doing defaults read org.vim.MacVim | grep MM?
Also, if you say resize the window, would it have fixed the problem?

Settings:
    MMAutosaveColumns = 230;
    MMAutosaveRows = 56;
    MMCurrentPreferencePane = General;
    MMTopLeftPoint = "{927, 1057}";```

I'll try starting with mvim --clean next time the problem occurs.
Resizing windows did not make a difference, but I tried disconnecting and reconnecting the external display and found that the problem eventually went away  (not right after reconnecting but a minute or so later).

Settings:
    MMAutosaveColumns = 230;
    MMAutosaveRows = 56;
    MMCurrentPreferencePane = General;
    MMTopLeftPoint = "{927, 1057}";```

I'll try starting with mvim --clean next time the problem occurs.
Resizing windows did not make a difference, but I tried disconnecting and reconnecting the external display and found that the problem eventually went away  (not right after reconnecting but a minute or so later).

Problem recurred just now,  I tried opening a new window with 'mvim --clean' but see the same problem in that new window. The geometry is different (230x56 instead of my default 120x56) and the color scheme is a bit different, but it has the same problem as a window opened with just 'mvim'.
This time I waited for a while rather than restarting MacVim or detaching/reattaching the external display.  The problem went away on its own after about 5 minutes, although at first it only rendered the top line of each file. Additional lines appeared as I moved the cursor up or down to each line...

Problem recurred just now,  I tried opening a new window with 'mvim --clean' but see the same problem in that new window. The geometry is different (230x56 instead of my default 120x56) and the color scheme is a bit different, but it has the same problem as a window opened with just 'mvim'.
This time I waited for a while rather than restarting MacVim or detaching/reattaching the external display.  The problem went away on its own after about 5 minutes, although at first it only rendered the top line of each file. Additional lines appeared as I moved the cursor up or down to each line...

Having the same problem. This is quite annoying, as I frequently have 4 or 5 windows open, with 2-5 buffers each. And trying to save things is hard because I can't see anything I'm typing. I've seen it go away once, but most of the time I end up restarting it.
Another tidbit, this doesn't happen to me only when waking from sleep (I think). When this happens while I'm editing, it does not remove the existing rendered text until I move my cursor around in the file, and then every line the cursor touches is blanked. If I hit ctrl-L, it blanks the whole window's text.
Attached is a pic of one vim window I hit ctrl-L on with multiple buffers, and another one behind it where I just started to move around (you can see the area around the cursor is blank).

Huh that's annoying… @schveiguy  can you also post your result of defaults read org.vim.MacVim | grep MM? Also, what kind of Mac are you using? And also, what font are you using?
Also, for both @schveiguy  and @mattooca , do your monitors support retina display? Or do you switch between retina and non-retina? I'm mostly just grasping at straws here since I can't seem to repro the issue.

I'm seeing the same problem. I'm running MacVim 8.2.2164 (169) on macOS 10.15.7.
$ defaults read org.vim.MacVim | grep MM
    MMAppearanceModeSelection = 1;
    MMAutosaveColumns = 127;
    MMAutosaveRows = 43;
    MMCurrentPreferencePane = Appearance;
    MMNoTitleBarWindow = 0;
    MMTitlebarAppearsTransparent = 1;
    MMTopLeftPoint = "{0, 1027}";

It occurs without sleeping. It persists for several minutes, and running ':redraw!' does not fix it. But after five-ish minutes, the line under the cursor suddenly becomes visible again, and ':redraw!' then brings the whole screen back.

I'm mostly just grasping at straws here since I can't seem to repro the issue.

It might be a good grasp.  If I had to guess, I'd say it happens mostly on my 4k screen, and it might be 100% of the time, I'm not sure.
I just spent about 5 days where I didn't have my screen, just the retina display of my macbook, and I didn't have the problem happen, but that's not really a meaningful detail, as it doesn't happen with any specific regularity.
My font is Menlo 14 point.
Here is the result of that defaults command:
    MMAutosaveColumns = 170;
    MMAutosaveRows = 83;
    MMCurrentPreferencePane = General;
    MMLastWindowClosedBehavior = 2;
    MMTopLeftPoint = "{1817, 1522}";
    MMZoomBoth = 1;

I did just notice that Macvim supports dark mode, I was on the "evening" setting for colors, because it wasn't available when I started using dark mode. I just switched back to the default scheme, and maybe this will make a difference?

The default MacVim color scheme never got updated for dark mode and doesn't work well under it. I need to fix it… Anyway that's a different issue.

Regarding monitors,  this is happening on a Macbook Pro with Retina display, but I usually have it attached to one or two lower-resolution external displays. The problem is happening right now,  I tried dragging MacVim windows to different monitors but it makes no difference.
One other funny detail:  it looks like the open / close paragraph syntax highlighting still works,  see the little blue box in attached screenshot.

I'm still struggling to repro this (even with an external 4K monitor), but if you see this again, try to change the font / font size. You could just do that by doing <cmd>- or +` to quickly toggle font sizes. I think it would reset the font cache which would bring it back to avoid losing work and allowing you to save.
Also, if changing font/font size does indeed solve the issue, let me know as that would confirm what I think is going wrong.

@mattooca can you clarify what you meant by this?

Attached image shows a new MacVim window opened up after the issue starts. The only visible text is up on the title bar, the VIM splash screen doesn't show up.

Are you saying that you saw that all of MacVim lost text rendering, then you opened a new window (either by Cmd-N or using the menu)? Or are you saying you already had a new window opened with text working, and then it lost its text rendering?

I opened up a new window with command-N after noticing the problem, and
that new window had the same problem.
By the way, next time the problem occurs I'll try your suggestion of
changing the font size. The issue didn't occur today.
On Thu, Feb 11, 2021, 8:14 PM Yee Cheng Chin<EMAIL_ADDRESS>wrote:

@mattooca https://github.com/mattooca can you clarify what you meant by
this?
Attached image shows a new MacVim window opened up after the issue starts.
The only visible text is up on the title bar, the VIM splash screen doesn't
show up.
Are you saying that you saw that all of MacVim lost text rendering, then
you opened a new window (either by Cmd-N or using the menu)? Or are you
saying you already had a new window opened with text working, and then it
lost its text rendering?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/macvim-dev/macvim/issues/1164#issuecomment-777956571,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABLBAL5XVTSTSTXXDVQUXEDS6STKJANCNFSM4V6KFQAA
.

The problem recurred this morning,  and I tried a few things.
Changing font, changing font size, going into full screen mode, changing the color scheme, and opening up new windows: none of those fixed the issue.
However, disconnecting the external display did fix the problem.  Text showed up again on all open windows almost immediately.  The text stayed visible when I then reconnected to the external display, at least so far  (I tried disconnecting about 10 minutes ago).

Oh… ok 😢. This is a little different from what I thought, but will help me prevent digging into a rabbit hole. Since I have trouble reproducing it, let me think about what to do over the weekend to see how this can be debugged and fixed.

Also having this issue, but disconnecting my external monitor did not correct the problem.  Waiting several minutes did.
MacVim 8.2.2164
2017 Macbook Pro w/ retina display
macOS 10.13.6
font DejaVu Sans Mono
% defaults read org.vim.MacVim | grep MM
2:    MMAutosaveColumns = 140;
3:    MMAutosaveRows = 60;
4:    MMCurrentPreferencePane = Advanced;
5:    MMTopLeftPoint = "{797, 1178}";`

I will add that not all open windows are affected equally.  I can have a window open that is displaying text and open a new window and have no text.

More info: It just happened again. I was on a second desktop in this case, and I swiped back to the first desktop to report. Instantly, when I went back to the other desktop, it was working again.

Happening again right now, this time swiping between desktops is not clearing the problem. Changing the font size does not clear the problem. When I moved the window from my 4k display to the retina display, it redrew the window completely and all text is now gone (except the highlights).

It's still in a bad state if you want me to try anything else.

@schveiguy  Can you try to capture the system logs from MacVim? You could either use Console.app to get it, or just use the terminal and do log show --debug --info --predicate='process=="MacVim"' --last 1h > macvim_logs.txt which will grab the logs from last hour and put it to a file macvim_logs.txt that you could attach here.
The wiki for this (https://github.com/macvim-dev/macvim/wiki/Debugging) is a little outdated, but you may also want to do this step first by doing:
defaults write org.vim.MacVim MMLogLevel 7

After we are done debugging you probably want to set it back to default logging level:
defaults delete org.vim.MacVim MMLogLevel

Thanks a lot for the help in debugging.

OK, so my mac went to sleep. I came back to do these steps. I first logged in, and saw that it in fact was still in the bad state. So I ran the log command. Then I ran the defaults command, and switched back to my macvim window and it's now recovering (the text is coming back on lines that I put my cursor on). I took a second log dump. The newest logs after I did the defaults happened at 16:38:11. Attached is the log file.
Should I leave the MMLogLevel at 7 for when this happens again?
macvim_logs2.txt

Yeah maybe leave it at that level for now if you could. It's just going to generate a little more logging but the logs you captured didn't provide quite enough info, so it would be nice to see if we could get more info from this.
I'm trying different ways to see if I could replicate this.

Happened again just now. And here is the weird thing. I was not even using MacVim, I was running a build in another window. And my eyes happen to notice a change on the MacVim window, where the text just disappeared.
I have the logging level set to 7, let me know if you want me to try anything.

And, it's now working again. Here is the last 30m of logs that happened during that time:
macvim_logs.txt

Happened again, recovered again quickly. Logs:
macvim_logs2.txt

And again, it's happening a lot more today than usual. But it's also clearing up quickly too.
This time I moved it to my retina screen, it got blanked, then I moved it back to my 4k screen, and it started working again.
macvim_logs3.txt

Unfortunately those logs do not have debug info on them. I think you need to somehow trigger the debug logging to be on per the steps in https://developer.apple.com/documentation/os/logging/customizing_logging_behavior_while_debugging?language=objc or something, but from local testing, seems like just having Console.app running would turn on debug logging as well. I think next time if you could, maybe have Console.app running in the background?
But now I feel like we are just piling on more work for you to do. Currently the CoreText renderer doesn't have a lot of logging anyway, so I think I'll push an update out soon (last update was in 2020…) and call this out as a known issue, while adding more logging in suspicious points in the renderer so we will get better signal to noise ration in the logging and debugging effort.
Also pinging @s4y to see if he encountered anything like this when implementing the current renderer (tldr: under certain situations such as waking from sleep or unplugging monitors, all texts stop rendering but other parts still draw)

I'm having the issue right now. How do I tell if debug output is on? I'm running the console.app as we speak, but I don't know what information to look for.

FYI, I'm also a software engineer, so I know how hard it is to debug something you can't reproduce. I'm happy to try anything you need to, MacVim is one of my core tools.

Is this correct?
steves@Stevens-MacBook-Pro auditserver % sudo log config --status --subsystem org.vim.MacVim
Password:
Mode for 'org.vim.MacVim'  INFO PERSIST_DEFAULT
steves@Stevens-MacBook-Pro auditserver % sudo log config --mode "level:debug" --subsystem org.vim.MacVim
steves@Stevens-MacBook-Pro auditserver % sudo log config --status --subsystem org.vim.MacVim
Mode for 'org.vim.MacVim'  DEBUG PERSIST_DEFAULT

OK, rendering is back to normal. Here's the last 15m log. I See "Debug" messages in there now.
macvim_log.txt

Oh cool thanks! The logs work now. I'm not fully understanding what I'm seeing but seems like there are a lot of spammy font resource loading, which I never see in my logs. That log was captured when you saw the bug right? Were you doing anything special? Otherwise I probably need to take a look.
Still probably going to push a release because there are a couple places where the additional logging would really help clear up exactly where things are broken.

when I turned on debug logging (and console.app), the bug was happening. I did not have debug logging on to capture the beginning of the problem (note that it happened as the OP said this time, when waking from a sleep).
Let me know if you want me to use a new release/test build.

Issue just happened, for like 10 seconds. Text started disappearing, and soon after started rendering again.
Here is a 5 minute log:
macvim_logs.txt

BTW, if the logs are just spam at this point, let me know and I will stop posting them.

Issue happened again for me this morning. As before, this only happens when I'm connected to an external display and I'm consistently able to get text back by disconnecting and reattaching the external display.
I collected log info this time:  2021_02_24_macvim_logs.txt. I detached the external display around 7:50 a.m. and reattached about 1 minute later.

This is a new situation.  It's suddenly not rendering italic text, but regular is fine.

Huh. Thanks for all the logs. They are of diminishing returns but I don't mind them. If I can't fix this issue I'm just hoping to push out an update this weekend with more logging and basic checks to try to identify where the issue is. So far this seems to be some font loading issues but I'm not 100% sure how yet.
Incidentally I managed to reproduce this once. I was so excited about this but then it went away and I couldn't repro afterwards…

This may be related or a separate issue, but lately I'm seeing the colors disappear after a long period of inactivity (MacVim in the background).  :redraw brings it back immediately.

If any of you see this issue again, do you mind following the instructions at https://github.com/macvim-dev/macvim/issues/1164#issuecomment-781639527 to capture logs and upload here? r170 contained more logging capability and it would be very useful, thanks.

I ran into this this morning. Here are my logs from the last few hours: https://gist.github.com/stevematney/a694694e43f36c07b992c33929a697fc

@stevematney  If you don't mind, do you mind enabling debug logging by doing the following? The drawback is just slightly more logging from MacVim, and debug logging are all logged to an OS memory buffer so it shouldn't use more disk space.
sudo log config --mode "level:debug" --subsystem org.vim.MacVim
defaults write org.vim.MacVim MMLogLevel 7

You can revert the debug logging by doing this:
sudo log config --mode "level:info" --subsystem org.vim.MacVim
defaults delete org.vim.MacVim MMLogLevel

I'm now wondering if I should have just made these logging non-debug-only…

An update: First, I don't have logs to attach, sorry. :-(
I run on a MBP 13" 2019 model (8GB, quad core i5 1.4GHz), no external monitor. I just unlocked the screen saver, switched to MacVim, and watched the line under the cursor disappear, although the remainder of the text was visible. When I moved the cursor up a line, that line also disappeared. A ':redraw' made all the text disappear, though the 'misspelling carats' remained visible. After about five minutes (trying switching to full screen, decreasing and increasing the font size, and minimizing/restoring the window to no avail), the line under the cursor became visible.
An interesting point: Even though text wrapping was enabled, and I was on the second line of a two line paragraph, only the "screen row" of text disappeared, and when it came back, only the screen row came back (until I did a ':redraw'). So it appears to be related to rows of text, rather than paragraphs/logical lines of text.

Just encountered this, again, in build 170.  Symptoms were similar to what timoth-know described...

watched the line under the cursor disappear, although the remainder of the text was visible. When I moved the cursor up a line, that line also disappeared. A ':redraw' made all the text disappear, though the 'misspelling carats' remained visible. After about five minutes (trying switching to full screen, decreasing and increasing the font size, and minimizing/restoring the window to no avail), the line under the cursor became visible.

macvim_logs.txt

I'm running build 170 on an M1 MacBook Air and this issue happens for me multiple times a day. MacVim seems to be fine when my computer wakes up from sleep but an hour or 2 later, text starts disappearing from the screen as I click on the text or as I use the up and down keys to scroll past the text. Logs attached.
defaults read org.vim.MacVim | grep MM
MMAutosaveColumns = 85;
MMAutosaveRows = 65;
MMTopLeftPoint = "{416, 1055}";
macvim_logs.txt

I can trigger this immediately by simulating memory pressure:
sudo memory_pressure -S -l critical -s 60

…so I suspect something is getting kicked out of a cache. Investigating but don't have any solid leads yet.

I'm sorry for the delay in getting to this, all. The linked pull request should fix it.

memory_pressure

That's a nice trick! I'll keep that in mind next time. Also, thanks for looking into this.

Ok this should be fixed! Thanks for the PR. Played around with it with the memory_pressure tool and confirmed the behavior and also that it's fixed.
One of those things that if we switched to ARC instead of reference counting we may have avoided this bug :/ (but then this is a CoreFoundation object, so maybe there may still be chances to have screwed this up due to how you need to specify how the bridging works)

One of those things that if we switched to ARC instead of reference counting we may have avoided this bug :/ (but then this is a CoreFoundation object, so maybe there may still be chances to have screwed this up due to how you need to specify how the bridging works)

@ychin This was making me think about ARC, too. I have some interest in investigating it; is there any history around talking about/trying to switch to ARC?

@s4y I think there is a lack of history of trying to switch to ARC :). But no I don't know what the history is. I think we just never looked into it, and since MacVim tends to follow the Vim philosophy of backwards compatibility, it takes a while before we are ahead enough to be able to say "we can support ARC" without breaking backwards compatibility.
Also, it's just not the kind of thing that brings immediate improvements, even though it does help prevent random memory leaks and potential retain bugs like this one.
I'm not really opposed to switching over. I was hoping to take a look at some point in the future, but if you want to investigate it that's totally fine. FWIW the new tabs PR (#1120) is using ARC, and I explicitly OKed it. No point in making a new project that uses manual reference counting.

@s4y, if you want to look into ARC, please use #1182 instead of this issue.

FYI I recently updated to Big Sur and haven't seen this problem since (still on build 170). But it seems I probably won't see it again, thanks for all the support!

The update has been pushed so if you get the latest version it should be good now.


Neural Style Transfer

Neural Style Transfer In Images!

@geekquad Thanks for contributing ! I have some time to review it, and it looks great !
When you have time, please add data source to either data/readme.md or in the exercise/readme.md. Thanks ! I'll merge once you have included the data source/credit.

Also maybe you could rename the root folder of your project from '007' to another placeholder name (say neural style transfer) to avoid unnecessary conflicts. I'll rename it appropriately when merged. Thanks !


Move FileClient and BlockFileCodec out of test-utils

Should close #5994.

whoops, should have been [FileClient](file_client::FileClient) and [BlockFileCodec](file_codec::BlockFileCodec)

whoops, should have been [FileClient](file_client::FileClient) and [BlockFileCodec](file_codec::BlockFileCodec)

@Rjected Yep just fixed it plus Cargo dependencies.


Expand README

A description and like to the dataset should be provided here. It is fine to use teh same material as here:
https://doi.org/10.7303/syn60239970

ReadMe expanded. Thanks a lot for pointing it out.
Tosin.


Add note about VS2017 validation warning

Also includes minor cleanup

OPS Build status updates of commit 2442113:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/add-ins/outlook-on-send-addins.md
:white_check_mark:Succeeded

docs/add-ins/troubleshoot-outlook-add-in-activation.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


[Bug]: BannerPlugin not working with the default stage PROCESS_ASSETS_STAGE_ADDITIONS

System Info
System:
OS: Windows 10 10.0.19045
Binaries:
Node: 18.17.1
npm: 6.14.4
Details
BannerPlugin only work for stage from PROCESS_ASSETS_STAGE_DEV_TOOLING (500 and onwards) instead of the default stage PROCESS_ASSETS_STAGE_ADDITIONS which is when you ‘Add additional sections to existing assets, like a banner or initialization code‘

Default stage rspack.Compilation.PROCESS_ASSETS_STAGE_ADDITIONS

Stage rspack.Compilation.PROCESS_ASSETS_STAGE_DEV_TOOLING

Reproduce link
https://codesandbox.io/p/github/DiegoSerra/rspack-project/master?import=true
Reproduce Steps
1 - npm create rspack@latest
2 - modifying the rspack.config.mjs file and adding to the list of plugins:
new rspack.BannerPlugin({
   banner: 'hello world',
   footer: true,
   // stage: rspack.Compilation.PROCESS_ASSETS_STAGE_DEV_TOOLING
})

3 - npm run build

the default stage not working is caused by the minifier remove the added banner, if you disable the minimize you can see the added banner.
optimization: {
  minimize:false
}

and since it's same as webpack, I don't think it's a bug


how can i train with a low resolution?

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior (full command you are using).
Expected behavior
A clear and concise description of what you expected to happen.
Full logs
If applicable, add full logs to help explain your problem.
Environment:

OS: [e.g. Ubuntu 18.04]
GPU: [e.g. RTX 2080 Ti *4]
CUDA Version [e.g. 10.2]

Additional context
Add any other context about the problem here.

any params should i update if i want to trian the yolact_edge with resolution of 224*224?

Please refer to #110, thanks.

I am closing this issue for now due to inactivity. If there are any further questions that I could help with, please feel free to reopen, thanks.


Update coordinates

Don't merge yet, some things need changing.

So, before we can release, and have a proper renamed project, we should try to eliminate as many TeamUnify references in the pom.
The source can go later, but the metadata should be correct.

So, should we remove TeamUnify in the <name /> or do you want a different name?
https://github.com/awkay/easy-i18n/pull/8/files#diff-600376dffeb79835ede4a0b285078036L11
Personally I think easy-i18n is pretty good already.


Added conversion for str for git timestamps

Added conversion to avoid error during testing triggering:
$ redis-benchmarks-spec-cli --use-branch --from-date 2023-02-11  --redis_port 12010 --redis_host *** --redis_pass *** --redis_user default
Traceback (most recent call last):
  File "/home/ubuntu/.local/bin/redis-benchmarks-spec-cli", line 8, in <module>
    sys.exit(main())
  File "/home/ubuntu/.local/lib/python3.8/site-packages/redis_benchmarks_specification/__cli__/cli.py", line 53, in main
    trigger_tests_cli_command_logic(args, project_name, project_version)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/redis_benchmarks_specification/__cli__/cli.py", line 259, in trigger_tests_cli_command_logic
    ) = request_build_from_commit_info(
  File "/home/ubuntu/.local/lib/python3.8/site-packages/redis_benchmarks_specification/__common__/builder_schema.py", line 151, in request_build_from_commit_info
    raise Exception(
Exception: Type of field commit_datetime is not bytes, string, int or float. Type (<class 'datetime.datetime'>). Value=2023-02-21 18:58:55+02:00

@filipecosta90, so what do you think, may I keep removing in this PR or should do separate one? Alternatively need to use this variable somehow..


Unable to select virtual environment not located at root

Environment data

VS Code version: 1.21.0
Extension version (available under the Extensions sidebar): 2018.2.1
OS and version: Windows 10 64 Bit
Python version: 2.7 and 3.6
Type of virtual environment used (if applicable): venv
Relevant/affected Python packages and their versions: N/A

Actual behavior
Selecting python executable to run in the lower left hand corner only displays options for the default installed python libraries and virtual environments found at workspace root.

Expected  behavior
Extension should either be able to search within the directory tree for virtual environments contained within folders (for example, I have a venv folder located at .\partner-api\venv), or users should be able to type in a full path to a python executable and have this added to the list.
Steps to reproduce:

Add folder to workspace
Add virtual environment root folder to created folder
Reload VSCode (ref issue #656)
See that new virtual environment has not been added to the list

Logs
Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)
none
Output from Console under the Developer Tools panel (toggle Developer Tools on under Help)
none

Changing image to be more focused.

We look for environments in a specific set of directories.
For instance we do look for environments in the workspace, but only directly in the current workspace, i.e. not 2-3 levels deep. Doing so, is not practical as we'd end up crawling the entire directory structure (imaging a user opening the root directory in VS Code).
The solution in such an instance is for you to use the setting venvPath as follows:
"python.venvPath": "folder path you'd like the extension to look for environments in"

Oops, please confirm this works for you.

Microsoft/vscode-docs#1568 is about documenting python.venvPath better. But as of right now the wildcard directory traversal hasn't seen enough demand to warrant trying to support it (if others come and vote for this then do leave a comment and we will consider re-opening).


Cannot create vagrant master after first attempt failed

I tried to follow the quick start guide. All went smoothly until the kontena master vagrant create command failed due to network errors (propably). kontena master ls shows that no master was created but trying to run command again gives an error Oops... cannot create Kontena Master! You can run only one Kontena Master with Vagrant. How can i get back to clean initial state?
14:14 $ kontena master vagrant create
Creating Vagrant machine kontena-master-sparkling-haze-90 ... -An error occurred while downloading the remote file. The error
message, if any, is reproduced below. Please fix this error and try
again.

transfer closed with 286653149 bytes remaining to read
done
Waiting for kontena-master-sparkling-haze-90 to start ... |^Cfail  # Ctrl + C due hang here

14:24 $ kontena master vagrant create
Oops... cannot create Kontena Master! You can run only one Kontena Master with Vagrant

Please do:
$ rm -rf ~/.kontena/vagrant_master/
and then recreate the master.

Okay, that did it. Thanks!


version 1.1.3: go sdk发布

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


rspec should not run in RAILS_ENV=production by default

rspec should be configured by default to not be run on RAILS_ENV=production
the file spec_helper.rb should contain some similar to this
raise "Not in test" unless ENV['RAILS_ENV'] == "test"

I suggest this since many tests start by truncating all the tables and if you by mistake run this on the production environment, well all your data will be lost

Hope someone else finds this useful and decides to include some default measures to avoid this from happening to somebody else

rspec-core knows nothing about RAILS_ENV or rails at all, and we're going to keep it that way.  Your request is reasonable for rspec-rails, though...do you mind re-opening your issue there since that's where any change would be?


Implement /sharedip

This is a work in progress, mostly want to know "amidoinitrite" and things I still need to implement and all that stuff.

It works for me. Don't I join them again to create the final message?

Yes, you do, and it's a kinda ugly solution...

Yeah fair enough

I can make a function in Chat for that

would let [ip, ...note] = target.split(","); possible solution so you only have to join it once after? (sorry for interrupting if it isnt better)

I did think of that, sparkychild, but introducing a splitOne command would be more readable.

Ok I made the splitOne command. I'll make a way to unmark IPs later.

Also, what the hell is going on with that regex in info.js I didn't change that...

I changed more stuff, @Zarel or someone else, can you sanity check?

No help commands?

I prefer to have this be undocumented so we don't get legions of people
requesting their IPs to be marked in ban appeals

...I'm not sure that would make a difference?

^

did


⚠ 366.ru has degraded performance

In 307261e, 366.ru (https://366.ru) experienced degraded performance:

HTTP code: 200
Response time: 1703 ms

Resolved: 366.ru performance has improved in 8de8f01.


Global task folder

Sorry for submitting many tickets but this one certainly is of low priority (tagged to later). The number of task files accumulated in ~/.sos is a bit worrisome to me. I'm wondering how many files do people usually have there? I removed my ~/.sos/task on Friday morning, and by this time I've got 61072 files in ~/.sos/task. This is two days working on just one project. I understand the advantages to have global task but I'm worried there will eventually be too many files there. I'm thinking maybe we have to move it to local .sos, or an option for it at least; or some way to clean it up when there are too many of them ...

sos purge --age 2d to remove all tasks older than 2 days?

Sure, sorry I should have said to implement a way that "automatically" remove them when there are too may. Yes I'm aware of that option, and even if not, a find can always suffice. But my point is that a regular user does not even realize it can be a problem! Moving to local .sos, on the other hand, can really mess up people's Dropbox or git repos. My initial thought is to set a max number of files allowed (what would it be??) and remove from oldest tasks when there are too many files -- and at the same time generate a warning.

We could use a database-like method to save all tasks into a large file but a daemon-like process would have to manage them on the remote host to accept tasks....

It would help with many small files, but still does not help with the potentially many obsolete signatures. Is it true that conceptually for /path/to/file.txt there should be one task / signature only? This constraint is more fundamental in helping reduce the number of files stored.

What does torque do about this? I remember that it only shows pending tasks but you can refer to completed ones with an ID

I think torque might simply insert to and display result from some databases, but for our case do we have to compare with existing tasks whenever a new one is submitted? That's more complicated than torque and a database may have io problem when it is too large?

One way to solve this problem is to let sos status automatically remove competed/failed tasks older than 1 month, or even 1 week if there are too many files. This will at least avoid infinite accumulation of task files.

Yes this is good housekeeping. But does "removing completed tasks" indicate users will have to rerun old tasks from a week ago if they do revisit their project, without -s build?

No. This only removes task and related out, err, shfiles. The  signatures are still there, and the signatures are mostly local and could be cleaned with projects.

I see. This is good idea.
But this still does not fix the problem when for example users run and complete lots of simulations, then realize they need to make a tiny change in script, and rerun these simulations. Then old tasks are saved and never will be removed. I was intially suggesting moving to local task because suppose we have a project folder for an SoS script (like vtools), then we can implement some command to only keep the signatures currently involved in the script of interest. Of course there are big flaws in my proposal as is, but I think my point of worry is at least valid.

Tasks are shared and do not have concept of a "source" project, especially when they are sent to remote host. This allows tasks to be monitors anywhere, not just from the machine from which the tasks are produced.

I was  only partly correct because although tasks do not have "source", the project knows its tasks and sos purge by default removes all project related tasks. So in your case, you can run sos purge before submitting a new batch of tasks.

I see, but

the project knows its tasks and sos purge by default removes all project related tasks

and I want exactly the opposite, ie, only keep project related tasks signature and remove all obsolete signature from the same project, while leaving signatures of other projects, current or obsolete, intact. Is that possible via some sos purge options?

sos purge purges all tasks related to workflows under the current directory, and leave tasks from other projects (outside of the current directory) intact. I think this is exactly what you need.

I understand what it does. But I'm talking about removing "obsolete" signatures of a project. For example, the code for the project has been changed many times and generated a lot of signatures / task err/out files that will never ever be of any value. Then when a person finally is settled with the project, he can use someting like sos purge --obsolete to remove any signatures historically related to the project, but will not touch any of the most up-to-date signatures so that executing SoS again will not trigger reruns for the same project.

But what is wrong with sos purge before the next run?

I have a hard time imagining how this can be a serious problem with one project. I would prefer not to add any command/option until the extra signatures/task files become a real problem.

The quota on the cluster system for $HOME directory is 30GB. I started to receive warnings recently RE quota limit exceeded. Turns out SoS was the culprit:
32G	./.sos

Pretty impressive considering I only have purged it all a couple of months ago!

This is a bit hard to believe because these are all tiny files. Can you try to get a head count of different types of files and their sizes? Something like du -ch .sos/tasks/*.tasks | grep total.

Ahh I already removed them all ... but I think I know what's going on -- as raised in #840 I do have generated some very large global data object that got pickled to disk for every task I run.

Ok I have another computer that also has a lot in .sos that I was able to try what you suggested:
gaow@trico:~/.sos/tasks$ du -ch *.task | grep total
177M	total
gaow@trico:~/.sos/tasks$ du -ch *.def | grep total
35G	total
gaow@trico:~/.sos/tasks$ du -ch *.err | grep total
396M	total
gaow@trico:~/.sos/tasks$ du -ch *.out | grep total
7.3M	total

In any case, some automatic clean up would be helpful in the long run.

Yes, it is clearly the size of your .def file. You can PM me one so that I can see what data is large.

Here we go:
M400_85afea1462030d119a1b198067b997b2.def.tar.xz.txt
I had to trick github to accept this attachment by adding .txt to the tar.xz extension. When you download and remove .txt you should see a tar.xz file of 112KB. Decompression this file gives you a def file of 52MB.

Update: my script that submits 30K computations to remote cluster at trunk_size = 300 (about 100 tasks submitted) has resulted in 5.9GB def files within a few hours.

Right, and I think this info should be dropped for a task because most of those files are not relevant to an individual task. _output would be enough to keep. (same deal for input and long parameters etc). Is that right?

The pickled output can be compressed quite a lot so it can be a good idea to compress it.
-rw-r--r--      1 bpeng1  MDANDERSON\Domain Users   1973771 Oct 30 10:28 output
-rw-r--r--      1 bpeng1  MDANDERSON\Domain Users    131672 Oct 30 10:32 output.gz

The problem is that individual task could opt to use input or `output, say, in a single-file case.

Or, we can scan task to check the sign of using complete output.

We could also move output etc to the master task level if all subtasks have the same output.

We could also move output etc to the master task level if all subtasks have the same output.

This seems reasonable.

The pickled output can be compressed quite a lot so it can be a good idea to compress it.

Yes ... particularly with xz we were able to reduce the size from 50MB to 100K! But gz is already good if we have to worry about performance.

I have implemented both options, let me know how it works.

Note that I used zlib.compress which is most convenient but not necessarily most efficient in terms of space and time. Since Python 3 now (supports xz compression](https://docs.python.org/3/library/lzma.html), you can play around it and replace the zlib compression with xz compression. The code should be easy to change with compression and decompression both as one liners.

Great! Since my current run has completed (though left behind it large SoS def footprint) I'm going to have to work in exploring the results of computations ... I hope you do not mind I keep this ticket open (and assigned to me) so that I"ll remember to try it out in my next runs.

You could start a few jobs and just check the size of def filed, or you can check it later if you are busy now.

Let me know how the patch works for you so that I can make a next release. Thanks.

Sure, i'll report this back by the end of today!

I just submitted a new batch. The def file size looks interesting:
[MW] ll *.def
-rw-rw-r-- 1 gaow gaow 811K Oct 31 18:07 M400_b565bcda6c0047da4964359a1df260e0.def
-rw-rw-r-- 1 gaow gaow 1.6M Oct 31 18:07 M400_4de2b3868dc263d0687ed02962190a53.def
-rw-rw-r-- 1 gaow gaow 2.4M Oct 31 18:07 M400_3aa68b7371acdfa92848fa556a4d6a58.def
-rw-rw-r-- 1 gaow gaow 3.2M Oct 31 18:07 M400_97c34a1a06176f2dcf2d5f35a47bfdd4.def
-rw-rw-r-- 1 gaow gaow 4.0M Oct 31 18:08 M400_f877f045f6ec1989dc4c9bd7e303f9f5.def
-rw-rw-r-- 1 gaow gaow 4.8M Oct 31 18:08 M400_548679b625cc5958d053324b3c6af8f8.def
-rw-rw-r-- 1 gaow gaow 5.6M Oct 31 18:08 M400_3656b36d2f36bef84ee16b2dc12bffde.def
-rw-rw-r-- 1 gaow gaow 6.4M Oct 31 18:08 M400_0e84b9fee8409b4903aee4e45b16f35a.def
-rw-rw-r-- 1 gaow gaow 7.2M Oct 31 18:08 M400_a1f67b86a12705588a9e1f995d8da565.def
-rw-rw-r-- 1 gaow gaow 7.9M Oct 31 18:08 M400_3824d64dccc4b2bb54341d2fa7e0c820.def
-rw-rw-r-- 1 gaow gaow 8.7M Oct 31 18:08 M400_0416d0b9f70d52c4470f092796eac418.def
-rw-rw-r-- 1 gaow gaow 9.5M Oct 31 18:08 M400_c7937db80a6a939ad6459fbe9d103c6d.def
-rw-rw-r-- 1 gaow gaow  11M Oct 31 18:08 M400_50cb1ad01798c6c6c3ca02119e7af3a9.def
-rw-rw-r-- 1 gaow gaow  12M Oct 31 18:08 M400_67045974cc4d3ba98b61d46af234aeaf.def
-rw-rw-r-- 1 gaow gaow  12M Oct 31 18:08 M400_09014c2c7ba5d9aec433a927af22fc2d.def
-rw-rw-r-- 1 gaow gaow  13M Oct 31 18:08 M400_9388ca7358a85a16aa8999c155e379c4.def
-rw-rw-r-- 1 gaow gaow  14M Oct 31 18:08 M400_522b199c8d3a895d83d3d760e6866485.def
-rw-rw-r-- 1 gaow gaow  15M Oct 31 18:08 M400_5fc193bc7a03b3d8d04c6a5a5ae7f809.def
-rw-rw-r-- 1 gaow gaow  16M Oct 31 18:08 M400_8baf89cb7ffcb46e88eec2c6f5ddaf83.def
-rw-rw-r-- 1 gaow gaow  16M Oct 31 18:09 M400_ca62686448784c798d26caa0d2e47482.def
-rw-rw-r-- 1 gaow gaow  17M Oct 31 18:09 M400_e62139c20629c31f8e550f9a3b20be1a.def
-rw-rw-r-- 1 gaow gaow  18M Oct 31 18:09 M400_8c0f8e010fc0b9df0e67c0065c9525e2.def
-rw-rw-r-- 1 gaow gaow  19M Oct 31 18:09 M400_f2d56cdd16725dabdd10c078e92124a4.def
-rw-rw-r-- 1 gaow gaow  19M Oct 31 18:09 M400_5f28730aa4efb49740515a7fb4a14e67.def
-rw-rw-r-- 1 gaow gaow  20M Oct 31 18:09 M400_b9e31b422a18b1b194e544f3fb25c8aa.def
-rw-rw-r-- 1 gaow gaow  21M Oct 31 18:09 M400_214f527383e955d22df67b8fc108538a.def
-rw-rw-r-- 1 gaow gaow  22M Oct 31 18:09 M400_c090355f170a26ac10f4f3be3fd5ff34.def
-rw-rw-r-- 1 gaow gaow  23M Oct 31 18:09 M400_29827ded66eca836fdcfe5a9f447caa1.def
-rw-rw-r-- 1 gaow gaow  23M Oct 31 18:09 M400_fc2c59b3f7265b8e9b87f65d0a008f0f.def
-rw-rw-r-- 1 gaow gaow  24M Oct 31 18:09 M400_f63eab6e3b59619d574f14068361ec74.def
-rw-rw-r-- 1 gaow gaow  25M Oct 31 18:10 M400_0658847468afa44a181872a887242644.def
-rw-rw-r-- 1 gaow gaow  26M Oct 31 18:10 M400_6be2ffea6581fec607d5a4bb0d882651.def
-rw-rw-r-- 1 gaow gaow  27M Oct 31 18:10 M400_f4c79cb607d3a9e414fb2aa71c5cdd43.def
-rw-rw-r-- 1 gaow gaow  27M Oct 31 18:10 M400_fad8817a7bb960417866d8252d46b15f.def
-rw-rw-r-- 1 gaow gaow  28M Oct 31 18:10 M400_cc6dd956d8cfd37001dd06ca58ec7954.def
-rw-rw-r-- 1 gaow gaow  29M Oct 31 18:10 M400_16358fa7b4f12946c90bff0d2511140c.def
-rw-rw-r-- 1 gaow gaow  30M Oct 31 18:10 M400_18224aebf395bb7e567d1735d8b789b8.def
-rw-rw-r-- 1 gaow gaow  31M Oct 31 18:10 M400_5af02b6020df8bfbbe59e32242fa1425.def
-rw-rw-r-- 1 gaow gaow  31M Oct 31 18:11 M400_2ad0b6b9c906b44e8d59a310d1508a6c.def
-rw-rw-r-- 1 gaow gaow  32M Oct 31 18:11 M400_7ec269df53aebd792fdfab583169e496.def
-rw-rw-r-- 1 gaow gaow  33M Oct 31 18:11 M400_4f08c5a399958972d779dd80f558ff37.def
-rw-rw-r-- 1 gaow gaow  34M Oct 31 18:11 M400_6876b363ffc30b7a630e650d584d389d.def
-rw-rw-r-- 1 gaow gaow  34M Oct 31 18:11 M400_2f852adc905afacbcd8eb9b83d4a82f9.def
-rw-rw-r-- 1 gaow gaow  35M Oct 31 18:11 M400_823ec15f2cfb8b3188331dd013f59b83.def
-rw-rw-r-- 1 gaow gaow  36M Oct 31 18:11 M400_d0a8d98dd18e4569b4b387613a102704.def
-rw-rw-r-- 1 gaow gaow  37M Oct 31 18:12 M400_9c4e58316fd93aca14b0e80160fdb02a.def
-rw-rw-r-- 1 gaow gaow  38M Oct 31 18:12 M400_0c1059714256b9c959812fe8bb14f28a.def
-rw-rw-r-- 1 gaow gaow  38M Oct 31 18:12 M400_3c781d730a7d71ea54143cd54aae54f8.def
-rw-rw-r-- 1 gaow gaow  39M Oct 31 18:12 M400_33d14dbe356434b00a8cd2e6c23ccb25.def
-rw-rw-r-- 1 gaow gaow  40M Oct 31 18:12 M400_15244edd0ec5f9e9275fe53d1f412773.def
-rw-rw-r-- 1 gaow gaow  41M Oct 31 18:12 M400_b3a06e2cf24692495169ade2763379cb.def
-rw-rw-r-- 1 gaow gaow  42M Oct 31 18:13 M400_c19dcec06fd811cb1c40b915f6e95fcd.def
-rw-rw-r-- 1 gaow gaow  42M Oct 31 18:13 M400_feb7d0acec10abe94a45b8df1cce3ab7.def
-rw-rw-r-- 1 gaow gaow  43M Oct 31 18:13 M400_3a6f7d57e37904d364f66c50070bcc8a.def
-rw-rw-r-- 1 gaow gaow  44M Oct 31 18:13 M400_5cabc925c7c43150ff216cffba3bd90c.def
-rw-rw-r-- 1 gaow gaow  45M Oct 31 18:13 M400_56f48e5fa67ead0a38d2d43edfd5439b.def
-rw-rw-r-- 1 gaow gaow  46M Oct 31 18:14 M400_d67aeb99de9a3ab9a3a6d3a5bbf5ead2.def
-rw-rw-r-- 1 gaow gaow  46M Oct 31 18:14 M400_99241ea214382ea0eb98315712e11046.def
-rw-rw-r-- 1 gaow gaow  47M Oct 31 18:14 M400_987cd9422737e176d5dd85166afde3a8.def
-rw-rw-r-- 1 gaow gaow  48M Oct 31 18:14 M400_05ac579462409c2b7fc4f9e87967a069.def

It looks like information accumulate? I have >25K jobs. The relevant parameter setting for task is:
        trunk_size = 400, trunk_workers = 1

that's 25K/400 master jobs.

Information of all 25K jobs are still a bit redundant if they have to present in all master jobs.
As you can see the file size doesnt look right.

But I need to verified this with one early and one later task files.

Here, in xz format:
defs.tar.xz.txt
Perhaps we should also consider xz format "default"? The compression rate is significantly better.

BTW currently submitting 25K jobs results in tasks folder of size  2.5G

You can replace zlib with lzma in sos_task.py, rerun your tests and check the size of the def files.

You were not using the latest trunk because the first line of your task file has
SOSTASK1.1

whereas the latest one should be 1.2.
The '1.2' version should have
[bpeng1@BCBMC02TX0SHHV2:~/Downloads]$ ls -l *.def
-rw-r--r--@ 1 bpeng1  895809667  52161681 Oct 31 18:48 M400_60dba1ac51f7314c0ce29ee8928a4b24.def
-rw-r--r--@ 1 bpeng1  895809667    829714 Oct 31 18:48 M400_b565bcda6c0047da4964359a1df260e0.def
-rw-r--r--  1 bpeng1  895809667  15429505 Oct 31 19:59 v1.2.def

although this is still larger than I expected.

Oops right, I'm so sorry ... I did not update the SoS on the remote machine! Currently my queue on the cluster is full. This batch of jobs will not finish until tomorrow morning. I guess I can wait for your continued updates, then update my remote machine to rerun the jobs.

Ok. The xz compr

Not sure what is going on but the xz compression works much better than zlib.

You mean fact that xz compresses 100 times better than zlib? It's surprising because no benchmark suggests that. But the concept of context-based compression sounds interesting:
https://en.wikipedia.org/wiki/Lempel–Ziv–Markov_chain_algorithm#Overview
I guess we should happily live with it if there is no performance issue. I'm going to submit a test job as soon as my current batch finishes tomorrow.

OK, just replaced zlib with lzma. My explanation is that we have large trunks of repeated stuff (output). They are not identical as each task has one more element so output cannot be moved to master task level. zlib is local compression so it compress each output separately, whereas lzma might have done something more global and really compressed repeated output.
There is indeed some performance penalty, but the benefit clearly outweight the penalty:
-rw-r--r--@ 1 bpeng1  895809667  52161681 Oct 31 18:48 M400_60dba1ac51f7314c0ce29ee8928a4b24.def
-rw-r--r--  1 bpeng1  895809667    114230 Oct 31 21:24 lzma.def

This is something more aggressive. Previously, automatic detection of signature vars are used to include variables that are used by tasks, and now they are used to exclude input, output etc if they are not used. The patch would exclude output from your task files, and substantially reduce their sizes, but it might break some code if the static step analyzer does not pick up output when it is actually used.
We will see how well this works.

I just submitted the 2nd run. The previously 5.2GB task folder is now 3.9 MB, so I guess I have nothing to complain about! Will have  to see unwanted side effects with other jobs though because my current job batch is running happily.

OK. Although there might still be problems with too many small files, I will close this ticket for now.

Okay I'm reopening this ticket because of many complaints I receive from a colleague of mine who recently runs many quick & small sos tasks on the cluster. The symptom she reported is that SoS process simply hangs without any progress for 10min. I asked her to remove ~/.sos and try again. It took > 5min to delete the folder, but it fixed the hang issue. Then I got the question such as why we do not automatically clean things up or why not using time-stamps ...
I think people are generally aware of existence of .sos, and because many are familiar with .snakemake, they know how removing .sos can help getting a fresh run. But not so much for ~/.sos where our task signatures are stored. And it can be big (see discussions in other threads of this ticket).
I'm wondering the following:

Is there a way to automatically remove old signatures of the same step? I guess we need to define what it is meant by "the same step" -- I guess same parameter, input and output files, but not the actual code which people might edit a lot?

Sometimes there are temporary variables in global parameters and can be quite large. is there a way to skip keeping signatures for those? That is, something like temp() for variables?

According to this line of code we remove tasks older than 30 days if it is not running. It certainly does not help for cases with "lots of small tasks in two days".
The slowness caused not only by large number of small files, but also by the time to check the status, because sos needs to check time stamp even content of multiple files.. Let me see if I can cache results between status checks.

Yes the 30 day clean up is useful to some degree. I was wondering about only keeping the "most recent successful version of script" but it seems perhaps difficult?
It would be nice to try to optimize there and then I'll have to try lots of small tasks myself to verify how good it is. I guess the fact that it locates on a cluster file system makes the check even more time consuming.

I am working on a patch that

check the status of a task, but also returns relevant files and their stamps.
Cache the results in a pickle file.
When we check the next time, if all relevant files are still available and have the same time stamp, return status directly.

In this way all failed and completed tasks would be cached ...

Let me know how well this works.

Great! I guess it will take some time to accumulate the task signatures before problems show up, but i'll certainly get there soon and see if there is still a concern. I'll leave the ticket open before I  get there.

It took my new mac about 1 sec to get status of 10,000 tasks from cache, compared to about 8 seconds before. This is on a SSD drive with fast access to small files so I expect bigger improvement on servers and clusters.


Vickel: Watch stltacticals.com

Vickel requests the watch of the watch_keyword stltacticals\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

stltacticals\.com has been seen in 0 true positives, 0 false positives, and 0 NAAs.

Approved by Jeff Schaller in Charcoal HQ


Allows arrays in Like Clauses

Fixes:
Previous Like Clauses were being queried as "orWhere" causing unexpected results with multiple queries. This pull request changes it to a "where" clause.
Example:
Before this query users?username[contain]=farid&email[contain]=gmail would return the following query:
select * from users where username like '%farid%' or email like '%gmail%'
With this pull request the query will now be:
select * from users where username like '%farid%' and email like '%gmail%'
Improves:
Accepts arrays to allow multiple like clauses with the same filter.
Example:
users?username[contain][]=farid&username[contain][]=hossein will return the following query:
select * from users where (username like '%farid%' or username like '%hossein%')

Closing this since I accidentally added a separate commit to this pull request. Will create 2 separate pull requests.


libc/stdio: Enable LIBC_FLOATINGPOINT by default if ARCH_FPU is enabled

Summary
and DEFAULT_SMALL is false
Impact
avoid the wrong configuration for the newbie
Testing
Pass CI

Looks like at least boards/arm/stm32f7/nucleo-144/configs/f746-pysim/defconfig should be modified as well.

boards/risc-v/mpfs/m100pfsevp/configs/nsh/defconfig needs to be updated

@hartmannathan could you please review?

@hartmannathan could you please review?

Sure, i'll review soon. Thanks


Feat/bug-report-and-feature-request-templates

I have created two additional issue templates (aside from the user story template in #2):

A bug report template (inspired by this post and GitHub Docs)
A feature request template (inspired by eslint and GitHub Docs)

What do you think? :-)
(PS I can't figure out how to preview these issue templates locally, so there might be some bugs that I'm only able to see once we have merged into main)

@lwjohnst86 @rqding I'm still struggling a bit to figure out the difference between a user story (added in #2 ) and a feature request (added here). Can you help clarify and help make the difference clearer in the templates when you review?

 Looks great, merging :grin:


Add Edit Birthday Feature

Add feature task #46

Codecov Report

Merging #45 (3ec1576) into master (9760de6) will decrease coverage by 0.01%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##             master      #45      +/-   ##
============================================
- Coverage     40.97%   40.95%   -0.02%
  Complexity      266      266
============================================
  Files            77       77
  Lines          1374     1377       +3
  Branches        144      146       +2
============================================
+ Hits            563      564       +1
- Misses          781      782       +1
- Partials         30       31       +1

Impacted Files
Coverage Δ

...java/seedu/address/logic/commands/EditCommand.java
47.29% <0.00%> (+0.72%)
:arrow_up:

.../seedu/address/logic/parser/EditCommandParser.java
0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9760de6...3ec1576. Read the comment docs.


Move resources into dedicated files with component prefixes

This is a rather subjective PR, so I'm soliciting as much feedback as I can get.
When I work with the Helm chart, I find it rather difficult to find the correct file and instead mostly search for resource names. I would like templates to have some form of structure, so I've split the individual components' big YAML bundles into smaller files, while preserving their component association by prefixing the files with them.
I think this is cleaner and helps in maintaining the chart, but I'm not hellbent on getting this change in if there's no agreement on this being "cleaner" in the first place.

/cc @mjudeikis

/cc @xrstf

/lgtm

/approve


Overlapped alternate/twin axes

Description
When there're more than one alternate/twin axes, they would overlap together.
Steps to reproduce
import proplot as pplt

fig, axs = pplt.subplots(share=0)

ox1 = axs.altx(color='r', label='alternate 1')
ox2 = axs.altx(color='b', label='alternate 2')

Expected behavior:
The spaces between axes should be adjusted autoamtically.

Actual behavior:

Manual fixes
import proplot as pplt

fig, axs = pplt.subplots(share=0)

ox1 = axs.altx(color='r', label='alternate 1')
ox2 = axs.altx(color='b', label='alternate 2')
ox2.spines["top"].set_position(("axes", 1.2))

Proplot version
matplotlib 3.3.4
proplot 0.8.1

I think this is a duplicate of #51. Definitely on the to-do list


Feature Request: Advance Snippet to add 'import' statement at start of the file

The code snippets are the most helpful and productive tool to quickly add the code. To improve the feature it provides, there can be one more property added bodyAtTop.
Use Case
Suppose you have a simple snippet to compose a component in the front end.
"Link": {
	"prefix": "link",
	"body": [
		"<Link to={$1}>",
		"  $0",
		"</Link>"
	],
	"description": "Link"
}

It will insert the <Link> component but not the import statement of a component, i.e., import {Link} from 'react-router-dom'. It can be very helpful if a snippet can do that. So updated snippet can be like below.
"Link": {
	"prefix": "link",
	"body": [
		"<Link$1 to={Routes$2.$3.path}>",
		"  $0",
		"</Link$1>"
	],
	"bodyAtTop": [
		"import {Link} from 'react-router-dom'"
	],
	"description": "Link"
}

Whatever is there in bodyAtTop will be inserted in the first line of the file.

This is a great idea but snippets are static and cannot run code to know what "atTop" means? Above the copyright header? Below the last import statement? I don't believe this can be done without running code and makes this issue a duplicate of https://github.com/microsoft/vscode/issues/10561

Alternatively, you can have an extension that does a snippet-style completion which does all the things "correctly"


[Event Request] Codeunit 90 - "Purch.-Post" - OnSumPurchLines2OnAfterIsRoundingLineInserted

Dear ALAppExtensions team,
through this issue we would like to request the addition of an event publisher OnSumPurchLines2OnAfterIsRoundingLineInserted in Codeunit 90 "Purch.-Post" with the following signature:
[IntegrationEvent(false, false)]
local procedure OnSumPurchLines2OnAfterIsRoundingLineInserted(var PurchaseHeader: Record "Purchase Header"; var PurchaseLine: Record "Purchase Line"; var OldPurchaseLine: Record "Purchase Line"; RoundingLineInserted: Boolean)
begin
end;

which would be used in procedure SumPurchLines2 as follows:
local procedure SumPurchLines2(PurchHeader: Record "Purchase Header"; var NewPurchLine: Record "Purchase Line"; var OldPurchLine: Record "Purchase Line"; QtyType: Option General,Invoicing,Shipping; InsertPurchLine: Boolean)
var
	PurchLine: Record "Purchase Line";
	TempVATAmountLine: Record "VAT Amount Line" temporary;
	TempVATAmountLineRemainder: Record "VAT Amount Line" temporary;
	IsHandled: Boolean;
	PurchLineQty: Decimal;
	BiggestLineNo: Integer;
begin
	IsHandled := false;
	OnBeforeSumPurchLines2(QtyType, PurchHeader, OldPurchLine, TempVATAmountLine, InsertPurchLine, IsHandled);
	if IsHandled then
		exit;

	TempVATAmountLineRemainder.DeleteAll();
	OldPurchLine.CalcVATAmountLines(QtyType, PurchHeader, OldPurchLine, TempVATAmountLine);
	with PurchHeader do begin
		GetGLSetup();
		GetPurchSetup();
		GetCurrency("Currency Code");
		OldPurchLine.SetRange("Document Type", "Document Type");
		OldPurchLine.SetRange("Document No.", "No.");
		OnSumPurchLines2OnAfterSetFilters(OldPurchLine, PurchHeader);
		RoundingLineInserted := false;
		if OldPurchLine.FindSet() then
			repeat
				if not RoundingLineInserted then
					PurchLine := OldPurchLine;
				// >>>> New >>>>
				OnSumPurchLines2OnAfterIsRoundingLineInserted(PurchHeader, PurchLine, OldPurchLine, RoundingLineInserted);
				// <<<< New <<<<
				case QtyType of
					QtyType::General:
						PurchLineQty := PurchLine.Quantity;
					QtyType::Invoicing:
						PurchLineQty := PurchLine."Qty. to Invoice";
					QtyType::Shipping:
						begin
							if IsCreditDocType() then
								PurchLineQty := PurchLine."Return Qty. to Ship"
							else
								PurchLineQty := PurchLine."Qty. to Receive"
						end;
				end;
				...

Thanks in advance!

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 10392.


Some changes are needed to support Ubuntu 19.10 which uses PHP 7.3

Hi when using the script on Ubuntu 19.10 the following happens.
Updating package lists Upgrading packages Installing required packages php7.2-fpm not found, skipping php7.2 not found, skipping php7.2-cli not found, skipping php7.2-dev not found, skipping php7.2-curl not found, skipping php7.2-xmlrpc not found, skipping php7.2-xml not found, skipping php7.2-json not found, skipping

sudo apt install software-properties-common

Installing the package
Then register the PPA

add-apt-repository` ppa:ondrej/php

latest version 1.8.8 should now install correct PHP versions for 19.10


Keybinding in a hover should be in parentheses not brackets?

Follow up on #195994
Currently we wrap the kb in brackets:

But we use parentheses in other places where the keybinding is shown, eg hover over problems or terminal pane label, or other icons.

Hi @ulugbekna thank you for posting this issue. Indeed when I chose the square brackets I was not sure what type of brackets to use. In the editor toolbar menu for example, we appear to be using square brackets. I'll ask in the UX channel about this.


[Feature]: Limit some query that can cause OOM on cosmos node.

Summary
In Oct 3, Cosmoshub Lava provider's QOS availability score dropped so much

And many provider's cosmos node met oom issue. (can check in discord incentivized-cosmos-hub channel)
I have checked the log when the cosmos node ram continuesly increased.
/cosmos/tx/v1beta1/txs?query=tx.height%3E%3D22022712&query=tx.height%3C%3D22022752&pagination.limit=100&pagination.offset=0 was the query which caused the OOM issue
Is it possible to limit this kind of query?
Problem Definition
/cosmos/tx/v1beta1/txs?query=tx.height%3E%3D22022712&query=tx.height%3C%3D22022752&pagination.limit=100&pagination.offset=0 this kind of api query can cause cosmos node memory leak.
Proposed Feature
Limit the query like /cosmos/tx/v1beta1/txs?query=tx.height%3E%3D22022712&query=tx.height%3C%3D22022752&pagination.limit=100&pagination.offset=0  on cosmos chains apis

Hi @silentnoname we are taking this into consideration internally planning on how we can solve this issue, will update on the progress.


Add note about not removing communities and grants from .zenodo.json

Description

Add a note about keeping Zenodo communities and grants to prevent the issue described in #3405 from happening again.
Link to documentation: https://esmvaltool--3417.org.readthedocs.build/en/3417/community/code_documentation.html#list-of-authors

Before you get started

[x] ☝ Create an issue to discuss what you are going to do

Checklist
It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

[x] 🛠 This pull request has a descriptive title
[x] 🛠 Documentation is available
[x] 🛠 The list of authors is up to date
[x] 🛠 All checks below this pull request were successful

To help with the number of pull requests:

🙏 We kindly ask you to review two other open pull requests in this repository

Thanks for reviewing! I opened https://github.com/citation-file-format/citation-file-format/issues/497 to report the issue.


build actions take a lot of minutes

I cloned this repository, and it promptly used up all my github action minutes. Yes, ALL of them.
Here's a usage report for my account:

It seems ridiculous that so many minutes would have been used for any of the builds. This is clearly a problem with the template.

Yep, this is expected currently. It does testing on all devices in several stages. Very beefy CI. Feel free to slim it down, but the heart of the issue is that we aren't providing a cheffed docker image with dependencies already baked in, like Bevy and Trunk. Installing those takes (very) long and eats up a lot of resources.
I'd love to see a custom Docker image made. Until then, I'm your best bet is to turn off minute hogs like MacOS CI.

I think a warning on the install instructions is the least you could do here.


Update cryptocurrency tests with TestKit [ECR-3054]

Overview
Update cryptocurrency tests with TestKit.

See: https://jira.bf.local/browse/ECR-3054
Definition of Done

[x] There are no TODOs left in the code
[x] Change is covered by automated tests
[x] The coding guidelines are followed
[x] Public API has Javadoc
[x] Method preconditions are checked and documented in the Javadoc of the method
[x] Changelog is updated if needed (in case of notable or breaking changes)
[ ] The continuous integration build passes

Coverage increased (+0.6%) to 86.88% when pulling f8b47296568d53923b13af1a8f7273ce33f78143 on ECR-3054_CS into 16087f8102ce8126d47dcf554f445267e3ad0524 on master.