Build Failed

Task :app:easylauncherStageDebug FAILED
FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':app:easylauncherStageDebug'.

java.lang.reflect.InvocationTargetException

Can you share the full exception? Either sharing a full build scan --scan or just the --stacktrace

@mateuszkwiecinski  here the full Exception
`#!/bin/bash -eo pipefail
cd android
sudo chmod +x ./gradlew
(sdkmanager --update && yes | sdkmanager --licenses) && echo accepted || echo ACCEPTED
./gradlew clean
cd ..
npx jetify
cd android && ./gradlew app:assemblestage
[=======================================] 100% Unzipping... emulator/qemu/linux-
All SDK package licenses accepted.======] 100% Computing updates...
ACCEPTED
Downloading https://services.gradle.org/distributions/gradle-7.3-all.zip
...............10%...............20%...............30%...............40%...............50%...............60%................70%...............80%...............90%...............100%
Welcome to Gradle 7.3!
Here are the highlights of this release:

Easily declare new test suites in Java projects
Support for Java 17
Support for Scala 3

For more details see https://docs.gradle.org/7.3/release-notes.html
Starting a Gradle Daemon (subsequent builds will be faster)

Configure project :react-native-reanimated
AAR for react-native-reanimated has been found
/home/circleci/project/node_modules/react-native-reanimated/android/react-native-reanimated-68-jsc.aar
Checking the license for package Android SDK Platform 31 in /opt/android/sdk/licenses
License for package Android SDK Platform 31 accepted.
Preparing "Install Android SDK Platform 31 (revision: 1)".
"Install Android SDK Platform 31 (revision: 1)" ready.
Installing Android SDK Platform 31 in /opt/android/sdk/platforms/android-31
"Install Android SDK Platform 31 (revision: 1)" complete.
"Install Android SDK Platform 31 (revision: 1)" finished.

The Kotlin Gradle plugin was loaded multiple times in different subprojects, which is not supported and may break the build.
This might happen in subprojects that apply the Kotlin plugins with the Gradle 'plugins { ... }' DSL if they specify explicit versions, even if the versions are equal.
Please add the Kotlin plugin to the common parent project or the root project, then remove the versions in the subprojects.
If the parent project does not need the plugin, add 'apply false' to the plugin line.
See: https://docs.gradle.org/current/userguide/plugins.html#sec:subprojects_plugins_dsl
The Kotlin plugin was loaded in the following projects: ':react-native-webview', ':segment_analytics-react-native', ':segment_sovran-react-native'

Task :app:clean
Task :react-native-app-auth:clean UP-TO-DATE
Task :react-native-async-storage_async-storage:clean UP-TO-DATE
Task :react-native-blob-util:clean UP-TO-DATE
Task :react-native-community_art:clean UP-TO-DATE
Task :react-native-community_geolocation:clean UP-TO-DATE
Task :react-native-community_netinfo:clean UP-TO-DATE
Task :react-native-device-info:clean UP-TO-DATE
Task :react-native-exception-handler:clean UP-TO-DATE
Task :react-native-gesture-handler:clean UP-TO-DATE
Task :react-native-inappbrowser-reborn:clean UP-TO-DATE
Task :react-native-mapbox-gl_maps:clean UP-TO-DATE
Task :react-native-masked-view_masked-view:clean UP-TO-DATE
Task :react-native-pdf:clean UP-TO-DATE
Task :react-native-permissions:clean UP-TO-DATE
Task :react-native-push-notification:clean UP-TO-DATE
Task :react-native-safe-area-context:clean UP-TO-DATE
Task :react-native-screens:clean UP-TO-DATE
Task :react-native-share:clean UP-TO-DATE
Task :react-native-splash-screen:clean UP-TO-DATE
Task :react-native-svg:clean
Task :react-native-vector-icons:clean UP-TO-DATE
Task :react-native-version-check:clean UP-TO-DATE
Task :react-native-view-pdf:clean UP-TO-DATE
Task :react-native-webview:clean UP-TO-DATE
Task :realm:clean UP-TO-DATE
Task :segment_analytics-react-native:clean UP-TO-DATE
Task :segment_sovran-react-native:clean UP-TO-DATE
Task :sentry_react-native:clean UP-TO-DATE

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 1m 20s
29 actionable tasks: 2 executed, 27 up-to-date
Jetifier found 1528 file(s) to forward-jetify. Using 36 workers...

Configure project :react-native-reanimated
AAR for react-native-reanimated has been found
/home/circleci/project/node_modules/react-native-reanimated/android/react-native-reanimated-68-jsc.aar

The Kotlin Gradle plugin was loaded multiple times in different subprojects, which is not supported and may break the build.
This might happen in subprojects that apply the Kotlin plugins with the Gradle 'plugins { ... }' DSL if they specify explicit versions, even if the versions are equal.
Please add the Kotlin plugin to the common parent project or the root project, then remove the versions in the subprojects.
If the parent project does not need the plugin, add 'apply false' to the plugin line.

What went wrong:
Execution failed for task ':app:easylauncherStageDebug'.

java.lang.reflect.InvocationTargetException

Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.

Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 19s
90 actionable tasks: 90 executed
Exited with code exit status 1
CircleCI received exit code 1`

Is your build environment dockerized? Maybe you hit this issue: https://github.com/usefulness/easylauncher-gradle-plugin/issues/201?
Can you share full stacktrace, by running the gradle command with --stacktrace argument? You pasted full console log, but it doesn't have all the precious information about the source of the error

Yes i'm using circleci. @mateuszkwiecinski

Cool! So this means this is a duplicate of #201. Feel free to share the details I requested above, in case your issue is any different :)

this works for me @mateuszkwiecinski
https://github.com/usefulness/easylauncher-gradle-plugin/issues/201#issuecomment-1195274416


"createDataRowArray" is not defined error in Lumira 2.0

Hi All,
Please help me on the below issue.
Recently we have upgrade from SAP Design Studio 1.6 SP5 to SAP Lumira 2.0 and when i verify i am getting error in one of the Dashboard.Rest all of the reports are working fine.Getting strange error Script error.
Below the error
Error during script processing. Contact the Application Designer to resolve the issue.
When i open the Dashboard and see any script errors i dont see any errors
Same dashboard when i open it in Design Studio 1.6 SP5 i am not getting any error.
Raised incident with SAP on this issue.They have suggested to upgrade to SAP Lumira SP03.I have done but still getting the same error.
Can anyone help on the issue?
Thanks
Shiva Shankar

Adding to the Point

We have used Bookmarks to the Dashboard.
2.Added SDK from below link

https://github.com/org-scn-design-studio-community/lumiradesignercommunityext/blob/master/releases/org.scn.community.sdk.extensions.zip?raw=true
Just want to check am i missing anything apart from this SDK's?

Below the error

Error

Hi Mike,
Came to know DATA Iterator SDK is not working properly .its throwing error in Lumira 2.0a dn came to know this is because of Dataiterator SDK component
Below is the error message
!ENTRY com.sap.ip.bi.base.application 4 0 2017-11-22 17:48:17.980
!MESSAGE Message: org.mozilla.javascript.EcmaError: ReferenceError: "createDataRowArray" is not defined. (DATAITERATOR_MAXDMD.getRows()#6)
Stack trace: com.sap.ip.bi.zen.rt.framework.jsengine.JsEngineException: org.mozilla.javascript.EcmaError: ReferenceError: "createDataRowArray" is not defined. (DATAITERATOR_MAXDMD.getRows()#6)
at com.sap.ip.bi.zen.rt.framework.jsengine.rhino.RhinoJsEngine.handleError(RhinoJsEngine.java:194)
at com.sap.ip.bi.zen.rt.framework.jsengine.rhino.RhinoJsEngine.doRunScript(RhinoJsEngine.java:88)
at com.sap.ip.bi.zen.rt.framework.jsengine.AbstractJsEngine.runScript(AbstractJsEngine.java:12)
Can you please help on fixing this.Thanks


vm: multisig template

closes: https://github.com/spacemeshos/go-spacemesh/issues/3268
depends on: https://github.com/spacemeshos/go-scale/pull/16
vm:
changed method signature for Verify from
Verify(ctx Context, raw []byte) bool
to
Verify(ctx Context, raw []byte, dec *scale.Decoder) bool
it simplifies decoding and enables variable sized fields in the signature.
multisig:
there are 3 multisig (K/N) variants 1/N, 2/N, 3/N
N is a number of keys that own multisig, N is limited by 10 and provided when account is spawned.
K is a number of signatures required to pass threshold, template can't have less keys than K.
multisig relies on canonical ed25519 signatures. K owners needs to sign a hash of the payload.
see type definitions in https://github.com/spacemeshos/go-spacemesh/blob/20f5a38aab38daff830b08f611ed5aef2e1bd53d/genvm/templates/multisig/types.go
public keys for spawn are encoded as a variable sized collection of 32byte arrays.
signature is encoded as a constant sized collection with tuples [(ref uint8, sig [64]byte)]. collection has to be sorted in the ascending order.

main question i have is about the design. why not implement single sig acct as 1-1 and we will just have one type of wallet?

i don't know yet if we will drop singlesig. i personally don't care, and will do what other will decide.
from technical point of view singlesig transactions is slightly shorter, due to the absence of var sized fields and removed ref field from signature

bors merge

bors merge


installing modsecurity 2.7.4 causes IBM HTTP Server not responding to requests

MODSEC-404: I have been installed mod security 2.6.4 with custom rules on my server and it works correctly. Because of new security issue on modsecurity i decided to update modsecurity to 2.7.4. I Installed 2.7.4 version and there was no error during compilation and installation. Restarting IHS with new modsecurity module causes my web server not responding to my requests. i commented all rules but the problem not be solved yet!

IBM HTTP Server packaging APR module version 1.2.12, this version using /dev/random for the random number generation. ModSecurity 2.7.1 and above calling the apr_generate_random_bytes() from APR module for Hash key generation, since the /dev/random is very slow on Virtual machine, all the request will be waiting on the random number generation.
Workaround for this problem is use the below ModSecurity directives to avoid calling the random number generation from APR module.
SecHashEngine On
SecHashKey "NoRand" "KeyOnly"
Best Regards,
Venkat

Thank you for the note @venkatreddykp.


[wip]Add telescope integration for fuzzy-searching branches

Adds optional telescope integration, used when searching for a branch

I've gotta rebase this, it's only 4 files -_-


Need a way to avoid importing basilleaf/django-list-modified.git

This is currently in requirements.txt and is actually needed. We need a better solution.

The problem is that the real django-last-modified project is from 2012, and django has moved on since then and it's no longer compatible. If you use the real project, you get:
File "/home/rfrench/anaconda2/lib/python2.7/site-packages/last_modified/last_modified.py", line 8, in
from django.utils.importlib import import_module
ImportError: No module named importlib
Because they were supporting backwards compatibility with python 2.6, which is no longer needed.

This is no longer compatible with Django 2.1 and Python 3.6, so its use has been removed entirely. At some point we will need to figure out what kind of caching we want the browser to do.


Optimize ORC reading when file contains many tiny stripes

Fixes #3663

@dain Addressed comments.

I think we should take a slightly different approach to this.  When we are creating the CachingOrcDataSource we can determine the cache regions and simply pass them into constructor.  That way the CachingOrcDataSource doesn't need to understand stripe details at all which should really simplify the code.
Additionally, I think we can add another simple optimization.  In the OrcReader constructor we can check the overall file size and if it is smaller than maxMergeDistance, we can create s single CachingOrcDataSource for the whole file.

Decoupled CachingOrcDataSource with stripes.

ping @dain

@dain back to you

Looks good


Review of Codes

Great job so far 👍
checking all the codes on your part of the project, you have done well, implementing all necessary requirements.
I also checked the corrections and adjustment we both suggested and i can see you have done it.
I have nothing to say than to appreciate working with you
Congratulations me made it to the end of the project.
Happy coding

Hello @Ellyboi
After checking on your code I noticed that everything is just fine, you did used the modules and proper set up for webpack, at this moment the only thing to look at will be to organize your code a little bit more, but overall your project is working really good! congratulations


Error when setting up SDKs if previous download failed

Repro steps:

Setup environment (python, cmake, ninja, etc)
git clone https://github.com/floooh/sokol-samples.git
cd sokol-samples
./fips setup emscripten
Cancel build in middle of download with CTRL + C
./fips setup emscripten (again)

Expected: Previous zip download is detected as corrupt, and the download is attempted again
Actual: Previous zip download is detected as present, and python zip throws an exception that crashes fips
Error is:
λ fips setup emscripten
=== setup emscripten SDK:
'emsdk-portable-64bit.zip' already exists

uncompressing 'emsdk-portable-64bit.zip'...
Traceback (most recent call last):
  File "fips", line 17, in <module>
    fips.run(fips_path, proj_path, sys.argv)
  File "C:\Users\my_user_name\development/fips\mod\fips.py", line 49, in run
    verb.verbs[verb_name].run(fips_path, proj_path, verb_args)
  File "C:/Users/my_user_name/development/fips/verbs\setup.py", line 17, in run
    emscripten.setup(fips_dir, proj_dir)
  File "C:\Users\my_user_name\development/fips\mod\emscripten.py", line 111, in setup
    uncompress(get_archive_path(fips_dir), get_sdk_dir(fips_dir), 'emsdk-portable')
  File "C:\Users\my_user_name\development/fips\mod\emscripten.py", line 74, in uncompress
    with zipfile.ZipFile(src_path, 'r') as archive:
  File "C:\Users\my_user_name\AppData\Local\Programs\Python\Python36\lib\zipfile.py", line 1100, in __init__
    self._RealGetContents()
  File "C:\Users\my_user_name\AppData\Local\Programs\Python\Python36\lib\zipfile.py", line 1167, in _RealGetContents
    raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

Yeah, that's a bit unfortunate. For a quick workaround, delete the fips-sdks directory in the same directory where the fips directory is and try the fips setup emscripten again.
I'll keep the ticket open for the case that I'll get around to implement a proper fix, but it's quite low-prio at the moment.
Thanks for the bug report :)

...it's now possible to cleanup any old or partial emscripten SDK installs with:
./fips emsdk uninstall


expandable watch node

It would be nice if you could have an expandable watch node (similar to the watch 3d). Right now it's hard to preview large strings or nested lists of elements with long names. Also it would be great if you can quickly copy parts of a watch node's output.

Modifying DynamoCore.Configuration.Configuarions provides access to the watch node size variables

I could possibly slap a serialized version of every object( processing nested lists will be tricky, will need recursion) inside a textbox controller.
To make the textbox expandable, I'll need to monitor the form's window resize event and update every time.

Done as a winforms app. Will post it in the next version.


Dark icons and colors shown in light themes

Not clear what the reproducible steps are, but @alexandrudima both seen it after a update: Light themes with dark icons and some other dark colors shining though in a light theme.
Debugging this I saw that both the 'vs' and the 'vs-dark' classes are set.
Bug is in the workbenchThemeService. When applying the persisted colors, the 'vs' and the 'vs-dark' classes are not cleared.

verified by @bpasero


KOGITO-4616 - Revise BDD tests debugging instructions and add GoLand

KOGITO-4616
Many thanks for submitting your Pull Request :heart: :robot:!
Please make sure your PR meets the following requirements:

[x] You have read the contributors' guide
[x] Pull Request title is properly formatted: [KOGITO-XYZ] Subject
[x] Pull Request contains a link to the JIRA issue
[x] Pull Request contains a description of the issue
[x] Pull Request does not include fixes for issues other than the main ticket

@sutaakar As promised, I added GoLand docs + also revised some of the badly formatted lines and dead links.


As a user, I want to have a custom default description for the code

I am already copy pasting the description from one code to the other

Opened #30, so this one can be closed


[C++][Compute] Add quotient and modulo kernels

Add a pair of binary kernels to compute the:

quotient (result after division, discarding any fractional part, a.k.a integer division)

mod or modulo (remainder after division, a.k.a % / %% / modulus).
The returned array should have the same data type as the input arrays or promote to an appropriate type to avoid loss of precision if the input types differ.

Reporter: Ian Cook / @ianmcook
Related issues:

[C++] Arithmetic kernels for numeric arrays (is a child of)
[C++] Implement mod compute function (is duplicated by)
[C++][Compute] Add remainder and quotient compute::Function (is related to)

PRs and other links:

GitHub Pull Request #11116

Note: This issue was originally created as ARROW-12755. Please see the migration documentation for further details.

A modulo kernel would also be nice for duration types (which are just integers under the hood). For instance, in time series a common task is autodetect the frequency at which time stamps are provided, which can be done by computing the GCD of all time deltas.

I wanted to switch to pyarrow but after being bitten by this I'm not sure how mature a library that doesn't implement divmod is :/.


fixed typo of variable

What is the purpose of the change
fixed type of variable
Brief changelog
rename "copyinvokers" into "copyInvokers" in java code
Verifying this change
XXXXX
Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Make sure there is a GITHUB_issue field for the change (usually before you start working on it). Trivial changes like typos do not require a GITHUB issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[ ] Format the pull request title like [Dubbo-XXX] Fix UnknownException when host config not exist #XXX. Each commit in the pull request should have a meaningful subject line and body.
[ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
[ ] Run mvn clean install -DskipTests & mvn clean test-compile failsafe:integration-test to make sure unit-test and integration-test pass.
[ ] If this contribution is large, please follow the Software Donation Guide.

Codecov Report

Merging #2875 into master will decrease coverage by 0.09%.
The diff coverage is 100%.

@@            Coverage Diff            @@
##           master    #2875     +/-   ##
=========================================
- Coverage   63.68%   63.59%   -0.1%
=========================================
  Files         577      577
  Lines       25930    25930
  Branches     4541     4541
=========================================
- Hits        16514    16489     -25
- Misses       7261     7282     +21
- Partials     2155     2159      +4

Impacted Files
Coverage Δ

...bo/rpc/cluster/support/FailoverClusterInvoker.java
88.88% <100%> (ø)
:arrow_up:

.../apache/dubbo/qos/protocol/QosProtocolWrapper.java
64.1% <0%> (-17.95%)
:arrow_down:

.../remoting/transport/netty4/NettyClientHandler.java
75% <0%> (-11.12%)
:arrow_down:

...e/dubbo/remoting/transport/netty/NettyChannel.java
54.11% <0%> (-8.24%)
:arrow_down:

...ng/exchange/support/header/HeartbeatTimerTask.java
73.68% <0%> (-5.27%)
:arrow_down:

...rpc/protocol/dubbo/telnet/InvokeTelnetHandler.java
54.21% <0%> (-3.62%)
:arrow_down:

...org/apache/dubbo/rpc/protocol/AbstractInvoker.java
59.67% <0%> (-3.23%)
:arrow_down:

...pache/dubbo/remoting/transport/AbstractServer.java
45.36% <0%> (-3.1%)
:arrow_down:

...a/org/apache/dubbo/monitor/dubbo/DubboMonitor.java
87.85% <0%> (-1.87%)
:arrow_down:

...dubbo/remoting/exchange/support/DefaultFuture.java
67.11% <0%> (-1.35%)
:arrow_down:

... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 133d570...8a0a6d2. Read the comment docs.


FM-Bench features are symbolic links?

I extract FM-Bench features using python evaluations.py --config configs/fmbench_eval.yaml，but I find the kpts and descs are symbolic links. So When I run FM-Bench Pipline_Demo, the returned fid of fopen('.keypoints') is -1. And I get Invalid file identifier. Use fopen to generate a valid file identifier.
There is the result when I run file 0006* in the features folder:
0006_l.descriptors: symbolic link to /data_ssd/FM-Bench/Features_aslfeat_ms/TUM/0005_l.descriptors
0006_l.keypoints:   symbolic link to /data_ssd/FM-Bench/Features_aslfeat_ms/TUM/0005_l.keypoints
0006_r.descriptors: data
0006_r.keypoints:   data

@TLzh Encountered the same problem as you. How did you solve it? Looking forward to your help. Thank you very much.


Bug: Name not showing properly in new Sidebar View

I'm using the new Sidebar View in Home Assistant as it works better to show the various plex libraries however the name is not showing properly here.

Fixed in 2.1.1.


Daily note shortcut only works with .md file open

Describe the bug
With no files open in VSCODE the create daily note shortcut does not create and open a new daily note.
To Reproduce
Steps to reproduce the behavior:

Open VSCODE
Wait for Dendron plugin to load
Close all open files
Click Control+Shift+I
Nothing happens

Expected behavior
I would like a new daily note to be created when no files are open and I click control shift I
Screenshots
n/a
Desktop (please complete the following information):

OS: Windows
dendron.log

Browser: Edge

Version: Dendron version 0.17.1, Windows Version 10, Edge version 87

Additional context
Add any other context about the problem here.
Dendron Log file
Please attach the output of >Dev:Dendron: Open Logs here

Probably related to #374

Yeah, just repro'd this. I can't even see logs generated when I try the daily note shortcut without an open file.
i.e. when a file is open I see this line:
{"level":30,"time":1619359059775,"pid":24112,"hostname":"CHeXPS","name":"dendron","ctx":"CreateDailyJournal","journalName":"journal","fname":"daily.journal.2021.04.25"}
But when a file is not open, this never occurs in the logs.

Actually, figured this out - it's due to the when clause of the shortcut. We default to editorFocused but if you remove that clause, cmd+shift+i/ctrl+shift+i will work as you like:

This should be changed in the defaults then, I would guess.

Nice catch. This will be included in Monday's release!
On Sun, Apr 25, 2021 at 7:06 AM flammehawk @.***> wrote:

This should be changed in the defaults then, I would guess.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dendronhq/dendron/issues/382#issuecomment-826330370,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AABPIPQVKDPDVDGUY6666ELTKQOVBANCNFSM4UEIRBMQ
.


WIP: vcs: Move difftest control logic to separated DifftestEndpoint module

The difftest control logic was independent of testbench, move it to a separated module DifftestEndpoint. The clock/reset/fsdb related logic, which were strongly related to testbench, were kept in the orignal tb_top module.
Also add BOSC to the copyright of modified files.

I think it's good to separate difftest logic to a new module. This makes the TOP cleaner.

@klin02 please help review whether the changes are good and do not bring unexpected issues

For WIP PR, we can convert it to a draft on the GitHub webpage under the "Reviewers" tab


Add mixed features

Two small things where one is needed and the other an improvement in ergonomics.

Possibility to set chain policy. The default policy is accept, and we need to set it to drop to block packets not matching our accept rules.

Making InterfaceName more ergonomic by making it Clone, but also implementing ToSlice for just a reference to one.

This change is

Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable

Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


Additional default .ol-viewport css settings

Fixes #12439 and #12566
Adds default settings to .ol-viewport to reset some settings observed to break or give poor user experience if set by third party libraries.

That would be nicer, although I had avoided using initial due to it not working in IE.  Starting with IE fallbacks for the most obvious potential problems, followed by the all: initial as suggested would be an alternative, as the fallbacks could be easily removed if support is dropped at a future release.

I think we can start ignoring Internet Explorer specifics, especially when they are easy to work around on the application level.
Also, I'd set all: initial on the respective elements programmatically when they are created, to make things work correctly even when ol.css is not used.

The effects of using initial for divs are too far reaching, and I have now noticed that even just the clearing the background has broken the control button white borders in the deploy-preview examples.  However css which sets the background of divs by type selector only is unlikely to be encountered.  It is also difficult to know if fonts have been set intentionally, as in the examples layout.css, or as an unwanted effect of another library as in the Mapbox-gl Layer example.  That leaves canvas, where it would be easy to set the style directly on the element in createCanvasContext2D.

Replaced by #12626


Whitelist required to watch AcornTV app (british version of netflix)

Energized Protection - block ⚡

Let's make an annoyance free, better open internet, altogether!

Issue Submit Form

  Provide the following infos properly, which will help us to resolve your issue quickly.

Issue(s):
Type x in between [ ] and make sure there isn't any space between brackets. Example; for Your Selected Issue(s), type like this - [x]
You can select more than one category of issues if you need to!

[X ] Whitelist
[ ] Blacklist
[ ] App Broken
[ ] Website Broken
[ ] Request
[ ] Other Issue

Pack(s):
Write the name of the pack(s) you are using.

Pack(s) Name:

Extension(s):
Also name the extension(s) if you are using any. If you don't then leave it blank.

Extension(s) Name: Hosts list that include api.rlje.net:
https://raw.githubusercontent.com/jerryn70/GoodbyeAds/master/Hosts/GoodbyeAds.txt
https://raw.githubusercontent.com/EnergizedProtection/block/master/basic/formats/hosts
https://raw.githubusercontent.com/EnergizedProtection/block/master/bluGo/formats/hosts
https://raw.githubusercontent.com/EnergizedProtection/block/master/blu/formats/hosts
https://raw.githubusercontent.com/EnergizedProtection/block/master/ultimate/formats/hosts.txt

Domain(s):
If you are submitting this issue for whitelist/blacklist issue, send us the domain(s) for whitelisting/blacklisting here. Kindly use the Code Tag to prevent tracking.

Domain(s):

api.rlje.net

Your Config:
Just to ensure there is no issues or conflicts with other app/software/magisk module/extension/source list.
Make sure you are running Energized Protection Service only.

Client:

Version:

Details:
Write us a lil bit more about your issue or query. You can attach any screen shot or log of the issue or advert, this will help to highlight it.

Your Issue Detail:
Whitelist Request:
api.rlje.net

Devices effected:
All amazon firesticks
Hosts list that include api.rlje.net:
https://raw.githubusercontent.com/jerryn70/GoodbyeAds/master/Hosts/GoodbyeAds.txt
https://raw.githubusercontent.com/EnergizedProtection/block/master/basic/formats/hosts
https://raw.githubusercontent.com/EnergizedProtection/block/master/bluGo/formats/hosts
https://raw.githubusercontent.com/EnergizedProtection/block/master/blu/formats/hosts
https://raw.githubusercontent.com/EnergizedProtection/block/master/ultimate/formats/hosts.txt
Effects of having list above enabled without api.rlje.net on whitelist:
When user tries to use AcornTV app, which is a paid subscription service popular in Britain that allows the user to watch certain movies and tv shows (Very similar to Netflix), this is what comes up. There is only a blank black screen where the movies and shows should be listed.

Effects of whitelisting:

Thank you for making Energized Protection great with your kind help!

A project by Ador with ❤

fixing upstream
https://github.com/jawz101/MobileAdTrackers

So Jawz101/MobileAdTrackers is where we need to start reporting issues to so that the changes effect all of the energized list?

no.  I just did an update a few weeks ago that pulled in a bit of false positives.  Since Energized uses mine as one of its base lists and has a larger userbase I've been watching his issues to remove any that were reported.
I've also been cleaning it up on my own quite a bit but he hasn't done a list update of his sources in the past few weeks.

@jawz101 @jawz101 Thanks for cleaning your list and letting us use it too !!

It's a fun hobby.  I want it to just be limited to mobile ads & tracking networks and let other lists handle browser ads.  I just had this idea to take the OpenDNS 1 million most queried domains and whittle it down from there, and then add that to the blocklist.  I looked for ones that had certain strings in the domain name, didn't fall in some categorical lists that some site filtering progs use (ex: education, govt, web hosting, etc.), weren't common servers (dns, ntp, smtp, etc.), weren't randomly generated subdomains (googlevideo/netflix/etc.) and got it down to about 29,800.  At some point I just couldn't sit on it any more without others using it.
I was a little shocked that some businesses were using Energized (I just suggest sticking with StevenBlack's) but mine is getting normal again and is a great addition for both iOS and Android devices.


Volumes not correctly mapped

Hello!
I have the following property in my application.properties:
quarkus.kubernetes.mounts.airflow-dags.path=/opt/airflow/dags

which generates the following
in the deployment.yaml at level spec > template > spec > containers
          volumeMounts:
            - mountPath: /opt/airflow/dags
              name: airflow-dags
              readOnly: false

but we're missing at level spec > template > spec
      volumes:
        - name: airflow-dags
          emptyDir: { }

needed for it to work.
The error I get is
Error: INSTALLATION FAILED: Deployment.apps "my-service" is invalid: spec.template.spec.containers[0].volumeMounts[0].name: Not found: "airflow-dags"

Hello @antssilva96 , you're only mounting one part of the volume, but missing the second part which states what kind of volume is. See more information at https://quarkus.io/guides/deploying-to-kubernetes#mounting-volumes.
Anyway, the issue you created is about Quarkus Kubernetes and should be reported in http://github.com/quarkusio/quarkus.

Thanks for the help! I'm new to kubernetes and helm so still figuring out some stuff!
I don't see any option for mounting a volume with just an empty dir, I assume this means the volume must already be created in the cluster ?
Thanks once again!


Security: Create functions for CRC32x checksums for a file

Create functions
int
pgmoneta_create_crc32c_file(char* path, char** crc);

int
pgmoneta_create_crc32c_buffer(void* buffer, size_t size, char** crc);

This need to be done with AVX instructions

I would like to work on this.

You likely need to stick to AVX2 - see https://en.wikipedia.org/wiki/Advanced_Vector_Extensions - or SSE...
I would just begin with a basic implementation, and then we can add compiler switches like #ifdef HAVE_AVX2 to have a more optimized version.
There are a lot of articles around it, like

https://news.ycombinator.com/item?id=32305900
https://github.com/topics/crc32?l=c

Remember you have to develop this code yourself...

@jesperpedersen There is a CRC32C implementation already present in hashmap.c. Do we want to implement a similar function here? It uses SSE4.2 instructions.


Avoid double-free on unsuccessful getting PRNG seeding

Fixes #16631

Checklist

[ ] documentation is added or updated
[ ] tests are added or updated

Looks like other RNGs are not affected.

I think it would also be worthwhile fixing the caller to not free this pointer on error.
I've created #16640 to do that.  I think the two should stand independently.


Add Worktrees support

Add support for git worktrees.
fixes #1322

I tried this locally and it works. Thanks for the fix.


fix panic

fixes: #13545

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: hzxuzhonghu
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pilot/OWNERS [hzxuzhonghu]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: hzxuzhonghu, rshriram
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pilot/OWNERS [hzxuzhonghu,rshriram]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@hzxuzhonghu: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
f0a75d636d79d7b3639f61e4eac2e3d4d757b502
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


How to make benchmarks run fine on windows?

I want to benchmark for some of windows default commands like rd, robocopy.
but I can't make it work because of error.

Benchmark 1: rmdir /S /Q D:/tmp/
Error: The preparation command terminated with a non-zero exit code. Append ' || true' to the command if you are sure that this can be ignored.

It works well with release files of this project, without rmdir lines in batch script.
Please help.

As expected for windows right now since we're just forwarding to https://github.com/XAMPPRocky/remove_dir_all/blob/master/Cargo.toml. In that note, I noticed that the parallel feature is disabled by default. If you enable it here (https://github.com/SUPERCILEX/fuc/blob/cb34600eeb590fbb3f30f88ee447154ee356c744/fuc_engine/Cargo.toml#L24), you should get about the same performance.
Can you try that and share benchmarks?

@SUPERCILEX not relevant, do you know some windows internals?
I hope to enhance windows default file operation by minifilter driver because copyhook implementation seems have some limitations.
But I don't know about minifilter and windows internals and many other fast handlers out there are using copyhook.
I hope to get some advice on this.
Thanks.

benches.zip
I can't wait to finish remove benchmark running on HDD.
But I guess almost last phase.
need to adjust values or run benches on SSD...
HDD takes too much time.

Thanks! So that's with the parallel feature enabled? Seems like a win for the large benchmarks.

Yes, I enabled parallel feature.

Sweet! I just pushed https://github.com/SUPERCILEX/fuc/commit/844b20b9c94154f34ebe60b6166355bb8f2a52df

benches (2).zip
Added copy benchmarks, just stdlib, rayon, cpz.
I'll try this with robocopy later.


Eureka discovered service static (angular) content not served through gateway (spring-cloud-gateway)

Hello,
I have an issue where an Angular 7 UI is not served when the route is called through the gateway.
Gateway service (gateway): http://localhost:8060 ---> Routes are located
UI Service (appui) : http://localhost:8091/ ----> This works
Routed URL: http://localhost:8060/appui ----> This does NOT work
Versions cloud : Greenwich.SR1 , boot: 2.1.5.RELEASE, angular: 7
Routes are auto located in the gateway.
[{"route_id":"app_ui","route_object":{"predicate":"org.springframework.cloud.gateway.support.ServerWebExchangeUtils$$Lambda$388/0x0000000800442840@4b6ae744","filters":["OrderedGatewayFilter{delegate=org.springframework.cloud.gateway.filter.factory.StripPrefixGatewayFilterFactory$$Lambda$391/0x0000000800441840@50ab56e2, order=0}","OrderedGatewayFilter{delegate=org.springframework.cloud.gateway.filter.factory.RewritePathGatewayFilterFactory$$Lambda$393/0x0000000800441040@7b5b5bfe, order=0}"]},"order":0},{"route_id":"CompositeDiscoveryClient_APPUI","route_definition":{"id":"CompositeDiscoveryClient_APPUI","predicates":[{"name":"Path","args":{"pattern":"/appui/**"}}],"filters":[{"name":"RewritePath","args":{"regexp":"/appui/(?<remaining>.*)","replacement":"/${remaining}"}}],"uri":"lb://APPUI","order":0},"order":0},{"route_id":"CompositeDiscoveryClient_GATEWAY","route_definition":{"id":"CompositeDiscoveryClient_GATEWAY","predicates":[{"name":"Path","args":{"pattern":"/gateway/**"}}],"filters":[{"name":"RewritePath","args":{"regexp":"/gateway/(?<remaining>.*)","replacement":"/${remaining}"}}],"uri":"lb://GATEWAY","order":0},"order":0}]

In the gateway I've a routebuilder defined to add a trailing / onto the UI route.
**** Note: This is just the latest version, I've tried numerous combinations.**
@Bean
    RouteLocator routes(final RouteLocatorBuilder routeBuilder) {
        return routeBuilder.routes()
                .route("app_ui",
                        route -> {
                            Route.AsyncBuilder uri = route
                                    .path("/appui/**")
                                    .filters(
                                            f -> f.stripPrefix(1).rewritePath("/appui/(?<RID>.*)", "/${RID}/")
                                    )
                                    .uri("http://localhost:8091");
                            System.out.println(uri);
                            return uri;
                        }
                )
                .build();
    }

The gateway logs below show that the route is hitting the ui service but returning the files themselves but not the rendered content:
2019-06-06 13:15:37.823 DEBUG 18900 --- [ctor-http-nio-2] o.s.w.s.adapter.HttpWebHandlerAdapter    : [7e7b22d4] HTTP GET "/appui"
2019-06-06 13:15:38.125 DEBUG 18900 --- [ctor-http-nio-2] o.s.w.s.adapter.HttpWebHandlerAdapter    : [7e7b22d4] Completed 304 NOT_MODIFIED
2019-06-06 13:15:38.139 DEBUG 18900 --- [ctor-http-nio-2] o.s.w.s.adapter.HttpWebHandlerAdapter    : [7e7b22d4] HTTP GET "/styles.3d37d63d23d319945b7f.css"
2019-06-06 13:15:38.142 DEBUG 18900 --- [ctor-http-nio-2] o.s.w.r.handler.SimpleUrlHandlerMapping  : [7e7b22d4] Mapped to ResourceWebHandler ["classpath:/META-INF/resources/", "classpath:/resources/", "classpath:/static/", "classpath:/public/"]
2019-06-06 13:15:38.149 DEBUG 18900 --- [ctor-http-nio-2] o.s.w.r.resource.ResourceWebHandler      : [7e7b22d4] Resource not found
2019-06-06 13:15:38.153 DEBUG 18900 --- [ctor-http-nio-3] o.s.w.s.adapter.HttpWebHandlerAdapter    : [564a0604] HTTP GET "/runtime.26209474bfa8dc87a77c.js"
2019-06-06 13:15:38.155 DEBUG 18900 --- [ctor-http-nio-3] o.s.w.r.handler.SimpleUrlHandlerMapping  : [564a0604] Mapped to ResourceWebHandler ["classpath:/META-INF/resources/", "classpath:/resources/", "classpath:/static/", "classpath:/public/"]
2019-06-06 13:15:38.158 DEBUG 18900 --- [ctor-http-nio-3] o.s.w.r.resource.ResourceWebHandler      : [564a0604] Resource not found
2019-06-06 13:15:38.158 DEBUG 18900 --- [ctor-http-nio-8] o.s.w.s.adapter.HttpWebHandlerAdapter    : [69d992f7] HTTP GET "/polyfills.8bbb231b43165d65d357.js"
2019-06-06 13:15:38.159 DEBUG 18900 --- [ctor-http-nio-1] o.s.w.s.adapter.HttpWebHandlerAdapter    : [3733013b] HTTP GET "/main.d366f2c101e009ba6c55.js"
2019-06-06 13:15:38.161 DEBUG 18900 --- [ctor-http-nio-8] o.s.w.r.handler.SimpleUrlHandlerMapping  : [69d992f7] Mapped to ResourceWebHandler ["classpath:/META-INF/resources/", "classpath:/resources/", "classpath:/static/", "classpath:/public/"]
2019-06-06 13:15:38.162 DEBUG 18900 --- [ctor-http-nio-1] o.s.w.r.handler.SimpleUrlHandlerMapping  : [3733013b] Mapped to ResourceWebHandler ["classpath:/META-INF/resources/", "classpath:/resources/", "classpath:/static/", "classpath:/public/"]
2019-06-06 13:15:38.165 DEBUG 18900 --- [ctor-http-nio-8] o.s.w.r.resource.ResourceWebHandler      : [69d992f7] Resource not found
2019-06-06 13:15:38.166 DEBUG 18900 --- [ctor-http-nio-1] o.s.w.r.resource.ResourceWebHandler      : [3733013b] Resource not found
2019-06-06 13:15:38.186 DEBUG 18900 --- [ctor-http-nio-8] a.w.r.e.AbstractErrorWebExceptionHandler : [69d992f7] Resolved [ResponseStatusException: 404 NOT_FOUND] for HTTP GET /polyfills.8bbb231b43165d65d357.js
2019-06-06 13:15:38.186 DEBUG 18900 --- [ctor-http-nio-3] a.w.r.e.AbstractErrorWebExceptionHandler : [564a0604] Resolved [ResponseStatusException: 404 NOT_FOUND] for HTTP GET /runtime.26209474bfa8dc87a77c.js
2019-06-06 13:15:38.186 DEBUG 18900 --- [ctor-http-nio-1] a.w.r.e.AbstractErrorWebExceptionHandler : [3733013b] Resolved [ResponseStatusException: 404 NOT_FOUND] for HTTP GET /main.d366f2c101e009ba6c55.js

Here are the logs from the UI service:
2019-06-06 13:15:38.061 DEBUG 11120 --- [io-8091-exec-10] o.s.web.servlet.DispatcherServlet        : GET "/", parameters={}
2019-06-06 13:15:38.063 DEBUG 11120 --- [io-8091-exec-10] o.s.b.a.w.s.WelcomePageHandlerMapping    : Mapped to ParameterizableViewController [view="forward:index.html"]
2019-06-06 13:15:38.063 DEBUG 11120 --- [io-8091-exec-10] o.s.w.s.v.ContentNegotiatingViewResolver : Selected 'text/html' given [text/html, application/xhtml+xml, image/webp, image/apng, application/signed-exchange;v=b3, application/xml;q=0.9, */*;q=0.8]
2019-06-06 13:15:38.063 DEBUG 11120 --- [io-8091-exec-10] o.s.w.servlet.view.InternalResourceView  : View name 'forward:', model {}
2019-06-06 13:15:38.063 DEBUG 11120 --- [io-8091-exec-10] o.s.w.servlet.view.InternalResourceView  : Forwarding to [index.html]
2019-06-06 13:15:38.064 DEBUG 11120 --- [io-8091-exec-10] o.s.web.servlet.DispatcherServlet        : "FORWARD" dispatch for GET "/index.html", parameters={}
2019-06-06 13:15:38.065 DEBUG 11120 --- [io-8091-exec-10] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped to ResourceHttpRequestHandler ["classpath:/META-INF/resources/", "classpath:/resources/", "classpath:/static/", "classpath:/public/", "/"]
2019-06-06 13:15:38.089 DEBUG 11120 --- [io-8091-exec-10] o.s.web.servlet.DispatcherServlet        : Exiting from "FORWARD" dispatch, status 304
2019-06-06 13:15:38.089 DEBUG 11120 --- [io-8091-exec-10] o.s.web.servlet.DispatcherServlet        : Completed 304 NOT_MODIFIED

My Gateway application.yml is as follows:
server:
  port: 8060

spring:
  application:
    name: gateway

  cloud:
    gateway:
      discovery:
        locator:
          enabled: true
          lowerCaseServiceId: true

eureka:
  client:
    fetch-registry: true
    register-with-eureka: true
    serviceUrl:
      defaultZone: http://localhost:8082/eureka
    healthcheck:
      enabled: true

management:
  endpoint:
    gateway:
      enabled: true

  endpoints:
    web:
      exposure:
        include: gateway

My UI application.yml:
server:
  port: 8091

spring:
  application:
    name: appui

management:
  health:
    binders:
      enabled: true

eureka:
  client:
    serviceUrl:
      defaultZone: http://localhost:8082/eureka
    healthcheck:
      enabled: true

Could the loadbalanced route be affecting the rewriting or adding of the trailing / for the UI or is it something else ?
Thanks!
ps; asked on stackoverflow - https://stackoverflow.com/questions/56517305/eureka-discovered-service-static-angular-content-not-served-through-gateway-s

Solved...
I needed to add a '/' when entering the URI in the browser.
http://localhost:8081/appui/
I'm not sure if my other changes were also necessary.

Hi @kingo999, i have the same issue with you, but the issue is not disappear despite i have added the '/' to the end of the uri just like you have done. the only difference  is my ui project is a nodejs project and also use the angular framework, the error log is the same with you.
the gateway uri: http://localhost:20200/
the ui uri through gateway: http://localhost:20200/appui/
the ui service uri: http://localhost:4000/
I am very confused why the gateway route the appui page to http://localhost:4000/ perfectly, and wrongly route its resources to http://localhost:20200/** rather than http://localhost:4000/**, can you help me please?

@jianjunchang without specifics it will be very difficult to answer your question.

Hi @spencergibb, thanks for your reply, my issue is very very same with the @kingo999 's, so his description for the issue is just what i can provide, and i want to know how can i config the gateway to make the static resources loaded from ui server (eg: http://localhost:4000/main.js) rather than from the gateway (eg: http://localhost:20200/main.js), because the main.js is actually in the ui server.


Bump kotlin-dsl plugins

Follow up for #25344
Fix for https://github.com/gradle/gradle-private/issues/3884

I've triggered a build for you. Click here to see all failures if there's any.

Pre-tested commit build failed. Click here to see all build failures.

I've triggered a build for you. Click here to see all failures if there's any.


WIP new models that enable dynamic url filtering

...based on self-service user input that is persisted globally in the database
WIP. Not working yet. No tests. DO NOT MERGE:)
This is for #28

so the way this works is like  so:
In [1]: from report.models import *

In [2]: filter = UrlFilter.objects.all()[0]

In [3]: filter.run_query()
{'url_paths__path': 'longer', 'url_paths__sequence': 3}
Out[3]: <UrlQueryset [<Url: https://txt.ist/this/is/a/longer/path/and/stuff/proof.json?foo=bar&baz=bore&whoop=azz#fraggy>]>

@ecumike I added minimal tests: https://github.com/IBM/page-lab/pull/36/files#diff-6987c66d1bf14362d0b44549634fafa9


Add "unspecified traffic sign"

Sometimes, we map highway=traffic_sign without specifying the traffic_sign=* (which is usually the primary tag for historic reasons).
Those traffic signs are missing a preset, which I added as unsearchable.
I created a separate file for that because I think there is no way to give a preset two different, unrelated tags to match on.

Wiki https://wiki.openstreetmap.org/wiki/Tag:highway%3Dtraffic_sign
Usage 1k+ https://taginfo.openstreetmap.org/tags/highway=traffic_sign#map, all over the world

Testing

https://pr-984--ideditor-presets-preview.netlify.app/id/dist/#background=Bing&disable_features=boundaries&id=n10311996320&locale=en&map=19.00/52.46748/13.47302
https://www.openstreetmap.org/node/10311996320


Add SPM, Linux support, GitHub Actions and perform some cleanup

Fixes #167 and #95
This PR is based on #172 and replaces the Carthage/Xcode project with a minimal, cross-platform layout that follows the SwiftPM conventions and integrates libgit2 through an (external) .systemLibrary target. The advantage of this approach is that it is now easy to build and test the package on platforms like Linux with minimal setup:
apt install libgit2-dev
swift test

To verify this, it also adds a GitHub Actions workflow that performs a full build/test run on Ubuntu 20.04.

@fwcd Thanks for the PR! Could you either rebase or propose changes to resolve the merge conflicts for this please?

@fwcd Thanks for the PR! Could you either rebase or propose changes to resolve the merge conflicts for this please?

Also, I want to make sure we don't clobber the use of Carthage. I'll test it out on my end but it will reduce back and forth if you make that does not break as well.

Also, I want to make sure we don't clobber the use of Carthage. I'll test it out on my end but it will reduce back and forth if you make that does not break as well.

@fwcd Just following up on the comments above!

I'm testing this as a package directly from https://github.com/fwcd/SwiftGit2
Getting "'git2.h' file not found" in shim.h and "Could not build Objective-C module 'Clibgit2'" in CheckoutStrategy.swift.
I've installed libgit2 via brew.

I ended up getting this working, but I had to change a few things to build with Xcode (swift build worked fine).
So I was getting an error in shim.h that git2.h was not found.
I checked the package.swift file and saw that the pkgConfig value was git2.
Running pkg-config --cflags git2 shows nothing found, so that means that the headers can't be found during compilation. Running pkg-config --cflags libgit2 points to the correct includes folder for the home-brew install.
So, by changing pkgConfig to libgit2 as per below, the compiler can now find libgit2 headers.
.systemLibrary(
	name: "Clibgit2",
	pkgConfig: "libgit2",
	providers: [
		.brew(["libgit2"]),
		.apt(["libgit2-dev"]),
	]
),

There's also a type issue that has to be fixed in Diffs.swift. You can see the changes I've made to my fork here:
https://github.com/fwcd/SwiftGit2/compare/spm-linux...solanne-io:spm-linux
I'm now using my fork successfully in Xcode, and the library is working fine in my code.
One last thing to note, if you're using a sandbox you'll need to disable library validation in order to use the swiftgit2 library. Not sure how this code can be signed and used via SPM.

@fwcd Getting around to PR reviews -- Do you want to salvage anything, based on recent master?

Can't promise anything, sorry. Been pretty busy recently and the project I needed this library for ended up not using it. The PR comment was just something I randomly stumbled on while browsing around.

@fwcd Totally relatable. (See the state of this repo :)) Are you fine with closing this PR? If you ever find time again, we can reopen

Sure! Feel free to cherry-pick anything useful.


dexie-export-import: Out-of-memory Crash On Import

Dexie.js version 2.0.4
dexie-export-import version 1.0.0-beta.12
-> Using default options
While attempting to import a very large (~1.5 GB) previously exported database, chrome repeatedly ran out of memory and crashed. The very same code works fine for smaller files. It is running in a chrome worker, and given a Blob that is properly formatted:
const db = await Dexie.import(blob, {
    progressCallback: progressCallback
});

After creating an Allocation sampling profile, it became apparent that there was a significantly larger portion of memory being used in comparison to the file size. It rose to ~2GB before crashing.
Here are the most significant profiler results:

I'll attempt to fix this and submit a PR.

I can confirm this issue. For me, I get the out of memory crash for a database that is about 800 MB in size.

I've fixed the issue and submitted a PR (#874).
In short: the json streaming parser was not making objects available (and thus not allowing them to be released from memory) until they're completely finished parsing, which made it impossible to read partial objects without parsing the entire json. This was causing loadUntilWeGotEnoughData() to parse the entire json, where it really only needed a tiny amount of information from the beginning.

Fixed in PR #872, released in<EMAIL_ADDRESS>


Bug: Infinite Loop

After uses gws command in virtual terminal, starts an infinite loop with this following code:
sys_connect: AF_GRAMADO ok
sys_connect: sProcess fail
sc2: [19] write

Ok. We have two programs in this test. The window server, called gwssrv and the default client, called gws. :)


Serve schemas

closes #82
to bootstrap data I needed to define the env variable before bootstrap: https://github.com/scitran/core/commit/ce1ba92e0923ef8ee3c0e584c83700482999f1e9
I am not sure if that is the right way. @ryansanford @gsfr any suggestions?

The diff makes sense so far... but I want to make sure I understand correctly.
Gunnar mentioned to us earlier about how this will start to cause loopback resolution, and I see that in the code. Can you walk us through the path you travelled? I'm not entirely opposed to this landing, but when it does I'd like to understand what we can do in the future to avoid loopback calls.

@rentzso
manually adding ENV vars should not be required when going through run-dev.sh.
Change to docker/uwsgi-entrypoint.sh should be backed out. See https://github.com/scitran/core/blob/master/docker/README.md for example on how to bootstrap in a raw scitran/core env and not the flywheel dev environment. If there's a new ENV var that is required, those instructions should be updated, and the ENV var should be added to sample.config

@ryansanford
Thanks for the info.
What changes are needed in fly/fly after adding the required ENV in the scitran/core?
@kofalt
Something that I have considered, but excluded quickly is to continue to load the schemas from the filesystem while serving them to external clients from the API.
This is problematic as it may create inconsistencies between two schemas. Also I have noticed some weirdness in json schemas resolution from the filesystem.
A possible alternative to avoid the loopback would be to statically serve schemas, maybe on a different endpoint. The downside of this is that we will have always to hardcode all the schemas. Serving them in the API would allow us in the future to create/modify them on the fly. Not sure if this will be needed but it is nice to have.

@rentzso
In fly/fly, add it to docker/dev.env. All vars there get consumed by scitran/core containers.

@ryansanford I have added the env variable in fly/fly and update the docs/sample.config
@kofalt Is it ok for you to merge or do you need more time to review the code?

@ryansanford I have tried to build the docker containers as in https://github.com/scitran/core/blob/master/docker/README.md
everything was fine except bootstrap-data.sh
I have got this error:
Collecting git+https://github.com/scitran/reaper.git@30215c66a33b18685e1608dbe952e78c370d8765
  Cloning https://github.com/scitran/reaper.git (to 30215c66a33b18685e1608dbe952e78c370d8765) to /tmp/pip-xfTF7O-build
fatal: unable to look up current user in the passwd file: no such user

It seems not possible to clone the reaper repo. Can you check it? Thanks!

@rentzso
I just merged a correction to the bootstrap example in docker/README.md, which should resolve your error. Please try again using this updated command.

@rentzso Are the only differences you've observed regarding $ref?
Can you contrast for me a code snippet that loads a JSON schema from a URL, versus a snippet that loads from a file? I'd like to play with this to see if I can make headway.

You can find one of the "weirdness" that I was forced to use for $ref in input/download.json:
you will see two references to subschemas defined in the same document, the only way to make them work was for the ones at line 51-52 to reference the same download.json schema and at line 8-9 NOT to reference it.
After serving the schemas this should become more coherent.

@ryansanford I have tested with the new scripts and also data creation works.
@kofalt let me know if you need to give more thought about the issue or if I can merge.
Thanks!

@rentzso: Let's go ahead & merge this for now.


Any way to ignore unknown commands?

Click v7.0
I'm trying to write a wrapper for git for my own purposes. I'm new to using click, so correct me if I'm doing this the wrong way.
My goal is to have special sub commands do specific functionalities, and if the argument is not a special sub command, default to running git with the given arguments.
I have something like this
import click

@click.group(
    invoke_without_command=True,
    context_settings = {'ignore_unknown_options': True, 'allow_extra_args': True}
)
@click.pass_context
def cli(ctx):
    if not ctx.invoked_subcommand:
        # run git with given arguments instead
        ...

@cli.command()
def special_command():
    # do some special functionality

What I want is when I run the cli with arguments/commands that isn't the special command, to run git with those arguments. But the cli gives me a Error: No such command ... if I give it any command other than the special command. Is this possible? Or am I doing this the wrong way?
I also want to add that I'm running this through setuptools and the entry point is cli
Example: My git wrapper is called cli
cli special_command will run the special command
cli status will run git status

Maybe you could so something with nargs like this?
Anything that needs to be in your special command comes first and gets captured by click, everything else is sent to git.
import click
import subprocess

@click.group(
    invoke_without_command=True,
    context_settings = {'ignore_unknown_options': True, 'allow_extra_args': True}
)
@click.pass_context
@click.option('-spec', '--special', is_flag=True, help='Enables bonus feature')
@click.argument('git', nargs=-1)
def cli(ctx, special, git):
    print(special)
    if special:
        # do some processing
        print("doing some processing")
    subprocess.run(git)

if __name__ == "__main__":
    cli()

`
Closing this because it isn't directly related to click development. Please use Stack Overflow for questions about your own code. This tracker is for issues related to the project itself. Be sure to include a [minimal, complete, and verifiable example](https://stackoverflow.com/help/mcve).


Draft: PoC: Vulkan Framework v2.0

Description
Earlier I merged part of the Vulkan Builders by mistake and then quickly reverted this change. This meant that I dropped vulkan builders for a while. I finally had a chance to revisit this concept. This time around the vulkan builder is complete and the test case for the builder passes on my local machine.
The Context
The context is the heart of the framework. v1.0 represented this using the Device class. This time around, I hope, to build functionality on top of the context by composing classes and interfaces. Keeping the context as simple as possible will allow the development of single responsibility components.
Testing
The most important thing going forward with framework v2.0 is the testability of the project. This is key to longevity and stability.  To test Vulkan components I added the vkb__register_gpu_tests cmake function. This is only used when the VKB_BUILD_GPU_TESTS  is enabled. A check is carried out internally which decided wether to register the test or not.
option(VKB_BUILD_GPU_TESTS "Enable generation and building of GPU tests." OFF)

vkb__register_gpu_tests(
  NAME "vulkan_context_test"
  LIBS
    vkb__vulkan
  SRC
    gpu/context.test.cpp
)

I then wrote the following test
TEST_CASE("create basic instance", "[vulkan][context]")
{
	// failed to initialise meta loader
	REQUIRE((vulkan::init_meta_loader() == VK_SUCCESS));

	vulkan::ContextBuilder builder;

	// clang-format off
  builder
    .configure_instance()
      .application_info(vulkan::default_application_info(VK_API_VERSION_1_2))
      .enable_validation_layers()
      .enable_debug_logger()
      .done()
    .select_gpu()
      .score_device(
        vulkan::scores::combined_scoring({
          vulkan::scores::device_preference({VK_PHYSICAL_DEVICE_TYPE_DISCRETE_GPU, VK_PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU, VK_PHYSICAL_DEVICE_TYPE_VIRTUAL_GPU}),
          vulkan::scores::has_queue(VK_QUEUE_GRAPHICS_BIT, 1),
        })
      )
      .done()
    .configure_device()
      .enable_queue(VK_QUEUE_GRAPHICS_BIT, 1)
      .done();
	// clang-format on

	vulkan::ContextPtr context = builder.build();
	REQUIRE(context != nullptr);
	REQUIRE(context->instance != VK_NULL_HANDLE);
	REQUIRE(context->gpu != VK_NULL_HANDLE);
	REQUIRE(context->device != VK_NULL_HANDLE);
}

Explanation
There are three stages to building the context. Configure the instance, Select a GPU and finally create a device. The context does not have any knowledge of Vulkan extensions and so VkSurfaceKHR usage is injected using builder functions. The only exception to this rule is queue selection in the device build stage which has a very specific queue allocation process.
In instance creation we are able to enable specific extensions or do more complex tasks like enable debug loggers and validation layers. Its worth noting that these complex tasks are not hacked into the builder but simply use its framework to implement more complex logic.
When we call vkCreateInstance, a set of extensions and layers are categorized as optional or required. When registering an extension you may also add a callback event that states what to do if the extension is enabled. (I might extend this with an enum to customize the scenario further).
GPU selection is score-based and scoring functions can be made for any use-case. At the moment I have implemented common scoring functions like device preference, queue requirements and presentation support for a VkSurfaceKHR. Extracting this into functions and away from the builder massively simplifies the interface and allows samples to make their own custom scoring mechanisms.
Device creation is mainly about collating and enabling the correct queues. Queues are selected by first family and more families are utilised if the required queue count is not met. If presentation support is required then we prioritise that queue family first. Later I may extend this to have more fine-grain control over queue selection for samples that want to demonstrate specific scenarios (i guess another lambda could work XD use_queue_selection_policy(func))
Implementation
InstanceBuilder and DeviceBuilder's core functionality comes from ExtensionBuilder. ExtensionBuilder is designed in such a way that it can only be used for these two classes. I purposely did this to enforce a single concern onto the context builder. All build functions between the builders are made private and are only accessibly by the main context builder.
pNextChain acts as a container to construct Vulkan pNextChains from arbitrary structs. Vulkan sType are deduced from the struct type simplifying the amount of information needed to interact with the container.

I've been wanting to review this for some time now, but tbh I'm having a hard time. Does this mean all samples will use the builder pattern as showed above for the tests? Or what changes (in general) will this introduce in regards to writing samples?
Can we spare some time during our next call to take a look at this?

Hey, @SaschaWillems,
Framework v2.0 has lots of PRs open and no real way of deciphering the order in which they need to merge or be reviewed. I can only dedicate a few hours to the improvements at a time so I appreciate that the bigger picture here is getting lost in the chaos - I will definitely work on improving this approach.
This PR is being actively developed and will be a bucket for the Vulkan-specific stuff. I will discuss this in the next WG call. The aim of this PR is to replace the Core folder with the minimum amount of wrappers and classes.
This PR also contains some things from other PRs such as the image asset stuff. This is so that I can add some of the core systems which convert sg::Mesh's to vulkan::Mesh's.
#493 demonstrates how the platform and strict sample lifecycle has been removed and replaced with sample_main
#493 also demonstrates how samples can be individually compiled and dynamically loaded which solves the slow build and interdependent compile targets issue which has long plagued samples
#598 demonstrates how we can load images from files and decode/encode them between formats without depending on Vulkan or other parts of the framework
#619 builds on top of #598 and demonstrates how we can simplify the GLTF loader in the process (the loader is something I have definitely over-engineered in the past!)
When we combine all these a sample's footprint may look like this in the future:
EXPORT_CLIB int sample_main(PlatformContext *platform_context)
{
  // configure context for sample
	vulkan::ContextBuilder builder;

	builder
	    .configure_instance()
	    .application_info(vulkan::default_application_info(VK_API_VERSION_1_2))
	    .enable_validation_layers()
	    .enable_debug_logger()
	    .done();

	builder
	    .select_gpu()
	    .score_device(
	        vulkan::scores::combined_scoring({
	            vulkan::scores::device_preference({VK_PHYSICAL_DEVICE_TYPE_DISCRETE_GPU, VK_PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU, VK_PHYSICAL_DEVICE_TYPE_VIRTUAL_GPU}),
	            vulkan::scores::has_queue(VK_QUEUE_GRAPHICS_BIT, 1),
	        }))
	    .done();

	builder
	    .configure_device()
	    .enable_queue(VK_QUEUE_GRAPHICS_BIT, 1)
	    .done();

	vulkan::ContextPtr context = builder.build();

  // get filesystem
  // the naming of this function is poor and we could always allocate the filesystem in main... for now this is the pattern
  auto fs = vfs::_default(platform_context);
  // possibly this in the future?
  // auto fs = platform_context->fs();

  // load scene
  auto registry = sg::create_registry();
  sg::NodePtr model_root;

  GltfLoader loader{registry};
  loader.load_from_file("Model Name", fs, "assets/models/scene.gltf", &model_root);

  // creating vulkan meshes
  // a system that finds all meshes in the scene and allocates a vulkan::Mesh for them
  vulkan::systems::allocate_vulkan_meshes(context, registry);

  // event loop
  EventPipeline pipeline;
  // ... define sample event pipeline and add systems (we can also add vulkan systems)
  // ... the default samples may decide to use default pipeline configurations
}

This is for new samples of course. For old samples, I will make a legacy shim over the older interfaces.
Let's discuss this in the next call. I will also find a better approach to the different components and try and find more time to complete my PRs. for instance. I missed the comments in #493 which I will resolve this evening so that it can be reviewed

Awesome. Thanks for the detailed explanation 👍

@SaschaWillems I spent some time throwing this together https://github.com/KhronosGroup/Vulkan-Samples/wiki/Framework-v2.0
This should help provide some context. I will add more detail to it and a section on a basic sample (or possible an implementation of a basic sample)

Any ETA on when we can merge this? I'm trying to change some things in the current framework, and ever tiny change results in a full rebuild that takes ages. On my system every change to the framework takes between 15 to 30 minutes to rebuild :(
This makes it tedious and frustrating to work on the current framework.

https://github.com/KhronosGroup/Vulkan-Samples/pull/404 <-- @SaschaWillems review that when you get a chance.  It might help :)

I tried to build this solution, but got 53 failing projects (Win10, VS2022).
The first failing one is the shader_compiler, complaining about an unresolved external:
45>------ Build started: Project: shader_compiler, Configuration: Debug x64 ------
45>main.cpp
45>MSVCRTD.lib(exe_main.obj) : error LNK2019: unresolved external symbol main referenced in function "int __cdecl invoke_main(void)" (?invoke_main@@YAHXZ)
45>C:\git\Vulkan-Samples\build\VS2022_64\bin\Debug\shader_compiler.exe : fatal error LNK1120: 1 unresolved externals
45>Done building project "shader_compiler.vcxproj" -- FAILED.

How would I resolve that?


Check in lockfiles

Creating this draft as a test to see how much faster CI would be with lock files checked in
answer: much much faster. CI is faster using lockfiles than when there is a cache hit
I am creating this PR as a point of discussion. Looking for input from @open-telemetry/javascript-approvers. Do you think we should check in lockfiles?
There is a good summary of the tradeoffs here: https://classic.yarnpkg.com/blog/2016/11/24/lockfiles-for-all/. It is about yarn, but applies to npm as well.
Discussion here: https://github.com/open-telemetry/opentelemetry-js/discussions/1830

Totally agree, i've always used it in my projects without issue. I generally take some time every once in a while to update deps or i just run dependabot

Totally agree, i've always used it in my projects without issue. I generally take some time every once in a while to update deps or i just run dependabot

Totally agree, i've always used it in my projects without issue. I generally take some time every once in a while to update deps or i just run dependabot

I think we should keep discussion in the linked discussion. That way comments on the PR are only related to the technical issues with the PR.

Totally agree, i've always used it in my projects without issue. I generally take some time every once in a while to update deps or i just run dependabot

I think we should keep discussion in the linked discussion. That way comments on the PR are only related to the technical issues with the PR.

I think we should proceed with this further then.

I think we should proceed with this further then.

ok

ok

Seems like all maintainers are on board. Marked as ready for reviews and will leave for a day or so to ensure nobody has any complaints before merging.

Seems like all maintainers are on board. Marked as ready for reviews and will leave for a day or so to ensure nobody has any complaints before merging.

Are the PRs from renovate-bot taking care of updating lock files also?
I miss also some hints for devs what they shall do if they add/remove/modify a dependency. I use lock files in some projects where I usually deleted lock file + node_modules folder and then run npm install. But here we have lerna so not sure if this works as intended regarding the linked lerna subprojects.
Finally I think https://github.com/open-telemetry/opentelemetry-js/blob/main/CONTRIBUTING.md#install-dependencies should be changed to npm ci.

Are the PRs from renovate-bot taking care of updating lock files also?
I miss also some hints for devs what they shall do if they add/remove/modify a dependency. I use lock files in some projects where I usually deleted lock file + node_modules folder and then run npm install. But here we have lerna so not sure if this works as intended regarding the linked lerna subprojects.
Finally I think https://github.com/open-telemetry/opentelemetry-js/blob/main/CONTRIBUTING.md#install-dependencies should be changed to npm ci.

Are the PRs from renovate-bot taking care of updating lock files also?

Yes. Per renovate docs https://docs.renovatebot.com/faq/

I miss also some hints for devs what they shall do if they add/remove/modify a dependency. I use lock files in some projects where I usually deleted lock file + node_modules folder and then run npm install. But here we have lerna so not sure if this works as intended regarding the linked lerna subprojects.

It works the same way, but the lock file is regenerated and possibly different. Dependencies cross-linked by lerna are omitted entirely from the package-lock. When lerna bootstrap is run, the package.json files are copied to a backup location and temporary ones are put in place which only contain external dependencies. Then npm install is run. Then symlinks are created between lerna dependencies. Then the original package.json files are moved back.

Finally I think https://github.com/open-telemetry/opentelemetry-js/blob/main/CONTRIBUTING.md#install-dependencies should be changed to npm ci.

It would be lerna bootstrap --ci, but only if you wanted to get the locked version of the dependencies for some reason. A bootstrap will work just as well locally and the lock file will be updated if it needs to be.

Are the PRs from renovate-bot taking care of updating lock files also?

Yes. Per renovate docs https://docs.renovatebot.com/faq/

I miss also some hints for devs what they shall do if they add/remove/modify a dependency. I use lock files in some projects where I usually deleted lock file + node_modules folder and then run npm install. But here we have lerna so not sure if this works as intended regarding the linked lerna subprojects.

It works the same way, but the lock file is regenerated and possibly different. Dependencies cross-linked by lerna are omitted entirely from the package-lock. When lerna bootstrap is run, the package.json files are copied to a backup location and temporary ones are put in place which only contain external dependencies. Then npm install is run. Then symlinks are created between lerna dependencies. Then the original package.json files are moved back.

Finally I think https://github.com/open-telemetry/opentelemetry-js/blob/main/CONTRIBUTING.md#install-dependencies should be changed to npm ci.

It would be lerna bootstrap --ci, but only if you wanted to get the locked version of the dependencies for some reason. A bootstrap will work just as well locally and the lock file will be updated if it needs to be.


fix(docs): Small typoes in the Supported installation page

-> https://docs.docker.com/installation/

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "master" git@github.com:zmarouf/docker.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f

Ammending updates the existing PR. You DO NOT need to open a new one.

I was already on it. Thanks Gordon! :)

LGTM

cherry-picked


New API timing doesn't work with fast applications

What issue do you have? Please be as thorough and explicit as possible.
The new 7.5 second timing results in some events not firing if a scene is turned on and then off quickly.
For example, I have two lightstrips in a closet which turn on as part of a scene when the door opens. I will often close the door before the second light has turned on, and then the turn off events won't fire correctly and the lights remain on.
I wasn't having this issue prior to the timing change in the most recent version.
Details of your setup.

Do you use (1) Homebridge (+ config-ui), (2) Homebridge (CLI) or (3) HOOBS?

Homebridge + configui

Which version of Homebridge/HOOBS do you have?

1.3.0 beta 53

Which platform do you run Homebridge/HOOBS on (e.g. Raspberry Pi/Windows/HOOBS Box)? Please also mention your version of Node.js/NPM if known.

Raspi 4
Node.js Version v14.15.4
NPM version v6.14.10

Which version of this plugin (homebridge-govee) do you have? Has the issue started since upgrading from a previous version?

2.9.0
Yes

Which Govee devices do you have that are causing issues? Please include product models if applicable.

Three H6141 light strips.
Please paste any relevant logs below.

[07/02/2021, 19:46:07] [deCONz] Wardrobe Right Door: set homekit contact from 1 to 0
[07/02/2021, 19:46:07] [Govee] [Wardrobe Right Top] current state [off].
[07/02/2021, 19:46:14] [Govee] [Wardrobe Right Bottom] current state [off].
[07/02/2021, 20:04:19] [deCONz] Wardrobe Right Door: set homekit contact from 0 to 1
[07/02/2021, 20:04:20] [Govee] [Wardrobe Right Top] current state [on].
[07/02/2021, 20:04:27] [Govee] [Wardrobe Right Bottom] current state [on].
[07/02/2021, 20:04:38] [Govee] [Wardrobe Right Top] current CCT via adaptive lighting [5376K].
[07/02/2021, 20:04:40] [deCONz] Wardrobe Right Door: set homekit contact from 1 to 0
[07/02/2021, 20:04:45] [Govee] [Wardrobe Right Bottom] current CCT via adaptive lighting [5376K].
[07/02/2021, 20:04:53] [Govee] [Wardrobe Right Top] current state [off].
[07/02/2021, 20:05:00] [Govee] [Wardrobe Right Bottom] current state [off].

In the above logs, the door is opened, and the lights are waiting to come on. By time they come on, the door has closed. When the door is opened again, the lights turn off.

Hi @radiocolin
I appreciate the 7.5 second delay between requests seems a very long time, but many users were finding that updates in quick succession were sometimes being ignored by Govee even though the response from their server was a success. I have been testing multiple updates in quick succession with my own lights (through scenes) and I found anything less than this 7.5 value was not always reliable.
Moving forward...
If you are happy to install the latest beta of the plugin I have just pushed (v2.10.0-0), I have added a new config option to change this 7500ms delay (you'll find it in the Optional Settings part of the plugin settings screen). Note that the value has a minimum of 500 and is in milliseconds.
This gives you the flexibility to reduce it down to a number that would work for you with your automation, but, I can't guarantee that the lights will always behave if the updates are sent in quick succession.
Hope this helps!

That will do nicely, can you tell me what the old value was? That seemed to work well for me.

Before was a slightly different method,

no delay for on/off
1 second delay for colour updates
2 second delay for brightness.

If you found it worked well before then you could try the minimum value of 500ms and see how you get on .

Just released v2.10.0 which, in some ways, reverts the changes I made in v2.9.0.
The default value for the new controlInterval setting is now 500ms, of course with the option of increasing it if anyone experiences connectivity issues.
Trying to make this plugin work well for everyone one way or another!


"Out of memory" error. Can anything be done?

rs train and rs predict fail with CUDA error: out of memory.
I have a nVidia GeForce GT 710 which is admittedly pretty low end. The specs say it has 2GB of memory, but nvidia-smi only says 978MB (~1 "giga" byte) 🤔.
When I set batch_size = 1, and images_size = 256 (and in rs download I download the 256x256 (ie. no @2x suffix)), I still get the same error. It takes a few seconds before I get a python error of RuntimeError: CUDA error: out of memory``, rather than ~1 second, so it feels like it's lasting more before OOMing. But it still fails. This happens on rs trainandrs predict`.
Is there any way to make robosat use less memory so that I can at least run this on my GPU rather than my CPU? Or must I just accept that my hardware isn't good enough? I know very little about graphics cards, cuda, or torch, or computer vision stuff.

I run it on my CPU with by installing torch with pip install --upgrade https://download.pytorch.org/whl/cpu/torch-0.4.0-cp36-cp36m-linux_x86_64.whl, and setting cuda = false in model.toml. It works, and for tiny areas, I can get results in a few hours. But I'd love if it could be faster.

$ nvidia-smi
Wed Oct 10 18:21:07 2018
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 390.87                 Driver Version: 390.87                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GT 710      Off  | 00000000:01:00.0 N/A |                  N/A |
| 40%   38C    P8    N/A /  N/A |    177MiB /   978MiB |     N/A      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0                    Not Supported                                       |
+-----------------------------------------------------------------------------+

and
$ nvcc
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2017 NVIDIA Corporation
Built on Fri_Nov__3_21:07:56_CDT_2017
Cuda compilation tools, release 9.1, V9.1.85

My OS is Ubuntu 18.04.

Looks like you only have 978 MiB (~1GB) CUDA memory available on your GPU. That's quite low indeed.
Here's what you can try:

The models we use in rs train should be able to go as low as 32x32 pixels (the size needs to be divisible by 32 for the repeated factor of two downsampling and then upsampling to work out). You might run into border problems with images as small as 32x32 but maybe try 256, or 128 first. Keep the @2 suffix for high-dpi images, though. You want to change image size, not quality. Also you don't have to re-download the images. Instead what you can do is use mercantile's tile.children function and PIL's Image.crop function and then cut a downloaded 512x512 tile into its 4 256x256 tiles. Repeat to get 128x128. Repeat to get 64x64.
Use a smaller model; here and here we construct the model with its default num_filters=32 (see here). See if decreasing this for the decoder helps or if the peak memory usage is really due to the resnet encoder.
Try the FPN pull request (https://github.com/mapbox/robosat/pull/75) it should be lighter and give you the same if not better results.
Use a smaller pre-trained model encoder. Here we are using a resnet50 and here we create channels which are in sync with the resnet50. You can try a resnet34 or even resnet18 and then instead of 2048, 1024, 512, 256 channels use 512, 256, 128, 64. You can do the same trick in the FPN branch which combines the idea to use the lighter FPN and the idea here to use a smaller encoder.
Always run watch nvidia-smi (e.g. in a separate tmux pane) to check the GPU utilization and if you can increase your batch size further.

If it turns out it's still not fitting into your GPU memory or you learn that you will have to trade off too much model performance to get it to work (which both might be the case since we optimize robosat for server usage and not mobile devices) I strongly recommend getting either an AWS p2 instance or a similar GPU e.g. at paperspace.com.
You should be able to run and prepare every single step in the pipeline except rs train reasonably fast on your laptop and if you then need to spend 10-20 bucks to train on a cloud GPU for a day or two I think that's a fair investment we can expect from users.

Thanks for your help, I'll look into those things. Although I previously mentioned using a laptop, I've since bought a desktop, looks like I didn't spend enough! 😛
It looks like AWS p2 start at 61GB of GPU ram, and nVidida GeForce GTX 1080 TI have 11GB. Is that the sort of "minimum memory" you're designing robosat for?

Th p2s K80 and the GTX 1080 TI both have around 12 GB of GPU memory. That's what we are using for prototyping, experimentation, training. But as I outlined above there's plenty of room to make it work on smaller GPUs and we only fully utilize the 12 GB right now with large batch sizes and 512x512 images.
Keep me posted how it works out for you!

Thanks for your reply. I'm looking into changing the num_filters & resnet50, nothing to report yet. The FPN PR doesn't apply to master branch, so I can't test that.

The models we use in rs train should be able to go as low as 32x32 pixels ...  cut a downloaded 512x512 tile into its 4 256x256 tiles. Repeat to get 128x128. Repeat to get 64x64.

Can you explain how to lay out this data? Should I turn one 512x512 z18 tile into 4 256x256 z19 tiles?, then into 8 127x127 zx20 tiles? (I can do this with a bash script + imagemagick) Then work on this new z20 tiles? Or do I split a 512x512 z18 tiles into 8 tiles and leave them in the same directory? But you mention python code (I've used python professionally for years), what would I change? Would that involve changing BufferedSlippyMapDirectory to somehow think there were more (but smaller) tiles? 😕

For training the tile's z doesn't matter so you can take your first approach.

That's great. Do I need to change any settings? image_size is in the model toml file, I presume I should change that to whatever size I use?

Yeah the image size should be e.g. 256 if your slippy map tile images are of size 256x256.
In addition I just rebased the FPN branch https://github.com/mapbox/robosat/pull/75 feel free to give it a go.

I had "success" with splitting the tiles up. With a batch=1, I was able to run rs train on my CPU and the speed was about 6sec per 512×512 tile. By splitting it 4 times, I got 32×32 images, and was just about able to squeeze that into 1GB of memory. It was ~930MB IIRC (batch=1 ofc). I got 0.6s per tile then. 10 tiles faster! But there are 64 times more images... So ~6½ times slower! 🤦
I can run rs predict using the graphics card, and that's faster than the CPU, and only takes ~300MB of memory on 512×512 images, and does ~20 tiles per second. So I can train on CPU, but predict on GPU.
I have noticed that I have a laptop with a 2GB nvidia card. I'll try that over the karlsruhe hack weekend.

Progress: I'm running it on a machine (laptop) with 2004MiB of GPU memory. With batch=1 and splitting tiles once into 256×256, and it works! nvidia-smi tells me it's using 1329MiB of memory. rs train is running at 1.32 batch/sec (i.e 0.75 sec per batch). That's still about 10× faster than the CPU approach.


6.5.0 on ANL BGQ systems (Vesta, Cetus/Mira)

Original issue: https://charm.cs.illinois.edu/redmine/issues/94

No body.

Original date: 2013-03-28 03:24:21

The known issues blocking some of these builds (#119, #131) have been dealt with. We want to get everything from the 6.5 release wrapped up so we can announce the installed paths for users. If I don't see an issue complete and a path recorded tomorrow, I'm going to start nagging.

Original date: 2013-04-02 22:02:14

Builds to be found in ~phil/charm-6.5

Original date: 2013-04-03 19:27:14

#104 prevents us building the 5 xlc * (smp, async-smp) * debug targets, but the the remaining 43/48 targets are built. I've confirmed that each of them can successfully link a complete binary, viz. megatest.
For reference, the set of builds occupies 6.6G in my home directory. The megatest binaries added another 1.6G while I had them sitting there.

Original date: 2013-04-04 14:52:43

Test update

Original date: 2013-04-04 14:53:17

Test update again

Original date: 2013-04-04 15:07:10

test update 3

Original date: 2013-04-10 18:46:16

Confirmed access to these locations.


Allows use the xkb options that for example allows you to specify a key combination to change the layouts.

Example of xrdp_keyboard.ini for russian keyboard (Please add this to wiki)
[default_rdp_layouts]
rdp_layout_us=0x00000409
rdp_layout_de=0x00000407
rdp_layout_fr=0x0000040C
rdp_layout_it=0x00000410
rdp_layout_jp=0x00000411
rdp_layout_jp2=0xe0010411
rdp_layout_jp3=0xe0200411
rdp_layout_jp4=0xe0210411
rdp_layout_ru=0x00000419
rdp_layout_se=0x0000041D
rdp_layout_pt=0x00000816
rdp_layout_br=0x00000416
rdp_layout_pl=0x00000415
[rdp_keyboard_ru]
keyboard_type=4
keyboard_subtype=1
model=pc105
options=grp:alt_shift_toggle
rdp_layouts=default_rdp_layouts
layouts_map=layouts_map_ru
[layouts_map_ru]
rdp_layout_us=us,ru
rdp_layout_ru=us,ru

Please add this to wiki

You also can do it.  If your configuration is common among Russian users, consider changing xrdp_keyboard.ini in the repository.

Merge as Jay +1-ed to #363.  The difference between #363 and #364 is the branch.

don't work in debian 8.5, build with X11rdp-o-matic, added entries from above, options not passed:
lc@debdev:~$ setxkbmap -print -verbose
Trying to build keymap using the following components:
keycodes:   xfree86+aliases(qwerty)
types:      complete
compat:     complete
symbols:    pc+us+inet(pc105)+ru:2
geometry:   pc(pc105)
xkb_keymap {
xkb_keycodes  { include "xfree86+aliases(qwerty)"   };
xkb_types     { include "complete"  };
xkb_compat    { include "complete"  };
xkb_symbols   { include "pc+us+inet(pc105)+ru:2"    };
xkb_geometry  { include "pc(pc105)" };
};

Some more works needed to apply keyboard options.  Please wait for a while.

@coolemza I did it #417. Can you try?

sure, what is the way to build this pull request with xrdp-o-matic? i tried "xrdp-o-matic --branch devel --justdoit" then applied pull request to xrdp_0.9.0+devel source dir, than again "xrdp-o-matic --branch devel --justdoit" but got errors like:
dpkg-source: error: cannot represent change to xorg/X11R7.6/build_dir/xorg-server-1.9.3/xfixes/.libs/libxfixes.a: binary file contents changed

Edit X11rdp-o-matic.sh, s/neutrinolabs/metalefty/g.
And then do with --branch apply-keyboard-options.

@coolemza The fix has been merged into devel branch.


pkg/report: improve report titles

Codecov Report

Merging #1526 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1526   +/-   ##
=======================================
  Coverage   56.08%   56.08%
=======================================
  Files         142      142
  Lines       26442    26442
=======================================
  Hits        14829    14829
  Misses      10904    10904
  Partials      709      709

Impacted Files
Coverage Δ

pkg/report/linux.go
84.61% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d29b9e8...e865d68. Read the comment docs.

LGTM


Split code in more packages

+namespaces

Packages:
Core (logging interface + config)
Internal (internal logger, helpers)
Xmlconfig
Network (targets, LayoutRenderers)
Classis asp
Webservices (WCF, rest)
Database

Silverlight renderers
Abstractions (interfaces)

Should be resolved with the following issues:
#2582 - Move WCF targets to NLog.WCF package
#2676 - Move DatabaseTarget to NLog.DatabaseTarget package
#2680 - Move NetworkTarget to NLog.NetworkTarget package
#2786 - Move MailTarget to NLog.SmtpMailTarget package
#2787 - Move System.Configuration to NLog.AppSettings package
See also #268

I think this should be 5.0, as 5.0 is later then planned IMO

extensions packages should target NLog.Core then (or maybe NLog.Abstractions)

#2582 - Move WCF targets to NLog.WCF package
#2676 - Move DatabaseTarget to NLog.DatabaseTarget package
#2680 - Move NetworkTarget to NLog.NetworkTarget package
#2786 - Move MailTarget to NLog.MailTarget package

Proposal: Nlog.Database, NLog.Network (including layout renderers), NLog.Mail

Also good.

@snakefoot what do you think the benefits should be of this change?
At first it was improving dependencies, but that isn't an issue with .NET standard 2 anymore (only one dependency). (of course we more platforms)
IMO the big improvement are

smaller code base - so easier to maintain
abstractions in separate package for custom implementations
smaller build times
maybe also:  changing a target won't break the core

But with this in mind, maybe another split would be better? (eg. no layout renderers in the core package).
What do you think?

My focus is on limited NetCore platforms and compilation with NetCore Native.
Reducing the amount of direct references from NLog.dll to system-libraries. Because with NetCore Native it will try to shake everything away not being used.
Must admit one of the strengths of NLog.dll is actually the release of unified NLog.dll where everything is in "sync". No need to chase around to find the perfect console-logger or file-logger.
But the exotic targets like WCF, Mail, Database, NetworkTarget, WebServiceTarget would be good to get out.

Sub issue: https://github.com/NLog/NLog/issues/3366
Useful tool probably: https://github.com/CurseStaff/GitSplit

started on this

todo

[ ] split PlatformDetector into file / non-file

split PlatformDetector into file / non-file

Maybe merge it into an environment-interface like IAppDomain

Small interfaces are better to handle, so move it to a separate class

issues:

[ ] internallogger (core) uses System.Configuration.ConfigurationManager
[ ] lower: ConditionParser (core) hard references all expressions

Think that configuration from appsettings should be removed.
Still think it is a good idea to keep NLog a little fat. Think that the support for loading xml config should stay in the core.
Since the InternalLogger supports console + file then it might also make sense to have simple console and file target in NLog. But without use of mutex and FileWatcher (that could be concurrent-file-target)
Sent from my Sony Xperia
---- Julian Verdurmen wrote ----
issues:

internallogger (core) uses System.Configuration.ConfigurationManager
lower: ConditionParser (core) hard references all expressions

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/NLog/NLog/issues/1148?email_source=notifications&email_token=ACXZ7HHSK3FDUDGMYFXATU3PYQ4IRA5CNFSM4BYHUQJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWX6PCY#issuecomment-498067339, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACXZ7HBVQYMJKKRKJ5LP2SDPYQ4IRANCNFSM4BYHUQJQ.

Still think it is a good idea to keep NLog a little fat. Think that the support for loading xml config should stay in the core.

I try to keep it as slim as possible. I prefer to decouple it.

Since the InternalLogger supports console + file then it might also make sense to have simple console and file target in NLog. But without use of mutex and FileWatcher (that could be concurrent-file-target)

Good one, but I prefer to have a core package that won't get changed much (compared to these targets)

Logfactory references LoggingConfigurationFileLoader (2x)

@snakefoot any good idea how to fix this? Too bad the Logfactory is newed statically

Maybe the LogFactory could lookup a config-file-loader-interface from the IServiceResolver?
Sent from my Sony Xperia
---- Julian Verdurmen wrote ----

Logfactory references LoggingConfigurationFileLoader (2x)

@snakefoothttps://github.com/snakefoot any good idea how to fix this? Too bad the Logfactory is newed statically
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/NLog/NLog/issues/1148?email_source=notifications&email_token=ACXZ7HBJHZ6C5DYY5EANLKDPY3RKDA5CNFSM4BYHUQJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODW6AG3A#issuecomment-498860908, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACXZ7HEKDAM35LIBCF364JDPY3RKDANCNFSM4BYHUQJQ.

Also think that these "packages" should remain one package called NLog and not having a meta package:

NLog.Abstractionshttp://NLog.Abstractions (public interfaces)

NLog.Corehttp://NLog.Core (logging + config + layouts +internal log + filters) -- default targets? (Console, chainsaw, memory etc)

NLog.NLog.Fluenthttp://NLog.NLog.Fluent? (Should be moved out of its namespace)

-NLog.XmlConfigFilehttp://NLog.XmlConfigFile
Sent from my Sony Xperia
---- Julian Verdurmen wrote ----

Logfactory references LoggingConfigurationFileLoader (2x)

@snakefoothttps://github.com/snakefoot any good idea how to fix this? Too bad the Logfactory is newed statically
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/NLog/NLog/issues/1148?email_source=notifications&email_token=ACXZ7HBJHZ6C5DYY5EANLKDPY3RKDA5CNFSM4BYHUQJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODW6AG3A#issuecomment-498860908, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACXZ7HEKDAM35LIBCF364JDPY3RKDANCNFSM4BYHUQJQ.

Logfactory references LoggingConfigurationFileLoader (2x). any good idea how to fix this?

I would probably keep XmlLoggingConfiguration in the core-package with the ability load from file. Then create a new package called NLog.XmlFileLoggingConfiguration that adds support for include-files with filewatcher and autoreload.
Just like NLog.Extension.Logging is has NLogLoggingConfiguration that provides autoreload-support without depending on NLog. Then XmlFileLoggingConfiguration should also handle autoreload without depending on NLog-core.

Another example of how NLog-package should reduce its references. Allowing it to be used in Blazor-ClientSide-Apps:
https://stackoverflow.com/questions/57151534/blazor-client-side-app-does-not-build-when-project-has-nlog-dependency

Is this still intended for NLog 5.0 ?

Yes, but I started too big I guess.

Yes, but I started too big I guess.

Meaning that we should not do it in NLog 5.0. Or that it takes longer time than expected?

I like this for NLog 5. But I have limited time these days so many we could start with doing the minimal.
https://github.com/NLog/NLog/issues/1148#issuecomment-216004968 is too large for me at least.

And the minimal is only splitting out NLog.Wcf ? Or does it include other low-hanging fruits?

The absolute minimal is that the dependencies are minimal for the "core" package.

Know that each package created means one more package to maintain and publish on each release)

Well a large package is also hard to maintain. I like to improve a lot, but the changes are too large (too many files, to long build times)

Created  #5198 to discuss if NLog v6 is for trimming-support, and post-pone NLog-core to v7.

NLog v5.2 will no longer perform reflection of Assembly-types, unless explicitly requested by the user.
See also: https://nlog-project.org/2023/05/30/nlog-5-2-trim-warnings.html