[rt.openssl.org #3849] openssl s_client delays on Windows

Migrated from rt.openssl.org#3849 (status was 'new')
Requestors:

<EMAIL_ADDRESS>
From<EMAIL_ADDRESS>on 2015-05-13 22:55:57:
OpenSSL version: 1.0.2a after applying the patch from http://rt.openssl.org/Ticket/Display.html?user=guest&pass=guest&id=3464
OS: Windows (tested on Windows 2012 r2 64 bit and Windows 7 64 bit)

openssl s_client introduces 2 delays of 1 second when closing the connection, clearly visible when running

echo | openssl.exe s_client -connect google.com:443 -debug

The openssl.exe was compiled from source after applying the patch from http://rt.openssl.org/Ticket/Display.html?user=guest&pass=guest&id=3464

These delays do not happen in Unix and Cygwin.

A similar delay of 1 second is also introduced at start, when displaying the message "Loading 'screen' into random state".

Closing, please open a new issue if this is still relevant.


Small fix to the Order date_extended PR

Small fix for #77  - don't show extended from and extended until if the value is not known

(tests are not working, because they are not working on current master)

@PetrDlouhy please update the latest with master, master tests are working as expected now.

Codecov Report

Merging #89 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #89   +/-   ##
=======================================
  Coverage   76.28%   76.28%
=======================================
  Files          34       34
  Lines        1809     1809
=======================================
  Hits         1380     1380
  Misses        429      429

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ada4bea...ba78d78. Read the comment docs.

@Alir3z4 Done.


[enhancement] please enable installation to SD card

I'd certainly welcome if you enabled this app to be installed to an SD card instead of the limited internal flash memory.  Please have a look at https://developer.android.com/guide/topics/data/install-location.  I don't see a reason why your app can't be installed on the SD memory. Thank you for your consideration.

See Applications that should NOT install on external storage. K-9 Mail uses quite a few of those features.
It's unfortunate that adoptable storage seems to reuse android:installLocation instead of introducing a new attribute. Compared to apps-on-SD-card (API 8+) adoptable storage is actually useful because it allows storing app-private data on external storage. I guess we'll set android:installLocation="auto" and hope nobody with Android versions prior to 6.0 will move the app to external storage without knowing what this entails.
See also #1187


Help Pls

I get an error when trying to take the screenshot to test installation.
Console:
C:\Users\jonat\AppData\Local\Programs\Python\Python39\Lib\site-packages>python clashroyalebuildabot/screen.py
adb.exe: more than one device/emulator
Traceback (most recent call last):
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python39\Lib\site-packages\clashroyalebuildabot\screen.py", line 41, in <module>
    main()
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python39\Lib\site-packages\clashroyalebuildabot\screen.py", line 34, in main
    cls = Screen()
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python39\Lib\site-packages\clashroyalebuildabot\screen.py", line 12, in __init__
    window_size = subprocess.check_output(['adb', 'shell', 'wm', 'size'])
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python39\lib\subprocess.py", line 424, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python39\lib\subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['adb', 'shell', 'wm', 'size']' returned non-zero exit status 1.

Try running the following to get rid of your old devices/emulators before you start the ADB server.
adb kill-server
adb start-server

To check that it's working,  adb devices should only return one device.
If it doesn't, try setting the ANDROID_SERIAL environment variable to the serial number of the Bluestacks emulator (returned by the previous command).


Last update 093a939...ffa3a23


[GSB] Yet more cleanup to realize fewer potential archetypes

Clean up and rework some PotentialArchetype-wielding APIs in the GenericSignatureBuilder, including:

Use UnresolvedType or EquivalenceClass * rather than PotentialArchetype * in a number of places where we don't need a potential archetype.
Simplify enumerateRequirements() to operate on equivalence classes and same-type component anchors. It eliminates an O(n^2) algorithm (where n is small, but still) and fixes a canonicalization issue
Reimplement archetype "anchor" computation in a more efficient, sensible manner.

Three crashers fixed, 5% speedup in type-checking time for the standard library, and few places that realize potential archetypes.

@swift-ci please smoke test and merge

@swift-ci please smoke test Linux

@swift-ci please smoke test Linux

@swift-ci please smoke test and merge

@swift-ci please smoke test and merge

@swift-ci please smoke test macOS

@swift-ci please smoke test macOS

macOS test failure is unrelated.


Highligted words in visual mode are invisible in neovim

I am running WSL2 + windows terminal + nvim v0.9.1 and the following plugins.

It appears there may be direct way to target this state, but there is a workaround to set the "Search" higlight group to reverse, and flip the fg/bg. This means that the search higlight will persist and appear the same even when selected in visual mode.
Fix would look like this:

Setting the configuration for nord theme to the above did not change anything for me. Was this the intended fix?

Apologies for the confusion, the screenshot was only inteded to show that with the described fix, the highlighting persists even when the line is selected in visual mode.
The fix should be done to the nord plugin itself, but you can override the highlight group in your own config in the meantime. Somewhere after the line where you declare the color as nord, put this to modify the "Search" higlight group: hi! Search guifg=#88C0D0 guibg=#3B4252 ctermfg=6 ctermbg=0 gui=reverse term=reverse

change nord.vim
hi("Visual", "",s:nord2_gui,"", s:nord1_term,"","") to
hi("Visual", "NONE",s:nord2_gui,"NONE", s:nord1_term,"","")
work for me.


adds a popup message to let the user know when there their popup bloc…

…k is enabled
@Bobfrat , @DanielJMaher , this will give the user a notice when the popup blocker is enabled.

:+1: Good looks!

Picture?!?! @FBRTMaka


String representation for boolean values in query string

I have the same issue #931 and I think it's convenient to have the option to parametrize the desired boolean representation of boolean values.

As stated in #931, I'd like to leave this out of the core. I'd recommend subclassing a serializer to get this specific behavior for your application.
:beers:

Thanks for the response. I think is not possible to get this behavior by subclassing the serializer because AFQueryStringPair is a private class. Also I added a property on the serializer to parametrize the behavior.


How do I get query parameters?

There are no ways to get the optional query parameters that are passed in the link. Even in $ _GET - empty.
What steps will reproduce the problem?
rout: 'article/view/all' => 'article/view-all'
action: ArticleController::actionViewAll()
url: /article/view/all?sort=name
Yii::$app->request->get();
Yii::$app->request->getQueryParams();
$_GET;

What is the expected result?
What do you get instead?
Empty parameters in all cases.
Additional info

Q
A

Yii version
<IP_ADDRESS>

PHP version
5.6.23

Operating system
Windows 7

What's the URL?

@samdark Я же написал "/article/view/all?sort=name".

@i-panov it looks like error in configuration maybe. Or in code. Just look into framework code: $_GET is never clears by framework.
https://github.com/yiisoft/yii2/blob/master/framework/web/Request.php#L273
https://github.com/yiisoft/yii2/blob/master/framework/web/Request.php#L616
It's the only places in whole framework when $_GET appears. So, the question is - who clear you $_GET?

So you aren't getting sort in $_GET? As @MysteryDragon said, Yii does nothing about that by itself.

It seems the problem was in the incorrect configuration of .htaccess (although it was default, ie as it was originally in the framework). When I replaced RewriteRule ^(.*)$ /index.php?/$1 [L] by RewriteRule . index.php everything worked! And by the way, I finally got a debug-panel! I wrote about the problems with it in this request, but I did not decide, and finally it happened!

It seems the problem was in the incorrect configuration of .htaccess (although it was default, ie as it was originally in the framework). When I replaced RewriteRule ^(.*)$ /index.php?/$1 [L] by RewriteRule . index.php everything worked! And by the way, I finally got a debug-panel! I wrote about the problems with it in this issue, but I did not decide, and finally it happened!

I met the same problem. Thank you for save my life and time.
> RewriteEngine On
> RewriteCond %{REQUEST_FILENAME} !-f
> RewriteCond %{REQUEST_FILENAME} !-d
> #RewriteRule ^(.*)$ /index.php?/$1 [L]
> RewriteRule . index.php


Added alternate pattern for 'lint' in CONTRIBUTING.md

Hi Kevin,
I updated the alternate pattern for 'lint' in CONTRIBUTING.md. I manage to make the 'npm run lint' work by changing this. Could you please take a look?

Close this request. Will bring this up until someone has the same issue.


Complete UI Diagram

In  DeveloperGuide.adoc - Design Section. Refer to instructions in the subheading to proceed.

Will update issues based on Stephanie's feedback


[SPARK-6949] [SQL] [PySpark] Support Date/Timestamp in Column expression

This PR enable auto_convert in JavaGateway, then we could register a converter for a given types, for example, date and datetime.
There are two bugs related to auto_convert, see [1] and [2], we workaround it in this PR.
[1]  https://github.com/bartdag/py4j/issues/160
[2] https://github.com/bartdag/py4j/issues/161
cc @rxin @JoshRosen

Test build #30514 has started for   PR 5570 at commit 3c373f3.

@JoshRosen can you take a look at this? I don't really know the py4j stuff.

Test build #30515 has started for   PR 5570 at commit ceb3779.

Test build #30514 has finished for   PR 5570 at commit 3c373f3.

This patch fails PySpark unit tests.

This patch merges cleanly.

This patch adds the following public classes (experimental):

class DateConverter(object):

class DatetimeConverter(object):

This patch does not change any dependencies.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30514/
Test FAILed.

Test build #30515 has finished for   PR 5570 at commit ceb3779.

This patch fails PySpark unit tests.

This patch merges cleanly.

This patch adds the following public classes (experimental):

class DateConverter(object):

class DatetimeConverter(object):

This patch does not change any dependencies.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30515/
Test FAILed.

Test build #30525 has started for   PR 5570 at commit 2e7566d.

Test build #30525 has finished for   PR 5570 at commit 2e7566d.

This patch fails PySpark unit tests.

This patch merges cleanly.

This patch adds the following public classes (experimental):

class DateConverter(object):

class DatetimeConverter(object):

This patch does not change any dependencies.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30525/
Test FAILed.

Looks like the change broke something in MLlib.

Test build #30541 has started for   PR 5570 at commit d17d634.

Test build #30541 has finished for   PR 5570 at commit d17d634.

This patch fails PySpark unit tests.

This patch merges cleanly.

This patch adds the following public classes (experimental):

class DateConverter(object):

class DatetimeConverter(object):

This patch does not change any dependencies.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30541/
Test FAILed.

Test build #30545 has started for   PR 5570 at commit eb4fa53.

Test build #30545 has finished for   PR 5570 at commit eb4fa53.

This patch passes all tests.

This patch merges cleanly.

This patch adds the following public classes (experimental):

class DateConverter(object):

class DatetimeConverter(object):

This patch does not change any dependencies.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30545/
Test PASSed.

Thanks. I'm going to merge this in master.


Fix more nil pointer deference.

Go is not my thing, but this solved my remaining nil pointer issues.
Fixes #51, #15 hopefully
🍔

@paulirish you rock!


GPU process crash on Linux with nvidia

I confirm (by marking "x" in the [ ] below: [x]):

[x] This is not a troubleshooting question. Troubleshooting questions go here: http://www.mattermost.org/troubleshoot/.
[x] This doesn't reproduce on web browsers (such as in Chrome). If it does, issue reports go to the Mattermost Server repository.
[x] I have read contributing guidelines.

Summary

Environment

Operating System: Ubuntu 20.04.3 LTS
Mattermost Desktop App version: 5.0.2
Mattermost Server version: 6.1.0

Other relevant environment info:

GPU: NVIDIA Quadro M4000
nvidia driver version: Driver Version: 470.82.00

Steps to reproduce

Launch mattermost-desktop from command line
Use mattermost as usual
Rapidly toggle certain slow-loading menu items such as the emoji selector, custom emoji page, or something. I'm not exactly sure.

Expected behavior
Mattermost does not freeze or crash
Observed behavior
In rare situations, mattermost freezes/crashes with
[437644:1210/114605.189733:ERROR:validation_errors.cc(106)] Invalid message: VALIDATION_ERROR_UNEXPECTED_NULL_POINTER (null field 1)
[437644:1210/114605.189934:ERROR:gpu_child_thread.cc(58)] Mojo error in GPU process: Validation failed for gpu.mojom.GpuChannel [VALIDATION_ERROR_UNEXPECTED_NULL_POINTER (null field 1)]
[437644:1210/114605.190090:ERROR:interface_endpoint_client.cc(656)] Message<PHONE_NUMBER> rejected by interface gpu.mojom.GpuChannel
[437644:1210/114605.190361:ERROR:shared_image_manager.cc(190)] SharedImageManager::ProduceGLTexturePassthrough: Trying to produce a representation from a non-existent mailbox.
[437644:1210/114605.190567:ERROR:raster_decoder.cc(2132)] [.RenderWorker-0x36805d64000]GL ERROR :GL_INVALID_VALUE : glCopySubTexture: unknown mailbox
[437644:1210/114605.197356:ERROR:shared_image_factory.cc(338)] UpdateSharedImage: Could not find shared image mailbox
[437644:1210/114605.197556:ERROR:shared_image_stub.cc(188)] SharedImageStub: Unable to update shared image
[437644:1210/114605.224047:ERROR:shared_image_manager.cc(214)] SharedImageManager::ProduceSkia: Trying to Produce a Skia representation from a non-existent mailbox.
[437644:1210/114605.225096:ERROR:shared_image_manager.cc(214)] SharedImageManager::ProduceSkia: Trying to Produce a Skia representation from a non-existent mailbox.

Please note that there are no other crashes or instabilities detected with the GPU on this machine whatsoever. There are no NVIDIA-related errors in dmesg, journalctl, or /var/log/Xorg.0.log, and all other graphics-accelerated applications, including running mattermost in Chrome, work perfectly.
However dmesg does show the mattermost crashes:
[1808946.403561] traps: mattermost-desk[3343234] trap int3 ip:55bb4a1202e4 sp:7fffe1d29570 error:0 in mattermost-desktop[55bb47d7e000+65ed000]
[2330081.952971] traps: mattermost-desk[4186538] trap int3 ip:55e3c207f2e4 sp:7ffd698a3140 error:0 in mattermost-desktop[55e3bfcdd000+65ed000]
[2416312.275566] traps: mattermost-desk[295182] trap int3 ip:560b773992e4 sp:7ffe0604cd70 error:0 in mattermost-desktop[560b74ff7000+65ed000]

Possible fixes
My best guess is that there is a race condition somewhere in the code. A texture for a slow-loading menu is loaded to GPU, but, because of my rapid navigating, the destination gets destroyed before it loads.

@dllu Does this happen if you use Google Chrome or the Chromium browser as well? If you turn off GPU acceleration does it happen?

Yes, I also use Google Chrome and Chromium. Both work perfectly fine with GPU acceleration.
I got some more crashes in Mattermost today.
16:10:02.403 › show back button
16:10:05.733 › show back button
16:10:07.529 › show back button
16:10:13.366 › show back button
16:10:13.863 › hide back button
[47494:0111/120732.264827:ERROR:validation_errors.cc(106)] Invalid message: VALIDATION_ERROR_UNEXPECTED_NULL_POINTER (null field 1)
[47494:0111/120732.265034:ERROR:gpu_child_thread.cc(58)] Mojo error in GPU process: Validation failed for gpu.mojom.GpuChannel [VALIDATION_ERROR_UNEXPECTED_NULL_POINTER (null field 1)]
[47494:0111/120732.265114:ERROR:interface_endpoint_client.cc(656)] Message<PHONE_NUMBER> rejected by interface gpu.mojom.GpuChannel
[47494:0111/120732.265229:ERROR:validation_errors.cc(106)] Invalid message: VALIDATION_ERROR_UNEXPECTED_NULL_POINTER (null field 1)
[47494:0111/120732.265287:ERROR:gpu_child_thread.cc(58)] Mojo error in GPU process: Validation failed for gpu.mojom.GpuChannel [VALIDATION_ERROR_UNEXPECTED_NULL_POINTER (null field 1)]
[47494:0111/120732.265340:ERROR:interface_endpoint_client.cc(656)] Message<PHONE_NUMBER> rejected by interface gpu.mojom.GpuChannel
[47494:0111/120732.265473:ERROR:shared_image_manager.cc(190)] SharedImageManager::ProduceGLTexturePassthrough: Trying to produce a representation from a non-existent mailbox.
[47494:0111/120732.265601:ERROR:shared_image_manager.cc(190)] SharedImageManager::ProduceGLTexturePassthrough: Trying to produce a representation from a non-existent mailbox.
[47494:0111/120732.265675:ERROR:raster_decoder.cc(2132)] [.RenderWorker-0x32e6002fb100]GL ERROR :GL_INVALID_VALUE : glCopySubTexture: unknown mailbox
[47494:0111/120732.265752:ERROR:shared_image_manager.cc(190)] SharedImageManager::ProduceGLTexturePassthrough: Trying to produce a representation from a non-existent mailbox.
[47494:0111/120732.265819:ERROR:shared_image_manager.cc(190)] SharedImageManager::ProduceGLTexturePassthrough: Trying to produce a representation from a non-existent mailbox.
[47494:0111/120732.265884:ERROR:raster_decoder.cc(2132)] [.RenderWorker-0x32e6002fb100]GL ERROR :GL_INVALID_VALUE : glCopySubTexture: unknown mailbox
12:07:32.723 › Renderer process for a webcontent is no longer available: crashed
12:07:52.482 › Error getting system idle time: Error: Render frame was disposed before WebFrameMain could be accessed
    at Object._send (<anonymous>)
    at Object.n.send (electron/js2c/browser_init.js:165:413)
    at Object.b.send (electron/js2c/browser_init.js:161:2492)
    at /opt/Mattermost/resources/app.asar/index.js:61077:56
    at Map.forEach (<anonymous>)
    at ViewManager.sendToAllViews (/opt/Mattermost/resources/app.asar/index.js:61077:18)
    at Object.sendToMattermostViews (/opt/Mattermost/resources/app.asar/index.js:59985:24)
    at UserActivityMonitor.<anonymous> (/opt/Mattermost/resources/app.asar/index.js:850:19)
    at UserActivityMonitor.emit (events.js:376:20)
    at UserActivityMonitor.sendStatusUpdate (/opt/Mattermost/resources/app.asar/index.js:66355:10)
12:07:53.484 › Error getting system idle time: Error: Render frame was disposed before WebFrameMain could be accessed
    at Object._send (<anonymous>)
    at Object.n.send (electron/js2c/browser_init.js:165:413)
    at Object.b.send (electron/js2c/browser_init.js:161:2492)
    at /opt/Mattermost/resources/app.asar/index.js:61077:56
    at Map.forEach (<anonymous>)
    at ViewManager.sendToAllViews (/opt/Mattermost/resources/app.asar/index.js:61077:18)
    at Object.sendToMattermostViews (/opt/Mattermost/resources/app.asar/index.js:59985:24)
    at UserActivityMonitor.<anonymous> (/opt/Mattermost/resources/app.asar/index.js:850:19)
    at UserActivityMonitor.emit (events.js:376:20)
    at UserActivityMonitor.sendStatusUpdate (/opt/Mattermost/resources/app.asar/index.js:66355:10)

I'll try without GPU acceleration and see if it keeps happening.

Yes, I also use Google Chrome and Chromium. Both work perfectly fine with GPU acceleration.
I got some more crashes in Mattermost today.
16:10:02.403 › show back button
16:10:05.733 › show back button
16:10:07.529 › show back button
16:10:13.366 › show back button
16:10:13.863 › hide back button
[47494:0111/120732.264827:ERROR:validation_errors.cc(106)] Invalid message: VALIDATION_ERROR_UNEXPECTED_NULL_POINTER (null field 1)
[47494:0111/120732.265034:ERROR:gpu_child_thread.cc(58)] Mojo error in GPU process: Validation failed for gpu.mojom.GpuChannel [VALIDATION_ERROR_UNEXPECTED_NULL_POINTER (null field 1)]
[47494:0111/120732.265114:ERROR:interface_endpoint_client.cc(656)] Message<PHONE_NUMBER> rejected by interface gpu.mojom.GpuChannel
[47494:0111/120732.265229:ERROR:validation_errors.cc(106)] Invalid message: VALIDATION_ERROR_UNEXPECTED_NULL_POINTER (null field 1)
[47494:0111/120732.265287:ERROR:gpu_child_thread.cc(58)] Mojo error in GPU process: Validation failed for gpu.mojom.GpuChannel [VALIDATION_ERROR_UNEXPECTED_NULL_POINTER (null field 1)]
[47494:0111/120732.265340:ERROR:interface_endpoint_client.cc(656)] Message<PHONE_NUMBER> rejected by interface gpu.mojom.GpuChannel
[47494:0111/120732.265473:ERROR:shared_image_manager.cc(190)] SharedImageManager::ProduceGLTexturePassthrough: Trying to produce a representation from a non-existent mailbox.
[47494:0111/120732.265601:ERROR:shared_image_manager.cc(190)] SharedImageManager::ProduceGLTexturePassthrough: Trying to produce a representation from a non-existent mailbox.
[47494:0111/120732.265675:ERROR:raster_decoder.cc(2132)] [.RenderWorker-0x32e6002fb100]GL ERROR :GL_INVALID_VALUE : glCopySubTexture: unknown mailbox
[47494:0111/120732.265752:ERROR:shared_image_manager.cc(190)] SharedImageManager::ProduceGLTexturePassthrough: Trying to produce a representation from a non-existent mailbox.
[47494:0111/120732.265819:ERROR:shared_image_manager.cc(190)] SharedImageManager::ProduceGLTexturePassthrough: Trying to produce a representation from a non-existent mailbox.
[47494:0111/120732.265884:ERROR:raster_decoder.cc(2132)] [.RenderWorker-0x32e6002fb100]GL ERROR :GL_INVALID_VALUE : glCopySubTexture: unknown mailbox
12:07:32.723 › Renderer process for a webcontent is no longer available: crashed
12:07:52.482 › Error getting system idle time: Error: Render frame was disposed before WebFrameMain could be accessed
    at Object._send (<anonymous>)
    at Object.n.send (electron/js2c/browser_init.js:165:413)
    at Object.b.send (electron/js2c/browser_init.js:161:2492)
    at /opt/Mattermost/resources/app.asar/index.js:61077:56
    at Map.forEach (<anonymous>)
    at ViewManager.sendToAllViews (/opt/Mattermost/resources/app.asar/index.js:61077:18)
    at Object.sendToMattermostViews (/opt/Mattermost/resources/app.asar/index.js:59985:24)
    at UserActivityMonitor.<anonymous> (/opt/Mattermost/resources/app.asar/index.js:850:19)
    at UserActivityMonitor.emit (events.js:376:20)
    at UserActivityMonitor.sendStatusUpdate (/opt/Mattermost/resources/app.asar/index.js:66355:10)
12:07:53.484 › Error getting system idle time: Error: Render frame was disposed before WebFrameMain could be accessed
    at Object._send (<anonymous>)
    at Object.n.send (electron/js2c/browser_init.js:165:413)
    at Object.b.send (electron/js2c/browser_init.js:161:2492)
    at /opt/Mattermost/resources/app.asar/index.js:61077:56
    at Map.forEach (<anonymous>)
    at ViewManager.sendToAllViews (/opt/Mattermost/resources/app.asar/index.js:61077:18)
    at Object.sendToMattermostViews (/opt/Mattermost/resources/app.asar/index.js:59985:24)
    at UserActivityMonitor.<anonymous> (/opt/Mattermost/resources/app.asar/index.js:850:19)
    at UserActivityMonitor.emit (events.js:376:20)
    at UserActivityMonitor.sendStatusUpdate (/opt/Mattermost/resources/app.asar/index.js:66355:10)

I'll try without GPU acceleration and see if it keeps happening.

The issue you're showing here has since been fixed here: https://github.com/mattermost/desktop/issues/1888
If you're getting crashes in the GPU process (as shown in the original issue) then that might still be an issue, but if you'd like to avoid this error:
12:07:52.482 › Error getting system idle time: Error: Render frame was disposed before WebFrameMain could be accessed
    at Object._send (<anonymous>)
    at Object.n.send (electron/js2c/browser_init.js:165:413)
    at Object.b.send (electron/js2c/browser_init.js:161:2492)
    at /opt/Mattermost/resources/app.asar/index.js:61077:56
    at Map.forEach (<anonymous>)
    at ViewManager.sendToAllViews (/opt/Mattermost/resources/app.asar/index.js:61077:18)
    at Object.sendToMattermostViews (/opt/Mattermost/resources/app.asar/index.js:59985:24)
    at UserActivityMonitor.<anonymous> (/opt/Mattermost/resources/app.asar/index.js:850:19)
    at UserActivityMonitor.emit (events.js:376:20)
    at UserActivityMonitor.sendStatusUpdate (/opt/Mattermost/resources/app.asar/index.js:66355:10)

Then you can download v5.0.3-rc1, which should have a fix: https://github.com/mattermost/desktop/releases/tag/v5.0.3-rc1

@dllu have you tried upgrading nvidia-driver to 515?
I have similar GPU process crash issues on 510 with Quadro P4000 (Thinkpad P71), but not in Mattermost.

Closing as inactive, likely resolved by v5.0.3


Fix tests by adding missing mocks

The latest changes in v0.6.0 seemed to have caused the tests to fail, due to some missing Slack API mocks. This PR adds those mocks, and also upgrades nock to the latest version so that .query() can be used. Thanks for considering!

<3 Awesome. Thanks so much for taking the time to fix tests…


Side by Side editing?

I find myself editing two pages at the same time (mainly moving text around, reorganizing etc) and I would find such a feature helpful.
So for example instead of having Editor / Preview you could have an option to open Page 1 / Page 2 and work on them at the same time.
Thank you!

Good way to do that is to work on 2 screens with 1 opened window with Bookstack on each.

Thanks for the suggestion @branton5 and I understand how you could see this being helpful but realistically this would complicate the editor system significantly while achieving something that could be solved via features most operating systems offer (Window management).
Therefore I'm going to close this out.

Small followup - for easy window snapping behaviour on MacOS checkout the app "Magnet" from the App Store!


On the Interpolation Transformer

Hi, thanks for the awesome work.
I have been going through the codes to better understand the paper.
I wish to understand how the interpolation transformer is implemented, but could not find it.
I assume the transformer(GPT) of Net2NetTransformer of tats/tats_transformer.py is the autoregressive transformer, but there does not seem to be another transformer for interpolation?
I found the Casual Attention to be existent, but not the Interpolation Casual Attention.
Is it not provided or am I missing something? Could you kindly specify where to look?

You should check the argument n_unmasked . For example, here:
https://github.com/SongweiGe/TATS/blob/8ea1b587a74736d420b70cc2b52ac1683682ec6c/tats/modules/gpt.py#L96

That's correct!!


[SPARK-8001][Core]Make AsynchronousListenerBus.waitUntilEmpty throw TimeoutException if timeout

Some places forget to call assert to check the return value of AsynchronousListenerBus.waitUntilEmpty. Instead of adding assert in these places, I think it's better to make AsynchronousListenerBus.waitUntilEmpty throw TimeoutException.

@andrewor14 could you take a look?

Merged build triggered.

Merged build started.

Test build #33884 has started for   PR 6550 at commit 607674a.

Test build #33884 has finished for   PR 6550 at commit 607674a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

lgtm

master 1.4 thanks


How to convert Tensor to dlpack?

Is there any library that supports dlpack?

Hello. The problem is being analyzed. We will get back to you as soon as possible.

Sorry， MindSpore is not supoort currently.

Thanks


Disable WINDOWS UPDATES

I don't know how to check whether the disable windows updates worked or not, because after disabling them, I can still click "Check for Updates" and the button still works as before. I don't know if its meant to be that way or not.
Is there any way to check if they're 100% disabled?

You really shouldn't disable these.

The thing we Disabled is "Automatic Update" Not the Manual Update.
So if you click on the Check the Update Button it still checks for the Update.

If you are manually clicking on "check for updates", you are kind of asking for updates. As @blusewill answered we do disable automatic updates. So if you disable them and don't manually download & install them u most likely will not get updates.
disable the ability for updates entirely isn't recommended and not that easy on a script base (I think)
Is this good for you? If yes please close the issue and else just go ahead and ask :)


Provide Memory Benchmarking Feature to Benchmarking Code

Description
This PR adds GPU memory logging features to the benchmark script according to #8 and an updated benchmark README for usage instructions.
There are 2 approaches to logging memory,

Using Nvidia's SMI CLI tool
Using Huggingface's HFTrainer's API

Note: Issue #19 is created to address the grouping of memory values using a common prefix and will be addressed in future
Usage
1. Nvidia's SMI CLI tool
Set environment variable MEMORY_LOGGING=nvidia to use run_benchmarks.sh with nvidia logging
For each experiment,

Before the experiment calls subprocess.run, it will open an async nvidia-smi process to monitor only gpu indices in $CUDA_VISIBLE_DEVICES and log to FILE_MEM inside Experiment.save_dir
After the experiment subprocess call is completed. It terminates the async process.
At the end of each experiment, reads and aggregates an average memory over time (MiB / per sec) for each device and finally saves the average memory across all devices into gpu_mem in the main result logging function Experiment.write_results
Since it is an independent process called to measure the device, no expected slowdown in training speed

Each experiment directory will have a gpu log that contains <Timestamp>, <GPU Name>, <GPU ID>, <GPU Memory Used>

The memory readings will be reflected in the results raw_summary.csv under the column 'nvidia_mem_reserved' where the raw values are reported in MiB
2.  Torch CUDA through Huggingface's HFTrainer's API
Set environment variable MEMORY_LOGGING=huggingface to use run_benchmarks.sh with huggingface logging (default)
HFTrainer has a feature to log memory through the skip_memory_metrics=False training argument. In their documentation, it is mentioned that setting this argument to False will affect training speed. In our tests so far (below), we do not see significant difference in throughput (tokens/sec) when using this argument.
A set of finegrain GPU readings will show as additional columns in the results raw_summary.csv where the raw values are reported in bytes

3. Log Both
Set environment variable MEMORY_LOGGING=all to use run_benchmarks.sh with both logging methods
4. Difference between Nvidia-SMI Utility and Torch CUDA through HFTrainer API
1. The Nvidia-SMI Utility is a coarse measurement tool that captures anything takes up GPU memory. It is simple and non-intrusive as it doesn't involve probing the trainer. It uses the NVML library to fetch reserved memory for each device ID -

All running computer processes using that device
All gpu memory used inside the training script
Any other caching of gpu memory by the training script

Note: To get accurate measurements, no other processes should be running on the device apart from the target process itself.
2. The HFTrainer API is a more precise tool that logs memory usage for a couple of operations inside HFTrainer
It uses torch.cuda.memory_allocated to probe the trainer by taking snapshots of allocated memory and storing the differences between the before and after of each stage. The following stages are probed -

Before Trainer init
Trainer.__init__,
Trainer.train,
Trainer.evaluate,
Trainer.predict.

Note: Any gpu memory accessed and used outside any of these stages or not part of HFTrainer will not be tracked. If the train script does not use the Huggingface trainer then this API will not work as well.
Note: Details on Memory Calculations from HFTrainer for GPTQ-LoRA + FSDP
This is an example of the memory values that HFTrainer will produce in the outputs of train()
output_metrics = {
    'train_runtime': 191.2491,
    'train_samples_per_second': 0.209,
    'train_steps_per_second': 0.052,
    'train_tokens_per_second': 428.342,
    'train_loss': 1.0627506256103516,
    'init_mem_cpu_alloc_delta': 4096,
    'init_mem_gpu_alloc_delta': 0,
    'init_mem_cpu_peaked_delta': 0,
    'init_mem_gpu_peaked_delta': 0,
    'train_mem_cpu_alloc_delta': 839086080,
    'train_mem_gpu_alloc_delta': -17491768832,
    'train_mem_cpu_peaked_delta': 0,
    'train_mem_gpu_peaked_delta':<PHONE_NUMBER>4,
    'before_init_mem_cpu':<PHONE_NUMBER>,
    'before_init_mem_gpu':<PHONE_NUMBER>6,
    'epoch': 0.01
}

We refer to the keys of the memory metrics in this order

before_init_mem_X as stage0
init_mem_X as stage1
train_mem_X as stage2
...

We currently compute the memory values in the report by taking the largest of sums. For example:
For allocated memory value
max([
  stage0_mem + stage1_allocated_delta,
  stage0_mem + stage1_allocated_delta + stage2_allocated_delta,
  ...
])

For peak memory value
max([
  stage0_mem + stage1_allocated_delta + stage1_peaked_delta,
  stage0_mem + stage1_allocated_delta + stage2_allocated_delta + stage2_peaked_delta,
  ...
])

Notice that we do not include stage0_mem alone when computing the max value. This is to avoid misleading comparisons between GPTQ-LoRA and other approaches that support low-memory mode. GPTQ-LoRA + FSDP currently does not support low-memory mode as mentioned #18.
The stage0_mem value of GPTQ-LoRA + FSDP will reflect a larger  value as it is loaded fully before the trainer is initialized and then subsequently will be sharded internally in trainer.prepare.
This might cause some misleading comparisons when other variants are loaded in low-memory mode and have smaller stage0_mem memory consumption than GPTQ-LoRA + FSDP before its sharding. Once low-memory mode is supported for GPTQ-LoRA, we will include stage0_mem back inside the max computation
Tests
Memory Measurement Accuracy and Potential Side Effects
1. No Significant Slowdown From Using HFTrainer Memory Probes API on QLoRA Training
In both the Mistral7B model and Mixtral model, introducing the memory probes do not show a significant impact to the throughput of the training run (50 steps). Generally with larger batch sizes and models, the overhead of memory logging becomes insignificant.
A. <100 toks/sec slowdown after introducing the memory probes for Mistral,

model_name_or_path
numgpus
perdevicebatchsize
throughputwithno mem probe(toks/sec)
throughputwithmem probe(toks/sec)

mistralai/Mistral-7B-v0.1
1
4
3465
3432

mistralai/Mistral-7B-v0.1
2
2
2973
2931

mistralai/Mistral-7B-v0.1
1
8
3489
3508

mistralai/Mistral-7B-v0.1
2
4
3383
3298

B. <100 toks/sec slowdown after introducing the memory probes for Mixtral

model_name_or_path
numgpus
perdevicebatchsize
throughputwithno_mem_probe(toks/sec)
throughputwithmem_probe(toks/sec)

mistralai/Mixtral-8x7B-Instruct-v0.1
1
4
1785
1776

mistralai/Mixtral-8x7B-Instruct-v0.1
2
2
1518
1442

mistralai/Mixtral-8x7B-Instruct-v0.1
1
8
1938
1933

mistralai/Mixtral-8x7B-Instruct-v0.1
2
4
1757
1724

2. Torch/HF shows more granularity of memory usage with peak memory and actual allocated memory than Nvidia's reserved memory. This is more helpful when analyzing the actual memory allocated for each model.
We compare the 2 memory tracking methods (Nvidia vs Torch/HF) on single devices for both GPTQ-LoRA and QLoRA. Nvidia's peak mem reserved reports larger values than Torch/HF peak mem alloc,  torch mem alloc shows the actual memory usage is lesser.

model_name_or_path
numgpus
perdevicebatchsize
peak nvidia mem reserved(GiB)
peak torch mem alloc(GiB)
torch mem alloc(GiB)

mistralai/Mistral-7B-v0.1
1
4
19.46
15.86
4.84

TheBloke/Mistral-7B-v0.1-GPTQ
1
4
19.97
15.89
4.87

mistralai/Mixtral-8x7B-Instruct-v0.1
1
4
37.49
36.22
25.2

TheBloke/Mixtral-8x7B-Instruct-v0.1-GPTQ
1
4
36.59
35.53
24.51

NousResearch/Llama-2-70b-hf
1
4
71.12
68.16
37.35

TheBloke/Llama-2-70B-GPTQ
1
4
70.51
65.9
36.29

3. Memory Usage Decreases on Distributed Finetuning
When running large models on multiple devices, torch mem alloc shows the memory usage decreases as the models are sharded (Comparing to table above).

model_name_or_path
numgpus
perdevicebatchsize
peak nvidia mem reserved(GiB)
peak torch mem alloc(GiB)
torch mem alloc(GiB)

mistralai/Mistral-7B-v0.1
2
4
20.97
16.59
2.73

TheBloke/Mistral-7B-v0.1-GPTQ
2
4
23.75
16.26
3.01

mistralai/Mixtral-8x7B-Instruct-v0.1
2
4
32.59
29.33
13.22

TheBloke/Mixtral-8x7B-Instruct-v0.1-GPTQ
2
4
51.1
27.74
12.79

Verified that torch mem alloc for GPTQ-LoRA on Llama2-70B hovers at 19GiB when sharded after trainer.prepare and during training. The values are similar to the manual probed values from this #15.

model_name_or_path
numgpus
perdevicebatchsize
peak nvidia mem reserved(GiB)
peak torch mem alloc(GiB)
torch mem alloc(GiB)

NousResearch/Llama-2-70b-hf
2
2
51.49
46.52
19.17

TheBloke/Llama-2-70B-GPTQ
2
2
78.69
45.4
18.65

Benchmarks
Run tox -e run_benches to produce benchmarks. Full benchmark details can be referenced here
4.  For small models, LoRA runs faster than the Quantized PEFT methods. One likely reason could be that it doesnt require an additional dequantization operation before the base layer+LoRA matmuls. While this is so, we also observe the significantly larger memory it consumed compared to the Quantized PEFT methods.

model_name_or_path
TrainingType
Accel.ConfigType
numgpus
perdevicebatchsize
peak nvidia mem reserved(GiB)
peak torch mem alloc(GiB)
torch mem alloc(GiB)
throughput(toks/sec)

mistralai/Mistral-7B-v0.1
lora
none
1
4
29.03
26.11
15.12
3597

mistralai/Mistral-7B-v0.1
lora
accelerated-peft-bnb
1
4
19.46
15.86
4.84
3428

TheBloke/Mistral-7B-v0.1-GPTQ
lora
accelerated-peft-autogptq
1
4
19.97
15.89
4.87
3254

5. We observe that on single device finetuning for larger models (e.g. 49B Mixtral), PEFT begins to run out of memory while the Quantized PEFT methods continue to maintain low memory consumption.

model_name_or_path
TrainingType
Accel.ConfigType
numgpus
perdevicebatchsize
peak nvidia mem reserved(GiB)
peak torch mem alloc(GiB)
torch mem alloc(GiB)
throughput(toks/sec)

mistralai/Mixtral-8x7B-Instruct-v0.1
none
none
1
4
79.14
OOM
OOM
OOM

mistralai/Mixtral-8x7B-Instruct-v0.1
lora
none
1
4
79.06
OOM
OOM
OOM

mistralai/Mixtral-8x7B-Instruct-v0.1
lora
baseline-peft-bnb
1
4
47.18
46.42
25.73
1396

mistralai/Mixtral-8x7B-Instruct-v0.1
lora
accelerated-peft-bnb
1
4
37.5
36.22
25.2
1764

TheBloke/Mixtral-8x7B-Instruct-v0.1-GPTQ
lora
accelerated-peft-autogptq
1
4
36.58
35.53
24.51
1864

6. In distributed finetuning for large models like Llama2-70B, GPTQ-LoRA shows the lowest memory consumption with the same throughput.

model_name_or_path
TrainingType
Accel.ConfigType
numgpus
perdevicebatchsize
peak nvidia mem reserved(GiB)
peak torch mem alloc(GiB)
torch mem alloc(GiB)
throughput(toks/sec)

NousResearch/Llama-2-70b-hf
lora
none
2
2
79
OOM
OOM
OOM

NousResearch/Llama-2-70b-hf
lora
accelerated-peft-bnb
2
2
51.4
46.52
19.17
418

TheBloke/Llama-2-70B-GPTQ
lora
accelerated-peft-autogptq
2
2
78.5
45.4
18.65
426

7. Increasing the batch size, GPTQ-LoRA is the only experiment that doesnt run out of memory.

model_name_or_path
TrainingType
Accel.ConfigType
numgpus
perdevicebatchsize
peak nvidia mem reserved(GiB)
peak torch mem alloc(GiB)
torch mem alloc(GiB)
throughput(toks/sec)

NousResearch/Llama-2-70b-hf
lora
none
2
4
OOM
OOM
OOM
OOM

NousResearch/Llama-2-70b-hf
lora
accelerated-peft-bnb
2
4
OOM
OOM
OOM
OOM

TheBloke/Llama-2-70B-GPTQ
lora
accelerated-peft-autogptq
2
4
78.48
70.67
18.65
451

can you run tox -e lint from the top level directory

can you run tox -e lint from the top level directory, the linting is not automated yet #7 . Also do we plan to activate the memory by default in run_benchmarks.sh?

can you run tox -e lint from the top level directory, the linting is not automated yet #7 . Also do we plan to activate the memory by default in run_benchmarks.sh?

okay linted. Do we want to set nvidia-smi to be the default mem logging approach or once we establish that the speed degradation is significant with using the HF memory logging API to use this instead?

okay linted. Do we want to set nvidia-smi to be the default mem logging approach or once we establish that the speed degradation is insignificant with using the HF memory logging API to use this instead?
@achew010 lets merge that commit on top of this one and let me review.

@achew010 i approived. after we update the csv we can merge

We have noted the memory keys should be renamed; to be addressed later in https://github.com/foundation-model-stack/fms-acceleration/issues/19

@achew010 can we move all the memory computation logic out of write_result into gather_report. That way the results.json only holds the raw data. Then gather_report can hold all the logic to preprocess the data for human consumption.

@achew010 also one more consideration is that memory we should only have huggingface mem probes in the benchmark.csv. This is because the script.sh cannot easily replay the nvidia-smi measurements.


Unit tests failing with material-table/core 4.1.0.

I am using Jest and Enzyme to write unit test cases for react app. In those unit test files, where we have import statements for Material Table, the complete test suit is failing. The error message provided by Jest is below:
Test suite failed to run
Jest encountered an unexpected token
This usually means that you are trying to import a file that Jest cannot parse, e.g. it's not plain JavaScript.

I checked the next recent version 4..0.0, the issue persists there too. But with version 3.2.5, all tests were passing. See the below screenshot for more detail.

Screenshots
(Image 1)

(Image 2)

As the error message suggests possible solutions for the failing tests. I applied 2nd point from the "what you can do" section (Image 2) and added "material-core/table" in the transformIgnorePatterns property in the package.json file. The tests are passing with this change.
So, why do we need to add it to transformIgnorePatterns for tests to work?
Desktop (please complete the following information):

OS: Windows 10
Browser: NA
React version:  16.13.1
Material-Table/Core - 4.0.1
Jest: 26.6.3

We changed how we bundle this library. That's why that is needed, I will update docs.

We changed how we bundle this library. That's why that is needed, I will update docs.

Thanks, @oze4 for the information.

Hi @debajit-saha , what did you add to your transformIgnorePatterns? I can't seem to get mine to work

This is what I have in transformIgnorePatterns. Try this.
"transformIgnorePatterns": [
"node_modules/(?!(jest-)?@material-table/core)"
]

Still breaking for me unfortunately. Works if I drop back to 3.2.5.
node_modules\@material-table\core\dist\index.js:2
    import "./utils/polyfill";
    ^^^^^^

    SyntaxError: Cannot use import statement outside a module

      2 | import { basicGet } from '../../api/axiosGet'
      3 | import { makeStyles } from '@material-ui/core/styles';
    > 4 | import MaterialTable from '@material-table/core';
        | ^
      5 | import CircularProgress from '@material-ui/core/CircularProgress';
      6 |
      7 |

      at ScriptTransformer._transformAndBuildScript (node_modules/@jest/transform/build/ScriptTransformer.js:537:17)
      at ScriptTransformer.transform (node_modules/@jest/transform/build/ScriptTransformer.js:579:25)
      at Object.<anonymous> (components/table-components/generic-table.js:4:1)

And here's my jest settings in my package.json
  "jest": {
    "setupFiles": [
      "<rootDir>/__mocks__/windowMock.js",
      "<rootDir>/__mocks__/canvasMock.js",
      "<rootDir>/__mocks__/barGraphMock.js"
    ],
    "transformIgnorePatterns": [
      "<rootDir>/node_modules/(?!(jest-)?@material-table/core)"
    ],

    "transform": {
      "^.+\\.[t|j]sx?$": "babel-jest",
      "\\.svg$": "./__setup__/svgTransform.js"
    },
    "moduleNameMapper": {
      "\\.(jpg|jpeg|png|gif|eot|otf|webp|svg|ttf|woff|woff2|mp4|webm|wav|mp3|m4a|aac|oga)$": "<rootDir>/__mocks__/fileMock.js",
      "\\.(css|less)$": "<rootDir>/__mocks__/styleMock.js"
    }
  },

I'm running Jest with     jest --env=jsdom

@cwpinto23 we changed our transpiler in order to cut the library size nearly in half. That is when this issue started.
I just added "type": "module" to our package.json file and changed the build script to use import statements (that was preventing me from using type: module to begin with)..
Would you mind updating to version 4.2.2 and giving this another try? It should fix the Jest error (since now it is officially a module package).
Sorry about that.

@debajit-saha see updates above

FYI
You should no longer need the transform in your Jest config in v4.2.2

@oze4 , still getting the same error I posted above. I tried it with and without the transformers.
I'm on Webpack 5 btw

I wonder if this line may be part of the issue.. "^.+\\.[t|j]sx?$": "babel-jest", (it doesn't pick up on just .js files).
Can you add the following line to your transforms just to test this theory?
    "transform": {
      "^.+\\.[t|j]sx?$": "babel-jest",
      "\\.js$": "babel-jest",       // <-- this line
      "\\.svg$": "./__setup__/svgTransform.js"
    },

@cwpinto23

Also, are you ignoring node_modules by any chance? If not, would you mind adding that as well?
From our Jest config:
https://github.com/material-table-core/core/blob/ff12774312423b57cc1c93ddc1aa261d28da90cf/jest.config.js#L170

Also, it may be silly but could you try deleting node_modules and running npm install again? Just to make sure you have the new package.json file with type: module?

If I recall correctly we had to do the same thing because one of our deps was using modules (same situation you're in).
Jest doesn't understand es6 by default, so you have to run it thru babel first.
https://stackoverflow.com/a/64223627/10431732
In fact, the biggest reason our library size is half of what it was is because we no longer need that injected babel polyfill code that makes it work (and also bloats prod builds).
Since it's a React project we reasoned you're already using es6 features, which is why this issue only occurs with Jest. And not while running a dev server, etc...Jest doesn't use es6 by default.

Alright, so I:

deleted and reinstalled node modules
created a babel.config.js that I set to  module.exports = {presets: ['@babel/preset-env']}
updated my transform to

"transform": {
      "^.+\\.(js|jsx)$": "babel-jest",
      "\\.svg$": "./__setup__/svgTransform.js"
    },

I also tried specifically pointing to the babel.config.js in my transform
 "transform": {
      "\\.js$": ["babel-jest", { "configFile": "./babel.config.js" }],
      "^.+\\.[t|j]sx?$": "babel-jest",
      "\\.svg$": "./__setup__/svgTransform.js"
    },

Confirmed I have the version of package.json specified as a module

Here's the full error (slightly different from before), seems to be focusing on MTablePagination:
node_modules\@material-table\core\dist\index.js:1
    ({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,global,jest){var a=Object.assign;import"./utils/pol
yfill";import t from"react";import{defaultProps as l}from"./defaults";import{propTypes as i}from"./prop-types";import o from"./mate
rial-table";import{withStyles as r}from"@material-ui/core";o.defaultProps=l,o.propTypes=i;const p=e=>({paginationRoot:{width:"100%"
},paginationToolbar:{padding:0,width:"100%"},paginationCaption:{display:"none"},paginationSelectRoot:{margin:0}});export default r(
p,{withTheme:!0})(e=>t.createElement(o,a(a({},e),{ref:e.tableRef})));export{MTableAction,MTableActions,MTableBody,MTableBodyRow,MTa
bleCell,MTableEditCell,MTableEditField,MTableEditRow,MTableFilterRow,MTableGroupRow,MTableGroupbar,MTableHeader,MTablePagination,MT
ableSteppedPagination,MTableToolbar}from"./components";
                                                                                ^^^^^^

    SyntaxError: Cannot use import statement outside a module

      2 | import { basicGet } from '../../api/axiosGet'
      3 | import { makeStyles } from '@material-ui/core/styles';
    > 4 | import MaterialTable from '@material-table/core';
        | ^
      5 | import CircularProgress from '@material-ui/core/CircularProgress';
      6 |
      7 |

      at ScriptTransformer._transformAndBuildScript (node_modules/@jest/transform/build/ScriptTransformer.js:537:17)
      at ScriptTransformer.transform (node_modules/@jest/transform/build/ScriptTransformer.js:579:25)
      at Object.<anonymous> (components/table-components/generic-table.js:4:1)

Worst case scenario, I can continue to use 3.2.5

Can you add the "type": "module" to your package.json file?

Just to make sure.. you did run npm i -D @babel/preset-env correct?

Yes to the preset-env, here's the start of my devDependencies:
  "devDependencies": {
    "@babel/core": "^7.15.0",
    "@babel/plugin-proposal-class-properties": "^7.14.5",
    "@babel/plugin-transform-runtime": "^7.15.0",
    "@babel/polyfill": "^7.12.1",
    "@babel/preset-env": "^7.15.0",
    "@babel/preset-react": "^7.14.5",

I tried adding the "type": "module" to my package.json as well, but same error as before

Is your project public? If not I'd like to create a POC repo that mirrors your setup as closely as possible. It doesn't make sense why it's not working for you :(

Ah, sorry, private repo for my company. I'll try and setup a public repo on my personal that's a bare minimum repeat of the setup that triggers the error

Are you using a home grown webpack config or are you using create-react-app?
I can get something started - I don't mind helping :)

All good, I got one up here: https://github.com/cwpinto23/test-material-table

It def seems like something specific to that project. Just have to find it lol. It doesn't make sense how we don't even get that error when testing. It seems like it works for some but not for others?
I'm going to create a fresh react project with homegrown webpack, etc.. to see if I get the error there as well.
I tried changing so many things but still got the error in your demo. I don't understand why.

cc @Domino987 any ideas?

Yes its the project but I am not sure why yet. I am looking into it as well.

@cwpinto23 @debajit-saha we went ahead and changed our build steps back to how it was before (a little larger library size, but at least these edge case bugs will be resolved).
v4.3.0 should work for you like normal. Please let me know if you have any questions.

Yep, verified with v4.3.0. All tests are passing w/o adding to transformIgnorePatterns. Thanks, @oze4

Going to close this one out for now.

@oze4 , just wanted to say 4.3.0 is working on my end. Thanks for all the help!


Starboard operator crashes with panic: runtime error: index out of range [1] with length 1

What steps did you take and what happened:
Hi, I ran across this wonderful project recently and wanted to onboard it for one of my projects. I have tried installing Starboard operator with both Helm and Kubectl with custom CRDs. The scanning works for few seconds and then my kubernetes cluster controller crashes with panic runtime error and end up in CrashLoopBackOff state.

I also tried reducing the scope of the targeted namespaces and made the operator to run in SingleNamespace mode but of no avail. Also attaching the Debug log for reference purposes.
 I have also updated the operators with multiple replicas to distribute the workloads however all the replicas are crashing with the same panic error.
I understand that it is not a direct issue pertaining to Starboard as it pertains to kubernetes controller crashing, however I wanted to bring it to notice to gain valuable inputs from the Aquasec community.
Thanks in advance
Environment:

Starboard version (use starboard version): 0.12.0
Kubernetes version (use kubectl version): 1.19.13 running in Azure Kubernetes Service
OS (macOS 10.15, Windows 10, Ubuntu 19.10 etc): Windows 10

Notice that running the Starboard Operator (or any K8s operator) with multiple replicase requires additional consideration. In particular enabling leader election so there's only one active replica in your cluster.
We should document this one better,  but to get started see the description of the OPERATOR_LEADER_ELECTION_ENABLED variable in the Operator Configuration documentation.


New thesis template - title page and footer

I guess, it comes every 5 years....

[x] New title page (implemented in 285f61b,  1ddfdaa and  e5ea0f1)
[x] New footer on every pages (url and logo) (implemented in 34315c5 and  b7a5476)
[ ] Triple check that there are no other surprises

[ ] Triple check that there are no other surprises


Create a WMWorkload property with a StepChain steps/parentage mapping

Fixes #8681
Replaces #8687
Seangchan, your proposal had a few issues and in the end it wouldn't provide a list of output datasets per Step, so I just went ahead and created this new PR (stealing one of your methods).
I added a couple of unit tests to test it a) one without input dataset and b) with input dataset. They both work fine, but further tests are of course needed.
BTW, I created a wide'ish structure to make sure we have all the necessary information in there. I'd actually recycle self.stepMapping and use only the new one. Can you please have a look at it to see whether we need more info (or less)?

Thank you very much Alan, patch looks good. If I understand correctly, this will include steps even if there is no output dataset or parent dataset, right? I will close my patch then.

Seangchan, I made one line change to findParentStepWithOuputDataset such that it returns a string instead of list.
I also renamed OutputDatasets to OutputDatasetMap as you suggested.
Finally, I made a third unit test to make sure all these mappings are correct.
I still would like to remove self.stepMapping and use only the new mapping structure, which has all the information we need.
Then we need to discuss which information exactly we want to put in the request json and its format. How about something like
[{'ParentDset': 'blah1', 'OutDset': 'blah2'}, {'ParentDset': 'blah2', 'OutDset': 'blah3'},...]
or maybe we rename 'OutDset' with 'ChildDset'?

[{'ParentDset': 'blah1', 'OutDset': 'blah2'}, {'ParentDset': 'blah2', 'OutDset': 'blah3'},...]
or maybe we rename 'OutDset' with 'ChildDset'?

I think ChildDset is good. This structure is more flexible. But if only have one to many relations between parent and child datasets. we could do  [{'ParentDset': 'blah1', 'ChildDsets': ['blah2']}, .....]
in a way to imply the constraint. I don't have particular preference though.

test this please

Actually, since we want to map which step outputs what, how about:
{'Step1': {'ParentDset': 'blah1', 'ChildDsets': ['blah2']},
 'Step2': {'ParentDset': 'blah2', 'ChildDsets': ['blah3', 'blah4],
...}

?
And None for no ParentDset or [] for no ChildDsets (KeepOutput=False). Note the singular vs plural trick too.

test this please

@ticoann I'm merging it and cutting a new release for testbed, but please go through it carefully once more. I'm updating the PR description soon.


ekf2: fixed calculation of static pressure error

Fixes #11235

Is this a regression?

Backported for v1.9.2. https://github.com/PX4/Firmware/commit/1fe70b2d6a1029005c46ad2d0f4c1dd4485c3da2


Added support to write() integer data as phase.

Also added units to set_measured_amplitude_analytic().

Closed in favor of #13.


DOC: tutorial-1: typo: Celsius

Thanks!
On Tuesday, October 16, 2018, Philip James<EMAIL_ADDRESS>wrote:

Merged #159 https://github.com/pybee/briefcase/pull/159 into master.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/pybee/briefcase/pull/159#event-1907442461, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AADGy7EiCJ8rELXmH0cMXF1YUBgks3Dxks5ulhSYgaJpZM4WK8fj
.


Published Parquet documentation missing arrow::async_reader

Describe the bug
As of today, the published docs at https://docs.rs/parquet/latest/parquet/ have a missing link to arrow::async_reader:

Checking out the repo locally, I can reproduce this with cargo doc --open, whereas with cargo doc --all-features --open I am able to see the async docs. I'm not sure exactly how docs are published, but presumably this process just needs to add --all-features?
To Reproduce
git clone https://github.com/apache/arrow-rs
git checkout 5374289f
cd parquet
cargo doc --open

Likely related to #1349


Paginated search queries now don't return a token on the last page

Related Issue(s):

https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/issues/242

Merge dependencie(s):

https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/241

Description:

Paginated search queries now don't return a token on the last page.
Made some fixes to the respective tests. In particular test_pagination_token_idempotent had and indentation issue
Improved execute_search no longer performs count to get total matched, uses  es_response["hits"]["total"]["value"] instead

PR Checklist:

[x] Code is formatted and linted (run pre-commit run --all-files)
[x] Tests pass (run make test)
[x] Documentation has been updated to reflect changes, if applicable
[x] Changes are added to the changelog

es_response["hits"]["total"]["value"] might not be accurate if there are more than 10,000 hits. Then a lower bound will be indicated with the "gte" relation in es_response["hits"]["total"]["relation"].
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-your-data.html#track-total-hits
So I'd suggest to use the count from the search only if es_response["hits"]["total"]["relation"] == "eq". The async count tasks are still useful when the count exceeds the default threshold.

Maybe we can add to this test or do something similar to make sure that the last page isn't returning a token? #244

Maybe we can add to this test or do something similar to make sure that the last page isn't returning a token? #244

Greetings, a test for this already exists test_pagination_item_collection.
It used to test for 7 requests for 6 items, now I've fixed it for 6 requests for 6 items.

es_response["hits"]["total"]["value"] might not be accurate if there are more than 10,000 hits. Then a lower bound will be indicated with the "gte" relation in es_response["hits"]["total"]["relation"].
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-your-data.html#track-total-hits
So I'd suggest to use the count from the search only if es_response["hits"]["total"]["relation"] == "eq". The async count tasks are still useful when the count exceeds the default threshold.

Greetings, thank you so much for the feedback.
Do you think this approach is correct?
        search_task = asyncio.create_task(
            self.client.search(
                index=index_param,
                ignore_unavailable=ignore_unavailable,
                body=search_body,
                size=limit,
            )
        )

        count_task = asyncio.create_task(
            self.client.count(
                index=index_param,
                ignore_unavailable=ignore_unavailable,
                body=search.to_dict(count=True),
            )
        )

        try:
            es_response = await search_task
        except exceptions.NotFoundError:
            raise NotFoundError(f"Collections '{collection_ids}' do not exist")

        hits = es_response["hits"]["hits"]
        items = (hit["_source"] for hit in hits)

        matched = es_response["hits"]["total"]["value"]
        if es_response["hits"]["total"]["relation"] != "eq":
            if count_task.done():
                try:
                    matched = count_task.result().get("count")
                except Exception as e:
                    logger.error(f"Count task failed: {e}")
        else:
            count_task.cancel()

I think either getting the correct count or no count at all would be the preferred behaviour.
The context extension is deprecated at this point, I don't know if there is an alternative available. Maybe we should handle this in a separate PR.
https://github.com/stac-api-extensions/context
https://github.com/radiantearth/stac-api-spec/issues/396

I think either getting the correct count or no count at all would be the preferred behaviour.

Is there situations where the count can fail ? Is this why count_task.result().get("count") is surrounded in a try/except?

Is there situations where the count can fail ? Is this why count_task.result().get("count") is surrounded in a try/except?

Probably the same reason a search request could fail, eg. invalid collection, bad query, ...
Without an accurate count, it is impossible to tell if we're on the last page. The only case where you are sure you're on the last page is when the current page size is less than the limit.

I don't know why it's in a try except myself? A lot of the db stuff was done by an old contributor.

es_response["hits"]["total"]["value"] is accurate up to 10,000 results. If the actual count_task fails - which is probably unlikely - we can use this maybe because people are not going to paginate through more than 10,000 results.

Indeed, if you are paging through more than 10,000 hits, I think there is no harm in 1 extra request with an empty response
😉

Another option would be to implement this workaround: https://stackoverflow.com/a/67200853/9339603
Pro's:

uniform handling regardless of number of hits ($.hits.total.relation: eq / gte)
no need track count in the pagination token

Con's:

edge cases like limit=0
performance impact?

Another option would be to implement this workaround: https://stackoverflow.com/a/67200853/9339603

@StijnCaerts I've tried implementing this approach
        search_after = None
        page = 1

        if token:
            decoded_token = urlsafe_b64decode(token.encode()).decode().split(",")
            search_after = decoded_token[:-1]
            page = int(decoded_token[-1]) + 1

        query = search.query.to_dict() if search.query else None

        index_param = indices(collection_ids)

        search_task = asyncio.create_task(
            self.client.search(
                index=index_param,
                ignore_unavailable=ignore_unavailable,
                query=query,
                sort=sort or DEFAULT_SORT,
                search_after=search_after,
                size=limit+1,
            )
        )

        count_task = asyncio.create_task(
            self.client.count(
                index=index_param,
                ignore_unavailable=ignore_unavailable,
                body=search.to_dict(count=True),
            )
        )

        try:
            es_response = await search_task
        except exceptions.NotFoundError:
            raise NotFoundError(f"Collections '{collection_ids}' do not exist")

        hits = es_response["hits"]["hits"]
        items = (hit["_source"] for hit in hits[:limit])

        next_token = None
        if len(hits) > limit:
            if hits and (sort_array := hits[limit-1].get("sort")):
                next_token = urlsafe_b64encode(
                    ",".join([str(x) for x in sort_array] + [str(page)]).encode()
                ).decode()

        matched = None
        if count_task.done():
            try:
                matched = count_task.result().get("count")
            except Exception as e:
                logger.error(f"Count task failed: {e}")

        return items, matched, next_token

but I'm getting these errors on these tests:
FAILED stac_fastapi/tests/api/test_api.py::test_app_query_extension_limit_gt10000 - elasticsearch.BadRequestError: BadRequestError(400, 'search_phase_execution_exception', 'Result window is too large, from + size must be less than or e...
FAILED stac_fastapi/tests/api/test_api.py::test_app_query_extension_limit_10000 - elasticsearch.BadRequestError: BadRequestError(400, 'search_phase_execution_exception', 'Result window is too large, from + size must be less than or e...

As I understand elasticsearch itself doesn't allow limits above 10,000, so when you pass 10,000 +1 it responds as a bad request.

Ended up applying this https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/243#issuecomment-2096698581 with edge  case handling, in this case the 10,000.

what do you think @jonhealy1 ?
I don't really like this hardcoded 10,000, but it's basically set by:

elasticsearch setting which only allows requests with up to 10,0000 by default.
stac-fastapi.api limits to 10,000 by default aswell

Can we import the max limit from stac-fastapi?

Can we import the max limit from stac-fastapi?

I was digging deeper and it seems we can ask elasticsearch indexes, but if not defined the default 10,000 will not be there...

Can we import the max limit from stac-fastapi?

I guess yes, we can use this:
max_result_window =  stac_fastapi.types.search.Limit.le

Can we import the max limit from stac-fastapi?

Yes, we can use this:
max_result_window =  stac_fastapi.types.search.Limit.le

I prefer option one too

I prefer option one too

added

I just left one small remark. Otherwise it looks good to me, thanks for the contribution!


Injected field is null after injecting

Hello, when I am instantiating prefab which need dependency goes error Null reference.
When that gameObject in the scene all works. I have used Start, Awake error the same.
Here some code:
https://gist.github.com/Neuyazvimyi/2c19830c2012fe60a632ca11d78517cb

My bad, sorry!


Have a gem to automatially run the specs

It would be great have something like guard-spec to automatically run the concerned specs whenever a change is made. This would really help to make TDD better.

Any suggestions?
I was about to do it myself using guard-rspec but don't know how configure the Guardfile

Why not use guard itself? We'll just need to figure it out. Don't think there's a need for a new gem.

Yes that's why I mean. Guard in not included as a gem in Daru yet.

Sure. Make a separate PR.

This is done.


add dblp to social links

Pretty self-explanatory I guess.
Many thanks for the awesome theme!

Field should be added to _config.yml as well.


Why is the letter 'y' treated differently when used as a key

const yaml = require("js-yaml");

const object = {
  x: 0.5,
  y: 0.9
};

const result = yaml.dump(object, {});

console.log(result);

// x: 0.5
// 'y': 0.9

Why does the 'y' character get apostrophes around it, whilst no other letters do?
Code playground here with it all set up: https://stackblitz.com/edit/js-yaml-y-charcter?devtoolsheight=33&file=index.js

Just answered my own question. It's because a 'y' character is treated as boolean true in yaml.

The actual answer to this question is to change the code to:
const result = yaml.dump(object, {
  noCompatMode: true
});

The the expected behaviour happens :)


Add security context to istio-csr deployment

Description

The default security context may be too permissive compared to what a cluster operator requires; configure it to be as nonrestrictive as possible.
Allow configuring the seccompprofile in case more restrictive custom policies need to be applied
Fixes #367

/ok-to-test

@SgtCoDFish - flakey test and worth a retry?
Or something obviously wrong I am missing?

/test pull-cert-manager-istio-csr-test-ecc


NarayanaTS no longer working

❗ please do not add sensitive information in issues, you can provide extra information via email using issue number as reference ❗
Describe the issue
NarayanaTS is no longer working. Appears to be with all databases, btm is working fine, though practically the same setup is used.
Reporter
Ricardo

mysql.zip
mariadb.zip

Seems to be partially introduced by https://github.com/ibissource/iaf/pull/2508.


Document problems with setters / parameters, and alternatives

This is a reminder for me to write a doc about this.
I covered over-parameterization in the rationale doc and elsewhere (https://github.com/kubernetes/design-proposals-archive/blob/8da1442ea29adccea40693357d04727127e045ed/architecture/declarative-application-management.md#parameterization-pitfalls).
Setters have kind of been an attractive nuisance, similar to vars in kustomize. I would recommend against the use of setters entirely, or at least try to minimize their use.
Setters are just template parameters with different syntax. Unlike templating tools, however, when a few simple parameters are not sufficient, in kpt it's unclear what to do next in order to solve the problem.
Setters should not be thought of as interfaces to packages. They are ad hoc transformers of the last resort. Packages aren't encapsulated and don't have interfaces. They're unencapsulated bundles of resources. Packages may contain abstractions, defined by Kubernetes resource types, but don't have abstractions.
Setters need to be added to packages largely manually, which is error-prone. They imply human editing of text representations of configuration rather than tools and interface surfaces generating configuration and performing transformations to it. Parameters are not an API, nor a resource definition.
Making it easier to create packages is a goal.
Setters are an obstacle to operating on packages with the package orchestrator, to editing them with the UI, to automating transformations, and to creating blueprints automatically, such as using exported resources.
We should strive to require no special markup of blueprints at all, much as kustomize transforms unmodified base resources. Needing to be aware of inline comments makes functions much harder to write. Carrying human-authored comments through, similar to compiler debug info, is fine and desirable.
The functions don't exist today that will be necessary to make this feasible, but we should start building in that direction rather than continuing to focus on what is, at best, a workaround. If a package contains setters, that’s probably because we didn’t create the right functions, someone had to identify string substitution targets by hand, and other users will need to do that for every package containing the parameterized resource types.
As much as possible, we (the community) should build transformer functions that at least know what their targets are, as with kustomize's built-in transformers, and possibly what where the source values should come from. Well defined inputs would create "APIs" that could be used to chain functions together more easily. Some functions will require resource-type-specific logic, and some horizontal domain-specific logic. This is logic that would otherwise be reimplemented in every "as code" style package.
We are trying to create a new approach in the spectrum between ClickOps and Programming, where using and even creating blueprints can be done by ClickOps, and programming is required less frequently than with Infrastructure as Code. When programming is required, we need SDK / language options to satisfy both the middle (relatively easy but low reuse) and right (harder but high reuse) of the spectrum.
Some "functions" may be embedded in what we think of as user interface surfaces, such as GUIs, CLIs, terminal consoles, mobile apps, ChatOps agents, etc. For instance, the Backstage UI plugin operates on the ResourceList as a function would.
Separating the code from the data enables us to apply the full toolbox of standard UX principles, techniques, and tools to the user experience.

For posterity, another setter-like approach, but which uses annotations: https://knot8.io/. Comments are problematic, but not the main problem, so I may want to reference this in the writeup.

One difference with knot8 is that it could be viewed as a mechanism for defining field-path shortcuts for fields that need to be frequently changed, and there's no extra source of truth introduced.

ytt uses comments as markup even more heavily than other tools, including for embedded code and imports as well as to mark data values to change.
https://carvel.dev/ytt/docs/v0.41.0/how-it-works/
This approach makes it infeasible to operate on as a data format, other than using the built-in templating mechanism.

Very looking forward to this. I'd like to refer to this issue when answering the user questions regarding how to use setters for XXX.


Conceal not working

The concealment doesn't work.
I'm using Typst with Tinymist.
I use Lazy as a plugin manager.
Conceal works perfectly on tex files with vimtex.
I use
  { "MrPicklePinosaur/typst-conceal.vim", lazy = false },

I could not get it to work either, unless I disabled Treesitter syntax highlighting. seems its only compatible with regular vim highlighting


chore: Wipe out submodule

Something seems to have gotten messed up a while ago, but the zombie submodule references have been resolved.
Resolves #678

The submodule update command that caused problems previously now succeeds, LGTM!


Possible to provide standalone source code files?

Hello, I recently came across the PAQ file format and the family of archiving tools associated with it. I've been quite satisfied with zpaq715 so far, and now I've become aware of zpaqfranz. What I particularly like is that some features have been added here that I found missing in zpaq, such as the ability to quickly get information about all saved versions in the form of a table. Great work, and it's nice to see active development happening here.
It's only been a few days, but I already think that zpaq might be my personal discovery of the year. I can hardly believe I didn't come across it sooner. At this point, I'd like to ask a question: the fact that the source code on GitHub has been merged into a single CPP file makes it very easy to compile it on various platforms. It was also straightforward to get a current version running on my FreeBSD. One small drawback is that navigating in the large file is quite cumbersome if you want to understand how the program works. It seems to be a generated compilation of a series of standalone source code files. Is there a possibility that these source code files or the script to create the compiled file could also be uploaded to GitHub?
Thank you and best regards,
Matthias

Hello, ...and it's nice to see active development happening here.

I am almost ready to release a specific function to detect (already in zpaqfranz) and RECOVER (first time ever) from SHA-1 collisions

(...) One small drawback is that navigating in the large file is quite cumbersome if you want to understand how the program works. It seems to be a generated compilation of a series of standalone source code files. Is there a possibility that these source code files or the script to create the compiled file could also be uploaded to GitHub?

Actually no, it is not created from several different sources through some script, because I had to modify them as I went along to make them compatible, even in very subtle situations (e.g., macros called off and variables always called off).
It's "handwork"
At the beginning of the source there are the individual portions of the software, and where I got them from
For example
14 [BSD 2-Clause license]         xxHash Copyright (C) 2012-2020 Yann Collet          https://github.com/memcached/memcached/blob/master/xxhash.h
 /// LICENSE_START.14
 /// LICENSE_END.14

If you search for LICENSE_START.14 and LICENSE_END.14 you'll find this example
zpaqfranz is made by zpaq.715 UNION (other software, like hashers, fixed by hand) UNION my own code (it is easy to spot, by style and sometimes even by italian variables too 😄 )
Therefore no
BUT
If you're curious, all you have to do is ask, and I'll answer to the best of my ability

Even if it was a "handwork" it could be beneficial to potential users of the source code (not to the end users of binaries, of course, but to the people who want to read the code, understand it or modify it) to have the original sources from different origins in separate files, and then for each release to, like the author of the question imagined, "produce a generated compilation of a series of standalone source code files."
If I understood correctly, as the precedence, that the original zpaq already embedded libdivsufsort (with the MIT license) in the libzpaq.cpp at some point in time of its development, but the resulting libzpaq.cpp is still some order of magnitude smaller (I think around 200K or less then) , and the original author could have had additional reasons for his solution as he, as far as I understand, planned to change the license of his own work and publish something like a "minimal but sufficient" version of libzpaq.cpp and of zpaq.cpp using the library, intentionally removing already implemented features or simplifying the code to achieve similar effect but in less "fixed" parts including less interface options, and then stop his own contributions, leaving a version "good enough and minimal enough to be left alone." It was consistent with author's vision and approach, and the resulting library is still a masterpiece (even having its own JIT (!) compiling engine for x86 and x64 in that 200K library, allowing it to, if I understand, even today define additional compression algorithms externally of the library, still to be executed as the native machine x86 or x64 code. And I don't know what the state of all that is for the cases of ARM-based processors. But if one would want to develop in that direction, one would probably be motivated to split even the libzpaq to smaller files, at least before a "very stable" version is reached again.
I understand that there's never a single answer and single perspective to some organizing decisions, but my vote would be for split files for the projects that intend to not be "frozen in time" or where the author would be interested in the contributions from others.

Even if it was a "handwork" (...) to have the original sources from different origins in separate files (...)

It is practically impossible to understand how it works, and how it was developed, "top down"
The way I think is best is to start from the development forum, and from the various threads "bottom up". Then study the older versions to get an idea of the direction of development

If I understood correctly (...) And I don't know what the state of all that is for the cases of ARM-based processors.

I can answer: use -DNOJIT for arm CPU-like, -DBIG for PowerPCs, -DANCIENT etc
Evertyhing is in the source code (starting comments)

But if one would want to develop in that direction, one would probably be motivated to split even the libzpaq to smaller files, at least before a "very stable" version is reached again.

I admit I'm not very interested. It's a burdensome job, and of little importance (for me)

I understand that there's never a single answer and single perspective to some organizing decisions, but my vote would be for split files for the projects that intend to not be "frozen in time" or where the author would be interested in the contributions from others.

Historically zpaq has had very little feedback, zpaqfranz even less, in the order of a few dozen
The time it would require of me could be used for additional and useful functions
If anyone is really interested, all they need to do is read Mahoney's documentation and the source for version 7.15, zpaqd and unzpaq 206

(Not directly connected with this topic, but slightly tangential as it's hard for me to figure out locally. I also don't know if it's "expected" as I think I've read somewhere that it's less tested on Intel: on my machine "hw" version with the -hw parameter seems to always crash. The technologies reported by cpuz:  Instruction sets	 MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, EM64T, VT-x, AES, AVX )

The hw use asm code for sha1
Works only on CPU that support SHA1 istructions
Like AMD or newer Intel

So does -hw ignore what the cpuid returns? I've tried to call the piece of your code ihavehw() in an independent program and it seems it detect 0 for SHA?
1838: new ecx<PHONE_NUMBER>
1843: new ebx 641
SSSE3 :OK
SSE41 :OK
SHA   :NO

No cpuid test for zpaqfranzhw
It is up to you
Your cpu does not support sha1
Therefore zpaqfranzhw -hw will crash
Standard zpaqfranz autodetect hw support, even on Linux, running walton's code
(No zpaqfranzhw on Linux)
Zpaqfranzhw use 7z code by Pavlov authorization
Everything is in the source

Thanks. I could find the relevant statement in the code now:
 For performances reason, no run-time CPU compatibility checks,

Do you know the measurable difference of cpuid being executed even if the cpuid is to be run only once for the whole time of the program (just before the first call to the functions which depend on the new instruction set)? I believed everything else (as an example, actually calculating even one checksum) would have to take much more time than e.g. two cpuid instructions.

Of course I know
Measured too
But the problem is not here, but in checking a boolean flag every time, disrupting cache loading, even with a preferred register (not honored by all compiler)
You can try yourself
And you'll find that cpuid is faster than boolean
Btw I use a specific AMD build. Maybe I'll relase

Rather OT but...
zpaqfranzhw (aka: -DHWSHA1) has a very small performance increase, compared with "normal" zpaqfranz.
I report some numerical data on my test set
zpaqfranzhw 58.11 (AMD native only) 37.046
zpaqfranzhw 58.11 (AMD native)      37.157/37.437 (12)
zpaqfranz   58.11 (generic)         37.844
zpaqfranz   54.10 (old release)     43.344

The AMD-only code is about 3% faster than the "generic" one
Or, the "generic" is just as fast as specific asm code (on the same HW accelerated hardware) 😄 , running (about) 13% faster than the older releases (aka: zpaq 7.15 implementation)
Therefore some die-hard user can "play" with the .asm, but without a big gain
=>
short version: use zpaqfranz.exe 🥇


Add "Copy link" to "Login required" dialog

Use cases:

I am testing new headscale/tailscale installation and I'm logging in into my OIDC provider with various users. It's simpler for me to open auth link in private mode instead of logging out as admin user
Browser do not opens for me anyway. I do not see any error messages in logs - neither app nor system. It's better to have possibility to get connection URL for such situations

Good idea. I'll see about adding it for people to copy manually if necessary.


Feature: clean model when a field is removed from view, such as when a condition is no longer satisfied.

This feature extends the schema-validate directive to allow users to configure an individual field or the entire form to one of a predefined set of behaviors when the $destroy event is broadcast. The common use case would be when a field is no longer visible due to a condition, which is evaluated by ng-if, no longer being met. Data that had previously been collected was remaining in the model, which would provide erroneous results on submission. By default, if no configuration is done, this feature would set the model value to undefined, just as if the field had never been visible in the form at all.
In order to properly handle complex nested arrays and objects, a new service (sfUnselect) was created. Based on the sfSelect service, sfUnselect will traverse a given path and attempt to set the endpoint to the provided value. If at any point an expected object or array is undefined, the service returns and does not create the missing object or array.
This addresses both https://github.com/Textalk/angular-schema-form/issues/344 and https://github.com/Textalk/angular-schema-form/issues/204
Also, in case it comes up in testing, this is related to https://github.com/Textalk/angular-schema-form/pull/370, as the checkboxes will not work with this new feature until the schema-validate directive is correctly spelled.
Please pardon the commit log; I managed to fumble my git-fu roll when trying to create two pull requests.

Hi @jbsaff and thank you for the PR! This is a much sought after feature and even though it changes current behavior I think that is a good call. It's probably what you want anyway and its the current behavior that surprises people. Nice work :+1:
I made some comments, I think it's a good idea to add a global option for it instead of checking formDefaults explicitly. It was unfortunate that the change in indentation broke the diff :) was a bit hard to read!

Thanks for the feedback. I'm pretty comfortable in Java, but needed something client-side for my current project. I'll incorporate your feedback into another commit. Thanks for your work on ASF. It's going to save me a ton of time!

@davidlgj Fixes checked in, and I think I got my branch rebased to match Textalk/development.
Both the code and documentation should be updated to include the globalOption and form defined destroyStrategy. I had to rework my approach to account for the difference between having the destroyStrategy property declared as undefined and not having it declared at all.
Thanks!

:+1: Awesome! I'm hoping to test and merge start of next week :)
One thing that struck me though, what happens if you remove the entire form? Like when navigating away in a single page app, or an ng-if on something around the form. Won't that trigger the $destroy?Will that then remove all the attributes from the model?

Well crud. Yes, if there is an ng-if wrapping the form, the entire form would be subject to that $destroy.  I'll look for another solution.

Okay, I've given it a thought. There are actually two parts to the logic that ought to trigger data cleanup. One is removal from the DOM, which can be identified by the $destroy. The other is that the condition on the form element is not satisfied (which might trigger a $destroy due to how conditions are wired in with ng-if). I'm going to head in that direction and see how it goes.

I was thinking that there might be a solution if the $destroy is called on
the first on sf-schema $scope then we might be able to set a flag that
stops the destruction of the fields.
fre 1 maj 2015 15:22 James Saffell<EMAIL_ADDRESS>skrev:

Okay, I've given it a thought. There are actually two parts to the logic
that ought to trigger data cleanup. One is removal from the DOM, which can
be identified by the $destroy. The other is that the condition on the form
element is not satisfied (which might trigger a $destroy due to how
conditions are wired in with ng-if). I'm going to head in that direction
and see how it goes.
—
Reply to this email directly or view it on GitHub
https://github.com/Textalk/angular-schema-form/pull/371#issuecomment-98132337
.

Trying to understand the implications of the Angular docs for the $destroy event. It says that the listener is called just prior to the actual destruction occurring. Does that mean that it happens after all of  the children have finished processing the $destroy in a depth-first fashion? If so, the $on $destroy for sfSchema would be too late to set the flag, as all of the children have already gone. I'm going to try to improve my understanding of the situation and determine what the event firing order is.

I've put together an initial (and possibly naive) implementation of a flag to retain the model instead of allowing the destroyStrategy to be used when the sfSchema element is destroyed.

I'll merge it locally and test it out!

Ok, so I didn't get through i tonight, I'll try to pick it up again soon. Keeping the flag in a service won't work since that service is shared across all instances of schema form you might have on the page, but I think I got a solution for that :) ...just got to try it out first!

Okay, thanks for the assist. I appreciate the help polishing it up!

@jbsaff  I've started to merge it in, but I've also changed some things. I really hope you don't mind. You've done some solid work and I couldn't have got this done without your PR to base it on!
Initial manual testing looks good, I pushed the branch so you can see it here if you feel like checking it out: https://github.com/Textalk/angular-schema-form/tree/jbsaff-feature/cleanModelUsingDestroyStrategy
It needs some cleaning and some tests but I think it works :)
Major points of what I have changed:

I changed the option defaultStrategy value to always take a string, so its "null", "empty" (was ''), "remove" (was undfined) and "retain". This greatly simplified the logic when trying to figure out if you should inherit a global option or not.
I changed the use of a service to flag for "external destruction", instead I put it as a variable on scope. This means several instances of schema form won't clash with each other.

@davidlgj I'm glad to hear my work was a useful starting point. I try not to take any ego into a project; if my stuff is good, it will stay. The changes you described sound like they will make the feature far more useable within ASF, so I'm very much in favor. Thanks for the reviews and suggestions. It made my first contribution to a project a lot of fun.

Only thing I noticed is that when an array element (like in a tabarray) is removed, the console fills up with "TypeError: Cannot convert undefined or null to object". If I set the global option to null, I get "TypeError: Cannot set property '[object Array]' of undefined" instead. It looks like the Remove cleans up the model (as expected), then the new feature tries to do the same thing and gets an error. Do we expect that, in a case where sfSelect would return undefined, we would still want to attempt to set the value using the destroyStrategy?

Of course! We might not have a model if the user hasn't filled out the entire form :) Added  a check!
This seems to work for me, but in certain scenarios, like when having nested objects we can end up being left with empty object {}, since there is no field with a schema-validate directive at that level... I'm gonna try and move it to the decorator directive instead.
btw I've been testing with this schema and form definition: https://gist.github.com/02ed18af7570ef149c92

Ok, moved it around a bit and added some tests. I tried doing removal of empty objects, but that is a bit tricky. Could easily end up mucking up arrays by removing the empty object needed for it to display an empty form. I'm not convinced it's actually needed, so I'm gonna let it be as it is. I think its ready enough for merging into the development branch :) Thanks for the PR and all the help!

Nice work @jbsaff & @davidlgj look forward to using it and replacing my own workarounds :)

@davidlgj  Having the behavior in the decorator directive looks like it works for the case where the destroy is triggered by something outside the form. However, when a form field destroy is triggered by a condition no longer being satisfied within that form, the sf-decorator tag remains in the DOM and the cleanup of the model isn't triggered.
Would it make sense to split the behavior off into a directive that can be applied to the outermost div in each decorator template? It would require add-ons/extensions to opt-in to using the new directive, which could be a drawback. I like the way that moving the event handling up from the schema-validate level means that it will only fire once per field, instead of once for every input (long checkbox/radio lists, commonly).

@jbsaff You're absolutely right! A separate directive is probably the cleanest, I'll try to re-work soon.


taur build bug

When using centaur the camera gets bugged in water and building is bugged the cross doesnt match the build selection

Note for later: ProjectileUtil.getHitResult()


Add support for Discordapp

Is your feature request related to a problem? Please describe.
It would be nice if we can get support for discordapp.com
There is no software like this out there that has support for discord.
There are plenty of API Wrappers for discord out there
(Link to a wrapper for golang:) https://github.com/bwmarrin/discordgo
Describe the solution you'd like
Discord has a rich API and Role based secrurity (Eg limiting who can send/read/see channels etc.)
Additional context
discordapp.com
https://discordapp.com/developers/applications/

Discord go api clinet Ref : https://bwmarrin.github.io/discordgo/

@codenio Could you implement discord support?,
I don't know anything about golang :(

any news here?

Nope sadly

@kingdevnl sorry about this.
creating the Integration test for Discordapp is the hardest part than actually implementing the functionality.
let me try my level best. it might take time.

@codenio Please feel free to ping me if you need any help. We can also work together in adding integration test. Thanks for picking this up :)

@PrasadG193 thanks for the support.
yes, we could try together as earlier.

Any new info about this?

I'm wondering how much work it would be to implement it all, Hahahaha

@kingdevnl, when you can't help yourself. you have to wait for the community to respond to your requests.


Unicode32-bit.bin does not exist?

It says "Unable to copy the appropriate binary file as AutohotkeySC.bin because said file does not exist:
............\Unicode32-bit.bin
Do you still want to continue?"
How to fix this?
Edit: Ok, I've found out it is originally be placed at
%Program Files%\AutoHotkey\Compiler\AutoHotkeySC.bin
There has never been any "Compiler" folder. Maybe there was a problem during installation process.

The compiler folder is where the installer puts the compiler. If there is no Compiler folder, you probably chose not to install the compiler when you installed AutoHotkey (I guess you got Ahk2Exe from here or somewhere else, separately).
In any case, Ahk2Exe is functioning as intended.


[20.09] virtualbox: 6.1.16 -> 6.1.18

Motivation for this change
release-20.09 currently carries a VirtualBox 6.1.16 marked broken due to incompatibility with Linux 5.10. I've backported the version upgrade and marked the relevant packages as unbroken.
This the backport of #110550.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Closed in favor of #112744. I've seen this too late. :)


Hard crash on DB.close()

I experience a hard crash when closing a DB.
I have experimented with a number of diverse settings. No matter the load or the contents of the database it always seems to happen.
This is the case both in master with Python 3 and in Python 2 with the fix from #40. I also tested with both librocksdb-dev 5.17 (from Debian stretch-backports) and the facebook/rocksdb master (>6.1.2).
At one point I got a detailed stack trace and memory map but I can't seem to recover them. I will edit this post if I can produce them later on. I do know that the error was "double free or corruption".
Closing a DB is a critical feature for my use case. I need to be able to reliably populate DBs in subprocesses and transfer the control to the master process afterwards. This is not possible unless the database is properly closed before the transfer. I have tried the del db; gc.collect() strategy from #10 and the unit tests, but, as expected, del is not deterministic and the file lock is not freed in most cases. In any case, if del run it would likely also crash in the close() method.
I believe #46 might be experiencing the same issue.
Please, let me know if this is the case for everyone or if it is an environment issue.

@oersted
Did you try pybind11 branch (https://github.com/twmht/python-rocksdb/tree/pybind11)? It should provide most of the features including stable close function.
And also let me know if there are any features you want but not in the branch.

Thank you, yes I did take a look at pybind11, I was not sure how stable it was. I am currently using the merge operations heavily and I can't find them in the docs at first sight. Are they implemented?
I have also experimented with plain get-put instead of merge, and it seems to work well, so I might try pybind11 anyways.

Yup, merge operators are still missing features in pybind11 branch. But I will add it as soon as possible.

@twmht How is progress on merge operators in pybind11? I have managed to get around this until now, but it is becoming a more important performance bottleneck.

@oersted
yes. It's almost done. Just need some more unit test. The update will be as soon as possible.

@twmht Thanks :)

Any update on this @twmht @oersted ?
Being able to close the database would be lovely :)

@maxibor I found that @iFA88 created a fork (https://github.com/iFA88/python-rocksdb) that fixed this issue. Original thread: #46. I repost it here to increase visibility.