color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

Caused by: java.lang.RuntimeException: Failure delivering result ResultInfo{who=null, request=1187240990, result=-1, data=null} to activity {com.*/com.websitebeaver.documentscanner.DocumentScannerActivity}: CvException [org.opencv.core.CvException: cv::Exception: OpenCV(4.1.0) /build/master_pack-android/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'
Hi,
This error is driving me crazy, It causes the app to crash, It shows there is an error in color.cpp related to (-215:Assertion failed) !_src.empty() in function 'cvtColor'
I am using android 11
The camera opens normally and takes the picture, The crash happens when clicking the right button.
I am using the library like this
const {scannedImages} = await DocumentScanner.scanDocument();
if (scannedImages.length > 0) {
const base64Images = [];
for (const image of scannedImages) {
const base64Image = await RNFS.readFile(image, 'base64');
base64Images.push(base64Image);
}
// setScannedImages(base64Images);
return base64Images; // Return the scanned images
      And I followed the setup instructions written here in this project

This error does not happen in the emulator I think it is related to how the phone handles image paths

try to create a new React Native project, install only this plugin, and see if it happens.

Thank you for your quick response
The issue is with the
https://github.com/WebsiteBeaver/android-document-scanner
repository
Can you check pull request #30 in that repository I tried to fix it but I am unable to test
@dmarcs

@dmarcs I finally got my hands on one of the Motorola devices that seem to be affected by this issue. I discovered some issues on android.

Below is a log from the demo of android-document-scanner. What happens is I run the demo and without any interactions it just crashes within 1-3 minutes with the following log(i've added logs to ImageUtil.kt first two lines):

2023-05-15 16:28:45.020 27225-27225/com.websitebeaver.documentscanner.demo D/DEBUG: /storage/emulated/0/Android/data/com.websitebeaver.documentscanner.demo/files/Pictures/DOCUMENT_SCAN_0_20230515_162644953650719.jpg
2023-05-15 16:28:45.020 27225-27225/com.websitebeaver.documentscanner.demo D/DEBUG: File exists: true
2023-05-15 16:28:45.022 27225-27225/com.websitebeaver.documentscanner.demo E/cv::error(): OpenCV(4.1.0) Error: Assertion failed (!_src.empty()) in cvtColor, file /build/master_pack-android/opencv/modules/imgproc/src/color.cpp, line 182
2023-05-15 16:28:45.026 27225-27225/com.websitebeaver.documentscanner.demo E/org.opencv.imgproc: imgproc::cvtColor_11() caught cv::Exception: OpenCV(4.1.0) /build/master_pack-android/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'
2023-05-15 16:28:45.027 27225-27225/com.websitebeaver.documentscanner.demo D/AndroidRuntime: Shutting down VM
2023-05-15 16:28:45.028 27225-27225/com.websitebeaver.documentscanner.demo E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.websitebeaver.documentscanner.demo, PID: 27225
    java.lang.RuntimeException: Unable to resume activity {com.websitebeaver.documentscanner.demo/com.websitebeaver.documentscanner.DocumentScannerActivity}: CvException [org.opencv.core.CvException: cv::Exception: OpenCV(4.1.0) /build/master_pack-android/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'
    ]
        at android.app.ActivityThread.performResumeActivity(ActivityThread.java:3121)
        at android.app.ActivityThread.handleResumeActivity(ActivityThread.java:3152)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1398)
        at android.os.Handler.dispatchMessage(Handler.java:102)
        at android.os.Looper.loop(Looper.java:148)
        at android.app.ActivityThread.main(ActivityThread.java:5443)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:728)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:618)
     Caused by: CvException [org.opencv.core.CvException: cv::Exception: OpenCV(4.1.0) /build/master_pack-android/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'
    ]
        at org.opencv.imgproc.Imgproc.cvtColor_1(Native Method)
        at org.opencv.imgproc.Imgproc.cvtColor(Imgproc.java:2309)
        at com.websitebeaver.documentscanner.utils.ImageUtil.getImageFromFilePath(ImageUtil.kt:41)
        at com.websitebeaver.documentscanner.DocumentScannerActivity$cameraUtil$1.invoke(DocumentScannerActivity.kt:88)
        at com.websitebeaver.documentscanner.DocumentScannerActivity$cameraUtil$1.invoke(DocumentScannerActivity.kt:75)
        at com.websitebeaver.documentscanner.utils.CameraUtil.startForResult$lambda-0(CameraUtil.kt:41)
        at com.websitebeaver.documentscanner.utils.CameraUtil.$r8$lambda$QtmUm3JcXlX2aa1YkJnrPoMUzS8(CameraUtil.kt)
        at com.websitebeaver.documentscanner.utils.CameraUtil$$ExternalSyntheticLambda0.onActivityResult(D8$$SyntheticClass)
        at androidx.activity.result.ActivityResultRegistry$1.onStateChanged(ActivityResultRegistry.java:148)
        at androidx.lifecycle.LifecycleRegistry$ObserverWithState.dispatchEvent(LifecycleRegistry.java:354)
        at androidx.lifecycle.LifecycleRegistry.forwardPass(LifecycleRegistry.java:265)
        at androidx.lifecycle.LifecycleRegistry.sync(LifecycleRegistry.java:307)
        at androidx.lifecycle.LifecycleRegistry.moveToState(LifecycleRegistry.java:148)
        at androidx.lifecycle.LifecycleRegistry.handleLifecycleEvent(LifecycleRegistry.java:134)
        at androidx.lifecycle.ReportFragment.dispatch(ReportFragment.java:68)
        at androidx.lifecycle.ReportFragment.dispatch(ReportFragment.java:144)
        at androidx.lifecycle.ReportFragment.onStart(ReportFragment.java:109)
        at android.app.Fragment.performStart(Fragment.java:2244)
        at android.app.FragmentManagerImpl.moveToState(FragmentManager.java:1002)
        at android.app.FragmentManagerImpl.moveToState(FragmentManager.java:1148)
        at android.app.FragmentManagerImpl.moveToState(FragmentManager.java:1130)
        at android.app.FragmentManagerImpl.dispatchStart(FragmentManager.java:1960)
        at android.app.FragmentController.dispatchStart(FragmentController.java:163)
        at android.app.Activity.performStart(Activity.java:6267)
        at android.app.Activity.performRestart(Activity.java:6307)
        at android.app.Activity.performResume(Activity.java:6312)
        at android.app.ActivityThread.performResumeActivity(ActivityThread.java:3110)
        	... 8 more

When I actually try to capture the image, i get a black screen during the cropping step, however, i can see the cropped image after it was cropped, with the following log:

2023-05-15 16:31:38.098 27607-27607/com.websitebeaver.documentscanner.demo D/DEBUG: /storage/emulated/0/Android/data/com.websitebeaver.documentscanner.demo/files/Pictures/DOCUMENT_SCAN_0_20230515_163133-1312137198.jpg
2023-05-15 16:31:38.098 27607-27607/com.websitebeaver.documentscanner.demo D/DEBUG: File exists: true
2023-05-15 16:31:39.754 27607-27744/com.websitebeaver.documentscanner.demo D/OpenGLRenderer: Use EGL_SWAP_BEHAVIOR_PRESERVED: true
2023-05-15 16:31:39.810 27607-27744/com.websitebeaver.documentscanner.demo I/Adreno-EGL: <qeglDrvAPI_eglInitialize:379>: EGL 1.4 QUALCOMM build: Nondeterministic_AU_msm8974_LA.BF.1.1.3_RB1__release_AU (I3193f6e94a)
    OpenGL ES Shader Compiler Version: E<IP_ADDRESS>
    Build Date: 10/09/15 Fri
    Local Branch: mybranch15039904
    Remote Branch: quic/LA.BF.1.1.3_rb1.2
    Local Patches: NONE
    Reconstruct Branch: NOTHING
2023-05-15 16:31:39.817 27607-27744/com.websitebeaver.documentscanner.demo I/OpenGLRenderer: Initialized EGL, version 1.4
2023-05-15 16:31:39.847 27607-27744/com.websitebeaver.documentscanner.demo W/OpenGLRenderer: Bitmap too large to be uploaded into a texture (2340x4160, max=4096x4096)
2023-05-15 16:31:39.861 27607-27744/com.websitebeaver.documentscanner.demo W/OpenGLRenderer: Bitmap too large to be uploaded into a texture (2340x4160, max=4096x4096)
2023-05-15 16:31:39.861 27607-27744/com.websitebeaver.documentscanner.demo W/OpenGLRenderer: Bitmap too large to be uploaded into a texture (2340x4160, max=4096x4096)
2023-05-15 16:31:40.161 27607-27744/com.websitebeaver.documentscanner.demo W/OpenGLRenderer: Bitmap too large to be uploaded into a texture (2340x4160, max=4096x4096)
2023-05-15 16:31:40.162 27607-27744/com.websitebeaver.documentscanner.demo W/OpenGLRenderer: Bitmap too large to be uploaded into a texture (2340x4160, max=4096x4096)
2023-05-15 16:31:40.162 27607-27744/com.websitebeaver.documentscanner.demo W/OpenGLRenderer: Bitmap too large to be uploaded into a texture (2340x4160, max=4096x4096)

Finally when I run example from react-native-document-scanner-plugin on the same device, i get the similar behaviour to # 2(so take an image, image is visible, go to cropper, black screen), but it ultimately crashes after you crop it, with the following log:

2023-05-15 16:45:06.775 29203-29253/com.example.reactnativedocumentscanner W/OpenGLRenderer: Bitmap too large to be uploaded into a texture (2340x4160, max=4096x4096)
2023-05-15 16:45:06.775 29203-29253/com.example.reactnativedocumentscanner W/OpenGLRenderer: Bitmap too large to be uploaded into a texture (2340x4160, max=4096x4096)
2023-05-15 16:45:06.775 29203-29253/com.example.reactnativedocumentscanner W/OpenGLRenderer: Bitmap too large to be uploaded into a texture (2340x4160, max=4096x4096)
2023-05-15 16:45:07.091 29203-29253/com.example.reactnativedocumentscanner W/OpenGLRenderer: Bitmap too large to be uploaded into a texture (2340x4160, max=4096x4096)
2023-05-15 16:45:07.092 29203-29253/com.example.reactnativedocumentscanner W/OpenGLRenderer: Bitmap too large to be uploaded into a texture (2340x4160, max=4096x4096)
2023-05-15 16:45:07.093 29203-29253/com.example.reactnativedocumentscanner W/OpenGLRenderer: Bitmap too large to be uploaded into a texture (2340x4160, max=4096x4096)
2023-05-15 16:45:08.230 29203-29253/com.example.reactnativedocumentscanner W/OpenGLRenderer: Bitmap too large to be uploaded into a texture (2340x4160, max=4096x4096)
2023-05-15 16:45:08.246 29203-29253/com.example.reactnativedocumentscanner W/OpenGLRenderer: Bitmap too large to be uploaded into a texture (2340x4160, max=4096x4096)
2023-05-15 16:45:08.263 29203-29253/com.example.reactnativedocumentscanner W/OpenGLRenderer: Bitmap too large to be uploaded into a texture (2340x4160, max=4096x4096)
2023-05-15 16:45:08.280 29203-29253/com.example.reactnativedocumentscanner W/OpenGLRenderer: Bitmap too large to be uploaded into a texture (2340x4160, max=4096x4096)
2023-05-15 16:45:09.975 29203-29203/com.example.reactnativedocumentscanner I/Choreographer: Skipped 100 frames!  The application may be doing too much work on its main thread.
2023-05-15 16:45:09.983 29203-29203/com.example.reactnativedocumentscanner I/art: Rejecting re-init on previously-failed class java.lang.Class<com.reactnativedocumentscanner.-$$Lambda$DocumentScannerModule$Q4ouggW8dLKZVyX_qanZKL8Mvdo>
2023-05-15 16:45:09.995 29203-29203/com.example.reactnativedocumentscanner D/AndroidRuntime: Shutting down VM
2023-05-15 16:45:10.012 29203-29203/com.example.reactnativedocumentscanner E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.example.reactnativedocumentscanner, PID: 29203
    java.lang.NoClassDefFoundError: com.reactnativedocumentscanner.-$$Lambda$DocumentScannerModule$Q4ouggW8dLKZVyX_qanZKL8Mvdo
        at com.reactnativedocumentscanner.DocumentScannerModule.lambda$scanDocument$1(DocumentScannerModule.java:70)
        at com.reactnativedocumentscanner.-$$Lambda$DocumentScannerModule$VopbdLiWknFSlGfEZAKQOPDvLEo.invoke(lambda)
        at com.websitebeaver.documentscanner.DocumentScanner.handleDocumentScanIntentResult(DocumentScanner.kt:119)
        at com.reactnativedocumentscanner.DocumentScannerModule$1.onActivityResult(DocumentScannerModule.java:41)
        at com.facebook.react.bridge.ReactContext.onActivityResult(ReactContext.java:308)
        at com.facebook.react.ReactInstanceManager.onActivityResult(ReactInstanceManager.java:758)
        at com.facebook.react.ReactDelegate.onActivityResult(ReactDelegate.java:90)
        at com.facebook.react.ReactActivityDelegate.onActivityResult(ReactActivityDelegate.java:112)
        at com.facebook.react.ReactActivity.onActivityResult(ReactActivity.java:68)
        at android.app.Activity.dispatchActivityResult(Activity.java:6442)
        at android.app.ActivityThread.deliverResults(ActivityThread.java:3716)
        at android.app.ActivityThread.handleSendResult(ActivityThread.java:3763)
        at android.app.ActivityThread.-wrap16(ActivityThread.java)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1403)
        at android.os.Handler.dispatchMessage(Handler.java:102)
        at android.os.Looper.loop(Looper.java:148)
        at android.app.ActivityThread.main(ActivityThread.java:5443)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:728)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:618)
2023-05-15 16:45:12.755 29203-29203/com.example.reactnativedocumentscanner I/Process: Sending signal. PID: 29203 SIG: 9

Related to https://github.com/WebsiteBeaver/android-document-scanner/issues/29

@alexglazkov9 I might need to add a check to prevent Bitmap too large to be uploaded into a texture (2340x4160, max=4096x4096), but I don't think this is what causes color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'.
It seems like there are 2 different issues here.

You get a black screen during the crop step
OpenCV isn't able to read the image

We need to figure out why OpenCV can't read the image.
In the OpenCV docs it says

If the image cannot be read (because of missing file, improper permissions, unsupported or invalid format), the function returns an empty matrix ( Mat::data==NULL )

You verified that the file isn't missing. It's possible that you don't have permission to read the file.
Are you able to consistently reproduce the color.cpp bug? If you are, can you try to manually read an image that has smaller dimensions?
https://github.com/WebsiteBeaver/android-document-scanner/blob/master/documentscanner/src/main/java/com/websitebeaver/documentscanner/utils/ImageUtil.kt
val image: Mat = Imgcodecs.imread(filePath)

I have made a pull request here
https://github.com/WebsiteBeaver/android-document-scanner/pull/32
bare in mind my changes require minimum android sdk 29
Maybe someone can splice old and new codes together

Fixed in https://github.com/WebsiteBeaver/android-document-scanner/pull/33 by making sure image isn't empty before calling cvtColor. If it is try reading image without OpenCV.


projectmembers api call 503's after turning auth off

Rancher Version: v1.2.0-pre4-rc7 UI v1.2.20
Steps to Reproduce:

Turn on auth (i do not believe it matters which but I used github and local)
Add new members to your default environment
Turn off auth
Navigate to the edit environment page for the default environment.

Results: The api call to projectmembers returns a 503
Expected: I should be able to get project members, if there is an error retrieving project members that were added when auth is on we should handle this more gracefully on the api side so that the entire UI doesnt blow up and users can still edit there environments.

We cannot get details of the project members that were added when auth was on, after auth is turned off. Since we lose the data to contact the thirdparty auth provider when we disabled auth.
Will fix such that we dont error out with 503.

I tested with master build (22 Nov), the project members doesn't return 503 anymore.


CUDA error trying to run demo following requirements

Hi,
I have installed using the following requirements
Ubuntu 16 on P2xlarge AWS Tesla k80
NVIDIA-SMI 384.130 driver
CUDA release 9.0, V9.0.176
CUDNN libcudnn7_<IP_ADDRESS>-1+cuda9.0_amd64.deb
Pytorch==1.0.0, scipy==1.2.0
Python3
All modules were compiled without errors
but when running the example CUDA_VISIBLE_DEVICES=0 python demo_MiddleBury.py
I got the following error:
revise the unique id to a random numer 91561
Namespace(SAVED_MODEL=None, alpha=[0.0, 1.0], arg='./model_weights/91561-Thu-May-21-18:36/args.txt', batch_size=1, channels=3, ctx_lr_coe=1.0, datasetName='Vimeo_90K_interp', datasetPath='', dataset_split=97, debug=False, depth_lr_coe=0.001, dtype=<class 'torch.cuda.FloatTensor'>, end_frame=100, epsilon=1e-06, factor=0.2, filter_lr_coe=1.0, filter_size=4, flow_lr_coe=0.01, force=False, frame_input_dir='/content/DAIN/input_frames', frame_output_dir='/content/DAIN/output_frames', log='./model_weights/91561-Thu-May-21-18:36/log.txt', lr=0.002, netName='DAIN', no_date=False, numEpoch=100, occ_lr_coe=1.0, patience=5, rectify_lr=0.001, save_path='./model_weights/91561-Thu-May-21-18:36', save_which=1, seed=1, start_frame=1, time_step=0.5, uid=None, use_cuda=True, use_cudnn=1, weight_decay=0, workers=8)
cudnn is used
The testing model weight is: ./model_weights/best.pth
The unique id for current testing is: 85504
RubberWhale
/home/ubuntu/.local/lib/python3.5/site-packages/torch/nn/modules/upsampling.py:129: UserWarning: nn.UpsamplingNearest2d is deprecated. Use nn.functional.interpolate instead.
warnings.warn("nn.{} is deprecated. Use nn.functional.interpolate instead.".format(self.name))
/home/ubuntu/.local/lib/python3.5/site-packages/torch/nn/modules/upsampling.py:129: UserWarning: nn.Upsample is deprecated. Use nn.functional.interpolate instead.
warnings.warn("nn.{} is deprecated. Use nn.functional.interpolate instead.".format(self.name))
/home/ubuntu/.local/lib/python3.5/site-packages/torch/nn/functional.py:2423: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
error in correlation_forward_cuda_kernel: no kernel image is available for execution on the device
Traceback (most recent call last):
File "demo_MiddleBury.py", line 131, in
y_s,offset,filter = model(torch.stack((X0, X1),dim = 0))
File "/home/ubuntu/.local/lib/python3.5/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/ubuntu/DAIN/networks/DAIN.py", line 152, in forward
time_offsets=time_offsets[::-1])
File "/usr/lib/python3.5/contextlib.py", line 77, in exit
self.gen.throw(type, value, traceback)
File "/home/ubuntu/.local/lib/python3.5/site-packages/torch/cuda/init.py", line 326, in stream
yield
File "/home/ubuntu/DAIN/networks/DAIN.py", line 149, in forward
self.forward_flownets(self.flownets, cur_offset_input, time_offsets=time_offsets),
File "/home/ubuntu/DAIN/networks/DAIN.py", line 205, in forward_flownets
temp = model(input)  # this is a single direction motion results, but not a bidirectional one
File "/home/ubuntu/.local/lib/python3.5/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/ubuntu/DAIN/PWCNet/PWCNet.py", line 221, in forward
corr6 = self.corr(c16, c26)
File "/home/ubuntu/.local/lib/python3.5/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, kwargs)
File "/home/ubuntu/DAIN/PWCNet/correlation_package_pytorch1_0/correlation.py", line 59, in forward
result = CorrelationFunction(self.pad_size, self.kernel_size, self.max_displacement,self.stride1, self.stride2, self.corr_multiply)(input1, input2)
File "/home/ubuntu/DAIN/PWCNet/correlation_package_pytorch1_0/correlation.py", line 27, in forward
self.pad_size, self.kernel_size, self.max_displacement,self.stride1, self.stride2, self.corr_multiply)
RuntimeError: CUDA call failed (correlation_forward_cuda at correlation_cuda.cc:80)
frame #0: std::function<std::string ()>::operator()() const + 0x11 (0x7f3d75f03fe1 in /home/ubuntu/.local/lib/python3.5/site-packages/torch/lib/libc10.so)
frame #1: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x2a (0x7f3d75f03dfa in /home/ubuntu/.local/lib/python3.5/site-packages/torch/lib/libc10.so)
frame #2: correlation_forward_cuda(at::Tensor&, at::Tensor&, at::Tensor&, at::Tensor&, at::Tensor&, int, int, int, int, int, int) + 0x5e6 (0x7f3d7299ef26 in /usr/local/lib/python3.5/dist-packages/correlation_cuda-0.0.0-py3.5-linux-x86_64.egg/correlation_cuda.cpython-35m-x86_64-linux-gnu.so)
frame #3:  + 0x16042 (0x7f3d729ab042 in /usr/local/lib/python3.5/dist-packages/correlation_cuda-0.0.0-py3.5-linux-x86_64.egg/correlation_cuda.cpython-35m-x86_64-linux-gnu.so)
frame #4:  + 0x1627e (0x7f3d729ab27e in /usr/local/lib/python3.5/dist-packages/correlation_cuda-0.0.0-py3.5-linux-x86_64.egg/correlation_cuda.cpython-35m-x86_64-linux-gnu.so)
frame #5:  + 0x12e76 (0x7f3d729a7e76 in /usr/local/lib/python3.5/dist-packages/correlation_cuda-0.0.0-py3.5-linux-x86_64.egg/correlation_cuda.cpython-35m-x86_64-linux-gnu.so)

frame #9: python3() [0x4ec9a3]
frame #11: python3() [0x4fc63e]
frame #14: THPFunction_do_forward(THPFunction, _object) + 0x15c (0x7f3db0235bdc in /home/ubuntu/.local/lib/python3.5/site-packages/torch/lib/libtorch_python.so)
frame #17: python3() [0x5b4846]
frame #21: python3() [0x4ecab7]
frame #25: python3() [0x4ec9a3]
frame #27: python3() [0x4fc63e]
frame #29: python3() [0x5b4846]
frame #33: python3() [0x4ecab7]
frame #37: python3() [0x4ec9a3]
frame #39: python3() [0x4fc63e]
frame #41: python3() [0x5b4846]
frame #44: python3() [0x54548f]
frame #47: python3() [0x4ecab7]
frame #51: python3() [0x4ec9a3]
frame #53: python3() [0x4fc63e]
frame #55: python3() [0x5b4846]
frame #58: python3() [0x544f43]
frame #60: python3() [0x622642]
Please help!

I have tested CUDA and pytorch using this demo code and it ran correctly on gpu
https://github.com/jcjohnson/pytorch-examples pytorch tensors
Also tested CUDNN with MNIST sample code from CUDNN
All installation appear to be normal
Python version is 3.5.2

Solved the issue adding compute_30 to all setup.py and recompiling all torch modules.
It was necesary for Tesla K80

@AlverGant hi bro, i try to add compute_30 like this

and recompiling modules but error persist.

@AlverGant hi bro, i try to add compute_30 like this

and recompiling modules but error persist.

@AlverGant hi bro, i try to add compute_30 like this

and recompiling modules but error persist.

Hi @Exspiravit that is exactly what I did, is your GPU a Tesla K80? If not you have to set it accordingly as http://arnon.dk/matching-sm-architectures-arch-and-gencode-for-various-nvidia-cards/
Are you using CUDA 9.0? and CUDNN 7 <IP_ADDRESS>-1?

Yes i use gpu tesla k80 and have cudnn with cuda 10.0, i dont know if i have some diferences of you.

You are having the same error that I had "cuda call failed" the differences is that I am using cuda 9 and torch 1.0.0 as recommended in the requirements from the developer

And I am using an older driver NVIDIA-SMI 384.130

Colab Pro allows for tesla V100 gpus, which require      '-gencode', 'arch=compute_70,code=sm_70',
in the compiler args file. "DAIN/my_package/compiler_args.py"
By default, '-gencode', 'arch=compute_75,code=sm_75', is uncommented. In order for me to fix it, I needed to uncomment line 44 which has '-gencode', 'arch=compute_70,code=sm_70', and maybe this isn't the correct way, but I commented line 42 which had  '-gencode', 'arch=compute_75,code=sm_75'.
I then recompiled DAIN and DAIN PyTorch. Sorry if this is confusing, I'm not exactly experienced at this stuff.

If you do not want to build CUDA programs.
We provide the CUPY version of those packages.
The cupy files do not need to be built.
please refer to:
https://github.com/laomao0/cupy_packages


Regression in ffmpeg 6

There is a regression since ffmpeg 6 that breaks the segmenting.
Test script:
# Generate test file of 20 sec, with keyframes every 2sec
ffmpeg -loglevel warning -fflags +genpts \
    -f lavfi -i testsrc=duration=20:size=1280x720:rate=30 \
    -f lavfi -i sine=frequency=1000:duration=20 \
    -c:v libx264 -g 60 \
    -c:a aac -b:a 192k \
    -f mp4 -y -movflags +faststart /tmp/test.mp4;

# Seek to 10 sec and create three 2s segments (at 12,14,16)
ffmpeg -loglevel warning -ss 10 -i /tmp/test.mp4 -to 16 -copyts -c copy \
    -f segment \
    -segment_time_delta 0.2 \
    -segment_format mpegts \
    -segment_times 12,14,16 \
    -segment_start_number 5 \
    -segment_list_type csv \
    -segment_list pipe:1 /tmp/test_%03d.ts;

We can set up a simple dockerfile:
ARG VERSION=3.22
FROM alpine:${VERSION}
RUN apk add --no-cache ffmpeg
ENTRYPOINT [ "ffmpeg" ]

And then run scripts above with different versions.
docker build --build-arg VERSION=3.14 -t alpine-ffmpeg:3.14 .
docker run --rm alpine-ffmpeg:3.14 -version

Versions prior 6
ffmpeg version 4.2.4 (from alpine:3.11)
ffmpeg version 4.3.1 (from alpine:3.12)
ffmpeg version 4.3.3 (from alpine:3.13)
ffmpeg version 4.4.1 (from alpine:3.14)
ffmpeg version 5.0.3 (from alpine:3.16)
ffmpeg version 5.1.4 (from alpine:3.17)
test_005.ts,0.000000,12.000000
test_006.ts,12.000000,14.000000
test_007.ts,14.000000,16.000000
test_008.ts,16.000000,16.166667

Versions after 6
ffmpeg version 6.0.1 (from alpine:3.18)
ffmpeg version 6.1.1 (from alpine:3.19)
ffmpeg version 6.1.2 (from alpine:3.21)
test_005.ts,0.000000,16.166667

It seems that it requires now relative offet to the seeked time in segment_times instead of the absolute time.
Modified segmenter script:
ffmpeg -loglevel warning -ss 10 -i /tmp/test.mp4 -to 16 -copyts -c copy \
    -f segment \
    -segment_time_delta 0.2 \
    -segment_format mpegts \
    -segment_times 2,4,6 \
    -segment_start_number 5 \
    -segment_list_type csv \
    -segment_list pipe:1 /tmp/test_%03d.ts;

New output:
test_005.ts,0.000000,12.000000
test_006.ts,12.000000,14.000000
test_007.ts,14.000000,16.000000
test_008.ts,16.000000,16.166667

Hotfix in a4b556b2ceec81a3362d712f86e21e34feeef895


Disable Kzg when it's not needed

PR Description
When Deneb is enabled and vc is started separately it throws on trustedSetupPath missing. Actually it's not needed on vc side and for the most subcommands. Disabled
Fixed Issue(s)

Documentation

[x] I thought about documentation and added the doc-change-required label to this PR if updates are required.

Changelog

[x] I thought about adding a changelog entry, and added one if I deemed necessary.

@rolfyone I've reworked the PR to avoid passing anything to the builder.
Plus, @tbenr discovered that PR still fails with --network=auto which surfaced another failing area - spec building. Fixed it too.

actually is this the actual kzg_noop config item? because if it is we should maybe leave it?

@tbenr done
@rolfyone what do you mean by leave it?


[web_sys] no method named 'document' found for struct "Window"

Describe the Bug
no method named document found for struct web_sys::features::gen_Window::Window
I using rust-webpack-template
I referring RustでWebAssemblyやってみた。
Error Message
no method named `document` found for struct `web_sys::features::gen_Window::Window` in the current scope
method not found in `web_sys::features::gen_Window::Window`rustc(E0599)
lib.rs(27, 27): method not found in `web_sys::features::gen_Window::Window`

Source Code
lib.rc
use wasm_bindgen::prelude::*;
use web_sys;

// When the `wee_alloc` feature is enabled, this uses `wee_alloc` as the global
// allocator.
//
// If you don't want to use `wee_alloc`, you can safely delete this.
#[cfg(feature = "wee_alloc")]
#[global_allocator]
static ALLOC: wee_alloc::WeeAlloc = wee_alloc::WeeAlloc::INIT;

// This is like the `main` function, except for JavaScript.
#[wasm_bindgen(start)]
pub fn main_js() -> Result<(), JsValue> {
    // This provides better error messages in debug mode.
    // It's disabled in release mode so it doesn't bloat up the file size.
    #[cfg(debug_assertions)]
    console_error_panic_hook::set_once();

    // Your code goes here!
    let window = web_sys::window().expect("no global `window` exists");

    window.alert_with_message("Hello world!");

    let document = window.document().expect("should have a document on window");

    let body = document.body().expect("document should have a body");
    let val = document.create_element("p")?;
    val.set_inner_html("Hello from Rust!");
    body.append_child(&val)?;

    Ok(())
}

Cargo.toml
# You must change these to your own details.
[package]
name = "rust-webpack-template"
description = "My super awesome Rust, WebAssembly, and Webpack project!"
version = "0.1.0"
authors = ["You<EMAIL_ADDRESS>categories = ["wasm"]
readme = "README.md"
edition = "2018"

[lib]
crate-type = ["cdylib"]

[profile.release]
# This makes the compiled code faster and smaller, but it makes compiling slower,
# so it's only enabled in release mode.
lto = true

[features]
# If you uncomment this line, it will enable `wee_alloc`:
#default = ["wee_alloc"]

[dependencies]
# The `wasm-bindgen` crate provides the bare minimum functionality needed
# to interact with JavaScript.
wasm-bindgen = "0.2.45"

# `wee_alloc` is a tiny allocator for wasm that is only ~1K in code size
# compared to the default allocator's ~10K. However, it is slower than the default
# allocator, so it's not enabled by default.
wee_alloc = { version = "0.4.2", optional = true }

# The `web-sys` crate allows you to interact with the various browser APIs,
# like the DOM.
[dependencies.web-sys]
version = "0.3.22"
features = ["console", "Window"]

# The `console_error_panic_hook` crate provides better debugging of panics by
# logging them with `console.error`. This is great for development, but requires
# all the `std::fmt` and `std::panicking` infrastructure, so it's only enabled
# in debug mode.
[target."cfg(debug_assertions)".dependencies]
console_error_panic_hook = "0.1.5"

# These crates are used for running unit tests.
[dev-dependencies]
wasm-bindgen-test = "0.2.45"
futures = "0.1.27"
js-sys = "0.3.22"
wasm-bindgen-futures = "0.3.22"

The cause was not adding "Document" to featuers of Cargo.toml.


Should we add Revise.jl

Should we add Revise.jl to have update in the boat without restarting it, at least during the dev phase?
It might still be buggy though, but if use in sync with tmux, it should be okay. What do you all think?

Actually, I couldn't get it to work even when I run it locally. It seems that Revise requires some kind of trigger. Normally I would go to the REPL and make a call to the function that I just changed but this is a running process so it's different. It would be nice if you can get it work first.

The manual trigger we need to put into a callback is Revise.revise(). As a last resort, we could just add !revise as a bot command. That most probably would be both, a lot easier than putting it into some Discord.jl-internal routines and at the same time less of a performance hit because we don't run it all times but only on request. We might even restrict its usage to mods. The only other point we need to take into account is that we might have to switch some ordinary function calls with invokelatest calls. Like there: https://github.com/timholy/Revise.jl/issues/194

Due to the way how the handlers are called in Discord.jl (via call operator on array/dict elements) we don't even need invokelatest. Just a Revise.revise() AFAIU

so it might be as easy as
function commander(c::Client, m::Message, ::Val{:revise})
    Revise.revise()
end


Create an initial mock bank (ASPSP) SpringBoot project for Open Banking

Story
As a TPP
I want to be able to test my application against a bank simulator
so that I can be assured my application will work against a real ASPSP
Acceptance criteria

The mock bank supports v3.0 onwards of the Open Banking Read/Write REST API.
The mock bank does not authenticate or authorise the client - any client with appropriate network access can send/receive HTTP requests/responses.
Open Banking API objects are converted to FR domain objects and stored in mongo.

Tasks
List of tasks required to implement this story

[x] Create initial SpringBoot maven based project with the groupId com.forgerock.sba.aspsp and artifactId forgerock-sba-mock-aspsp.
[x] Port "common" code (e.g. repositories, mongo document classes, FR domain model and converters) from openbanking-aspsp.
[x] Copy the required snippets of code from openbanking-openbanking-model, thereby allowing a self contained project.
[x] Refactor the package structure, making the separation of layers more obvious.
[x] Use embedded MongoDB for SpringBoot tests, rather than docker-compose.
[x] Generate the OB model objects within this project (e.g. as a sub-module).
[x] Add a simple example controller for the payments API (to prove the project works end-to-end using postman).

Release Notes
Affected App: forgerock-sba-mock-aspsp

All tasks complete.


⚠ Jenkins Plugins Artifactory has degraded performance

In e541601, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1257 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 0b417ee after 12 minutes.


abort_source: move-assign operator: call base class unlink

calling node_algorithms::unlink(this_ptr()) with an engaged subscription results in use-after-free that is reproduced by the added unit test.
Fixes #1336
Signed-off-by: Benny Halevy<EMAIL_ADDRESS>

./build/release/tests/unit/abort_source_test
Running 7 test cases...
....
n not permitted)
random-seed=1432522134

*** No errors detected


onClickAction not working on Kotlin?

Hi, I am having the following problem with the following code:
 val dynamicItem = MaterialAboutActionItem.Builder()
                .text("aaa")
                .subText("bbb")
                .icon(IconicsDrawable(c)
                        .icon(CommunityMaterial.Icon.cmd_refresh)
                        .color(ContextCompat.getColor(c, colorIcon)
                        ).sizeDp(18))
                .build()

        dynamicItem.onClickAction = MaterialAboutItemOnClickAction {
            dynamicItem.subText = "ccc"
        }
        otherCardBuilder.addItem(dynamicItem)

I want to change the subtext from dynamic item to "ccc" when someone taps it. The 'onClickAction' is working fine, it is always called. However, I can't manage to solve why the method.subText isn't working. I am using the latest Kotlin version (1.2.10) and never had a problem before today.

Hi, sorry about the late response. Been caught up over the Christmas period. Have you tried calling refreshMaterialAboutList(); after changing the subText?

Works fine, thanks!
I would have never guessed this.


Multipart request ignores/overwrites user defined Content-Type

Expected Behavior
When submitting a POST multipart request, the boundary parameter that Insomnia inserts (boundary=X-INSOMNIA-BOUNDARY) into the Content-Type header should be appended to the value that the user has already specified.  This allows for custom media types such as application/vnd.github or additional parameters like ;charset=UTF-8 which may be required by the server.
Actual Behavior
Right now the Content-Type header specified by the user is completely replaced with this:
Content-Type: multipart/form-data; boundary=X-INSOMNIA-BOUNDARY

Reproduction Steps
No response
Is there an existing issue for this?

[X] I have searched the issue tracker for this problem.

Additional Information
No response
Insomnia Version
2022.3.0
What operating system are you using?
Windows
Operating System Version
Windows 10 Enterprise version 20H2
Installation method
Download installer
Last Known Working Insomnia version
No response

Hi @john3300, thank you for reporting this!
I can reproduce this as well:

I'm not entirely sure if feature-wise this is something Insomnia should support though, as with other tools, the Content-Type header is also overwritten as well or locked out from the user to edit it. Here's an example from another API client:

This will need some internal discussion. We'll update you here if there's any changes or fixes in the meantime.

@filfreire I was also having trouble setting the 'Content-Type' header on Insomnia and just tried it on Postman. While Postman does set Content-Type: multipart/form-data; boundary= uneditable header by default, you can 'uncheck' it and set your own Content-Type header like so:

It would be nice to have something like this on Insomnia as well.

Thanks for the confirmation and update.
Given that this prevents me from being able to use this tool with certain
servers that require non-standard media types I would argue that the other
API client is wrong too.  Two wrongs don't make a right.  :-)
On Mon, May 16, 2022 at 4:25 AM Filipe Freire @.***>
wrote:

Hi @john3300 https://github.com/john3300, thank you for reporting this!
I can reproduce this as well:
[image: Screenshot 2022-05-16 at 10 20 26]
https://user-images.githubusercontent.com/11976836/168560944-35a46124-30ae-43fe-bd8e-be8505e54ea1.png
I'm not entirely sure if feature-wise this is something Insomnia should
support though, as with other tools, the Content-Type header is also
overwritten as well or locked out from the user to edit it. Here's an
example from another API client:
[image: Screenshot 2022-05-16 at 10 23 03]
https://user-images.githubusercontent.com/11976836/168561776-81438f4f-2f5a-4e65-b3c4-2cf6d217b4a1.png
This will need some internal discussion. We'll update you here if there's
any changes or fixes in the meantime.
—
Reply to this email directly, view it on GitHub
https://github.com/Kong/insomnia/issues/4780#issuecomment-1127437774,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACRLEWQXPOVY5NAMIJBHTVTVKIIAXANCNFSM5V3OK7TA
.
You are receiving this because you were mentioned.Message ID:
@.***>

a


Modify PR's fields

Hi,
I really appreciate this extension, because we had to do multiples cherry-picks sometimes (~10).
But, we had to modify too the title, the description and the work items linked to the PR. And we can't do this for now.
I'll keep watching this evo about these issues.
Thanks for this work.

Hi,
Please, add a link to the pull request from where the cherry peak is made in the description of the new pull request, like this:
Cherry picked from !number
Thanks.

Thanks for using the extension! We'll investigate making these additions, but we'd also be open to taking a PR 😄

These suggestions have been implemented in #18 - we're looking forward to hearing how it works for you!


Directory wildcard problem

lsd 0.16.0 is omitting directory names when trying to target a specific file across multiple directories via wildcard:
mkdir a b c
touch {a,b,c}/test.txt
# Normal ls output
ls */test.txt
a/test.txt  b/test.txt  c/test.txt
# lsd output
lsd --icon never */test.txt
test.txt  test.txt  test.txt

@Peltoche @meain
https://github.com/dvvvvvv/lsd/tree/relative-path
I've been working on this issue for a few days. The above branch contains the following modifications:

meta::Name includes PathBuf instead of String

Name::render function accepts DisplayOption enum value
DisplayOption::FileName: draw file name(using when drawing tree)
DisplayOption::Relative{ base_path }: draw relative path to base_path
DisplayOption::None: draw path it self

the file path passed directly has None DisplayOption. so it can draw file paths like ls

$ lsd ../Cargo.*
 ../Cargo.lock   ../Cargo.toml
$ lsd meta/*.rs
 meta/date.rs        meta/inode.rs   meta/owner.rs         meta/symlink.rs
 meta/filetype.rs    meta/mod.rs     meta/permissions.rs   meta/windows_utils.rs
 meta/indicator.rs   meta/name.rs    meta/size.rs
$ lsd /home/dvvv/src/lsd/src/*.rs
 /home/dvvv/src/lsd/src/app.rs       /home/dvvv/src/lsd/src/flags.rs
 /home/dvvv/src/lsd/src/color.rs     /home/dvvv/src/lsd/src/icon.rs
 /home/dvvv/src/lsd/src/core.rs      /home/dvvv/src/lsd/src/main.rs
 /home/dvvv/src/lsd/src/display.rs   /home/dvvv/src/lsd/src/sort.rs

I realized while writing. The relative path option does not seem to be necessary. wow

but This doesn't seem to be rusty code. The Name structure must always have a PathBuf: This is a duplicate of the Meta structure.
I've tried changing it to a Path reference that refers to Meta's PathBuf, but that's not possible because move occurs when the Meta structure is created(as far as I know) .
I've tried changing the Name structure to a Path structure, but that doesn't fit the module named Meta.
I would like to hear you guys opinion.

@dvvvvvv Would you mind to create a PR with your change so we can review it?

hi all, any update on this? this is the only use case where i have to drop back to /bin/ls

This should be fixed now.
Feel free to reopen if this issue still persistes.


Rich validation errors

So far EOF validation implementations return their own error code it is rather difficult to match them one-to-one.
Here is an idea to extend the return type to a tuple:

error code (8 bits) (as it is now)
error category (8 bits)
element index (32 bits?)

The error category is "more standardized" way of informing at what phase of validation the error happened. E.g. header, body, type section, code section, subcontainer.
The index informs at which element the error happened. We need to find a global or category-based recursive indexing.
First of all, we need to find a convenient method of numbering all subcontainers.

This is the categorization currently used in the EOF validation fuzzer.
EOFErrCat get_cat(evmone::EOFValidationError err) noexcept {
  using enum evmone::EOFValidationError;
  switch (err) {
  case invalid_prefix:
  case eof_version_unknown:
  case header_terminator_missing:
  case type_section_missing:
  case code_section_missing:
  case data_section_missing:
  case section_headers_not_terminated:
  case zero_section_size:
  case incomplete_section_size:
  case incomplete_section_number:
  case too_many_code_sections:
  case too_many_container_sections:
  case invalid_type_section_size:
    return EOFErrCat::header;
  case invalid_section_bodies_size:
    return EOFErrCat::body;
  case invalid_first_section_type:
  case inputs_outputs_num_above_limit:
  case max_stack_height_above_limit:
  case toplevel_container_truncated: // ?
    return EOFErrCat::type;
  case undefined_instruction:
  case truncated_instruction:
  case invalid_container_section_index:
  case stack_underflow: // stack?
  case unreachable_instructions:
  case invalid_code_section_index:
  case invalid_rjump_destination:
  case callf_to_non_returning_function:
  case invalid_dataloadn_index:
  case no_terminating_instruction:
  case invalid_non_returning_flag:         // ?
  case invalid_max_stack_height:           // stack?
  case stack_height_mismatch:              // stack?
  case stack_overflow:                     // stack?
  case stack_higher_than_outputs_required: // stack?
  case incompatible_container_kind:
  case jumpf_destination_incompatible_outputs:
    return EOFErrCat::code;
  case unreferenced_subcontainer:
  case eofcreate_with_truncated_container:
  case ambiguous_container_kind:
    return EOFErrCat::subcont;
  case unreachable_code_sections:
    return EOFErrCat::ref;
  case container_size_above_limit:
    return EOFErrCat::other;
  case success:
    assert(!"success");
    __builtin_trap();
  case impossible:
    assert(!"impossible");
    __builtin_trap();
  }
}

This starts to move into specifying an implementation instead of a data structure.  Besu layout parsing integrates the header parsing and header content parsing, so it always gets order different than evmone.  EEST accepted multiple error codes in https://github.com/ethereum/execution-spec-tests/pull/759, which besu has adapted our error codes to include.

So if they fail for different categories it's a fuzz break?

That's the intention. If we cannot make it work there is no point in standardizing it.


Dismissing from application switcher closes the switcher

Open the application switcher from the bottom right android button.
Swipe to dismiss ChatSecure.
See the application switcher close.
This is on Android 4.4.

Is this closed because it's fixed?

actually, it is not. apologies! re-opening...


A question of  compute P and Q weithou P = I - P, Q = I - Q

def matrix_pade_approximant(p,I):     p_sqrt = pade_p[0]*I     q_sqrt = pade_q[0]*I     p_app = I - p     p_hat = p_app     for i in range(5):         p_sqrt += pade_p[i+1]*p_hat         q_sqrt += pade_q[i+1]*p_hat         p_hat = p_hat.bmm(p_app)     return torch.linalg.solve(q_sqrt, p_sqrt)
why not  p_sqrt = I - p_sqrt,q_sqrt = I - q_sqrt, This seems inconsistent with the formula in the paper

Hi,
Thanks for your interest in our work!
This is because we initialize P and Q with: p_sqrt = pade_p[0]*I, q_sqrt = pade_q[0]*I (here pade_p[0] and pade_q[0] are both 1), and the pade' coefficients pade_p[i>0] and pade_q[i>0] are negative in the code (but assumed positive in the paper by default).
Your idea is correct if we rewrite the code as follows:
   def matrix_pade_approximant(p,I):
    p_sqrt = 0.
    q_sqrt = 0.
    p_app = I - p
    p_hat = p_app
    for i in range(5):
        p_sqrt += pade_p[i+1]*p_hat
        q_sqrt += pade_q[i+1]*p_hat
        p_hat = p_hat.bmm(p_app)
   p_sqrt = I + p_sqrt
   q_sqrt = I + q_sqrt
   return torch.linalg.solve(q_sqrt, p_sqrt)

Let me know if you have further concerns!
Kind regards,
Yue Song

Hi,
Thanks for your interest in our work!
This is because we initialize P and Q with: p_sqrt = pade_p[0]*I, q_sqrt = pade_q[0]*I (here pade_p[0] and pade_q[0] are both 1), and the pade' coefficients pade_p[i>0] and pade_q[i>0] are negative in the code (but assumed positive in the paper by default).
Your idea is correct if we rewrite the code as follows:
   def matrix_pade_approximant(p,I):
    p_sqrt = 0.
    q_sqrt = 0.
    p_app = I - p
    p_hat = p_app
    for i in range(5):
        p_sqrt += pade_p[i+1]*p_hat
        q_sqrt += pade_q[i+1]*p_hat
        p_hat = p_hat.bmm(p_app)
   p_sqrt = I + p_sqrt
   q_sqrt = I + q_sqrt
   return torch.linalg.solve(q_sqrt, p_sqrt)

Let me know if you have further concerns!
Kind regards,
Yue Song

Thank you for your reply.

Hi,
the input matrix must be square matrix? Because of  $I$ is Identity matrix.

Yes, the matrix should be square and symmetric; otherwise there would be no principal square root.

Yes, the matrix should be square and symmetric; otherwise there would be no principal square root.

Thanks for your reply.


Refactor and add magnetostatics features

Currently we have some specific magnetic field generation routines. We can generalize the functions using steady current coils, and add direct Bio-Savart calculation.

First step implemented in 3f922d7; more to come later


Implement cooldown periods

The cooldown periods in Status Staking Summary are not implemented.
UPDATE: Explanding this.
There's two "periods" that need to be implemented: deposit_cooldown_period and withdrawal_cooldown_period
These are meant to work as follows:

depost_cooldown_period starts when accounts stake funds without_ a lock up time. As long as accounts are within the deposit_cooldown_period, they can immediately unstake their funds (mutliplier points will be lost then)
withdrawal_cooldown_period starts after accounts signalled they want to unstake and their deposit_cooldown_period has passed.  This period should also apply to the process of claiming rewards (rewards will stay in rewardCollector vault until then but can be immediately "re-staked")

This issue is considered done when the following has been implemented:

deposit_cooldown_period functionality when unstaking funds

Unit tests
documentation
Application properties
Bonus: formal verification

withdrawal_cooldown_period functionality when unstaking funds

Unit tests
docs
Application properties
Bonus: formal verification

Notice that implementing withdrawal_cooldown_period for reward collector vault depends on #34 .
So it's possible to first only implement it for unstake()

Updated the issue description accordingly.
@mart1n-xyz please let me know if there's anything wrong or a detail missing you think should be added here.

Looks good to me

This means, users will have to send an additional transaction to signal the StakeManager that their account can be processed.
I think this can be executed in the first processAccount() call. Then, I guess, this would need to be managed by StakeManager. However, I think it would simplify the rest of the discussion, no? Or is this not feasible?

We simply don't allow locking of funds that are still in deposit cooldown and wait for cooldown to finish
We signal staking to the StakeManager with the existing funds, retroactively mind the rewards + multiplier points for the already passed time, then set the lock for the funds moving forward.
From a dev perspective, I'd prefer 1. for simplicity. On the contrary, as a user, I know the frustration that you deposit the stake and it is not yet earning anything - it is a strong incentive to lock.

Scenario 5
I think the following would be a good compromise here:

We can also treat a vault that has a depositCooldownUntil > 0 as "locked", meaning, users have to wait until deposit cooldown is over, until they can add more stake to the vault (or they can spin up a new vault if they don't want to wait)

However, as of now, if I'm understanding correctly, this will cause the account to progress in epochs. The question then is: what happens when the deposit cooldown period for the first round of funds is over? Can we still retroactively mint rewards and MP for those, even though the account is already much further down in terms of epochs?
I think this should be possible if we indeed track the cooldown in StakeManager. We can predict on aggregate how many MPs and stake will exit cooldown and become active in the next epoch (recorded at stake transaction). And the rest of the user triggered calculations can happen with the first processAccount().

You make a good point that all deposits within an epoch are treated the same - as a single deposit. With 2 week cooldown and 1 week epoch, we can have max 1 epoch of additional deposit. Not sure it helps but it gives some specifics to scenarios.

I think this can be executed in the first processAccount() call. Then, I guess, this would need to be managed by StakeManager. However, I think it would simplify the rest of the discussion, no? Or is this not feasible?

@mart1n-xyz so the idea with this approach would be: basically don't even let the StakeManager know that a vault has some funds and only let the StakeManager know ones the deposit cooldown in the vault has elapsed.
In other words, I'd probably introduce two new functions (and change one)

StakeVault.deposit(uint256 amount) which just transfer funds to vault, without letting manager know (locktime is 0), deposit cooldown is initialized
StakeVault.stake(amount) can be called when deposit cooldown is over (registers funds with manager, starts processAccount), but on longer transfer tokens (as this has already happened.
StakeVault.depositAndStake(uint256 amount, uint256 increaseLocktime) - The same as the above two combined, basically when staking with a lock up time

Something along those lines.

From a dev perspective, I'd prefer 1. for simplicity. On the contrary, as a user, I know the frustration that you deposit the stake and it is not yet earning anything - it is a strong incentive to lock.

So here, I think it's important to keep in mind that: due to the deposit cooldown you'll not earn anything anyways, if you decide to unstake before deposit cooldown is elapsed and,
Once deposit cooldown is over and you actually register with the manager, you will mint your rewards retroactively.
So one could say that your funds are earning rewards silently.

I'm starting to implement some of the ideas discussed in the issue.
While I'm crafting a test for a new deposit(uint256 amount) function, which is bascially transfering funds to the vault and initializing the depositCooldownUntil marker, I realized there's another thing that needs clarification, even if we block additional deposits from happening while there's a cooldown.
Imagine the following scenario:

User deposits funds into vault (deposit cooldown is initialized)
During cooldown, no more funds can be deposited
During cooldown, funds that are cooling down can be withdrawn
User waits for cooldown to finish
User can call stake() to actually stake (and retroactively mint rewards)
However, user decides to deposit() again (which is now possible due to lockup period being over)
depositCooldownUntil gets extended by cooldown period

^ Here, if we keep a trivial check in stake() that block.timestamp has to be > depositCooldownUntil, the user is not able to stake() the funds from the first deposit(). The new cooldown threshold prevents the user from staking funds that went through their own cooldown already.
How do we want to handle this? I see a few options:

In addition to a "withdrawable" balance, we'll keep track of stakeableBalance, which is the amount of any deposit that went through the cooldown period. Then, when a user calls stake(amount) we'll operate on stakeableBalance and can still have deposits in cooldown phase. However, this adds another complexity: Given that we want to retroactively mint rewards we need to keep track of every deposit's timestamp. Which will bring us back to solution 3 of scenario 5:

We actually keep track of every stake operation without lock up period individually, so something like:
mapping(uint256 depositLockUntil => uint256 amount) deposits;

But then, this will also introduce the additional complexity that, when a user calls stake(amount) we need to decide which of the deposits we actually want to register with the StakeManager.

We can avoid the complexity of 1) and instead force the user to first stake() any unstaked deposits. In other words, step 5 in the scenario laid out above is not an option. deposit() will revert as long as there's unstaked funds in the vault.

How does everyone feel about this?

I just realized, we'll have a similar complexity with the withdraw_cooldown_period.
The way I would see this work is the following:

User has deposited and then staked some funds
User decides to unstake(amount) - this will deregister amount from StakeManager an initialized withdrawCooldownUntil
Once withdrawCooldownUntil is over, users can call withdraw() on any "withdrawable" funds (there might be already "withdrawable" funds, namely, any deposits that weren't staked and got past their deposit cooldown period)

This works fine if the user decides to unstake() all staked funds.
But lets imagine this scenario:

User staked 1000 tokens
User unstakes 500 tokens (withdraw cooldown initialized)
Withdraw period has elapsed, but user hasn't withdrawn yet
User unstakes remaining 500 (or less) tokens
Withdraw cooldown is initialized again

So here, we similar need to keep track of funds that were withdrawn before.
In an additional scenario where the user unstake()s while there's already unstaked funds and the withdraw cooldown hasn't elapsed yet, we're again in a state where we'd need to keep track of all withdrawals and their timestamps.
I guess, similarly, we could avoid this complexity by saying, users can only unstake() when there's no withdraw_cooldown_period active.

Here's another one that maybe @mart1n-xyz can answer as this might have an impact on how well the reward system is balanced. Here's a scenario:

User deposit() into vault (no lock up time, deposit cooldown period starts)
Deposit cooldown has elapsed, so user will now call stake() to get the funds registered with the manager

In step 2) it's theoretically possible to give stake() a lock up time.
Questions:

Do we want o allow that, or should the stake step after a deposit cooldown period always have 0 lock up time?
If we do allow for it, and we retroactively mint the rewards for the deposit cooldown period, will also retroactively apply the lockup time that is passed to stake() ?

In other words, do we want to allow users to also retroactively apply a lockup time for their deposit that was done without a lockup time?

How does everyone feel about this?

I still feel the solution to majority of these problems indeed lies in tracking the cooldowns in StakeManager and not async in StakeVault. Even if not, in this case, we can make sure processAccount is called if any additional stake is being added (called from within deposit()), activating the previous deposit and emptying the cooldown queue for the new deposit.

I guess, similarly, we could avoid this complexity by saying, users can only unstake() when there's no withdraw_cooldown_period active.

I think this is preferable, I have actually experienced this behaviour in other staking contracts. We want to limit response (selling) to price action and having a gradual unstaking is a strong mechanism here. Otherwise, we'd just delay the response by 2 weeks, this way we may somewhat smooth it out.

Do we want to allow that, or should the stake step after a deposit cooldown period always have 0 lock up time?

If we have this deposit()/stake() split, we need to have the user specify the lock up time in deposit(). This is because for locked vaults, the cooldown does not apply (obviously, we need MIN_LOCKUP_TIME>2 weeks). This is another reason why I think we need to get the call to StakeManager.
Let me know if I am missing something.

Here's a draft branch that shows what this looks like in action (note: it's a WIP)

StakeVault.deposit(uint256 amount)
https://github.com/logos-co/staking/commit/8c5dd440404d6184937fa65deec67e00b24e159b#diff-b710313a5571054e746fc0e0d1332e5894fc76a55ffb035711d912c00bf8f826R63-R66
StakeVault.stake(uint256 amount)
https://github.com/logos-co/staking/commit/8c5dd440404d6184937fa65deec67e00b24e159b#diff-b710313a5571054e746fc0e0d1332e5894fc76a55ffb035711d912c00bf8f826R77-R79
StakeVault.depositAndStake(uint256 amount, uint256 time)
https://github.com/logos-co/staking/commit/8c5dd440404d6184937fa65deec67e00b24e159b#diff-b710313a5571054e746fc0e0d1332e5894fc76a55ffb035711d912c00bf8f826R81-R90

Again, it's a WIP. Some things that are missing is the unstake/withdrawal handling and also the ability to call _stake() with an additional timestamp that tells StakeManager from which point onwards it needs to retroactively mint rewards.

Thank you for explaining, this makes sense. I may have been missing your point.

a) do we allow for apply a lock time now?

Yes, it makes sense to allow it. As you mention, user can call lock() anyway. I agree with the deposit() -> cooldown -> stake() -> lock(lockTime).
I do not see why we should retroactively grant lock time from cooldown - it is user's fault in the first place that he didn't lock at the onset. Allowing for this introduces inconsistency. The dominant strategy would be to always go this path since you have the same outcome as with depositAndStake() but can withdraw anytime within those 2 weeks.

Another scenario, this time considering the withdraw_cooldown_period.
The withdraw_cooldown_period is similar to the deposit cooldown period, just that it will be initialized when a user unstakes funds. So here's the flow:

User has some funds staked
User calls unstake(amount)
This will initialize withdrawCooldownUntil
User will have to call withdraw() once block.timestamp > withdrawCooldownUntil

Very similar to deposit cooldown, similar things apply. If the user unstake()s again, while in withdraw cooldown, the cooldown period gets reset (this reasonable?)
But, what if the following happens:

User has some funds staked
User deposits into vault more funds (no staking) - initializes deposit cooldown
User unstakes some of the staked funds - initializes withdraw cooldown
User wants to withdraw funds from deposit done in 2) but can't know because there's a withdraw cooldown

Does this mean that withdraw cooldown has always highes priorty? Or maybe we also don't allow for unstaking while in deposit cool down?

Does this mean that withdraw cooldown has always highes priorty? Or maybe we also don't allow for unstaking while in deposit cool down?

I'd argue for disabling unstaking while there's an active withdrawal cooldown as per one of my previous comments. Reseting the cooldown is another option but I think suboptimal. Here, I'd again consider that we are dealing with epoch time units - hence withdrawals within one epoch can be treated as a single one.

This is blocked by #91 and inherently blocked by #48

Unassigning and rolling back to "TODO" state.
At this point we don't even know if we want to keep cooldown periods.


raiseUnknownParameters raises error when query starts with sub delimiter

The router currently returns a "400 Bad Request" error when:

raiseUnknownParameters is set to true in the router configuration
the query parameters start with a & (while it is unorthodox, this is allowed by RFC 3986)

Here an example based on the tutorial:
http://server/api/list?&page=1&isCompleted=true
which yields

"errors":{"":"is an unknown parameter"}

I would have thought that no such error would be raised in this case, since there is no unknown parameter (i.e. none).

Hey there, I've just published the new v1.0.0 release fixes this issue
Migration guide to 1.0.0 available here


Subgraphs moved to paused state once the network came back online

The agent found no GRT in the wallet, thus he couldn't reallocate when the network came back online.
All subgraphs that were previously syncing, were moved to nodeid "removed", thus pausing everything.

@trader-payne have you seen this issue recently?

I haven't tried to reproduce it in a long time. But I can try


cups-zj-58: init at 2018-02-22

Motivation for this change
Adds cups drivers for Zjiang ZJ-58
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[NA] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

Success on x86_64-linux (full log)
Partial log (click to expand)
gcc  -o rastertozj rastertozj.o -lcupsimage -lcups
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/3b7k9wmdjbgkicacnpijl7kh1mbp3klg-cups-zj-58-2018-02-22
shrinking /nix/store/3b7k9wmdjbgkicacnpijl7kh1mbp3klg-cups-zj-58-2018-02-22/lib/cups/filter/rastertozj
strip is /nix/store/b0zlxla7dmy1iwc3g459rjznx59797xy-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/3b7k9wmdjbgkicacnpijl7kh1mbp3klg-cups-zj-58-2018-02-22/lib
patching script interpreter paths in /nix/store/3b7k9wmdjbgkicacnpijl7kh1mbp3klg-cups-zj-58-2018-02-22
checking for references to /tmp/nix-build-cups-zj-58-2018-02-22.drv-0 in /nix/store/3b7k9wmdjbgkicacnpijl7kh1mbp3klg-cups-zj-58-2018-02-22...
/nix/store/3b7k9wmdjbgkicacnpijl7kh1mbp3klg-cups-zj-58-2018-02-22

Success on aarch64-linux (full log)
Partial log (click to expand)
gcc  -o rastertozj rastertozj.o -lcupsimage -lcups
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/1p275qj51xr5w8vk0isr9d7gbj62pd5z-cups-zj-58-2018-02-22
shrinking /nix/store/1p275qj51xr5w8vk0isr9d7gbj62pd5z-cups-zj-58-2018-02-22/lib/cups/filter/rastertozj
strip is /nix/store/lvx1acn1ig1j2km8jds5x3ggh3f2wa8v-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/1p275qj51xr5w8vk0isr9d7gbj62pd5z-cups-zj-58-2018-02-22/lib
patching script interpreter paths in /nix/store/1p275qj51xr5w8vk0isr9d7gbj62pd5z-cups-zj-58-2018-02-22
checking for references to /build in /nix/store/1p275qj51xr5w8vk0isr9d7gbj62pd5z-cups-zj-58-2018-02-22...
/nix/store/1p275qj51xr5w8vk0isr9d7gbj62pd5z-cups-zj-58-2018-02-22

Failure on x86_64-darwin (full log)
Partial log (click to expand)
Package ‘cups-zj-58-2018-02-22’ in /private/var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/lnl7-mac/pkgs/misc/cups/drivers/zj-58/default.nix:26 is not supported on ‘x86_64-darwin’, refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowBroken = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowBroken = true; }
to ~/.config/nixpkgs/config.nix.

@GrahamcOfBorg eval

@Mic92 thanks for the cleanup :+1: i always forget that install is just so much cooler than mkdir + cp

Success on x86_64-linux (full log)
Partial log (click to expand)
gcc  -o rastertozj rastertozj.o -lcupsimage -lcups
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/r84vc3k9pshy8wcvk4a8s94r62aha088-cups-zj-58-2018-02-22
shrinking /nix/store/r84vc3k9pshy8wcvk4a8s94r62aha088-cups-zj-58-2018-02-22/lib/cups/filter/rastertozj
strip is /nix/store/b0zlxla7dmy1iwc3g459rjznx59797xy-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/r84vc3k9pshy8wcvk4a8s94r62aha088-cups-zj-58-2018-02-22/lib
patching script interpreter paths in /nix/store/r84vc3k9pshy8wcvk4a8s94r62aha088-cups-zj-58-2018-02-22
checking for references to /tmp/nix-build-cups-zj-58-2018-02-22.drv-0 in /nix/store/r84vc3k9pshy8wcvk4a8s94r62aha088-cups-zj-58-2018-02-22...
/nix/store/r84vc3k9pshy8wcvk4a8s94r62aha088-cups-zj-58-2018-02-22

Success on x86_64-linux (full log)
Partial log (click to expand)
/nix/store/r84vc3k9pshy8wcvk4a8s94r62aha088-cups-zj-58-2018-02-22

Failure on x86_64-darwin (full log)
Partial log (click to expand)
Package ‘cups-zj-58-2018-02-22’ in /private/var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/lnl7-mac/pkgs/misc/cups/drivers/zj-58/default.nix:23 is not supported on ‘x86_64-darwin’, refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowBroken = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowBroken = true; }
to ~/.config/nixpkgs/config.nix.

Failure on x86_64-darwin (full log)
Partial log (click to expand)
Package ‘cups-zj-58-2018-02-22’ in /private/var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/lnl7-mac/pkgs/misc/cups/drivers/zj-58/default.nix:23 is not supported on ‘x86_64-darwin’, refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowBroken = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowBroken = true; }
to ~/.config/nixpkgs/config.nix.

Success on aarch64-linux (full log)
Partial log (click to expand)
gcc  -o rastertozj rastertozj.o -lcupsimage -lcups
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/97xzqh031fkzz5gnpxxmfwgfvf11ply9-cups-zj-58-2018-02-22
shrinking /nix/store/97xzqh031fkzz5gnpxxmfwgfvf11ply9-cups-zj-58-2018-02-22/lib/cups/filter/rastertozj
strip is /nix/store/lvx1acn1ig1j2km8jds5x3ggh3f2wa8v-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/97xzqh031fkzz5gnpxxmfwgfvf11ply9-cups-zj-58-2018-02-22/lib
patching script interpreter paths in /nix/store/97xzqh031fkzz5gnpxxmfwgfvf11ply9-cups-zj-58-2018-02-22
checking for references to /build in /nix/store/97xzqh031fkzz5gnpxxmfwgfvf11ply9-cups-zj-58-2018-02-22...
/nix/store/97xzqh031fkzz5gnpxxmfwgfvf11ply9-cups-zj-58-2018-02-22

Success on aarch64-linux (full log)
Partial log (click to expand)
these derivations will be built:
  /nix/store/8mvysvvghy33by6l12f9fsks5s38rdzb-cups-zj-58-2018-02-22.drv
waiting for locks or build slots...
/nix/store/97xzqh031fkzz5gnpxxmfwgfvf11ply9-cups-zj-58-2018-02-22

no problem.


Front-end develop/debug experience is much worse than pre-Webpack era

I was trying to work on #856 and kept bashing my head against the wall: many things that were possible and helping code or debug are now gone.
Granted, part of my frustration is caused by my current lack of understanding of Webpack/Babel and the like, but I am going to only list here the technical issues that I encountered.
I'm happy to RTFM, but at the very least it should be as easy to code/debug/test as it used to be.

[ ] debugger statements are being dropped when running Webpack. This is due to UglifyJsPlugin which drops them by default.
Current workarounds include adding drop_debugger: false to the compress object in the Webpack config or commenting out the plugin locally. What do we want to do in repo? Both options are meh, but first one is less meh since ESLint ensures debugger statements do not end up in master. Is there a better solution?

[ ] When reaching breakpoints, it is now not possible to evaluate code using the browser REPL. To reproduce, add var foo = "bar"; debugger; anywhere, run webpack, open the browser devtools and make sure to reach the breakpoint. When evaluating foo, browser says VM72:ReferenceError: foo is not defined.
I did not find an alternative yet and this is driving me crazy 😅

[ ] Likely a symptom of the same issue, but none of the globals are available anymore. This means it is not possible to open a browser console, select an element and evaluate $($0) anymore because $ is undefined1.
Temporary workaround I have found is to add window.$ = $; in lounge.js to have access to $. We obviously will not be able to ship this in production, which means that it will be hell to debug a production instance after next release if not fixed (hence priority label).

[ ] Sometimes npm run start-dev loses it and will not rebuild anymore. This forces a whole server restart, which is annoying as we lose history and such. Not sure what to do here, I ended up restarting the server every time so far...

1 $0 simply references the element currently selected by the inspector

This has some recommendations:

Use webpack devtools for debugging in browser: Better debugging experience, real source code, real module names, at the cost of slower build and difference to production build (Developer, for every app)

But this does not solve the 3rd point (and yay, more complexity...).

I would consider this actually a win for us, because we no longer pollute globals, and thus the entire client code is contained and less likely to be tampered with.

I don't entirely agree as things like jQuery are indeed available globally within the client code, so it's not a super clear win (though when we start expanding the client, it will be). However, I do not call making debugging harder a win :)
What solutions do we have? Is it reasonable to have a function (say debugMode()) available in the client code which, once called from a browser console, unleashes the globals? I don't know, maybe it's stupid, but I'm open to ideas and suggestions.

Could be an OSX issue.

I don't think so (at least not the OS-X fsevents bug as  they have a check for it, warn the user, and it's rare enough to not affect me).
I've noticed that, sometimes, when a build fails, Webpack won't rebuild anymore. It seems more like a Webpack bug than an OSX bug. Right now I'm running npm start and npm run watch in 2 different terminal windows. Not a huge deal, but still a pain, and it kills the point of npm run start-dev. Not sure what to do about that.

BTW, for whomever ends up here, @xPaw is attempting to fix the first 2 items of the list above in his PR #858. I have yet to test it.

Likely a symptom of the same issue, but none of the globals are available anymore. This means it is not possible to open a browser console, select an element and evaluate $($0) anymore because $ is undefined.
Temporary workaround I have found is to add window.$ = $; in lounge.js to have access to $. We obviously will not be able to ship this in production, which means that it will be hell to debug a production instance after next release if not fixed

@xPaw, any idea how we can go around this for v2.2.0? I understand that not leaking globals is a good thing conceptually, but it also means that debugging a production issue will be way more painful and that easily cancels out the win.
Throwing some ideas out there, maybe they're all 💩 so feel free to suggest anything better:

?debug=1 would load a page where these globals are available. This is not ideal because it forces a page reload.
A function knowingly accessible as a global (like debug() or enableDebug() or whatever) that makes the globals available. Advantage is you run it from the browser devtools so no need to refresh.

Note that these ideas are very similar (one could add a debug checkbox in the settings to the list, but it's the same, and does not provide anything more, except localStorage persistency at the cost of an unpleasant extra parameter to normal users), but I don't know how feasible it is in practice...
Again, taking any better idea than that.

@xPaw, any ideas? I'd hate to ship our next version without being able to use jQuery selectors, among other things.
For example, my main instance of The Lounge is actually the npm release and I very often open the dev tools to debug it with real data. We wouldn't be able to do so afterwards. This is also true when running from master with NODE_ENV=production npm run build (after #858 lands), so only way to have a "debuggable" instance would be to run from source AND no in production mode. Not good :S

So, what's the current state of these issues, @astorije ?

Regarding the globals, the idea is that you can access $ and stuff from breakpoints. So most of the time bugs come up in a function. So we can set a breakpoint in that function, and everything should be available there. Do we really need them globally available outside of this? If so, then I would be happy with either of the options that @astorije suggests.

For example, you can't open a console and start fiddling with $('#input') as $ is undefined.
This could be addressed at least short term by doing window.$ = $; in lounge.js. It would allow for v2.2.0 to go out shortly and be correctly debuggable while we have a better solution in v2.3.0.
Also, setting breakpoints does not work correctly in production environment because of the uglification. That being said, I need to re-try that now that #858 has landed. I'll do that tomorrow.

I think this is just an issue with webpack. I don't think there's anything we can do here.

I'm afraid you are right... I guess I'll use npm start and npm run watch in 2 terminals instead of npm run start-dev.

For example, you can't open a console and start fiddling with $('#input') as $ is undefined.
This could be addressed at least short term by doing window.$ = $; in lounge.js. It would allow for v2.2.0 to go out shortly and be correctly debuggable while we have a better solution in v2.3.0.

Meh, fine I guess. I mean, you could just load it in the console manually, but I'm not really bothered.

Also, setting breakpoints does not work correctly in production environment because of the uglification. That being said, I need to re-try that now that #858 has landed. I'll do that tomorrow.

So, you've ticked this in the post description, so does this mean that it's fixed now? The map should have sorted this, so I assume the fix worked for that.

Meh, fine I guess. I mean, you could just load it in the console manually, but I'm not really bothered.

Turns out, when running NODE_ENV=production npm run build, $ is now available from the dev tools console. So one can run things like $("#input") or else to debug an issue specifically on a release and/or in production.
Not sure what we did that could have changed that. It was definitely not possible prior to #858. It also does not make sense to me that it's now possible, but I won't complain. Note that it's not true for other globals, except Mousetrap. Some kind of wild west in there. Ah, the magic of obscure build systems :)

So, you've ticked this in the post description, so does this mean that it's fixed now? The map should have sorted this, so I assume the fix worked for that.

Yes, but I wanted to test the behavior on production post-#858. It's still the case: while you can place a breakpoint somewhere from the dev tools and it will correctly be reached, you can't evaluate the content of variables from the console. It's possible to temporarily turn off the source maps and evaluate the uglified variable names (breakpoints are shared between lounge.js and bundle.js) though. There might be some configuration issue here, or maybe that's how source maps are supposed to work. Either way, this is not impeding the release.

So between $ now being accessible (random) and last item being Webpack-related and/or OSX-related and/or magic (what is this, Ruby on Rails?!), we are good here, closing.
Thanks for your efforts and patience! :)


Add helpers functions used in deploy scripts

This PR adds the following new functions, all located in ./helpers:

decryptSecrets
writeEnvFiles
executeCommands
retryCommand

These are used in deploy scripts of multiple repos (hollowverse/hollowverse, hollowverse/letsEncrypt, and possibly hollowverse/hollowbot in the future).

How are we going to do code quality control in this repo?

How are we going to do code quality control in this repo?

@msafih I think it is possible to make this repo self-hosting. But I'm still not sure how to share the code quality scripts between repos. It seems we will have to copy-paste these commands unless we come up with a functionality similar to what commonpkg had. What do you think about adding these commands to the common.js file?


Can't install 3 alfa version in Ubuntu (pop os)

Error after make command:
NDEBUG  -MMD -MF apps/openssl-bin-version.d.tmp -MT apps/openssl-bin-version.o -c -o apps/openssl-bin-version.o apps/version.c
In file included from include/openssl/macros.h:11,
                 from include/openssl/opensslconf.h:14,
                 from ./e_os.h:14,
                 from apps/include/apps.h:13,
                 from apps/version.c:13:
apps/version.c: In function 'version_main':
include/openssl/opensslv.h:91:54: error: expected ')' before numeric constant
   91 | # define OPENSSL_VERSION_TEXT "OpenSSL 3.0.0-alpha1 "23 Apr 2020""
      |                                                      ^~
apps/version.c:109:16: note: in expansion of macro 'OPENSSL_VERSION_TEXT'
  109 |                OPENSSL_VERSION_TEXT, OpenSSL_version(OPENSSL_VERSION));
      |                ^~~~~~~~~~~~~~~~~~~~
apps/version.c:108:31: warning: format '%s' expects a matching 'char *' argument [-Wformat=]
  108 |         printf("%s (Library: %s)\n",
      |                              ~^
      |                               |
      |                               char *
make[1]: *** [Makefile:22578: apps/openssl-bin-version.o] Error 1
make[1]: Leaving directory '/home/bestann/openssl-3.0.0-alpha1'
make: *** [Makefile:3033: tests] Error 2

There's a workaround: https://wiki.openssl.org/index.php/OpenSSL_3.0#Installation_and_Compilation_of_OpenSSL_3.0
There's also an actual fix, which is included in current master: #11624


Add torchao to optimum as a pytorch backend configuration

Summary:
att.
for now we just added int4 weight only quantization, using TorchAoConfig from https://huggingface.co/docs/transformers/main/en/quantization/torchao
Test Plan:
python examples/pytorch_llama.py
gpt2 model
Base:
prefill: 9486.25 decode: 83.75
AWQ:
prefill: 9496.02 decode: 83.62
GPTQ
prefill: 9814.27 decode: 97.31
torchao int4wo
prefill: 10007.93 decode: 84.66
llama2
Base:
prefill: 2275.32 decode: 18.92
AWQ:
prefill: 2344.19 decode: 18.21
GPTQ:
prefill: 2881.87 decode: 26.47
torchao int4wo
prefill: 3035.82 decode: 24.51
Reviewers:
Subscribers:
Tasks:
Tags:

@jerryzh168 great prefill performance !
can you also add torchao to the installed libs in our docker images.

@IlyasMoutawwakil I feel we should be able to get better performance if qkv are fused in llama model, which is what people typically do today, not sure if there is a huggingface model in the hub that has it though
by adding torchao to docker, do you mean these: https://github.com/huggingface/optimum-benchmark/tree/main/docker ?

@IlyasMoutawwakil I feel we should be able to get better performance if qkv are fused in llama model, which is what people typically do today, not sure if there is a huggingface model in the hub that has it though

Would require changes in transformers modeling to support that, would make for a great PR there, some frameworks (like ipex in optimum-intel) do that manually.

by adding torchao to docker, do you mean these: https://github.com/huggingface/optimum-benchmark/tree/main/docker ?

yes exactly, the same as torchvision and torchaudio, I would rather have torchao installed there to avoid any cpu/cuda version issues.

yeah we are doing that manually in our benchmarks as well, maybe we can worry about it a bit later. I'll make docker changes first

@IlyasMoutawwakil I added torchao to cpu and cuda dockers, it's not available for rocm yet


DatePicker: Year not respecting selected theme

Detailed description

Describe in detail the issue you're having.

The Year value (2020) besides the month is showing as black while in a dark theme

Is this a feature request (new component, new icon), a bug, or a general issue?

Bug

Is this issue related to a specific component?

Date Picker

What did you expect to happen? What happened instead? What would you like to see changed?

The year should be the same color as the month

What browser are you working in?

Chrome

What version of the Carbon Design System are you using?

Latest / See Reproduction

What offering/product do you work on? Any pressing ship or release dates we should be aware of?

CS360 + Others.

Steps to reproduce the issue

https://github.com/virkt25/carbon-ng-demo

Closing because it seems to have been resolved in https://github.com/carbon-design-system/carbon/pull/6248


Provide Rana with Project Team contact details

We need to provide Rana with email addresses for Slack and Git accounts of project team members.

I have sent Rana all of our email addresses.


Can't prove range check constraints

Description
Aim
Generate a proof that includes a range check constraint.
Expected behavior
That my proof could be generated successfully for a valid input.
Bug
Attempting to prove for a circuit containing a Field range check fails:

thread 'main' panicked at 'not implemented: Field comparison is not implemented yet, try to cast arguments to integer type', crates/noirc_evaluator/src/ssa/acir_gen.rs:339:29

Attempting to prove  for a circuit containing a u8 range check progresses further, but still fails:
ACIR gates generated : 15
Loading noir code from: "/var/folders/n4/d6xwpnvd3mb6vrsqh0k9rv9h0000gn/T/.tmp8bm2iQ"
panicked at 'called `Result::unwrap()` on an `Err` value: Syntax("invalid type: sequence, expected u32")', crates/acir/src/circuit/mod.rs:30:9

Stack:

Error
    at module.exports.__wbg_new_693216e109162396 (/Users/aztec/.nvm/versions/node/v16.13.0/lib/node_modules/@noir-lang/noir-cli/node_modules/@noir-lang/noir_wasm/noir_wasm.js:200:17)
    at wasm://wasm/009f874a:wasm-function[1148]:0x232fd6
    at wasm://wasm/009f874a:wasm-function[2431]:0x266915
    at wasm://wasm/009f874a:wasm-function[1674]:0x258cc7
    at wasm://wasm/009f874a:wasm-function[1752]:0x25c228
    at wasm://wasm/009f874a:wasm-function[2162]:0x2651bc
    at wasm://wasm/009f874a:wasm-function[1991]:0x26268b
    at wasm://wasm/009f874a:wasm-function[2100]:0x26456c
    at wasm://wasm/009f874a:wasm-function[1800]:0x25db79
    at wasm://wasm/009f874a:wasm-function[1324]:0x243434

Error thrown: RuntimeError: unreachable
thread 'main' panicked at 'range start index 32 out of range for slice of length 0', library/core/src/slice/index.rs:52:5

To reproduce
Field range bug

nargo build the follow circuit:

fn main(x : Field, y : pub Field) {
    constrain x < y;
}

nargo prove proof with inputs

# Verifier.toml
x = 1
y = 2

u8 range bug
As above, but with circuit:
fn main(x : u8, y : pub u8) {
    constrain x < y;
}

Environment
(Specify your setup and versions of dependencies.)

OS: macOS 12.4 Intel

For nargo users

noir-lang/noir commit cloned: b3eb4c13192cb940d9d8a39bc8eb68e80d3284fa
Proving backend

[ ] default

Clang: (run clang --version)

[x] wasm-base

Your second example with u8 successfully proves and verifies for me on the same commit

u8 example works for me as well.

Can confirm this bug goes away if I build barretenberg instead of using wasm-base. Here's a fully stack trace attempting the same u8 comparison on linux.
RUST_BACKTRACE=1 nargo prove proof
ACIR gates generated : 15
internal/modules/cjs/loader.js:905
  throw err;
  ^

Error: Cannot find module<EMAIL_ADDRESS>    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:902:15)
    at Function.Module._load (internal/modules/cjs/loader.js:746:27)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:76:12)
    at internal/main/run_main_module.js:17:47 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}
thread 'main' panicked at 'range start index 32 out of range for slice of length 0', /mnt/user-data/joss/.cargo/git/checkouts/aztec_backend-a697fb631cbad807/8dc5b28/src/acvm_interop/proof_system.rs:205:5
stack backtrace:
   0: rust_begin_unwind
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/panicking.rs:142:14
   2: core::slice::index::slice_start_index_len_fail_rt
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/slice/index.rs:53:5
   3: core::slice::index::slice_start_index_len_fail
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/slice/index.rs:42:9
   4: aztec_backend::acvm_interop::proof_system::remove_public_inputs
   5: aztec_backend::acvm_interop::proof_system::<impl acvm::ProofSystemCompiler for aztec_backend::acvm_interop::Plonk>::prove_with_meta
   6: nargo::cli::prove_cmd::run
   7: nargo::cli::start_cli

This looks to be a Noir issue that was fixed (however I cannot find an original issue if there was one), but the wasm-base is missing the updated version of Noir. Using the barretenberg.wasm file directly allows us to access the most recent Noir changes without having to push updates to the noir-cli. This work has yet to be merged into the aztec_backend, but installing nargo with the backend below should result in successful proving and verification:
aztec_backend = { optional = true, package = "barretenberg_wasm", git = "https://github.com/noir-lang/aztec_backend", branch = "mv/fix-static-lib" }

Just tested - confirmed fixed


Support for 1.20?

Hi GeraldTM, its this time of the year where I can get back to stuff and now I implemented a multi version build. Feel free to take this over to your version as well if you want to!
Did you do any specific changes regarding functionality in your fork of the project?

No, never got around to it

Okay thanks for taking over then, Ill close this


Avoid writing to stdout in default reporter if --json is enabled. Fixes #3941

Check #3941 for more details.

Codecov Report

Merging #3945 into master will decrease coverage by 0.05%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master    #3945      +/-   ##
==========================================
- Coverage   58.06%   58.01%   -0.06%
==========================================
  Files         196      196
  Lines        6758     6764       +6
  Branches        6        6
==========================================
  Hits         3924     3924
- Misses       2831     2837       +6
  Partials        3        3

Impacted Files
Coverage Δ

...ackages/jest-cli/src/reporters/default_reporter.js
4.41% <0%> (-0.43%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5a509c7...375f1e3. Read the comment docs.

Sweet! Please run yarn test and make sure it passes.


how can i change the max_len??

Describe the Bug
A clear and concise description of what the bug is.
Version Info

[x] I'm using the latest version

Minimal Codes To Reproduce
import keras_bert

pass

What should I do when my sentence length is greater than 512 bytes

What should I do when my sentence length is greater than 512 bytes

see https://github.com/google-research/bert/issues/66

What should I do when my sentence length is greater than 512 bytes

see google-research/bert#66

Okay got it, that makes sense. Thanks for your help!


PHOENIX-2165 Make zip file name and directory consistent for phoenix-*-cluster.zip

Any other changes you would like to see here? Thanks

@mujtabachohan - would you mind reviewing/committing?

Looks like this was committed already a while back. Please let us know if anything is awry, but otherwise the PR and JIRA can be closed.


Add a condition 'DoNotReplaceQueryParams' paramerter for ProxyExchangeHandlerFunction  in GatewayMvc

Is your feature request related to a problem? Please describe.
In my scenario, my request uri is "http://fool?key1=value1@%26@&key2=value2" , it wiil proxy for another server. i use embed jetty, i will auto decode uri parameters to compose HttpServletRequest.
so the parametersMap in HttpServletRequest becomes {key1=value1@&@,key2=value2} ,  that's not my aim. the character '@%26@' is special for another server , i don't want  any change in query parameters until another server received.
ProxyExchangeHandlerFunction
the method Handle, will replaceQueryParams from uri parameters to serverRequest params .
	public ServerResponse handle(ServerRequest serverRequest) {
		URI uri = uriResolver.apply(serverRequest);
		boolean encoded = containsEncodedQuery(serverRequest.uri());
		// @formatter:off
		URI url = UriComponentsBuilder.fromUri(serverRequest.uri())
				.scheme(uri.getScheme())
				.host(uri.getHost())
				.port(uri.getPort())
                              // should not replace in every scenario
				.replaceQueryParams(serverRequest.params())
				.build(encoded)
				.toUri();
		// @formatter:on

		// TODO: Streams.collect()?
		HttpHeaders filteredRequestHeaders = filterHeaders(
				this.requestHttpHeadersFilters.orderedStream().map(Function.identity()),
				serverRequest.headers().asHttpHeaders(), serverRequest);

		boolean preserveHost = (boolean) serverRequest.attributes()
				.getOrDefault(MvcUtils.PRESERVE_HOST_HEADER_ATTRIBUTE, false);
		if (preserveHost) {
			filteredRequestHeaders.set(HttpHeaders.HOST, serverRequest.headers().firstHeader(HttpHeaders.HOST));
		}
		else {
			filteredRequestHeaders.remove(HttpHeaders.HOST);
		}

		// @formatter:off
		ProxyExchange.Request proxyRequest = proxyExchange.request(serverRequest).uri(url)
				.headers(filteredRequestHeaders)
				// TODO: allow injection of ResponseConsumer
				.responseConsumer((response, serverResponse) -> {
					HttpHeaders httpHeaders = filterHeaders(this.responseHttpHeadersFilters.orderedStream()
							.map(Function.identity()), response.getHeaders(), serverResponse);
					serverResponse.headers().putAll(httpHeaders);
				})
				.build();
		// @formatter:on
		return proxyExchange.exchange(proxyRequest);
	}

Describe the solution you'd like
i don't want  any change in query parameters until another server received.
Describe alternatives you've considered
we could add a condition 'DoNotReplaceQueryParams' for ProxyExchangeHandlerFunction constructer parameter
Additional context
None

See comments on #3248

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.


[spark] Add text generation in pyspark

Description
Brief description of what this PR is about
Add text generation in pyspark. Since text generation models are not traceable. So it's only supported in pyspark now.

Codecov Report
Patch coverage: 74.08% and project coverage change: +1.73 :tada:

Comparison is base (bb5073f) 72.08% compared to head (ef1e3a0) 73.82%.

:exclamation: Current head ef1e3a0 differs from pull request most recent head c7f5bdc. Consider uploading reports for the commit c7f5bdc to get more accurate results

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #2477      +/-   ##
============================================
+ Coverage     72.08%   73.82%   +1.73%
- Complexity     5126     6835    +1709
============================================
  Files           473      675     +202
  Lines         21970    29923    +7953
  Branches       2351     3091     +740
============================================
+ Hits          15838    22090    +6252
- Misses         4925     6334    +1409
- Partials       1207     1499     +292

Impacted Files
Coverage Δ

api/src/main/java/ai/djl/modality/cv/Image.java
69.23% <ø> (-4.11%)
:arrow_down:

...rc/main/java/ai/djl/modality/cv/MultiBoxPrior.java
76.00% <ø> (ø)

...rc/main/java/ai/djl/modality/cv/output/Joints.java
71.42% <ø> (ø)

.../main/java/ai/djl/modality/cv/output/Landmark.java
100.00% <ø> (ø)

...main/java/ai/djl/modality/cv/output/Rectangle.java
72.41% <0.00%> (ø)

...i/djl/modality/cv/translator/BigGANTranslator.java
21.42% <0.00%> (-5.24%)
:arrow_down:

.../modality/cv/translator/ImageFeatureExtractor.java
0.00% <0.00%> (ø)

.../ai/djl/modality/cv/translator/YoloTranslator.java
27.77% <0.00%> (+18.95%)
:arrow_up:

...ain/java/ai/djl/modality/cv/util/NDImageUtils.java
67.10% <0.00%> (+7.89%)
:arrow_up:

api/src/main/java/ai/djl/modality/nlp/Decoder.java
63.63% <ø> (ø)

... and 228 more

... and 342 files with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


fix: holopin formatting

Description

I believe this is a formatting issue. Updating the holopin badge
Issue reference

This PR is linked to #164
Checklist

[ ] Code compiles correctly
[ ] Created/updated tests
[ ] Extended the documentation

Hi @mikeee,
Sorry I didn't notice the formatting but I found this PR in go-sdk has the same issue.
Hope we can avoid the same mistake there.

I'm not sure if it matters with yaml, I think I'm unable to issue stickers as I'm not part of the holopin org 👍

The formatting is not an issue. See the Holopin bot readme