[android] Download is slow due to buffer settings

Package
dio
Version
5.3.3
Operating-System
Android 13
Output of flutter doctor -v
Doctor summary (to see all details, run flutter doctor -v):
[!] Flutter (Channel [user-branch], 3.13.0-0.2.pre, on macOS 12.6.5 21G531 darwin-arm64, locale en-IT)
    ! Flutter version 3.13.0-0.2.pre on channel [user-branch] at /Users/xxxxxx/Development/flutter
      Currently on an unknown channel. Run `flutter channel` to switch to an official channel.
      If that doesn't fix the issue, reinstall Flutter by following instructions at https://flutter.dev/docs/get-started/install.
    ! Upstream repository unknown source is not a standard remote.
      Set environment variable "FLUTTER_GIT_URL" to unknown source to dismiss this error.
[!] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    ✗ cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/macos#android-setup for more details.
[✓] Xcode - develop for iOS and macOS (Xcode 14.0.1)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2022.2)

Dart Version
Dart 3.1.0 (build 3.1.0-262.2.beta) • DevTools 2.25.0
Steps to Reproduce
We currently use Dio to download files from google drive.
await dio.download(
        'https://www.googleapis.com/drive/v3/files/$id',
        path,
        queryParameters: {
          'alt': 'media',
        },
        options: Options(headers: {
          HttpHeaders.acceptEncodingHeader: '*',
          'Authorization': token,
        },
        responseType: ResponseType.stream,
        ),
        onReceiveProgress: (count, total) =>
            onDownloadProgress.call(count, total),
      );

Code works but it is slow and we noticed it is related to buffer.
For example, it takes 2 secs to download a file of about 258KB with a 200Mps connection and progress prints all below values. The same thing on iOS, where we don't use dio, just print 1 value and completes in a few millisecs. Is it a bug? Or is it possible to instruct dio to download files in chunks and set chunk size to 1024 * 1024?
I/flutter (18530): CloudBackup:: Downloading sqlite db file.
I/flutter (18530): CloudBackup: downloading sqliteFile 762 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 3518 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 4896 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 6274 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 7652 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 9030 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 10408 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 11786 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 13164 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 14542 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 14992 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 16370 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 17748 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 19126 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 20504 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 21882 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 24638 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 26016 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 27394 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 28772 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 30150 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 31376 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 32754 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 34132 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 35510 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 36888 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 38266 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 39644 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 41022 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 42400 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 43778 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 45156 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 46534 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 47760 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 49138 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 50516 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 51894 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 53272 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 54650 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 56028 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 57406 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 58784 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 60162 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 61540 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 62918 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 64132 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 65510 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 66888 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 68266 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 69644 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 71022 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 72400 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 73778 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 75156 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 77912 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 79290 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 80516 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 81894 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 83272 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 84650 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 86028 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 87406 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 88784 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 90162 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 91540 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 92918 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 94296 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 95674 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 96900 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 98278 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 99656 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 101034 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 102412 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 103790 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 105168 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 106546 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 107924 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 109302 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 110680 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 112058 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 113284 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 114662 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 118796 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 120174 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 121552 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 122930 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 124308 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 125686 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 127064 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 128442 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 129668 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 131046 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 132424 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 133802 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 135180 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 136558 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 137936 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 139314 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 140692 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 142070 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 143448 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 144826 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 146052 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 147430 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 148808 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 150186 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 152942 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 154320 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 155698 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 157076 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 158454 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 159832 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 161210 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 162436 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 163814 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 165192 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 166570 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 167948 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 169326 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 170704 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 172082 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 173460 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 174838 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 176216 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 177594 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 178820 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 180198 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 181576 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 182954 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 184332 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 185710 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 187088 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 188466 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 189844 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 191222 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 192600 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 193978 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 195204 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 196582 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 197960 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 199338 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 200716 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 202094 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 203472 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 204850 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 206228 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 207606 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 210362 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 211588 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 212966 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 214344 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 215722 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 217100 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 218478 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 219856 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 221234 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 222612 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 223990 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 225368 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 226746 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 227972 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 229350 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 230728 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 232106 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 233484 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 234862 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 236240 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 237618 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 238996 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 240374 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 241752 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 243130 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 244356 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 245734 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 247112 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 248490 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 249868 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 251246 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 252624 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 254002 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 255380 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 256758 of 258048

Expected Result
File should be downloaded faster. This might be a log for a scenario with about 100 KB chunk size.
I/flutter (18530): CloudBackup:: Downloading sqlite db file.
I/flutter (18530): CloudBackup: downloading sqliteFile 105168 of 258048
I/flutter (18530): CloudBackup: downloading sqliteFile 256758 of 258048

Actual Result
As the above logs.

We no longer support Dio v3.
Please provide a minimal example of your comparison.

Hi Alex,

Sorry, pasted wrong number, using ^5.3.3
Issue is reproducible also with native_dio_adapter.dart.
Strangely, same code doesn't work with http2 adaper.

The ios code uses a native plugin. But the sample provided should be good to reproduce issue. I didn't count the logs lines pasted above but I guess we print 100 lines of progress for a 200KB file. Progress depends much on system and configuration, but it is normal to expect zero or just a few lines of progress before a 200KB download completes.

Tested on macOS. It seems perfectly reached my maximum bandwidth. (https://hnd-jp-ping.vultr.com/vultr.com.100MB.bin)

Sorry, I missed to specify this occurs on android. We are testing with v13, maybe issue also occur with v12.

Sorry, I missed to specify this occurs on android. We are testing with v13, maybe issue also occur with v12.

What is v13?

Android 13

I tried your file above.
Look at first logs:  7777, 15968, 15970, 24161, 32352, 32354
Is that like a buffer size issue?
 CloudBackup: downloading 7777 of 104857600
I/flutter (21600): CloudBackup: downloading 15968 of 104857600
I/flutter (21600): CloudBackup: downloading 15970 of 104857600
I/flutter (21600): CloudBackup: downloading 24161 of 104857600
I/flutter (21600): CloudBackup: downloading 32352 of 104857600
I/flutter (21600): CloudBackup: downloading 32354 of 104857600
etc...

Tested on Android 11:
Launching lib/main.dart on ONEPLUS A6010 in debug mode...
Running Gradle task 'assembleDebug'...
✓  Built build/app/outputs/flutter-apk/app-debug.apk.
Installing build/app/outputs/flutter-apk/app-debug.apk...
Debug service listening on ws://<IP_ADDRESS>:50091/QhHYdlfBDfU=/ws
Syncing files to device ONEPLUS A6010...
I/flutter ( 2282): 0:00:24.937422, speed: 4266.666666666667KB/s
I/flutter ( 2282): 0:00:25.503617, speed: 4096.0KB/s
I/flutter ( 2282): 0:00:25.167354, speed: 4096.0KB/s

Tested on Android 13:
Launching lib/main.dart on PHB110 in debug mode...
Running Gradle task 'assembleDebug'...
✓  Built build/app/outputs/flutter-apk/app-debug.apk.
Installing build/app/outputs/flutter-apk/app-debug.apk...
Debug service listening on ws://<IP_ADDRESS>:50490/SY4CVCRm4_E=/ws
Syncing files to device PHB110...
I/flutter (22557): 0:00:25.496580, speed: 4096.0KB/s
I/flutter (22557): 0:00:25.543291, speed: 4096.0KB/s
I/flutter (22557): 0:00:25.027422, speed: 4096.0KB/s

I'm not sure about your statement regarding the buffer size. It seems we can get constant results between native platforms, regardless the implementation. The progress log only tells how many bytes are accepted in the current stream event. Other than that, if you dont throttle logs for it, it might cause performance regressions too.

I just tried printing %
onReceiveProgress: (received, total) {
          if (total != -1) {
            print('downloading: ${(received / total * 100).toStringAsFixed(0)} %');
          }

I/flutter (10195): downloading: 2 %
I/flutter (10195): downloading: 10 %
I/flutter (10195): downloading: 14 %
I/flutter (10195): downloading: 18 %
I/flutter (10195): downloading: 22 %
I/flutter (10195): downloading: 26 %
I/flutter (10195): downloading: 30 %
I/flutter (10195): downloading: 34 %
I/flutter (10195): downloading: 38 %
I/flutter (10195): downloading: 42 %
I/flutter (10195): downloading: 43 %
I/flutter (10195): downloading: 47 %
I/flutter (10195): downloading: 51 %
I/flutter (10195): downloading: 55 %
I/flutter (10195): downloading: 59 %
I/flutter (10195): downloading: 63 %
I/flutter (10195): downloading: 67 %
I/flutter (10195): downloading: 71 %
I/flutter (10195): downloading: 75 %
I/flutter (10195): downloading: 79 %
I/flutter (10195): downloading: 83 %
I/flutter (10195): downloading: 87 %
I/flutter (10195): downloading: 94 %
I/flutter (10195): downloading: 98 %
I/flutter (10195): downloading: 100 %

And if I disable onReceiveProgress, download takes same amount of time.

To narrow the scope of the issue, could you remove the log in onReceiveProgress and leave it as a empty callback? Then we can determine whether it's a logging throttle issue or caused by the callback.

Logs removed
onReceiveProgress: (received, total) {
          if (total != -1) {
          //  print('downloading: ${(received / total * 100).toStringAsFixed(0)} %');
          }

It takes same amount of time.
It indeed makes sense, coz considering % logs above, we can expect callback to be invoked 1 or 2 times.
Instead we see %: 2, 10, 14, 18, ... etc... 42, 43, ...etc ... which shouldn't occur for a 200KB file with a 200Mbps connection.

Which your same anount of time equals to? Mine or yours?

Mine, the same file (200KB) takes same amount of time to download, with or without printing log inside callback.
At first glance, I see two potential issues:

As you mentioned, a callback issue. Dio invokes callback too often. Hence, for example, calling onReceiveProgress 30 times for a process that, if not stalled from callback, would complete in a few millisecs.
There might be a low level issue, coz if I disable onReceiveProgress (not just leaving it empty but removing the callback itself), download stills take the same big amount of time to complete.

Using a small file (100-300KB) to test would make things more evident.

I added your code to also print speed, which in my case is InfinityKB/s
I/flutter (16212): downloading: 2 %
I/flutter (16212): downloading: 10 %
I/flutter (16212): downloading: 14 %
I/flutter (16212): downloading: 18 %
I/flutter (16212): downloading: 22 %
I/flutter (16212): downloading: 26 %
I/flutter (16212): downloading: 30 %
I/flutter (16212): downloading: 34 %
I/flutter (16212): downloading: 38 %
I/flutter (16212): downloading: 42 %
I/flutter (16212): downloading: 43 %
I/flutter (16212): downloading: 47 %
I/flutter (16212): downloading: 51 %
I/flutter (16212): downloading: 55 %
I/flutter (16212): downloading: 59 %
I/flutter (16212): downloading: 63 %
I/flutter (16212): downloading: 67 %
I/flutter (16212): downloading: 71 %
I/flutter (16212): downloading: 75 %
I/flutter (16212): downloading: 79 %
I/flutter (16212): downloading: 83 %
I/flutter (16212): downloading: 87 %
I/flutter (16212): downloading: 91 %
I/flutter (16212): downloading: 95 %
I/flutter (16212): downloading: 99 %
I/flutter (16212): downloading: 100 %

I/flutter (16212): 0:00:00.848312,speed: InfinityKB/s
I/flutter (16212): 0:00:00.687079,speed: InfinityKB/s
I/flutter (16212): 0:00:00.515818,speed: InfinityKB/s

Maybe this helps:
I tested the download by downloading in chunks
A)
I/flutter (16212): CloudBackup: downloading sqliteFile 1048576 of 104857600
I/flutter (16212): CloudBackup: downloading sqliteFile 2097152 of 104857600
I/flutter (16212): CloudBackup: downloading sqliteFile 3145728 of 104857600
I/flutter (16212): CloudBackup: downloading sqliteFile 4194304 of 104857600
I/flutter (16212): CloudBackup: downloading sqliteFile 5242880 of 104857600

Printed values are 1048576, 2097152, 3145728, 4194304, 5242880, etc
When using Dio with your 100MB file logs were:
B)
I/flutter (21600): CloudBackup: downloading 15968 of 104857600
I/flutter (21600): CloudBackup: downloading 15970 of 104857600
I/flutter (21600): CloudBackup: downloading 24161 of 104857600
I/flutter (21600): CloudBackup: downloading 32352 of 104857600
I/flutter (21600): CloudBackup: downloading 32354 of 104857600

Printed values are 15968 , 15970 , 24161 , 32352 , 32354 , etc
So, with solution A we downloaded 1MB when DIO downloaded 15.9KB. And so on so forth.
So, we can conclude that is a low level issue.
To test things yourselves, here is our chunkedDownloader:
library chunked_downloader;

/// Original from plugin https://pub.dev/packages/chunked_downloader
/// We just added minor changes to inject queryParameters

import 'dart:async';
import 'dart:io';
import 'dart:typed_data';

import 'package:async/async.dart';
import 'package:flutter/cupertino.dart';
import 'package:http/http.dart' as http;
import 'package:http/http.dart';

/// Progress Callback
/// [progress] is the current progress in bytes
/// [total] is the total size of the file in bytes
typedef ProgressCallback = void Function(int progress, int total, double speed);

/// On Done Callback
/// [file] is the downloaded file
typedef OnDoneCallback = void Function(File file);

/// On Error Callback
/// [error] is the error that occured
typedef OnErrorCallback = void Function(dynamic error);

/// Custom Downloader with ChunkSize
///
/// [chunkSize] is the size of each chunk in bytes
///
/// [onProgress] is the callback function that will be called when the download is in progress
///
/// [onDone] is the callback function that will be called when the download is done
///
/// [onError] is the callback function that will be called when the download is failed
///
/// [onCancel] is the callback function that will be called when the download is canceled
///
/// [onPause] is the callback function that will be called when the download is paused
///
/// [onResume] is the callback function that will be called when the download is resumed
///
class ChunkedDownloader {
  final String url;
  final String saveFilePath;
  final int chunkSize;
  final ProgressCallback? onProgress;
  final OnDoneCallback? onDone;
  final OnErrorCallback? onError;
  final Function? onCancel;
  final Function? onPause;
  final Function? onResume;
  StreamSubscription<StreamedResponse>? stream;
  ChunkedStreamReader<int>? reader;
  Map<String, String>? headers;
  Map<String, String>? queryParameters;
  double speed = 0;
  bool paused = false;
  bool done = false;
  static const bool kDebugMode = false;

  ChunkedDownloader({
    required this.url,
    required this.saveFilePath,
    this.headers,
    this.queryParameters,
    this.chunkSize = 1024 * 1024, // 1 MB
    this.onProgress,
    this.onDone,
    this.onError,
    this.onCancel,
    this.onPause,
    this.onResume,
  });

  /// Start the download
  /// @result {Future<ChunkedDownloader>} the current instance of the downloader
  Future<ChunkedDownloader> start() async {
    // Download file
    try {
      int offset = 0;
      var httpClient = http.Client();
      // Set queryParameters
      var uri;
      if (queryParameters != null) {
        uri = Uri.parse(url).replace(queryParameters: queryParameters);
      } else {
        uri = Uri.parse(url);
      }
     // Uri uri = Uri.parse(url);
     // var finalUri = uri.replace(queryParameters: queryParameters);
      var request = http.Request('GET', uri);
      // Set headers
      if (headers != null) {
        request.headers.addAll(headers!);
      }
      debugPrint('ChunkedDownloader: uri ${uri}');
      var response = httpClient.send(request);

      // Open file
      File file = File('$saveFilePath');

      stream = response.asStream().listen(null);
      stream?.onData((http.StreamedResponse r) async {
        // Get file size
        int fileSize = int.tryParse(r.headers['content-length'] ?? '-1') ?? -1;
        reader = ChunkedStreamReader(r.stream);
        try {
          Uint8List buffer;
          do {
            // TODO: better pausing
            while (paused) {
              await Future.delayed(const Duration(milliseconds: 500));
            }
            // Set start time for speed calculation
            int startTime = DateTime.now().millisecondsSinceEpoch;

            // Read chunk
            buffer = await reader!.readBytes(chunkSize);

            // Calculate speed
            int endTime = DateTime.now().millisecondsSinceEpoch;
            int timeDiff = endTime - startTime;
            if (timeDiff > 0) {
              speed = (buffer.length / timeDiff) * 1000;
            }

            // Add buffer to chunks list
            offset += buffer.length;
            if (kDebugMode) {
              print('Downloading ${offset ~/ 1024 ~/ 1024}MB '
                  'Speed: ${speed ~/ 1024 ~/ 1024}MB/s');
            }
            if (onProgress != null) {
              onProgress!(offset, fileSize, speed);
            }
            // Write buffer to disk
            await file.writeAsBytes(buffer, mode: FileMode.append);
          } while (buffer.length == chunkSize);

          // Rename file from .tmp to non-tmp extension
          //await file.rename(saveFilePath);

          // Send done callback
          done = true;
          if (onDone != null) {
            onDone!(file);
          }
          if (kDebugMode) {
            print('Downloaded file.');
          }
        } catch (error) {
          if (kDebugMode) {
            print('Error downloading: $error');
          }
          if (onError != null) {
            onError!(error);
          }
        } finally {
          reader?.cancel();
          stream?.cancel();
        }
      });
    } catch (error) {
      if (kDebugMode) {
        print('Error downloading: $error');
      }
      if (onError != null) {
        onError!(error);
      }
    }
    return this;
  }

  /// Stop the download
  void stop() {
    stream?.cancel();
    reader?.cancel();
    if (onCancel != null) {
      onCancel!();
    }
  }

  /// Pause the download
  void pause() {
    paused = true;
    if (onPause != null) {
      onPause!();
    }
  }

  /// Resume the download
  void resume() {
    paused = false;
    if (onResume != null) {
      onResume!();
    }
  }
}

And here is sample code:
var chunkedDownloader = await ChunkedDownloader(
          url: 'https://hnd-jp-ping.vultr.com/vultr.com.100MB.bin',
          saveFilePath: path,
          chunkSize: 1024 * 1024, // Buffer set to 100KB
          headers: {
            'Content-Type': 'application/octet-stream',
            'HttpHeaders.acceptEncodingHeader': '*',
          },
          onError: (error) {},
          onProgress: (received, total, speed) =>
              onDownloadProgress.call(received, total),
          onDone: (file) {})
          .start();

Labeling for further investigations.

I added your code to also print speed, which in my case is InfinityKB/s
I/flutter (16212): downloading: 2 %
I/flutter (16212): downloading: 10 %
I/flutter (16212): downloading: 14 %
I/flutter (16212): downloading: 18 %
I/flutter (16212): downloading: 22 %
I/flutter (16212): downloading: 26 %
I/flutter (16212): downloading: 30 %
I/flutter (16212): downloading: 34 %
I/flutter (16212): downloading: 38 %
I/flutter (16212): downloading: 42 %
I/flutter (16212): downloading: 43 %
I/flutter (16212): downloading: 47 %
I/flutter (16212): downloading: 51 %
I/flutter (16212): downloading: 55 %
I/flutter (16212): downloading: 59 %
I/flutter (16212): downloading: 63 %
I/flutter (16212): downloading: 67 %
I/flutter (16212): downloading: 71 %
I/flutter (16212): downloading: 75 %
I/flutter (16212): downloading: 79 %
I/flutter (16212): downloading: 83 %
I/flutter (16212): downloading: 87 %
I/flutter (16212): downloading: 91 %
I/flutter (16212): downloading: 95 %
I/flutter (16212): downloading: 99 %
I/flutter (16212): downloading: 100 %

I/flutter (16212): 0:00:00.848312,speed: InfinityKB/s
I/flutter (16212): 0:00:00.687079,speed: InfinityKB/s
I/flutter (16212): 0:00:00.515818,speed: InfinityKB/s

@AlexV525 this class is buggy and poorly coded, had to rework it.
But the concept is correct and I gain perf improvement this way without using Dio.
Idea is, buffer lenght is chunkSize, buffer is filled with bytes from reader, and we write file a buffer at time. This reduces the numbers of write required to create the file and speed thungs up. Ofc, I can do that because we know file size and pass it to construct (no need to fetch it from content-lenght, which is not supported in many cases).
Indeed that might be a feature to add to Dio: if fileSize is known, we can pass fileSize and chunkSize and download perf will be boosted.
However, the underneath problem related to poor performsnce of flutter stream remains unfixed.

Thisbis the class, lmk if you see improvement
library chunked_downloader;

/// Inspired by plugin https://pub.dev/packages/chunked_downloader
/// It was reworked for the most part

import 'dart:async';
import 'dart:io';

import 'package:async/async.dart';
import 'package:flutter/cupertino.dart';
import 'package:http/http.dart' as http;
import 'package:http/http.dart';

/// Progress Callback
/// [progress] is the current progress in bytes
/// [total] is the total size of the file in bytes
typedef ProgressCallback = void Function(int progress, int total, double speed);

/// On Done Callback
/// [file] is the downloaded file
typedef OnDoneCallback = void Function(File file);

/// On Error Callback
/// [error] is the error that occured
typedef OnErrorCallback = void Function(dynamic error);

/// Custom Downloader with ChunkSize
///
/// [chunkSize] is the size of each chunk in bytes
///
/// [onProgress] is the callback function that will be called when the download is in progress
///
/// [onDone] is the callback function that will be called when the download is done
///
/// [onError] is the callback function that will be called when the download is failed
///
/// [onCancel] is the callback function that will be called when the download is canceled
///
/// [onPause] is the callback function that will be called when the download is paused
///
/// [onResume] is the callback function that will be called when the download is resumed
///
class ChunkedDownloader {
  final String url;
  final String saveFilePath;
  final int chunkSize;
  final ProgressCallback? onProgress;
  final OnDoneCallback? onDone;
  final OnErrorCallback? onError;
  final Function? onCancel;
  final Function? onPause;
  final Function? onResume;
  StreamSubscription<StreamedResponse>? stream;
  ChunkedStreamReader<int>? reader;
  Map<String, String>? headers;
  Map<String, String>? queryParameters;
  double speed = 0;
  bool paused = false;
  bool done = false;
  int? fileSize;
  static const bool kDebugMode = false;
  static bool downloadCompleted = false;
  static bool downloadError = false;

  ChunkedDownloader({
    required this.url,
    required this.saveFilePath,
    this.fileSize,
    this.headers,
    this.queryParameters,
    this.chunkSize = 1024 * 1024, // default is 1 MB
    this.onProgress,
    this.onDone,
    this.onError,
    this.onCancel,
    this.onPause,
    this.onResume,
  });

  static const String _debugPrefix = 'ChunkedDownloader:';

  /// Start the download
  /// @result {Future<ChunkedDownloader>} the current instance of the downloader
  Future<void> start() async {
    // Download file
    try {
      int offset = 0;
      var httpClient = http.Client();
      // Set queryParameters
      var uri;
      if (queryParameters != null) {
        uri = Uri.parse(url).replace(queryParameters: queryParameters);
      } else {
        uri = Uri.parse(url);
      }
     // Uri uri = Uri.parse(url);
     // var finalUri = uri.replace(queryParameters: queryParameters);
      var request = http.Request('GET', uri);
      // Set headers
      if (headers != null) {
        request.headers.addAll(headers!);
      }
      //debugPrint('$_debugPrefix uri ${uri}');
      var response = httpClient.send(request);

      // Initialize file
      File file = File(saveFilePath);
      // Number of iterations of while/do loop
      // while the file is downloaded in chunks
      int cycles = 0;
      // Buffer is the bytes readed from readerChunk
      List<int> buffer = [];
      // bytes if the total numbers od bytes at each cycle
      List<int> bytes= [];

      stream = response.asStream().listen(null);
      stream?.onData((http.StreamedResponse r) async {
        // Get file size. Many servers don't provide [content-length], so
        // standard use of this class is to pass fileSize as parameter
        // In case of Presence we can do it coz fileSize is known.
        fileSize = fileSize ?? int.tryParse(r.headers['content-length'] ?? '-1') ?? -1;
        reader = ChunkedStreamReader(r.stream);
        try {

          do {
            // Wait indefinitely if paused
            await waitWhile(() => (paused));

            // Set start time for speed calculation
            int startTime = DateTime.now().millisecondsSinceEpoch;

            // Read chunk
            buffer = (fileSize! - bytes.length  >= chunkSize) ?
                  await reader!.readChunk(chunkSize)
                  : await reader!.readChunk(fileSize! - bytes.length);
            bytes = bytes + buffer;

            // Calculate speed
            int endTime = DateTime.now().millisecondsSinceEpoch;
            int timeDiff = endTime - startTime;
            if (timeDiff > 0) {
              speed = (buffer.length / timeDiff) * 1000;
            }
            // Add buffer to chunks list
            offset += buffer.length;
            if (kDebugMode) {
              debugPrint('$_debugPrefix Downloading ${offset ~/ 1024 ~/ 1024}MB '
                  'Speed: ${speed ~/ 1024 ~/ 1024}MB/s');
            }
            if (onProgress != null) {
              onProgress!(offset, fileSize!, speed);
            }
            // Write buffer to disk
            // the first cycle we override the initial file
            // coz flutter create new file with 1024 bytes
            // default size. On subsequent cycle we append
            // new bytes to previous ones.
            if (cycles == 0) {
              await file.writeAsBytes(bytes);
            } else {
              await file.writeAsBytes(buffer, mode: FileMode.append);
            }
            cycles++;
          } while (bytes.length < fileSize!);
          //debugPrint('$_debugPrefix downloaded file size ${file.lengthSync()}');
          // Send done callback
          done = true;
          if (onDone != null) {
            onDone!(file);
            downloadCompleted = true;
          }
          if (kDebugMode) {
            debugPrint('$_debugPrefix Downloaded file.');
          }

        } catch (error) {
          downloadError = true;
          bytes.clear();
          buffer.clear();
          if (kDebugMode) {
            debugPrint('$_debugPrefix Error downloading: $error');
          }
          if (onError != null) {
            onError!(error);
          }
        } finally {
          reader?.cancel();
          stream?.cancel();
          bytes.clear();
          buffer.clear();
        }
      });
    } catch (error) {
      if (kDebugMode) {
        debugPrint('$_debugPrefix Error downloading: $error');
      }
      if (onError != null) {
        onError!(error);
      }
    }
    // Do not return until download is !completed and !failed
    // Note: this is required because we cannot
    // await stream?.onData((http.StreamedResponse r)
    // and await won't work with this func
    await waitWhile(() => (!downloadCompleted && !downloadError));
  }

  /// Stop the download
  void stop() {
    stream?.cancel();
    reader?.cancel();
    if (onCancel != null) {
      onCancel!();
    }
  }

  /// Pause the download
  void pause() {
    paused = true;
    if (onPause != null) {
      onPause!();
    }
  }

  /// Resume the download
  void resume() {
    paused = false;
    if (onResume != null) {
      onResume!();
    }
  }
}

// This is used to wait until test condition is not false
Future waitWhile(bool condition(), [Duration pollInterval = Duration.zero]) {
  var completer = new Completer();
  check() {
    if (!condition()) {
      completer.complete();
    } else {
      new Timer(pollInterval, check);
    }
  }
  check();
  return completer.future;
}

Hello @AlexV525 can you please run this demo on your side?
It downloads the same 1MB file with three different methods.
Dio takes between 250ms-1.000ms to download 1MB file (200Mps).
Although it looks like the slowest one, we must consider also the write File time.
However, values between 400ms-1.000ms look huge anyway.
Sample
// ignore_for_file: prefer_const_constructors
import 'dart:io';

import 'package:flutter/material.dart';
import 'package:http/http.dart' as http;
import 'package:path_provider/path_provider.dart';
import 'package:dio/dio.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
        useMaterial3: true,
      ),
      home: const MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({super.key, required this.title});
  final String title;

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {

  void activateDownload() async {
    Directory appDocDir = await getApplicationDocumentsDirectory();
    String path = appDocDir.path;

    final file = File('$path/example.jpg');
    final url =
        'https://freetestdata.com/wp-content/uploads/2022/02/Free_Test_Data_1MB_JPG.jpg';

    print('1) Download File $url');
    await downloadFile(file.path, url, onProgress: (bytes, total) {
      if (total != null) {
        //print('donwload file progress ${(100 * (bytes / total)).round()}%');
      }
    });

    print('\n2) Dio');
    print('\nDownload File $url');
    await downloadFileWithDio(url, file.path);

    print('\n3) DownTest Code');
    print('\nDownload File $url');
    await downloadTest(url);

  }

  static Future<void> downloadFile(
      String filePath,
      url, {
        void Function(int, int?)? onProgress,
      }) async {
    final sink = File(filePath).openWrite();
    var httpClient = http.Client();
    var request = http.Request('GET', Uri.parse(url));
    final response = await httpClient.send(request);
    var startTime = DateTime.timestamp();
    if (onProgress == null) {
      await sink.addStream(response.stream);
    } else {
      var bytes = 0;
      await sink.addStream(response.stream.map((e) {
        bytes += e.length;
        onProgress(bytes, response.contentLength);
        return e;
      }));
      print(
          'Downloaded $bytes bytes in ${DateTime.timestamp().difference(startTime).inMilliseconds}ms');
    }
    await sink.flush();
    await sink.close();
  }

  static Future<void> downloadFileWithDio(
      url,
      String filePath,
     ) async {
    Dio dio = Dio();
    var startTime = DateTime.timestamp();
    await dio.download(url, filePath,
      onReceiveProgress: (bytes, total) {
        //int percentage = ((received / total) * 100).floor();
        if (bytes == total) {
          print(
              'Downloaded $bytes bytes in '
                  '${DateTime.timestamp().difference(startTime).inMilliseconds}ms');
        }
      });
    }

  static Future<void> downloadTest(url) async {
    var httpClient = http.Client();

    var request = http.Request('GET', Uri.parse(url));
    var response = await httpClient.send(request);
    var totalLength = 0;
    var startTime = DateTime.timestamp();
    await response.stream.listen((data) {
      totalLength += data.length;
    }).asFuture();
    print(
        'Downloaded $totalLength bytes in ${DateTime.timestamp().difference(startTime).inMilliseconds}ms');
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        backgroundColor: Theme.of(context).colorScheme.inversePrimary,
        title: Text(widget.title),
      ),
      body: const Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            Text(
              'You have pushed the button this many times:',
            ),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: () {activateDownload();},
        tooltip: 'Increment',
        child: const Icon(Icons.add),
      ), // This trailing comma makes auto-formatting nicer for build methods.
    );
  }
}

I just tested with your example, and there is nothing related to file writing. The performances are poor whether using GET+Stream or the download method, so it should be an adapter issue.

This is hard to debug more than I thought. I tried to separate flows but all flows seem working well. The next step I may try is to rebuild all flows from the bottom to the top.

I think there is an issue with how TLS sockets are handled on Android. I have noticed a similar issue on https://github.com/localsend/localsend/issues/384#issuecomment-1513917835

After some debugging, I have noticed, that HTTP is much faster than HTTPS. In my test case, HTTP reached 50 MB/s while HTTPS only reached 7 MB/s.
I am not sure how to fix this as I only use the Flutter API (both with dio and with http package).
It seems that Android is the device causing slow transmission speeds. Windows <-> macOS reached 60 MB/s with HTTPS.

You can try to download a large file via unencrypted HTTP - it should be much faster. Strangely, other platforms don't have this issue.

Edit: I think, the TLS socket is just too CPU demanding. I have tried out to run a test on an old iPhone 5S and it has a similar problem. Turning off encryption offers 2x download / upload speeds. I guess newer iPhones are just too performant to have this problem.

Thanks for the input, and sad that the issue didn't get any notice from the Dart team.
However during my tests, when I was using https://freetestdata.com/wp-content/uploads/2022/02/Free_Test_Data_1MB_JPG.jpg as the source, I can see that http and HttpClient had a more normal speed but our package doesn't.

Transferring a 24mb file from my laptop to my Samsung s22  is only 600KB/s.
I turned off wifi on my phone and turned it back on and the speed went back to normal!

Testing in another network location, the download elapsed duration is slightly higher than the other libraries, but the regression seems not occurred.
I/flutter ( 6803): 1) http write sink https://freetestdata.com/wp-content/uploads/2021/09/png-5mb-1.png
I/flutter ( 6803): Downloaded 5244276 bytes in 4442ms
I/flutter ( 6803): 2) Dio Download https://freetestdata.com/wp-content/uploads/2021/09/png-5mb-1.png
I/flutter ( 6803): Downloaded 5244276 bytes in 4586ms
I/flutter ( 6803): 3) http Stream -> Future https://freetestdata.com/wp-content/uploads/2021/09/png-5mb-1.png
I/flutter ( 6803): Downloaded 5244276 bytes in 4448ms

I/flutter ( 6803): 1) http write sink https://freetestdata.com/wp-content/uploads/2022/02/Free_Test_Data_1MB_JPG.jpg
I/flutter ( 6803): Downloaded 1054294 bytes in 909ms
I/flutter ( 6803): 2) Dio Download https://freetestdata.com/wp-content/uploads/2022/02/Free_Test_Data_1MB_JPG.jpg
I/flutter ( 6803): Downloaded 1054294 bytes in 1039ms
I/flutter ( 6803): 3) http Stream -> Future https://freetestdata.com/wp-content/uploads/2022/02/Free_Test_Data_1MB_JPG.jpg
I/flutter ( 6803): Downloaded 1054294 bytes in 897ms

@iosephmagno Would you be able to run the same code using the 3.16 or a newer version of Flutter?

@AlexV525 it is the same here 👍

@AlexV525 it is the same here 👍

Does the same means the issue does not presist on a newer version?

Yes did a quick test and seems so. We can close this if more tests are not needed.


chore(ci) omit tests from license scan

What this PR does / why we need it:
Removes the test directory before initiating the license scan, to omit test-only dependencies from license reports.
We include our application and its tests in the same repo. The tests aren't compiled into any release artifacts, so it doesn't make sense to include them in the license report we ship with those artifacts. They're also the source of a fair number of license issue reports that we need to investigate and ignore.
Per FOSSA, there's no Go-specific functionality to ignore certain build tags, but deleting those files has the same effect, and we conveniently have those under the same directory (not units, but they're not usually the source of spurious reports because they usually don't add dependencies their package didn't already have).
See test.txt for a POC example with some extra ls calls thrown into the job.

Added go mod tidy as well and had to rearrange some stuff outside of test that linked to test (it was a package that was only used by stuff under /test, I think we had it elsewhere because we didn't have an internal directory under test/ earlier).
This breaks in act for some reason, so https://github.com/rainest/kubernetes-ingress-controller/runs/7657773046?check_suite_focus=true

Closed pending feedback from legal.


CLI returns no such file or directory for version parameter in version 1.1.12

Something is corrupt...
    # npm update -g profitbricks-cli 1.1.12
    npm http GET https://registry.npmjs.org/profitbricks-cli
    npm http 304 https://registry.npmjs.org/profitbricks-cli

    # profitbricks -V
    : No such file or directory

I have to confirm that I was getting the same problem with older versions as well after an update.
My workaround was to call it via node:
/usr/local/lib/node_modules/profitbricks-cli$ node profitbricks.js datacenter list

So I assume the update procedure messes up with links or something.

I'm looking into it
On Tue, Jan 26, 2016 at 10:58 AM, Szymon Bochniak<EMAIL_ADDRESS>wrote:

I have to confirm that I was getting the same problem with older versions
as well after an update.
My workaround was to call it via node:
/usr/local/lib/node_modules/profitbricks-cli$ node profitbricks.js datacenter list
So I assume the update procedure messes up with links or something.
—
Reply to this email directly or view it on GitHub
https://github.com/profitbricks/profitbricks-cli/issues/26#issuecomment-174935305
.

--
Jasmin Gacić

So I just did this:

$ profitbricks -V
1.1.12
$ profitbricks datacenter list
Datacenters
Id                                    Name    Location

b0ac144e-e294-415f-ba39-6737d5a9d419  docker  us/las

On windows I was getting a similar error but after restarting the console it all cleared up

C:\Users\jasmin.gacic>npm update -g profitbricks-cli 1112

C:\Users\jasmin.gacic>projects -V

C:\Users\jasmin.gacic>profitbricks -V
1.1.12

Perhaps try restarting the bash/console after the update.
Also my current version of npm is at on windows 3.3.12 and on ubuntu 3.5.3 you can achieve that by running npm install -g npm@latest

Upgrading npm to the latest version and restarting the console didn't helped. I'm still getting the no such file or directory message.

Can you do this

npm uninstall -g profitbricks-cli
and then
npm install -g profitbricks-cli

Done and worked fine, but the issue is still not fixed.

@botchniaque your latest commit https://github.com/profitbricks/profitbricks-cli/commit/416d43d2304ae14b19ef641925392383b60b7180 has changed file mode for profitbricks file.

Oops, sorry then. That would explain why I was having the issue before - must have messed up.

Thanks, the new version 1.1.14 solved the issue a bit confusing:
    # npm update -g profitbricks-cli 1.1.14
    .
    .
    .
    # profitbricks -V
    1.1.15

I'm glad it helped.
When I published the npm package the code and npm version were not in sync. So I published it one more time and now it is at version 1.1.15. If it is bothering you too much just do npm install -g profitbricks-cli@latest


io.paperdb.PaperDbException: Couldn't rename file

What is the reason for this exception and how to handle this?

Could you provide a more detailed stack trace?

W/System.err: io.paperdb.PaperDbException: Couldn't rename file /data/user/0/com.redtaxi.driver/files/io.paperdb/travelTime.pt to backup file /data/user/0/
W/System.err:     at io.paperdb.DbStoragePlainFile.insert(DbStoragePlainFile.java:117)
W/System.err:     at io.paperdb.Book.write(Book.java:36)
W/System.err:     at com.redtaxi.driver.fragments.RTTripBillFragment.removePrefsValues(RTTripBillFragment.java:158)
W/System.err:     at com.redtaxi.driver.fragments.RTTripBillFragment.onCreateView(RTTripBillFragment.java:67)
W/System.err:     at android.support.v4.app.Fragment.performCreateView(Fragment.java:2087)
W/System.err:     at android.support.v4.app.FragmentManagerImpl.moveToState(FragmentManager.java:1113)
W/System.err:     at android.support.v4.app.FragmentManagerImpl.moveToState(FragmentManager.java:1295)
W/System.err:     at android.support.v4.app.BackStackRecord.run(BackStackRecord.java:801)
W/System.err:     at android.support.v4.app.FragmentManagerImpl.execPendingActions(FragmentManager.java:1682)
W/System.err:     at android.support.v4.app.FragmentManagerImpl$1.run(FragmentManager.java:541)
W/System.err:     at android.os.Handler.handleCallback(Handler.java:739)
W/System.err:     at android.os.Handler.dispatchMessage(Handler.java:95)
W/System.err:     at android.os.Looper.loop(Looper.java:148)
W/System.err:     at android.app.ActivityThread.main(ActivityThread.java:5417)
W/System.err:     at java.lang.reflect.Method.invoke(Native Method)
W/System.err:     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
W/System.err:     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)

Paper renames the original file to file "*.bak" on writing attempt to have a way to recover in case if writing is failed.
https://github.com/pilgr/Paper/blob/master/paperdb/src/main/java/io/paperdb/DbStoragePlainFile.java#L115
The issues may happen because the original data file is locked for some reasons. Do you use the same Paper instance for all read/write operations? Using different Paper instances may cause data inconsistency and crashes like this.
How often does issue happen? Can you provide a test case or a sample code to reproduce it?

I'm using Paper.book().read() and Paper.book().write() throughout the app. My app also have two services running in background which also uses the above methods. Will this cause the issue?

Two services running in one process or in different processes? Paper doesn't support data consistency when accessed from different processes.

They are running in two separate processes. How to handle this? Please suggest me a solution.

Keep all the code accessing Paper in one process, let's say the main process where Paper can be accessed from activities and services. This main process should expose Binder interface with all the necessary API to get data. So another process can use Binder interface to access data without knowing anything about Paper underhood.
Service with Paper, Process1 <-[Another service, Process2]


User & group params are missing

#1 looks like it removed the class params that define the user/group consul-template should run as.
There's several references to $consul_template::user and $consul_template::group params throughout the manifests (Debian init script, resources for managing the user/group if manage_user/manage_group is defined)
This kind of also means there's some parity issues between different operating systems with different init scripts (at least, on Debian it was going to try and start/stop consul-template as a different user)
Personally, I haven't had time to look into addressing this yet but just wanted to make the issue known.

Closed by PR #25


WIP: make ClientContext container-only, never initializing

WIP pending more tests,
but should be good enough for review.
All unit tests pass against both old and regenerated GCJ
but we should integ to make sure.
Adding Neo since a related change broke his code earlier.

Codecov Report

Merging #318 into master will increase coverage by 0.3%.
The diff coverage is 70%.

@@             Coverage Diff             @@
##             master     #318     +/-   ##
===========================================
+ Coverage     69.81%   70.11%   +0.3%
- Complexity      355      358      +3
===========================================
  Files            73       73
  Lines          1925     1921      -4
  Branches        157      158      +1
===========================================
+ Hits           1344     1347      +3
+ Misses          513      506      -7
  Partials         68       68

Impacted Files
Coverage Δ
Complexity Δ

.../google/api/gax/core/FixedCredentialsProvider.java
0% <ø> (ø)
0 <0> (ø)
:arrow_down:

...va/com/google/api/gax/grpc/ChannelAndExecutor.java
75% <ø> (+75%)
3 <0> (+3)
:arrow_up:

...gle/api/gax/grpc/InstantiatingChannelProvider.java
47% <0%> (-0.96%)
9 <0> (ø)

...in/java/com/google/api/gax/grpc/ClientContext.java
100% <100%> (+14.28%)
2 <1> (-1)
:arrow_down:

...n/java/com/google/api/gax/grpc/ClientSettings.java
48.97% <33.33%> (+2.04%)
5 <0> (+1)
:arrow_up:

.../java/com/google/longrunning/OperationsClient.java
93.54% <90%> (-1.91%)
17 <0> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7772e37...259dd74. Read the comment docs.

I tested this against both current GCJ and the one generated by googleapis/toolkit#1319. Each test includes mvn test on all projects and mvn verify on pubsub.
I'll merge this, but hold off on toolkit until a gax version bump.


Add missing css so that wiredep can also pick up.

When user includes angular-upload in his bower.json, and if wiredep is used, wiredep doesn't pick up the css file to be included in the build.  So the user ends up manually adding a override section in his bower.json to include both the js and css file.
The button looks broken with the css and it's not obvious what is wrong for those who use the angular generator for manage the build and etc.

Fixed


Evaluate and remove tmpfs workaround in favor of kubernetes/50596

See: https://github.com/kubernetes/kubernetes/pull/50596
Will evaluate and remove tmpfs workaround.

This does not work yet.
Filed https://github.com/kubernetes/kubernetes/pull/51927 to fix it. Let's see how it goes.

Shouldn't it be the flexVolume driver doing the cleanup if it added some data there? https://github.com/kubernetes/kubernetes/pull/51927#issuecomment-327544251 suggests that doing RemoteAll in the infrastructure code rather than in the individual driver code sounds dangerous.

Confirmed, this work with kubelet 1.7.5+


Parsing invalid JSON causes NodeRED crash

Here a JSON response is assumed, when (for whatever reason) a different response is given NodeRED crashes entirely.
https://github.com/SmartParksOrg/node-red-contrib-earthranger/blob/dev/src/nodes/earthranger-connection/earthranger-connection.ts#L51
We can see this in the following logs:
Mar 27 11:38:47 smartparks-gonarezhou Node-RED[1108008]: 27 Mar 11:38:47 - [red] Uncaught Exception:
Mar 27 11:38:47 smartparks-gonarezhou Node-RED[1108008]: 27 Mar 11:38:47 - [error] SyntaxError: Unexpected token < in JSON at position 0
Mar 27 11:38:47 smartparks-gonarezhou Node-RED[1108008]:     at JSON.parse (<anonymous>)
Mar 27 11:38:47 smartparks-gonarezhou Node-RED[1108008]:     at IncomingMessage.<anonymous> (/root/.node-red/node_modules/@smartparks/node-red-contrib-earthranger/dist/nodes/earthranger-connection/earthranger-connection.js:33:34)
Mar 27 11:38:47 smartparks-gonarezhou Node-RED[1108008]:     at IncomingMessage.emit (node:events:539:35)
Mar 27 11:38:47 smartparks-gonarezhou Node-RED[1108008]:     at endReadableNT (node:internal/streams/readable:1345:12)
Mar 27 11:38:47 smartparks-gonarezhou Node-RED[1108008]:     at processTicksAndRejections (node:internal/process/task_queues:83:21)

The fix would be to surround the JSON.parse(str) in a try/catch block and fail gracefully. The same goed for this: https://github.com/SmartParksOrg/node-red-contrib-earthranger/blob/dev/src/nodes/earthranger-connection/earthranger-connection.ts#L106

agreed upon suggestion and tested it in the lastest commit. will be released in 1.3.3


fix for LazyGraphModule

Try to fix https://github.com/thuml/depyf/issues/9 .

Comment: add compatibility support for PyTorch after https://github.com/pytorch/pytorch/pull/118201 and https://github.com/pytorch/pytorch/pull/117911 .


Added preliminary support for multi-machine F1 cosimulation (WIP)

This is a WIP PR at the moment. I'd like to get feedback.
We're quickly approaching a point where we'll have multiple top-level files for F1 (or even FPGA) infrastructure. This is necessary because we'll be building 16x8 and 4x4 manycore designs on F1. Some of those will have 4 DDRs (A, B, C, D) and some will only have 1 DDR (C), which means different top-level files (@leonardxiang can attest to this), and different ROMs (see the PR).
What I've done here, is work with the Makefile.machine.include file to define multiple types of machines. However, instead of having multiple Makefile.machine.include files (which would vastly complicate our F1 infrastructure) there's only one Makefile.machine.include file. This file requires MACHINE_TYPE to be set, which then includes the specific machine configuration from an underlying makefile.
I like this approach because it appears similar to our previous approach. It's not a radical departure from the previous commit. It's also extensible. With a little effort, I can build multiple AGFIs for Amazon with different sizes/configurations. With each machine also defining it's own top-level file, we can support different top-level targets (E.g. the VCU 128 board).
I don't like this approach because it requires MACHINE_TYPE to be set to ensure that the correct binary is compiled/loaded. If we have multiple F1 images, people would need to switch between the different MACHINE_TYPE variables.
Maybe this is a good motivation for only compiling one (LARGE) F1 design, and running cosimulation regression on the 4x4 (as we already do).
Changelist:

Moved Makefile.machine.include into hardware directory
Created 4x4_DDRC, and 16x8_DDRC directories
Added machine.mk fragments for defining machine parameters (used by Makefile.machine.include)
Added Makefile variable MACHINE_TYPE that must be defined at compilation time
Added HARDWARE_MACHINE_FILE variable that contains file-path of machine.mk

This PR only works for the library test programs (at the moment)

Hi Dustin,
Maybe pull in Bandhav to take a look. I believe we had a similar system for
testing the ASIC?
On Wed, Nov 13, 2019 at 12:06 PM Dustin Richmond<EMAIL_ADDRESS>wrote:

@drichmond commented on this pull request.
In hardware/Makefile.machine.include
https://github.com/bespoke-silicon-group/bsg_f1/pull/459#discussion_r345976010
:

+$(error $(shell echo -e "$(RED)BSG MAKE ERROR: HARDWARE_PATH is not defined$(NC)"))
+endif

+KNOWN_MACHINE_TYPES = $(notdir $(shell find $(HARDWARE_PATH) -maxdepth 1 -mindepth 1 -type d))
+# MACHINE_TYPE: The type of machine to target.
+ifndef MACHINE_TYPE
+$(error $(shell echo -e "$(RED)BSG MAKE ERROR: MACHINE_TYPE is not defined$(NC)"))
+endif
+
+# Confirm that MACHINE_TYPE is valid
+ifeq ($(filter $(MACHINE_TYPE),$(KNOWN_MACHINE_TYPES)),)
+$(info $(shell echo -e "$(RED)BSG MAKE ERROR: Unknown MACHINE_TYPE $(MACHINE_TYPE)"))
+$(error $(shell echo -e "$(RED)BSG MAKE ERROR: Valid MACHINE_TYPE values are $(KNOWN_MACHINE_TYPES)"))
+endif
+
+HARDWARE_MACHINE_PATH = $(HARDWARE_PATH)/$(MACHINE_TYPE)
The downside of using MACHINE_TYPE is that it needs to be defined in a
file somewhere. I haven't quite figured out where the best place to define
it is.
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/bespoke-silicon-group/bsg_f1/pull/459?email_source=notifications&email_token=AEFG5ABQHYZBJE6HTWNNKDLQTRM4HA5CNFSM4JNAZ7VKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCLO33NQ#discussion_r345976010,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEFG5ADWXMFMUKDXMZEBMVLQTRM4HANCNFSM4JNAZ7VA
.

We do! And he's on this PR.
This is slightly more complex than the manycore design because we run more than a single simv, or simv_debug program.

@drichmond
To toss an idea, we have a different approach for a similar problem where we had to simulate multiple process corners for the tapeout. For each run (meaning for each machine config), we clone a separate run directory (separate bsg_manycore) and point the machine path to simv path to simv generated in bsg_design for each run.
Approach above needs zero flow changes and can be applied to bsg_f1 as well I think. Roughly, we can have a makefile in bsg_bladerunner that clones multiple copies of bsg_f1 each pointing to a simv of our choice. Few things about this:

There'd be space overhead due to multiple clones. Not a serious issue if we do a shallow clone.
Very straight forward to do parallel runs.
Very easy to debug as each run has completely separate environment.
Needs zero flow changes to existing infrastructure.

Here's how it was done for tapeout:
https://github.com/bespoke-silicon-group/bsg_manycore/blob/master/software/regress/Makefile
What do you think?

The flow to compile machines with different configurations (which is main objective of this pr?); lgtm.
@leonardxiang Any specific issues preventing us from parameterizing the toplevel for different configs?

@drichmond
To toss an idea, we have a different approach for a similar problem where we had to simulate multiple process corners for the tapeout. For each run (meaning for each machine config), we clone a separate run directory (separate bsg_manycore) and point the machine path to simv path to simv generated in bsg_design for each run.
Approach above needs zero flow changes and can be applied to bsg_f1 as well I think. Roughly, we can have a makefile in bsg_bladerunner that clones multiple copies of bsg_f1 each pointing to a simv of our choice. Few things about this:

There'd be space overhead due to multiple clones. Not a serious issue if we do a shallow clone.
Very straight forward to do parallel runs.
Very easy to debug as each run has completely separate environment.
Needs zero flow changes to existing cosim infrastructure.

Here's how it was done for tapeout:
https://github.com/bespoke-silicon-group/bsg_manycore/blob/master/software/regress/Makefile
What do you think?

@vb000 In these different process corners, you had the same top-level file right?
I like the ability to parallel runs (especially in separate environments)

We do a pretty similar thing in BlackParrot except without the code clone. Separate build directories and separate run directories which symlink to the respective simv

We do a pretty similar thing in BlackParrot except without the code clone. Separate build directories and separate run directories which symlink to the respective simv

This is also a feasible solution, but I fear a 'state explosion' problem. I posed this to @mrutt92 too.
We have regression and testbench directories. Each directory has specint cuda, spmd, library, and python directories with tests. If we add M machines, then each of those directories will have M new directories for each machine - so we're adding  ~ M * 5 * 2 new directories to our tree to support different machine types.
It would certainly do the job, and it would allow us to run all our regression tests in parallel.

I suppose what I'm asking is - do developers, the people who might use this repository, prefer directory hierarchies, configuration files, or magic?
We've had a fair amount of debate recently about how to make this whole environment easier to use.

@vb000 In these different process corners, you had the same top-level file right?
I like the ability to parallel runs (especially in separate environments)

Yes, same toplevel. But that's only because we taped out one design. Need not be same toplevel, as each simv could be compiled from different toplevels.


Not found in repository <-> no error when checking

Output of restic version
restic 0.8.3
compiled with go1.10 on linux/amd64
How did you run restic exactly?
/usr/local/bin/restic -r sftp:xxxxxxxxx@xxxxxxxxx.xxxxxxxxx.xx:/restic.xxxxxxxxx restore latest --target /
What backend/server/service did you use to store the repository?
SFTP (Hetzner Storage Box)
Expected behavior
I expect no errors.
When I run the following command:
/usr/local/bin/restic -r sftp:xxxxxxxxx@xxxxxxxxx.xxxxxxxxx.xx:/restic.xxxxxxxxx check --read-data
I get the following result:
<EMAIL_ADDRESS>password:
enter password for repository:
password is correct
create exclusive lock for repository
load indexes
check all packs
check snapshots, trees and blobs
read all data
[10:30:24] 100.00%  51187 / 51187 items
duration: 10:30:24
no errors were found
Actual behavior
When restoring I get these errors (truncated):
ignoring error for /www/xxxxxxxxx/_combined-5745.pdf: id 503eda8ba97951f19dbcf0f6ea94d0c6576e0e9a91942e9a2640fe7a338f80d3 not found in repository
ignoring error for /www/xxxxxxxxx/_combined-5746.pdf: id 47687078cee87101caa22c0e4345782201f5193e935b4e5ffac8d8e3f605637a not found in repository
ignoring error for /www/xxxxxxxxx/_combined-5763.pdf: id b8756cd759871d4a9350e258ea42f21c56357a417e932b5bf7e98b283157285f not found in repository
ignoring error for /www/xxxxxxxxx/_combined-5781.pdf: id 88b50858a4c42b8c46026e063a366fe55e9c4d003baa08d9c8bad7ff664d4a43 not found in repository
ignoring error for /www/xxxxxxxxx/_combined-5819.pdf: id 8142c934d18844512e5a13bedbfc07c65c2d50e09b3071dc0785d9dc39224249 not found in repository
ignoring error for /www/xxxxxxxxx/_combined-5856.pdf: id 57fdb16ff8bc2017a683e0490cda15bd932239ff250cec144e569eb52c30c334 not found in repository
ignoring error for /www/xxxxxxxxx/_combined-5912.pdf: id 1d6fa838e56116af06949a4775da636262b35d71269fe8433ba189fa63a7aba4 not found in repository
Do you have any idea what may have caused this?
I'm currently doing an new restore (might take a while and I'll report back). I'm hoping it is related to a transport error, but the phrase "not found in repository" makes me wonder if there is an error somewhere?
Also, when restoring those error files, they show up as zero byte files - which is contradictory to the "ignoring error" phrase.
Do you have an idea how to solve the issue?
Nope. If it is related to transport error, it should display something else as an error message. Now it looks like there is an error in the repository while restoring ...

Uh, this is odd. The restic check command will read all data for all snapshots and make sure the right data is returned. I'm confident that when check returns no errors, all data is there.
I don't have any idea what may have happened here. You can try to help debugging this:

Restore a subset of the data which still shows the error, by using the --include option to restic restore, ideally that'd be just a single file or a directory with a few files in it.
Rebuild restic and create a debug log (instructions)
Run restic restore with the debug log and attach it to the issue. Please make sure to replace any sensitive details included in there (file names, passwords etc) before posting it.

Maybe we can see what's going on...
Let us know if you got stuck somewhere.

Ok, so I'm now into 80% of my additional restore, and I don't get any "ignoring errors" anymore (and the previous zero byte files are now extracted perfectly), so I'm thinking either: the previous transmission was interrupted, or the storage box I was using was overloaded.
Either way, the notification was not completely correct - since the file is indeed in the repository. So maybe you could adjust the wording a bit there? My initial thought was: "Ah, bollocks ... and there I thought Restic was reliable. Not." while actually, it was :)

Thanks for the feedback. It's hard to say from the outside what's going on here... Did you get any other messages at the start of the restore, complaining that index files couldn't be loaded? That's the only way I can come up with how this error message could have happened...

Nope. No additional errors in the beginning. Can it be that the connection was interrupted that it starts showing error messages?


Fix bug that Sys::Socket#accept does not work well in Windows.

Fix bug that Sys::Socket#accept does not work well in Windows.
socklen_t
MinGW data type
https://ja.osdn.net/projects/mingw/lists/archive/users/2020-June/000536.html
Windows accept
https://learn.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-accept

All tests are passed now.


merchants: add basicswapdex

reddit thread 11 months ago https://www.reddit.com/r/Monero/comments/111j8c1/an_overview_of_how_xmrbtc_swaps_and_htlc_swaps/
monerokon2023 talk https://particl.news/monerokon-2023-presentation-kewde/
tor url: https://discord.me/particl/post/basicswap-website-live-on-tor
i2p on roadmap https://basicswapdex.com/roadmap
are they open source? do they have XMR/FIAT pairings?
preview page: https://deploy-preview-2230--barolo-time-757cf9.netlify.app/community/merchants/

are they open source?

i believe the project is mit (checking)

do they have XMR/FIAT pairings?

no

the description ive used is directly pulled from their site (very colorful) and should probably be changed so its inline with the others.

Trade cryptocurrencies using atomic swaps with unmatched ease of use. No terminal commands. No order matching issues. No manual swap steps to complete.
could be:
Decentralised atomic swaps with emphasis on ease of use?

Im ok with it as is, but im not creative. Either or is fine with me, but often i use the description as written by the project (so long as its accurate and doesnt contain profanity, and has quotes so its not "me" who wrote it as ab endorsement)


Hi , XPaxos has been source-opened completely?  including all the features that a multi-paxos system should have ?

Yes, it's have include all feature already.


[IMP] travis: Add PyPi deploy

Adds compatibility for auto-deploy as introduced in LasLabs/python-quality-tools#3

Current coverage is 97.64% (diff: 100%)

Merging #29 into master will not change coverage

@@             master        #29   diff @@
==========================================
  Files            59         59
  Lines          1574       1574
  Methods           0          0
  Messages          0          0
  Branches         39         39
==========================================
  Hits           1537       1537
  Misses           19         19
  Partials         18         18

Powered by Codecov. Last update c143f0a...6200274

Coverage remained the same at 98.729% when pulling 620027403ad06a7cffa07445cbd40d67ef22dcb3 on feature/master/travis-pypi into c143f0a68157648d2d96f710c170936eac87b0f9 on master.

It seems it's not you:
2.68s$ travis_after_success
Submitting coverage to coveralls.io...
/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:334: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  SNIMissingWarning
/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:132: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
Traceback (most recent call last):
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/bin/coveralls", line 11, in <module>
    sys.exit(main())
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/coveralls/cli.py", line 62, in main
    result = coverallz.wear()
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/coveralls/api.py", line 93, in wear
    response = requests.post(self.api_endpoint, files={'json_file': json_string})
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/requests/api.py", line 110, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/requests/adapters.py", line 497, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed


Get 404 when trying reach token endpoint to create a new token

Bug Report
Current Behavior
I have followed the steps to deploy Quirrel on my own hosting. but when I get to this step:
https://docs.quirrel.dev/deploying#acquire-a-token
I get an 404 when I try to do that request.
I can access ui and login so thats not an problem :)
Expected behavior/code
I can do https://docs.quirrel.dev/deploying#acquire-a-token
Environment

Quirrel version: 1.6.1

Hi @davidsoderberg! Have you set the PASSPHRASE environment variable?

Oh shoots, it's actually called PASSPHRASES(plural). Does that fix your issue? https://github.com/quirrel-dev/quirrel/blob/54641d15bc224d1819e3d5c54b91024aeea0c680/src/api/main.ts#L11

@allcontributors please add @davidsoderberg for docs

No that did not solve my problem :(

Tried it locally, works for me. What image are you using precisely? I'm on ghcr.io/quirrel-dev/quirrel:main.

I am running the dockerfile in this repos main branch at railway.app.

Tried that one too, and it works for me. Could you provide a bash script that I can use to reproduce your exact bug on a fresh machine?

Or, before you do that, what's the exact curl command you run?

The one specified in documentation.
curl --user ignored:{PASSPHRASE} -X PUT {QUIRREL_SERVER_URL}/tokens/{NAME_OF_TOKEN}

So e.g. curl --user ignored:mysupersecurepassphrase -X PUT david-quirrel.railway.app/tokens/test?

Yes exactly

But with https://

If you go to /documentation, can you see the /tokens endpoint in the OpenAPI Docs?

No

That points towards the PASSPHRASES environment variable not being set properly. Can you verify it actually gets passed into the container?

Yes that was the problem, but mainly it was because of the doc did not state correct ENV.


Realm avoid Association property

ProductName:	Mac OS X
ProductVersion:	10.10.5
BuildVersion:	14F1021

/Applications/Xcode.app/Contents/Developer
Xcode 7.2
Build version 7C68

/usr/bin/pod
0.39.0
Realm (1.0.2)

/bin/bash
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin14)

(not in use here)

/usr/bin/git
git version 2.5.4 (Apple Git-61)

#User.h
@interface User : RLMObject
@property int64_t id;
@property NSString *name;
@property NSString *screen_name;
@property NSData *color;
@end
@interface User (Test)
@property (nonatomic, assign) NSInteger age;
@end

#User.m
@implementation User

+ (NSString *)primaryKey
{
    return @"id";
}
@end

@implementation User (Test)

- (void)setAge:(NSInteger)age {
    objc_setAssociatedObject(self, @selector(age), @(age), OBJC_ASSOCIATION_RETAIN_NONATOMIC);
}

- (NSInteger)age {
    return [objc_getAssociatedObject(self, _cmd) integerValue];
}
@end

The user's age also be saved? how avoid Association property ?

See the Ignored Properties section of the documentation for information on how to prevent specific properties from being included in your Realm.

I'm closing this issue as it seems to be sufficiently answered.


4e: GET routes for /api/<category, product, tag>/:id - returns a single lookup value

[ ] Functional for Products, Categories, and Tags
[ ] Include associated table records.


Style/ArgumentsForwarding` should allow or forbid specific argument names

Is your feature request related to a problem? Please describe.
As it is today, Style/ArgumentsForwarding is great for cutting down on the redundant noise of argument passing.  Using anonymous argument names clearly signals that the arguments are either ignored or forwarded.  It was the first autocorrection I ran after upgrading to ruby 3.2.
However, argument names can be used to document more than just "this is forwarded".  E.g: these names can document what they are represent, and where they are forwarded.  This is especially important for gem authors and public APIs.  As ruby language servers mature and achieve greater adoption, this documentation becomes more useful even on internal or private methods.
If the names are simple *rest, *args, **kwargs, **opts, and similar, then the names are (generally) not really serving any purpose but noise.  But if the names are **scope_options or *authenticator_args and so on, now they might be worth preserving.
Describe the solution you'd like
I would like (at least) two new configurable attributes: AllowedNames and ForbiddenNames.  Both options should be an array of either regular expressions or strings.  The defaults could be the same as today: forbid all and allow none.
We might add separate configuration options for each of the rest, kwrest, and block parameters, but I personally would prefer to keep it simpler with only these two options.
Describe alternatives you've considered

Just let rubocop autocorrect names that are unique and helpful... but only a little bit helpful.
Add # rubocop:disable Style/ArgumentForwarding every time I feel the names are important to the documentation.

Additional context
When I upgraded to ruby 3.2, I used rubocop to autocorrect almost every single Style/ArgumentsForwarding offense in my codebase.  For at least 9 out of ten changes, I agreed completely.  For almost all of the rest, it wasn't really worth the effort to deal with them on a case-by-case basis... but if I'd had these configuration attributes, it would have been worth my time to configure.
Thanks!

I've opened https://github.com/rubocop/rubocop/pull/12397 to resolve this issue. It's a different approach from the proposal, but it addresses the issue effectively.


Download go modules in docker container

This makes the building process faster, without waiting for dependencies

This means that we're letting a docker container touch files in your GOPATH. Can we oversee the effect on the ownership and permissions of these files if they are newly created?
We could also opt for not binding the local go directory, but mounting a volume (either a named, anonymous or tmpfs mount). This way we still have separation, but the packages do not have to be re-downloaded between runs.

This means that we're letting a docker container touch files in your GOPATH. Can we oversee the effect on the ownership and permissions of these files if they are newly created?
We could also opt for not binding the local go directory, but mounting a volume (either a named, anonymous or tmpfs mount). This way we still have separation, but the packages do not have to be re-downloaded between runs.

This means that we're letting a docker container touch files in your GOPATH. Can we oversee the effect on the ownership and permissions of these files if they are newly created?
We could also opt for not binding the local go directory, but mounting a volume (either a named, anonymous or tmpfs mount). This way we still have separation, but the packages do not have to be re-downloaded between runs.

Great idea! I mounted a volume instead.

This means that we're letting a docker container touch files in your GOPATH. Can we oversee the effect on the ownership and permissions of these files if they are newly created?
We could also opt for not binding the local go directory, but mounting a volume (either a named, anonymous or tmpfs mount). This way we still have separation, but the packages do not have to be re-downloaded between runs.

Great idea! I mounted a volume instead.


[netlify adapter] add note on Netlify node version

this is just a tweak to readme, to address confusion from users https://twitter.com/BrittneyPostma/status/1416101106648813572

I think the minimum version is 12.20, not 14. For some odd reason netlify defaults to 12.16.
And maybe .node-version instead of .nvmrc would be a more suiting filename

I tweaked the docs a bit based on the comments above. I wonder instead of documenting if we should just automatically create these files like https://github.com/sveltejs/kit/pull/1924 is suggesting?

@ben sorry i took a haitus from github notifications and only just saw this. i think a simple fix like the readme warning works well here since I don't expect Netlify to stick to v12 too much longer. can leave the ignore discussion to that other PR


The demo does not work because of the new insecure origins policy

The getUserMedia() function no longer works when hosted via insecure origins (non-https).

Just a quick note that you can still see the demo if you clone the repo and view it locally. As long as you have Node.js installed you can do the following in the funky directory:
npm install http-server -g
http-server
go to <IP_ADDRESS>:8080

I will try that, but will be cool to have the demo online
Thanks!

hi,
is there any way to make this demo working ?
sorry i didn't understand thomasmaclean solution.
Thanks

@purestudio
running locally
To make it work locally you need to have a server running. It could be a node server like @thomasmaclean suggested or any other server.
If you have PHP installed it could be a command php -S localhost:8080 as well
Then you need to navigate to http://localhost:8080/examples
make it work locally
This library is using old deprecated API to make it work you need to change those lines

I can prepare PR, but author needs to approve it.  Because it looks like project is abandoned I will not do it.
Will you approve my PR @43081j

yup, sorry for not keeping up with this, its one of a couple of aged repos now i haven't had any time to properly maintain.
if you do a PR i will be happy to review it

Fixed in 68f86a3 which has been merged into matser

Fixed in 68f86a3 which has been merged into matser


Upgrade to Django 4.1

This updates the Django version to 4.1 and normalizes the app structure to make sure all app configs are properly detected.
Impact

[ ] New migrations
[ ] New/Updated API fields or mutations
[ ] Deprecated API fields or mutations
[ ] Removed API types, fields, or mutations
[ ] Documentation needs to be updated

Pull Request Checklist

[ ] Privileged queries and mutations are guarded by proper permission checks
[ ] Database queries are optimized and the number of queries is constant
[ ] Database migration files are up to date
[ ] The changes are tested
[ ] GraphQL schema and type definitions are up to date
[ ] Changes are mentioned in the changelog

This will need to be reverted for Django 4.x: https://github.com/saleor/saleor/pull/11073


Google Cloud NLP integration

This PR Integrates Google Cloud NLP service for identifying place mentions in the search query.
This is the first step towards having a single search bar instead of two one for the question and one for the place selection.
A big limitation is that currently the id associated to the place is a Freebase MID. We will therefore need a mechanism to map back to dcids at some points. Do you have any suggestions @shifucun ?

Please fix the lint error with ./run_test.sh -f


Can't give players permission to use a pixelmon pc in my claim.

i seem to be having trouble with /cf interact-block-secondary pixelmon:pc true, I can't seem to get it to let people use the pc even with that entered it still tells players in my claim they do not have my permission to interact with it.I was not sure if this seems to be a issue on my side or if more people have this same issue.
When i use the command it gives me the feedback that the command worked but yet people still can't use the pc.

im also running pixelmon generations v2.5.1 if that changes how this command needs to be entered.


Release Google.Cloud.Functions.V2 version 1.5.0

Changes in this release:
New features

Change netstandard2.1 target to netstandard2.0 (commit 82bea85)

Created release for Google.Cloud.Functions.V2-1.5.0


[Bugfix] Fix eval_fanout of validation dataloader for lp task in multi-task learning

Issue #, if available:
The way to set the eval_fanout when creating the validation datalaoder for a link prediction task in multi-task learning setting is incorrect. It will crash the training script.
Description of changes:
Update the code to use the global eval_fanout for the validation dataloader.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Could you update the PR description to explain what the bug is and how the current code fixes this bug.

Could you update the PR description to explain what the bug is and how the current code fixes this bug.

Added


The "cast" command is not working when using console

The "cast" command doesn't work from console.
When using the command /cast Smoke-Bomb tx 8270 ty 66 tz 8272 tworld world in that world as a player it works fine. When using this command in the console the particle doesn't appear while with debugging it says it did cast the spell
Steps to reproduce:

/mvtp world
/tp 0 100 0
/cast Smoke-Bomb tx 0 ty 100 tz 0 tworld world duration 3000
Expected Result: Smoke-Bomb casts.
Execute the following command through console:
/cast Smoke-Bomb tx 0 ty 100 tz 0 tworld world duration 3000
Expected Result: Smoke-Bomb casts.
Actual Result: Nothing seems to happen.

The Smoke-Bomb particle:
Smoke-Bomb:
    icon: feather
    effects:
        cast:
        -   class: EffectSingle
            location: target
            effectlib:
                class: SphereEffect
                particle: CLOUD
                particles: 10
                radius: 0
                radiusIncrease: 0.25
        -   class: EffectSingle
            location: target
            effectlib:
                class: SphereEffect
                particle: CLOUD
                particles: 15
                radius: 0
                radiusIncrease: 0.2
        -   class: EffectSingle
            location: target
            effectlib:
                class: SphereEffect
                particle: CLOUD
                particles: 20
                radius: 0
                radiusIncrease: 0.15

When casting from console you also need to specify the source location. Try adding "px 8270 py 66 pz 8272 pworld world".
The p stands for player, this is used to mimic a player casting a spell.

We tried doing that with this command:
/cast Smoke-Bomb tx -125 ty 40 tz -520 tworld AWorld px -100 py 40 pz -500 pworld AWorld
This however didn't result in a particle at that location

So I tried this out myself, and I can't figure out why it wouldn't be working for you. I was about about the "p" location parameters, in your case everything is happening at the target, so it doesn't really matter except it might look weird on dynmap.
I was able to "cast Smoke-Bomb tx 0 ty 65 tz 0 tworld world" from the console and see it in-game.
One thing that may help, or at least will make this a little less complex and more consistent, is to add a few parameters:
Smoke-Bomb:
    icon: feather
    effects:
        cast:
        -   class: EffectSingle
            location: target
            effectlib:
                class: SphereEffect
                particle: CLOUD
                particles: 10
                radius: 0
                radiusIncrease: 0.25
        -   class: EffectSingle
            location: target
            effectlib:
                class: SphereEffect
                particle: CLOUD
                particles: 15
                radius: 0
                radiusIncrease: 0.2
        -   class: EffectSingle
            location: target
            effectlib:
                class: SphereEffect
                particle: CLOUD
                particles: 20
                radius: 0
                radiusIncrease: 0.15
    parameters:
        duration: 3000
        range: 1
        allow_max_range: true

That will make it so simply doing "/cast Smoke-Bomb" in-game works for testing, it will appear 1 block in front of you. It may also help your console-casting issue, in case it was being counted as  "miss" for some reason. Though I did try different coordinate with cast both above ground and underground, and it seemed to work pretty reliably for me. I hope you can figure it out, let me know if I can help more.

I added the parameters and it didn't appear when run from console. I also tested the command from a different world, the particle didn't appear at the target location.
We are running version 7.0-snapshot.

I'm out of ideas, sorry!
If it works for me using your exact spell configs and command, there's not a whole lot else I can do.

Hey Nathan,
Sorry for not coming back to you earlier.
We have tried some things and using "precast" instead of "cast" did the trick.


Add sanitizer option

please try introducing a memory error and show that the CI fail in that case.

please try introducing a memory error and show that the CI fail in that case.

https://github.com/f3d-app/f3d/runs/4826250319?check_suite_focus=true#step:16:363
Failing as expected considering the leak I introduced

Failing as expected considering the leak I introduced

Nice !


add has_and_belongs_to_many

Not ready yet
need change try_load method for support multiple id (need that for polymorphic)


refactor: remove deprecated Dockerfiles

Description

#98

This PR removes directories that contain Dockerfiles and shell scripts to build deprecated images. Note that this is going into a refactor branch

Review guidelines

Estimated Time of Review: 2 minutes

Checklist before merging:

[ ] Confirm that the relevant changelog(s) are up-to-date in case of any user-facing changes

Correct


Create PledgedMerc Vault Token

This needs some more testing, but wanted to get your feedback on how it
is looking so far. Big things to check out:

Adapted solmate's 4626 to be OZ friendly
Clone a PledgedMerc vault when creating a new gauge
Restrict the gauge's pledge and depledge to the vault token

lgtm. feel free to merge when ready


Make use of %license macro

Make use of %license macro

Merging. The failed tests have nothing to do with these changes.


Typo fixes in Parts.hs

Fixed the words concatenate and separate

Cool, thanks

Wow that was quick.


fix: Support enums with unbounded wildcards

This is a fix for enums with unbounded wildcards. We get a NullPointerException otherwise because enumConstants is null.

Thanks!


Visual Studio 2017 (or old vers) Can't install It on my pc cuz of admin HELP!!!

Well i need help to find a way to use Visual studio and download it without admin
Computer : Windows 10, school security type such as WINDOWS DEFENDER Group strategy and more shit like this.
I need it for PERSONAL reasons If you really need to know Well let me know.

this is repository for Visual Studio CODE not Visual Studio 2017. I highly doubt you're going to get any help here!

Sorry we can't help with that, please file issues against big brothers.


Modus table scrollbar development

Description
implement vertical and horizontal scrollbar for modus table.
References #1307
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation update

How Has This Been Tested?

Checklist

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[x] Any dependent changes have been merged and published in downstream modules
[x] I have checked my code and corrected any misspellings

Closing this PR
new PR: https://github.com/trimble-oss/modus-web-components/pull/1538


rename does not work

Renaming either hangs or completes, but does not do the rename.
For example:

$ bean rename -a $A2 Bean1
Found Bean with name/address: /987bf3591c32
Connected!
Renaming Bean to: Bean1
Command completed successfully

Or hang and after waiting for few minutes control-c:

$ bean rename -a $A1 Shuckle1
Found Bean with name/address: Bean+/987bf358352b
Connected!
Renaming Bean to: Shuckle1

but after those tests names are still Blue+:

$ sudo blescan
Scanning for devices...
Device (new): 98:7b:f3:58:35:2b (public), -92 dBm
Flags: <06>
Tx Power: <04>
Incomplete 128b Services:
Complete Local Name: 'Bean+'
Device (new): 98:7b:f3:59:1c:32 (public), -58 dBm
Flags: <06>
Tx Power: <04>
Incomplete 128b Services:
Complete Local Name: 'Bean+'

I also restarted Bean+ devices several times, but that did not help.

Fixed  in 0.4.0 version


imports hotfix

fixed bad imports in README
fixed go get command in README

Codecov Report

Merging #1 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master       #1   +/-   ##
=======================================
  Coverage   95.87%   95.87%
=======================================
  Files          17       17
  Lines         436      436
=======================================
  Hits          418      418
  Misses         16       16
  Partials        2        2

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cf136df...f03082d. Read the comment docs.


Add blossom-ci to github workflows

Enable Blossom CI integration by adding the blossom-ci.yml file.

My bad, this repo doesn't have a GitLab side


Feature: 404 page

404 page:
**A styled 404 page to replace this empty 404 page:

Test URL: https://404-page--stock--wbstry.hlx.page/pages/404.html

It selects a random video each time you refresh the page.

XD Specs: (A), (B), (C)

@So1ene fix the conflicts and we can merge this.

@auniverseaway the conflicts were fixed!

Closing per @williambsm request.