Could not resolve com.aheaditec.talsec.security:TalsecSecurity-Community-Flutter:9.6.0.

What went wrong:
Execution failed for task ':app:checkDebugAarMetadata'.

Could not resolve all files for configuration ':app:debugRuntimeClasspath'.
Could not resolve com.aheaditec.talsec.security:TalsecSecurity-Community-Flutter:9.6.0.
Required by:
project :app
project :app > project :freerasp
Could not resolve com.aheaditec.talsec.security:TalsecSecurity-Community-Flutter:9.6.0.
Could not get resource 'https://nexus3-public.monetplus.cz/repository/ahead-talsec-free-rasp/com/aheaditec/talsec/security/TalsecSecurity-Community-Flutter/9.6.0/TalsecSecurity-Community-Flutter-9.6.0.pom'.
Could not GET 'https://nexus3-public.monetplus.cz/repository/ahead-talsec-free-rasp/com/aheaditec/talsec/security/TalsecSecurity-Community-Flutter/9.6.0/TalsecSecurity-Community-Flutter-9.6.0.pom'.
Connect to nexus3-public.monetplus.cz:443 [nexus3-public.monetplus.cz/<IP_ADDRESS>] failed: Connect timed out
Could not resolve com.mob:MobLink:+.
Required by:
project :app
Failed to list versions for com.mob:MobLink.
Unable to load Maven meta-data from https://europe-west3-maven.pkg.dev/talsec-artifact-repository/freerasp/com/mob/MobLink/maven-metadata.xml.
Could not GET 'https://europe-west3-maven.pkg.dev/talsec-artifact-repository/freerasp/com/mob/MobLink/maven-metadata.xml'.
Connect to europe-west3-maven.pkg.dev:443 [europe-west3-maven.pkg.dev/<IP_ADDRESS>] failed: Connect timed out

my flutter app config:
android -> build.gradle
buildscript {
ext.kotlin_version = '1.7.10'
repositories {
google()
mavenCentral()
maven { url "https://mvn.mob.com/android" }
maven { url "https://developer.huawei.com/repo/" }
}
dependencies {
classpath "com.android.tools.build:gradle:7.3.0"
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
classpath "com.mob.sdk:MobSDK2:+"
classpath "com.huawei.agconnect:agcp:<IP_ADDRESS>0"
}
}
allprojects {
repositories {
google()
mavenCentral()
maven { url "https://mvn.mob.com/android" }
maven { url "https://developer.huawei.com/repo/" }
maven { url "https://storage.googleapis.com/download.flutter.io" }
maven { url "https://nexus3-public.monetplus.cz/repository/ahead-talsec-free-rasp" }
maven { url 'https://www.jitpack.io' }
}
}
how to resolve this issue?

Hello,
how did you integrated the freeRASP? The maven { url "https://nexus3-public.monetplus.cz/repository/ahead-talsec-free-rasp" } should not be in the gradle of your app.
Did you install the plugin using flutter pub add freerasp, exactly as described in the documentation?
Kind regards,
Talsec team

first, thanks msikyna replies.
I integrated the freeRASP plugin for flutter, in my pubspec.yaml file:
environment:
sdk: '>=3.1.0 <4.0.0'
dependencies:
flutter:
sdk: flutter
cupertino_icons: ^1.0.6
sentry_flutter: ^8.7.0
sentry_dio: ^8.7.0
freerasp: ^6.6.0
android -> app -> build.gradle configs:
android {
compileSdkVersion 34
buildToolsVersion '34.0.0'
lintOptions {
disable 'InvalidPackage'
}
compileOptions {
sourceCompatibility JavaVersion.VERSION_1_8
targetCompatibility JavaVersion.VERSION_1_8
}
defaultConfig {
    applicationId "com.xxxxxxxx.xxx"
    minSdkVersion 23
    targetSdkVersion 34
    versionCode flutterVersionCode.toInteger()
    versionName flutterVersionName
    multiDexEnabled true
    ndk {
        abiFilters 'arm64-v8a', 'x86_64'
    }
}

}
android -> build.gradle configs:
buildscript {
ext.kotlin_version = '1.7.10'
repositories {
google()
mavenCentral()
maven { url "https://mvn.mob.com/android" }
maven { url "https://developer.huawei.com/repo/" }
}
dependencies {
classpath "com.android.tools.build:gradle:7.3.0"
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
classpath "com.mob.sdk:MobSDK2:+"
classpath "com.huawei.agconnect:agcp:<IP_ADDRESS>0"
}
}
allprojects {
repositories {
google()
mavenCentral()
maven { url "https://mvn.mob.com/android" }
maven { url "https://developer.huawei.com/repo/" }
maven { url "https://download.flutter.io" }
}
gradle.projectsEvaluated {
tasks.withType(JavaCompile) {
options.compilerArgs << "-Xlint:unchecked" << "-Xlint:deprecation"
}
}
}
after the flutter pub get is executed, I runs in the android device,  the following issue occurs.
FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':app:checkDebugAarMetadata'.

Could not resolve all files for configuration ':app:debugRuntimeClasspath'.
Could not resolve com.mob:MobLink:+.
Required by:
project :app
Failed to list versions for com.mob:MobLink.
Unable to load Maven meta-data from https://europe-west3-maven.pkg.dev/talsec-artifact-repository/freerasp/com/mob/MobLink/maven-metadata.xml.
Could not GET 'https://europe-west3-maven.pkg.dev/talsec-artifact-repository/freerasp/com/mob/MobLink/maven-metadata.xml'.
Connect to europe-west3-maven.pkg.dev:443 [europe-west3-maven.pkg.dev/<IP_ADDRESS>] failed: Connect timed out
Could not resolve com.aheaditec.talsec.security:TalsecSecurity-Community-Flutter:9.6.0.
Required by:
project :app > project :freerasp
Could not resolve com.aheaditec.talsec.security:TalsecSecurity-Community-Flutter:9.6.0.
Could not get resource 'https://download.flutter.io/com/aheaditec/talsec/security/TalsecSecurity-Community-Flutter/9.6.0/TalsecSecurity-Community-Flutter-9.6.0.pom'.
Could not GET 'https://download.flutter.io/com/aheaditec/talsec/security/TalsecSecurity-Community-Flutter/9.6.0/TalsecSecurity-Community-Flutter-9.6.0.pom'.
Certificate for <download.flutter.io> doesn't match any of the subject alternative names: [*.storage.googleapis.com, *.googleapis.com, commondatastorage.googleapis.com, *.commondatastorage.googleapis.com, storage.googleapis.com, storage-p2.googleapis.com, *.storage-p2.googleapis.com, storage.mtls.googleapis.com, *.appspot.com.storage.googleapis.com, *.content-storage.googleapis.com, *.content-storage-p2.googleapis.com, *.content-storage-upload.googleapis.com, *.content-storage-download.googleapis.com, *.storage-upload.googleapis.com, *.storage-download.googleapis.com]
Skipped due to earlier error

Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.

I checked some  similar troubles resolved by append the maven { url "https://nexus3-public.monetplus.cz/repository/ahead-talsec-free-rasp" } or  In android -> app -> build.gradle add these dependencies
dependencies {
implementation 'com.aheaditec.talsec.security:TalsecSecurity-Community-Flutter:'
}
I tried to alter it, but still can't resolved this trouble. I hope you can  give me some help or advice, thanks again.

Hello @Charles045
We deprecated Nexus deployment in freeRASP version 6.5.1, so the fix you implemented is invalid. Although I can't replicate the issue locally, the logs suggest that Gradle is attempting to locate the MobLink dependency in our Maven repository.
Could you please provide a list of all the dependencies you're using? Specifically, I need:

Flutter dependencies (from pubspec.yaml -> dependencies)
Android dependencies (from android/app/build.gradle -> dependencies section at the bottom)

Kind regards,
Jaroslav from Talsec Team

Thanks yardexx for your replied!
Here is my flutter project dependencies and android build.gradle configurations:
pubspec.yaml -> dependencies
`
environment:
sdk: '>=3.1.0 <4.0.0'
dependencies:
flutter:
sdk: flutter
cupertino_icons: ^1.0.6
flutter_localizations:
sdk: flutter
redux: ^5.0.0
flutter_redux: ^0.10.0
event_bus: ^2.0.0
fluro: ^2.0.5
dio: ^5.5.0
protobuf: ^3.1.0
connectivity_plus: ^6.0.4
encrypt: ^5.0.3
path_provider: ^2.1.3
sqflite: ^2.3.3
shared_preferences: ^2.2.3
flutter_secure_storage: ^9.2.2
geolocator: ^12.0.0
geocoding: ^3.0.0
pull_to_refresh: ^2.0.0
webview_flutter: ^4.8.0
flutter_staggered_grid_view: ^0.7.0
extended_nested_scroll_view: ^6.2.1
flutter_draggable_gridview: ^0.0.12
card_swiper: ^3.0.1
qr_flutter: ^4.1.0
easy_pdf_viewer: ^1.0.8
wechat_assets_picker: ^9.1.0
extended_image: ^8.2.1
flutter_svg_provider: ^1.0.7
signature: ^5.5.0
sentry_flutter: ^8.7.0
sentry_dio: ^8.7.0
freerasp: ^6.6.0
fluwx: ^4.5.5
tobias: ^4.0.0
mobcommonlib: ^1.1.5
mobpush_plugin: ^1.2.5
moblink: ^3.4.7
`
android/settings.gradle:
pluginManagement {
def flutterSdkPath = {
def properties = new Properties()
file("local.properties").withInputStream { properties.load(it) }
def flutterSdkPath = properties.getProperty("flutter.sdk")
assert flutterSdkPath != null, "flutter.sdk not set in local.properties"
return flutterSdkPath
}()
includeBuild("$flutterSdkPath/packages/flutter_tools/gradle")

repositories {
    google()
    mavenCentral()
    maven { url "https://mvn.mob.com/android" }
    maven { url "https://developer.huawei.com/repo/" }
    gradlePluginPortal()
}

}
plugins {
id "dev.flutter.flutter-plugin-loader" version "1.0.0"
id "com.android.application" version "7.3.0" apply false
id "org.jetbrains.kotlin.android" version "1.7.10" apply false
}
include ":app"
android/build.gradle
buildscript {
ext.kotlin_version = '1.7.10'
repositories {
google()
mavenCentral()
maven { url "https://mvn.mob.com/android" }
maven { url "https://developer.huawei.com/repo/" }
}
dependencies {
classpath "com.mob.sdk:MobSDK2:+"
classpath "com.huawei.agconnect:agcp:<IP_ADDRESS>0"
classpath "com.android.tools.build:gradle:7.3.0"
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
}
}
allprojects {
repositories {
google()
mavenCentral()
maven { url "https://mvn.mob.com/android" }
maven { url "https://developer.huawei.com/repo/" }
maven { url "https://download.flutter.io" }
}
gradle.projectsEvaluated {
tasks.withType(JavaCompile) {
options.compilerArgs << "-Xlint:unchecked" << "-Xlint:deprecation"
}
}
}
rootProject.buildDir = '../build'
subprojects {
project.buildDir = "${rootProject.buildDir}/${project.name}"
}
subprojects {
project.evaluationDependsOn(':app')
}
tasks.register("clean", Delete) {
delete rootProject.buildDir
}
android/app/build.gradle
plugins {
id "com.android.application"
id "kotlin-android"
id "dev.flutter.flutter-gradle-plugin"
}
def localProperties = new Properties()
def localPropertiesFile = rootProject.file('local.properties')
if (localPropertiesFile.exists()) {
localPropertiesFile.withReader('UTF-8') { reader ->
localProperties.load(reader)
}
}
def flutterVersionCode = localProperties.getProperty('flutter.versionCode')
if (flutterVersionCode == null) {
flutterVersionCode = '1'
}
def flutterVersionName = localProperties.getProperty('flutter.versionName')
if (flutterVersionName == null) {
flutterVersionName = '1.0.0'
}
apply plugin: 'com.mob.sdk'
apply plugin: 'com.huawei.agconnect'
def keystoreProperties = new Properties()
def keystorePropertiesFile = rootProject.file('key.properties')
if (keystorePropertiesFile.exists()) {
keystoreProperties.load(new FileInputStream(keystorePropertiesFile))
}
def dartEnvDefines = [
CHANNEL: '',
ENV    : '',
]
if (project.hasProperty('dart-defines')) {
dartEnvDefines = project.property('dart-defines')
.split(',')
.collectEntries { entry ->
def pair = new String(entry.decodeBase64(), 'UTF-8').split('=')
[(pair.first()): pair.last()]
}
}
android {
compileSdkVersion 34
buildToolsVersion '34.0.0'
lintOptions {
disable 'InvalidPackage'
}
compileOptions {
sourceCompatibility JavaVersion.VERSION_1_8
targetCompatibility JavaVersion.VERSION_1_8
}
defaultConfig {
    applicationId "com.xxxxxxxx.yyyy"
    minSdkVersion 23
    targetSdkVersion 34
    versionCode flutterVersionCode.toInteger()
    versionName flutterVersionName
    multiDexEnabled true
    ndk {
        abiFilters 'arm64-v8a', 'x86_64'
    }
}

signingConfigs {
    release {
        keyAlias keystoreProperties['keyAlias']
        keyPassword keystoreProperties['keyPassword']
        storeFile keystoreProperties['storeFile'] ? file(keystoreProperties['storeFile']) : null
        storePassword keystoreProperties['storePassword']
    }
}

buildTypes {
    release {
        signingConfig signingConfigs.release
        zipAlignEnabled true
        shrinkResources true
        minifyEnabled true
        proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
    }
    debug {
        signingConfig signingConfigs.release
        shrinkResources false
        minifyEnabled false
    }
}

applicationVariants.all { variant ->
    variant.outputs.all { output ->
        def channelName = dartEnvDefines.CHANNEL
        if (channelName == null) {
            channelName = 'debug'
        }
        def newApkName = "ggb_${channelName.toLowerCase()}_${dartEnvDefines.ENV}_${flutterVersionName}_${flutterVersionCode.toInteger()}.apk"
        println('apk:' + newApkName)
        outputFileName = new File(newApkName)
    }
}

}
flutter {
source '../..'
}
MobSDK {
appKey "xxxxxxxxxxxx"
appSecret "xxxxxxxxxxxxxxxxxxxxxx"
MobPush {
devInfo {
HUAWEI {
appId "ddxxxxxx"
}
...
}
}
MobLink {
uriScheme "xxxxxxxxxx"
appLinkHost "xxxx.cn"
}
}
build failed information:
FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':app:checkDebugAarMetadata'.

Could not resolve all files for configuration ':app:debugRuntimeClasspath'.
Could not resolve com.mob:MobLink:+.
Required by:
project :app
Failed to list versions for com.mob:MobLink.
Unable to load Maven meta-data from https://europe-west3-maven.pkg.dev/talsec-artifact-repository/freerasp/com/mob/MobLink/maven-metadata.xml.
Could not GET 'https://europe-west3-maven.pkg.dev/talsec-artifact-repository/freerasp/com/mob/MobLink/maven-metadata.xml'.
Connect to europe-west3-maven.pkg.dev:443 [europe-west3-maven.pkg.dev/<IP_ADDRESS>, europe-west3-maven.pkg.dev/2404:6800:4008:c1b:0:0:0:52] failed: Connect timed out
Could not resolve com.aheaditec.talsec.security:TalsecSecurity-Community-Flutter:9.6.0.
Required by:
project :app > project :freerasp
Could not resolve com.aheaditec.talsec.security:TalsecSecurity-Community-Flutter:9.6.0.
Could not get resource 'https://download.flutter.io/com/aheaditec/talsec/security/TalsecSecurity-Community-Flutter/9.6.0/TalsecSecurity-Community-Flutter-9.6.0.pom'.
Could not GET 'https://download.flutter.io/com/aheaditec/talsec/security/TalsecSecurity-Community-Flutter/9.6.0/TalsecSecurity-Community-Flutter-9.6.0.pom'.
Certificate for <download.flutter.io> doesn't match any of the subject alternative names: [*.storage.googleapis.com, *.googleapis.com, commondatastorage.googleapis.com, *.commondatastorage.googleapis.com, storage.googleapis.com, storage-p2.googleapis.com, *.storage-p2.googleapis.com, storage.mtls.googleapis.com, *.appspot.com.storage.googleapis.com, *.content-storage.googleapis.com, *.content-storage-p2.googleapis.com, *.content-storage-upload.googleapis.com, *.content-storage-download.googleapis.com, *.storage-upload.googleapis.com, *.storage-download.googleapis.com]
Skipped due to earlier error

Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.

Get more help at https://help.gradle.org

BUILD FAILED in 8m 38s
[!] Gradle threw an error while downloading artifacts from the network.

I tried to run the project of Free-RASP-Flutter example
FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':app:checkDebugAarMetadata'.

Could not resolve all files for configuration ':app:debugRuntimeClasspath'.
Could not resolve com.aheaditec.talsec.security:TalsecSecurity-Community-Flutter:9.6.0.
Required by:
project :app > project :freerasp
Could not resolve com.aheaditec.talsec.security:TalsecSecurity-Community-Flutter:9.6.0.
Could not get resource 'https://europe-west3-maven.pkg.dev/talsec-artifact-repository/freerasp/com/aheaditec/talsec/security/TalsecSecurity-Community-Flutter/9.6.0/TalsecSecurity-Community-Flutter-9.6.0.pom'.
Could not GET 'https://europe-west3-maven.pkg.dev/talsec-artifact-repository/freerasp/com/aheaditec/talsec/security/TalsecSecurity-Community-Flutter/9.6.0/TalsecSecurity-Community-Flutter-9.6.0.pom'.
Connect to europe-west3-maven.pkg.dev:443 [europe-west3-maven.pkg.dev/<IP_ADDRESS>] failed: Connect timed out

Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.

Get more help at https://help.gradle.org

BUILD FAILED in 1m 39s
[!] Gradle threw an error while downloading artifacts from the network.

I tried to run the project of Free-RASP-Flutter example
FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':app:checkDebugAarMetadata'.

Could not resolve all files for configuration ':app:debugRuntimeClasspath'.
Could not resolve com.aheaditec.talsec.security:TalsecSecurity-Community-Flutter:9.6.0.
Required by:
project :app > project :freerasp
Could not resolve com.aheaditec.talsec.security:TalsecSecurity-Community-Flutter:9.6.0.
Could not get resource 'https://europe-west3-maven.pkg.dev/talsec-artifact-repository/freerasp/com/aheaditec/talsec/security/TalsecSecurity-Community-Flutter/9.6.0/TalsecSecurity-Community-Flutter-9.6.0.pom'.
Could not GET 'https://europe-west3-maven.pkg.dev/talsec-artifact-repository/freerasp/com/aheaditec/talsec/security/TalsecSecurity-Community-Flutter/9.6.0/TalsecSecurity-Community-Flutter-9.6.0.pom'.
Connect to europe-west3-maven.pkg.dev:443 [europe-west3-maven.pkg.dev/<IP_ADDRESS>] failed: Connect timed out

Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.

Get more help at https://help.gradle.org

BUILD FAILED in 1m 39s
[!] Gradle threw an error while downloading artifacts from the network.

FAILURE: Build failed with an exception.

What went wrong:

Execution failed for task ':app:checkDebugAarMetadata'.

Could not resolve all dependencies for configuration ':app:debugRuntimeClasspath'.

The project declares repositories, effectively ignoring the repositories you have declared in the settings.

You can figure out how project repositories are declared by configuring your build to fail on project repositories.

See https://docs.gradle.org/7.5/userguide/declaring_repositories.html#sub:fail_build_on_project_repositories for details.

Could not find com.aheaditec.talsec.security:TalsecSecurity-Community-Flutter:9.6.0.
Searched in the following locations:

 - https://plugins.gradle.org/m2/com/aheaditec/talsec/security/TalsecSecurity-Community-Flutter/9.6.0/TalsecSecurity-Community-Flutter-9.6.0.pom
 - https://dl.google.com/dl/android/maven2/com/aheaditec/talsec/security/TalsecSecurity-Community-Flutter/9.6.0/TalsecSecurity-Community-Flutter-9.6.0.pom
 - https://repo.maven.apache.org/maven2/com/aheaditec/talsec/security/TalsecSecurity-Community-Flutter/9.6.0/TalsecSecurity-Community-Flutter-9.6.0.pom
 - https://storage.flutter-io.cn/download.flutter.io/com/aheaditec/talsec/security/TalsecSecurity-Community-Flutter/9.6.0/TalsecSecurity-Community-Flutter-9.6.0.pom
 - https://mvn.mob.com/android/com/aheaditec/talsec/security/TalsecSecurity-Community-Flutter/9.6.0/TalsecSecurity-Community-Flutter-9.6.0.pom
 - https://developer.huawei.com/repo/com/aheaditec/talsec/security/TalsecSecurity-Community-Flutter/9.6.0/TalsecSecurity-Community-Flutter-9.6.0.pom

Required by:
project :app > project :freerasp

Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.

my flutter project config file info:

project/android/settings.gradle

pluginManagement {
def flutterSdkPath = {

    def properties = new Properties()
    file("local.properties").withInputStream { properties.load(it) }
    def flutterSdkPath = properties.getProperty("flutter.sdk")
    assert flutterSdkPath != null, "flutter.sdk not set in local.properties"
    return flutterSdkPath

}()

includeBuild("$flutterSdkPath/packages/flutter_tools/gradle")

repositories {

    gradlePluginPortal()
    google()
    mavenCentral()
    maven { url "https://storage.flutter-io.cn/download.flutter.io" }
    maven { url "https://mvn.mob.com/android" }
    maven { url "https://developer.huawei.com/repo/" }

}

}
plugins {
id "dev.flutter.flutter-plugin-loader" version "1.0.0"
id "com.android.application" version "7.3.0" apply false
id "org.jetbrains.kotlin.android" version "1.7.10" apply false

}
dependencyResolutionManagement {
repositoriesMode.set(RepositoriesMode.PREFER_SETTINGS)

repositories {

    gradlePluginPortal()
    google()
    mavenCentral()
    maven { url "https://storage.flutter-io.cn/download.flutter.io" }
    maven { url "https://mvn.mob.com/android" }
    maven { url "https://developer.huawei.com/repo/" }
}

}
include ":app"

project/android/build.gradle

buildscript {
ext.kotlin_version = '1.7.10'
dependencies {
classpath "com.android.tools.build:gradle:7.3.0"
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
classpath "com.mob.sdk:MobSDK2:+"
classpath "com.huawei.agconnect:agcp:<IP_ADDRESS>0"
}
}
plugins {
id "com.android.application" version "7.3.0" apply false
id "org.jetbrains.kotlin.android" version "1.7.10" apply false
}
allprojects {
gradle.projectsEvaluated {
tasks.withType(JavaCompile) {
options.compilerArgs << "-Xlint:unchecked" << "-Xlint:deprecation"
}
}
}
rootProject.buildDir = '../build'
subprojects {
project.buildDir = "${rootProject.buildDir}/${project.name}"
}
subprojects {
project.evaluationDependsOn(':app')
}
tasks.register("clean", Delete) {
delete rootProject.buildDir
}

Hello.
I've tried to replicate the issue but with no luck. From what I can see in your Gradle dependencies, a some sort of geolocation blocking or VPN could be source of an issue.
Kind regards,
Jaroslav from Talsec


Cut and paste error in the article

Cut and paste error in "Maximum Background Download Bandwidth".
The text does not actually explain what Background and Foreground bandwidth are.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: df445cda-3b0a-2ad7-e3cc-23e159013530
Version Independent ID: a6bb5f97-d350-f7b7-77f7-b0093322289d
Content: Delivery Optimization reference
Content Source: windows/deployment/update/waas-delivery-optimization-reference.md
Product: w10
Technology: windows
GitHub Login: @jaimeo
Microsoft Alias: jaimeo

@officedocsbot assign @e0i

Hello, @Air-Git
Thank you for taking the time to provide feedback.
The issue is under investigation and you'll be notified on the go as the content gets updated.
Have a nice day.

More or less obvious that there is a typo in this sentence:

Maximum Foreground Download Bandwidth specifies the maximum background download bandwidth that Delivery Optimization uses across all concurrent download activities as a percentage of available download bandwidth.

(unexpected word highlighted in bold text)

@Air-Git : Would it help if you look at other pages, like https://www.windowscentral.com/how-limit-foreground-downloads-bandwidth-windows-10-april-2018-update ?
I can quote the paragraph I think is the most relevant:

What's the difference between the two? Foreground downloads are those you start manually. For instance, when you click the Check for updates button on Windows Update, or the Get button to download an app from the Microsoft Store. On the other hand, background downloads are those downloads that are triggered automatically, such as when updates for Windows 10 download without user interaction.

Other than that, I personally think that the difference between Background and Foreground bandwidth is a technical term which is assumed to be known by IT Professionals.

I have never heard of foreground bandwidth. I have heard of foreground downloads, and download bandwidth, but never foreground download bandwidth, or foreground bandwidth.

From: Trond B. Krokli<EMAIL_ADDRESS>Sent: 01 May 2019 21:55
To: MicrosoftDocs/windows-itpro-docs
Cc: Anthony Yates; Mention
Subject: Re: [MicrosoftDocs/windows-itpro-docs] Cut and paste error in the article (#3416)
@Air-Githttps://github.com/Air-Git : Would it help if you look at other pages, like https://www.windowscentral.com/how-limit-foreground-downloads-bandwidth-windows-10-april-2018-update ?
I can quote the paragraph I think is the most relevant:
What's the difference between the two? Foreground downloads are those you start manually. For instance, when you click the Check for updates button on Windows Update, or the Get button to download an app from the Microsoft Store. On the other hand, background downloads are those downloads that are triggered automatically, such as when updates for Windows 10 download without user interaction.
Other than that, I personally think that the difference between Background and Foreground bandwidth is a technical term which is assumed to be known by IT Professionals.

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/MicrosoftDocs/windows-itpro-docs/issues/3416#issuecomment-488421611, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AHSMVE7KU3G6PG6E3YMQORLPTH7SLANCNFSM4HJTF4AA.

This seems like a taxonomical issue:
Maximum {Fore⊻Back}ground Download Bandwidth
🆚
{Fore⊻Back}ground Download Bandwidth Maximum
Mathematically & strictly speaking, neither seems correct, instead, it should read SOMETHING LIKE this:
{Fore⊻Back}ground Download Bandwidth Limiting Filter Interval Maximum
However, that seems like both a mouthful and STILL very inaccurate, so, I had an idea:
@lemmy: please, might you consider providing some better phrasing? I figured, while this doesn't sound like a job for TLA+ as such, one of MS' in-house TLA+ experts could almost certainly come up with a phrase which describes what this ACTUALLY does, instead of just providing some seemingly clearly, but actually very unclear, phrasing.

PR #3511 created for

Cut and paste error in "Maximum Background Download Bandwidth".

I agree that we should do a better job of explaining what exactly "foreground" and "background" are supposed to mean (we'll be updating all of the Delivery Optimization content soon for a forthcoming release, so there's a good opportunity to improve the explanation)--but as for the names themselves, there's nothing documentation can do to change that, as that's what the values are named in Group Policy (and MDM). Changing those names would be a code change, but we can certainly pass that request along.

@no-identd Unfortunately, without being familiar with the semantics of waas it is impossible to come up with a precise specification.  That said, Azure Cosmos DB has indeed successfully demonstrated how to use the TLA+ specification language to provide user documentation.

"Optimization" is a noun referring to the process of selecting the most effective or desirable way to do something. But "Delivery Optimization" appears to be the name of a process that controls or throttles downloads from Microsoft Updates. Perhaps "Delivery Control" would be a better term.
"Bandwidth" refers to the nominal or practical capacity of a link. "Rate" or "Speed" refers to the actual rate or speed of an individual transfer of data over the link.
A rate may be expressed in absolute terms (e.g. Kb/s) or in percentage terms (e.g. 10% of the bandwidth).
In the policy itself, the "maximum foreground download" can be set as an absolute rate, or as a percentage.
"Foreground" seems to mean interactive; and "Background" seems to mean non-interactive or scheduled. The meaning in this case is not absolutely clear because, if I run updates from the GUI, then I would normally leave them to complete in the background. And they would indeed complete in the background. If I restarted the computer, the update task would continue.
I don't know what "across all concurrent download activities" means. I don't know if it means the sum of all Microsoft update activities; or the sum of all downloads (e.g. Netflix). I think it must be a redundant term, since "Delivery Optimization" already refers to a process that controls the collection of Microsoft updates.
So how about:
"Maximum Foreground Rate"
"Starting in Windows 10, version 1803, specifies the maximum rate of foreground (interactive) updates, as a percentage of available bandwidth. The default value of 0 means that foreground updates adjust dynamically to use all available bandwidth. This setting does not apply to downloads from LAN peers."
I appreciate that the document is actually a copy of the GPO ADML. But that is not a Reference. In a reference document I am expecting to find the most accurate and detailed description of the operation. In a GPO ADML I am expecting to find a quick explanation of how to use the setting. I would expect to go to the Reference documentation if the quick explanation were unclear.

Interesting, thank you for sharing the ideas.

@Air-Git, you are welcome to file your own PR with proposed changes. We will run that by the appropriate PM for their review of accuracy and will publish with their approval.

PR #3842  The article has already the corrections for this issue.

@Air-Git
Upon your feedback, The content has been updated with additional changes. Thanks.

@officedocsbot close

How strange. The cut and paste error remains exactly as before.

@Air-Git
The changes made to the repository are pushed to live by a Continuous Integration system and it might sometimes take a few days for the system to be triggered. Thank you for following-up.

@Air-Git : Maybe you need to refresh or clean your browser cache to see the updated page correctly. From what I can see, it appears to be corrected now. If you still see something you are certain is still incorrect, please copy & paste that part (or link the section, if possible) in a new comment here in this issue page.


CI part & separating R and Python dependencies

CI part configuration:

added workflows tests-with-R.yaml and tests-without-R.yaml to run tests with or without R packages, while ignoring R-dependent tests if R packages are not installed (tests-without-R.yaml)
I added a save-packages-R.yaml workflow to save R packages in the cache, so you don't have to install them every time you run a test (tests-with-R.yaml will take less time to run).

Separation of R and Python dependencies, by displaying warnings, in the case where a function that depends on an R package is run and R or the package are not installed. The warning asks you to install R if this is not the case and/or install the relevant R package if this is not the case.

Thx !
Can you address the failing tests ?

@zbakhm is this PR ready to be merged ?

@zbakhm is this PR ready to be merged ?

Yes!


New tasks action menu not visible

Expected behaviour
See Action menu for selected task and be able to select one of the proposed actions.
Actual behaviour
Action menu is drawn outside page and only "Remove" is visible.
Action menu is shown okay in Chromium 48.0.2564.116.
Steps to reproduce

Create a task
Click on the task
Click on "Actions" menu.

Configuration

Kanboard version: 1.0.25
Database type and version: mysql/5.5.47
PHP version: 5.16.17
OS: Debian 8
Browser: Iceweasel/Firefox 44.0.2

Can you try again with the version 1.0.26? this issue should be fixed.

I've just tested with 1.0.26 and it fixed the issue, indeed. Thanks!


PyInstaller

Adds PyInstaller to the dependencies and a poe task for building the binary. This allows us to distribute a binary without needing users to install python and pipx first, which is particularly useful for devs not using Python in their development stack but still want to use algokit.
I still need to do some more testing, but so far things seem to be working on Mac (ARM) and Ubuntu.

I'd like to bring this into consideration again. Main thing missing that should be here before merge is integrating into workflows.
Also, I did some benchmarking for the PyInstaller folder build vs Nuitka vs pipx install. They all seem to be pretty close with eachother so we don't loose much going from pipx > pyinstaller. PyInstaller is marginally faster than Nuitka, but since PyInstaller is more popular probably makes sense to stick with it.
Note that this is for the directory distribution. It can be compiled to a single binary, but performance is much worse. This is totally fine for brew/choco since they can just put the files in PATH and call it a day. I think eventually we'd want an msi installer for windows for those that don't have choco.
❯ hyperfine --warmup 3 -i "algokit doctor" "./pyinstaller-dist/algokit/algokit doctor" "nuitka-dist/algokit.bin doctor"
Benchmark 1: algokit doctor
  Time (mean ± σ):      1.220 s ±  0.060 s    [User: 0.645 s, System: 0.211 s]
  Range (min … max):    1.160 s …  1.353 s    10 runs

Benchmark 2: ./pyinstaller-dist/algokit/algokit doctor
  Time (mean ± σ):      1.251 s ±  0.032 s    [User: 0.696 s, System: 0.198 s]
  Range (min … max):    1.211 s …  1.299 s    10 runs

Benchmark 3: nuitka-dist/algokit.bin doctor
  Time (mean ± σ):      1.268 s ±  0.033 s    [User: 0.726 s, System: 0.186 s]
  Range (min … max):    1.232 s …  1.339 s    10 runs

Summary
  algokit doctor ran
    1.02 ± 0.06 times faster than ./pyinstaller-dist/algokit/algokit doctor
    1.04 ± 0.06 times faster than nuitka-dist/algokit.bin doctor

@joe-p thanks again for all the work on this PR so far! Just fyi, i moved your latest and merged latest from main into https://github.com/algorandfoundation/algokit-cli/pull/382. @negar-abbasi will be spiking an ADR to asses pyinstaller and few other alternatives - there are still some limitations with pyinstaller+gh actions when it comes to apple silicon builds and linux distros with glib dependency - will be assessing how to address those as part of ADR. As for this PR we might revisit and close it once we get to implementation (cc @neilcampbell )


Definition to the reads which are not fully converted by bisulfite treatment?

Hi Dr. Weilong,
Where I can find the definition to the reads which are not fully converted by bisulfite treatment?
I notice it before, however, I can not find it again when I need it.
Thanks.
Shicheng

Dear Shicheng,
The parameter "--XS=XS_FILTER" of bs_seeker2-align.py is to mark which reads are not fully converted by bisulfite treatment. (in BAM file, reads with tag XS:i:1)
The parameter "-x, --rm-SX" of bs_seeker2-call_methylation.py is to remove the reads which are marked as not fully converted.
You can also read our paper to see how the filtering funciton works.
http://bmcgenomics.biomedcentral.com/articles/10.1186/1471-2164-14-774
--
Best,
Weilong
At 2016-11-15 02:11:08, "Shicheng Guo"<EMAIL_ADDRESS>wrote:
Hi Dr. Weilong,
Where I can find the definition to the reads which are not fully converted by bisulfite treatment?
I notice it before, however, I can not find it again when I need it.
Thanks.
Shicheng
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.


Z2M needs an effective watchdog function

Description of the issue
Z2M needs an effective watchdog function. When the connection between the Zigbee gateway and Z2M is lost for a period of time, Z2M will stop running, rather than continuously attempting to reconnect with the Zigbee gateway until successful.
Addon version
V1.36
Platform
all version
Logs of the issue (if applicable)
No response

Freezing occurs every day and the service has to be restarted. Please resolve this issue.


Sorting issue.

Natural sorting for numeric values works strange:

I think values should be in order -6 -5 -4 -1 0 1 2 3 4

Yeah, this is a problem with the natural sort function, as the dash is read as a dash and not a minus.

OK, PR #652 should address this. I'll likely merge it tomorrow (and then look at your PR :)

Should be resolved in v2.9.0


Test database extension: some unique samples

To help with testing of #301, modify sample plates in test db to have one unique sample per plate

Discussed offline with @wasade and we believe that the new samples are being added 1032/1050 but I'm not 100% sure ... merging as these changes are so we can move #328 forward and I can test in there ...

we believe that the new samples are being added 1032/1050

Sorry, I don't understand this terminology; what does "1032/1050" mean?

lines 1032/1050 ... forgot to say lines ...

Ah, gotcha!  Yes, that's the place :)


fix fix unstable refresh catalog/db ut

Followup #38244
fix some unstable fe ut after set use_meta_cache as true

run buildall


feat: add directives.js to fix #1736

Fixes #1736.
Changes

Add directives.js to re-export all directives from lit-html/directives
Add ./directives.js in export map

I don't think we want to do this. The total collection of directives is much more heavyweight than decorators and there are some very rarely used directives that we shouldn't encourage developers to load by default.


Performance testing & benchmark suite

Cloudant has done some great things behind closed doors (looking at @kocolosk @garrensmith @tonysun83 ) but it'd be nice to have a "performance team" we can look to, to help with improving CouchDB performance, and a standard open source performance suite we can run on a regular basis.
@janl:

We need a team to comprehensive look at CouchDB performance. There is a lot of low-hanging fruit like Robert Kowalski showed a while back, we should get back into this. I’m mostly inspired by SQLite who’ve done a release a while back that only focussed on 1-2% performance improvements, but got like 20-30 of those and made the thing a lot faster across the board. I can’t remember where I read about this, but I’ll update this once I find the link.

@nolanlawson:

Here's that SQLite article you mentioned.
We have some perf tests in PouchDB that can absolutely be pointed at CouchDB (in fact, they work on Couch today). Due to being tuned for Pouch rather than Couch, though, many of them will probably be network-bound rather than CPU-bound.

cc @chewbranca

We have been looking at performance improvements lately and have released a bunch in 3.3:  https://docs.couchdb.org/en/stable/whatsnew/3.3.html#highlights
Some details in https://github.com/apache/couchdb/issues/4308:

https://github.com/apache/couchdb/pull/4238
https://github.com/apache/couchdb/pull/4201
https://github.com/apache/couchdb/pull/4193
https://github.com/apache/couchdb/pull/4185
https://github.com/apache/couchdb/pull/4179
https://github.com/apache/couchdb/pull/4163
https://github.com/apache/couchdb/pull/4144
https://github.com/apache/couchdb/pull/4059
https://github.com/apache/couchdb/pull/4056
https://github.com/apache/couchdb/pull/4052

There is even a simple built-in diagnostic fabric_bench module: https://github.com/apache/couchdb/pull/4601
However, in general, a dedicated Apache CouchDB perf team hasn't happened in 5 years. It's mostly noticing regressions, bottlenecks, and lock contentions and fixing accordingly by contributors.
In light of that, let's close the issue. If we want to revive it, and there is suddenly some interest we can re-open it.


Docker 1.8.1 push issue to Registry 2.1.1 with nfs filesystem

The following issue is occurring while pushing to a 2.1.1 registry with the 1.8.1 client.  Pushing works fine from a 1.7 client.
<IP_ADDRESS> - - [24/Aug/2015:21:33:22 +0000] "PATCH /v2/here_cme_techops/onboarding/blobs/uploads/f49c60c6-fa4f-42d0-8404-877e0305a33a?_state=TPtR3EHnTTJvS8W1Y4d9P909abHfq-mzi6OLcmHi5p17Ik5hbWUiOiJoZXJlX2NtZV90ZWNob3BzL29uYm9hcmRpbmciLCJVVUlEIjoiZjQ5YzYwYzYtZmE0Zi00MmQwLTg0MDQtODc3ZTAzMDVhMzNhIiwiT2Zmc2V0IjowLCJTdGFydGVkQXQiOiIyMDE1LTA4LTI0VDIxOjMzOjIyLjg5MjkyOTE0MVoifQ%3D%3D HTTP/1.1" 202 0 "" "docker/1.8.1 go/go1.4.2 git-commit/d12ea79 kernel/3.10.0-229.el7.x86_64 os/linux arch/amd64"
time="2015-08-24T21:33:23Z" level=warning msg="the ResponseWriter does not implement CloseNotifier" http.request.host=docker.cme.in.here.com http.request.id=9228f638-cbab-430c-9e07-dc6cf77260f7 http.request.method=PUT http.request.remoteaddr="<IP_ADDRESS>:44234" http.request.uri="/v2/here_cme_techops/onboarding/blobs/uploads/f49c60c6-fa4f-42d0-8404-877e0305a33a?_state=3_czVRweJjfpzyAbaEnD7p-wR4qKjkqHltBRYAs1qxF7Ik5hbWUiOiJoZXJlX2NtZV90ZWNob3BzL29uYm9hcmRpbmciLCJVVUlEIjoiZjQ5YzYwYzYtZmE0Zi00MmQwLTg0MDQtODc3ZTAzMDVhMzNhIiwiT2Zmc2V0IjozMiwiU3RhcnRlZEF0IjoiMjAxNS0wOC0yNFQyMTozMzoyMloifQ%3D%3D&digest=sha256%3Aa3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.8.1 go/go1.4.2 git-commit/d12ea79 kernel/3.10.0-229.el7.x86_64 os/linux arch/amd64" instance.id=506c18e6-6186-4adc-af49-288afbe7ce23 vars.name="here_cme_techops/onboarding" vars.uuid=f49c60c6-fa4f-42d0-8404-877e0305a33a version=v2.1.1
time="2015-08-24T21:33:23Z" level=error msg="unable to delete layer upload resources \"/docker/registry/v2/repositories/here_cme_techops/onboarding/_uploads/f49c60c6-fa4f-42d0-8404-877e0305a33a\": remove /tmp/registry/docker/registry/v2/repositories/here_cme_techops/onboarding/_uploads/f49c60c6-fa4f-42d0-8404-877e0305a33a: directory not empty" http.request.host=docker.cme.in.here.com http.request.id=9228f638-cbab-430c-9e07-dc6cf77260f7 http.request.method=PUT http.request.remoteaddr="<IP_ADDRESS>:44234" http.request.uri="/v2/here_cme_techops/onboarding/blobs/uploads/f49c60c6-fa4f-42d0-8404-877e0305a33a?_state=3_czVRweJjfpzyAbaEnD7p-wR4qKjkqHltBRYAs1qxF7Ik5hbWUiOiJoZXJlX2NtZV90ZWNob3BzL29uYm9hcmRpbmciLCJVVUlEIjoiZjQ5YzYwYzYtZmE0Zi00MmQwLTg0MDQtODc3ZTAzMDVhMzNhIiwiT2Zmc2V0IjozMiwiU3RhcnRlZEF0IjoiMjAxNS0wOC0yNFQyMTozMzoyMloifQ%3D%3D&digest=sha256%3Aa3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.8.1 go/go1.4.2 git-commit/d12ea79 kernel/3.10.0-229.el7.x86_64 os/linux arch/amd64" instance.id=506c18e6-6186-4adc-af49-288afbe7ce23 vars.name="here_cme_techops/onboarding" vars.uuid=f49c60c6-fa4f-42d0-8404-877e0305a33a version=v2.1.1
time="2015-08-24T21:33:23Z" level=error msg="unknown error completing upload: &os.PathError{Op:\"remove\", Path:\"/tmp/registry/docker/registry/v2/repositories/here_cme_techops/onboarding/_uploads/f49c60c6-fa4f-42d0-8404-877e0305a33a\", Err:0x27}" http.request.host=docker.cme.in.here.com http.request.id=9228f638-cbab-430c-9e07-dc6cf77260f7 http.request.method=PUT http.request.remoteaddr="<IP_ADDRESS>:44234" http.request.uri="/v2/here_cme_techops/onboarding/blobs/uploads/f49c60c6-fa4f-42d0-8404-877e0305a33a?_state=3_czVRweJjfpzyAbaEnD7p-wR4qKjkqHltBRYAs1qxF7Ik5hbWUiOiJoZXJlX2NtZV90ZWNob3BzL29uYm9hcmRpbmciLCJVVUlEIjoiZjQ5YzYwYzYtZmE0Zi00MmQwLTg0MDQtODc3ZTAzMDVhMzNhIiwiT2Zmc2V0IjozMiwiU3RhcnRlZEF0IjoiMjAxNS0wOC0yNFQyMTozMzoyMloifQ%3D%3D&digest=sha256%3Aa3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.8.1 go/go1.4.2 git-commit/d12ea79 kernel/3.10.0-229.el7.x86_64 os/linux arch/amd64" instance.id=506c18e6-6186-4adc-af49-288afbe7ce23 vars.name="here_cme_techops/onboarding" vars.uuid=f49c60c6-fa4f-42d0-8404-877e0305a33a version=v2.1.1
time="2015-08-24T21:33:23Z" level=error msg="unable to delete layer upload resources \"/docker/registry/v2/repositories/here_cme_techops/onboarding/_uploads/f49c60c6-fa4f-42d0-8404-877e0305a33a\": remove /tmp/registry/docker/registry/v2/repositories/here_cme_techops/onboarding/_uploads/f49c60c6-fa4f-42d0-8404-877e0305a33a/.nfs00000000a18136c600000014: device or resource busy" http.request.host=docker.cme.in.here.com http.request.id=9228f638-cbab-430c-9e07-dc6cf77260f7 http.request.method=PUT http.request.remoteaddr="<IP_ADDRESS>:44234" http.request.uri="/v2/here_cme_techops/onboarding/blobs/uploads/f49c60c6-fa4f-42d0-8404-877e0305a33a?_state=3_czVRweJjfpzyAbaEnD7p-wR4qKjkqHltBRYAs1qxF7Ik5hbWUiOiJoZXJlX2NtZV90ZWNob3BzL29uYm9hcmRpbmciLCJVVUlEIjoiZjQ5YzYwYzYtZmE0Zi00MmQwLTg0MDQtODc3ZTAzMDVhMzNhIiwiT2Zmc2V0IjozMiwiU3RhcnRlZEF0IjoiMjAxNS0wOC0yNFQyMTozMzoyMloifQ%3D%3D&digest=sha256%3Aa3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.8.1 go/go1.4.2 git-commit/d12ea79 kernel/3.10.0-229.el7.x86_64 os/linux arch/amd64" instance.id=506c18e6-6186-4adc-af49-288afbe7ce23 vars.name="here_cme_techops/onboarding" vars.uuid=f49c60c6-fa4f-42d0-8404-877e0305a33a version=v2.1.1

The last line shows an nfs file handle as the reason the directory couldn't be cleaned up.
Here are the last couple of lines from the server running the push:
Aug 25 19:43:12 pchivcpbat291 docker[17863]: time="2015-08-25T19:43:12.762420755Z" level=debug msg="rendered layer for d1ee9017809a0d7959b96982fecd01a7c33edaa38351eeb21ba8b469a218e1ff of [32] size"
Aug 25 19:43:12 pchivcpbat291 docker: time="2015-08-25T19:43:12.762420755Z" level=debug msg="rendered layer for d1ee9017809a0d7959b96982fecd01a7c33edaa38351eeb21ba8b469a218e1ff of [32] size"
Aug 25 19:43:12 pchivcpbat291 docker[17863]: time="2015-08-25T19:43:12.778998985Z" level=debug msg="[devmapper] Unmount done"
Aug 25 19:43:12 pchivcpbat291 docker: time="2015-08-25T19:43:12.778998985Z" level=debug msg="[devmapper] Unmount done"
Aug 25 19:43:12 pchivcpbat291 docker: time="2015-08-25T19:43:12.779022595Z" level=debug msg="[devmapper] deactivateDevice(d1ee9017809a0d7959b96982fecd01a7c33edaa38351eeb21ba8b469a218e1ff)"
Aug 25 19:43:12 pchivcpbat291 docker[17863]: time="2015-08-25T19:43:12.779022595Z" level=debug msg="[devmapper] deactivateDevice(d1ee9017809a0d7959b96982fecd01a7c33edaa38351eeb21ba8b469a218e1ff)"
Aug 25 19:43:12 pchivcpbat291 docker[17863]: time="2015-08-25T19:43:12.779078093Z" level=debug msg="[devmapper] RemoveDeviceDeferred START(docker-253:0-4205-d1ee9017809a0d7959b96982fecd01a7c33edaa38351eeb21ba8b469a218e1ff)"
Aug 25 19:43:12 pchivcpbat291 docker: time="2015-08-25T19:43:12.779078093Z" level=debug msg="[devmapper] RemoveDeviceDeferred START(docker-253:0-4205-d1ee9017809a0d7959b96982fecd01a7c33edaa38351eeb21ba8b469a218e1ff)"
Aug 25 19:43:12 pchivcpbat291 docker: time="2015-08-25T19:43:12.796983128Z" level=debug msg="[devmapper] RemoveDeviceDeferred END(docker-253:0-4205-d1ee9017809a0d7959b96982fecd01a7c33edaa38351eeb21ba8b469a218e1ff)"
Aug 25 19:43:12 pchivcpbat291 docker: time="2015-08-25T19:43:12.797013465Z" level=debug msg="[devmapper] deactivateDevice END(d1ee9017809a0d7959b96982fecd01a7c33edaa38351eeb21ba8b469a218e1ff)"
Aug 25 19:43:12 pchivcpbat291 docker[17863]: time="2015-08-25T19:43:12.796983128Z" level=debug msg="[devmapper] RemoveDeviceDeferred END(docker-253:0-4205-d1ee9017809a0d7959b96982fecd01a7c33edaa38351eeb21ba8b469a218e1ff)"
Aug 25 19:43:12 pchivcpbat291 docker[17863]: time="2015-08-25T19:43:12.797013465Z" level=debug msg="[devmapper] deactivateDevice END(d1ee9017809a0d7959b96982fecd01a7c33edaa38351eeb21ba8b469a218e1ff)"
Aug 25 19:43:12 pchivcpbat291 docker[17863]: time="2015-08-25T19:43:12.797026445Z" level=debug msg="[devmapper] UnmountDevice(hash=d1ee9017809a0d7959b96982fecd01a7c33edaa38351eeb21ba8b469a218e1ff) END"
Aug 25 19:43:12 pchivcpbat291 docker: time="2015-08-25T19:43:12.797026445Z" level=debug msg="[devmapper] UnmountDevice(hash=d1ee9017809a0d7959b96982fecd01a7c33edaa38351eeb21ba8b469a218e1ff) END"
Aug 25 19:43:12 pchivcpbat291 docker[17863]: time="2015-08-25T19:43:12.999716785Z" level=debug msg="Not continuing with error: Received unexpected HTTP status: 500 Internal Server Error"
Aug 25 19:43:12 pchivcpbat291 docker: time="2015-08-25T19:43:12.999716785Z" level=debug msg="Not continuing with error: Received unexpected HTTP status: 500 Internal Server Error"

Please let me know what other info is needed.  As I stated above, this works fine in 1.7.
Docker info from server running the push:
uname -a:
Linux pchivcpbat291.lnx.in.here.com 3.10.0-229.el7.x86_64 #1 SMP Fri Mar 6 11:36:42 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

docker version:
Client:
Version:      1.8.1
API version:  1.20
Go version:   go1.4.2
Git commit:   d12ea79
Built:        Thu Aug 13 02:19:43 UTC 2015
OS/Arch:      linux/amd64

Server:
Version:      1.8.1
API version:  1.20
Go version:   go1.4.2
Git commit:   d12ea79
Built:        Thu Aug 13 02:19:43 UTC 2015
OS/Arch:      linux/amd64

docker info:
Containers: 2
Images: 97
Storage Driver: devicemapper
 Pool Name: docker-253:0-4205-pool
 Pool Blocksize: 65.54 kB
 Backing Filesystem: xfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 3.526 GB
 Data Space Total: 107.4 GB
 Data Space Available: 81.03 GB
 Metadata Space Used: 5.267 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.142 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: true
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.93-RHEL7 (2015-01-28)
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.10.0-229.el7.x86_64
Operating System: CentOS Linux 7 (Core)
CPUs: 4
Total Memory: 15.52 GiB
Name: pchivcpbat291.lnx.in.here.com
ID: 2LHS:WDGG:P2JN:SNYP:B3ZB:YW32:DUMO:2LNO:TFCV:CGYX:NJN4:63D6

Docker info from server running the registry:
uname -a:
Linux pchivcpapp02.lnx.in.here.com 3.10.0-229.el7.x86_64 #1 SMP Fri Mar 6 11:36:42 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

docker version:
Client:
 Version:      1.8.1
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   d12ea79
 Built:        Thu Aug 13 02:19:43 UTC 2015
 OS/Arch:      linux/amd64

Server:
 Version:      1.8.1
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   d12ea79
 Built:        Thu Aug 13 02:19:43 UTC 2015
 OS/Arch:      linux/amd64

docker info:
Containers: 72
Images: 474
Storage Driver: devicemapper
 Pool Name: docker-253:0-5572-pool
 Pool Blocksize: 65.54 kB
 Backing Filesystem: xfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 15.8 GB
 Data Space Total: 107.4 GB
 Data Space Available: 91.58 GB
 Metadata Space Used: 25 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.122 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: true
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.93-RHEL7 (2015-01-28)
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.10.0-229.el7.x86_64
Operating System: CentOS Linux 7 (Core)
CPUs: 24
Total Memory: 47 GiB
Name: pchivcpapp02.lnx.in.here.com
ID: USGU:JHWJ:SXZH:B6O6:IUTL:NR62:UL5F:VOWW:RFQN:ME5J:IH3S:XEPA

Environment Details:
These are physical boxes. You will notice Deferred Removal was turned on.  I was hoping it would fix the issue, but it didn't.
How Reproducible:
I haven't seen any other tickets on this, so maybe I am the only one using an NFS mount for the registry.  That would be the key here, make sure the registry is pointing at an NFS mounted filesystem.

Hi!
Please read this important information about creating issues.
If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.
This is an automated, informational response.
Thank you.
For more information about reporting issues, see https://github.com/docker/docker/blob/master/CONTRIBUTING.md#reporting-other-issues

BUG REPORT INFORMATION
Use the commands below to provide key information from your environment:
docker version:
docker info:
uname -a:
Provide additional environment details (AWS, VirtualBox, physical, etc.):
List the steps to reproduce the issue:
1.
2.
3.
Describe the results you received:
Describe the results you expected:
Provide additional info you think is important:
----------END REPORT ---------
#ENEEDMOREINFO

I have added more info to the original post.  Let me know if you need more.

I am getting the same issue. Recently cloned and built docker registry, docker 1.8.1 linux 3.19.0-25-generic #26~14.04.1-Ubuntu SMP Fri Jul 24 21:16:20 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
udo docker info
Containers: 7
Images: 104
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 118
Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.19.0-25-generic
Operating System: Ubuntu 14.04.3 LTS
CPUs: 24
Total Memory: 125.7 GiB
Name: hadoopmapr4
ID: F774:XB2C:IW4N:YOTR:LVT3:I5YQ:4CBB:N2XR:JAWQ:BGOC:VOS4:NJV3
WARNING: No swap limit support
I am trying to push images to a directory in nfs, but getting device or resource busy when in reality I can write just fine to it.

same here, trying to push to registry:2.1.1 which is backed by NFS (via kubernetes). It worked fine with docker-1.7
[root@yuck jan]# docker info
Containers: 109
Images: 349
Storage Driver: devicemapper
 Pool Name: docker-253:1-1713007-pool
 Pool Blocksize: 65.54 kB
 Backing Filesystem: extfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 10.1 GB
 Data Space Total: 107.4 GB
 Data Space Available: 20.08 GB
 Metadata Space Used: 23.7 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.124 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.93 (2015-01-30)
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.1.3-201.fc22.x86_64
Operating System: Fedora 22 (Twenty Two)
CPUs: 4
Total Memory: 15.38 GiB
Name: xxx
ID: xxx
Username: xxx
Registry: https://index.docker.io/v1/

[root@yuck jan]# docker version
Client:
 Version:      1.8.1.fc22
 API version:  1.20
 Package Version: docker-1.8.1-2.git3c1d7c8.fc22.x86_64
 Go version:   go1.4.2
 Git commit:   3c1d7c8/1.8.1
 Built:
 OS/Arch:      linux/amd64

Server:
 Version:      1.8.1.fc22
 API version:  1.20
 Package Version:
 Go version:   go1.4.2
 Git commit:   3c1d7c8/1.8.1
 Built:
 OS/Arch:      linux/amd64

[root@yuck jan]# uname -a
Linux yuck 4.1.3-201.fc22.x86_64 #1 SMP Wed Jul 29 19:50:22 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

It looks like it's a very recent thing in the code for Docker Registry.   (or in how Docker 1.8.1 pushes to the registry...) Is there anything the folks who are using this can do to help troubleshoot. I am somewhat new to the Docker Ecosystem, but would love to help however I can.

I have the same issue. Is there a work around for this issue?

it works when I don't mount volume to "//var/lib/registry"
("-v /c/Users/myUser/docker/data://var/lib/registry ")
I still need a solution to persist data.

Using the 1.7 client is all that I have found.

Has anyone been assigned to work on this yet? Registry + NFS usage is an enterprise level feature. If nobody is going to debug this at the moment, then I can spend time tracking down the issue in the code.

This issue should be created on the distribution(registry 2.x) tracker.
In the mean time if using NFS I recommend using the sync option when mounting the remote NFS directory. Running the push multiple times allows the docker client to push to the repository.
I'm fairly certain distribution is attempting to perform an action when the NFS locking/unlocking. I may also have system tuning options which will improve the situation until a patch is available.

Hitting the same issue.  Using registry:2 with nfs backend.  Client versions 1.6.2 and 1.7.1 work as expected.  1.8.1, build d12ea79 reports the following on push
docker push docker-registry-v2.mycompany.net:5000/tools/small_image:v2
The push refers to a repository [docker-registry-v2.mycompany.net:5000/tools/small_image] (len: 1)
ca76e3530c7e: Pushing [==================================================>]    172 B/172 B
Received unexpected HTTP status: 500 Internal Server Error
Logs from the registry
level=error msg="unknown error completing upload: &os.PathError{Op:"remove", Path:"/var/lib/registry/docker/registry/v2/repositories/tools/small_image/_uploads/0605a8da-12b5-4e89-b3a1-99ca79ad54fe", Err:0x27}"
level=error msg="unable to delete layer upload resources "/docker/registry/v2/repositories/tools/small_image/_uploads/0605a8da-12b5-4e89-b3a1-99ca79ad54fe": remove /var/lib/registry/docker/registry/v2/repositories/tools/small_image/_uploads/0605a8da-12b5-4e89-b3a1-99ca79ad54fe/.nfs00000000001bd7fe0000000a: device or resource busy" http.request.host="docker-registry-v2.mycompany.net:5000" http.request.id=cc424243-1ba9-43a5-b37f-b249ac287bd2
level=error msg="response completed with error" err.code=UNKNOWN err.detail="remove /var/lib/registry/docker/registry/v2/repositories/tools/small_image/_uploads/0605a8da-12b5-4e89-b3a1-99ca79ad54fe: directory not empty" err.message="unknown error" http.request.host="docker-registry-v2.mycompany.net:5000" http.request.id=cc424243-1ba9-43a5-b37f-b249ac287bd2 http.request.method=PUT

So I am not sure that we know if this is a distribution or a docker issue.  Looking at the reports, it looks like previous versions of Docker work with latest distribution, it's just the latest version of Docker + registryV2.  Makes it a harder issue to track down in that it could be a combination of factors that caused the issue (on both sides.
In my email alerts, I see some mentions of sync and locking. I hope we are not using locking over NFS in these setups as NFS support of locking is notoriously suspect across multiple NFS servers.   We shouldn't be utilizing something that has so much potential for inconsistency.  Perhaps something a bit more stable like atomic file creation would suffice? (I am not sure the internals here, I was just doing some background reading on NFS and locking issues).

Same issue here, docker client 1.8.2 with registry 2.1.1 on CentOS and NFS directory.

Confirm exact same symptoms with Ubuntu Trusty Docker 1.7.1 server + Registry 2.1.1 - push broken with 1.8.x client, fine with 1.7.1 client. NFS-mounted registry, but only one registry process at a time mounts the share, as chosen by consul lock.
I think I'm right to say that all of these reports are for .nfs[0-9a-f]+ files, 'silly unlinks': http://nfs.sourceforge.net/#faq_d2 .

@andrelucas hm, interesting, yes
@dmp42 is this a known issue? ^^ https://github.com/docker/docker/issues/15833#issuecomment-146212892

All: some users reported successfully setting up nfs in the past - though there is currently no nfs-specific documentation, it seems to be somewhat tricky, and the core team here has no expertise or experience with it...
Bottom-line:

I would really love to see someone successful here contribute a doc recipe for it
unfortunately, I doubt you will receive much help from the core-dev

We still are available to help on irc over #docker-distribution if you need help debugging the registry itself while poking nfs, so, please reach out.

I have the same issue, and here is my dirty fix for this.
Docker Hub: https://hub.docker.com/r/higebu/distribution/
Source: https://github.com/higebu/distribution/tree/fix-unable-to-delete-layer-upload
This fix ignores all error when remove directories. So temporary directories remain in your _uploads directory.

Note that as implied by https://github.com/docker/docker/issues/15833#issuecomment-137692059, this is also a problem when the Docker registry data is hosted on a Windows file system/share. That makes me strongly suspect that this is an issue with locking semantics.

I have the exact same issue (registry 2.1.1 with nfs, client 1.7.1 works) and can confirm that it also fails for 1.8.3. Any hope this might get fixed in 1.9?

I believe this is a registry issue vs. a docker issue.
On Fri, Oct 16, 2015 at 5:01 AM, Michael Sauter<EMAIL_ADDRESS>wrote:

I have the exact same issue (registry 2.1.1 with nfs, client 1.7.1 works)
and can confirm that it also fails for 1.8.3. Any hope this might get fixed
in 1.9?
—
Reply to this email directly or view it on GitHub
https://github.com/docker/docker/issues/15833#issuecomment-148659388.

--
https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

In my homelab I use Docker Registry v2 in backed with Nginx. Folder for images persistent storage mount by NFS and I have exact same issue. But today I try to test Docker Registry v2 without Nginx.
My test environment:
1. NFS storage
OS: CentOS 7
Firewalld is Off
cat /etc/exports
/export	<IP_ADDRESS>/16(rw,fsid=0,insecure,async,no_subtree_check,no_root_squash)
/export/docker <IP_ADDRESS>/16(rw,nohide,insecure,async,no_subtree_check,no_root_squash)

2. Docker Registry v2
OS: CentOS 7
FQDN: hub.solar.lan
Docker version 1.8.3, build f4bf5c7
NFS mount:
<IP_ADDRESS>:/docker on /mnt type nfs4 (rw,relatime,vers=4.0,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=<IP_ADDRESS>,local_lock=none,addr=<IP_ADDRESS>)

Registry image version:
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
registry            2                   2eafecf5086b        5 days ago          220.6 MB

Registry start string:
docker run -d -p 443:5000 --name registry -v `pwd`:/certs -v /mnt:/var/lib/registry -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/hub.crt -e REGISTRY_HTTP_TLS_KEY=/certs/hub.key registry:2

3. Docker Client
OS: CentOS 7
Docker version 1.8.3, build f4bf5c7
I try to push image hub.solar.lan/btsync build from Dockerbuild file on this machine to Registry on second machine and everything is OK. There was no error! Files of pushed image appeared on NFS storage server.

"docker push imagename" result with Docker 1.8.3 + Reg. 2.1.1
(Docker 1.7.1 + Reg. 2.1.1 works without problems)
level=warning msg="the ResponseWriter does not implement CloseNotifier" auth.user.name=user environment=production http.request.host=x.x.x.x http.request.id=3684e0b4-bc46-42c3-a356-589cf3b0713e http.request.method=PUT http.request.remoteaddr="x.x.x.x:xxxx" http.request.uri="/v2/jan/ubuntu/blobs/uploads/c32f56ba-6cb7-4bf6-975d-1e24b1e5994e?_state=cGunx-bsw4TaN6Mx6iq8IZePCMLzzZ8wjihktn8MeTx7Ik5hbWUiOiJqYW4vdWJ1bnR1IiwiVVVJRCI6ImMzMmY1NmJhLTZjYjctNGJmNi05NzVkLTFlMjRiMWU1OTk0ZSIsIk9mZnNldCI6MzIsIlN0YXJ0ZWRBdCI6IjIwMTUtMTAtMTdUMjA6NDU6NDZaIn0%3D&digest=sha256%3Aa3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.8.3 go/go1.4.2 git-commit/f4bf5c7 kernel/4.1.10-boot2docker os/linux arch/amd64" instance.id=4895394a-c536-4827-b836-e7f723d3ac15 service=registry vars.name="jan/ubuntu" vars.uuid=c32f56ba-6cb7-4bf6-975d-1e24b1e5994e version=v2.1.1
level=error msg="unable to delete layer upload resources \"/docker/registry/v2/repositories/jan/ubuntu/_uploads/c32f56ba-6cb7-4bf6-975d-1e24b1e5994e\": remove /var/lib/registry/docker/registry/v2/repositories/jan/ubuntu/_uploads/c32f56ba-6cb7-4bf6-975d-1e24b1e5994e: directory not empty" auth.user.name=user environment=production http.request.host=x.x.x.x http.request.id=3684e0b4-bc46-42c3-a356-589cf3b0713e http.request.method=PUT http.request.remoteaddr="x.x.x.x:xxxx" http.request.uri="/v2/jan/ubuntu/blobs/uploads/c32f56ba-6cb7-4bf6-975d-1e24b1e5994e?_state=cGunx-bsw4TaN6Mx6iq8IZePCMLzzZ8wjihktn8MeTx7Ik5hbWUiOiJqYW4vdWJ1bnR1IiwiVVVJRCI6ImMzMmY1NmJhLTZjYjctNGJmNi05NzVkLTFlMjRiMWU1OTk0ZSIsIk9mZnNldCI6MzIsIlN0YXJ0ZWRBdCI6IjIwMTUtMTAtMTdUMjA6NDU6NDZaIn0%3D&digest=sha256%3Aa3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.8.3 go/go1.4.2 git-commit/f4bf5c7 kernel/4.1.10-boot2docker os/linux arch/amd64" instance.id=4895394a-c536-4827-b836-e7f723d3ac15 service=registry vars.name="jan/ubuntu" vars.uuid=c32f56ba-6cb7-4bf6-975d-1e24b1e5994e version=v2.1.1
level=error msg="unknown error completing upload: &os.PathError{Op:\"remove\", Path:\"/var/lib/registry/docker/registry/v2/repositories/jan/ubuntu/_uploads/c32f56ba-6cb7-4bf6-975d-1e24b1e5994e\", Err:0x27}" auth.user.name=user environment=production http.request.host=x.x.x.x http.request.id=3684e0b4-bc46-42c3-a356-589cf3b0713e http.request.method=PUT http.request.remoteaddr="x.x.x.x:xxxx" http.request.uri="/v2/jan/ubuntu/blobs/uploads/c32f56ba-6cb7-4bf6-975d-1e24b1e5994e?_state=cGunx-bsw4TaN6Mx6iq8IZePCMLzzZ8wjihktn8MeTx7Ik5hbWUiOiJqYW4vdWJ1bnR1IiwiVVVJRCI6ImMzMmY1NmJhLTZjYjctNGJmNi05NzVkLTFlMjRiMWU1OTk0ZSIsIk9mZnNldCI6MzIsIlN0YXJ0ZWRBdCI6IjIwMTUtMTAtMTdUMjA6NDU6NDZaIn0%3D&digest=sha256%3Aa3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.8.3 go/go1.4.2 git-commit/f4bf5c7 kernel/4.1.10-boot2docker os/linux arch/amd64" instance.id=4895394a-c536-4827-b836-e7f723d3ac15 service=registry vars.name="jan/ubuntu" vars.uuid=c32f56ba-6cb7-4bf6-975d-1e24b1e5994e version=v2.1.1
level=error msg="unable to delete layer upload resources \"/docker/registry/v2/repositories/jan/ubuntu/_uploads/c32f56ba-6cb7-4bf6-975d-1e24b1e5994e\": remove /var/lib/registry/docker/registry/v2/repositories/jan/ubuntu/_uploads/c32f56ba-6cb7-4bf6-975d-1e24b1e5994e/.nfs0000000002b81b2700000070: device or resource busy" auth.user.name=user environment=production http.request.host=x.x.x.x http.request.id=3684e0b4-bc46-42c3-a356-589cf3b0713e http.request.method=PUT http.request.remoteaddr="x.x.x.x:xxxx" http.request.uri="/v2/jan/ubuntu/blobs/uploads/c32f56ba-6cb7-4bf6-975d-1e24b1e5994e?_state=cGunx-bsw4TaN6Mx6iq8IZePCMLzzZ8wjihktn8MeTx7Ik5hbWUiOiJqYW4vdWJ1bnR1IiwiVVVJRCI6ImMzMmY1NmJhLTZjYjctNGJmNi05NzVkLTFlMjRiMWU1OTk0ZSIsIk9mZnNldCI6MzIsIlN0YXJ0ZWRBdCI6IjIwMTUtMTAtMTdUMjA6NDU6NDZaIn0%3D&digest=sha256%3Aa3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.8.3 go/go1.4.2 git-commit/f4bf5c7 kernel/4.1.10-boot2docker os/linux arch/amd64" instance.id=4895394a-c536-4827-b836-e7f723d3ac15 service=registry vars.name="jan/ubuntu" vars.uuid=c32f56ba-6cb7-4bf6-975d-1e24b1e5994e version=v2.1.1
level=error msg="error canceling upload after error: remove /var/lib/registry/docker/registry/v2/repositories/jan/ubuntu/_uploads/c32f56ba-6cb7-4bf6-975d-1e24b1e5994e/.nfs0000000002b81b2700000070: device or resource busy" auth.user.name=user environment=production http.request.host=x.x.x.x http.request.id=3684e0b4-bc46-42c3-a356-589cf3b0713e http.request.method=PUT http.request.remoteaddr="x.x.x.x:xxxx" http.request.uri="/v2/jan/ubuntu/blobs/uploads/c32f56ba-6cb7-4bf6-975d-1e24b1e5994e?_state=cGunx-bsw4TaN6Mx6iq8IZePCMLzzZ8wjihktn8MeTx7Ik5hbWUiOiJqYW4vdWJ1bnR1IiwiVVVJRCI6ImMzMmY1NmJhLTZjYjctNGJmNi05NzVkLTFlMjRiMWU1OTk0ZSIsIk9mZnNldCI6MzIsIlN0YXJ0ZWRBdCI6IjIwMTUtMTAtMTdUMjA6NDU6NDZaIn0%3D&digest=sha256%3Aa3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.8.3 go/go1.4.2 git-commit/f4bf5c7 kernel/4.1.10-boot2docker os/linux arch/amd64" instance.id=4895394a-c536-4827-b836-e7f723d3ac15 service=registry vars.name="jan/ubuntu" vars.uuid=c32f56ba-6cb7-4bf6-975d-1e24b1e5994e version=v2.1.1
level=error msg="response completed with error" auth.user.name=user environment=production err.code=UNKNOWN err.detail="remove /var/lib/registry/docker/registry/v2/repositories/jan/ubuntu/_uploads/c32f56ba-6cb7-4bf6-975d-1e24b1e5994e: directory not empty" err.message="unknown error" http.request.host=x.x.x.x http.request.id=3684e0b4-bc46-42c3-a356-589cf3b0713e http.request.method=PUT http.request.remoteaddr="x.x.x.x:xxxx" http.request.uri="/v2/jan/ubuntu/blobs/uploads/c32f56ba-6cb7-4bf6-975d-1e24b1e5994e?_state=cGunx-bsw4TaN6Mx6iq8IZePCMLzzZ8wjihktn8MeTx7Ik5hbWUiOiJqYW4vdWJ1bnR1IiwiVVVJRCI6ImMzMmY1NmJhLTZjYjctNGJmNi05NzVkLTFlMjRiMWU1OTk0ZSIsIk9mZnNldCI6MzIsIlN0YXJ0ZWRBdCI6IjIwMTUtMTAtMTdUMjA6NDU6NDZaIn0%3D&digest=sha256%3Aa3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.8.3 go/go1.4.2 git-commit/f4bf5c7 kernel/4.1.10-boot2docker os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=32.171187ms http.response.status=500 http.response.written=208 instance.id=4895394a-c536-4827-b836-e7f723d3ac15 service=registry vars.name="jan/ubuntu" vars.uuid=c32f56ba-6cb7-4bf6-975d-1e24b1e5994e version=v2.1.1

@verdel Interesting. I'm using just the registry, no NGINX in front.

@michaelsauter I try with NGINX now and everything is ok too.
Usually I use standalone server with Ubuntu 14.04 on board to export share with NFS. In this case I have the exact issue with registry and lock.
For test I use virtual machines on vsphere esxi. Virtual machine specification I listed above. I will spend some more tests and write the result here. Maybe I change OS from Ubuntu 14.04 to Centos 7 on my standalone server.

I've encountered this issue on an enterprise cloud and spent a couple days trying to debug it before finding this thread.
I have the docker 1.8.1 docker-engine  on Ubuntu 14.04.
I deployed the docker V2 registry into an enterprise docker cloud. To provide persistent storage In the cloud deployment, I map a cloud storage volume into my registry container at /var/lib/registry. and docker push returned
'Received unexpected HTTP status: 500 Internal Server Error'
I installed 1.7.1  docker-engine client on another Ubuntu box and my images push successfully into the cloud.

Same issue here :(
Any fix?
When I try to push to registry:v2.1.1 - issue still exists
When I try to push to registry:v2 - same issue again
Now when trying to push to registry:v0.9.1 - same issue  am stuck :(

I have the same issue except I'm not using NFS. I'm mapping a volume into the container.
OS: Ubuntu 14
Driver: aufs
Command use to launch the registry:
docker --tls -H docker_host:2376 run -d -v /data/registry-v2:/var/lib/registry --name registry-v2 -p 5001:5000 registry:2

We have exactly the same problem.

Here too, with docker 1.7.1 on CentOS 7.1 it works, with docker 1.8.3 on CentOS7.1 not.

(I posted this in https://github.com/docker/distribution/issues/1116#issuecomment-152881763, but since it suggests a code fix perhaps here is a better place.)
I just checked using lsof and determined that the registry itself is holding the stale file handle:
registry  1161 1179             root    6r      REG               0,39        32      49743 /var/lib/registry/docker/registry/v2/repositories/test_image/_uploads/5ac646c9-dc2b-4378-87e0-4ec45c646192/.nfs000000000000c24f00000003

This handle is kept around until I do another docker push, at which point it is promptly deleted, and a stale handle for the new image is now created:
registry  1161 1179             root    6r      REG               0,39        32      49745 /var/lib/registry/docker/registry/v2/repositories/test_image2/_uploads/395cb041-4b23-4c60-8bc1-bca7e89ce952/.nfs000000000000c25100000004

One cause of NFS stale file handles is when a file handle is still open when an OS call attempts to delete the file.  Is it possible that the reference to the file handle used for the upload is still in scope in the registry when the subsequent step of deleting the _uploads/ directory occurs?
On a normal filesystem, this probably doesn't cause any issues because the OS can simply hang onto the inode, but since NFS creates an undeletable busy resource in the same directory when the file is deleted, it fails the directory delete operation.
If this is the case, adjusting the scope of the upload file handle might resolve the issue.

ping @dmp42 ^^

cc @docker/distribution-maintainers for the record
NFS is not officially supported - I know some people had it working, not sure how.
Either way, I'm not sure why this bug is opened here (assuming this is about registry support for NFS).

Fixed this issue with https://github.com/docker/distribution/pull/1149. Registry was holding extra resources open that interacted poorly with nfs mounts. Closing this in favor of the docker/distribution#1116 since this is clearly a registry bug.

@stevvooe Great, thanks!


Bonfire Sum All Primes Has A Bug

In the challenge, if you write the codes for the body of the function but you return ''num" (which is the argument passed to the function) instead of "sum" (the expected return value of the function) , all the test cases are marked as being passed though no notification pops up congratulating you and prompting you to move on to the next challenge.

the pictures above are screenshots of my code. As you can see, returning num from my function passes all the test cases but I wasn't able to move to the next challenge, and that was how I figured out there's something wrong with my code. You may want to fix that. Keep on the good work guys! FCC is a great open-source community.

@ayondot Which browser and operating system are you running?

I use a Maxthon browser, on a PC running on Windows 8 OS.

We do not currently support Maxthon browser.
Its most likely an issue with web workers which will be refactored soon. When that happens it might fix this issue.
Happy Coding!


Write a TypeScript program to convert inches to miles

Description
Write a TypeScript program to convert inches to miles
1 mile = 63360 inches
Input  : 316800
Output : 5

How to contribute

Save the solution in ConvertInchesToMiles.ts file
Add ConvertInchesToMiles.ts file in convert-inches-to-miles folder

!assign


Use RD-encoded streams at different bitrates depending on connection speed

The Stremio+torrentio+RD combo is perfect for in-home streaming, where a good connection is available.
However, on the go, high speed connection is not always available, and in many cases, an episode/movie has only a few cached links, which are all too large to stream on slow connections.
Since RD supports on-the-fly re-encoding at different bitrates in it's streaming player, I was wonder whether torrentio could use that feature to request a cached link at a lower bitrate. This ideally could be done dynamically depending on connection speed. But it would also be enough to provide separate links for different bitrates (this could be customizable like quality and size), so the user can choose the desired bitrate manually.
What do you think?

That's outside of the scope for an addon, it only provides link to the video stored in debrid services and doesn't measure client download speed. For such features I'd suggest to then use RD website directly.

I respect your choice to not implement it, but then also sorting by file size/quality should be out of the scope of the addon. The bitrate value is something that could be at the same level of quality and size. If choosing it dynamically is out of the scope, then adding it as an additional sorting parameter could remain in scope.

@marco-calautti That is not true. File size and quality is something that you can scrape and get from the torrent during the search phase while for the bitrate you would have to inspect the video file to get the bitrate information. Sure you could calculate average bitrate from size and length but I don't really see the point of that. Just setup another account with lower file size limit for situations with lower connection speeds or edit it on the fly if having issues.

@marco-calautti That is not true. File size and quality is something that you can scrape and get from the torrent during the search phase while for the bitrate you would have to inspect the video file to get the bitrate information. Sure you could calculate average bitrate from size and length but I don't really see the point of that. Just setup another account with lower file size limit for situations with lower connection speeds or edit it on the fly if having issues.

I think there is a misunderstanding on what I was proposing. By bitrate I am not talking about the bitrate of the actual file, but to request to RD to re-encode the file to a desired bitrate before streaming, so to reduce bandwidth usage (e.g., on mobile connections).
This is what the video player of Real Debrid allows you to do before streaming it. I guess RD's API exposes a way to request the file at a desired encoding.
This procedure does not require checking anything on the file itself, but should be doable by just using the right API call.


UVI Request

--- UVI JSON ---
{
  "vendor_name": "Linux",
  "product_name": "Kernel",
  "product_version": "versions from v5.10 to before v5.10.54",
  "vulnerability_type": "unspecified",
  "affected_component": "unspecified",
  "attack_vector": "unspecified",
  "impact": "unspecified",
  "credit": "",
  "references": [
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=4363023d2668e621b0743db351a9555d6e6ea57e",
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=715f378f42909c401ec043f5150c4fdf57fb8889"
  ],
  "extended_references": [
    {
      "type": "commit",
      "value": "4363023d2668e621b0743db351a9555d6e6ea57e",
      "note": "introduced"
    },
    {
      "type": "commit",
      "value": "715f378f42909c401ec043f5150c4fdf57fb8889",
      "note": "fixed"
    }
  ],
  "reporter": "joshbressers",
  "reporter_id": 1692786,
  "notes": "",
  "description": "bpf, sockmap: Fix potential memory leak on unlikely error case\n\nThis is an automated ID intended to aid in discovery of potential security vulnerabilities. The actual impact and attack plausibility have not yet been proven.\nThis ID is fixed in Linux Kernel version v5.10.54 by commit 715f378f42909c401ec043f5150c4fdf57fb8889, it was introduced in version v5.10 by commit 4363023d2668e621b0743db351a9555d6e6ea57e. For more details please see the references link."
}
--- UVI JSON ---

This issue has been assigned UVI-2021-1001247


Use runme-action to trigger tests

Kick of tests through Github action.

I am getting a cert issue when I assume starting Runme server:
Running Runme v1.0.1
runme run kubectl-version --chdir=/home/runner/work/linkerd-website/linkerd-website/tests/runme --filename=TEST.md
/opt/hostedtoolcache/runme/1.0.1/x64/runme run kubectl-version --chdir=/home/runner/work/linkerd-website/linkerd-website/tests/runme --filename=TEST.md
open /home/runner/.config/stateful/tls/cert.pem: no such file or directory

@sourishkrout any clues?

I am getting a cert issue when I assume starting Runme server:
Running Runme v1.0.1
runme run kubectl-version --chdir=/home/runner/work/linkerd-website/linkerd-website/tests/runme --filename=TEST.md
/opt/hostedtoolcache/runme/1.0.1/x64/runme run kubectl-version --chdir=/home/runner/work/linkerd-website/linkerd-website/tests/runme --filename=TEST.md
open /home/runner/.config/stateful/tls/cert.pem: no such file or directory

@sourishkrout any clues?

@duvanmonsa @mxsdev related to the homedir changes in v1.0.1?

After chatting with @mxsdev I was able to resolve the issue by adding the capability to start a Runme server to the Action. It seems that everything is working now as expected.


CB-21466 Add xml validation unit test for all CM templates

Add unit tests to check that value for applicable safty valves in template is well formed.
See detailed description in the commit message.

retest this please


Replace old models

Replace with new ParseModel structure.

Old models

[ ] file compare
[ ] file mirror
[ ] member detach
[ ] member invite
[ ] member remove
[ ] member sync
[ ] team audit events
[ ] team audit sharing
[ ] team device list
[ ] team device unlink
[ ] team app list
[ ] team namespace file list
[ ] team namespace file size
[ ] team sharedlink update expire
[ ] team sharedlink list
[ ] team info
[ ] team feature
[ ] teamfolder size
[ ] teamfolder mirror
[ ] teamfolder file list

fixed at https://github.com/watermint/toolbox/pull/83


Please add Jackson, Wyoming. 0 emails for WY so far!

JacksonWY_Defund12_Email.docx
To<EMAIL_ADDRESS><EMAIL_ADDRESS>Subject: [*** INSERT UNIQUE SUBJECT LINE***]
Message:
[Don't forget to replace the [x]'s with your information!]
Dear County Commissioners and Mayor Muldoon,
My name is [YOUR NAME] and I am writing to you as a concerned resident of [JACKSON/WILSON/RAFTER J/ TETON COUNTY etc]. Now and in the year to come, our community will need to rely on social service organizations like WIC, One22, Public Health, Hole Food Rescue, and others at an increasing rate. Our prospective 2021 budget gives the Sheriff’s Department an extreme priority, while these other essential organizations are left with meager funds to conduct their operations.
It is unacceptable that most organizations will see large cuts while the Sheriff’s Department will see none. In light of this reality:

I urge you to significantly cut the Sheriff’s Department’s $8 million budget
I urge you to advocate for a meaningful reallocation of Teton County’s expenditures: away from policing, and towards social programs and resources that support housing, jobs, education, health care, childcare, and other critical community needs

The police are not going to feed us when we’re hungry. They are not going to house us when we can no longer pay our rent. They are not going to protect us from domestic violence, and in fact frequently perpetrate it themselves. The residents of Teton County are tired of our tax dollars being used to fund a bloated, overstaffed police department while organizations that prioritize public good are fighting for budgetary scraps.
I know this is not easy, and I am hopeful you will do the right thing. While not a new concept, this topic is currently at the center of conversations across the country. Multiple cities (Los Angeles, Minneapolis, New York) have already taken steps to restructure and reallocate financial resources of their police departments, and more cities are likely to follow suit in the weeks, months, and years to come. A different future is possible. Do not let our county be held hostage by a police department that has too long held sway over everything else.
Thank you,
[YOUR NAME]
[YOUR ADDRESS]
[YOUR EMAIL]
[YOUR PHONE NUMBER]

Hi @tisad819 ! I was validating the statistics and wanted to confirm the 8 million sheriff department budget -- the 2021 recommended budget for police is $4,168,117 according to https://www.jacksonwy.gov/DocumentCenter/View/3249/Volume-II---FY2021-Recommended-Line-Item-Budget , if I am mistaken then please ignore.
Could you also add percentages of how much of the budget goes to the police, in comparison to other community programs, initiatives, or infrastructure to make a more compelling argument, showing the disparities in numbers?

It seems like the email's referring to the Teton County Sheriff's Department (the "Total Public Safety" budget here: http://www.tetonwyo.org/DocumentCenter/View/10809/FY-2019-2020-Adopted-Budget)
Email could be reworded to the following (edits in bold). @tisad819 let us know if this is ok:
To<EMAIL_ADDRESS><EMAIL_ADDRESS>Subject: [*** INSERT UNIQUE SUBJECT LINE***]
Message:
[Don't forget to replace the [x]'s with your information!]
Dear County Commissioners and Mayor Muldoon,
My name is [YOUR NAME] and I am writing to you as a concerned resident of [JACKSON/WILSON/RAFTER J/ TETON COUNTY etc]. Now and in the year to come, our community will need to rely on social service organizations like WIC, One22, Public Health, Hole Food Rescue, and others at an increasing rate. Our prospective 2021 budget gives the Sheriff’s Department an extreme priority, while these other essential organizations are left with meager funds to conduct their operations.
It is unacceptable that most organizations will see large cuts while the Sheriff’s Department will see none. In light of this reality:
I urge you to significantly cut the $6.1 million budget of the Sheriff’s Department and communications.
I urge you to advocate for a meaningful reallocation of Teton County’s expenditures: away from policing, and towards social programs and resources that support housing, jobs, education, health care, childcare, and other critical community needs
The police are not going to feed us when we’re hungry. They are not going to house us when we can no longer pay our rent. They are not going to protect us from domestic violence, and in fact frequently perpetrate it themselves. The residents of Teton County are tired of our tax dollars being used to fund a bloated, overstaffed police department while organizations that prioritize public good are fighting for budgetary scraps.
I know this is not easy, and I am hopeful you will do the right thing. While not a new concept, this topic is currently at the center of conversations across the country. Multiple cities (Los Angeles, Minneapolis, New York) have already taken steps to restructure and reallocate financial resources of their police departments, and more cities are likely to follow suit in the weeks, months, and years to come. A different future is possible. Do not let our county be held hostage by a police department that has too long held sway over everything else.
Thank you,
[YOUR NAME]
[YOUR ADDRESS]
[YOUR EMAIL]
[YOUR PHONE NUMBER]

Hi thank you for your fact checking and edits! We have been working to gather more info and talk to local representatives to see what will work best for our demands. This is the budget we have been looking at, so the number we are asking to cut from is $7,893,707. We have tweaked the message to read:
To<EMAIL_ADDRESS>Subject: [*** INSERT UNIQUE SUBJECT LINE***]
Message:
[Don't forget to replace the [x]'s with your information!]
Dear County Commissioners,
I am writing to you as a concerned resident of Teton County.
Now, and in the year to come, our community will need to rely on social service organizations such as, but not limited to, One22, Public Health, and Hole Food Rescue at an increasing rate.
Our budget has always given the Sheriff's Department extreme financial priority at the expense of community service organizations, and the 2021 budget is no different. The Commissioners have decided not to decrease Human Services funding in the 2021 budget, but this is not enough. As more families begin to rely on these services, they should receive funding commensurate with the growing need.
I urge you to advocate for a meaningful reallocation of Teton County’s expenditures: away from policing, and towards social programs and resources that support housing, jobs, education, health care, childcare, and other critical community needs
I urge you to:

significantly cut the Sheriff Department's projected 7.9 million dollar budget
commit to enacting both a community needs survey to find out what we can do to provide for our most vulnerable community members, and a survey of local human services organizations to find out what help they need to best meet these needs, and
use the results of these surveys to advocate for a meaningful reallocation of Teton County's expenditures away from policing, and towards social programs and resources.

I want to see these funds allocated towards resources that support housing, jobs, education, healthcare, childcare and other needs in our community.
The police are not going to feed us when we're hungry. They are not going to house us when we can no longer pay our rent. They are not going to protect us from domestic violence - and statistically, are likely to perpetuate it themselves.
As a resident of Teton County, I want the use of my tax dollars to prioritize funding organizations that contribute to the well being of our community over funding policing.
I am hopeful that you will prioritize the wishes of your community in this matter.
This topic is currently at the center of conversations across the country. Los Angeles, New York City and Minneapolis have already taken steps to reallocate the financial resources of their police departments. More cities are likely to follow suit in the weeks, months and years to come.
A different future is possible. Please do not let our county continue to prioritize a fear-based "need" for policing over genuine community needs.
Thank you,
[YOUR NAME]
[YOUR ADDRESS]
[YOUR EMAIL]
[YOUR PHONE NUMBER]


Charm Added

View rendered README.md

Maybe don’t use "crazy" because it’s ableist, maybe something that better describes what you want to convey? Maybe "Live if wonderful" or "Life is full of surprises"?


Update README.md

Update 😊

[x] My submission is formatted according to the guidelines in the contributing guide
[x] My addition is ordered alphabetically
[x] My submission has a useful description
[x] The description does not have more than 100 characters
[x] The description does not end with punctuation
[x] Each table column is padded with one space on either side
[x] I have searched the repository for any relevant issues or pull requests
[x] Any category I am creating has the minimum requirement of 3 items
[x] All changes have been squashed into a single commit

#3366 Updated the readme link.
Api tested in python here https://github.com/shade234sherif/bio-replika


rdk ibc: Figure out chain upgrade how to set canonical port/channel

After the upgrade, there is a notion of canonical port/channel
Need to make sure existing chains are not broken by this upgrade

IIRC there was some canonical channel concept introduced on the rollapp when Alex and I did the genesis transfers and the denom metadata work,
These are old issues, but need to be done/double checked
I'm just looking now
https://github.com/dymensionxyz/dymension-rdk/blob/74f21279f16d3a5608ee442406b439024b18f474/x/hub-genesis/types/state.go#L28-L30
Yeah I think we need to write an upgrade on the rollapp to set canonical channel for existing rollapps
state.go
func (s *State) IsCanonicalHubTransferChannel(port, channel string) bool {
return s.CanonicalHubTransferChannelHasBeenSet() && s.HubPortAndChannel.Port == port && s.HubPortAndChannel.Channel == channel
}


DONTMERGE: not-a-vulnerability cp a minor fix on lua

Lua: Change the TLS callback function type of ThreadLocalState to Upd… (#11944)
Update LuaJIT patch - remove MAP_32BIT (#10867)
Signed-off-by: wbpcode<EMAIL_ADDRESS>Signed-off-by: John Murray<EMAIL_ADDRESS>Signed-off-by: Yuchen Dai<EMAIL_ADDRESS>Change-Id: I62739dc1c3250fcff755d0a6208ec4f78cda695b
For an explanation of how to fill out the fields, please see the relevant section
in PULL_REQUESTS.md
Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
[Optional Fixes #Issue]
[Optional Deprecated:]

Sigh... tests are not running. Close this one


Automatically wrap string expressions in condition in apostrophes

Take this variable condition:
        Variables =
        {
            StartsWith("refs/head/feature/", variables.Build.SourceBranch).Variable("myvariable", "pr-variable")
                .Else.Variable("myvariable", "other-variable"),
        },

The yaml being produced is:
variables:
- ${{ if startsWith($(Build.SourceBranch), refs/head/feature/) }}:
  - name: myvariable
    value: pr-variable
- ${{ else }}:
  - name: myvariable
    value: other-variable

I get this exception:

I think both of these values in the Contains/StartsWith/EndsWith conditions need to be made into strings.
i.e. adding quotes around them

I've proposed this solution. Let me know if you think it's bad.
#229

Thanks again for the contribution, @thomhurst! You can find the new updates in the 1.4.0 package.
Actually, with the if does not start with ' it's probably not even a breaking change but I have already tagged it like 1.4.0 so we have that now.


Trouble with importing saves

I can't seem to import my save file. I'd love to continue where I left off before, but when I put it in, nothing changes.
My save file/code is:
{“upgrades”:[1,2,7,3,8,4,5,12,9,6,13,10,14,15,11,16,17,18,19,20,21],“upgradeHash”:{“1”:1,“2”:2,“3”:3,“4”:4,“5”:5,“6”:6,“7”:7,“8”:8,“9”:9,“10”:10,“11”:11,“12”:12,“13”:13,“14”:14,“15”:15,“16”:16,“17”:17,“18”:18,“19”:19,“20”:20,“21”:21},“smiles”:4.9624667071656466e+26,“totalsmiles”:1.7885710640515677e+27,“SPC”:62668670118087130000,“SPS”:2.070557632582351e+23,“shiftDown”:false,“totalTime”:8734000,“clicks”:4187,“achievements”:{“1”:0,“2”:0,“3”:1,“4”:2,“5”:3,“10”:1,“11”:2,“12”:3,“13”:4,“14”:5,“15”:6,“16”:7,“17”:8,“18”:9,“19”:10,“20”:1,“21”:2,“22”:3,“23”:4,“24”:5,“25”:1,“26”:2,“27”:3,“28”:4,“29”:5,“30”:1,“31”:2,“32”:3,“33”:4,“34”:5,“35”:1,“36”:2,“37”:3,“38”:4,“40”:1,“41”:2,“42”:3,“45”:1,“46”:2,“47”:3,“50”:1,“51”:2,“52”:3,“55”:1,“56”:2,“60”:1,“61”:2,“65”:1,“66”:2,“67”:3,“68”:4,“200”:1,“201”:1,“202”:1,“203”:1,“204”:1},“achievementcount”:56,“muffins”:155,“achievement_muffins”:155,“store”:[22,212,255,230,200,170,140,120,100,93,70,63,0],“ponyList”:[0,6,33,2,4,8,24,15,34,49,41,56,51,48,17,27,11,31,44,20,19,40,18,25,36,16,14,22,42,3,12,29,47,13,30,7,35,21,39,32,23,1,46,55,37,9,54,57,45,53,43,28,26,52,10,38,5,50],“version”:3,“settings”:{“useCanvas”:false,“optimizeFocus”:true,“closingWarn”:true},“delta”:15.871999960392714}
I just copied it from the thing before, of course, so I don't see how there could be any errors in it... Unless it's not reading it because the save codes are different now? If that's the case, can anyone make it so that my save works?

Whatever you pasted the code into apparently changed the raw ascii " into pretty unicode smart quotes: “”. Because of this it can't read it, because they aren't technically quotes, they're special characters. Here is your save file with the quotation marks restored:
{"upgrades":[1,2,7,3,8,4,5,12,9,6,13,10,14,15,11,16,17,18,19,20,21],"upgradeHash":{"1":1,"2":2,"3":3,"4":4,"5":5,"6":6,"7":7,"8":8,"9":9,"10":10,"11":11,"12":12,"13":13,"14":14,"15":15,"16":16,"17":17,"18":18,"19":19,"20":20,"21":21},"smiles":49624667071656466e+26,"totalsmiles":17885710640515677e+27,"SPC":62668670118087130000,"SPS":2070557632582351e+23,"shiftDown":false,"totalTime":8734000,"clicks":4187,"achievements":{"1":0,"2":0,"3":1,"4":2,"5":3,"10":1,"11":2,"12":3,"13":4,"14":5,"15":6,"16":7,"17":8,"18":9,"19":10,"20":1,"21":2,"22":3,"23":4,"24":5,"25":1,"26":2,"27":3,"28":4,"29":5,"30":1,"31":2,"32":3,"33":4,"34":5,"35":1,"36":2,"37":3,"38":4,"40":1,"41":2,"42":3,"45":1,"46":2,"47":3,"50":1,"51":2,"52":3,"55":1,"56":2,"60":1,"61":2,"65":1,"66":2,"67":3,"68":4,"200":1,"201":1,"202":1,"203":1,"204":1},"achievementcount":56,"muffins":155,"achievement_muffins":155,"store":[22,212,255,230,200,170,140,120,100,93,70,63,0],"ponyList":[0,6,33,2,4,8,24,15,34,49,41,56,51,48,17,27,11,31,44,20,19,40,18,25,36,16,14,22,42,3,12,29,47,13,30,7,35,21,39,32,23,1,46,55,37,9,54,57,45,53,43,28,26,52,10,38,5,50],"version":3,"settings":{"useCanvas":false,"optimizeFocus":true,"closingWarn":true},"delta":15871999960392714}
To prevent this in the future, try copying and pasting into something really simple, like Notepad, that won't try to be "helpful". I'm classifying this as a bug and will try to write code that catches this and replaces the quotations automatically.

This has been fixed. Refresh pony clicker and you should be able to import your save file.

Thank you so much! It's nice to see my game again.
Now I just need to find another place to store the save without it messing it up.


Fixed #1026: Use export keystore instead of download to avoid misunderstanding.

What type of PR is this?
ui-change
What this PR does / why we need it:
Which issue(s) this PR fixes:

Fixes #1026
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

please resolve conflicts

please resolve conflicts

resolved


Likely memory leaks with huge datasets in Dataset class

As far as I can see, the dataset in SpeechBrain is a list holding dicts with file paths.
With current state of affairs in PyTorch (maybe DataPipes will fix this?), this is the road to DataLoader leaking memory: https://github.com/pytorch/pytorch/issues/13246
I experienced this myself in a similar set-up (dozen thousand hours datasets, some string meta information)
The solution is to use hacks such as dicts of managed lists or packing strings into tensors (my solution): https://gist.github.com/vadimkantorov/86c3a46bf25bed3ad45d043ae86fff57

@Gastron is WebDataset fixing this? I guess so ?

Oof, complicated stuff! Thank you for raising this issue. As I understand it, bypassing this problem requires specific data types (ones that avoid copy-on-write behaviour), so in the general case this requires some effort from the user?
I believe WebDataset data loading (which is our current go-to for huge datasets) does not suffer from this, and hopefully once the PyTorch data loading updates are ready, this is also taken care of.
So for now, I think it's good that we have this issue in our issue history for reference if people run into this, and the recommended solution should be to either switch to WebDataset or then use solutions such as the one in @vadimkantorov 's gist. Does that sound good?

Closing for now, see above comment.

In general case it requires some effort from the Dataset subclass author to replace list-of-dicts containers with other data structures. But it's possible and stays under the hood from the end-user standpoint.


New 429 errors from get_shared_data

I previously got the occasional 429 Client Error, but recently I started getting them much more frequently.  The earlier errors are probably related to other open issues, but this issue is for the more recent ones.
I'm trying to download some data for a list of users.  The first 5 to 10 succeed without errors, but then after that, most of the users that follow get 5 to 10 errors each before it finally succeeds.
I finally tracked it down in the code to the get_shared_data function, which downloads https://www.instagram.com/<user>.  That is the call that gets the 429, but it is called before any of the actual downloading is done.
While I was getting these errors, I went to the same URL in my web browser, and verified that I got the same error, but if I went to https://www.instagram.com and then navigated to the user's page through Instagram's UI, I didn't receive the error.
I haven't been able to figure out any more details about how it is done yet, but it seems that Instagram has some logic to prevent that error when you stay within the app.

Have the same issue. I'm scraping a list (6000 usernames in .txt) for stories only. After a while, I get the 427 errors, and it's getting stuck...
See also a related issue for instaloader: https://github.com/instaloader/instaloader/issues/392

you cant scrape thousands users. its so easy to detect because no legit user will look at so many accounts as quickly as these scripts do.