Fastlane build fails with  Unable to make field private final java.lang.String java.io.File.path

New Issue Checklist

[x] Updated fastlane to the latest version
[x] I read the Contribution Guidelines
[x] I read docs.fastlane.tools
[x] I searched for existing GitHub issues

Issue Description
I am unable to deploy this lane
 lane :slackbuild do    gradle(        task: 'clean'      )      gradle(        task: 'assemble',        build_type: 'Debug'      )     slack(message: "Build Successful!")     upload_to_slack()   end
Command executed
fastlane slackbuild
Complete output when running fastlane, including the stack trace and command used
I am getting this error
**Execution failed for task ':ondemand.driver.app:extractDeepLinksDebug'.

Unable to make field private final java.lang.String java.io.File.path accessible: module java.base does not "opens java.io" to unnamed module @4547f45b**
[07:12:30]: Exit status of command '/Users/stevehechio/StudioProjects/GetbodaOndemandDriver/gradlew assembleDebug -p .' was 1 instead of 0.
Task :ondemand.driver.app:preBuild UP-TO-DATE
Task :ondemand.driver.app:preDebugBuild UP-TO-DATE
Task :ondemand.driver.app:compileDebugAidl NO-SOURCE
Task :ondemand.driver.app:compileDebugRenderscript NO-SOURCE
Task :ondemand.driver.app:dataBindingMergeDependencyArtifactsDebug
Task :ondemand.driver.app:dataBindingMergeGenClassesDebug
Task :ondemand.driver.app:generateDebugResValues
Task :ondemand.driver.app:generateDebugResources
Task :ondemand.driver.app:injectCrashlyticsMappingFileIdDebug

Task :ondemand.driver.app:processDebugGoogleServices
Parsing json file: /Users/stevehechio/StudioProjects/GetbodaOndemandDriver/ondemand.driver.app/google-services.json

Task :ondemand.driver.app:generateDebugBuildConfig
Task :ondemand.driver.app:mergeDebugResources
Task :ondemand.driver.app:checkDebugAarMetadata
Task :ondemand.driver.app:dataBindingGenBaseClassesDebug
Task :ondemand.driver.app:createDebugCompatibleScreenManifests
Task :ondemand.driver.app:extractDeepLinksDebug FAILED

FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':ondemand.driver.app:extractDeepLinksDebug'.

Unable to make field private final java.lang.String java.io.File.path accessible: module java.base does not "opens java.io" to unnamed module @4547f45b

 [REPLACE THIS WITH YOUR INFORMATION]

Environment
🚫 fastlane environment 🚫
Stack

Key
Value

OS
10.15.7

Ruby
2.7.2

Bundler?
false

Git
git version 2.23.0

Installation Source
/usr/local/Cellar/fastlane/2.177.0/libexec/bin/fastlane

Host
Mac OS X 10.15.7 (19H1217)

Ruby Lib Dir
<EMAIL_ADDRESS>

OpenSSL Version
OpenSSL 1.1.1i  8 Dec 2020

Is contained
false

Is homebrew
true

Is installed via Fabric.app
false

Xcode Path
/Applications/Xcode.app/Contents/Developer/

Xcode Version
12.4

System Locale

Error

No Locale with UTF8 found 🚫

fastlane files:
`./fastlane/Fastfile`
# This file contains the fastlane.tools configuration
# You can find the documentation at https://docs.fastlane.tools
#
# For a list of all available actions, check out
#
#     https://docs.fastlane.tools/actions
#
# For a list of all available plugins, check out
#
#     https://docs.fastlane.tools/plugins/available-plugins
#

# Uncomment the line if you want fastlane to automatically update itself
# update_fastlane

default_platform(:android)
# https://hooks.slack.com/services/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
# for tests https://hooks.slack.com/services/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

platform :android do
 before_all do
    ENV["SLACK_URL"] = "https://hooks.slack.com/services/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"
 end

  # Add this to your Fastfile
  desc "Upload the APK to Slack channel"
  private_lane :upload_to_slack do |options|
    file_path = lane_context[SharedValues::GRADLE_APK_OUTPUT_PATH]
    file_name = file_path.gsub(/\/.*\//,"")
    sh "echo Uploading " + file_name + " to Slack"
    sh "curl https://slack.com/api/files.upload -F token=\"token\" -F channels=\"#channel\" -F title=\"" + file_name + "\" -F filename=\"" + file_name + "\" -F file=@" + file_path
  end
  lane :slackbuild do
   gradle(
       task: 'clean'
     )
     gradle(
       task: 'assemble',
       build_type: 'Debug'
     )
    slack(message: "Build Successful!")
    upload_to_slack()
  end

  lane :build do
    gradle(task: "clean assembleDebug")
  end

  after_all do |lane|
    # call on success
    slack(message: "Successfully built another version of On-demand Driver")
  end
  error do |lane, exception|
    slack(
        message: exception.message,
        success: false)
  end

end

`./fastlane/Appfile`
json_key_file("") # Path to the json secret file - Follow https://docs.fastlane.tools/actions/supply/#setup to get one
package_name("co.ke.getboda.ondemand.driver.app") # e.g. com.krausefx.app

fastlane gems

Gem
Version
Update-Status

fastlane
2.184.1
✅ Up-To-Date

Loaded fastlane plugins:
No plugins Loaded
Loaded gems

Gem
Version

did_you_mean
1.4.0

atomos
0.1.3

CFPropertyList
3.0.3

claide
1.0.3

colored2
3.1.2

nanaimo
0.3.0

xcodeproj
1.19.0

rouge
2.0.7

xcpretty
0.3.0

terminal-notifier
2.0.0

unicode-display_width
1.7.0

terminal-table
1.8.0

plist
3.6.0

public_suffix
4.0.6

addressable
2.7.0

multipart-post
2.0.0

word_wrap
1.0.0

tty-screen
0.8.1

tty-cursor
0.7.1

tty-spinner
0.9.3

artifactory
3.0.15

babosa
1.0.4

colored
1.2

highline
2.0.3

commander
4.6.0

unf_ext
<IP_ADDRESS>

unf
0.1.4

domain_name
0.5.20190701

http-cookie
1.0.3

faraday-cookie_jar
0.0.7

faraday_middleware
1.0.0

gh_inspector
1.1.3

mini_magick
4.11.0

naturally
2.2.1

rubyzip
2.3.0

security
0.1.3

xcpretty-travis-formatter
1.0.1

dotenv
2.7.6

bundler
2.1.4

simctl
1.6.8

retriable
3.1.2

multi_json
1.15.0

signet
0.15.0

httpclient
2.8.3

google-apis-core
0.3.0

google-apis-playcustomapp_v1
0.3.0

google-apis-androidpublisher_v3
0.5.0

rake
13.0.3

digest-crc
0.6.3

google-cloud-storage
1.31.1

emoji_regex
3.2.2

uri
0.10.0

excon
0.82.0

ruby2_keywords
0.0.4

faraday-net_http_persistent
1.1.0

faraday-net_http
1.0.1

faraday-excon
1.1.0

faraday-em_synchrony
1.0.0

faraday-em_http
1.0.0

faraday
1.4.2

fastimage
2.2.4

json
2.5.1

jwt
2.2.3

trailblazer-option
0.1.1

declarative
0.0.20

uber
0.1.0

representable
3.1.1

mini_mime
1.1.0

os
1.1.1

memoist
0.16.2

googleauth
0.16.2

rexml
3.2.5

webrick
1.7.0

google-cloud-errors
1.1.0

google-cloud-env
1.5.0

google-cloud-core
1.6.0

google-apis-iamcredentials_v1
0.4.0

google-apis-storage_v1
0.4.0

aws-eventstream
1.1.1

aws-sigv4
1.2.3

aws-partitions
1.465.0

jmespath
1.4.0

aws-sdk-core
3.114.1

aws-sdk-kms
1.43.0

aws-sdk-s3
1.95.1

forwardable
1.3.1

logger
1.4.2

cgi
0.1.0

date
3.0.0

timeout
0.1.0

stringio
0.1.0

ipaddr
1.2.2

openssl
2.1.2

zlib
1.1.0

ostruct
0.2.0

strscan
1.0.3

io-console
0.5.6

delegate
0.1.0

fileutils
1.4.1

singleton
0.1.0

open3
0.1.0

nokogiri
1.11.1

racc
1.5.2

yaml
0.1.0

psych
3.1.0

mutex_m
0.1.0

generated on: 2021-06-04

 [REPLACE THIS WITH YOUR INFORMATION]

In my case, that problem is about permissions. if you open android studio as administrator (or using sudo) then if you open the project again as normal user, android studio may not delete some files because when you open the project as administrator, some project files permissions are changed. If you lived those steps, I think you should clone the project from your VCS (git, tfs, svn etc.) again. Otherwise you have to configure project file - folder permissions.

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest fastlane version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
Friendly reminder: contributions are always welcome! Check out CONTRIBUTING.md for more information on how to help with fastlane and feel free to tackle this issue yourself 💪
This issue will be auto-closed if there is no reply within 1 month.

I have tried updating fastlane but I keep getting the error

Have the same problem.
`Execution failed for task ':app:processDevDebugMainManifest'.

Unable to make field private final java.lang.String java.io.File.path accessible: module java.base does not "opens java.io" to unnamed module @3b705af8
`

Well, the problem seems to come from the JDK version. If you are using a java version that is not yet supported by android studio, you'll probably land on the problem. I just downgraded to java 11 and everything worked perfectly.


Fixes to prevent clang-diagnostic errors

When building projects with Visual Studio and cmake and in addition using clang-tidy we see clang-diagnostic errors in the cereal codebase during the clang-tidy execution.  The binaries are built using the msvc compiler (visual studio 16.6.0, Microsoft (R) C/C++ Optimizing Compiler Version 19.26.28805 for x64), successfully.  clang-tidy uses the llvm tool chain and thus the clang compiler (version 9.0.0).
This scenario occurs during the clang-tidy run.  clang-tidy is executed using the build commands generated by cmake, as such _MSC_VER is defined even though the clang compiler is being utilised.
The following diagnostics are generated during the clang-tidy run:
C:\Src****************\out\build\Windows-x64-Debug_deps\cereal-src\package\include\cereal\details\static_object.hpp(67,29): error G70D0A859: 'cereal::detail::StaticObject<cereal::detail::bind_to_archives<datapath::udp_stream::send_session_configuration, cereal::detail::(anonymous namespace)::polymorphic_binding_tag> >' must have external linkage when declared 'dllexport' [clang-diagnostic-error]
class CEREAL_DLL_EXPORT StaticObject
^
And...
C:\Src******************\out\build\Windows-x64-Debug_deps\cereal-src\package\include\cereal\details\polymorphic_impl.hpp(658,91): error GD8FB81D5: unexpected type name 'type': expected expression [clang-diagnostic-error]
savePolymorphicSharedPtr( ar, ptr, ::cereal::traits::has_shared_from_this::type() ); // MSVC doesn't like typename here
......
A guard is required to prevent the un-versioned _MSC_VER test from being taken when compiling with clang under these circumstances.  The clang compiler identifies itself using the clang pre-defined macro which can be used to provide a guard.

This would also fix:

https://github.com/USCiLab/cereal/issues/652
https://github.com/USCiLab/cereal/issues/582


Clarify that repo download if for reference and later usage

Clarify that repo download if for reference/later usage

Thanks @jjasghar !!


[ISSUE #172][ISSUE #174]Modify the File Source Example; Update README; Fix incompatible types bug

Add a filter usage example and modify the document.

Codecov Report

Merging #173 (77db264) into main (dcd0c77) will not change coverage.
The diff coverage is 0.00%.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f8f755e...77db264. Read the comment docs.

#174 fix incompatible types bug

https://github.com/apache/rocketmq-streams/issues/172

@hyl-xidian can you remove the fix bug part? The others looks fine to me.

@hyl-xidian can you remove the fix bug part? The others looks fine to me.

Got it, I have removed the fix bug part.


tests: vkUpdateDescriptorSet parameter validation

Add a positive test to ensure that the parameter_validation layer ignores
vkUpdateDescriptorSet parameters that are not related to the type of
descriptor set being updated.

:+1:

This is a good case and useful beyond just parameter_checker.
Could be enhanced to include *TEXEL_BUFFER and *BUFFER updates that validate that verify that other struct members are properly ignored.

@tobine I have updated this test to include TEXEL_BUFFER and BUFFER updates.


Connection Limit Exceeded and CPU Burnout Issues

Bug description
I hope this message finds you well. As part of the critical project team, I am reporting two urgent issues that require immediate attention.
Connection Limit Exceeded:
Our application has encountered instances where the connection limit has been exceeded, significantly impacting the user experience. It is crucial to address this matter urgently to ensure the seamless operation of the system.
CPU Burnout:
Additionally, our server is experiencing CPU burnout, leading to performance degradation and potential downtime. We must promptly resolve this problem to ensure the stability and reliability of our project.
As we prepare for significant growth, it is essential to address these challenges to maintain a smooth user experience and ensure the overall success of the project. I will create detailed issue reports for each problem and keep everyone informed about the progress.
Technical Details:
Our project utilizes Prisma ORM with a GraphQL API and is built on Node.js/TypeScript. We are running on GCP-k8 with six pods, each having 24 GB of RAM and 8 CPU. After careful consideration, we have set a connection pool limit of 16 with a 20-second timeout. Despite these configurations, we are still facing connection pool limit exceedance, leading to unresponsive queries and extended response times.
Additionally, we have noticed instances where the Database CPU experiences burnout, negatively affecting overall performance. We have attached graphs for reference.
In the past, we encountered heap memory burning issues, which improved after removing the promise.all function. However, there is still room for further stability enhancement.
We highly appreciate your immediate attention to these issues to ensure the smooth functioning of our project. Your support is vital as we work towards resolving these challenges.

How to reproduce
NA
Expected behavior

The application should be able to handle an increasing number of users without encountering connection limit issues or experiencing CPU burnout.

Prisma information
schema.prisma
  provider        = "prisma-client-js"
  previewFeatures = ["metrics"]
}

datasource db {
  provider = "mysql"
  url      = env("DATABASE_URL")
}

enum enumName {}

model modelName {}

...
...
...

model modelTwoName {}

utils.ts
import { PrismaClient } from '@prisma/client';

export interface Context {
  user: () => Promise<User | null | undefined>
  prisma: PrismaClient
  request: any
}

...
...
...

genre.ts
import { Context } from '../../utils'
import { GraphQLResolveInfo } from 'graphql'
import { genreReturnType } from '../ReturnType'
import { errorName } from '../../error/error'

export const genreMethods = {
    query: {
        async genre(_parent, args, ctx: Context, _info: GraphQLResolveInfo) {
            try {
                const item = await ctx.prisma.genre.findUnique({ ...args, include: genreReturnType })
                return item
            } catch (err) {
                console.log(err)
                throw err
            }
        },
    }
}

export default genreMethods

Environment & setup

Docker Image: node:16-alpine
Database: MySQL/CloudSQL
Node.js version: 16.16.0

Prisma Version
4.1.0

Hi @kashif-trailfive, I noticed you are using a relatively old version, 4.1.0 is now 1 year old.
Did you consider upgrading, maybe? (Are they any blockers to upgrade the Prisma version?)
On the performance side, we introduced jsonProtocol in 4.11.0 and made it generally available in 5.0.0, many users sent us positive feedback, though this is not about database performance.
I also noticed that you top SELECT query return on average 10k rows.
You could try running these queries with EXPLAIN and see what is slow and maybe discover a missing index https://dev.mysql.com/doc/refman/8.0/en/explain.html

Some questions for clarification:

Connection Limit Exceeded:
Our application has encountered instances where the connection limit has been exceeded, significantly impacting the user experience. It is crucial to address this matter urgently to ensure the seamless operation of the system.

What kind of connection limit exactly? From Prisma or your database? What error message exactly did you get?

CPU Burnout:
Additionally, our server is experiencing CPU burnout, leading to performance degradation and potential downtime. We must promptly resolve this problem to ensure the stability and reliability of our project.

Do I understand correctly that you have 1 database server, and that the available resources on that one are being fully utilized while you are running your application? Is that normal traffic or testing?

Hi,
Thank you for your responses and your deep analysis. We will diligently follow all the guidelines. Below are our answers to some questions.
Regarding the Prisma version, it has already been deployed on staging with the latest 5.x version. However, we need to thoroughly test it, as the Apollo Server version also requires an upgrade. We are upgrading it slowly to ensure backward compatibility. The next upgrade will be to the latest version.
Yes, this is normal production traffic.

Is this normal production traffic or some load testing?

Yes , The CPU/Memory usage on pods is generally normal and under the limit. However, sometimes when the connection pool limit is exceeded, servers/pods keep running, but queries stop responding. To address this, we have implemented a script that detects such issues and automatically restarts them. It may sound funny, but this patch was necessary to ensure minimal disruption and keep our users' experience uninterrupted.

All the graphs you shared are from the database, correct? What does CPU look like on the pods/servers?

The connection limit issue arises from Prisma. The error message we received is related to exceeding the connection pool limit. Additionally, when the database CPU is overloaded, users may experience unexpected logouts.

What kind of connection limit exactly? From Prisma or your database? What error message exactly did you get?

Our server can handle up to 10,000 connections, and this limit has never been reached or exceeded.

CPU Burnout:
Additionally, our server is experiencing CPU burnout, leading to performance degradation and potential downtime. We must promptly resolve this problem to ensure the stability and reliability of our project.

Do I understand correctly that you have 1 database server, and that the available resources on that one are being fully utilized while you are running your application? How many connections and queries should that database server be able to handle?

Our project utilizes Prisma ORM with a GraphQL API and is built on Node.js/TypeScript. We are running on GCP-k8 with six pods, each having 24 GB of RAM and 8 CPU. After careful consideration, we have set a connection pool limit of 16 with a 20-second timeout.

Yes, that's correct. The number of connections is calculated using the formula: Number of CPU * 2 + 1.

So you are running 6 "servers" that are each running your application? The connection pool of Prisma is set to 16 connections (with 20 second timeout) on each of those "servers"?

10000 connections

What does the database set up and configuration look like? What CPU and memory? How many connections does your database server allow at a time?

Despite these configurations, we are still facing connection pool limit exceedance, leading to unresponsive queries and extended response times.

From Prisma, we are getting this error
Error: Timed out fetching a new connection from the connection pool. (More info: http://pris.ly/d/connection-pool, Current connection limit: 16)

time to time , even without database CPU 100%

What exactly do you mean by that? The same "connection limit exceeded" or something else? What is the exact error message you are getting?

Additionally, we have noticed instances where the Database CPU experiences burnout, negatively affecting overall performance. We have attached graphs for reference.

We can disregard this issue for now since it was related to the heap memory crash, which occurred due to the use of promise.all(). While addressing the connection timeout issue, we replaced promise.all() with a loop to fetch results, which also improved the heap memory situation to some extent. However, we are now facing a separate concern with the CPU reaching 100% utilization. It might be related or unrelated; we are not entirely sure at the moment.

Is this the same "CPU Burnout" described above or something else?

In the past, we encountered heap memory burning issues, which improved after removing the promise.all function. However, there is still room for further stability enhancement.

We were unsure of the exact number of promises/queries executed in the past, especially with the sudden increase in user traffic. As a result, we might need to conduct a reindexing of the database or explore other optimization measures to address the performance issues. Currently, we are closely monitoring the system and implementing scripts to automatically restart the servers/pods when connection pool limit exceedance occurs, which helps to maintain smoother operation during high traffic periods.

How many promised/queries were you trying to executed in the past? How are you running them now instead?

Yes, we believed that Prisma has indexed the queries. Currently, we are not fetching millions of results, but the small query execution is still taking an excessive amount of time. Moreover, we doubted that when one query is in execution, another one comes in and congests the system, leading to performance issues. We are unsure whether this is a code-related problem or another factor affecting the performance.

If I read this table correctly, you have a small number of queries that produce most of the load on your system. They all run for a full second or (a lot!) longer. They are frequently called, and return a lot of rows.

Do you know which code triggers these queries?
Yes, we are aware of the code that triggers these queries. As GraphQL has the N+1 problem, we are actively working on optimization efforts to mitigate its impact. Despite the database size not being in the millions, we are still facing performance issues, which suggests that there might be some inefficient code parts causing these challenges.
Do these queries have appropriate indexes defined or do they really have to be that slow?
Yes, we are actively checking the indexes. As indicated in the image, the tables with the prefix '_tablename' are indexed tables, and Prisma handles the indexing. However, we are currently investigating whether these indexes are functioning optimally, as certain indexing algorithms may not work effectively with MySQL.
Can you share the relevant schema, Prisma Client query and then the full generated SQL query for any query >1s?
model User {
 ...
 favoriteGenres   Genre[]     @relation(name: "UserFavoriteGenre")
 ...
}
model Genre {
 ...
 users   User[]    @relation(name: "UserFavoriteGenre")
 ...
}

Query:  {
  query: 'query User($where: UserWhereUniqueInput!) {\n' +
    '  user(where: $where) {\n' +
    '    id\n' +
    '    name\n' +
    '    email\n' +
    '    favoriteGenres {\n' +
    '      id\n' +
    '      name\n' +
    '      projects {\n' +
    '        id\n' +
    '        name\n' +
    '      }\n' +
    '    }\n' +
    '  }\n' +
    '}',
  variables: { where: { id: 'cl74mncks00041q22520h2da7' } },
  operationName: 'User'
}
Query: SELECT `prisma-prod2`.`User`.`id`, `prisma-prod2`.`User`.`updatedAt`, `prisma-prod2`.`User`.`createdAt`, `prisma-prod2`.`User`.`email`, `prisma-prod2`.`User`.`name`, `prisma-prod2`.`User`.`status`, `prisma-prod2`.`User`.`artistName`, `prisma-prod2`.`User`.`password`, `prisma-prod2`.`User`.`secretHash`, `prisma-prod2`.`User`.`role`, `prisma-prod2`.`User`.`facebookId`, `prisma-prod2`.`User`.`googleId`, `prisma-prod2`.`User`.`korgId`, `prisma-prod2`.`User`.`avatarUrl`, `prisma-prod2`.`User`.`bio`, `prisma-prod2`.`User`.`headerImage`, `prisma-prod2`.`User`.`emailConfirmToken`, `prisma-prod2`.`User`.`emailConfirmExpires`, `prisma-prod2`.`User`.`joinedAt`, `prisma-prod2`.`User`.`inviteAccepted`, `prisma-prod2`.`User`.`emailConfirmed`, `prisma-prod2`.`User`.`deletedAt`, `prisma-prod2`.`User`.`lastLogin`, `prisma-prod2`.`User`.`lastLoginIp`, `prisma-prod2`.`User`.`isSuper`, `prisma-prod2`.`User`.`resetToken`, `prisma-prod2`.`User`.`resetExpires`, `prisma-prod2`.`User`.`magicLinkToken`, `prisma-prod2`.`User`.`magicLinkExpires`, `prisma-prod2`.`User`.`age`, `prisma-prod2`.`User`.`gender`, `prisma-prod2`.`User`.`musicTaste`, `prisma-prod2`.`User`.`typeOfMusician`, `prisma-prod2`.`User`.`recordingsCount`, `prisma-prod2`.`User`.`timelineItemsCount`, `prisma-prod2`.`User`.`relatedTimelineItemsCount`, `prisma-prod2`.`User`.`followsCount`, `prisma-prod2`.`User`.`followedByCount`, `prisma-prod2`.`User`.`projectsCount`, `prisma-prod2`.`User`.`tracksOwnedCount`, `prisma-prod2`.`User`.`devicesCount`, `prisma-prod2`.`User`.`notificationsCount`, `prisma-prod2`.`User`.`clapsCount`, `prisma-prod2`.`User`.`commentsCount`, `prisma-prod2`.`User`.`allProjectsCount`, `prisma-prod2`.`User`.`isDeveloper`, `prisma-prod2`.`User`.`dummy`, `prisma-prod2`.`User`.`isValidated`, `prisma-prod2`.`User`.`validatedAt`, `prisma-prod2`.`User`.`fileId`, `prisma-prod2`.`User`.`userId`, `prisma-prod2`.`User`.`phoneNumber`, `prisma-prod2`.`User`.`phoneNumberConfirmed`, `prisma-prod2`.`User`.`likedId`, `prisma-prod2`.`User`.`deactivateReason`, `prisma-prod2`.`User`.`isDeactivated`, `prisma-prod2`.`User`.`fcm_token`, `prisma-prod2`.`User`.`facebookLink`, `prisma-prod2`.`User`.`instagramLink`, `prisma-prod2`.`User`.`location`, `prisma-prod2`.`User`.`soundCloudLink`, `prisma-prod2`.`User`.`clappedProjectsCount`, `prisma-prod2`.`User`.`deleteAfter` FROM `prisma-prod2`.`User` WHERE `prisma-prod2`.`User`.`id` = ? LIMIT ? OFFSET ?
Params: ["cl74mncks00041q22520h2da7",1,0]
Duration: 158ms
Query: SELECT `prisma-prod2`.`_UsersClappedProject`.`B`, `prisma-prod2`.`_UsersClappedProject`.`A` FROM `prisma-prod2`.`_UsersClappedProject` WHERE `prisma-prod2`.`_UsersClappedProject`.`B` IN (?)
Params: ["cl74mncks00041q22520h2da7"]
Duration: 418ms
Query: SELECT `prisma-prod2`.`_MusicalInstrumentUser`.`B`, `prisma-prod2`.`_MusicalInstrumentUser`.`A` FROM `prisma-prod2`.`_MusicalInstrumentUser` WHERE `prisma-prod2`.`_MusicalInstrumentUser`.`B` IN (?)
Params: ["cl74mncks00041q22520h2da7"]
Duration: 364ms
Query: SELECT `prisma-prod2`.`_UserFavoriteGenre`.`B`, `prisma-prod2`.`_UserFavoriteGenre`.`A` FROM `prisma-prod2`.`_UserFavoriteGenre` WHERE `prisma-prod2`.`_UserFavoriteGenre`.`B` IN (?)
Params: ["cl74mncks00041q22520h2da7"]
Duration: 457ms
Query: SELECT `prisma-prod2`.`Genre`.`id`, `prisma-prod2`.`Genre`.`name`, `prisma-prod2`.`Genre`.`createdAt`, `prisma-prod2`.`Genre`.`updatedAt`, `prisma-prod2`.`Genre`.`description`, `prisma-prod2`.`Genre`.`rank` FROM `prisma-prod2`.`Genre` WHERE `prisma-prod2`.`Genre`.`id` IN (?,?,?,?)
Params: ["cl82wppz4125841p1swti93jdp","clbuyambj0015zk1nqz6anopz","clbuyambj0019zk1nen7vwej0","clbw67x0700090v1nb6gy6jyj"]
Duration: 406ms
Query: SELECT `prisma-prod2`.`Geolocation`.`id`, `prisma-prod2`.`Geolocation`.`createdAt`, `prisma-prod2`.`Geolocation`.`updatedAt`, `prisma-prod2`.`Geolocation`.`userId`, `prisma-prod2`.`Geolocation`.`latitude`, `prisma-prod2`.`Geolocation`.`longitude` FROM `prisma-prod2`.`Geolocation` WHERE `prisma-prod2`.`Geolocation`.`userId` IN (?)
Params: ["cl74mncks00041q22520h2da7"]
Duration: 411ms

It appears that the problem might be related to the IN part of the query, but we are not entirely certain at this point.
Do they really need to return that many rows?
This is indeed a valid question. The frontend team is responsible for deciding how much data they need to fetch from the endpoint. In GraphQL, when they include more data points with 'include,' it generates additional SQL statements in the final query. As a result, the number of rows returned can increase significantly based on their requirements.
Could these query results maybe be cached, even in a local variable?
Yes, we attempted to maintain a local cache, but due to the nature of the project data, the cache kept increasing in size. Nonetheless, we successfully implemented the cache with the help of Apollo Server, and we believe it is functioning effectively.

Do you know which code triggers these queries? Do these queries have appropriate indexes defined or do they really have to be that slow? Can you share the relevant schema, Prisma Client query and then the full generated SQL query for any query >1s? Do they really need to return that many rows? Could these query results maybe be cached, even in a local variable?

Note: A colleague mentioned that you could check Query Insights to see the CPU consumption of your queries: https://cloud.google.com/sql/docs/mysql/using-query-insights#specific-query

Note: A colleague mentioned that you could check Query Insights to see the CPU consumption of a specific query: https://cloud.google.com/sql/docs/mysql/using-query-insights#specific-query

Yes, we have identified two queries causing problems. It's challenging to locate them in the code as the ORM is generating the queries, and the frontend is handling data inclusion. Therefore, we are extensively working to find the bottleneck. Here is the screenshot from our Query insight.

Some POST method latency as well:

@kashif-trailfive Your comment https://github.com/prisma/prisma/issues/20272#issuecomment-1641681226 was formatted a bit confusingly for me (you put the responses first, and then the quote what you were responding to), so I quickly went through your post and moved that around (I did not delete any text). Please double check if I got it all correct. I will follow up later.

@janpio  Thanks for your reply, it's very helpful!

If I was you @kashif-trailfive, I would do 3 things now:

We have an endpoint called 'AllProjects' that also retrieves information about users associated with each project, so many to many. When the frontend calls this endpoint with 20 projects, it generates 66 underlying queries to retrieve users(right now), which involve many-to-many joins with 'genre' and 'musical instrument' tables. As a result, so if you see making 5000 calls to the 'AllProjects' endpoint in an hour leads to 64000-74000 calls to the 'users' endpoint, and this number increases proportionally for further queries. The reason for this behavior is due to the use of GraphQL Prisma API 'include' feature. The more data we include, the more queries are built up. We are currently optimizing this process to reduce the query load as much as possible.

Really focus on understanding the top queries (or even just start with the top one) and make them/it disappear from that list. I can't think of any reason why a query would need to return avg >10.000 results and take avg 12 seconds to do that- and still be useful in an app.
Can you share the full query from the top queries?
Usually, it is possible to go back to understand what Prisma Client query would create this SQL query. From the leading underscore of the table _UserFavoriteGenre I can already know that this is part of a many-to-many relation of Prisma for example, indeed the one between User and Genre you shared above. Why would such a query ever return >10.000 results? Do you know where you run a Prisma Client query that would query that relation?

Now, this is interesting. We are not entirely sure what will happen if we increase the connection pool size, as there is no clear documentation available on the topic. Currently, we have set the maximum connections to 1024, and it seems to be working fine. However, I would like to understand what would be an optimal size for a large pool of connections. Can we use Prisma DataProxy with a long-running server instead of a serverless setup, and would it be beneficial for performance?

Your database connections are bored. CPU and memory of your pods also seem to be bored. On the other hand you are getting connection pool timeouts. That error indicates a capacity and saturation problem - in that the queries an application tries to perform, can not be performed in the available time (the pool timeout) - so you get an error. This happens very quickly if you have some queries that are very slow, as they "hog" connections for a long time and then the queue gets to long - and the error explodes in your face.
What happens if you double or triple the connection limit of each pod? If CPU and memory are fine, you should be able to use a lot more of those available 10.000 connections from your database! (Unfortunately the slow queries will probably still take over all connections in the connection pool sooner or later. So you might need to increase further, or do the following:

max_execution_time is seems to be very helpful. Let me share an interesting thing with you: we had a conversation with the Google team, and they mentioned that we have all options available in MYSQL. However, we discovered that query caching was only accessible for Enterprise Plus users. Despite having an Enterprise Plus version (we have Enterprise) we decided to update our database version from 8.26 to 8.35, and to our surprise, everything started functioning smoothly. The CPU usage dropped significantly to just 0.5%, and the query response times improved to milliseconds. Honestly, I can't pinpoint the exact reason for this improvement, but we should investigate further to understand the underlying cause. It's essential to figure it out, as any potential issue could arise again in the future.
You see, 5000 project queries triggered 79000 user queries. We have also requested our FE team to analyze this in detail. Additionally, we are currently working on implementing New Relic for our backend with Kubernetes (k8s).

 That could ensur that no queries can hog any connection to long (MySQL has a setting max_execution_time). Not that this will of course make parts of your application explode - but then you can figure out where exactly that is and why these queries are so slow.

So to recap, in our opinion you have some bad database queries that kill your database CPU, and also block the Prisma connection pool and lead to timeouts there.

We have an endpoint called 'AllProjects' that also retrieves information about users associated with each project, so many to many. When the frontend calls this endpoint with 20 projects, it generates 66 underlying queries to retrieve users(right now), which involve many-to-many joins with 'genre' and 'musical instrument' tables. As a result, so if you see making 5000 calls to the 'AllProjects' endpoint in an hour leads to 64000-74000 calls to the 'users' endpoint, and this number increases proportionally for further queries. The reason for this behavior is due to the use of GraphQL Prisma API 'include' feature. The more data we include, the more queries are built up. We are currently optimizing this process to reduce the query load as much as possible.

Less queries with less data returned is always better, so really make sure you need all that data (also all the fields being returned, not just the rows) and then make sure all the queries really are fast by using indexes. Any query you can avoid, can not cause a problem. Any query that needs to return less, will allow you servers to handle more requests.

This new screenshot of the queries being executed already looks much better. The throughput of queries in the connection pool should be seriously unblocked with that, as they are all much faster than before.

Really focus on understanding the top queries (or even just start with the top one) and make them/it disappear from that list. I can't think of any reason why a query would need to return avg >10.000 results and take avg 12 seconds to do that- and still be useful in an app.
Can you share the full query from the top queries?
Usually, it is possible to go back to understand what Prisma Client query would create this SQL query. From the leading underscore of the table _UserFavoriteGenre I can already know that this is part of a many-to-many relation of Prisma for example, indeed the one between User and Genre you shared above. Why would such a query ever return >10.000 results? Do you know where you run a Prisma Client query that would query that relation?

Now, this is interesting. We are not entirely sure what will happen if we increase the connection pool size, as there is no clear documentation available on the topic. Currently, we have set the maximum connections to 1024, and it seems to be working fine. However, I would like to understand what would be an optimal size for a large pool of connections.

That is one of the big questions of database usage - and it depends on a million different considerations.
Generally, we recommend we start with a pretty low number (hence our really conservative default), and only play with that if the traffic volume you get can not be handled - as is the case for you! Then you can increase it step by step and observer what that does to your app. If you have a bigger pool, that means Prisma can open and then use more connections at a time. This comes at a cost - opening a connection is slow, and managing them takes CPU and memory. So you have to watch the resource usage of your pods. A higher connection pool liit/size should lead to more resources being used. But when they are open, you have more "pipes" through which you can execute queries, which means that a "connection pool timeout" is less probable.

Can we use Prisma DataProxy with a long-running server instead of a serverless setup, and would it be beneficial for performance?

You can absolutely use it with a long running server as well. As your database can handle a lot of connections though, and your application servers also seem to be pretty happy and underutilized, I would not necessarily recommend that route right now yet. I think you can get this under control with easier measures.

3. Your database connections are bored. CPU and memory of your pods also seem to be bored. On the other hand you are getting connection pool timeouts. That error indicates a capacity and saturation problem - in that the queries an application tries to perform, can not be performed in the available time (the pool timeout) - so you get an error. This happens very quickly if you have some queries that are very slow, as they "hog" connections for a long time and then the queue gets to long - and the error explodes in your face.
What happens if you double or triple the connection limit of each pod? If CPU and memory are fine, you should be able to use a lot more of those available 10.000 connections from your database! (Unfortunately the slow queries will probably still take over all connections in the connection pool sooner or later. So you might need to increase further, or do the following:

max_execution_time is seems to be very helpful.

It's one of those things that can really protect your from some bad queries (as you had them in the overview before) killing your whole application. It's like a kill switch that prevents bad queries from causing too much trouble. (Unless of course these queries are the core functionality of your app - but then you are motivated to fix that quickly as well 😆 )

Let me share an interesting thing with you: we had a conversation with the Google team, and they mentioned that we have all options available in MYSQL. However, we discovered that query caching was only accessible for Enterprise Plus users. Despite having an Enterprise Plus version (we have Enterprise) we decided to update our database version from 8.26 to 8.35, and to our surprise, everything started functioning smoothly. The CPU usage dropped significantly to just 0.5%, and the query response times improved to milliseconds. Honestly, I can't pinpoint the exact reason for this improvement, but we should investigate further to understand the underlying cause. It's essential to figure it out, as any potential issue could arise again in the future.

Wow, that is pretty amazing. We would be really interested to hear from you when/if you figure out what is/was going on there with 8.26. Mayyyybe some index previously was not used accidentally, and this was fixed - that would be the most simple explanation really. For now I am just happy for you! 🥳

You see, 5000 project queries triggered 79000 user queries. We have also requested our FE team to analyze this in detail. Additionally, we are currently working on implementing New Relic for our backend with Kubernetes (k8s).

Both very good ideas.
(From my own experience: Be careful with New Relic, using an APM like that is usually so impressive that you will want to use it on every project in the future and optimize all the things it points out 😄. Flipside is that the cost can be pretty heavy sometimes.)

Thank you for your suggestion, which was very helpful. Yes, I think the table index somehow got blocked or encountered an issue, but now it's back in action. We will investigate the indexing in more detail, as we are currently relying on manual indexes in Prisma.

@janpio We are currently facing a heap memory leak at a very minute level (as well). After running in production for 20 or more hours, our pods keep restarting. Any idea what might be causing this? We have already addressed some areas in the code where we suspect Promise.all could be causing issues in old space. If you have any quick suggestions or specific areas to look into, we would greatly appreciate it.

Unfortunately nothing directly comes to mind, there are far to many reasons why this could be happening - connected to Prisma or not.
Back to the original problem, do I understand correctly that the situations is now generally under control? The database CPU is not eaten up any more, and the connection pool can now also handle the load you are putting at it?

@janpio Thank you for the update. Yes, I understand that there might not be a straightforward solution for this heap issue, considering the various possible reasons behind it.  It seems that one of the causes could be query overloading with data to fetch.
Regarding the original problem, The situation is now generally under control. It's a positive outcome that the database CPU is no longer overwhelmed, and the connection pool is effectively handling the workload.
With the successful resolution of the problem, I am going to close this issue with thanks :)

Thanks for the update.
If there are any other things you think Prisma is involved or influencing, please let us know. It was a pleasure to debug this together with your from the distance, and as you provided all the information you had (including diagrams and logs) we could really dive deep here and get to real conclusions. Thanks, and all the best!


Learning pattern

There is a learning pattern associated with the nav drawer nature, as it is something people don't see (and some don't know it's there) until it's open. Basically what Google recommends is to open the nav drawer when starting your app everytime until user himself / herself firstly tap / swipe to open the drawer = they finally got it how it works and that it's there. It seems like pretty obvious to us, but there are some users that must be taught to do that (unfortunatelly).
If you generate a new project in Android Studio with a drawer pattern, it is already implemented there, so check it out - just one SharedPref variable, that's it ;)

ok so I will check what code are used with sample project.

In the sample code the drawer will open when the instance state is null and the user has not learned the drawer

        // If the user hasn't 'learned' about the drawer, open it to introduce them to the drawer,
        // per the navigation drawer design guidelines.
        if (!mUserLearnedDrawer && !mFromSavedInstanceState) {
            mDrawerLayout.openDrawer(mFragmentContainerView);
        }

So my code is an exactly reproduction, but I use a static variable instead passing it through a bundle.


Overlay support (and other cleanup)

Reduce docker image size by cleaning apt
Use noninteractive mode and no-install-recommends to reduce noisy output when building
Use ENTRYPOINT instead of CMD

Allow commands to be run without specifying wrapdocker every time

Exit early if the docker daemon doesn't come up before the timeout.
Bash style cleanup

2 space indent (instead of mixed 4 spaces and tabs)
Inline then/do
Remove trailing line whitespace
Wrap simple variable usages

Cleanly shutdown docker daemon on exit to avoid resource leaks
Add support for OverlayFS (which allows recursive mounting)
Fallback to AUFS when OverlayFS is not available or when not configured on the host docker
Remove /var/lib/docker volume
Use loop device with AUFS to allow AUFS on AUFS
Update README
Use docker daemon instead of deprecated docker -d (>= v1.8)

Sorry to put so many changes in one PR, but otherwise it would have been a chain of branches of branches to deal with conflict. I used multiple commits to make it easier to review. Let me know if any of these changes are unacceptable and need to be removed to accept the others.

FWIW, these changes are largely from mesos-slave-dind (without the networking changes).
Credit goes to @sttts for the initial OverlayFS support.

When running it on Manjaro-Linux I get this:
ln: failed to create symbolic link '/sys/fs/cgroup/systemd/name=systemd': Read-only file system

What can I do?

@webwurst What command did you use to run the dind container?

Nothing special:
$ docker build -t karlkfi/dind .
$ docker run --privileged -t -i karlkfi/dind
ln: failed to create symbolic link '/sys/fs/cgroup/systemd/name=systemd': Operation not permitted

Haven't tried on Manjaro-Linux yet. I'll try to check that out in a vm.
On my Mac, with boot2docker (v1.8.1, aufs, kernel 4.0.9) and docker (v1.8.1) I get the following:
$ docker run --privileged -t -i karlkfi/dind
0+0 records in
0+0 records out
0 bytes (0 B) copied, 6.4819e-05 s, 0.0 kB/s
mke2fs 1.42.9 (4-Feb-2014)
/data/docker is not a block special device.
Proceed anyway? (y,n) Discarding device blocks: done
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
327680 inodes, 1310720 blocks
65536 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1342177280
40 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
	32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

Interestingly, I see your same error in https://github.com/tehranian/dind-jenkins-slave/issues/4, https://github.com/totem/docker-image-factory/issues/5, and https://github.com/docker/docker/issues/14109
One notes that the solution was to change the /var/run/docker.sock ownership/permissions, but that doesn't sound right to me...
Does the master:HEAD dind work for you?
What version of docker are you using?

Try the new version.

Just a quick test to give feedback. I am still seeing this error, but it seems to work anyway! Could run docker run alpine echo "dind" in this:
$ docker run --privileged -t -i karlkfi/dind
ln: failed to create symbolic link '/sys/fs/cgroup/systemd/name=systemd': Operation not permitted
Storage Filesystem: overlay
Host Storage Filesystem: overlay
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
INFO[0000] Option DefaultDriver: bridge
INFO[0000] Option DefaultNetwork: bridge
WARN[0000] Running modprobe bridge nf_nat br_netfilter failed with message: modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.0.1-1-user-ns-enabled/modules.dep.bin'
modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.0.1-1-user-ns-enabled/modules.dep.bin'
modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.0.1-1-user-ns-enabled/modules.dep.bin'
, error: exit status 1
WARN[0000] Your kernel does not support swap memory limit.
INFO[0000] Loading containers: start.

INFO[0000] Loading containers: done.
INFO[0000] Daemon has completed initialization
INFO[0000] Docker daemon                                 commit=d12ea79 execdriver=native-0.2 graphdriver=overlay version=1.8.1
INFO[0001] GET /v1.20/info
root@becaeeeb0755:/#

@webwurst Does the master:HEAD dind give you that error too?

Sorry for the delay. Here both outputs to compare:
$ docker build --rm -t jpetazzo/dind .
$ docker run --privileged -t -i jpetazzo/dind
ln: failed to create symbolic link '/sys/fs/cgroup/systemd/name=systemd': Operation not permitted
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
INFO[0000] [graphdriver] using prior storage driver "overlay"
WARN[0000] Running modprobe bridge nf_nat failed with message: , error: exit status 1
WARN[0000] Your kernel does not support swap memory limit.
INFO[0000] Loading containers: start.

INFO[0000] Loading containers: done.
INFO[0000] Daemon has completed initialization
INFO[0000] Docker daemon                                 commit=786b29d execdriver=native-0.2 graphdriver=overlay version=1.7.1
INFO[0001] GET /v1.19/info
root@244136ff09af:/# docker run alpine echo "dind"
INFO[0024] POST /v1.19/containers/create
ERRO[0024] Handler for POST /containers/create returned error: No such image: alpine (tag: latest)
ERRO[0024] HTTP Error                                    err=No such image: alpine (tag: latest) statusCode=404
Unable to find image 'alpine:latest' locally
INFO[0024] POST /v1.19/images/create?fromImage=alpine&tag=latest
INFO[0027] Image manifest for alpine:latest has been verified
latest: Pulling from alpine
31f630c65071: Already exists
alpine:latest: The image you are pulling has been verified. Important: image verification is a tech preview feature and should not be relied on to provide security.
Digest: sha256:c471fce1d08618adf4c6c0d72c047b9f3d5ef82cae0ca9a157ce1c800d42722f
Status: Downloaded newer image for alpine:latest
INFO[0029] POST /v1.19/containers/create
INFO[0029] POST /v1.19/containers/f252a591a6e6de0c1dff6b9978226639cf12e80c1df8daa8465e5b3a9a132dec/attach?stderr=1&stdout=1&stream=1
INFO[0029] POST /v1.19/containers/f252a591a6e6de0c1dff6b9978226639cf12e80c1df8daa8465e5b3a9a132dec/start
dind
INFO[0029] POST /v1.19/containers/f252a591a6e6de0c1dff6b9978226639cf12e80c1df8daa8465e5b3a9a132dec/wait

$ docker build --rm -t karlkfi/dind .
$ docker run --privileged -t -i karlkfi/dind
ln: failed to create symbolic link '/sys/fs/cgroup/systemd/name=systemd': Operation not permitted
Storage Filesystem: overlay
Host Storage Filesystem: overlay
INFO[0000] Option DefaultDriver: bridge
INFO[0000] Option DefaultNetwork: bridge
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
WARN[0000] Running modprobe bridge nf_nat br_netfilter failed with message: modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.0.1-1-user-ns-enabled/modules.dep.bin'
modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.0.1-1-user-ns-enabled/modules.dep.bin'
modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.0.1-1-user-ns-enabled/modules.dep.bin'
, error: exit status 1
WARN[0000] Your kernel does not support swap memory limit.
INFO[0000] Loading containers: start.

INFO[0000] Loading containers: done.
INFO[0000] Daemon has completed initialization
INFO[0000] Docker daemon                                 commit=d12ea79 execdriver=native-0.2 graphdriver=overlay version=1.8.1
INFO[0001] GET /v1.20/info
root@2f877e0cdd06:/# docker run alpine echo "dind"
INFO[0025] POST /v1.20/containers/create
ERRO[0025] Handler for POST /containers/create returned error: No such image: alpine (tag: latest)
ERRO[0025] HTTP Error                                    err=No such image: alpine (tag: latest) statusCode=404
Unable to find image 'alpine:latest' locally
INFO[0025] POST /v1.20/images/create?fromImage=alpine&tag=latest
INFO[0028] Image manifest for alpine:latest has been verified
latest: Pulling from library/alpine
31f630c65071: Already exists
library/alpine:latest: The image you are pulling has been verified. Important: image verification is a tech preview feature and should not be relied on to provide security.
Digest: sha256:c471fce1d08618adf4c6c0d72c047b9f3d5ef82cae0ca9a157ce1c800d42722f
Status: Downloaded newer image for alpine:latest
INFO[0031] POST /v1.20/containers/create
INFO[0031] POST /v1.20/containers/8aba5782201813fe05f371a8199460e5d2342082d4ccac1b4c469329446db6b6/attach?stderr=1&stdout=1&stream=1
INFO[0031] POST /v1.20/containers/8aba5782201813fe05f371a8199460e5d2342082d4ccac1b4c469329446db6b6/start
dind
INFO[0031] POST /v1.20/containers/8aba5782201813fe05f371a8199460e5d2342082d4ccac1b4c469329446db6b6/wait

@webwurst Looks like it works to me.Are you still having problems?
We're both getting errors about moddep that seems to be coming from docker daemon, but it still seems to be working. FWIW that error gets mentioned a lot in this docker thread: https://github.com/docker/docker/issues/15498

Yes works for me since your last change. Thx a lot!

@jpetazzo You wanna take a look, now that someone else has confirmed it works?

So, after discussing with @tianon and testing the new official docker image, I think it's the best place to concentrate efforts, since it is directly sync'ed upstream and actively maintained.
I invite you to test again with docker:dind and report upstream any issue you might have!
Thank you!

There's one critical difference between this docker image and the one you linked, @jpetazzo. This one is extensible to run the docker daemon in the background and some other app/script in the foreground. The official image only runs seems to run the docker daemon. It makes it really hard to use the official docker:dind as a base-image.
Running something else alongside the daemon is critical, because otherwise the volume mount paths don't match.

@jpetazzo So is the suggestion that dind development be discontinued here in favor of the docker repo?  For those of us who actually need dind (as opposed to being able to get by with mounting the docker port), @karlkfi's work promised to eliminate the failure modes associated with loopback consumption.  It would be good to see that get the attention it deserves.

@karlkfi: it is actually possible to obtain the behavior you're looking for. Can you describe the use-case a little bit more in depth? I'd love to find a way to satisfy your scenario with the new docker:dind images, and add more instructions to the README. Trust me, in the long run, everybody will win :-)
Meanwhile, I'm re-opening this issue so we can discuss here.

Hey @karlkfi, are you still into this issue? Just would like to know the current state. Or if I can test something on it ;)

I do still want some of this behavior upstreamed, but the Dockerfile I'm using for mesos-slave-dind is based on this repo, not the official docker image. I looked into "rebasing" against the official branch and decided it was going to be a ton of work I don't have time for right now.
Builder & Dockerfile: https://github.com/mesosphere/docker-containers/tree/master/mesos-slave-dind
Dockerhub: https://hub.docker.com/r/mesosphere/mesos-slave-dind/
The big thing I was changing in this PR was support for overlay on overlay and falling back to aufs on ext3 loop on aufs if overlay is not available. But this logic doesn't map directly to the official image because it's more flexible about which fs you can use.
The other problem is that mesos-slave-dind runs mesos-slave in the foreground and docker daemon in the background, but I can't figure out how to do that with the official docker image. It seems the new image hinges on having only one app per container, which sounds ideal, but doesn't work when you need the daemon and another app to agree on image paths.

@jpetazzo my use case for running dind then another process is: a jenkins image which has it's own isolated docker environment for building jobs.

@munkyboy or other people considering using d-in-d with Jenkins: I'd recommend that you check this blog post.

I'm not actively maintaining this repository anymore.
I will continue to consider small-impact PRs, but I would advise people to:

check the official docker images on the Docker Hub, as they have a dind variant specifically designed to run Docker-in-Docker;
do their own thing with a fork of this repository if that's easier for their workflow.

Thank you!


docker ps -a show huge Amount of Docker Containers on Mesos/Marathon Slave server

Good evening,
We've configured a marathon 0.10.0 and mesos 0.23.0 with 3 masters and 4 slaves ( 2 of them with short amount of mem and cpu), we're using as well docker 1.7
We're running a POC of these technologies and we're deploying 1 docker container of logstash, 1 docker container of kibana, 2 instances of mesos-dns and a docker container cluster of elasticsearch with two instances
We've found out that in one of the servers with small amount of resources, where some of the marathon instances can't run due to lack of resources, when we run a "docker ps -a" we got a list of  almost 100 elements.
Is this behavior the expected one? Since the environment performs a pre-check of the resources on the slaves previous to try to start the instance task on the slave, this situation shall not happen right?
Kind Regards

Mesos/Marathon don't do a pre-check before starting tasks. Marathon gets resource offers from Mesos and then based on the resource requirements specified in the app definition, it will schedule tasks that fit in those offers.
Could it be possible that the resources you defined in the app definition are not enough to start the tasks? In that case Marathon would schedule a task, the Mesos Slave would then try to start it, but it would fail, freeing up the resources. Once Marathon notices that the task failed to start and that there are enough resources for it, it will reschedule it, leading to another task failure.

Also if you start tasks not using Mesos, the auto-detection of Mesos will not be accurate.
I will close this now. If you have more questions, please ask.


Nested rsets was broken

Nested rsets was broken after commit dac7aba843e7fe9a47adb3696466811691334328
Did'n get the problem why it happend.
Here is the stack trace:
.....
ORA I 2016/06/20 14:26:07.875750 rset.go:900: E2S1S1S1S0 [Rset.open] 27. prec=0 scale=-127 => gct=OraF64
ORA I 2016/06/20 14:26:07.875794 rset.go:900: E2S1S1S1S0 [Rset.open] 29. prec=30 scale=0 => gct=N
ORA I 2016/06/20 14:26:07.875826 rset.go:900: E2S1S1S1S0 [Rset.open] 30. prec=30 scale=0 => gct=N
ORA I 2016/06/20 14:26:07.875860 rset.go:900: E2S1S1S1S0 [Rset.open] 31. prec=30 scale=0 => gct=N
ORA I 2016/06/20 14:26:07.875894 rset.go:900: E2S1S1S1S0 [Rset.open] 32. prec=0 scale=-127 => gct=OraF64
ORA I 2016/06/20 14:26:07.875923 rset.go:900: E2S1S1S1S0 [Rset.open] 33. prec=30 scale=0 => gct=N
ORA I 2016/06/20 14:26:07.876007 rset.go:886: E2S1S1S1S0 [Rset.Next]
fatal error: unexpected signal during runtime execution
[signal 0xb code=0x1 addr=0x90 pc=0x7fa22ca644fc]

runtime stack:
runtime.throw(0x9b91a0, 0x2a)
        /usr/local/go/src/runtime/panic.go:527 +0x90
runtime.sigpanic()
        /usr/local/go/src/runtime/sigpanic_unix.go:12 +0x5a

goroutine 18 [syscall, locked to thread]:
runtime.cgocall(0x78d940, 0xc8200cb208, 0x7fa200000000)
        /usr/local/go/src/runtime/cgocall.go:120 +0x11b fp=0xc8200cb1b8 sp=0xc8200cb188
git.restr.im/itv-rt/itv-api/vendor/github.com/rana/ora._Cfunc_OCIStmtFetch2(0x191bc80, 0x190f608, 0x200000080, 0x0, 0xc800000000)
        ??:0 +0x39 fp=0xc8200cb208 sp=0xc8200cb1b8
git.restr.im/itv-rt/itv-api/vendor/github.com/rana/ora.(*Rset).beginRow(0xc8202a6000, 0x0, 0x0)
        /go/src/git.restr.im/itv-rt/itv-api/vendor/github.com/rana/ora/rset.go:193 +0x565 fp=0xc8200cb328 sp=0xc8200cb208
git.restr.im/itv-rt/itv-api/vendor/github.com/rana/ora.(*Rset).Next(0xc8202a6000, 0x7fa2283c6000)
        /go/src/git.restr.im/itv-rt/itv-api/vendor/github.com/rana/ora/rset.go:263 +0xfc fp=0xc8200cb410 sp=0xc8200cb328
...

Sorry, but what is an "inherited rset"?
Is it fixed in some commit, but wrong with dac7aba ?
What test have you used to bisect and find the broking commit?
Although that commit is huge, so a possible source of troubles...
Can you reproduce the problem?
Can you run your code with
ulimit -v unlimited
env CGO_CFLAGS=-fsanitize=address CGO_LDFLAGS=-lasan <your program here>

? This address sanitizer should have catch the 0xb (SIGSEGV) errors.

I meant nested rsets.
No, in the master branch it's still broken.
I've used my own application. It's been working before this commit, but it doesn't work after.
Here is stack trace:
ASAN:SIGSEGV
=================================================================
==20465==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7f723f60f4fc sp 0x7f722ed6e060 bp 0x7f722ed6e0c0 T5)
    #0 0x7f723f60f4fb in kpcxc2r (/usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1+0x18bd4fb)
    #1 0x7f7240d672d7 (/usr/lib/x86_64-linux-gnu/libasan.so.1+0x582d7)
    #2 0x7f7240d65c0b (/usr/lib/x86_64-linux-gnu/libasan.so.1+0x56c0b)
    #3 0x45fe84 in runtime.sigfwd (/tmp/go-build514681893/command-line-arguments/_obj/exe/main+0x45fe84)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV ??:0 kpcxc2r
Thread T5 created by T3 here:
    #0 0x7f7240d32bba in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.1+0x23bba)
    #1 0x79a13e in _cgo_sys_thread_start /usr/local/go/src/runtime/cgo/gcc_linux_amd64.c:43
    #2 0x10c820505f9f (+0x205f9f)

Thread T3 created by T0 here:
    #0 0x7f7240d32bba in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.1+0x23bba)
    #1 0x79a13e in _cgo_sys_thread_start /usr/local/go/src/runtime/cgo/gcc_linux_amd64.c:43

==20465==ABORTING
exit status 1

Okay, ASAN does not help.
Can you share anything about your code? A minimal working test would be the best, but maybe a description (what kind of tested rset?) could start us, too.

Are you using PrepAndExe? As it closes the statement right after execution, it also closes the fresh Rset, too.
I've documented it harder in 8aa81bc.  Also, see Test_cursor1_session in z_resultSet_session_test.go - that test passes, so if it still fails on you, than please tweak that test till it reproduces the problem!

No, we don't use PrepAndExe.
Our workflow looks like:

  env, err := ora.OpenEnv(nil)
  // defer env.Close()
  if err != nil {
    panic(err)
  }

  if logging {
    ora.Cfg().Log.Logger = lg.Log
    ora.Cfg().Log.Rset.Next = true
  }

  srvCfg := ora.NewSrvCfg()

  srvCfg.Dblink = dbLink
  srv, err := env.OpenSrv(srvCfg)
  // defer srv.Close()
  if err != nil {
    panic(err)
  }

  sesCfg := ora.NewSesCfg()
  sesCfg.Username = user
  sesCfg.Password = pass
  sesCfg.StmtCfg.Rset.SetNumberInt(ora.OraI64)
  sesCfg.StmtCfg.Rset.SetNumberFloat(ora.OraF64)
  sesCfg.StmtCfg.Rset.SetBinaryDouble(ora.OraF64)
  sesCfg.StmtCfg.Rset.SetBinaryFloat(ora.OraF64)
  sesCfg.StmtCfg.Rset.SetFloat(ora.OraF64)
  sesCfg.StmtCfg.Rset.SetDate(ora.OraT)
  sesCfg.StmtCfg.Rset.SetTimestamp(ora.OraT)
  sesCfg.StmtCfg.Rset.SetTimestampTz(ora.OraT)
  sesCfg.StmtCfg.Rset.SetTimestampLtz(ora.OraT)
  sesCfg.StmtCfg.Rset.SetChar1(ora.OraS)
  sesCfg.StmtCfg.Rset.SetVarchar(ora.OraS)
  sesCfg.StmtCfg.Rset.SetLong(ora.OraS)

  ses, err := srv.OpenSes(sesCfg)

  if err != nil {
    panic(err)
  }

stmt, err := ses.Prep("CALL some_precedure(:1, :2, :3, :4, :5, :6)")

if err != nil {
  return err
}

rst := &ora.Rset{}

_, err = stmt.Exe(packageID.Value, regionID.Value, genresList.Value, from.Value, to.Value, rst)

if err != nil {
  return err
}

  for rst.Next() { // here it fails

  }

This flow fails only in cases when result contains nested rsets, otherwise, it works properly.


mention httpie-jwt-auth plugin on README

Hello guys,
We've released https://pypi.python.org/pypi/httpie-jwt-auth and would like it to be mentioned on README so that everyone could know and use it right away.
Best regards,
Hoat Le

Thanks for sharing 👍


Remove iteration clause from LOOP body code

While here, simplify the loops — the compiler should be able to determine appropriate types based on the loop bounds. Also reindent.

Thanks. I'll take a closer look.
I'm not too concerned about optimizing around types here since it's only done at load-time. I originally annotated them to help with my sanity in porting the code over, so any simplification is welcome


Broken Badge

Hello--
The small badge appears as a broken link to me when visiting standardjs.com and when I copy/paste the provided path from the github readme.

Problem Identified --
Chrome identifies that there is an SSL problem
Failed to load resource: net::ERR_SSL_VERSION_OR_CIPHER_MISMATCH
Apparently some older OS (like WinXP) don't have this SSL algorithm-- I have noticed that it is starting to be used now-a-days on more and more websites (probably when people update their certs?)
Closing this now since it is not relevant to standardjs


Notification UX

#618
#658

Basic views

Note to self, for background processes (sync, gateway provisioning, migration of dat) #658 I think Live Activities and ActivityKit are a good fit. In-app notifications are still needed to supplement live activities though, and can be used to communicate status updates for those activities when the app has focus.


Firefox 62.03 not loading

Summary
TypeError: e is undefined
Filename: blob:https://example.com/7716361f-8ed8-4fa6-8fe9-341b7ec05921
Line: 1828
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0
CyberChef version: 8.8.5s
Example
Loading the Cyberchef in my Firefox 62.03 not working. I downloaded the html file from https://gchq.github.io/CyberChef/ and uploaded it to my servers in my internal network. Not working... Is there a missing dependency?
https://gchq.github.io/CyberChef/ is working in my browser!

Same issue, I download to my desktop, and it is stuck at loading page, dont know why.

Duplicate with #388

Fixed in #403. Thanks to @bwhitn.


update broken token spec link

We are updating this page to fix a broken link and point users to the current soroban-token-spec resource

Preview is available here:http://soroban-docs-pr404.previews.kube001.services.stellar-ops.com/


Newer PHP-CS-Fixer version does not support older command-line arguments

debug.md
There's an error with beautification, It seems that there's newer version of php-cs-fixer that does not support older parameters/rules

Feel free to submit a Pull Request updating the beautifier. Applicable file is https://github.com/Glavin001/atom-beautify/blob/master/src/beautifiers/php-cs-fixer.coffee


Add presentation from SSUG Gothenburg

Presentation on PnP from Swedish SharePoint User Group meeting

Hi @pbjorklund, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by Microsoft and real humans are currently evaluating your PR.
TTYL, MSBOT;

Thx Patrik. Merging...

Good presentation on the PnP structure and especially like the value for different roles. Great job and hopefully was well taken by the audience.

Thanks Vesa! I know that at least one person got super exited afterwards,
just waiting to see if any PR's show up from him now :)
Mvh Patrik Björklund
On Thu, Jun 4, 2015 at 4:33 PM, Vesa Juvonen<EMAIL_ADDRESS>wrote:

Good presentation on the PnP structure and especially like the value for
different roles. Great job and hopefully was well taken by the audience.
—
Reply to this email directly or view it on GitHub
https://github.com/OfficeDev/PnP-Guidance/pull/32#issuecomment-108916176
.


Feedback on the completed thesis draft

Hi @donizk, below is the feedback on your thesis document.
Overall, great work with strong writing! Below are some of the things I identified to make your work even stronger. Please address these issues in advance of submitting the final version of your comp. Here is a reminder of the remaining deadlines in CMPSC 610:

Week Fifteen (April 24 - April 28) through Week Sixteen (May 1 - May 5):  Final revisions to senior thesis document.
May 5 by 11:59 PM: Submit the final version of senior thesis as a tagged release in the document's GitHub repository.

You also need to respond to the questions that appear in the feedback below, they are labeled as [Q:]. Please respond to the questions within 24 hours of this issue submission, then you can work on making changes to the document throughout the remaining time before the May 5th deadline. For each question please provide an explanation of how you plan to address that item or if you already addressed it in the document, please provide information with the section and page number where it has been addressed.
Feedback and Questions

Please make sure that your document meets all of the baseline requirements outlined in the syllabus. You should use the syllabus as a checklist  to confirm that you meet every baseline requirement.

In addition to finishing the final version of thesis document you also need to have a completed version of your computational
artifact that meets all of the baseline requirements outlined in the syllabus.

[Q:] Are there relevant references to make your claims about data reliability and issues in equity stronger? Please include references for arguments made in the Ethical Implications section that begins on page 15. Issues in Equity subsection especially deserves a deeper and stronger discussion.

[Q:] What does "statistical significance" mean? You need to explain what it is, how it is evaluated, what p value is, etc. in the document, perhaps on page 95.

The first regression Figure (112) should be explained in detail. Walk the reader through the sections shown in the figure and explain what each section is.

[Q:] How does a static dashboard and a confusion matrix accuracy result in ethical concerns? It feels like there is a step missing in your discussion of Future Ethical Implications.

Rewrite a few sentences on page 32 under "Data Description" to be more scientific and less personal.

Reformat vertical spacing on pages 48-51, ensuring vertical space before subheadings like "Hispanic Recording" and try to ensure that the subheading shows on the same page as the visual.

There are a few of typos that I saw (repeated "the", misspelled words, for example "seperated"). Please read your document carefully, run it through a spellchecker if possible, and improve its readability by fixing those errors.

[Q:] Are there relevant references to make your claims about data reliability and issues in equity stronger? Please include references for arguments made in the Ethical Implications section that begins on page 15. Issues in Equity subsection especially deserves a deeper and stronger discussion.
I did not reference any literature/other sources in the Ethical Implications. I can, however, find additional references to include to further bolster the claims made in this section. Beyond that, how should I consider deepening the discussion of Ethical Implications?
[Q:] What does "statistical significance" mean? You need to explain what it is, how it is evaluated, what p value is, etc. in the document, perhaps on page 95.
Statistical significance refers to the finding of significance in a variable used in the regression, determined by the interpretation of the p-values and confidence intervals, but also indicated by the asterisks next to these values on the regression summary output.
[Q:] How does a static dashboard and a confusion matrix accuracy result in ethical concerns? It feels like there is a step missing in your discussion of Future Ethical Implications.
I think that the nature of the static dashboard results in ethical concerns due to the inability to continuously add more data to it, making it confined to the data that the project was developed with. This may come as an issue, as the findings presented may not be as accurate, as the information presented may degrade with time. As for the confusion matrix, a potential ethical implication could be the fact that the confusion matrix was not implemented in the suggested manner, which could make the accuracy resulting from this unreliable -- meaning that the model generated may be more or less accurate than what is actually reported.

The following changes have been made to the document to adhere to the suggested revisions:

reformat Data Description
address problems with spelling and spacing
added to Future Ethical Implications and Issues in Equity sections
added a blurb about statistical significance and p-values.

@donizk Your thesis release 3.0.1 satisfies the baseline requirements and incorporated all feedback provided to you after the defense. Congratulations, you have passed CMPSC 610 and are now officially de-comped!
Please upload your comprehensive thesis document to DSpace. Also, if you would like to have hard copies of your comp, please see Sabina Gora, our building coordinator, to get that process started.


keygen: Fix public key comment

Fix support for adding -C to the .pub file
Set default comment to user@host

I was wondering if this should be part of the Key{} struct actually.
We never actually read this public key for the agent so running ssh-tpm-agent and then ssh-add -L won't show the comments, and they should.

I was wondering if this should be part of the Key{} struct actually.
We never actually read this public key for the agent so running ssh-tpm-agent and then ssh-add -L won't show the comments, and they should.

Yes this makes sense, so we can show the comment to Pinentry as well

@stigtsp Do you want to take a stab at this?
I can write some tests for the key stuff itself so you can more easily extend the struct and parsing.

Being implemented with this https://github.com/Foxboron/ssh-tpm-agent/pull/18


Move ObjectExpresssion helper declaration to top of scope

Closes #51. I implemented a new method declareIdentifier on Blocks so that they can handle declaration of variables themselves.

much neater 👍


Arrow buttons leak events

When you click the arrow buttons the body element receives events.
I assume this is not the intended behavior.
Here's a jsbin.
In previous versions of basic-elements this behavior did not appear.

@Jamtis Thanks for the report. I'll investigate, but I wanted to mention that it will be next week before I will have a chance to look at this.

No problem. Maybe I find the time to fix it myself and make a PR.

I updated the jsbin.
event.stopPropagation() prevents the events from leaking through.

Also I would suggest to switch to the tap event instead of click.

I've updated basic-arrow-direction with a fix for this. The new behavior is: if an arrow buttons handles the click, it stops propagation. However, if the arrow button doesn't handle the click, the event propagates. This situation comes up when the carousel is on the first (or last) item and the user clicks the left (or right) button.
While the tap event is appealing, I'm finding it a bit too magic for use just yet. Case in point: I tried using the tap event here, and it didn't work as expected. I'll monitor progress on that event, and will keep looking for an opportunity to use it.
This fix is on the master branch of the consolidated basic-web-components repository. It will go out in the next release, which should be 0.6.3, at which point it will make its way to the Bower-compatible basic-arrow-direction repository.

Nice that it's fixed.
About the tap event: That's exactly what I'd do too.
Big :+1: for your great support.

@Jamtis De nada — you make these components better with your issues and feedback.
Since this sounds like an acceptable resolution, I'm going to mark this as closed.
BTW, we're planning out our next investments in this component set, and would love your feedback. If there are areas where you'd like to see us invest — more improvements to basic-carousel, or new components, or anything else — then send your thoughts to me at<EMAIL_ADDRESS>Thanks.

This fix has now been released.


[dv] Clean up for style lint

Signed-off-by: Weicai Yang<EMAIL_ADDRESS>

@msfschaffner
I cleaned up all style lint error in DV, except

Syntax error within SV macros due to tool issue
rv_plic/dv. I'm not sure if you still want to maintain this TB, but some files are missing and it can't be run now.

@rasmus-madsen
I updated AES DV to fix below style errors:

trailing spaces (no space at the end of the line)
Line length exceeds max (100 characters)
Explicitly define a storage type for every parameter and localparam

@msfschaffner
I cleaned up all style lint error in DV, except

Syntax error within SV macros due to tool issue
rv_plic/dv. I'm not sure if you still want to maintain this TB, but some files are missing and it can't be run now.

That is maybe a better question for @eunchan or @cindychip. If it is not functional, should we update it or remove it? I guess this is something that would belong into scratch_dv (or whathever the name will be...)

@weicaiyang
thank you I will rebase once these get checked in.
I must say I find the limitations on linewidth to small.. I for one are strongly againts these wrapping lines. can you point me where this was decided I would like to understand the reason behind it :D

@weicaiyang
thank you I will rebase once these get checked in.
I must say I find the limitations on linewidth to small.. I for one are strongly againts these wrapping lines. can you point me where this was decided I would like to understand the reason behind it :D

@rasmus-madsen
Here is the link of the rule.
https://github.com/lowRISC/style-guides/blob/master/VerilogCodingStyle.md#line-length
I feel 100 char limit is quite useful when you diff 2 files and review code changes, the line won't be auto-wrapped in most of the monitors. And we also have some macros (likegfn, gn for common functions - get_full_name, get_name), to avoid too much line wrapping.
I personally add set colorcolumn=100 in .vimrc to know where to wrap the code in editor.


Add ability not to compile fabio from sources

optionally run under dedicated user
optionally install from compiled binary
provide upstart script for CentOS

LGTM :+1:


Fill subsampled luma AC buffer in dav1d layout

That is, with Block::W stride rather than fixed 32 stride.

Coverage increased (+0.5%) to 84.586% when pulling 6b4296cf5534ff8eea32bb07e150e0716f0c82d3 on barrbrain:dav1d-cfl-asm into a394931b873117f51582c34480c7672809a6b858 on xiph:master.


Missing gpg key: Debian repository is unsusable

Fllowing instruction Installation using repository on https://www.erlang-solutions.com/downloads/ result in a unusable repo, the GPG signature used to sign repo is not the one downloadable at repo root.
Can you please publish the missing signing public key?
root@d3501376823b:/# echo "deb http://binaries2.erlang-solutions.com/debian/ bullseye-elixir-1.15 contrib" | tee /etc/apt/sources.list.d/elixir.list
deb http://binaries2.erlang-solutions.com/debian/ bullseye-elixir-1.15 contrib
root@d3501376823b:/# wget https://binaries2.erlang-solutions.com/GPG-KEY-pmanager.asc
--2023-09-11 12:17:23--  https://binaries2.erlang-solutions.com/GPG-KEY-pmanager.asc
Resolving binaries2.erlang-solutions.com (binaries2.erlang-solutions.com)... <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, ...
Connecting to binaries2.erlang-solutions.com (binaries2.erlang-solutions.com)|<IP_ADDRESS>|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1335 (1.3K) [application/pgp-keys]
Saving to: ‘GPG-KEY-pmanager.asc’

GPG-KEY-pmanager.asc                           100%[=================================================================================================>]   1.30K  --.-KB/s    in 0s

2023-09-11 12:17:23 (65.9 MB/s) - ‘GPG-KEY-pmanager.asc’ saved [1335/1335]

root@d3501376823b:/# apt-key add GPG-KEY-pmanager.asc
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
OK
root@d3501376823b:/# apt update
Hit:1 http://security.debian.org/debian-security bullseye-security InRelease
Get:2 http://binaries2.erlang-solutions.com/debian bullseye-elixir-1.15 InRelease [7165 B]
Hit:3 http://deb.debian.org/debian bullseye InRelease
Hit:4 http://deb.debian.org/debian bullseye-updates InRelease
Err:2 http://binaries2.erlang-solutions.com/debian bullseye-elixir-1.15 InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 2D6FA62A5CE3E39E
Reading package lists... Done
W: GPG error: http://binaries2.erlang-solutions.com/debian bullseye-elixir-1.15 InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 2D6FA62A5CE3E39E
E: The repository 'http://binaries2.erlang-solutions.com/debian bullseye-elixir-1.15 InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

cc @gilacost

Working on it, there is an issue on the github runner and not reading the GPG Keys correctly.

Hi @dalbarado any news?

Continue troubleshooting, I already changed PGP, but aptly over github
runner didn't work as expected, because it didn't take pgp anymore.
Investigating the issue.
On Mon, 18 Sept 2023 at 03:20, Sebastien BLAISOT @.***>
wrote:

Hi @dalbarado https://github.com/dalbarado any news?
—
Reply to this email directly, view it on GitHub
https://github.com/esl/packages/issues/71#issuecomment-1722812672, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AGRO4LHEECJLWDSULSGTTZLX27R3NANCNFSM6AAAAAA4THQGM4
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
*
*
*Our
upcoming conferences:
*
Code BEAM Europe
https://codebeameurope.com: 19-20 October 2023
RabbitMQ Summit
https://rabbitmqsummit.com: 20 October 2023
Erlang Solutions cares
about your data and privacy;
please find all details about the basis for
communicating with you and the way we
process your data in our Privacy
Policy https://www.erlang-solutions.com/privacy-policy.html. You can
update your email preferences or opt-out from receiving Marketing emails
here.
https://forms.erlangsolutions.com/email-preference?epc_hash=JtO6C7Q2rJwCdZxBx3Ad8jI2D4TJum7XcUWcgfjZ8YY

any news? this is still a problem.

Problem is fixed now, I have to regenerate the entire repo using a new pgp which is published.


Remove version: CrystalDewWorld.CrystalDiskInfo.KureiKeiEdition version 8.16.0

[Automated] It returns code over 400 in all urls
Microsoft Reviewers: Open in CodeFlow

Close with reason: Duplicate;


single-valued indicators should not have accessors

moving average, exponential moving average, etc ... do not have accessors. However RSI does - for consistency, any indicator that only requires a single put value should not expose accessors.
The design guidelines should be updated accordingly.
(or, alternatively every indicator should expose accessors)
@chrisprice @tlsim - what do you think? I'm erring towards all indicators having accessors for a consistent API.

I think allowing an accessor to be specified is more in keeping with d3
(even v4) but agree we should be consistent whatever the choice.
Although now I'm thinking about it would a pre-map be so bad? Actually on
balance I think the consistency wins out...
On Mon, 23 May 2016 at 07:27, Colin Eberhardt<EMAIL_ADDRESS>wrote:

moving average, exponential moving average, etc ... do not have accessors.
However RSI does - for consistency, any indicator that only requires a
single put value should not expose accessors.
The design guidelines should be updated accordingly.
(or, alternatively every indicator should expose accessors)
@chrisprice https://github.com/chrisprice @tlsim
https://github.com/tlsim - what do you think? I'm erring towards all
indicators having accessors for a consistent API.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/d3fc/d3fc-technical-indicator/issues/9

Talking gibberish, MA and EMA etc ... do have accessors, rebound from sliding window.


Admin - Address index view not found

Bug Report
Hi,
There's a problem with customer addresses in the admin panel, After Editing an address returned view does not exist.
Bagisto Version

[x] master branch
[ ] 1.3.3
[ ] 1.3.2
[ ] 1.3.1
[ ] 1.3.0

Steps To Reproduce

Go to Admin Panel
Go to Customers menu
Edit a Customer
Select the address tap
Edit an address
Click on Save

Actual Result

Expected Result

Hi @AmooAti,
Thanks for reporting this.

Hi @devansh-webkul,
This issue also appears when you want to create an address for the customer but you click on the back button next to the title.
Also, There's no back button in the Edit Address form to cancel Editing and get back to the customer information.

Hi @AmooAti,
This issue has been fixed, let me know if you face any problems.


Add Seedbox.fr

Backstory
No action is required. Other than to stop paying for your subscription. More info in the notes.
Data

name: Seedbox

url: https://wiki.seedbox.fr/index.php/Comment_Résilier_son_abonnement

difficulty: easy

notes:
There are two ways of cancelling your pack, depending on how you ordered your services:
    **Without automatic payment:**

    Simply ignore the reminders and the pack will be automatically deleted from our servers.
    No action is required on your part.
    Ignore the automatic reminders sent by our system.

    **With automatic payment:**

    ***It is imperative to cancel the automatic payment before the renewal of your service, no refund will be made.***

     - Log in to your PayPal account.
     - Click on Preferences at the top of the page.
     - Click on "My money", then on "Update" next to "My pre-approved payments".
     - Select the payment, then click Cancel
     - All you have to do is ignore the renewal calls issued by the system and your pack will be deleted automatically when it expires.

domains: https://www.seedbox.fr

Thanks for your contribution!


Support pagination with a --next option

Would be neat if you could do pagination when running shot-scraper javascript - by running extra JavaScript that returns the URL of the next page to visit.

Here's a prototype I built to help me scrape through all of https://news.ycombinator.com/from?site=simonwillison.net following the more links:
diff --git a/shot_scraper/cli.py b/shot_scraper/cli.py
index 9bc48aa..eb3a80e 100644
--- a/shot_scraper/cli.py
+++ b/shot_scraper/cli.py
@@ -524,6 +524,21 @@ def accessibility(url, auth, output, javascript, timeout, log_console, skip, fai
     is_flag=True,
     help="Output JSON strings as raw text",
 )
<EMAIL_ADDRESS>+    'next_',
+    "--next",
+    help="JavaScript to run to find next page",
+)
<EMAIL_ADDRESS>+    "--next-delay",
+    type=int,
+    help="Milliseconds to wait before following --next",
+)
<EMAIL_ADDRESS>+    "--next-limit",
+    type=int,
+    help="Maximum number of --after pages",
+)
 @browser_option
 @user_agent_option
 @reduced_motion_option
@@ -536,6 +551,9 @@ def javascript(
     auth,
     output,
     raw,
+    next_,
+    next_delay,
+    next_limit,
     browser,
     user_agent,
     reduced_motion,
@@ -571,6 +589,7 @@ def javascript(
     if not javascript:
         javascript = input.read()
     url = url_or_file_path(url, _check_and_absolutize)
+    next_count = 0
     with sync_playwright() as p:
         context, browser_obj = _browser_context(
             p,
@@ -582,9 +601,27 @@ def javascript(
         page = context.new_page()
         if log_console:
             page.on("console", console_log)
-        response = page.goto(url)
-        skip_or_fail(response, skip, fail)
-        result = _evaluate_js(page, javascript)
+        result = []
+        while url:
+            response = page.goto(url)
+            skip_or_fail(response, skip, fail)
+            evaluated = _evaluate_js(page, javascript)
+            if next_:
+                result.extend(evaluated)
+            else:
+                result = evaluated
+            next_count += 1
+            if next_:
+                if next_limit is not None and next_count >= next_limit:
+                    raise click.ClickException(
+                        f"Reached --after-limit of {next_limit} pages"
+                    )
+                url = _evaluate_js(page, next_)
+                print(url)
+                if next_delay:
+                    time.sleep(next_delay / 1000)
+            else:
+                url = None
         browser_obj.close()
     if raw:
         output.write(str(result))

I ran it like this and it worked!
shot-scraper javascript \
    'https://news.ycombinator.com/from?site=simonwillison.net' \
    -i /tmp/scrape.js \
    --next '() => {
        let el = document.querySelector(".morelink[rel=next]");
        if (el) {
            return el.href;
        }
    }' -o /tmp/all.json --next-delay 1000

Needs more thought about how things like concatenating together results from multiple pages should work.
It would also be neat if this could return a {"method": "POST", "body": "..."} object as an alternative to returning a URL, then shot-scraper could hit subsequent pages using other HTTP methods. Maybe persist cookies too!

I was trying to scrape some Google Maps lists of places, but didn't manage as the first page that loads is a cookie notice that triggers a navigation event when accepted / rejected that results in Error: Execution context was destroyed, most likely because of a navigation., but this sounds like it could solve it?
To your question, maybe it could just return JSON-LD and leave the concat to downstream?


Ability to have Tab Depth Indicators

Feature request.  Take a look at the F# Power Tools.  Python is a tab oriented language syntactically and this would make a world of difference for readability.

Do you mean like Indent Guides? I don't see anything on the F# Power Tools that looks like "Tab Depth Indicators" - perhaps you could post a screenshot?


what are all the possible ways to require a beakerlib library?

Currently, beakerlib is able to parse metadata.yaml for libraries written in the form of library(X/Y) and {url: "X", name:"Y"}.
Please give me all the possible ways so I can implement it in beakerlib as well.

All supported library combinations should be documented in the require attribute. Let us know if there are any details still missing.


Add ability to use TypedDict Required/NotRequired

Add ability to use TypedDict Required/NotRequired
Recently Required and NotRequire was added to typing_extentions.
This PR adds ability to use them in TypedDict declaration:
from typing_extensions import NotRequired, Required, TypedDict
from pydantic import BaseModel

class DataTD1(TypedDict, total=False):
    a: int
    b: Required[str]

class DataTD2(TypedDict, total=True):
    a: NotRequired[int]
    b: str

class Model1(BaseModel):
    t1: DataTD1
    t2: DataTD2

Model1(t1={}, t2={})

pydantic.error_wrappers.ValidationError: 2 validation errors for Model1
t1 -> b
  field required (type=value_error.missing)
t2 -> b
  field required (type=value_error.missing)

Related issue number

Checklist

[x] Unit tests for the changes exist
[x] Tests pass on CI and coverage remains at 100%
[ ] Documentation reflects the changes where applicable
[ ] changes/<pull request or issue id>-<github username>.md file added describing change
(see changes/README.md for details)
[x] My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

please review

please update.

please review

please review

please review


Add TransformerM monad to simplify mark* functions.

This is way more clean, indeed! The only thing that is bothering me a little bit is that we're ignoring the output of the State monad. Do you know if there is any alternative monad that acts just like the State but doesn't have the output?

Hmm, maybe this doesn't make sense, since the bind operator won't have a type to work on... well, I will approve the changes, but let me know if you know some way to make this work


fix "reference to Process is ambiguous" error in build.sbt for sbt 0.13.15 and 1.0.3 (didn't test other versions).

Also added a trailing slash to destination directory in Makefile (otherwise, calling DESTDIR=/opt/leo3 make would create the shell script as /opt/leo3leo3).

Hi Martin, thanks for the Makefile correction.
Concerning the updated build.sbt: I get an error when testing the pull request locally on my machine:
sbt assembly
[info] Loading settings from idea.sbt ...
[info] Loading global plugins from /home/lex/.sbt/1.0/plugins
[info] Loading settings from plugins.sbt ...
[info] Loading project definition from /home/lex/dev/Leo-III/project
/home/lex/dev/Leo-III/build.sbt:44: error: object java.lang.Process is not a value
          val exitCode = Process("java", args) ! log
                         ^
/home/lex/dev/Leo-III/build.sbt:47: error: object java.lang.Process is not a value
          val exitCode2 = Process("rm", Seq((target / "tptp.tokens").toString, (target / "tptpLexer.tokens").toString)) ! log
                          ^
[error] sbt.compiler.EvalException: Type error in expression
[error] sbt.compiler.EvalException: Type error in expression

I use sbt 1.0.3. It seems to me that the import import scala.sys.process._ is indeed necessary as the sbt tool changed its behavior since 1.x to not support sbt.Process anymore (see http://www.scala-sbt.org/1.x/docs/sbt-1.0-Release-Notes.html). I recall changing this in 386e14eefbec08ddfd3a0af13c5bfb6d4babad50. Unfortunately, I forget to update the README to require sbt 1.x rather than 0.13.x.
Are you sure that your solution worked for you under sbt 1.x?
Best, Alex

My apologies, you are right. It seems my first attempts with 0.13.15 created a corresponding entry in project/build.properties which was overriding the newly installed version. I put the import back into the file and left only the patch for trailing destination directory paths.


chore(deps): upgrade dev dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-dev-dependencies-k8s-26-main" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 3707

Questions ?
Please refer to the Backport tool documentation


[Unexpected Behavior] Fatal Error

Well, when I was playing your SM 5.3 Theme, it was behaving as normal when all of a sudden...

When I clicked abort, I got this:

Thanks for reporting this Fatal error
I just fix recently and upload it to this repository.
Like I said in README.md "⚠This theme is still in development.⚠".
I know that there is a lot of bug or issue in my theme.
But during this time I have some big project to do (not this one :( )  so I don't have time to fix all Theme's bugs.
If you found an issue feel free to let me know in this repository (this page).
Not Outfox themselve Because It is Themer fault(ME).


GET requests or empty payloads make the SDK panic

The receiver panics when receiving a GET request of empty payload.
Given this client
func main() {
    cli, _ := cloudevents.NewDefaultClient()
    cli.StartReceiver(ctx, func(cloudevents.Event) {})
}

just curl localhost:8080.
📄 error trace
panic: runtime error: invalid memory address or nil pointer dereference
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x7b2946]

goroutine 19 [running]:
github.com/cloudevents/sdk-go/v2/protocol/http.(*Message).Finish(0x0, 0x0, 0x0, 0x0, 0x0)
        /go/pkg/mod/github.com/cloudevents/sdk-go/v2@v2.0.0-!r!c1/protocol/http/message.go:151 +0x26
github.com/cloudevents/sdk-go/v2/client.(*receiveInvoker).Invoke.func1(0xd91c20, 0x0, 0xc00006de78)
        /go/pkg/mod/github.com/cloudevents/sdk-go/v2@v2.0.0-!r!c1/client/invoker.go:41 +0x47
panic(0xba6860, 0x128d010)
        /usr/local/go/src/runtime/panic.go:969 +0x166
github.com/cloudevents/sdk-go/v2/protocol/http.(*Message).ReadEncoding(0x0, 0xc000020000)
        /go/pkg/mod/github.com/cloudevents/sdk-go/v2@v2.0.0-!r!c1/protocol/http/message.go:81 +0x5
github.com/cloudevents/sdk-go/v2/binding.ToEvent(0xd91c60, 0xc0000ae000, 0x7fce984681a0, 0x0, 0x0, 0x0, 0x0, 0xc00006ddbc, 0x2, 0x2)
        /go/pkg/mod/github.com/cloudevents/sdk-go/v2@v2.0.0-!r!c1/binding/to_event.go:28 +0x52
github.com/cloudevents/sdk-go/v2/client.(*receiveInvoker).Invoke(0xc00009e560, 0xd91c60, 0xc0000ae000, 0xd91c20, 0x0, 0x0, 0x0, 0x0)
        /go/pkg/mod/github.com/cloudevents/sdk-go/v2@v2.0.0-!r!c1/client/invoker.go:46 +0xfc
github.com/cloudevents/sdk-go/v2/client.(*ceClient).StartReceiver(0xc0000ce0a0, 0xd91c60, 0xc0000ae000, 0xb69e00, 0xcc3008, 0x0, 0x0)
        /go/pkg/mod/github.com/cloudevents/sdk-go/v2@v2.0.0-!r!c1/client/client.go:218 +0x287
github.com/cloudevents/sdk-go/v2/client.(*obsClient).StartReceiver(0xc00009e540, 0xd91c60, 0xc0000ae000, 0xb69e00, 0xcc3008, 0x0, 0x0)
        /go/pkg/mod/github.com/cloudevents/sdk-go/v2@v2.0.0-!r!c1/client/client_observed.go:93 +0xac
main.(*handler).Start.func1(0xc0000dc0c0, 0xc0000b03d0, 0xd91c60, 0xc0000ae000)
        /go/src/github.com/triggermesh/myproject/cmd/mycmdtarget/main.go:85 +0x63
created by main.(*handler).Start
        /go/src/github.com/triggermesh/myproject/cmd/mycmdtarget/main.go:84 +0x7a
exit status 2

The culprit is here:
https://github.com/cloudevents/sdk-go/blob/0fc16e13d87f0f2b1c99ec41183e35c0e68b6b5c/v2/binding/to_event.go#L23-L26
This comparison is false even when the request contains no payload.

@slinkydeveloper in case you haven't figured out what happened yet, the issue is described here: https://dev.to/pauljlucas/go-tcha-when-nil--nil-hic (thanks @odacremolbap for pointing that out).

whenever you assign to an interface (either explicitly via assignment or implicitly via return value), it takes on both the value and the concrete type. Once it takes on a concrete type, it will never compare equal to (the typeless) nil.

Same result when the Content-Type header is not application/cloudevents. Editing the issue description.

@slinkydeveloper I tackled this because it's causing issues with a project I'm currently working in.