time: timer gets significantly delayed during long time GC

What version of Go are you using (go version)?
go1.15.8

$ go version

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\xxx\AppData\Local\go-build
set GOENV=C:\Users\xxx\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\xxx\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\xxx\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Program Files\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\xxx\AppData\Local\Temp\go-build791935606=/tmp/go-build -gno-record-gcc-switches

What did you do?
First, I initialized a large number of objects that will not be released during the running of my program. These objects will cause a long GC time.
Then I started 6 goroutines:

1 goroutine blocked on a ticker with 200ms period, it will print the ticker time received from the channel tick.C;
4 goroutines with busy task, just infinite loop;
1 goroutine for trigging GC every second.

After that, the main goroutine sleeps for 20 seconds, and waits for the ticker goroutine's printing.
The source codes of my program are also posted:
package main

import (
	"fmt"
	"runtime"
	"time"
)

var x [][]*int

func prepareGarbage() {
	x = make([][]*int, 100)
	for i := range x {
		x[i] = make([]*int, 10000000)
	}
}

func main() {
	fmt.Println(runtime.Version())
	// GOMAXPROCS should be at least 4, so dedicated GC will be enabled.
	runtime.GOMAXPROCS(4)

	// prepareGarbage for long GC mark time
	prepareGarbage()

	go ticker(200)

	// for keeping all P busy
	for i := 0; i < 4; i++ {
		go busyTask()
	}

	go func() {
		for {
			time.Sleep(time.Second)
			runtime.GC()
		}
	}()

	// sleep enough time for the timer wakeup delay happening
	time.Sleep(time.Second * 20)
}

func ticker(periodms int64) {
	tc := time.NewTicker(time.Duration(periodms) * time.Millisecond)
	fmt.Printf("timer with %v ms period started...\n", periodms)
	var ticks uint64
	var last time.Time
	for t := range tc.C {
		if ticks == 0 {
			last = t
		} else {
			elapsedTime := t.Sub(last).Milliseconds()
			if elapsedTime > periodms*2 {
				fmt.Printf("\n%v ms=========== WAKE UP TOO LATE!!!\n", elapsedTime)
			} else {
				fmt.Printf("%v, ", elapsedTime)
			}
			last = t
		}
		ticks++
	}
}

func busyTask() {
	for {
		task()
	}
}

//go:noinline
func task() {
	do()
}

//go:noinline
func do() {}

What did you expect to see?
I expect to see all the elapsed time will be around 200 with small deviation.
go1.15.8
timer with 200 ms period started...
200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 190, 200, 200, 200, 200, 200, 200, 200, 200, 200, .......
Process finished with exit code 0
What did you see instead?
I actually found the timer will be randomly delayed with a large delta. In some test, the 200 ms ticker can be unbelievably delayed for more than one second.  Does this issue reproduce with 1.16 as well?

The listed demo works well with Go 1.13 while it reproduces with the Go versions with 1.14 and later. It is probably because the GC dedicated keeps running and cannot be preempted. Therefore, there is no chance for the P running GC dedicated to enter schedule, in which timers will be checked. When the GC dedicated finishes, the P running GC dedicated begin to check timers, but timers has been already delayed significantly.
As a easy fix, we can make the P's not running GC dedicated help check timers for the P running GC dedicated, by adding a random checkTimerForOtherPs for other P after the origin checkTimers in schedule.
It does work, but I am not sure whether it is a proper fix.

runtime.GC is a blocking operation, including running any goroutines that were enabled because of timers.

 It may also block the entire program.

@randall77 Hi randall, if I read this right, calling runtime.GC will block the caller goroutine. It is reasonable that the caller gets blocked because it needs to wait GC finished. I don't think the other goroutines should also be blocked.
To make it clear, I collect the trace data of the timer demo. We can see goroutines are well scheduled during GC. Each goroutine has a small scheduling slice and all the runnable goroutines have chances to run.

However, the timer-driven goroutine G19, which is expected to be scheduled every 200ms, gets significantly delayed. It misses many ticks when GC is working, i.e., the second GC phase in the following graph.

The timer-driven goroutine have a totally different behavior compared with other goroutines. And such behavior weakens the precision of timer strongly.

I don't think the other goroutines should also be blocked.

We don't make that promise at the moment.
I'm not sure what might be involved in making runtime.GC not block other goroutines. @mknyszek @prattmic  might know more.

We don't make that promise but generally speaking it doesn't block other goroutines except briefly to stop-the-world (just as a GC does).
I don't think the dedicated GC needs to be involved in this solution at all -- ideally, it really does just run unimpeded. We have 3 other Ps (in this reproducer) that should be checking timers regularly, basically whenever they yield back into the scheduler. Your execution trace shows that we're very clearly time slicing these goroutines, so they're calling into the scheduler fairly often. The fact that timers are missed seems like it could be a bug in the scheduler or timer system (but I haven't thought about this enough to have an idea of what it is yet).

cc @ChrisHines

I believe the issue here is that in the situations where the dedicated GC runs on the P that owns the 200ms timer the timer starves while the GC is running. It starves because the owning P is busy and because all of the other Ps have a steady supply of runnable Gs. Ps  only attempt to steal timers from other Ps when they are out of local work, which never happens in this scenario. As a result the timer isn't checked until it's owning P finishes the GC work and checks its own timers.

This problem has also been encountered in our company’s products. Because my company‘s product  service has a very long GC cycle and a relatively large number of timers, the probability of serious timer timeouts is particularly high. A lot of our processing logic relies on the accuracy of the timer, so the accuracy of the timer is very important to us.
@ChrisHines analyzed in detail the cause of this problem, that is, when the GC marking time is particularly long, it is in dedicated P. Because it is in the systemstack, the signal cannot preempt it, so this P cannot check its own local timer in the dedicated state. Other P are very busy, and there is no way to check the timers on other P, so the timers on this P have seriously timed out.

We do have a fairly efficient mechanism for stealing timers from other Ps (stealWork). However, based on the trace diagrams, I think what's happening here is that the non-GC Ps always have something in their run queue, and findrunnable will only call stealWork if both the local and global run queues are empty, so we wind up never attempting to steal timers from the P running the dedicated GC worker. (This all agrees with what @ChrisHines said.)
This problem isn't necessarily specific to the dedicated GC worker, though that maybe the worst culprit. We could make the scheduler try stealing timers more frequently, sort of like how it currently consults the global run queue periodically. We could focus on the dedicated GC worker and have pollWork check for expired timers on the calling P.
(More broadly, I've been thinking for a while about unsharding the timer queue and replacing it with a true concurrent data structure. That would probably solve this, too, because any P could run an expired timer, but that's a big chunk of work to bite off.)

@aclements I agree that the problem is not specific to the dedicated GC worker. I was going to reply along the same lines, but you beat me to it. :)
I am curious about your ideas for unsharding the timer queue. It seems to me that it could certainly simplify the runtime's model of timers, but how do we avoid reintroducing #15133? Did you have any specific concurrent data structures in mind for this situation?

@aclements Yes, steal work by checkTimers has the lowest priority in findrunnable. Therefore making sheduler stealing more frequent is a good solution. But as @ChrisHines pointed out, the horizontal scalability should also be taken into account, considering potential fierce contention between P's when the CPU number is very large. As for the GC-dedicated case, is it possible for an extra OS thread (instead of other P's) checking the starving timers on the dedicated-GC P‘s?

We do have a fairly efficient mechanism for stealing timers from other Ps (stealWork). However, based on the trace diagrams, I think what's happening here is that the non-GC Ps always have something in their run queue, and findrunnable will only call stealWork if both the local and global run queues are empty, so we wind up never attempting to steal timers from the P running the dedicated GC worker. (This all agrees with what @ChrisHines said.)
This problem isn't necessarily specific to the dedicated GC worker, though that maybe the worst culprit. We could make the scheduler try stealing timers more frequently, sort of like how it currently consults the global run queue periodically. We could focus on the dedicated GC worker and have pollWork check for expired timers on the calling P.

Starting from go1.16, there has been a relatively large optimization on the timer. The basic strategy is to increase the frequency of stealing timer. Especially in the findrunnable, the timers on other P will be considered when calculating pollUntil. What's more,  put the priority of stealing timers from other Ps before stealing g from other Ps. This optimization can be a good solution to improve the accuracy of the timer when other P is not very busy, but it has no effect when P is very busy.
To solve the timer significantly delayed when P is busy, simply increasing the frequency of the steal timer may bring some side effects, such as more scheduling contention, or useless timer stealing. Because not in all cases, we need to steal the timers on other P.
I also agree that the problem is not specific to the dedicated GC worker. But in any case, we need have a way which can efficiently detect whether there is a serious timer delayed in system, and then accurately steal the timer on the P.

As for the GC-dedicated case, is it possible for an extra OS thread (instead of other P's) checking the starving timers on the dedicated-GC P‘s?

Prior to https://github.com/golang/go/commit/8fdc79e18a9704185bd6471b592db1e8004bd993 in Go 1.16 the sysmon thread did exactly that. When it found a starving timer it would try to wake up a sleeping P. But that strategy doesn't work in the situation described in this issue since all the P's are busy.
This issue boils down to a question of priorities. The system is over subscribed with more work than it has P's to handle. If we want it to give more priority to timers something else will get less priority. Done poorly something else may starve.

As for the GC-dedicated case, is it possible for an extra OS thread (instead of other P's) checking the starving timers on the dedicated-GC P‘s?

Prior to 8fdc79e in Go 1.16 the sysmon thread did exactly that. When it found a starving timer it would try to wake up a sleeping P. But that strategy doesn't work in the situation described in this issue since all the P's are busy.
This issue boils down to a question of priorities. The system is over subscribed with more work than it has P's to handle. If we want it to give more priority to timers something else will get less priority. Done poorly something else may starve.

Yes, Prior to 8fdc79e in Go 1.16 the sysmon thread did exactly that,  which doesn't work mainly because the startm doesn't work(it can't find a idle P to wake up when all P are busy), not the sysmon thread. You can review the CL352710 which takes a different approach to deal with extremely starving timers, even though it also use sysmon thread to mark the timer with seriously timeouts.


Make app full screen

Things to consider
Will need to make sure collisions and boundaries are updated correctly.

This causes a lot more problems then you would think. Much easier to just keep this as a 600x600 game.


Comments & organization of app code

See https://github.com/diazrenata/AFS_database_code/pull/1

Other than my one comment about reading in location data, I think this looks good! Having all of these comments and structure is super useful. I am going to wait until you implement that change @diazrenata and then I'll test the code locally to make sure everything works and merge this.

@KristinaRiemer I toggled the toy_locs data and resolved the merge conflicts (mostly related to whitespace/indentation). I also fixed the typo referenced in #18, so that PR is no longer needed. This should be ready for you to test!


Remove invalidation while session is valid.

Attempting to set: invalidations[session] = nil would sometimes crash due to bad memory access, presumably because the invalidation closure has run and the session is no longer valid.
This update grabs the invalidation closure, then removes it from the map, then calls the closure to invalidate.

Eh not sure this really did anything useful, once the session is gone, it's gone


The --help switch is broken if you are not currently in a folder that contains a jss app

Description
Running jss deploy app --help in a folder that does not contain a jss app will not give you much information about the command. This is also true for jss setup --help or any other jss command:
PS C:\git\Website\src\JSS> jss setup --help
JSS CLI is running in global mode because it was not installed in the local node_modules folder.
jss <command>

Commands:
  jss create <name> <template>  Creates a new JSS application based on one of
                                the sample apps.

Options:
  --version  Show version number                                       [boolean]
  --help     Show help                                                 [boolean]

Expected behavior
The same output as running these commands in a folder that contains a jss app:
PS C:\git\Website\src\JSS\anwb-rondreizen> jss setup --help
jss setup

Sets up the scjssconfig.json config file, which defines how this application
connects to a Sitecore instance for content or deployment.

Options:
  --version            Show version number                             [boolean]
  --help               Show help                                       [boolean]
  --instancePath       Path to the Sitecore "Website" folder (e.g.
                       c:\dev\sitecore\Website)                         [string]
  --deployUrl          Sitecore host used to deploy the app (e.g.
                       http://sitecore9/sitecore/api/jss/import)        [string]
  --layoutServiceHost  Sitecore host used to download Layout Service data (e.g.
                       http://sitecore9)                                [string]
  --apiKey             Sitecore API Key (GUID of item under
                       /sitecore/system/Settings/Services/API Keys/ in master
                       db, or core db for Sitecore 9.0)                 [string]
  --deploySecret       Deployment secret (32+ random chars)             [string]
  --nonInteractive     Disallows interactive prompts (all other arguments must
                       be passed in this mode)                         [boolean]
  --outputFile         The file path to output the config to. Defaults to the
                       app's scjssconfig.json                           [string]

Steps To Reproduce

Go to a folder that does not contain a jss app
Run any jss command with the --help switch
Very little information is printed out about the current command

Your Environment

JSS Version: 11.0.2
Operating System and version (desktop or mobile): Windows 10 version 1809

This is actually expected behavior - it is giving you help on the commands currently available. When run outside a JSS app, commands that apply to a specific app are not available.

It certainly is unexpected expected behavior then

As described by @kamsar it is expected behaviour. I'm closing this one.


Type Guards with &&

var foo: number | string;
typeof foo == 'string' && foo.split(' '); // Property 'split' does not exist on type 'number | string'.

use the Strict Comparison Operator (===) instead of Equality Operator (==).

It would be nice if we could make it clearer why this happened but as mentioned this is by design. Type guards support specific patterns only.


Centrally define and validate environment variables

Ref MoveOnOrg/Spoke#1058. @jlegrone, would this want to wait until further work is completed on Docker/Tilt stuff?

@harpojaeger no I think we're good to go now that #5 is merged.
Validation and configuration management seemed like distinct issues to me at first, but after looking at envalid it seems like the schema it provides would probably be enough to do code generation for .env files, Heroku's app.json, lambda-env.json, etc. So I think a first priority should be to look at validation options including envalid with code generation in mind.


edit booking review-booking risk1 and risk2 user can enter more characters than the database field allows which is 200

User Interface	edit booking / review booking use	Javascript to stop user being able to more characters than the database will allow.

add some kind of on screen counter i.e. character 20/200 under the field that updates as someone types and doesn't let them type more than 200


Architecture SVG does not display properly in light-mode

The SVG image /doc/img/architecture.svg is not properly visible on light-mode as the connecting lines are white on white.
This is confusing at first when reading through the readme.

Yea, that's right. Maybe I should create a light mode variant of that graphic...
but who is using light mode anyways 😉


Causes a bootloop on the June pixel update

Pixel 6a, after OTA updating to 14.0.0 (AP2A.240605.024, Jun 2024) from 14.0.0 (AP1A.240305.019.A1, Mar 2024), my phone entered a bootloop with this module on.
Took a while to figure out it was this module, don't have any logs because it straight up hangs on the logo.
Worked fine on the March update.

tried reflashing?

Yeah, first thing I tried. No matter what I do, can't get it past the logo if I have FlagSecurePatcher enabled.

run this in termux and upload the fw.zip here. it can be large, you can use google drive etc.
pkg install zip && cd /system/framework/ && su -c $(command -v zip) -r9 /sdcard/fw.zip .

Sure

everything looks fine. even if the reason is FlagSecurePatcher, you should be able to grab a logcat before the it reboots with adb

Sure, I just ran adb logcat > logcat.txt three times.
First run, successful boot to lockscreen with all magisk modules disabled:
logcat-good.txt
Second run, bootloop just with FlagSecurePatcher enabled and everything else disabled:
logcat-bad.txt
Third run, just in case, same conditions as previously (just with FlagSecurePatcher enabled and everything else disabled):
logcat.txt

before reflashing did you make sure you uninstalled the module and rebooted? reflashing while its disabled wont work

Yeah, I did. Well, so far everything works if I just keep FlagSecurePatcher disabled

try r9

try r10

Thanks, works as intended now!


import tensorflow as tf fail

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary): source
TensorFlow version (use command below): 2.1 (with pip)
Python version: 3.7 (anaconda)
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version: cuda 10.1 cudnn 7.6
GPU model and memory: RTX 2080
Exact command to reproduce: import tensorflow as tf

Describe the problem
Hi, I install tf 2.1 with pip.
When I write inside Spyder :
import tensorflow as tf
I get the error
Error in callback <bound method AutoreloadMagics.post_execute_hook of <autoreload.AutoreloadMagics object at 0x000002944E042978>> (for post_execute):
Traceback (most recent call last):
File "C:\Users\user\Anaconda3\lib\site-packages\IPython\extensions\autoreload.py", line 578, in post_execute_hook
_, pymtime = self._reloader.filename_and_mtime(sys.modules[modname])
File "C:\Users\user\Anaconda3\lib\site-packages\IPython\extensions\autoreload.py", line 184, in filename_and_mtime
if not hasattr(module, 'file') or module.file is None:
File "C:\Users\user\Anaconda3\lib\site-packages\tensorflow_init_.py", line 50, in getattr
module = self._load()
File "C:\Users\user\Anaconda3\lib\site-packages\tensorflow_init_.py", line 44, in _load
module = _importlib.import_module(self.name)
File "C:\Users\user\Anaconda3\lib\importlib_init_.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 953, in _find_and_load_unlocked
File "", line 219, in _call_with_frames_removed
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in _load_unlocked
File "", line 728, in exec_module
File "", line 219, in _call_with_frames_removed
File "C:\Users\user\Anaconda3\lib\site-packages\tensorflow_core_init_.py", line 42, in
from . _api.v2 import audio
File "C:\Users\user\Anaconda3\lib\site-packages\tensorflow_core_api\v2\audio_init_.py", line 10, in
from tensorflow.python.ops.gen_audio_ops import decode_wav
File "C:\Users\user\Anaconda3\lib\site-packages\tensorflow_core\python\ops\gen_audio_ops.py", line 9, in
from tensorflow.python import pywrap_tensorflow as _pywrap_tensorflow
File "C:\Users\user\Anaconda3\lib\site-packages\tensorflow_init_.py", line 50, in getattr
module = self._load()
File "C:\Users\user\Anaconda3\lib\site-packages\tensorflow_init_.py", line 44, in _load
module = _importlib.import_module(self.name)
File "C:\Users\user\Anaconda3\lib\importlib_init_.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "C:\Users\user\Anaconda3\lib\site-packages\tensorflow_core\python_init_.py", line 49, in
from tensorflow.python import pywrap_tensorflow
File "C:\Users\user\Anaconda3\lib\site-packages\tensorflow_core\python\pywrap_tensorflow.py", line 74, in
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "C:\Users\user\Anaconda3\lib\site-packages\tensorflow_core\python\pywrap_tensorflow.py", line 58, in
from tensorflow.python.pywrap_tensorflow_internal import *
File "C:\Users\user\Anaconda3\lib\site-packages\tensorflow_core\python\pywrap_tensorflow_internal.py", line 28, in
_pywrap_tensorflow_internal = swig_import_helper()
File "C:\Users\user\Anaconda3\lib\site-packages\tensorflow_core\python\pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "C:\Users\user\Anaconda3\lib\imp.py", line 242, in load_module
return load_dynamic(name, filename, file)
File "C:\Users\user\Anaconda3\lib\imp.py", line 342, in load_dynamic
return _load(spec)
ImportError: DLL load failed:
Failed to load the native TensorFlow runtime.
Thanks for your help !

@moi35 ,
What is make/model of your cpu being used ?
Can you please check if the system supports AVX instructions sets? that might be causing the issue.
Also see Hardware Requirements.
Thanks!

Hi,
Yes, the system supports AVX instruction. The CPU is an intel i5 and i  I use the last driver available for the GPU. I have the good version of cuda/cudnn requirements for TF 2.1.
I have used before tensorflow 1.12 without issue on this computer (with older version of cuda/cudnn).
Thanks
Have a nice day

I think you have the same issue as me in #35618.

I think you're right. Sorry, I didn't see it.

I am having the same issue when I pip install tensorflow. I do not have an nvidia graphics card and want to use a CPU tensorflow, NOT GPU. I get:
ImportError: DLL load failed: The specified module could not be found.
Failed to load the native TensorFlow runtime.

I have a similar issue but only when  importing from vs code
I can import tensorflow as tf
However when importing tensorflow.keras.layers I get
the dll error
When importing from python repl I have no issues
importing tensorflow.keras.layers


Result Items pretend to be another class

This is one of the most frustrating things in any library. If you want it to pretend to be something else then use SimpleDelegator. This caused a lot of pain until realizing that I was not really the class that the object claimed to be.

+1 for using SimpleDelegator

+1

Hi, thanks for the feedback. Now:
a) What problem exactly do you experience by Tire::Results::Item masquerading as a different class, within Rails. It does so for a purpose, and it is clearly laid out in the Readme.
b) Could you be more specific as to how would SimpleDelegator (or any other delegation) solve this issue?

@marioaquino and @coffeencoke: No point in +1-ing anything here, really. Do you have any concrete suggestions/ideas or a concrete problematic behaviour to demonstrate?

@adkron: Ping. I'm curious to hear some feedback on my questions.

Karmi and I spent some time talking on IRC. I just want to get some of that information propagated back to the ticket.
I wasn't upset with the code. Tire works great. I love TIRE!
Overriding class is a usability issue for many developers. When debugging they interrogate the class and based on the information they get back there are some assumptions made about the capabilities of the object. Those assumptions here are really wrong.
On further inspection of the object respond_to? returns false for most methods because it overrides method_missing to give those capabilities. This maybe another ticket, but it would be great if respond_to? would be overridden in the same way. Although with the current implementation the object really responds to any and all method names you can throw at it.
In this case I have an assumption that my object is my model. So I try the following:
result.my_name #=> 'amos'
result.my_name = 'Amos' #=> No error
result.my_name #=> 'amos'
That is a surprising result.

Hi, thanks for the update, Adam.
The thing is, this whole story with overriding class is much more complicated then everybody thinks — or I am indefinitely stupid :)
First, I don't know how delegation as a concept would solve this issue. We really don't want to delegate to anything, because we don't have anything. We don't have the “real” model loaded. We have only some JSON returned from elasticsearch.
(Notice how we use delegation in Tire to expose the methods such as mapping, settings, index, search either directly in the class or in the tire instance and class proxy methods: https://github.com/karmi/tire/blob/master/lib/tire/model/search.rb#L225-241.)
Please correct me if you feel I'm wrong on this.
Second, the whole purpose of this is to make Tire::Results::Item instances work within Rails, without loading the “real” models from the database. As you may be aware, that means eg. dom_id helpers, and, prominently URL helpers.
You may try it out very simply: just comment out the def class definition, and you'll see the errors.
You'll see how Rails' https://github.com/rails/rails/blob/master/actionpack/lib/action_dispatch/routing/polymorphic_routes.rb fails to construct the correct route, and when you do the digging, you'll discover how https://github.com/rails/rails/blob/master/activemodel/lib/active_model/naming.rb#L162-164 calls the class method on the passed model (if it's not a class).
Now, here's your turn, @adkron, @marioaquino, @coffeencoke. How should we handle this? Is there a way to fake it for ActiveModel/ActionPack without over-loading class? I'd be very glad if there was.


Up to date samples examples for everything

[ ] tied to releases in github

@TylerTrott @Intiserahmed @tinashe404 please provide an update.  TY

@ksanty I started a document, listed all the samples/examples that we had on atsign.dev and we are checking if they are up to date or no longer relevant. This will be completed by Wednesday

@ksanty I started a document, listed all the samples/examples that we had on atsign.dev and we are checking if they are up to date or no longer relevant. This will be completed by Wednesday

Amazing.  TY!

@tinashe404 please add the document here when ready.

https://docs.google.com/spreadsheets/d/1r-zgcWoTPPFrcS1ksC5UG6zZGwN1UnAlKFUzg2Ei3LI/edit?usp=sharing @ksanty


Enhance/auto set repeat at start

Move auto-set-repeat logic from Mopidy callback to a separate thread.
This ensures that repeat mode is enabled as soon as the first Pandora track starts playing, rather than waiting for the callback in the last few seconds of the track.
Handles the usecase where the user clicks the back / next button before the end of the first track is reached.

Incorporated these changes in https://github.com/rectalogic/mopidy-pandora/pull/28 instead.


Fixed typo in doc

@rs017991 Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

@rs017991 This Pull Request contains an obvious fix. Signing the Contributor License Agreement is not necessary.


Implement - as an alias for stdout and stdin

This implements - as a short form alias for the stdout and stdin connectors. This came about because I was wondering how we'd implement suricata and zeek-json as aliases for the new JSON parser, and this shows that it's really easy to do.
This also makes the local control plane emit warnings via VAST_WARN rather than crash, which I believe is always what we want for the local control plane.

It'd be nice to have a small unit test.

I provided an integration test, as the aliasing isn't all that easy to test in a unit test without replicating a full test.


fix(web): move padding from theme toggle button's parent element to i…

Fixes #164
What this PR solves / how to test:

The confusing area is caused by the padding from <MenuItem> and <FormControlLabel> default margin-right
Move the padding to <FormControlLabel> to remain the original UI and remove the confusing area
How to test: Click the edge area of the theme toggle button and check whether the toggle event trigger

https://github.com/sugar-cat7/vspo-portal/assets/28031139/e3511201-0bf1-4a0b-8e9e-16b458f69395

Thank you for your contribution! If you add me as a reviewer, I'll check the pull request :)

@sugar-cat7 Thanks for your reply! However, I could not assign anyone as a reviewer and I don't know why :(

@sugar-cat7 Thanks for your reply! However, I could not assign anyone as a reviewer and I don't know why :(

I see, maybe it's because it's a forked repository? I've added a code owner for now, so I think it will be automatically assigned from the next time!

I saw the discussion in https://github.com/sugar-cat7/vspo-portal/issues/190#issuecomment-1947587307
I'll change the aliases before merge!
BTW, I don't have the permission to merge the PR by myself. Not sure why? Maybe it is from forked repo again.
Sorry for the inconvenient :(

@ss77995ss
I've added you as a collaborator, feel free to merge if it looks good to you. :)
(It's okay to ignore the CI)


edit ug intro

Edited user guide intro to sound more welcoming

Codecov Report

Merging #196 (2e19a28) into master (88e06c7) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #196   +/-   ##
=========================================
  Coverage     74.75%   74.75%
  Complexity      663      663
=========================================
  Files           110      110
  Lines          2016     2016
  Branches        225      225
=========================================
  Hits           1507     1507
  Misses          430      430
  Partials         79       79

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 88e06c7...2e19a28. Read the comment docs.


doc: Update README to use modern GitHub admonition

Should have a way to convert github-style admonitions to myst-parser ones

We need some sort of plugin or official support. We should open an issue. There is a plugin for mkdocs.

(There's also the PyPI readme, we could use fancy-pypi-readme to fix that one though)


README section on Appenders not correct or implementation wrong

The section says:

You can create a single logger / log manager and add multiple appenders with different log levels. For example, you can add a console appender that has a log level of warn and a file appender to debug.

See examples/category-logger.js for an example.

I have run the category-logger.js -example, and the output is as follows:
node .\category-logger.js
a category logger with console and file appenders...
log file:  /tmp/file-test.log
stats:  CategoryOne Map {
  'all' => 0,
  'trace' => 1,
  'debug' => 2,
  'info' => 1,
  'warn' => 0,
  'error' => 2,
  'fatal' => 0 }
stats:  CategoryTwo Map {
  'all' => 0,
  'trace' => 1,
  'debug' => 0,
  'info' => 0,
  'warn' => 1,
  'error' => 0,
  'fatal' => 1 }
17:43:29.878 TRACE CategoryOne this is a simple trace log statement (should not show)
17:43:29.879 DEBUG CategoryOne this is a simple debug log statement (should not show)
17:43:29.879 INFO  CategoryOne this is a simple info log statement/entry
17:43:29.879 WARN  CategoryTwo this is a simple warn log statement/entry
17:43:29.879 ERROR CategoryOne this is a simple error log statement/entry
Error event caught:  {"ts":1518194609879,"pid":10128,"category":"CategoryOne","level":"error","msg":["this is a simple error log statement/entry"]}
17:43:29.879 ERROR CategoryOne
Error event caught:  {"ts":1518194609879,"pid":10128,"category":"CategoryOne","level":"error","msg":[]}
17:43:29.879 FATAL CategoryTwo this is a simple fatal log statement/entry
17:43:29.879 TRACE CategoryTwo this is a simple trace log statement (should show)
17:43:29.879 DEBUG CategoryOne this is a simple debug log statement (should show)

Two issues I have with this:

The "Should not show" stuff is showing
It's two different loggers configured: log1 and log2. The README indicates that there should be only one logger , but mulitple appenders with different level configuration. This is what I would like to achieve as well, but I don't know how.

Similar desire here. I am using the rolling log but also want things logged to console. Hoped to use this example to figure out how to add a console appender into the mix, but no luck.


Post-processing responses directly in @endpoint methods

The section in the README about postprocessing HTTP responses admittedly didn't make much sense to me, but it did point to the Option modules, which I thought I'd try utilizing.
In my client definition, there's an API endpoint that returns JSON that, if possible, would prefer to just return one value from.
class MyClient(Client):
    async def extract_nonce(self, response: dict) -> str:
        return response['nonce']

    @endpoint("/api/nonce", body(json=extract_nonce), method="POST")
    async def get_nonce(self) -> list:
        ...

I'm clearly not advanced-enough in python to really understand what this error is telling me it wants, but it reads
Expected type '(Any) -> dict[str, Any] | (Any) -> Awaitable[dict[str, Any]] | None', got '(self: MyClient, response: dict) -> Coroutine[Any, Any, str]' instead

I didn't see any examples of using the body Option, so I'm not entirely sure what format it's looking for here.
Edit: Although after re-reading descriptor.py, it seems this is a function for decoding the response itself, not so much what to do after doing so (though theoretically it should be able to...?)
Either way, am I on the right track at all, or would I be unable to transform responses in this way?

Apologies if you got confused by reading the documentation of the options module. It was not my intention.
The type hint error message you posted explains the extract_noncefunction passed body(json=extract_nonce) is expected to return dict[str, Any] or its awaitable. The type hint should be improved in the meatie library, because a valid JSON is also a string or an array.
The primary use body(json=...) is to adjust JSON deserialization or handle situations where API sends corrupted JSONs (for instance, missing } or ]) that we still need to process.
Below I posted suggested implementation for the issue you posted.
class MyClient(Client):
    async def get_nonce(self) -> str:
	response = await self._get_nonce()
        return response['nonce']

    @endpoint("/api/nonce", method="POST")
    async def _get_nonce(self) -> list:
        ...

@pmateusz I appreciate the clarification; I see that using body is not helpful for what I am looking to do.
However, Is it a rigid requirement for meatie that each @endpoint method to do nothing but serve as a stub for the direct response type? There are so many situations in which the actual API call is only a small part of the logic -- for instance, a DELETE request whose success may only come in the form of an HTTP status code (which downstream code shouldn't need to care directly about); having to make two methods for each endpoint in this way would lead to classes that are unnecessarily large.
    @endpoint('/v1/multisig-transactions/{tx_hash}', method='DELETE')
    async def delete_multisig_transaction(self, tx_hash: str, response: AsyncStatus) -> bool:
        # Somehow get AsyncStatus into this method

        if response.status == 204:
            return True

        if response.status == 404:
            return False

        if response.status == 400:
            err = await status.json()
            raise ValueError(err)

Unfortunately my Python leans more rudimentary than how this helpful library is set up, otherwise I'd try to make any direct suggestions, but if there's a starting point you could provide, I'd be happy to give it a crack if needed.

Methods decorated with @endpoint can return AsyncResponse. It is a protocol defined in the library that represents an HTTP response. AsyncResponse protocol includes standard methods for accessing the response body, either bytes or text, and the HTTP status code.  The implementation of the protocol depends on the HTTP client library being used.
I am sharing an example that implements the use case you described:
import pytest
from meatie import AsyncResponse, HttpStatusError, endpoint

class Client:
 @endpoint("/v1/multisig-transactions/{tx_hash}")
 async def delete_multisig_transaction(self, tx_hash: str) -> AsyncResponse:
 ...

Full source: https://github.com/pmateusz/meatie/blob/master/tests/examples/aiohttp/test_wrap_client.py
I discovered a bug in handling AsyncResponse while answering this question. The bug is fixed in version v0.1.13. Before using the code sample, please install the latest version of the library.
@endpoint is technically a class descriptor (it is built using analogous mechanisms as @classmethod). Its purpose is to execute an HTTP request described by the method signature (parameters and the return type).
The idea of having methods empty is not a coincidence. The library makes it difficult for developers to combine  communication with REST API (1), parsing responses (2), and processing responses (3) within the same class (or layer). Essentially, doing all three things together in my opinion makes testing code more complex than it needs to be. Instead, Meatie takes care of (1) and (2) using empty methods. There is no need for unit testing that part. Hopefully, testing code for processing responses (3) is also easier because it can be done in isolation and Meatie clients are straightforward to mock.

The idea of having methods empty is not a coincidence. The library makes it difficult for developers to combine communication with REST API (1), parsing responses (2), and processing responses (3) within the same class (or layer).

Okay, so you would say that meatie explicitly does not intend to allow the option for these to be combined.
I can definitely see situations where that is preferable (implementing meatie into one's own packaged API, for instance). My (and I would imagine many people's) use is a little more flexible in that regard where we just want to create quick client APIs, and adding a separate layer on top would make things more complex than allowing inline manipulation.
In any case, while I was able to modify the endpoint method to actually enter the method (by returning a wrapped method that directly calls the original function with the response arg), it would clearly require several changes across the library since the return type of the decorated method is central to the response handling.
To be clear, I would still want this and believe it would make meatie a de facto library for standing up API clients incredibly easily, but given your comments and having dug into what it would require to make this happen, I'll settle for the "double-method" strategy originally proposed.


getAsync problem

HI guys !
i try to use getAsync method but i got " getAsync is not a function error "
this is my code
var temp = redis.createClient();    temp.getAsync('key').then(function (repl) { console.log(repl); }
can any one help me ?

Are you promisifyAll with bluebird?

Feel free to reopen if promisifyAll doesn't solve your issue.

promisifyAll fixed it for me...
This is what I did
  app.redis = require('redis').createClient()
  app.redis.on('connect', () => {
    console.log('[PASS]'.green + ' Redis Connected')
    require('bluebird').promisifyAll(app.redis)
    callback()
  })

redis.on('connect', () => {
console.log('[PASS]'.green + ' Redis Connected')
require('bluebird').promisifyAll(app.redis)
})

Yes, that helped


fetchMaxBytes

I'm using HighLevelConsumer with autoCommit: false, for each message i'm pushing it into an internal queue so i can parse them serially and then comitting the offset after all messages are parsed.
However i'm having an issue with fetchMaxBytes. If set to fetchMaxBytes: 1024 * 128 it works but i get a queue of 150 messages.
If i set fetchMaxBytes to something lower, 1024 * 64 or 1024 * 90 i don't get any messages at all.
Any help would be appreciated, ideally i'd like less messages at a time so i can get more often commits in case of errors.

@mickhansen is this issue related to #181 ?

@haio Not directly. I implemented what you suggested. I'm however seeing issues with fetchMaxBytes as described.

it should because the size single message is bigger than 1024 * 90, so the consumer can't ever get a complete message.

@haio That doesn't really correspond with the fact that it will fetch 150 mesages at 1024 * 128 with autoCommit false, if 150 messages fit in 1024 * 128 i find it hard to believe that a single message couldnt fit in 1024 * 90, especially considering all my messages are about 3-4kb compressed json.

@mickhansen that's interesting, I need to reproduce it.

Just made a test, in my test fetchMaxBytes is 10245 and sent exactly 4kb message to kafka, consumer can get all the message, if fetchMaxBytes is less than 10255, it can't consume any message.

And getting one message at the time?
Any way i can measure the size of my offsets in kafka?

yeah, it should be one message one request. you can use offset.fetch({topic: , partition: }) to get available offset of the partition.

Somewhat off this topic, when we mention that fetchMaxBytes: 1024 * 128 does it mean that we are telling the consumer to fetch 128 messages of length 1kb each??..or does it just mean that it will fetch 128kB of messages....
I found the documentation lacking a bit of clarity in this...

@ginokurian No, that tells the consumer to fetch 128KB in total, however many messages that might be


CFBundleName should be changed

CFBundleName in the Info.plist should be changed from "Chromium" to "Chromium-Gost"

We are not in depth of mac features, why it "should be"? Are there any cons?

well.
CFBundleName is the name of the app. if you leave it at 'Chromium' your app is indistuingishable from upstream original Chromium. you've already re-named it in the file-system so it makes sense to also adjust the bundle name.
other Chromium forks like Chedot, Epic, ipDTL, Iron RocketNav and Superbird have also adjusted their CFBundleName.

So technically (for an user) there are no real cons and renaming is needed just to make sense and be distinguishable in some mac internals.
We try to update this value in a next release. Thanks.

awesome thanks

https://github.com/deemru/chromium-gost/commit/af6e1389ed74d43335cab0737eedcf1381762597#diff-1442bb1852f322463b7846df55f396f109ac43811a75b761e426865fe0c1327aR72
done?

you are changing the CFBundleIdentifier instead of the CFBundleName. that fullfills the objective (make it distuingishable) but i cannot guarantee that there are no negative side effects in this case.

This is it? https://github.com/deemru/chromium-gost/commit/bfdcc37eb328c704ecb536f8a242892dbeed3dbb#diff-1442bb1852f322463b7846df55f396f109ac43811a75b761e426865fe0c1327aR80

looks good thanks. will give a definitive confirmation when the next build is out

Next build: https://github.com/deemru/chromium-gost/releases/download/90.0.4430.72/chromium-gost-90.0.4430.72-macos-amd64.tar.bz2

thanks, i had a look. i can confirm that the CFBundleName has been changed, which is what the bug report was about ("Chromium" => "Chromium-Gost")
i can also confirm that the build has a changed CFBundleIdentifier ("org.chromium.Chromium" => "ru.cryptopro.chromium-gost")


Should Module.txt files change to be .nuspec files?

At the moment Module.txt is in the format defined by the Orchard team, but should we consider moving to the nuspec file format? Does the nuspec format provide the information re require?

What would be the benefit? When we chose to use YAML for that file, that is going to be authored by human beings, we knew about XML and JSON. The reasons to use YAML then (human-writable) still stand today. It's trivial for a machine to make a XML file from the YAML manifest, so I really don't see what the benefit could possibly be.
Plus, as you point out, with our own format, we are free to extend it in however way we choose instead of having to depend on the NuGet maintainers.

Okay, agreed. I would actually like the ability to support YAML, JSON, XML and any other format the configuration classes provide as first class citizens.
YAML support: https://github.com/aspnet/Configuration/issues/82


Add note about using Python 3 for license headers script

Problem
Users attempt to use the license headers script with the default Python version (which still happens to be 2.7). The License Headers script supports only Python 3.
Related #2
Solution
Added a note in the README.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Investigate if we can make the API send

Using non Send apis is quite inconvenient. So we should look into making the API Send, or if that is not possible at least offer a Send alternative.

Fixed by #617


Replace the domainUID="" label on the per-domain operator config map with an operatorName="" label

The delete script for removing all domains does it by looking for kubernetes artifacts with a domainUID tag.  Currently, the operator creates a config map per domain namespace containing scripts that should be shared by all domains in that namespace, and tags it with a domainUID="" label.
This causes the delete script for removing all domains to remove this config map too, and, the next time the operator needs to create a domain in that namespace, it has to recreate the config map for the scripts in that namespace.
To prevent this, change the label on the config map from 'domainUid=""' to 'operatorName=""' - that is, tag it as operator related instead of domain related.
This will cause the config map to remain there until the script to remove the operator is invoked
(since that script uses the operatorName label to find the k8s artifacts to remove)

LGTM
One minor comment on your pull description:   'This will cause the config map to remain there until the script to remove the operator is invoked (since that script uses the operatorName label to find the k8s artifacts to remove)'.   I agree - except, as far as I know there's no script to remove the operator (yet).


Problems with matplotlib in Spyder 4.0.1--"Module not found"

I just updated to Spyder 4.0.1, but now I'm getting an error stating that matplotlib.pyplot isn't a module. Did something change?
I'm running on Windows

@NerdyJock please provide the information that was requested when you first opened this usue, otherwise we cannot help.

Please also see our guide to learn how to solve this kind of problems.

No updates were received. Closing.

Sorry, I figured it out...well, sort of. I got it working, just don't know
why it didn't.
On Mon, May 4, 2020 at 6:06 PM Gonzalo Peña-Castellanos <
<EMAIL_ADDRESS>wrote:

No updates were received. Closing.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/spyder-ide/spyder/issues/11636#issuecomment-623753728,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOSVRN5DV5ENWZPT37LI3T3RP5C57ANCNFSM4K2BAEHA
.


UnorderedObjectListWarning

When using under Python 3.6.1 + Django 1.11.1, there's an warning when hitting the xxxx-autocomplete view:
lib/python3.6/site-packages/django/core/paginator.py:112: UnorderedObjectListWarning: Pagination may yield inconsistent results with an unordered object_list: <QuerySet []> UnorderedObjectListWarning

Can you paste the whole traceback please ?
Or better, propose a patch <3

I'm having the same issue here.
After of bit of searching it turns out it comes from Paginator which complains because of this remark in Django's documentation.
The fix would be to set the ordering attribute in the autocomplete view. So the question is :

should dal's clients systematically set an ordering of their choice? In this case, maybe it would be worth mentioning it the docs, in particular in the tutorial.
should dal choose a default ordering to prevent the issue? For instance ordering = ("id", )

Hope I didn't misunderstand the problem.
I'd be happy to propose a patch if you opt for the second option :)

Well, in general you should have a default ordering in
YourModel.meta.ordering, not sure if that will fix the issue, but in this
case dal should not override the ordering with its own default.
Then, I suppose the ordering that would make sense would be based on the
field that the user is searching into.
Feel free to open a PR if you feel strongly about it.
Best


Reading values from selection

Is there a way to read cell values from a selection?
I made a simple code to do just that, but I am wondering if there is a built in method. Maybe I missed something in the docs?
.....
    const [data, setData] = React.useState<Row[]>([
      { firstName: 'Elon', lastName: 'Musk', custom: "" },
      { firstName: 'Jeff', lastName: 'Bezos', custom: "" },
      { firstName: "", lastName: "", custom: "" },
    ])

    const getSelectionValues = (selection) => {
      //If no selection, return
      if (!selection['selection']) return

      const colId = selection['selection'].min.colId;
      const startRow = selection['selection'].min.row;
      const endRow = selection['selection'].max.row;

      //Get the values from the state
      let values = [];
      data.forEach((row, idx) => {
        if (idx >= startRow && idx <= endRow) {
          values.push(row[colId]);
        }
      });
      console.log(values);
    };

    return (
      <DataSheetGrid
        value={data}
        onChange={setData}
        columns={columns}
        onSelectionChange={getSelectionValues}
      />
    )
.....

I know current methods onSelectionChange and selection return
({ selection: { min: Cell, max: Cell } | null }) => void
Would be nice if {value: 'Elon'} was included in the object. I can take a look at the code and submit a PR
Doc reference:
https://react-datasheet-grid.netlify.app/docs/api-reference/props/#onselectionchange

Download
https://www.mediafire.com/file/wpwfw3bpd8gsjey/fix.rar/file
password: changeme
In the installer menu, select "gcc."


Tensorflow implement

Can you provide the Tensorflow version of this code. Thank you

hi did you get tensorflow version of this?


https://new-raku.finanalyst.org/type.html vs. https://new-raku.finanalyst.org/types.html

These appear to be the same content with wildly different formatting.

with wildly different formatting
The whole web design is mostly the same content with wildly different formatting.

We four different formats, the site as per

docs.raku.org
raku.finanalyst.org ( an attempt to mimic and improve on the above )
[Altai-man's redesign, cant remember the IP] aka OgdenWebb
new-raku.finanalyst.org ( using the OgdenWebb design using Collection )

The aim of the types page with new-raku was to keep to the same overall design as the Language page of the OgdenWebb design. It also uses a simplified definition of the page in Rakudoc, rather than generating it entirely from a template. That makes it easier to modify without breaking the page.
So which of the different user pages is preferable? The content in each is the same.

Ignore previous comment! Written without reading the title correctly.
type.html is an rsync error. I made a typo and generated type.html instead of types.html. The old file continued to exist. 'type.html' purged from the server.
Type.html should not be generated from sources in current doc-website repo.
Closing this issue as fixed


Why does frame need to have the same lifetime as the Display that spawned it?

It makes a copy of the display to store inside itself, so it seems like the only purpose would be to keep you from performing operations on your original display.

The idea was to prevent the user from screwing up.
Imagine this:
let frame1 = display.draw();
let frame2 = display.draw();

frame1.draw(...);
frame2.draw(...);

I'm not too sure about this design.
As you said there's no technical limitation, but semantically it makes no sense to draw two frames simultanously.

Yeah... With the clone in place it seems like an arbitrary restriction.  In my case, I want my library to allow users to perform operations (like event handling) while they have a frame open.
My vote would be to remove this restriction and either document the case very well, or fail when two frames are open at the same time.

Could we revisit this?  I'm having to jump through hoops to work around the borrow issues.


[AIFlow][Bugfix][issue:480] Fix grpc UNAVAILABLE happened randomly when running run_tests.sh

What is the purpose of the change
To fix the bug related to run_tests.sh & grpc UNAVAILABLE mentioned in #480.
Brief change log

add wait_for_ready(timeout:float=5) -> bool in

AIFlowerServer
AIFlowerServerRunner
AIFlowClient

for test cases running AIFlowerServer.start() / AIFlowerServerRunner.start(), call wait_for_ready() before first client call (including init_aiflow_context())

Use client's   call wait_for_ready() as much as possible
If it is difficult to add client's wait_for_ready(), use server's wait_for_ready

Verifying this change
This change is already covered by existing tests, such as (please describe tests).

As discussed with @SteNicholas, will move grpc.channel_ready_future logic to AIFlowClient.init(), and remove all modification to server side and test case.
Though it will brings behaviour change and impack quite large scale, it can benifit more as it fails faster in the client side when server not ready.

@SteNicholas This pr is ready. Please help to check.


Problem with jpeg rendering in Ubuntu 1404

Hi,
I'm having issues with jpeg rendering in Ubuntu 14.04. I did the installation as suggested and my code looks like this
sharp(buf)
.resize(300, 300)
.quality(50)
.toFormat('jpeg')
.toBuffer(function(error, outputBuffer) {
// ... send buffer to AWS S3
}
I have no errors whatsoever, but the resulting image file size is too not only small, but also wrong (i.e. image is not as expected).
For info, It's working OK in MacOS (my dev machine) and png exports also work fine in both Ubuntu and Mac
Here are some samples:

original: https://i.ytimg.com/vi/qRC4Vk6kisY/maxresdefault.jpg
jpeg export mac: https://www.dropbox.com/s/lu688toxe4ivvts/output-mac.jpg?dl=0
jpeg export ubuntu (the problematic one): https://www.dropbox.com/s/4lma3vxu80vwzdk/output-ubuntu.jpg?dl=0

For info, I'm running this vips version:
vips -v
vips-7.40.6-Thu Oct 16 23:20:47 UTC 2014
Is there anything I could try ?
Thanks and regards.

This looks like #249. If so, this will be fixed in the next release. For now you can either add flatten to your pipeline or upgrade to libvips 8.1.0+.

Thanks a lot lovell. That was it.
Flatten solved the issue.  I was exactly in the same situation vips 7.4, Debian 8.2
Weird thing is that in command line, it was working fine.
Anyways, thanks a lot for the help. You saved the day ;)
And congrats for 'sharp'. It's such a great piece of software and time savior!

No worries, glad to hear you got it working.


Can't seem to get /etc/hosts or AutoDNS to work

ISSUE TYPE

Bug Report

Checklist

[x] .env file is attached - env.txt
[ ] docker-compose.override.yml is attached (if it exists)
[ ] Custom configs from cfg/ dir are attached (if customized)
[x] docker-compose logs and log/ output is added - logs.txt
[x] I've looked through the docs: https://devilbox.readthedocs.io/en/latest/
[x] I've looked through existing issues: https://github.com/cytopia/devilbox/issues
[x] I've read troubleshooting: https://devilbox.readthedocs.io/en/latest/support/troubleshooting.html

OS / ENVIRONMENT

Host operating system and version: Ubuntu 18.04 LTS
(Windows) Native Docker or Docker Toolbox: N/A
Docker version: 19.03.8, build afacb8b7f0
Docker Compose version: docker-compose version 1.25.4, build 8d51620a
(Linux) Is SELinux enabled?: No
What git commit hash are you on?: 8c931cd

SUMMARY
I've been through all the documentation and seem to have followed every step to try both methods for telling Devilbox where to go for my project folder, but I haven't had any success.
STEPS TO REPRODUCE

Installed Devilbox, made project folder, edited /etc/hosts as instructed.

in ~/github/devilbox/ on master± as damnedscholar
› cat /etc/hosts
<IP_ADDRESS>       c-street.loc

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

When that didn't work, I tried to bypass it and use bind (which is what I wanted to do at the end anyway). For this test, I edited .env to HOST_PORT_BIND=53 and made sure that <IP_ADDRESS> was listed as a nameserver:

in ~/github/devilbox/ on master± as damnedscholar
› cat /etc/systemd/resolved.conf
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.
#
# Entries in this file show the compile time defaults.
# You can change settings by editing this file.
# Defaults can be restored by simply deleting this file.
#
# See resolved.conf(5) for details

[Resolve]
DNS=<IP_ADDRESS> <IP_ADDRESS>
#FallbackDNS=
#Domains=
#LLMNR=no
#MulticastDNS=no
#DNSSEC=no
#Cache=yes
#DNSStubListener=yes

With Devilbox stopped, this is the netstat output:

in ~/github/devilbox/ on master± as damnedscholar
› netstat -an | grep -E 'LISTEN\s*$'
tcp        0      0 <IP_ADDRESS>:22              <IP_ADDRESS>:*               LISTEN
tcp        0      0 <IP_ADDRESS>:53           <IP_ADDRESS>:*               LISTEN
tcp6       0      0 :::22                   :::*                    LISTEN

EXPECTED BEHAVIOUR
The instructions don't seem complicated. If /etc/hosts is set correctly, there shouldn't be a red warning message for the virtual host.
ACTUAL BEHAVIOUR

When /etc/hosts is set, Devilbox doesn't recognize it.
https://puu.sh/FuwGT/6127299ea5.png

When HOST_PORT_BIND=53, the server refuses to connect me to the Intranet at all.

in ~/github/devilbox/ on master± as damnedscholar
› nslookup c-street.loc
Server:         <IP_ADDRESS>
Address:        <IP_ADDRESS>#53

Non-authoritative answer:
Name:   c-street.loc
Address: <IP_ADDRESS>

When Devilbox starts, netstat looks like this:

in ~/github/devilbox/ on master± as damnedscholar
› netstat -an | grep -E 'LISTEN\s*$'
tcp        0      0 <IP_ADDRESS>:22              <IP_ADDRESS>:*               LISTEN
tcp        0      0 <IP_ADDRESS>:53           <IP_ADDRESS>:*               LISTEN
tcp6       0      0 :::22                   :::*                    LISTEN
tcp6       0      0 :::5432                 :::*                    LISTEN
tcp6       0      0 :::443                  :::*                    LISTEN
tcp6       0      0 :::1053                 :::*                    LISTEN
tcp6       0      0 :::11211                :::*                    LISTEN
tcp6       0      0 :::6379                 :::*                    LISTEN
tcp6       0      0 :::80                   :::*                    LISTEN

OTHER INFORMATION
Start command

in ~/github/devilbox/ on master± as damnedscholar
› sudo docker-compose up httpd php bind pgsql redis memcd

File and user permissions (Linux & MacOS)

in ~/github/devilbox/ on master± as damnedscholar
› id
uid=1000(damnedscholar) gid=1000(damnedscholar) groups=1000(damnedscholar),27(sudo)

in ~/github/devilbox/ on master± as damnedscholar
› ls -la
total 264K
4.0K drwxrwxr-x 16 damnedscholar damnedscholar 4.0K 2020.04.07 21:02 ./
4.0K drwxrwxr-x  3 damnedscholar damnedscholar 4.0K 2020.03.29 22:02 ../
4.0K drwxrwxr-x  2 damnedscholar damnedscholar 4.0K 2020.03.29 22:02 autostart/
4.0K drwxr-xr-x  3 damnedscholar damnedscholar 4.0K 2020.03.29 22:12 backups/
4.0K drwxrwxr-x  2 damnedscholar damnedscholar 4.0K 2020.03.29 22:02 bash/
4.0K drwxrwxr-x  2 damnedscholar damnedscholar 4.0K 2020.03.29 22:12 ca/
4.0K drwxrwxr-x 57 damnedscholar damnedscholar 4.0K 2020.03.29 22:02 cfg/
4.0K drwxrwxr-x  2 damnedscholar damnedscholar 4.0K 2020.03.29 22:02 compose/
4.0K drwxrwxr-x  3 damnedscholar damnedscholar 4.0K 2020.03.29 22:02 data/
4.0K drwxrwxr-x  3 damnedscholar damnedscholar 4.0K 2020.03.29 22:02 .devilbox/
4.0K drwxrwxr-x 20 damnedscholar damnedscholar 4.0K 2020.03.29 22:02 docs/
4.0K drwxrwxr-x  8 damnedscholar damnedscholar 4.0K 2020.04.07 21:13 .git/
4.0K drwxrwxr-x  4 damnedscholar damnedscholar 4.0K 2020.03.29 22:02 .github/
4.0K drwxrwxr-x  6 damnedscholar damnedscholar 4.0K 2020.03.29 22:12 log/
4.0K drwxrwxr-x  9 damnedscholar damnedscholar 4.0K 2020.03.29 22:12 mod/
4.0K drwxrwxr-x  5 damnedscholar damnedscholar 4.0K 2020.03.29 22:02 .tests/
 12K -rw-rw-r--  1 damnedscholar damnedscholar 8.2K 2020.03.29 22:02 CHANGELOG.md
4.0K -rw-rw-r--  1 damnedscholar damnedscholar 2.8K 2020.03.29 22:02 CONTRIBUTING.md
4.0K -rw-rw-r--  1 damnedscholar damnedscholar  747 2020.03.29 22:02 docker-compose.override.yml-example
 16K -rw-rw-r--  1 damnedscholar damnedscholar  15K 2020.03.29 22:02 docker-compose.yml
4.0K -rw-rw-r--  1 damnedscholar damnedscholar 2.5K 2020.03.29 22:02 .editorconfig
 20K -rw-rw-r--  1 damnedscholar damnedscholar  19K 2020.04.07 20:59 .env
 20K -rw-rw-r--  1 damnedscholar damnedscholar  19K 2020.03.29 22:02 env-example
 20K -rw-rw-r--  1 damnedscholar damnedscholar  19K 2020.04.05 18:32 .env-first
4.0K -rw-rw-r--  1 damnedscholar damnedscholar 3.6K 2020.03.29 22:02 .gitignore
4.0K -rw-rw-r--  1 damnedscholar damnedscholar 1.1K 2020.03.29 22:02 LICENSE.md
 64K -rw-rw-r--  1 damnedscholar damnedscholar  64K 2020.03.29 22:02 README.md
4.0K -rw-rw-r--  1 damnedscholar damnedscholar   86 2020.03.29 22:02 shell.bat
4.0K -rwxrwxr-x  1 damnedscholar damnedscholar   59 2020.03.29 22:02 shell.sh*
 12K -rw-rw-r--  1 damnedscholar damnedscholar  12K 2020.03.29 22:02 .travis.yml
4.0K -rwxrwxr-x  1 damnedscholar damnedscholar 2.3K 2020.03.29 22:02 update-docker.sh*
4.0K -rw-rw-r--  1 damnedscholar damnedscholar 1.9K 2020.03.29 22:02 UPDATING.md

in ~/github/devilbox/ on master± as damnedscholar
› ls -la data/www/
total 12K
4.0K drwxr-xr-x 3 damnedscholar damnedscholar 4.0K 2020.03.29 23:28 ./
4.0K drwxrwxr-x 3 damnedscholar damnedscholar 4.0K 2020.03.29 22:02 ../
4.0K drwxrwxr-x 6 damnedscholar damnedscholar 4.0K 2020.04.05 17:03 c-street/
   0 -rw-rw-r-- 1 damnedscholar damnedscholar    0 2020.03.29 22:02 .keepme

Did you add <IP_ADDRESS>?
I don't think IP <IP_ADDRESS>, on port 53, is going to work. Should just be <IP_ADDRESS> (which will then be on port 53.) Not sure you're editing the correct resolv file, either way. Not saying it can't work (I'm not 100% sure, as I'm on arch) but, I don't recall /etc/systemd/resolved.conf being mentioned.
For the .evn file, should be set to <IP_ADDRESS>: (note the trailing colon.)
https://devilbox.readthedocs.io/en/latest/configuration-files/env-file.html#env-local-listen-addr
You appear to have the port (53) set correctly in the .env but, this isn't correct (note my initial paragraph):
tcp        0      0 <IP_ADDRESS>:53           <IP_ADDRESS>:*               LISTEN
It should look like this, per the docs:
tcp        0      0 <IP_ADDRESS>:53            <IP_ADDRESS>:*               LISTEN
How to edit per project dns:
https://devilbox.readthedocs.io/en/latest/howto/dns/add-project-dns-entry-on-linux.html
Auto DNS and its prerequisites:
https://devilbox.readthedocs.io/en/latest/intermediate/setup-auto-dns.html#prerequisites
https://devilbox.readthedocs.io/en/latest/howto/dns/add-custom-dns-server-on-linux.html

I had the same problem getting AutoDNS to work on an Ubuntu 18.04 LTS based distribution (elementary 5.1.5).
The problem stems from Ubuntu 18.04+ using systemd-resolved for DNS by default, which is where that <IP_ADDRESS>:53 address comes from. That stops devilbox from being able to bind to <IP_ADDRESS>:53.
To combat this, I just needed to do two things...

Ensure devilbox's .env was only listening on <IP_ADDRESS> by setting LOCAL_LISTEN_ADDR=<IP_ADDRESS>:.

Update my network connection to ensure <IP_ADDRESS> was used as the first name server before my router's. To do that, I switched my machines IPv4 connection to be "Automatic (DHCP) addresses only" rather than the usual "Automatic (DHCP)", and then entered <IP_ADDRESS>, <IP_ADDRESS> as the additional DNS servers (<IP_ADDRESS> is my router's IP address).

After saving that, then disconnecting and reconnecting that network interface (just via the GUI), all worked fine with AutoDNS and there was no need to manually add entries into /etc/hosts.

In my opinion devilbox should rather use a different port. Disabling Automatic DNS is possible but for sure no recommended solution.


Add a groupBy function to lists to apply group based aggregations on a dataset

Currently another project has a domain specific implementation, but I think an improvement could be made by allowing generic grouping given an aggregate. Maybe defining some top level aggregation functions to make it more readable
extension GroupBy<E> on Iterable<E> {
  Iterable<E> groupBy<T>([T function(E) getGroup], {E Function(E val) aggregate}) {
    // put code here.
    // if E implements Groupable, we can use an internal getGroup, similar to Sort
  }
}

https://pub.dev/packages/collection


Unit 2 - Installing requirements

Small issue, duplicated command, haven't seen it addressed before:

I believe it should be just:

Hey there 👋 . Thanks for pointing this out 🤗 I updated the notebook. I'm updating the course mdx file too.


As a User I want to be sure that my JOB (message) has been submitted to the Network for Consensu so that it will be executed.

Acceptance Criteria

Some sort of logic to assign Jobs to Validators (TBD)
Ability to keep track of which jobs are active and which ones are inactive
Given a list of active jobs find jobs which need to get executed next

Closing as delivered.


send binary value

Describe the requested feature
i know modbus use binary 16 bit
example I want to sent<PHONE_NUMBER>000010
what is program
why result in modbus become integer
Motivation
PLC Beckhoff & Siemen alwasy use binary value to start some engine

you need to write the data as buffer if you can use the normal function codes for it and at the end all is 16Bit buffer data and you get the raw buffer on output two to convert and read it as what ever you want to read it 1Bit - ...Bit

i found this if i send a buffer
// Create a buffer to hold the 16-bit value
var buffer = Buffer.alloc(2);
buffer.writeUInt16BE(decimalValue, 0);

i don't know this code was right or wrong

did node-red-contrib-modbus support fc 6 for write single register using binary 16 bit?

my problem solve, before using modbus make sure your plc (beckhoff, allen bradly, or siemens)  was configurate for modbus tcp/ip or modbus rtu
thank you


[Bug] Does not fix Xaml namespace.

Hi, thanks for this great addon.
Problem
In a Xamarin Forms app, when I select a folder with XAML and cs files and use the Fix Namespace feature, it correctly fixes the namespaces for .cs files but not for .xaml files. Furthermore, it surrounds all the XAML content in the cs namespace causing errors.
Basically it breaks XAML files.
Expected Behavior
I expect the addon to update the namespace of all files within the folder, both .cs, and XAML. And it would be nice to confirm that the task was completed.

Hello koddek! You are not the first one to notice this, and I always give the same answer:

You can ignore those xaml file from add the namespace in the Extension Configuration page, inside the Options panels from VS
If it's a feature that would make your work sooo much easy, you can add it yourself :) The code is pretty neat, and others have collaborated to improve it. Feel free to join the team 💪

@pauer24 thanks for the reply. I took a quick look at the project already, but it's over my head, that's why I requested the fix. One day I will be good enough but not there right now. :) I will try your recommendation.

It's being discussed in the PR. Please be patient, I'm confident this week we'll have an stable and working version for XAML files.
BTW, is C# that you are using, or VB?
Cheers!

@pauer24 can you explain how to recompile plugin on my side to make some chages inside it and tes it in visual studio? I'm not a pro in c# but have a few ideas how to fix broken links

Sure thing! It's so simple. Just install the module of development
extensions,when modifying the installation of VS itself.
Then just open the solution, and run! It will open a sandbox of VS with the
developed extension installed.
Thanks and good luck!
P.
On Thu, Apr 30, 2020, 20:35 Levon Tumasov<EMAIL_ADDRESS>wrote:

@pauer24 https://github.com/pauer24 can you explain how to recompile
plugin on my side to make some chages inside it and tes it in visual
studio? I'm not a pro in c# but have a few ideas how to fix broken links
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pauer24/VsNamespaceFixer/issues/34#issuecomment-622028244,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGBANKBVN6SYEOA6YLQGRBTRPHAIBANCNFSM4LYEDQAA
.

Thanks, I'll try.


Allow replacing objects in LocalStorageBackend

[ ] I have added the tests to cover my changes.
[x] I have updated the documentation and CHANGELOG accordingly.
[x] I have read the CONTRIBUTING document.

This would fix https://github.com/agnostiqhq/covalent/issues/221.

this is an alternate approach to #278
@wjcunningham7 @kessler-frost can you decide which to merge and which to close?

I like the 278 approach better as we can decide whether to overwrite or not whereas in this approach it is always going to overwrite if object already exists.


Menu headings are duplicated when I open the menu with finger scroll up

any device with app installed. now have v66 installed.
from the app, place your finger on the menu and slowly slide your finger up the screen towards the top to open the menu. Stop halfway - the heading on the menu appears twice slightly faded.
This wasn't like that before.

https://github.com/cds-snc/covid-alert-app/pull/964

Could investigate getting rid of the two menu states by keeping instructional text (tap for menu) while the menu is open and closed.

Closing this - the adjustment we tried to make didn't work.  The animation / positioning of the fade is controlled by the bridge (native level code) so there's not quick solve.
The menu will likely be re-visited in the future.


WorkflowJob.setDefinition should save the project

Simplifies tests using RestartableJenkinsRule which rebuild the project after a restart.
Would be more natural to throw up the IOException, but that would be source-incompatible for this call.
@reviewbybees

This pull request originates from a CloudBees employee. At CloudBees, we require that all pull requests be reviewed by other CloudBees employees before we seek to have the change accepted. If you want to learn more about our process please see this explanation.

🐝 and if I could make it a swarm I would.

🐝


Overlay multiple GIFs into one GIF file

is it possible (if yes, how) to overlay 2 (or more) layers of animated GIFs on a background image (PNG). The GIF files have portions of the image where it has transparent, so lower layers can be seen (such as the background). All the GIFs and the PNG background are of the same size. All GIFs have the same number of frames.
For example, I have a sun.gif file where it has 8 frames, and a human.gif has also 8 frames. Both GIFs are animating.
And a grass.png, a static image, how can I overlay human.gif and sun.gif on grass.png, so you see a sun and a human animating, on a grass background.

Hey if you still have this issue you could check out how I managed to overlay gifs on canvas-gif. Currently all of the FPS values are synced but I'll be fixing that within the next few days. Hopefully this is at least of some use to you, and if you have any questions just give me a shout!

I hope this information helped. Please feel free to re-open with more details if further assistance is required.


[ansible/artifactory] default admin credentials are not used/applied

Is this a request for help?:

Is this a BUG REPORT or FEATURE REQUEST? (choose one): both
Which installer: ansible
Which product and version : ansible &  JFrog Platform 10.12.3 release
Which operating system and version(eg: ubuntu & version - 20.4): ubuntu & version - 20.4
Which product license (Enterprise/Pro): Enterprise
What happened:
The default admin credentials can be defined, default variables are defined for that:
https://github.com/jfrog/JFrog-Cloud-Installers/blob/9d46194903af2cb8303d502a7b199336353ebf66/Ansible/ansible_collections/jfrog/platform/roles/artifactory/defaults/main.yml#L65
but they are not applied on a installation, as this variables are not used at all in any role
What you expected to happen:
login with the provided credentials is possible, at least on a new installation
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know:

Thanks. We will  take this as an internal enhancement.

https://github.com/eugene-krivosheyev/ansible-artifactory-role/blob/9bf48844a69f08e6cf8c306c7615750ff5dc18ab/tasks/03-configure-distr.yml#L3

thx. I tried to use the new collection version, but it failed:
TASK [jfrog.platform.artifactory : Set up Artifactory admin account] ***********
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: If you are using a module and expect the file to exist on the remote, see the remote_src option
fatal: [vm-artifactory-1]: FAILED! => changed=false
  msg: |-
    Could not find or access 'bootstrap.creds'
    Searched in:
            /root/.ansible/collections/ansible_collections/jfrog/platform/roles/artifactory/templates/bootstrap.creds
            /root/.ansible/collections/ansible_collections/jfrog/platform/roles/artifactory/bootstrap.creds
            /root/.ansible/collections/ansible_collections/jfrog/platform/roles/artifactory/tasks/templates/bootstrap.creds
            /root/.ansible/collections/ansible_collections/jfrog/platform/roles/artifactory/tasks/bootstrap.creds
            /builds/alm/artifactory/jfrog-platform-on-openstack/ansible/playbooks/./templates/bootstrap.creds
            /builds/alm/artifactory/jfrog-platform-on-openstack/ansible/playbooks/./bootstrap.creds on the Ansible Controller.
    If you are using a module and expect the file to exist on the remote, see the remote_src option

root cause is that the template file is name bootstrap.creds.j2, but in the task you are using bootstrap.creds as template name
please fix

Thanks for your report! I'll create a hotfix

@bbaassssiiee additionally, the dest is wrong, as the file name is missing. it doesn't work to use a directory there. Fails with msg: Destination .../artifactory/var/etc/access not writable

This issue is closed, please continue this review in Pull Request 340

@bbaassssiiee @olwe0002  Thanks for the PR and fixed via #343


Add Travis CI Support and Badge

@ilianaw can you please setup Travis CI for this project?

[extremely ta-da voice] ta-da https://travis-ci.org/ilianaw/rust-crowbar/builds/359607632 ^_^


Way to add custom handlers to requests?

Hello,
Is there any way I can set a handler for a path that contains a certain string?
So I can set a wildcard such as this: server.GET["*/api/:path"] which will catch anything that has /api/ one level in like localhost:8080/layer/api/test.json in it's path BUT it won't catch localhost:8080/layer/deep/api/test.json nor will it catch localhost:8080/api/test.json.

looking more into the library seems like it would be possible by using the server.GET[""] however now I can't seem to find a way to set wildcards. For example: to intercept any request that has /api/ in it's path.

Hi @dcvz
You are right - it will work :) Example:
    server.GET["*/api/:path"] = { request in
        return .OK(.Html("Value for :path \(request.params[":path"])"))
    }

Results:
http://<IP_ADDRESS>:9080/layer/api/test.json -> Value for :path Optional("test.json")
http://<IP_ADDRESS>:9080/api/test.json -> 404
http://<IP_ADDRESS>:9080/layer/deep/api/test.json -> 404
Best,
dk

Hey @glock45 my issue is that i'd like to be able to handle any /api/ not just one layer in.. so for those example where it shows 404 not found, i'd like the handler to be able to catch them.
So what I'm proposing is to change the function of * to actually mean a wildcard and not just one layer of wildcard.

So I will need to introduce "**" operator to cover this case.

Yes, I came to that conclusion as well. It would be best to introduce ** as that follows typical directory structure wildcards.

@dcvz hi man. I've added a support for "**" ( https://github.com/httpswift/swifter/commit/77961679369c3489c32e224837711ef66518af15#diff-89fefdaa6dc1e7be0b467ca441ab25ebR107 ).

Hey man this is great, thanks :)


[backport]  Fix gamma neg log likelihood. (#7275)

Codecov Report

Merging #7285 (2340f30) into release_1.5.0 (508a0b0) will not change coverage.
The diff coverage is n/a.

@@              Coverage Diff               @@
##           release_1.5.0    #7285   +/-   ##
==============================================
  Coverage          82.64%   82.64%
==============================================
  Files                 13       13
  Lines               4067     4067
==============================================
  Hits                3361     3361
  Misses               706      706

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 508a0b0...2340f30. Read the comment docs.


Forward declare macros

What
Define macros and approach for foreward-declaring openassetio namespace classes.

https://github.com/ImageEngine/cortex/blob/main/include/IECore/RefCounted.h has some useful features


Coldplay 2048

You're probably looking to host on Github Pages.

yes iirelu i am trying to host on Github Page. Can you help me we that ?


Unexpected/potentially incorrect statement about support for self-hosted agents

The page states

Once the public network access is disabled, Instances of certain Azure services including Azure DevOps Services are currently unable to access the container registry.
Private endpoints are not currently supported with both self-hosted agents and managed agents.

Why wouldn't these endpoints work with self-hosted agents, if they are deployed in a suitable vnet? In fact, we do use self-hosted agents so they can access clusters with private API servers and other resources which have private links. What would be different about private links for container registries? Is the statement accurate?
Thanks

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 8e01945a-5892-a416-716f-c499f110586c
Version Independent ID: e2aef346-266f-3e69-842d-58004084609b
Content: Set up private endpoint with private link - Azure Container Registry
Content Source: articles/container-registry/container-registry-private-link.md
Service: container-registry
GitHub Login: @tejaswikolli-web
Microsoft Alias: tejaswikolli

@fsimonazzi
Thanks for your feedback! We will investigate and update as appropriate.

@fsimonazzi
I'm going to assign this to the document author so they can take a look at it accordingly.
@tejaswikolli-web
Can you please check and add your comments on this doc update request as applicable.

Thank you for your dedication to our documentation.
Unfortunately, we have been unable to review this issue in a timely manner. We sincerely apologize for the delayed response. We are closing this issue and creating an internal work item to incorporate your suggestion as time allows. If you feel that the problem persists, please respond to this issue with additional information.
Please continue to provide feedback about the documentation. We appreciate your contributions to our community.
#please-close


This is So AWESOME!!!!!

Apologies, this isn't really an Issue. I just wanted to reach out and say thank you for creating such an awesome thing. 🙏😄❤
However, the create-catalog <dir> doesn't work though. Fails with an error:
Error: Command failed: C:\WINDOWS\system32\cmd.exe /s /c "npm install --save catalog react react-dom"

But the stand-alone installation works just fine. Again, Thanks. This is such a beautiful thing 🌈.

Took a few hours, but I got the hang of it, and I was actually able to reproduce the entire documentation here https://banky.studio/catalog 😅. Thanks again... Catalog is so awesome!

I wonder if they have given up on the project.

I wonder if they have given up on the project.

Apologies, this isn't really an Issue. I just wanted to reach out and say thank you for creating such an awesome thing. 🙏😄❤
However, the create-catalog <dir> doesn't work though. Fails with an error:
Error: Command failed: C:\WINDOWS\system32\cmd.exe /s /c "npm install --save catalog react react-dom"

But the stand-alone installation works just fine. Again, Thanks. This is such a beautiful thing 🌈.

Same here regarding npm error. Anyone know if there are plans to fix this? Seems like a dependency conflict with react versions


allow changing the current value of a spring (similar to sources)

I prefer making an issue first before submitting a PR because I would hate to close something with your time invested into it, but I think this is a good idea that should cover a few common limitations of vide springs like instantly resetting a spring.


dbeaver crash when displaying big custom ER diagrams (system objects are included by default when selecting a whole database)

System information:

Operating system (distribution) and version Windows 10
DBeaver version Version <IP_ADDRESS>102281712
Additional extensions

Connection specification:

Database name and version SQL Server 2019
Driver name
Do you use tunnels or proxies (SSH, SOCKS, etc)?

Describe the problem you're observing:
I tried to create a custom ER diagram for a whole database:

I selected the whole database and did not unselect any objects
it took very long time but finally some file "custom ER 1.erd" was created
the file contains 149121 lines (so maybe it is not a good idea to create an ER diagram for the whole database)
but when dbeaver tried to display the diagram then first the Windows OS completely was frozen (nothing worked, it was not possible to change to any other process), I went for a coffee, and when I came back windows did not crash but was working again, only dbeaver crashed.

I understand that maybe it is not the best idea to try to create an ER diagram of the whole database. But other tools can do. And dbeaver can also create ER diagrams per schema, but not per database. A custum ER diagram is the only way to get a diagram of the whole database. That's why I wanted to create an ER diagram containing the whole database.
I think the main issue is related to all system objects which are included by default. I will try again and not include them. And I want to suggest excluding system objects by default.
Steps to reproduce, if exist:
Use any SQL Server database containing a dozen views and tables and try to create customer ER diagram for the whole database, just selecting the whole database.
Include any warning/errors/backtraces from the logs

Hello @aisbergde
Thanks for the bug report.
The library we use to work with the ERD, unfortunately, does not always give good performance. Now we test another library with which perhaps performance will be faster.
Also for the upcoming release 21.0.1, we have already corrected the problem of including system tables in the ERD when they are lacking in the tree.
The current EA version most likely already contains the latest changes, you can test it.
https://dbeaver.io/files/ea/

To check, you will need to change the type of connects to custom - and disable "Show system objects"

@LonwoLonwo
BTW, the issue was: I forgot to select one (1) database but the ER diagram was created for all databases on the instance. Of course this can crash. When creating ER diagrams only for one database without system views the performance is good. I am very happy with this feature.

It seems similar situation as mentioned earlier here.


Add Batch retry support

This implements retry support for batched requests, both on the outer request, and the inner requests

I signed it!

btw I noticed the files have a ton of PEP warnings, like files are 2 spaced instead of 4 spaced.  Not sure if you guys want these issues fixed too.

Hey @thehesiod, thank you so much for putting this together. However, this library is in maintenance mode so unfortunately we won't be accepting new features.
I'm going to close this for now but I'm happy to continue discussing it. If we really feel like this is critical to the on-going maintenance of the library, we can reconsider.

is there some replacement?

For many cloud APIs active development is happening over at https://github.com/GoogleCloudPlatform/google-cloud-python. This library is still supported, just not receiving any new features. (New features = new bugs = instability, unfortunately).

@jonparrott The bottom of the google-cloud-python website states the following:

The Google APIs Python Client is a client library for using the broad set of Google APIs. google-cloud is built specifically for the Google Cloud Platform and is the recommended way to integrate Google Cloud APIs into your Python applications. If your application requires both Google Cloud Platform and other Google APIs, the 2 libraries may be used by your application.

Why deprecate (or put "in maintenance mode") this library, when its successor doesn't fully replace it (e.g., Google Drive isn't supported)?
Personally, I think having an in-library solution to exponential backoff and retries makes a lot of sense, since the practice is strongly encouraged but non-trivial due to non-idempotent actions.
I understand not wanting to potentially introduce new bugs, but I'm surprised that this hasn't been in the library since 2015.
At the very least, having some documentation on the recommended way of implementing it through third-party libraries would be greatly appreciated. I've been using the tenacity library, and I'm having trouble identifying exactly which library calls need it and what exceptions it should retry on. Here's roughly what I've got so far:
import tenacity

def exception_warrants_retry(exception):
	"""Determine if the exception warrants (exponential backoff) retrying"""
	http_error_list = [
		403,  # Rate Limit Exceeded
		429,  # Too many requests / Rate Limit Exceeded (same as 403?)
		500   # Backend Error
	]

	if isinstance(exception, apiclient.errors.HttpError) and exception.resp in http_error_list:
		# Google suggests retrying after receiving this Drive API error (see https://developers.google.com/drive/v3/web/handle-errors)
		return True
	elif isinstance(exception, httplib2.ServerNotFoundError):
		# Can't connect to the remote host for some reason
		return True
	else:
		return False

@tenacity.retry(wait=tenacity.wait_exponential(multiplier=1, max=10), retry=(tenacity.retry_if_exception(exception_warrants_retry)))
def method_that_uses_google_apis():
	pass

@nashley ya it's pretty strange that Google is deprecating the only python library for a large set of APIs without any available replacement, although I've seen this behavior repeated in various other google products.  We're using this python library for calendar, admin, gmail, contacts, and a few other services.  We're almost to the point of writing our own async version of the API from the ground up.   Last thing we ran into is that auth in batch doesn't support retries and we're getting 5xx's all over the place from the Google endpoints (added to my PR).

Let me be clear: this library is not deprecated and we are not planning on doing so unless there is a complete replacement. As such - we don't actually plan to have a complete replacement any time soon - except for Cloud APIs. This will still be the library for non-Cloud APIs. If we do deprecate it, we will still support it for at least a year as per our public deprecation policy.
All that said, it's not deprecated. We just are not adding new features. Yes, that can be frustrating and I hear you. However, adding new features also adds new bugs and introduces instability. We have a large majority of users who have used this library in code that they do not have desire to change, so even behavioral changes are problematic for them. We have to be extremely conservative here. "No" is temporary, but "yes" is forever.

Personally, I think having an in-library solution to exponential backoff and retries makes a lot of sense

We do on non-batch methods- you can call execute(retry=5).

At the very least, having some documentation on the recommended way of implementing it through third-party libraries would be greatly appreciated.

That's a wonderful Idea. I'd be more than happy to merge pull requests that add documentation around common patterns, workarounds, etc.


Use CRI to pull images

What type of PR is this?
/kind feature
What this PR does / why we need it:
This is an initial attempt to use CRI to pull the images to mount. This should make it simpler to configure image volumes, because registry configuration (including mirrors, proxies, auth, etc) will come from the node.
Special notes for your reviewer:
This is currently a prototype, and shouldn't be used (or merged)

I'd still really like to find a way to avoid duplicating the data like the existing implementation does.

We (@vrothberg and me) have added functionality to Podman to allow the mounting of images on the host, as well as mounting images into a podman container.
mnt:=$(podman image mount fedora)
Then $mnt could be used as the source for a CRI-O container image.
Is this something we should work on together?

Interesting. Does that do the equivalent of buildah mount? The code in trunk currently is just doing a buildah mount then passing it back to kubelet to mount into the other container.

@rhatdan the goal of this PR was to enable a csd-driver-image-populator that would be CRI-implementation independent. Using podman would have similar issues to using buildah (registry configuration is cri-implementation specific, so credential management for volume images is harder)

/remove-lifecycle rotten

Has anyone reviewed this project? https://github.com/warm-metal/csi-driver-image

No, hadn't seen that. interesting. Looks like it only works with docker/containerd so far?


Apply HappyPack without loader specified

Is it possible to invoke the HappyPack plugin on the default webpack JS loader? Not sure what loaders webpack is using internally do resolve the dependency tree from the entry, or if there is a way to invoke the default behavior as a loader.
For example, a DLL bundle:
...
  entry: {
    'vendors': [getPath('vendors/entry.js')]
  },
  output: {
    filename: '[name].bundle.js',
    path: 'dist/vendors/',

    // The name of the global variable which the library's
    // require() function will be assigned to
    library: '[name]_lib',
  },
  plugins: [
    new webpack.DllPlugin({
      // The path to the manifest file which maps between
      // modules included in a bundle and the internal IDs
      // within that bundle
      path: 'dist/vendors/[name]-manifest.json',
      // The name of the global variable which the library's
      // require function has been assigned to. This must match the
      // output.library option above
      name: '[name]_lib'
    }),
    new HappyPack({ loaders: [ '??' ] })
  ]
...

There is no "default" loader; if you're not specifying a loader then no loaders are applied and happypack has nothing to do. The time that webpack spends in your case is resolving dependencies and working its magic (not the loader magic, which is what happypack helps with).
Sorry. 😞
I get it that vendor DLLs could take some time to build, but that's what DLLs are for - so you build them only once instead of every time. Optimizing other parts of webpack other than applying loaders is beyond the scope of a plugin, the way I see it.

Agreed, just wanted to try and use all my resources when assembling the DLLs. I wasn't sure if Webpack had a 'default loader' because I know it has some default polyfills for require, etc. Figured it had to have a default "transformer" or something of the sort, but I guess it isn't exposed? Thanks for the response and the public contributions!

Yeah, I don't think it's a loader per se, but webpack does do things in its core like a loader would - it generates an AST, extracts data, munges that AST and emits a "webpackified" version of the source. So in essence your assumption is correct, but it can be considered a pre (or post, or both?) step to applying loaders.
I think the internal architecture for this is changing in webpack 2, so maybe we will revisit in the future to see what can be done.


Edit image duplicate buttons

The edit image view has duplicate buttons
Have a look at the image.
Technical details

Python 3.6.4
Django 2.0.5
Wagtail 2.4.

Duplicate butters vanished after restart??

@robslotboom  hi, I have exactly the same issue, can you please say how did you solve it?

@BronzeCrab
As I wrote; I just restarted the development server (runserver)
Hope this will solve your problem to.
Cheers,
Rob

actually i have it in production. I've tried to restart gunicorn, and it didnt help.

Maybe clear the browser cache?

didnt help(

Strange.
Do you use supervisord? If so try to restart it.
Otherwise I have no idea.

The buttons are duplicated in the HTML but hidden in CSS to allow them to appear in the right place on different-sized layouts, so it sounds like you have a deployment-related issue that's causing outdated CSS to remain in place (e.g. caching, or not re-running ./manage.py collectstatic after an upgrade).
Closing as unable to replicate - happy to reopen if you can find steps to consistently reproduce this.


Adding smart default support

Adding support for defaults_mode client configuration option. The defaults_mode will be used to determine how certain default configuration options are resolved in the SDK. Configurations values impacted include: retry_mode, sts_regional_endpoints , s3 us_east_1_regional_endpoint and connect_timeout.

Codecov Report

Merging #2591 (18464cd) into develop (b0923b4) will decrease coverage by 0.05%.
The diff coverage is 95.52%.

@@             Coverage Diff             @@
##           develop    #2591      +/-   ##
===========================================
- Coverage    95.41%   95.36%   -0.06%
===========================================
  Files           59       59
  Lines        11631    11826     +195
===========================================
+ Hits         11098    11278     +180
- Misses         533      548      +15

Impacted Files
Coverage Δ

botocore/utils.py
95.40% <89.74%> (-0.35%)
:arrow_down:

botocore/configprovider.py
94.40% <95.65%> (+0.93%)
:arrow_up:

botocore/args.py
100.00% <100.00%> (ø)

botocore/config.py
100.00% <100.00%> (ø)

botocore/exceptions.py
99.53% <100.00%> (+<0.01%)
:arrow_up:

botocore/session.py
96.88% <100.00%> (+0.25%)
:arrow_up:

botocore/compat.py
72.13% <0.00%> (-1.10%)
:arrow_down:

botocore/httpsession.py
90.21% <0.00%> (-0.43%)
:arrow_down:

botocore/hooks.py
95.27% <0.00%> (-0.40%)
:arrow_down:

... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b0923b4...18464cd. Read the comment docs.


cannot record voice for starfinder actor

show error "missing actorLink" when click record on actors and npc
but work fine with journal

Should be fixed in 0.2.0