Implement Valheim Plus mod support

Takes care of #63
I apologize if my Bash code isn't elegant but this is the best I could do with my limited knowledge. In general, the steps added are as follows:

Created an ENV variable that allows the user to specify whether V+ is enabled (by default disabled).
If the mod is enabled, during the update() function call, we first get an md5 checksum list of all the unmodified server files to keep as reference.
Then we call the new check_valheimplus() function to see if we need to install/reinstall the mod.
Inside check_valheimplus() we do either a check to see if the mod exists in the first place (this is usually on first start of container) of if the md5 checksums of the server files don't match the files found in the mod (this is for cases where Steam overrides the files).
Inside the install_valheimplus() function, we go ahead and download and install the mod in a volume specifically for it found at /opt/valheimplus (the reason we persist this volume is for checksum data as well as keeping the latest zip file downloaded there plus the unzipped files from the mod).
We then rsync all the files in the mod volume to both /opt/valheim_dl and /opt/valheim
I have also implemented a new run_bepinex.sh script that should take care of handling our server starting function better.
The update_valheimplus() function checks if the md5 checksums of the remote zip file and the local zip file differ, in which case it will run install_valheimplus() again to update the mod.

The only bit of concern here is that the valheim_plus.cfg file needs to have its ownership changed to reflect the UID and GID of the user running the container. This is so that users can easily modify the file without needing root access. However, I think if #20 was implemented, we could switch over to that

OK, so I noticed some issues with the way rsync is handling some files. BepInEx, once run actually creates some new files and directories that are only available in /opt/valheim and not in /opt/valheim_dl which causes the mod to keep reinstalling. Let me look into fixing that as well

@lloesche How averse are you to using md5 checksums to check if the server files were updated against /opt/valheim instead of using grep '^[*>]' "$logfile" > /dev/null 2>&1 as the output for that line is a bit unreliable

@armanatz thank you for your PR. I'll review it tomorrow. Can you give some details how the rsync output is unreliable? A line beginning with a * means a file was removed and a line beginning with a > means a file was added/replaced.

@armanatz I very much like the approach of checking steam for the current build version and comparing it with what we have local. I just looked over the PR and added some comments where I saw some obvious things. I like the overall design, but I'm not a fan of relying on line numbers in remote generated content. That'll be the first code to break.
Regarding container size, I'm not at all concerned with that. If the container image is 2GB larger at runtime but therefor we have cleanly separated download and install directories that's fine by me. Disk space is cheap and plentiful.
If we can't get the steam version parsing code to be more robust I'm also fine with keeping the current /opt/valheim_dl and /opt/valheim folders and (for plus users) creating yet another /opt/valheim_plus directory which would be a merger of /opt/valheim and UnixServer.zip. valheim-server can then decide whether it starts the vanilla server in /opt/valheim or the plus server in /opt/valheim_plus. The plus would also be that users can switch between both versions on the fly.
We want to be sure that we

do not break anything for existing users
make the code robust to changes on the IronGate, ValheimPlus and Steam side of things

Just did a quick search and https://github.com/leovp/steamfiles seems to have a very robust acf and appinfo parser implementation. Is that something you could tackle? If not I can invest some time on Wednesday.

I like the overall design, but I'm not a fan of relying on line numbers in remote generated content. That'll be the first code to break.

Yeah, I will look into parsing things a bit better. To be honest, my knowledge with sed is abysmally tiny.

Regarding container size, I'm not at all concerned with that. If the container image is 2GB larger at runtime but therefor we have cleanly separated download and install directories that's fine by me. Disk space is cheap and plentiful.

True, however there are some people who'd like to run the server on an RPi and I was thinking any space savings would be welcomed for them. I think if we can keep the behavior the way it is currently, it would also make understanding the folder structure of the container as well as future maintenance a bit easier as well since we are only dealing with a single game folder.

I just played around with that Python library. Works quite well. Here's my rough prototype for a acf to json converter:
root@2f403fbaf80b:/opt/valheim_dl/steamapps# cat ~/acf2json.py
import sys
import select
import json
from steamfiles import acf

if select.select([sys.stdin,],[],[],0.0)[0]:
    acf_content = sys.stdin.read()
    data = acf.loads(acf_content)
    print(json.dumps(data))

Will produce output like:
root@2f403fbaf80b:/opt/valheim_dl/steamapps# python3 ~/acf2json.py < appmanifest_896660.acf | jq .
{
  "AppState": {
    "appid": "896660",
    "Universe": "1",
    "name": "Valheim Dedicated Server",
    "StateFlags": "4",
    "installdir": "Valheim dedicated server",
    "LastUpdated": "1613959631",
    "UpdateResult": "0",
    "SizeOnDisk": "1045271601",
    "buildid": "6246034",
    "LastOwner": "76561198828919924",
    "BytesToDownload": "0",
    "BytesDownloaded": "0",
    "BytesToStage": "0",
    "BytesStaged": "0",
    "AutoUpdateBehavior": "0",
    "AllowOtherDownloadsWhileRunning": "0",
    "ScheduledAutoUpdate": "0",
    "InstalledDepots": {
      "1006": {
        "manifest": "6688153055340488873",
        "size": "59862244"
      },
      "896661": {
        "manifest": "521795651741005384",
        "size": "985409357"
      }
    },
    "UserConfig": {}
  }
}

I think this can be useful. We can either process it in Python or just use the lib to convert to json and then do further selection with jq. I'm fine with either. Should be pretty robust.
Although I'm not sure if this is worth installing an entire python stack. At that point we could just leave the existing rsync code which we know works reliable and does not require any metadata parsing and create a third /opt/valheim_plus directory.

@lloesche If all looks good, I think my recent changes should suffice with regards to merging this PR. Let me know your thoughts :)

@armanatz lgtm 👍 just had two questions about /opt/valheim_dl still in the README and was confused about two files that shouldn't be part of the PR. Likely just a quick rebasing issue.

@armanatz lgtm 👍 just had two questions about /opt/valheim_dl still in the README...

Woops, sorry. Forgot to make the changes to the README. It is fixed now.

and was confused about two files that shouldn't be part of the PR. Likely just a quick rebasing issue.

That should be fixed as well. Was an issue on my part.

merged, thank you! @armanatz

@lloesche Oh dang, there was a new update to V+ that changed its structure. I was going to message saying not to merge yet. I am doing a fix for it as we speak. Will submit a new PR then

@armanatz Hey, I just had to revert the PR because my server was restarting every 15 minutes on every update check.
Also valheim plus was enabled by default in the updater.

We'll have to redo this PR with your latest changes and also figure out why the updater is broken and restarts the server every 15 minutes with the changes you made. This time lets create it against the dev branch so we can test the image before merging into main.

@armanatz Hey, I just had to revert the PR because my server was restarting every 15 minutes on every update check.

That's odd, it doesn't happen for me. Let me recheck and see again. Maybe I missed something.

Also valheim plus was enabled by default in the updater.

That's weird. This wasn't from my side. Will double check.

We'll have to redo this PR with your latest changes and also figure out why the updater is broken and restarts the server every 15 minutes with the changes you made. This time lets create it against the dev branch so we can test the image before merging into main.

Yeap, on it. Will do changes against dev branch

Thank you two for your hardwork.
Once we have something on dev, I'll do my best to help with the testing

You had the code
VALHEIM_PLUS_ENABLED=${VALHEIM_PLUS_ENABLED:-true}

twice, once in -server once in -updater. In server it defaulted to false in updater it defaulted to true.

We should introduce a "default-vars" and set all the defaults in there and then include that inside of all of our scripts. That way we don't have to maintain the same default in two places.

You had the code
VALHEIM_PLUS_ENABLED=${VALHEIM_PLUS_ENABLED:-true}

twice, once in -server once in -updater. In server it defaulted to false in updater it defaulted to true.

 My bad. Will ensure to double check. Lack of sleep does that to you haha.

We should introduce a "default-vars" and set all the defaults in there and then include that inside of all of our scripts. That way we don't have to maintain the same default in two places.
I'll do that quick.

Yea, that sounds like a good idea. Thanks!

No worries, we caught it before it effected any users so all is good. I just pushed https://github.com/lloesche/valheim-server-docker/commit/704d05009dc468a19b30b237bb3d3aaa696a346c which creates a file defaults where we can maintain all our default env vars.
I was thinking to avoid any issues with server updates we should keep the current rsync update process instead of querying steam. It's just more reliable in that we only restart the server if files have actually changed. What do you think about the following structure:
/opt/valheim/dl/server   # here we let steamcmd download the dedicated server unmodified
/opt/valheim/dl/plus     # here we download and extract UnixServer.zip of ValheimPlus and store the updated_at date info
/opt/valheim/server      # here we rsync the vanilla server from /opt/valheim/dl/server and use the current restart logic
/opt/valheim/plus        # here we rsync with `--link-dest` (creating hard links) from /opt/valheim/server and /opt/valheim/dl/plus to create a ValheimPlus distribution

This would also be backwards compatible with folks who already have a /opt/valheim volume mount.
By using hard links in the last step we don't consume any more disk space than we do now (although as mentioned I'm not too concerned with +/- 1GB of disk space usage).
I also feel this makes every step of adding ValheimPlus more robust. If ValheimPlus for whatever reason breaks users can just turn it off and we will start Valheim from the unmodified /opt/valheim/server vanilla location.

@armanatz  By the way I just invited you as a collaborator for this repository. Let's create a new valheimplus branch and do the work in there. That way we can more easily collaborate on it.
Dockerhub is also set up to automatically build any branches we create as new tags. So a valheimplus branch would become available as lloesche/valheim-server:valheimplus.
That'll allow us better testing.

I also feel this makes every step of adding ValheimPlus more robust. If ValheimPlus for whatever reason breaks users can just turn it off and we will start Valheim from the unmodified /opt/valheim/server vanilla location.

That's is defo a important point. Modding on Valheim is, as far as I understand it, quite hacky so having a easy/quick way to fall back seems right.

Dockerhub is also set up to automatically build any branches we create as new tags. So a valheimplus branch would become available as lloesche/valheim-server:valheimplus.
That'll allow us better testing.

Indeed. Don't hesitate to ping me once you have something there, I have a few players that are willing to test the mod, I should be able to get you some feedback quite fast.

@armanatz let me know what you think of https://github.com/lloesche/valheim-server-docker/pull/88
@Hydeo thank you! Will definitely ping you once a PR is ready to test!

I also feel this makes every step of adding ValheimPlus more robust. If ValheimPlus for whatever reason breaks users can just turn it off and we will start Valheim from the unmodified /opt/valheim/server vanilla location.

@lloesche Actually V+ changed the way it handles things with BepInEx. So before 0.8.5, they used to ship a folder called valheim_server_Data/Managed which had unstripped Unity DLLs in it. This folder would modify the valheim_server_Data/Managed folder downloaded via Steam and overwrite things.
With 0.8.5, they changed the behavior by instead having a unstripped_corlib folder shipped with new releases. This folder has the same files as valheim_server_Data/Managed that was shipped before. However, now they load this folder into Doorstop instead.
What this effectively means is that none of the actual files on the server are modified. This also means that Steam won't always need to revalidate the game files if V+ is installed.
So what I am trying to say is that we don't have to worry about V+ breaking things. Basically, if the user disables VALHEIM_PLUS_ENABLED, then we just load things in the vanilla way and they are good to go

I was thinking to avoid any issues with server updates we should keep the current rsync update process instead of querying steam. It's just more reliable in that we only restart the server if files have actually changed.

I don't know if rsync is the right tool though over here. It also can cause some issues if in the future IronGate decides to create temp files or cache files directly in the server folder. Or in the case of BepInEx, it actually does create a cache directory on server start, therefore, anytime the update check runs, it will restart the server if you are running V+.
By saying that we run rsync whenever there is a modification to a file and then restarting the server, we may end up restarting unnecessarily when there actually were no updates to sync.
By using Steam as our single source of truth for updates, then we can guarantee that the information is correct as Steam sees it. If when Steam notifies that an update is there, then only we initiate a restart.

It also can cause some issues if in the future IronGate decides to create temp files or cache files

That's just an issue of not supplying the --delete argument and/or not checking for lines starting with *. If we only check for updated files that were transferred to the destination > then we should be good.
And there is no conceivable way where the game would modify its own files that were downloaded from Steam because then Steam would always report them as corrupt. So I feel this is a pretty robust method of checking for changes.

If when Steam notifies that an update is there, then only we initiate a restart.
I would agree if we had programmatic access to the Steam API. Calling steamcmd and parsing the output with a number of sed/awk/trs or even via the acf library seems less than ideal. It's a bit like scraping websites for information and hoping they don't change the format.

@Hydeo as promised #91 for you to play around with 🙂
Docker image is available as lloesche / valheim-server:dev

Awesome !
I'll play with it today, I'll come back to you as soon I as have :)


Blade bug: exception in some cases when the word "or" is in a comment

I hit a strange bug earlier and have boiled it down to a failing template.
With a fresh Laravel 5 project, if I replace welcome.blade.php with this:
<?php
class Foo {
	public function bar($callback) {
		return $callback();
	}
}
$foo = new Foo();
?>
{{ $foo->bar(function() {
	// Comments are okay here but if one contains the following word we get an exception.

	// or

	return "baz";
}) }}

An exception is thrown.
FatalErrorException in e05da09885ea068f3fec7a1a33d8fad4 line 16: Cannot use isset() on the result of a function call (you can use "null !== func()" instead)

On inspecting the compiled template we have
<?php
class Foo {
	public function bar($callback) {
		return $callback();
	}
}
$foo = new Foo();
?>
<?php echo e(isset($foo->bar(function() {
	// Comments are okay here but if one contains the following word we get an exception.

	//) ? $foo->bar(function() {
	// Comments are okay here but if one contains the following word we get an exception.

	// : return "baz";
})); ?>

Looks like something is going quite wrong in the Blade compiler.
If you remove the comment // or, the file runs properly and we get output of "baz". In that case it compiles to
<?php
class Foo {
	public function bar($callback) {
		return $callback();
	}
}
$foo = new Foo();
?>
<?php echo e($foo->bar(function() {
	// Comments are okay here but if one contains the following word we get an exception.

	return "baz";
})); ?>

The bug also exists in Laravel 4, which is where I found it, and also when the comment is the # or style or the /* or */ style.
(I know it is unwise to put logic like this in a Blade template, but it's a bug all the same.)

This is known I'm afraid. The or syntax is very limited. Sorry.

@GrahamCampbell, I'm not trying to use "or" syntax. I don't even know what "or" syntax you're referring to. My code breaks if the word "or" is in a comment -- that might happen for any number of reasons. Which ticket does this duplicate? Or if there is no duplicate, why have you closed it?

I agree with Graham, actually. You should not use PHP comments inside the output tags ({{ ... }}). You can use Blade comments.

What exactly are you agreeing with?
Is it documented that I should not use PHP comments inside output tags?
How can I use Blade comments inside output tags?

You can't, just don't add any comments inside the output tags.

Is that documented anywhere?
Why not? What if I want to? Do you disagree that this is a bug?

Technically it isn't documented, but there are also no examples encouraging anything like this. You abused the fact that these tags are compiled to raw PHP, where this would be possible.
Yes, I don't think this is a bug.

Not a bug, just a limitation.


Lots of Copyright comment blocks in main.js build

Version: beta.16
OS: OSX 10.12
So I just ran an ng build -prod --aot using beta.16 and was looking at the output. I noticed main.js has about 20 - 30 instances of Google Copyright notice (one above each block). The same happens for a build without AoT as well actually.
/**
 * @license
 * Copyright Google Inc. All Rights Reserved.
 *
 * Use of this source code is governed by an MIT-style license that can be
 * found in the LICENSE file at https://angular.io/license
 */

I'm guessing there should only really be one or none of those at the top of the generated dist/main.js
Screenshot: https://www.dropbox.com/s/4cuf1yxtik9drjy/Screenshot 2016-09-28 10.50.09.png?dl=0

@hansl can you have a look? This must be part of the bundle size.

I need to talk to the legal team about this. The google ones are probably okay to only put one, but third party libraries might be more complicated.

Just FYI I made a new beta16 project and ran the main bundle through a comment stripper, and non-gzipped size went from 815kb to 764kb.

Speaking of comments.
I'm using material-design-lite and building it from sass. My styles.bundle.js with --prod contains 31 occurrences of every single /* */ comment (not // comments) in _variables.less as it's included 31 times.
Perhaps I should create a separate issue for this tho?

CSS comments has been solved in #2593.

I see the comments with builds using ngCLI beta.17 as well.  Can we strip the credits into a credits.txt file that refers to the compiled .js/.css?  MIT doesn't require the presence of the copyright notice in binary distributions. Can a *.js file be considered a binary distribution?

Tracked by: https://github.com/angular/angular-cli/issues/4413
Addressed by: https://github.com/angular/angular-cli/pull/4694


fcrackzip: init at 1.0

patch adapted from https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=430387

forgot to commit all-packages.nix. Fixed


Not in white list. Check listen.allowed_clients

We use cachetool in our (automated) deploys. Once in a while this fails with a RuntimeException
FastCGI error: Not in white list. Check listen.allowed_clients.

It would be nice if cachetool could deal with this more gracefully (e.g. try again a few times and exit) instead of throwing an exception.
This happens on Ubuntu 16.04 + PHP 7.0.21

What's your listen.allowed_clients value in your php-fpm server?
From which machine are you invoking cachetool?

@gordalina There is no explicit configuration for listen.allowed_clients, which - according to the PHP documentation - means it should accept all. Additionally, we only use unix sockets for our FPM pools, not TCP sockets.

If you run cache tool after the error it works? How many times does it error before it works?

Running it immediately after usually works

We also faced this problem from time to time.
our listen.allowed_clients value is <IP_ADDRESS>
and listen value is <IP_ADDRESS>:9000
so maybe this problem is not completely relative to listen.allowed_clients
and our php version is also 7.0 & os version is Ubuntu 16.04

I found out that this is maybe a fastcgi client library problem (https://github.com/adoy/PHP-FastCGI-Client/issues/13)
It's seems that adoy/PHP-FastCGI-Client is not in actively maintained. maybe change a fastcgi client library is good idea ?

Yeah, looks like someone forked adoy/PHP-FastCGI-Client into https://github.com/hollodotme/fast-cgi-client
If you're interested in sending in a Pull Request to switch libraries, I'd be happy for it

Same problem for me. Sometimes I am getting same error

For us mostly it occurs during deployment when flushing opcache it increases the load quite drastically and cachetool prints out this error, i think it's because of timeout that occurs in the underlying library that is executing the flush operation through FPM. It ends up succeeding but we get the error and can't suppress it or handle it so be build fails. Due to the high load it just takes longer for FPM to response, ideally an option to change the timeout could also work

I've switched from adoy/PHP-FastCGI-Client to https://github.com/hollodotme/fast-cgi-client and scripts invalidation works now! 🍾 See PR above.
However I did it only for 3.x branch as we use PHP 7.0.
I also added this change to my PR as well: https://github.com/gordalina/cachetool/pull/94
P.S. Seems like tests failed because of bad build configuration (something wrong with GitHub token)

I'm also hitting this problem.
Migrating to https://github.com/hollodotme/fast-cgi-client seems like a solid idea.
What's stopping merging #96?

One of the big problems I'm experiencing is that after this failure, phpinfo() shows that PHP opcache becomes disabled and I have to restart the interpreter to restore it. Very costly. Does anyone here have any thoughts?

My problem was highly likely caused by a WordPress plugin calling opcache_reset() due to Redis object cache not being properly invalidated https://github.com/Yoast/wordpress-seo/issues/9315


Fail to parse expressions with non-range step values

hi,
The current version fails to parse expressions with non-range step values, even though the library's examples page suggest otherwise. For example, the following all return {:error, "Can't parse 0 as a range"}
Crontab.CronExpression.Parser.parse "0 0/5 14 * * *"    # example from library's example page
Crontab.CronExpression.Parse.parse "0/2 * * * *"

I can get around the issue by using ranges, e.g., 0-23/5 and 0-59/2 respectively for the examples above.
I can help in submitting a fix, if supporting such expressions is expected.

Oh, you're right. A PR would be welcome.
I think it's suppused to work as follows: REMAINDER/DIVIDER and evaluates as truthy if INTERVAL % DIVIDER == REMAINDER

Oh, you're right. A PR would be welcome.
I think it's suppused to work as follows: REMAINDER/DIVIDER and evaluates as truthy if INTERVAL % DIVIDER == REMAINDER

I think START/INTERVAL means run on START + INTERVAL * N where N begins at 0; for example, if 0 0 2/5 * * * (i.e., for the day of month field) and today is 2024-01-30, that expression's next few run dates would be:

2024-02-02 00:00:00
2024-02-07 00:00:00
2024-02-12 00:00:00
2024-02-17 00:00:00
2024-02-22 00:00:00
2024-02-27 00:00:00
2024-03-02 00:00:00


Allow customization of Twitch chat server parameters through ENV variables

This would leave an option for https://github.com/Chatterino/chatterino2/issues/1296 if compiling their own version isn't viable

We could also add a setting for choosing ws+tsl/tsl/raw irc.

Yup, could also do that. Is there a point in defaulting to 443? It looks like the most compatible one, but I don't know if there are any downsides to using it.

@pajlada that's actually 5Head since it will get around proxies

I am closing the PR and moving default port to 443 instead of 6697


Install and scaffold Tailwind

This PR installs Tailwind, and configures it based on the tokens of the project.

Yeah, my experience JIT with PostCSS 7 was quite buggy as well. Maybe the next major upgrade with Vue 3 will allow us to turn on JIT 🤞.


Release 0.9.4

I was looking into using your library in an application and just realized that the collections functionality I need doesn't appear until the 0.9.4 version. Unfortunately, I don't see a tag for that version or see it listed in npm.  Any idea when it will be released? Is this project currently in active development?

Just published to npm. Been moving fast lately, forgot to release before moving forward!


Feature Request: Video Subjects

Are there plans to allow for videos to be used as subjects?  We have a project in the planning stages using the Chimp & See data, and realized that Panoptes currently only supports .jpg, .png, .gif, .svg for subjects...

Yes it is our plan to support video subjects.


[WIP] Added the blocklet info to index file and make the datamap distributable with job

In this PR following tasks are completed.

Added the blocklet info to the carbonindex file, so datamap not required to read each carbondata file footer to the blocklet information. This makes the datamap loading faster.

Made the data map distributable and added the spark job. So datamap pruning could happen distributable and pruned blocklet list would be sent to driver.

This PR cannot compile as carbondata format changes are present.

Build Failed with Spark 1.6, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder/504/

Build Failed  with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder/3094/

Build Failed  with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder/3218/

Build Failed with Spark 1.6, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder/622/

SDV Build Failed with Spark 2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheSDVTests/38/

Build Failed  with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder/3391/

Build Failed with Spark 1.6, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder/794/

SDV Build Failed with Spark 2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheSDVTests/121/

SDV Build Failed with Spark 2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheSDVTests/122/

Build Failed with Spark 1.6, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder/795/

Build Failed  with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder/3393/

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/carbondata-pr-spark-2.1/91/

SDV Build Failed with Spark 2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheSDVTests/139/

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/carbondata-pr-spark-1.6/743/

Build Failed with Spark 1.6, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder/820/

Build Failed  with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder/3412/

SDV Build Failed with Spark 2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheSDVTests/313/

SDV Build Failed with Spark 2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheSDVTests/314/

retest this please

SDV Build Success with Spark 2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheSDVTests/319/

LGTM


Wizard should pass along actionContext so that individual steps can add to telemetry

Mentioned in this PR: https://github.com/Microsoft/vscode-azuretools/pull/218

Fixed in https://github.com/microsoft/vscode-azuretools/pull/503


Make the IR and other info move2kue collects available in the transformation scripts

Is your feature request related to a problem? Please describe.
Make all the information that move2kube collects and the IR within the transformation scripts.
Describe the solution you'd like
Perhaps something like ir('k1.k2.k3')
Describe alternatives you've considered
Or maybe something lke ir['k1']['k2']['k3']
Additional context
Perhaps to get app name or service name?

Config is also not available currently.

Centralized IR is no longer there in three new framework


Parameter JSDoc could be included in tooltip

Hi,
Version: VS 2015 / TS 1.5

When hovering over the property foo, it would be helpful if the JSDoc "The foo option" was also displayed. At present the only way to figure out what the property represents seems to be to delete and type it again:

looks like a dupe of #290

Yes, I think it's a duplicate.


Use redis in filer, set db but not work

Sponsors SeaweedFS via Patreon https://www.patreon.com/seaweedfs
Describe the bug
Use redis in filer, I set redis db = 2, but db 2 is empty, all data is in db 0.
System Setup
List the command line to start "weed master", "weed volume", "weed filer", "weed s3", "weed mount".
/data/seaweedfs/weed master -port 19333 -ip <IP_ADDRESS> -ip.bind <IP_ADDRESS>
/data/seaweedfs/weed filer -master=<IP_ADDRESS>:19333 -port=9998 -port.readonly=9999

filer.toml
[redis]
enabled = true
address  = "xxx"
password = "xxx"
db = 2

set redis db = 2
but it save to db 0
➜  ~ redis-cli
<IP_ADDRESS>:6379> SELECT 2
OK
<IP_ADDRESS>:6379[2]> DBSIZE
(integer) 0
<IP_ADDRESS>:6379[2]> KEYS *
(empty list or set)
<IP_ADDRESS>:6379[2]> SELECT 0
OK
<IP_ADDRESS>:6379> DBSIZE
(integer) 8409980
<IP_ADDRESS>:6379> keys /images/2ch.hk/mov/thumb/2249878/*
1) "/images/2ch.hk/mov/thumb/2249878/15576907789920s.jpg"
2) "/images/2ch.hk/mov/thumb/2249878/15577166357860s.jpg"
3) "/images/2ch.hk/mov/thumb/2249878/15577646385250s.jpg"
4) "/images/2ch.hk/mov/thumb/2249878/15576726272160s.jpg"
(8.55s)

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.

It's a typo in scaffold. Please change "db" to "database".


Deny view outputs based on wildcard suffix

Can now deny additional items with a filter class by appending a wildcard * character to a given DENYLIST config.
This applies to all filters except for those that have overridden filter() to not check for output types.

Latest commit needs the formatter run:
Not sure why it is only showing on the RHEL8 runner and not the others. But I get a diff locally after a build.
sh contrib/ci/detect-changes.sh
The build changed files in worktree:
 M src/main/java/emissary/output/filter/AbstractFilter.java
    ```


How to use DomainParser facade?

Please provide examples.

Hi! Anything specific you would like to know about?
Even though this issue is related to #24, I'll leave it open until fixed.

Thank you Kevin. The idea of using the resolve() method on what is presented as a "list" object wasn't obvious at all. Finally figured it out by cross-referencing the PDP documentation. I'll leave this here for others:
$pdp = app('pdp.rules')->resolve($raw_domain);
$registrable_domain = $pdp->registrableDomain()->toString();
$tld = $pdp->suffix()->toString();


Can not run global node packages

I can not run any globally installed node packages like grunt or gulp.
The bash says: Command not found.

The workaround was formatted. Here is how it should look like:
"term2": {
  'autoRunCommand': 'login -f `whoami`'
}

Dup of #50


How to remove dotted underline under datestamp?

Hi,
I want to remove dotted underline coming under date which i am getting via timeago plugin. Is it possible?
Thanks.

Never mind, i have fixed it.

For those that find this at a future date:
Just remove the bottom border on the abbr tag with CSS.  By default, browsers style abbr with a dotted border-bottom. For examplem to get rid of that:
abbr { border-width: 0px; }


NER recognition

Spanish files that are not fully recognized: SGsp0006.eml (place), Sgsp0005.sp (place and name).

Closing this as we are using new models and sample files. PRocess has been updated and issue no longer relevant.


fix version number 2024 -> 4024

As noted by https://github.com/Roald87/TwinCatChangelog/pull/100#issuecomment-2254952612


Add CANDLE_NVCC_CCBIN support for candle-kernels, and eliminate warnings.

On windows, it seems currently nvcc will not auto detect cl.exe position, so this allows the compiler bindir be stored as an environment variables, for convenience.
This also eliminate a warning under conditional compilation.

Merged, thanks!


Exception when no workspace is loaded.

Activating extension streetsidesoftware.code-spell-checker failed: Path must be a string. Received undefined.

fixed


Please support CoClassAttribute

F# doesn't support System.Runtime.InteropServices.CoClassAttribute.
At C#,
[Guid("AC4A48F4-590E-4FEF-807B-1CE30011866C")]
class CoClass {}

[ComImport, Guid("4862688B-AC55-42CA-A71F-B80C082CCA31"), CoClass(typeof(CoClass))]
interface Interface {}

var i = new Interface();

last line is compiled as
var i = (Interface)new CoClass();

But F#,
let i = Interface ()        // FS0804
let i = CoClass () :?> Interface    // FS0193
let i = CoClass() |> box :?> Interface    // ok

F# compiler should support CoClassAttribute like C# compiler.

I'm not inclined to support this.  The C# support is an idiosyncratic corner of the language, I doubt if the current language designers would approve this feature if it was being proposed today.
We could consider improving the error message to notice the CoClassAttribute on the interface and suggest new of the coclass instead.

There seems to be no great appetite for this so I am closing this issue.


include end time in export filenames

If we configure a "catch up file" at some factor as a more frequent
export configuration (e.g., 24 hr catch up, 4 hr regular period),
unless they use a different prefix/suffix in the name they'll clash
on naming. Easy way to make this foolproof is to have filenames
include start/end times.
This should not effect anybody that simply reads index for files to
download. This also shouldn't effect our sample app implementation
because its region is not configured for catch up files. But I will
also submit a corresponding change in the regex it uses to group
parts of a batch together:
https://github.com/google/exposure-notifications-android/blob/5eb8e99dc8ef0db4ee4754495d032e27094bcfe9/app/src/main/java/com/google/android/apps/exposurenotification/network/Uris.java#L66

Sample app repo has picked up the change to support an optional file name:
https://github.com/google/exposure-notifications-android/blob/765dab0af327d92cc6f5b7f16f5ad80d29361370/app/src/main/java/com/google/android/apps/exposurenotification/network/Uris.java#L66
Proceeding with this change now.
/unhold

/approve


Added Checks

The commands were missing some checks
so i added some

Denied, this is unnecessary as the bot handles it all in one spot here
Also, as the the master branch is currently not under development, please contribute to the rewrite branch instead.


Revel Framework like Render method

Initial version of Goal had a tool that scanned user's views directory and produced a package like this.
It was assumed user would import that package and use it as follows:
func (c App) MyAction() http.Handler {
	return c.RenderTemplate(generatedPackageName.Paths.To.Some.FileHTML)
}

So, this is similar to Revel Framework's c.RenderTemplate("path/to/some/file.html") but type safe: if you use a wrong (non-existent) path your app will not be compiled. So, you know early the template is missing instead of getting a runtime surprise in a form of Internal Error 500 when you're trying to visit some page of your app.
However, now I think it would be better to use something like Revel's c.Render() but reflectionless. It would be more difficult to make sure user renders the template that does exist. But that's possible. And as a result user would need to type just c.Render() instead of c.RenderTemplate(views.Paths.ControllerName.ActionNameHTML). We need a tool that will scan user's controllers and find all calls of c.Render and make sure their arguments are correct.
P.S.: I have already removed the old tool that was responsible for generation of the above mentioned packages. So, Goal is mostly unusable now. That's why this should be resolved ASAP.

I'm closing this for now as Render method has been added. Though a tool that would check existence of templates is still needed.


Implement a standardized execution scheme for chisel

Provide support for chisel options
Provide support for firrtl options when called as part of chisel compile
provide command line support the above options via scopt
provide and execution result class that can be used when chisel3 is part
of some externally controlled toolchain
IMPORTANT: Do not merge until after release, must be merge in tandem with execute harness PR in firrtl

This was superseded by execute-harness-plan-3 PR #325


[tests] fix flaky advertising proxy tests

There are failure logs show that the test scripts failed to verify updated SRP services via mDNS, but the SRP service status on the SRP client and server are verified okay.
2021-10-22T13:57:35.4956943Z FAIL: test (__main__.SingleHostAndService)
2021-10-22T13:57:35.4957783Z ----------------------------------------------------------------------
2021-10-22T13:57:35.4958403Z Traceback (most recent call last):
2021-10-22T13:57:35.4959313Z   File "./tests/scripts/thread-cert/border_router/test_advertising_proxy.py", line 203, in test
2021-10-22T13:57:35.4960362Z     self.host_check_mdns_service(host, host_address, 'my-service', service_port)
2021-10-22T13:57:35.4961477Z   File "./tests/scripts/thread-cert/border_router/test_advertising_proxy.py", line 220, in host_check_mdns_service
2021-10-22T13:57:35.4962481Z     self.assertEqual(service['port'], service_port)
2021-10-22T13:57:35.4963065Z AssertionError: 12345 != 12341
2021-10-22T13:57:35.4963345Z
2021-10-22T13:57:35.4963974Z ----------------------------------------------------------------------
2021-10-22T13:57:35.4964475Z Ran 1 test in 223.176s

(find the full log file at https://github.com/openthread/openthread/issues/6654#issuecomment-950533485
This may indicate that the mDNS publisher doesn't refresh its DNS cache timely, so this commit is trying to double the time waiting for mDNS updates.

aha, I just caught a failure with the added SRP server logs (https://github.com/openthread/openthread/runs/4048259822) and identified the root cause openthread/ot-br-posix#1071. Added the forth commit as a workaround fix since the real fix will require significant work in the advertising proxy.

Awesome, @wgtdkp ! Thanks for continuing to dig into this.

NP! I will work on resolving https://github.com/openthread/ot-br-posix/issues/1071 and I think we can merge this PR to make the CI stable?


Torque vs Angular Velocity

So I was confused on what the difference between body.torque and Body.setAngularVelocity() is. In my code, I am implementing a turning motion for a gear that stays in one place but is turning. So what would be the difference between angular velocity and torque for me?

Torque is rotational force which applied over time (rotational acceleration) results in angular velocity on the body (its rotational speed / direction).
So the relation here is similar to body.force effect on body.velocity, but relating to rotation instead.
When you apply torque you should usually do so over multiple updates as it takes time to accelerate.


Clarify the release decision for Cloak preview

There are a few routes that we can take to release Cloak for the preview launch. We should document the choice since items like our install instructions are dependent on it.

@aluzzardi already has experience doing the last release and mentioned a few options. After talking with him today, he will decide which approach is best and notify us of the decision here.

@vikram-dagger pinging you, so you can watch the issue too.

Ok so there's 2 routes. Both are technically feasible, need to clarify a couple of points before taking a decision:

0.3 channel: We release a parallel 0.3.0-alphaX version that's cloak only. 0.2.x continues to be maintained/released in parallel. 0.2.x is still the default.

Basically the goal is to change the tooling to only consider "stable" versions by default, ignore -alpha/-beta/etc unless explicitly provided
@gerhard How feasible / how much work is it? Specifically

homebrew: Is there a way to have stable and alpha versions co-existing, with stable being the default?

Or do we need to resort to a different tap / formula?

latest_version / install.sh: We'd need to only consider stable versions for bumping latest, but still have a manual way to install a specific version

Feature Flag: Merge dagger-cue and cloak together in the same binary

Similar to what we did in 0.2.x with DAGGER_EUROPA

However, 0.1.x and 0.2.x shared a lot of code, 0.2.x and cloak share none

Example:

dagger do test (cue based)
DAGGER_CLOAK=1 dagger dev (cloak based)

We're going to either need cloak to import dagger-cue's "main" or the other way around, and conditionally invoke one or the other
Open Question: Repo organization (/cc @sipsma)

Since both leave on the same repo, on a different branch, with no shared code, this is non trivial
Solutions:

Fork dagger/dagger's main branch into its own repository (e.g. dagger/dagger-cue), import it from dagger/dagger (cloak codebase)

Is this relevant to the "CUE SDK" project? /cc @jlongtine

History rewriting rebase of the cloak branch so that dagger-cue and dagger-cloak codebases co-exist

Will break all PRs

@vikram-dagger @slumbering Is there a docs impact, one way or the other?

Currently 0.2 docs live in the main branch and 0.3 docs live in the cloak branch with no overlap. I have an open item to discuss using Docusaurus versioning with @slumbering so both can live in the same branch and be rendered at different sub-urls under docs.dagger.io. This requires some investigation, so I'll be better able to answer the impact question after. For the preview launch, if we don't yet have a clean solution for the two sets of docs to co-exist, one solution would be to create a separate microsite only for 0.3 docs.

0.3 channel

Provided this option is possible in a reasonable way, it seems like the best. I definitely see this in practice plenty often in releases on Github where they can be marked "Pre-release" and thus not override the "latest" version while still being available.

Fork dagger/dagger's main branch into its own repository (e.g. dagger/dagger-cue), import it from dagger/dagger (cloak codebase)

If we have to go the feature flag route, this definitely seems like the simplest option (even if it feels weird).

Currently 0.2 docs live in the main branch and 0.3 docs live in the cloak branch with no overlap. I have an open item to discuss using Docusaurus versioning with @slumbering so both can live in the same branch and be rendered at different sub-urls under docs.dagger.io. This requires some investigation, so I'll be better able to answer the impact question after. For the preview launch, if we don't yet have a clean solution for the two sets of docs to co-exist, one solution would be to create a separate microsite only for 0.3 docs.

Could we:

Introduce a 0.3 to the sidebar.js exports, e.g. https://github.com/dagger/dagger/blob/v0.2.35/website/sidebars.js
Configure all pages which belong to 0.3 to display this sidebar, e.g. https://github.com/dagger/dagger/blame/v0.2.35/docs/introduction/1236-community.md#L3

This approach enabled us to have both 0.1 & 0.2 co-exist in the same branch as we were getting ready to release dagger v0.2 (a.k.a. Europa).
Shipping all versions concurrently, in the same place - i.e. https://docs.dagger.io - is a proven strategy of de-risking releases. There are a lot of things that we will need to duplicate if we take a different approach.

@aluzzardi ideally, I think that we should start shipping cloak as a dagger subcommand, i.e.
dagger cloak COMMAND

Existing dagger users should be able to get the cloak variant as a dagger upgrade, without changing anything. There should be no surprises. We could even start displaying a new notice about the default command behaviour changing when 0.3 gets released. In the same context, we can start encouraging existing users to try out the new version by starting their commands with dagger cloak.
I had a quick look at how we might achieve this technically, and while nothing obvious came up in the 30 minutes that I had for this, I would like to keep digging. Unless we know for sure that this is not going to work, dagger cloak sounds as the most elegant solution to me. WDYT?

With europa we used the feature flag like dagger --europa. We can do the same with dagger --cloak. No strong preference on it though.

Taking ownership of this come Tuesday/Wednesday, after kubehuddle.com

@gerhard let me know if you think that we need to create another issue for tracking the docs merge solution for @vikram-dagger or if it should all be tracked in this single issue

Basically the goal is to change the tooling to only consider "stable" versions by default, ignore -alpha/-beta/etc unless explicitly provided
@gerhard How feasible / how much work is it? Specifically

homebrew: Is there a way to have stable and alpha versions co-existing, with stable being the default?

Or do we need to resort to a different tap / formula?

latest_version / install.sh: We'd need to only consider stable versions for bumping latest, but still have a manual way

@gerhard Is this feasible?

@aluzzardi ideally, I think that we should start shipping cloak as a dagger subcommand, i.e.

I think that's a variant of the feature flag (option 2). I think option 1 is preferable if doable, as the feature flag / sub-command is temporary

Basically the goal is to change the tooling to only consider "stable" versions by default, ignore -alpha/-beta/etc unless explicitly provided

@gerhard How feasible / how much work is it? Specifically

homebrew: Is there a way to have stable and alpha versions co-existing, with stable being the default?

Or do we need to resort to a different tap / formula?

latest_version / install.sh: We'd need to only consider stable versions for bumping latest, but still have a manual way

@gerhard Is this feasible?

Yes it is. In Homebrew we can declare head with a specific branch - cloak in our case. FTR https://rubydoc.brew.sh/Formula#head-class_method

@aluzzardi ideally, I think that we should start shipping cloak as a dagger subcommand, i.e.

I think that's a variant of the feature flag (option 2). I think option 1 is preferable if doable, as the feature flag / sub-command is temporary

We have discussed this sync with @aluzzardi and ended up with the following action items:

Rename cloak command to dagger (in the cloak branch) - pay attention to SDKs (Go & TS - Erik, Python - Sam & Helder)
Get Homebrew --head working with the cloak branch - it uses Go to build (tests will need updating)
Produce a dagger v0.3.0-alpha.N binary
Ensure that install.sh script works on Linux with v0.3.0-alpha.N versions
Retire our Homebrew tap
Merge cloak into main as a PR (rebase on top of main first)
Create a new 0.2.x branch from the commit just before the merge

A Windows version for dagger v0.3.0-alpha.N is out of scope for now. As soon as we have the above implemented, we can switch focus to Windows.
There are a lot of unknowns. The focus is on figuring out if the above steps will work in practice. I will start with them tomorrow and get small PRs flowing. Ideally we will have most of this figured out by the end of this week.
In parallel, there is a bunch of docs-related questions to be answered. Stopping here for now. Will talk to @slumbering tomorrow about it.

@gerhard when you sync with @slumbering, can you sync with @vikram-dagger too (if he is online)? It will be good to keep him in the loop since he is the lead for the doc re-structure. Thanks!

Fork dagger/dagger's main branch into its own repository (e.g. dagger/dagger-cue), import it from dagger/dagger (cloak codebase)

Is this relevant to the "CUE SDK" project? /cc @jlongtine

@aluzzardi I think it probably will be relevant to that. I haven't spent much time thinking about that, because of the focus on dagger-classic.

We have discussed this sync with @aluzzardi and ended up with the following action items:
1. Rename `cloak` command to `dagger` (in the `cloak` branch) - pay attention to SDKs (Go & TS - @sipsma, Python - @samalba & @helderco)

2. Get Homebrew `--head` working with the `cloak` branch - [it uses Go to build](https://github.com/Homebrew/homebrew-core/blob/822caeb49a473450ee479bb8cc5db54d25026a0a/Formula/dagger.rb#L20) (tests will need updating)

3. Produce a `dagger v0.3.0-alpha.N` binary

4. Ensure that `install.sh` script works on Linux with `v0.3.0-alpha.N` versions

5. Retire our Homebrew tap

6. Merge `cloak` into `main` as a PR (rebase on top of `main` first)

7. Create a new `0.2.x` branch from the commit just before the merge

I don't see anything about shipping both code path (europa + cloak) and toggle via a feature flag. Is the idea to simply ship both 02.x + 0.3? I think it works, but better to make it explicit to avoid confusion.

Can we move the final checklist to the top of this issue, or to another issue?

@samalba I don't see anything about shipping both code path (europa + cloak) and toggle via a feature flag. Is the idea to simply ship both 02.x + 0.3? I think it works, but better to make it explicit to avoid confusion.

We are starting with Option 1:

main branch continues producing dagger v0.2.x releases (no changes)
cloak branch will start producing dagger v0.3.x releases (former cloak binaries)

A dagger v0.2.x binary will have the behaviour that everyone is used to, no changes.
A dagger v0.3.0-alpha.N binary will have the current cloak binary behaviour.
If we hit any significant blockers, we will re-evaluate. For now, Option 1 described above introduces the least amount of change. There is no risk of breaking existing v0.2.x behaviour since we wouldn't be introducing anything new into those releases or that release process.

What about using redirection for 0.1 and 0.2 towards docs.dagger.io/v1/{id}/getting-started and docs.dagger.io/v2/{id}/getting-started to avoid 404, then we'll not have any issue with futur versioning as v3 will be the new way. Every docs will be in its appropriate directory and every sidebar will be independant. IMHO it's a cleaner way to tackle documentation versioning.

TBH, I'm not sure we need the unique {id} at all if we are using version numbers in the URL and implementing redirections anyway from old to new URLs. In this situation, would the {id} be unnecessary additional overhead?

I think the problem with prefixing every url in the docs with /vx is that it assumes 1) the entire docs are versioned, and 2) the docs are for a single versionable product.
Things that don’t fit well in that model:

Dagger Cloud (not versioned)
Guides and tutorials (may work across versions of dagger)
Comparison to other software (seems weird to version)
SDKs (each SDK may be versioned independently in the future)
Extensions (each extension may be versioned; all are versioned independently of dagger itself)

Could it be that docusaurus “version everything” model is better suited for reference docs of a more narrower scoped product? For example a single SDK, say the Go SDK could presumably have its docs versioned as a whole.
wdyt?

I want to think about the previous comment a bit more as it has deep implications for how we version docs, but atm I have one follow-up comment/question:

SDKs (each SDK may be versioned independently in the future)
Extensions (each extension may be versioned; all are versioned independently of dagger itself)
a single SDK, say the Go SDK could presumably have its docs versioned as a whole.

In both above cases, it's not sure if it's necessary or a good idea for the docs to mirror versioning for a given SDK/extension version. Two reasons: Most users would only be interested in docs for the "latest" version, plus the deltas between minor versions of each SDK/extension will typically be small to avoid compatibility breaks. I'm therefore wondering if it would be necessary to maintain separate docs for each previous version of a given SDK/extension, or if we could maintain docs only for the current version + a changelog for reference?

I have learned today that using --head for Homebrew formula pre-releases (e.g. alpha) is not allowed:

Unstable versions (alpha, beta, development versions) are not acceptable for versioned (or unversioned) formulae.
Excerpt from https://docs.brew.sh/Versions

In other words, I expect this diff to be rejected by the Homebrew maintainers:
commit 8146077b81ad2d9715bb01ea758382366565a743 (HEAD -> point-dagger-head-to-current-head, origin/point-dagger-head-to-current-head)
Author: Gerhard Lazu<EMAIL_ADDRESS>Date:   Wed Oct 12 12:40:23 2022 +0100

    dagger: Install the next pre-release via --head

    We want to give users the opportunity to try the next version of Dagger.

    The simplest change was to add this via the `--head` option.
    Technically, that is the current HEAD of the dagger CLI.

    Signed-off-by: Gerhard Lazu<EMAIL_ADDRESS>
diff --git a/Formula/dagger.rb b/Formula/dagger.rb
index b83db0f2c07..019864a3262 100644
--- a/Formula/dagger.rb
+++ b/Formula/dagger.rb
@@ -5,7 +5,7 @@ class Dagger < Formula
       tag:      "v0.2.36",
       revision: "ea275a3bafbc5b5c611e0b81bf2dd8a8add72f6b"
   license "Apache-2.0"
-  head "https://github.com/dagger/dagger.git", branch: "main"
+  head "https://github.com/dagger/dagger.git", branch: "cloak"

   bottle do
     sha256 cellar: :any_skip_relocation, arm64_monterey: "3397d51bc458acbcfe9003fd55bd0fc086ed43f41d9f41a0c9cee48849aab809"
@@ -31,6 +31,16 @@ class Dagger < Formula
     generate_completions_from_executable(bin/"dagger", "completion")
   end

+  def caveats
+    <<~EOS
+      To install an experimental Dagger v0.3.0 alpha release, run:
+        brew install dagger --head
+
+      Learn more about the next Dagger v0.3.0 release (a.k.a. Cloak):
+        https://github.com/dagger/dagger/tree/cloak
+    EOS
+  end
+
   test do
     assert_match "v#{version}", shell_output("#{bin}/dagger version")

As I was submitting this PR and was going through the manual checks, I realized that formula tests were going to be messy (current differences between the current v0.3 vs v0.2). Instead of going down that rabbit hole, we should bump the dagger.rb formula to 0.3.0 when we are done with the final release.
As for the next step, we should add a new branch to our dagger Homebrew tap instead - I'm thinking v0.3.x currently - and publish pre-releases via GoReleaser. Anyone can install pre-releases from our Homebrew tap by running brew install dagger/tap/dagger https://raw.githubusercontent.com/dagger/homebrew-tap/v0.3.x/dagger.rb. Everything looks good from the GoReleaser perspective:

https://github.com/goreleaser/goreleaser/blob/c1965923b0c52429718b1dd4948b796d6c8aaad2/www/docs/customization/release.md
https://github.com/goreleaser/goreleaser/blob/c29971bddb38ac1e77da872e31412c0ef3fb009b/www/docs/customization/homebrew.md

I have also learned that our dagger Homebrew tap had 5x more installs in the last 90 days than the dagger bottle: 4084 vs 817. FWIW: install-on-request. This makes me think that we shouldn't deprecate it, but instead make both formulas behave the same (our tap should be the source of truth).
I have updated the tasklist at the top of this issue to reflect the new plan.
Next step: produce alpha binaries using GoReleaser.

@gerhard can we close this issue since we consolidated into #3283 ?

We have moved the task list to https://github.com/dagger/dagger/issues/3283 with @shykes and can now close this. Going forward, I will be capturing all auxiliary context on that issue, same as I was doing above. 🏁


Rework of alpine version tagging for docker images

The current docker image uses alpine:3.18.3 which is not the best for security updates.
Maybe it's time to provide more docker images tags.
I would suggest something like this :

Latest alpine version : latest, 1.3.0
Major alpine version : 1.3.0-alpine3
Minor alpine version : 1.3.0-alpine3.17,  1.3.0-alpine3.18

I do not feel the necessity to give a "Patch alpine version". Maybe I'm wrong.

Opened a pull request here: https://github.com/willfarrell/docker-autoheal/pull/115


drop queue should infer queue type

https://github.com/tembo-io/pgmq/blob/2783a6b6d61f0f9a65a0b6cfa109d88066ce7992/pgmq-extension/sql/pgmq.sql#L453 takes in a parameter designating whether the queue to drop is partitioned. When that flag is true, pgmq will handle deleting the objects/records from pg_partman as well. So if you run pgmq.drop_queue() on a partitioned queue without the flag, there will be some orphaned pg_partman stuff laying around.
the pgmq.meta table has an attribute that tracks whether table is partitioned. So on the drop_queue() call, I think we should drop the partitioned and replace it with a query to something like select is_partitioned from pgmq.meta where ....

/attempt #302

Can I get this assigned? @ChuckHend

implemented in https://github.com/tembo-io/pgmq/pull/319


problem launching bootstrap

URI::InvalidURIError: can not set user with opaque
  /root/.rbenv/versions/2.4.3/lib/ruby/2.4.0/uri/generic.rb:395:in `check_user'
  /root/.rbenv/versions/2.4.3/lib/ruby/2.4.0/uri/generic.rb:474:in `user='
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb:922:in `proxy_from_env'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb:860:in `proxy='
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb:528:in `initialize'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/fetcher.rb:241:in `new'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/fetcher.rb:241:in `connection'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/fetcher.rb:85:in `initialize'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/source/rubygems.rb:251:in `new'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/source/rubygems.rb:251:in `block in fetchers'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/source/rubygems.rb:249:in `map'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/source/rubygems.rb:249:in `fetchers'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/source/rubygems.rb:383:in `block in remote_specs'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/index.rb:11:in `build'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/source/rubygems.rb:382:in `remote_specs'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/source/rubygems.rb:89:in `specs'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/definition.rb:268:in `block (2 levels) in index'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/definition.rb:266:in `each'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/definition.rb:266:in `block in index'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/index.rb:11:in `build'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/definition.rb:263:in `index'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/definition.rb:257:in `resolve'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/definition.rb:170:in `specs'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/definition.rb:158:in `resolve_remotely!'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/installer.rb:279:in `resolve_if_needed'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/installer.rb:83:in `block in run'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/process_lock.rb:12:in `block in lock'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/process_lock.rb:9:in `open'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/process_lock.rb:9:in `lock'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/installer.rb:72:in `run'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/installer.rb:25:in `install'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/cli/install.rb:65:in `run'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/cli.rb:224:in `block in install'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/settings.rb:136:in `temporary'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/cli.rb:223:in `install'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/cli.rb:27:in `dispatch'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/cli.rb:18:in `start'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/exe/bundle:30:in `block in <top (required)>'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/lib/bundler/friendly_errors.rb:122:in `with_friendly_errors'
  /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bundler-1.16.0/exe/bundle:22:in `<top (required)>'
  /root/.rbenv/versions/2.4.3/bin/bundle:23:in `load'
  /root/.rbenv/versions/2.4.3/bin/bundle:23:in `<main>'

Environment
Bundler       1.16.0
  Platforms   ruby, x86_64-linux
Ruby          2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux]
  Full Path   /root/.rbenv/versions/2.4.3/bin/ruby
  Config Dir  /root/.rbenv/versions/2.4.3/etc
RubyGems      2.6.14
  Gem Home    /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0
  Gem Path    /root/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0:/root/.gem/ruby/2.4.0
  User Path   /root/.gem/ruby/2.4.0
  Bin Dir     /root/.rbenv/versions/2.4.3/bin
OpenSSL
  Compiled    OpenSSL 1.0.2k  26 Jan 2017
  Loaded      OpenSSL 1.0.2k-fips  26 Jan 2017
  Cert File   /etc/pki/tls/cert.pem
  Cert Dir    /etc/pki/tls/certs
Tools
  Git         <IP_ADDRESS>
  RVM         not installed
  rbenv       rbenv 1.1.1-28-gb943955

  chruby      not installed

Bundler Build Metadata
Built At          2017-10-31
Git SHA           10f20fa33
Released Version  true

Gemfile
Gemfile
# frozen_string_literal: true

source 'https://rubygems.org'

gem 'github-pages'
gem 'jekyll-octicons'

group :development, :test do
  gem 'html-proofer'
  gem 'parallel'
  gem 'rake'
  gem 'rubocop'
  gem 'typhoeus'
end

Gemfile.lock
<No /root/government.github.com/Gemfile.lock found>

--- TEMPLATE END ----------------------------------------------------------------
Unfortunately, an unexpected error occurred, and Bundler cannot continue.

Can you please share what's in your ENV, particularly the http proxy variables? Thanks!

%5b%5d%3d-c.ri   empty%3f-c.ri      keys-c.ri       size-c.ri
%5b%5d-c.ri      fetch-c.ri         length-c.ri     store-c.ri
assoc-c.ri       has_key%3f-c.ri    member%3f-c.ri  to_a-c.ri
cdesc-ENV.ri     has_value%3f-c.ri  rassoc-c.ri     to_hash-c.ri
clear-c.ri       include%3f-c.ri    rehash-c.ri     to_h-c.ri
delete-c.ri      index-c.ri         reject%21-c.ri  to_s-c.ri
delete_if-c.ri   inspect-c.ri       reject-c.ri     update-c.ri
each-c.ri        invert-c.ri        replace-c.ri    value%3f-c.ri
each_key-c.ri    keep_if-c.ri       select%21-c.ri  values_at-c.ri
each_pair-c.ri   key%3f-c.ri        select-c.ri     values-c.ri
each_value-c.ri  key-c.ri           shift-c.ri

I am though proxy servers


[Patch]: Added a self target to the status effect handler

Usage:
{
    "type": "minecraft:on_hit",
    "on_hit": {
      "type": "minecraft:status_effect",
      "target": "minecraft:self",
      "effect": "minecraft:poison",
      "level": 1
    },
  }

Closed in favoUr of #985


IPython doesn't reset matplotlib figures

Description
Hello, long time no see ! ;)
I'm trying to switch from the python console I loved so much to the IPython console. One problem I have is that clearing variables when I run a script does not close matplotlib figures. So if I run a script multiple time to debug it, the number of matplotlib windows keeps growing.
The best would be a true reset of the interpreter, but this operation is very slow with IPython.
I understand that you want to get rid of the python console, but right now I don't have a useful workflow without it. I want each run of my script to be completely independant.
What steps will reproduce the problem?

create a script:

import matplotlib.pylot as plt
plt.figure()
plt.plot(range(10))
plt.show()

configure script as "run in current IPython console" and "clear variables on run"
use autobamick backend to get an external window
run script multiple times

What is the expected output? What do you see instead?
I expect to see a unique matplotlib window with my beautiful plot. Instead I see multiple windows.
Please provide any additional information below
It gets worse if plt.figure() is removed from the script: in that case the plots are all stacked in one figure.
Version and main components

Spyder Version: 3.1.4
Python Version: 3.6.1
Qt Versions: 5.6.2, PyQt5 5.6 on Windows

Dependencies
pyflakes >=0.6.0 :  1.5.0 (OK)
pep8 >=0.6       :  1.7.0 (OK)
pygments >=2.0   :  2.2.0 (OK)
qtconsole >=4.2.0:  4.3.0 (OK)
nbconvert >=4.0  :  5.2.1 (OK)
pandas >=0.13.1  :  0.20.2 (OK)
numpy >=1.7      :  1.13.0 (OK)
sphinx >=0.6.6   :  1.6.2 (OK)
rope >=0.9.4     :  0.9.4-1 (OK)
jedi >=0.9.0     :  0.10.2 (OK)
matplotlib >=1.0 :  2.0.2 (OK)
sympy >=0.7.3    :  None (NOK)
pylint >=0.25    :  1.6.4 (OK)

@Nodd it will get fixed. We will add the open in new ipython interpreter that will start a barebones ipython console and then t should be faster and work as expected. @ccordoba12 is working on that

Hey, glad to see you!
As @goanpeca said, we will provide an option to execute in a dedicated IPython console and that will start a new (minimal) kernel every time. That will solve your problem ;-)
Closing as a duplicate of issue #4418.


Cognito

Coverage decreased (-0.2%) to 73.481% when pulling dd5a09566ea895bb3c68ff1f8f9e2cde38ca1b2c on millarm:cognito into 746cd5fa002cc57c3f2f28b07edaaab4f235a02f on Miserlou:master.

Hey @millarm - thanks for this! I've merged it in.
Would you mind sending another PR (after repulling from master) for the documentation which explains how to use this setting in the Responding to AWS Events section of the README?

Done, that, with details & examples, so the next time I come to implement this I can look in the Zappa documentation rather than my other project sources.  :-)


Update app-secrets.md

Sentence was hard to understand. Made it a little more clear and concise.

Hi @mikelunn, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. Real humans will now evaluate your PR.

TTYL, DNFBOT;

@Rick-Anderson to review

@mikelunn  your rewrite presents it as an alternative approach. I'd like to keep the word "avoid" to let folks know they shouldn't be storing secrets in code/config.

@Rick-Anderson Just to make sure I understand. The intention of the section "Environment Variables" is to encourage developers to use environment variables instead of code/config. Is that correct. The new Docs are great by the way!

@mikelunn correct.


proper way to get dimensions of surfaces?

I couldn't figure out how to get the dimensions of images & fonts.
I am currently doing this:
const imgTitle = {
  tex: canvas.createTextureFromSurface(canvas.loadSurface('./assets/title.png')),
  dim: { x: 0, y: 0, width: 158, height: 62  }
}

// ...later
const pos = { ...imgTitle.dim, x: 80, y: 60}
canvas.copy(imgTitle.tex, imgTitle.dim, pos)

where I just put in the right size
and
canvas.renderFont(font, "Press any key.", {
    blended: { color: { r: 255, g: 255, b: 255, a: 255 } },
  }, {
    x: 120,
    y: 130,
    width: 80,
    height: 14,
  })

where I guess, and tweak it until it sort of fits.
Is there a proper way to do this that can tell me the size of some text used by a font, or would there be interest in adding some utilities like this in a PR?
I am imagining things like love has. getLineHeight and getWrap and getWidth are super-useful for laying out text.
Seems like TTF_SizeText might be a good start, but I'm not sure how to wrap it in deno/deno_sdl2.

Not passing width and height will set a suitable height/width for your font based on SDL_QueryTexture().
Making this available to the user will be a little tricky since we don't really have a "query" mechanism in place to request resource information from Rust.
I'm happy to accept PRs adding this mechanism along with bindings to SDL_QueryTexture for the loaded font which is already being tracked in #25

Not passing width and height will set a suitable height/width for your font based on SDL_QueryTexture()

I get this error, leaving it off:
error: TS2345 [ERROR]: Argument of type '{ x: number; y: number; }' is not assignable to parameter of type 'Rect'.

canvas.renderFont(font, "Press any key.", {
      blended: { color: { r: 255, g: 255, b: 255, a: 255 } },
    }, {
      x: 120,
      y: 130
    })

Making this available to the user will be a little tricky since we don't really have a "query" mechanism in place to request resource information from Rust.

I'm not quite sure how to approach it. If I understand the code, it downloads a rust-compiled runtime that starts a socket on port 34254, and reads incoming JSON messages, turning them into operations, with no return values. How would it work?

I'm happy to accept PRs adding this mechanism along with bindings to SDL_QueryTexture for the loaded font which is already being tracked in #25

Sounds good. I will dig in and see if I can figure it out.

Ah! renderFont types are wrong. I'll fix it, meanwhile it should work with // @ts-ignore


Removes session initialization.

Testing if the session initialization in dosomething_global_init causes #5924.

:+1:

Reopened #5620.


Vendor all dependencies

1. What does this change do, exactly?
Vendors all Caddy dependencies. (Plugins not included.)
2. Please link to the relevant issues.
#200, #630, #632, #709, https://github.com/Homebrew/homebrew-core/pull/9233 - probably others too
3. Which documentation changes (if any) need to be made because of this PR?
Remains to be seen. Probably none for most use cases, but this could affect the build server.
The contributing instructions will need to be updated, in case contributors need to change or add dependencies. They'll need to use gvt. (Should be a simple go get to install gvt then gvt fetch -revision to update to a specific version, if not just master.)
4. Checklist
(irrelevant)

This is still an experiment!
I've held out on vendoring on purpose. Before the GO15VENDOREXPERIMENT, there were a variety of third party tools to help with dependency management, but the official go tooling has been ramping up support for vendoring in recent years, and I wanted to see how this would turn out. When the GO15VENDOREXPERIMENT was made official, toolchain support continued to improve and is still continuing to do so even up to Go 1.9. Because of this unstable ecosystem around dependency management and even vendoring specifically, I decided to wait.
Also, the old build server would not have done well with vendoring.
But now, we have new, autonomous build infrastructure. It runs standard go and git commands. The go command now respects the vendor/ folder and "just works" with go get. I'm hoping this will all work together to make reproducible builds possible.
Yes, this change adds 846 files and 391,183 lines of code to the repo. 😬
This PR has no consideration for plugins; I don't intend for this initial step to make Caddy builds with third-party plugins reproducible as well. It's possible that they could vendor their own dependencies but that requires a lot more experimenting; I want to make sure we don't run into problems like this: https://github.com/mattfarina/golang-broken-vendor -- but it is on my list to figure out eventually.
I will be experimenting with reproducing Caddy builds using this branch, and I ask anyone else interested in this to try it out as well. Before taking this step, we want to make sure we get it right!
(Note to reviewers: Obviously, you don't have to review all the changed files, as these are not part of the Caddy project! The reviews should be focused on our approach to vendoring and achieving reproducible builds. So it's not a normal code review, in that sense.)

Okay, case in point: the go tooling does not yet ignore the vendor/ folder when running go vet ./... or go test ./... but it will by Go 1.9. Disregard the CI failures for now, I guess.

Shouldn't you .gitignore everything in vendor except manifest?
The manifest in this setup seems like the lock file, and dependencies should be able to be reinstated based on its definition.

@francislavoie Great questions.

Shouldn't you .gitignore everything in vendor except manifest?

No, because then all developers who wanted to "go get" Caddy would have to use gvt or whatever tool helped make the vendor folder. One goal here is to stick to the stick to the standard go tooling rather than lock ourselves into yet another dependency.

The manifest in this setup seems like the lock file, and dependencies should be able to be reinstated based on its definition.

This is true, if the tool you're using supports that format. But most people will just need go get as usual; only maintainers will need to know what the exact revision is and/or update them (gvt wouldn't even necessarily be required for that but makes it convenient, it's pretty lightweight).

didn't really explore how dependency management works in Go yet.

It doesn't. 😅

Edit: I guess along with this, why not use Glide?

I don't think we're going for package management here, all we need is to vendor dependencies to achieve reproducible builds.

Edit2: Looking at the usually release schedule for Go (in the past 2 years), Go 1.9 should be expected in August I think? So this PR is aiming for that, if I'm correct?

Possibly. If I can find a nice way to run tests on CI and the build server without too many changes (i.e. to ignore the vendor/ folder), we can get it in sooner.

As much as this gives stability ie. clone once and build or use regular go get, I'm wary of keeping this much codes in the repo.
I think anyone willing to build from source is ready to execute more than one command anyway and I do not think the simplicity of go get should take higher priority than the size of the repo.

@abiosoft I know, the size of this makes me squeamish. It's not a comfortable choice and frankly one I don't/didn't really think is/was necessary.
But I realize that Caddy sits in a very vulnerable position: at the center of millions of secured connections, between clients connecting from around the world and sensitive data stored in the machine on which it's running. We can make Caddy open source, and we can make binaries conveniently available, but the only sure way to know that your binary is as intended is to compare it to compiling the source code yourself.
So it's not so much a matter of convenience (running one command or multiple -- and in this case, it's still just one command for everyone), it's security. Build reliability. And in the future, (for example) customers and open source users both will appreciate being able to obtain an older version (if absolutely needed) and be confident that it's the same as what worked for them before.
Doing go get github.com/mholt/caddy/caddy still pulls down all this code and compiles it, but it pulls it from various parts around the Internet, or various accounts on GitHub which can disappear or be compromised and are frankly out of our control. By vendoring we assume control over the code without assuming the maintenance burden (no more than if the code was in a separate repo). Instead of pulling down dozens of different repositories, the go command will find everything it needs in this repo.
Now, I'm still new to vendoring. So I find the feedback valuable, keep it coming. I'm still learning a lot.

Alright that explanation satisfies me. Still feels a bit gross but I understand the reasoning now.
FYI I brought up Glide because it does have a proper lock file. Your point about security and data ownership makes sense though, for long term.
I think there will need to be some clarification about how to contribute if you need to change dependencies in a PR. It was easier before, but now that they are all in the repo, that does change things a bit in those cases.

I'm just afraid it adds additional burden of patch porting from/to upstream.

@mholt

I know, the size of this makes me squeamish. It's not a comfortable choice and frankly one I don't/didn't really think is/was necessary.

I guess we all are not really comfortable with that choice

But I realize that Caddy sits in a very vulnerable position: at the center of millions of secured connections, between clients connecting from around the world and sensitive data stored in the machine on which it's running. We can make Caddy open source, and we can make binaries conveniently available, but the only sure way to know that your binary is as intended is to compare it to compiling the source code yourself.

Although I do not feel comfortable with this PR, I understand that this PR serves a greater purpose! We achieve transparency, security, reliability and reproducibility, which is something we should stand for.

@francislavoie

I think there will need to be some clarification about how to contribute if you need to change dependencies in a PR. It was easier before, but now that they are all in the repo, that does change things a bit in those cases.

@tw4452852

I'm just afraid it adds additional burden of patch porting from/to upstream.

Yup, this is true -- you are both right. Contributors who want to change or update a dependency will need gvt. And I will definitely update the contributing instructions when this is finalized. (It's easy though, just go get the gvt command, then gvt fetch -revision ... should do the trick.)

The repo weight shouldn't be a major concern. I've done this many times, it's fine. Just an experienced opinion.

Good decision @mholt , this a must for all projects like this, I did that on Iris a month ago and I can tell that it works like a charm. After this any caddy version will work no mater what will happen later on, with this change you are not depending on dependencies' repository and other's decisions. Caddy's users and mods should embrace this!

@mholt I'm not a Caddy contributor, so my interest in this is mainly general. My main objection about putting soooo much third-party code into /vendor is that it clutters up the Git log (who are this repo's contributors after such a massive commit? Who is the "Git author" attached to that first bulk commit?).
If you, somehow, could convince the people at @github to somehow exclude commits to vendor folders on Go projects ...

I'm a big proponent of checking in the vendor folder. It takes control over the availability of the dependencies, on top of the security and reliability advantages of lock files. Think left-pad.
I think the friction to updating dependencies is actually a feature, since it stops the owner of one of the 13 repositories Caddy depends on from slipping in a backdoor or bug, intentionally or not, and then slip it out the next day.
About the git history, gvt does not import the git history of the dependencies, so you only see as many commits as having a lockfile. The only thing that is hampered are line count stats, which were never useful IMHO.
Finally, consider that even if you stay on top of the current build with CI, there is no way at all to build Caddy as it was a year ago.
Disk is cheap, let's vendor.

@mholt Personally, I wouldn't characterize vendoring as immature. Package management is changing a lot, but the vendor folder is here to stay. If you don't want it to be included in ./..., you can use the explicit:
go test $(go list ./... | grep -v /vendor/)

Maybe a reason for waiting for go1.9:
https://twitter.com/deeeet/status/864303177599827968

@tw4452852 It's quite possible I'll change the CI scripts to use the workaround that @FiloSottile suggested, but yeah, 1.9 will make things convenient again. I'm also going to need to add vendorcheck to our CI tests.

The main holdup with this PR now is figuring out how to handle the fact that the "vendor" folder is in the top level of the repo but the top level of the repo is a package that all plugins import, and some other applications use it as a library, too (though this is very uncommon--even rare).
So as I understand how this works, other repos that import github.com/mholt/caddy or go get github.com/mholt/caddy will end up pulling down all the vendored code too... but this is not necessary or desirable unless the main function is the target, i.e. the developer wants to install Caddy from source.
I suppose doing a go get github.com/mholt/caddy/caddy (which is the current install command for the caddy executable) will still git clone the whole repo, which includes the vendor folder somewhere in there. (Yeesh, maybe I should have put the main package in a separate repo?)
I guess I'm wondering... is any of this a real concern? Should I move the github.com/mholt/caddy package (that gets imported by all plugins) into a subfolder? It would mean everyone would have to update their import paths, but I think I can make the website's build server ease this transition by rewriting import paths until plugin authors update. So at least that's not so bad. But if it doesn't really solve a problem, or if the problem isn't real, then I'd much rather not do this.
If this isn't a real problem, then I can probably merge this soon enough... thoughts?

So wait, is the problem that the Caddy API exposes types from the dependencies, or just some concern about repository size? Because developer disk is very very cheap, cheaper than your time spent thinking about this.

@FiloSottile I've looked, and as far as I can tell, Caddy does not expose types from the dependencies (phew) so that's not the problem; but I've read numerous places that libraries shouldn't vendor their dependencies; only main packages (commands) should do that.
If that reason is because of the problem with exporting dependency types, then maybe this isn't actually a problem and I can maybe merge this in today.

The reason libraries should not vendor is, by what I have read, that they should take the burden of checking that the transitive dependencies "always" are OK. If libraries also vendored, you'd get this painful big updates where problem would be hard to track down, and it would be hard for, say commonly used library, to spread a security patch, etc.

The reason many of us say not to vendor in libraries is the problem with exporting dependency types. The ecosystem point @bep makes is also valid, but I think Caddy is not a library like any other, it's a big monolithical piece of software that you can import, and programs that use Caddy as a library would IMHO benefit from using the same dependencies as the released version, if anything.

Alright -- thanks for the feedback. That makes sense, and I'm okay going forward.
Now if I can just get these CI tests to pass (by ignoring the vendor folder)...

I've synced with master, updated the CI tests and contributing instructions, and squashed commits.
This is now ready for review. @abiosoft @tw4452852 - I will probably merge by tonight or tomorrow if I don't hear from anyone. A review can be a simple, empty "Approve" or you can request changes if anything comes to mind.

I realize I'm commenting after the fact, but I wanted to add an informational note:

@FiloSottile I've looked, and as far as I can tell, Caddy does not expose types from the dependencies (phew) so that's not the problem; but I've read numerous places that libraries shouldn't vendor their dependencies; only main packages (commands) should do that.

If that reason is because of the problem with exporting dependency types, then maybe this isn't actually a problem and I can merge this in today.

In the ecosystem as it exists today, yes, it can be problematic for any project with importable libraries to commit, specifically because of the incompatible type leakage issue. That was one of the first major problems to be established with vendor after it defaulted to ON in Go 1.6. It sounds like you don't have a leakage problem, though, so 👍 .
The standard we're establishing with dep is that it's fine to commit vendor whether you're importable or not, or for libs or not, as vendor directories contained within dependencies are flattened up into a single, top-level vendor directory. (Any nested vendor dirs are unconditionally stripped out.) So, even if there are some types that leak through, your dependers that use dep would still be fine.

@FiloSottile  even with exported types, vendor worths it. If exported types are few there are plenty of solutions with the todays tools to make it work, it's tricky and needs maintenance but as I said, if there are 1-2 types there is no problem.  The issue would be also solved by the typealias feature that is ready on go1.8.1_typealias branch of golang github repository, so we should expect it for go 1.9 on August :)


Rename classes and function to more sensible names

Resolve #82, for detail please refer to issue.
Note that this is a code refactoring PR.
Remember to delete following files.

/data/*
/preferences.json

LGTM

Seems like there is merge conflict with pr #88 as UserGuide was updated. Maybe u can commit the new changes into this pr to resolve the conflict


Proposal: Prevent unnecessary re-renders

On maps with only a small amount of clusters, it often happens that all clusters are redrawn upon zoom, even though nothing has changed. The resulting flickering looks very buggy and distracting. As a workaround, we use an algorithm that compares the actual markers:
import {
  AlgorithmInput,
  Cluster,
  SuperClusterAlgorithm,
  SuperClusterOptions,
} from "@googlemaps/markerclusterer";
import { shallowEqual } from "fast-equals";

const summarize = (c: Cluster) => `${c.position} ${c.count}`;

export class CachedSuperClusterAlgorithm extends SuperClusterAlgorithm {
  constructor(opts: SuperClusterOptions = {}) {
    super(opts);
  }
  calculate(input: AlgorithmInput) {
    const prevClusters = this.clusters;
    const output = super.calculate(input);
    const { changed, clusters } = output;
    if (changed && prevClusters) {
      if (shallowEqual(prevClusters.map(summarize), clusters.map(summarize))) {
        return { changed: false, clusters };
      }
    }
    return output;
  }
}

I was wondering if it made sense to incorporate such a check into the library, and would be happy to contribute a PR if you think this would be a good idea.

@fgnass Thank you for opening this issue. 🙏
Please check out these other resources that might be applicable:

Support Console - If you have a support contract.
Discord - chat with other developers
StackOverflow - use the google-maps tag
Google Maps Platform issue trackers

This is an automated message, feel free to ignore.

Seems reasonable to add this. The flickering is definitely suboptimal. What use cases exist for not doing this?

I can't think of any, besides of performance, maybe. I'll create a PR and change the code so that the string-conversion only happens when the size of the arrays is equal, and also comparing the items one-by-one instead of mapping both arrays upfront.

While implementing my proposed solution, I realized that caching on algorithm level is not a good idea, since the renderer might still want to display the markers differently based on the ClusterStats which are likely to change, even if the position and count of an individual cluster stays the same.
I therefore moved the caching to markercluster.ts which now takes the output of the renderer into account
Here's a screen capture of the improved version:
https://user-images.githubusercontent.com/89450/156614421-e419df06-b7e6-4142-9bc4-28614cdc5560.mov
.

For reference, here's how the same example looks with the original implementation:
https://user-images.githubusercontent.com/89450/156614658-237bb46a-be5a-463b-980e-ed19e9e026d7.mov
:

Even though it's already much better, I'm still not super happy with the result. When looking at the performance panel in the dev tools, there are still a lot of dropped frames and tasks that take much too long.
One path of optimization that might be worth to test, would be to reduce the load on the garbage collector by reusing the cluster markers instead of throwing them away.
If I find some more time, I will give it a try. If you want, you can take a look at the current work in progress in my forked repo.
Another interesting find was, that the actual markers flicker a lot less, if they are constructed with {optimized: false}.

I found some time to try out a few more ideas. The one that provides the best results does the following:

After each render pass, remember the markers that have been rendered
Then, in the next render pass:

For each marker, look up the closest one from the previous pass
If it was not more than 45px away, reuse and update it

In order for this to work, renderers must return google.maps.MarkerOptions. If they return Marker instances like before, the algorithm falls back to the old behaviour.
I did some additional profiling to check where most of the CPU cycles are spent and noticed that the number of DOM nodes that are needed for a single marker has quite an impact. I updated the default renderer so that the label is now part of the SVG icon, which saves a lot of nodes.
Here's a screen capture that shows how markers now "morph" into their new state:
https://user-images.githubusercontent.com/89450/156791186-ac92e31c-d8b0-4812-b1ad-41f7fd078fbb.mov

Just my two cents worth -- this "problem" will likely persist as the original clusterers design was to use Marker objects as input.  A better approach is to use "data-points" that contain a latitude, longitude and data key.   Using the data-points approach saves on memory and CPU -- especially when it comes time to render markers on the map as only those markers in the visible area of the map need to be created.   I have tested this and found a data-point design to be significantly more efficient (and without any "flickering" !
Code is available for play at: https://github.com/bdcoder2/grid_clusterer

@bdcoder2 I checked out your example, but unfortunately it exposes the same kind of flickering as the original (at least on my machine). If you play back the screen recording frame by frame, you can see that all markers briefly disappear before the new ones are rendered.
https://user-images.githubusercontent.com/89450/156990458-1bac667f-4484-4910-a458-3c553ed4980a.mov
In my experience, using markers as input is only a problem if they are initially added to the map. Unless they are added to the DOM, their overhead is pretty much negligible.

@bdcoder2 I totally get your point, but this issue is not about the initial load or performance improvements in general, but solely about trying to reduce the number of markers that need to be added/removed on each render cycle. All of this without introducing any breaking changes.
@jpoehnelt I'm quite happy now with how the result turned out and would appreciate your feedback. Of course, the docs would need to be updated too, in order to mention that custom renderers should return MarkerOptions instead of Markers to benefit from the performance gains. Please let me know if you would be interested in a PR.

@fgnass I saw you mentioned that you're using svg for your markers; changing to png would also help performance. See this YouTube video and the links in the description for more detail. https://www.youtube.com/watch?v=rU9OcRrNkVg

Just FYI @fgnass https://stackoverflow.com/questions/73838247/markers-and-clusters-are-flashed-redraw-when-zoom-is-changing-googlemaps-js-mar

at
https://github.com/googlemaps/js-markerclusterer/blob/1193ff4/src/markerclusterer.ts#L114
what is the purpose/usage (there is no explanation in the documentation) of:
onAdd
it seems to bind to idle and render ... and this seems to be the cause of 'flashing' renders in my app ...
for me the markers do not flash when i circumvent the onAdd via
this.clusterer.onAdd = () => {};
and actually, i don't see where onAdd would be called from

onAdd is from the base class (OverlayView).
See https://developers.google.com/maps/documentation/javascript/customoverlays#add


feat: Communication

Hi there, :wave:
in line with my earlier oral feedback, addressing the use-case of wanting to address some audience and some potential consumership for video games, perhaps it might be prudent to utilize a more visual logbook of sorts.
Following your publications on the development progress, I noticed that their visual representation mostly aims at adapt software versioning users.
So I would kindly ask for some blog posts or similar to illustrate the current development. For example the next thing that would spring to my mind for a hassle free approach there would be to utilize some codaishin.github.io Github Page.
I hope you are doing well,
regards,
Lars/Quimoniz

Thank you for the feedback. I'll be certainly checking out Github pages, but I can't promise any time frame yet.


Incorrect status codes returned when unknown command is encountered (spec violation)

selendroid 0.9.0 on Android 19 (4.4)
According to the JsonWireProtocol, unknown commands should respond with 404 Not Found and unimplemented commands should respond with 501 Not Implemented. Selendroid incorrectly responds with 500 Internal Server Error.

Works now, just tried on master:
~ $ curl -v http://localhost:4444/wd/hub/not-a-command
* Hostname was NOT found in DNS cache
*   Trying <IP_ADDRESS>...
* Connected to localhost (<IP_ADDRESS>) port 4444 (#0)
> GET /wd/hub/not-a-command HTTP/1.1
> User-Agent: curl/7.37.1
> Host: localhost:4444
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Connection: close
< Content-Encoding: identity
<
* Closing connection 0


slides Per Column issue

This is a (multiple allowed):

[x] bug

[x] enhancement

[x] feature-discussion (RFC)

Swiper Version: EXACT RELEASE VERSION OR COMMIT HASH, HERE.

Live Link or JSFiddle/Codepen or website with isssue: PREFERABLY (IF YOU WANT YOUR ISSUE TO BE RESOLVED ASAP).

What you did
I tried to use slidesPerColumn with its css but its not working after new update.
please update Stackblitz code
Expected Behavior
Slider with two columns
Actual Behavior
All slides display under each other
P.S. Remember, an issue is not the place to ask questions. You can use Stack Overflow
for that.
Before you open an issue, please check if a similar issue already exists or has been closed before.

Your issue is closed because it is not filled correctly. Before you open an issue please review the contributing guideline.
Please fill the issue template correctly and:

Clearly describe the issue including steps to reproduce when it is a bug.
Make sure you fill in the earliest version that you know has the issue.
Provide live link or JSFiddle/Codepen or website with isssue

It will save a lot of time for other contributors to check your issue and fix it.

The author wrote everything correctly, in the new version the slider for several columns does not work correctly, and you just close the problem instead of solving it!

No, @AvdeevaEvgeniya, the issue is really missing the required information, if you have the same problem and want it to be fixed as soon as possible, consider opening a new issue yourself with all required information: library version, platform, reproduction link and steps, and a clear non ambiguous description of whats happening and how to reproduce it.
Complain will solve nothing.

Anyway, "I" found a workaround to this, use display: grid on .swiper-wrapper instead of the default flex
  .swiper-wrapper {
      display: grid;
      grid-template-columns: 1fr;
      grid-template-rows: auto auto; // here is the amount of slidesPerColumn you want
      grid-auto-flow: column;

      & > .swiper-slide {
        height: fit-content;
      }
    }

Hello everyone!
@vhoyer I used your solution and it worked successfully when used without the following configuration:
        centeredSlides: false,
        loop: false,

When I try to use with these tow options set to TRUE, the slider breaks and all the slides are gone. I tried several solutions with css only but I could not get it. I wonder if you have tried your solution like this.
By the way, sorry to go back again in the issue!
Thanks!


Language switch

Added a parameter for starting pattypan in a specific language,
Added language fallback

Wrong target branch, closing

@yarl suggestion: removing the lang parameter but making it driven by the settings file.
Then later we could just use the GUI to update the setting and restart the app...