Add argfile support

 Issue by XAMPPRocky
Friday Feb 14, 2020 at 11:09 GMT
Originally opened as https://github.com/clap-rs/clap/issues/1693

Describe your use case
I've received requests (XAMPPRocky/tokei#437) to add argfile support to tokei, it would be nice if this was provided in clap so that it can be shared and reused by all crates.
Argfiles aren't a specific standard, and there are varying levels of support. The best documentation I found on it was documentation on javac's CLI.
Describe the solution you'd like
CLI
tokei @argfile

Rust
App::new("My Super Program").argfile(true)

 Comment by CreepySkeleton
Friday Feb 14, 2020 at 12:32 GMT

Here's a formal description for the sake of possible implementors:

App::new("My Super Program").global_setting(AppSettings::AllowArgfiles) tells clap to look after positional arguments starting with @.  These args will be treated as @argfiles.
@argfiles can only be preceding so explicit cli options override them, i.e it's not possible to do something like app -J --foo @argfile1 bar @argfile2 baz. This fact should simplify the implementation considerably.
Each @argfile is treated as a list of command line options/args. In fact, the args simply inserted into processing queue in place of @argfile occurrence.

One line corresponds to one arg. The following @argfile corresponds to three args.

-J foo
Bar
"rm -rf /"

Each line is treated verbatim. Nothing is stripped, no escapes.
If the file end with a newline character it will be stripped (but only one!).
If a file is not found - it's a parsing error.

 Comment by pksunkara
Monday Feb 17, 2020 at 09:22 GMT

So, with #1697, I rewrote how input is being parsed. That would enable us to use @argfiles even in the middle.

 Comment by DarthUdp
Wednesday Feb 26, 2020 at 07:18 GMT

@pksunkara are you working on this or can I give it a go?

 Comment by pksunkara
Wednesday Feb 26, 2020 at 09:09 GMT

Please go ahead.

 Comment by pksunkara
Wednesday Feb 26, 2020 at 15:59 GMT

If the App is taking an argfile method, then it's better if it's implemented for all subcommands instead of just at the end.

 Comment by DarthUdp
Wednesday Feb 26, 2020 at 15:57 GMT

I will implement to @CreepySkeleton's spec then.

 Comment by pickfire
Saturday Mar 07, 2020 at 15:47 GMT

How about @argfile as a value of some commands, like curl -F f:1=@file ix.io?
edit: this looks different from what I thought

 Comment by XAMPPRocky
Sunday Mar 08, 2020 at 00:21 GMT

@pickfire Well since @file is the value in that case I would expect my app to handle that rather than the cli library. I've only seen that in the context of curl so I don't know if there's wider consensus on what it should do. It probably needs a seperate issue.

 Comment by michael-smythe
Thursday Apr 09, 2020 at 13:31 GMT

@pksunkara Glad too see this is being worked on! I just wanted to add a comment since this is closing out #1519. I think the spec described by @CreepySkeleton is a really good step and a useful starting point. One thing that I think it lacks is the ability to have comments within your config file. This would be useful if you have a large number of options that are presented by the app, and it would allow end-users to build very readable configs. I could see it being annoying to open up a config file to slightly alter how the app is running and not remembering exactly what each option does. With the ability to comment and/or use the actual Arg::with_name("name") to control the option/switch/flag within the config I believe it would increase the usability for the end-user. That being said any progress towards config files is good in my book, just something to consider for the long term. Thanks!

 Comment by kbknapp
Tuesday Apr 28, 2020 at 14:25 GMT

Couple of thoughts:

I like that this is gated behind an AppSettings variant because, although rare @foo does come up in CLIs. I wouldn't want us supporting argfiles to break some CLI. The cross section of CLIs that use @foo and want to use this new feature is probably so minimally niche that its fine that there is a minimal conflict here.
Comments is probably easy enough to implement. Any line starting with # is ignored. Inline comments -J=foo #this describes something is not allowed. Having to scan each line and split them is not worth it IMO.
Likewise blank lines should probably be ignored as well.
I'm also curious if anyone can think of something that breaks or gets complicated by ignoring blank lines or # lines?
I'm still not sure how I feel about clap handling file I/O during parsing. My main concern is two fold:

Code bloat (could be handled by a cargo feature flag, but then at that point do we need the AppSettings? Maybe still yes, just to make the parsing more simple with a branch. I'd be fine either way.)
All the idiosyncrasies that pop up with files (permissions, encodings, etc.)...which in turn leads back to sub-bullet 1 (code bloat)

Something I didn't see spelled out in the spec above is, are these @argfiles treated as positional arguments i.e.:

do they show up in the help message, and if so, how? (I'd say they go in the usage string, but nothing more)
Is the user expected to make a Arg place holder for the argfile? (I'd say no)

 Comment by CreepySkeleton
Tuesday Apr 28, 2020 at 18:39 GMT

I like that this is gated behind an AppSettings variant

Agreed. Whatever we'll end up with, it must be opt-in.
I have a bad feeling about comments/blank lines.

I don't remember clearly, but a few years ago I was working on a CLI app which was receiving URL parts through command line args (among other things), like docs.rs #anchor. As you see, there are (possibly rare) cases where arguments are expected to start with #. I'd say I'm OK with this part because it's too rare to really be accounting for. Maybe "# plus an optional space between # and the comment".

I would absolutely expect empty lines to be processed as empty values. It's pretty common - not common-common, but I saw quite a few of cases - where this kind of values is used to override previous occurrences, especially in combination with alias.
Imagine I've got a a script in my PATH. This script is literally alias a='application-with-long-name --option value' because I want this --option to have value 99% of time, but occasionally
I want an explicitly empty string there (maybe it's some sort of annotation). Not really sure people would use @argfile for this kind of thing, just a thing to consider.

I am purposely not expressing the difference between "empty lines" (\n\n) and "blank lines" (\n \t \n, ASCII whitespace only). They should treated in the same manner - passed over "as is".

The link to javac spec doesn't mention empty lines and comments, too.

These are argfiles, not configs. What comments can be used for there?

All the idiosyncrasies that pop up with files (permissions, encodings, etc.)...which in turn leads back to sub-bullet 1 (code bloat)

Argfiles can be used only on command line. If the user typed @file, this is a sort of contract: "This file exists and accessible". If that is not so, the user made a mistake, return ErrorKind::ArgfileInaccessible and render it as Can't read the '<path>' argfile: <error from OS>.
Encoding: UTF-8 only. If you're using something else nowadays, it's either some kind of database (argfiles aren't), or this is blasphemy.  Send the possessed hard drive to Vatican, the holy priests will perform an appropriate exorcism ritual on it and erase the possessed file. Seriously, who uses non-UTF-8 encodings in files, still?

do they show up in the help message, and if so, how? (I'd say they go in the usage string, but nothing more)
Is the user expected to make a Arg place holder for the argfile? (I'd say no)

Agreed on both, an AppSetting is enough.

 Comment by kbknapp
Tuesday Apr 28, 2020 at 19:41 GMT

[..] there are (possibly rare) cases where arguments are expected to start with #

Good point on the argument values that begin with #. However, that would only possibly apply to positional arguments, because --foo=#bar wouldn't be parsed as a comment with explicit disallowing of inline comments. Pragmatically, if encountering this error, I would change the line to "#value" and see if that fixed the issue. Either way I think it's such a small area that this little hiccup shouldn't devalue it.
As for what value comments have in an argfile, I'd agree probably not too much but can still see the use case where a complex CLI ships a default argfile but also comments what each switch/option is doing. I know in my day job there are instances where we ship some gnarly incantations where comments would absolutely help future workers quickly understand what's there.

I would absolutely expect empty lines to be processed as empty values. [..]

Again, that's just for positional values, so overriding a positional value isn't a good idea just in the general sense. I'd also fall back to, if this was truly the outcome I expected and my argfile wasn't doing what I thought it would, my initial guess would be to make the line "". Like Rust, I'd prefer explicit over implicit. Special white space (\n\t\n vs \n\n) :vomiting_face: haha

Seriously, who uses non-UTF-8 encodings in files, still?

Windows. :cry: Dealing with UTF-16, WTF-8, and having to do things like BOM sniffing doesn't sound super appealing to an argument parser unless it's well gated and opt-in only :stuck_out_tongue_winking_eye: I think @BurntSushi (sorry for the ping) has had some very strong thoughts on dealing with encodings on Windows.

 Comment by kbknapp
Wednesday Apr 29, 2020 at 14:00 GMT

here is roughly where I landed for ripgrep's configuration file

Thank you!

[..] I think just declaring that it must be valid UTF-8 is fine

I think this is where I'm landing too. It's also historically how some of my decisions have gone, so I'm ok with this so long as it's documented somewhere.

Otherwise, I think the single biggest downside to this approach is that the file fomat does not support environment variables.

This might be slightly different in clap's case becase of the intrinsic support for env vars already. A hypothetical argfile (ignoring the need for some escape):
--foo=SOME_ENV_VAR

Is semantically equivilant to foo_arg.env("SOME_ENV_VAR") to the point that there is no benefit listing it in the argfile, minus maybe just being super verbose/explicit? It's also possible I'm overlooking some angle :stuck_out_tongue_winking_eye:

 Comment by kbknapp
Wednesday Apr 29, 2020 at 14:33 GMT

Ah yes, you're correct. I was assuming the CLI creator added the env("..") to an argument. In the case they didn't (which is probably the most likely assumption) then yes, an argfile would allow the CLI user to specify a var to pull the value from.
Hmm...
So my initial thought is while I can see the value in that, I also think it may lead to some gotchas when that env var is unset, etc. In bash it's easy to use a var or default to some value, but being honest I have little interest in trying to account for all those edge cases in  an argfile format. I think I'm leaning towards just saying supporting env vars is something we're not doing for now, and if we find some easy way in the future all the better.
Especially since if the CLI creator did use the env("..") directive on fields that are likely to be pulled from ENV vars (applications that are highly in line with 12 factor guidelines, or secrets, etc.) and the existence of various env templates or use of things like envsubst, it's not too much of a workaround for those that need this.

 Comment by kbknapp
Wednesday Apr 29, 2020 at 15:26 GMT

When going through the ripgrep specs, it got me thinking; what are the thoughts on instead of dedicated CLI syntax @foo, what if we just add support to the App instance? Such that the App instance would support a list of paths in which to look for the argfile (in order presumably). This way we essentially side step all the syntax issues (help messages, conflicting with existing CLIs, etc.). In addition it would allow the clap consumer a choice of:

Informing user of specific locations to place argfile for "automatic" handling
Using a "partial/pre parse" idea I've been chewing on lately (...more below) so that they can use their own argument to support such argfiles if so desired --arg-file=foo either in place of standard locations, or in addition/overriding standard locations defined by their App struct.

Partial/Pre Parsing
A way to declare a subset of arguments to be parsed, then return from parsing early. With the expectation that parsing will be called again, and run to full completion.
This most often comes up when you want some argument to handle some aspect of the CLI itself (such as --color flags to control error messages during parsing, or in the above --arg-file which needs to determine a file to load prior to parsing the rest of the CLI).
From a user stand point, nothing changes. The run the CLI and things just work. From the a clap consumer stand point, it essentially allows you to pull out some values from the CLI (ignoring everything else), change something about the CLI and continue parsing.
// .. already have some App struct
let pre_matches = app.get_partial_matches("color"); // Only parse the `--color` option, ignore all else
app = if let Some("never") = pre_matches.value_of("color") {
    app.setting(AppSettings::ColorNever)
} else {
    app
};

let matches = app.get_matches();

Maybe it's biting off more than we want to chew for now, but this issue seems to come up time and time again where we want to parse some particular subset of arguments, and use those values to inform how the CLI itself behaves.


Support for the Raspberry Pi RP2040 soc

Is this request related to a missing driver support for a particular hardware platform, SoC or board? Please describe.
This request is for the support of RP2040 SoC by the Raspberry Pi foundation. This is a dual core
Cortex-M0+ device. The reason I think this would be a great addition to Zephyr is that it is a well documented SoC
with a well documented SDK too.
Describe why you are asking for this support?
Describe why you are asking for this support instead of contributing it directly to the tree
The reason I am asking for this support is because I am interested in using Zephyr on this chip.
However, I'm not that experienced with porting socs to zephyr, so i would probably need some guidance or help
taking on this task
Also seeing as Arduino, Adafruit and other distributors are creating boards for this soc.
If this is a new board or SoC, please state whether you are willing to maintain the Zephyr support for it if it is included in the main tree
If this SoC were to be included in the main tree i would be willing to maintain the code for it.
Additional context
Add any other context or graphics (drag-and-drop an image) about the hardware here.

CC @carlescufi

Same issue here: https://github.com/zephyrproject-rtos/zephyr/issues/32193

Not very helpful to have a bot close tracking issues like this one :/ Will only result in new ones being created and this adding to the noise rather than the intended opposite.

There is active interest in the issue.

#37134

#37134

The Pico and the Compute Module are very different

@voxpelli Sorry. I wanted to reference another github issue.

There is active interest in the issue.
(Any way to make the Stale bot ignore an issue?)

Active interest

The HAL request is here: #40817


README suggestions

I've spent some more time unironically reading the README and I have a few ideas:

Format ruby code with ruby codefence
Wordsmith the Features section to highlight the most interesting bits. In your opinion what are the 3-5 most important features? Let's put those at the top and expand them.
I wonder if we could get rid of the API section and link to the modules from above (in Features).
At least one badge 😄
Want to stick a logo in there? https://looka.com

So the reason why I traditionally avoid ruby code-fenses is because only RedCarpet seems to support GitHub Flavored Markdown (GFM) out-of-the-box. I use kramdown for YARD's markdown processing which is pure-Ruby and guaranteed to just-work on any Ruby. Kramdown did recently add a separate parser for GFM, but I wasn't able to shoe-horn it into YARD. Normally I'm kind of a stickler about being able to generate docs locally to inspect the formatting. We could make an exception and add GFM code-fenses so the Examples look less bland?
Normally I like to list the unique features of the library. Perhaps we could compromise and create key Feature bullet-points with sub-bullet points that go in-depth?
Could we still link to each module's documentation in the Examples sub-sections? Maybe put the link in the header name or a "Read more" link below the code example?
Code coverage, maybe a code-climate code quality score (if they don't ding us to much), and maybe a documentation coverage badge?
Is there any fine print for this service? Do they somehow retain the copyright or IP somehow? Provided if I had the time and inspiration, I'd fire up Inkscape and create an SVG logo involving model kits and terminals.

OK, I can get back to this now! Work distractions 😄

Let's add 'em!
I'll try it
Sure, good plan
I will investigate
I realized you have to pay... I could make a cheesy one if you want, colored text like I did for https://github.com/gurgeous/httpdisk. Totally up to you.


Retry is not working when skipDuplicateUrls is enabled

I'm facing the same issue,
If set skipDuplicateUrls param to true, cluster won't retry anymore.

I'm facing the same issue,
If set skipDuplicateUrls param to true, cluster won't retry anymore.

Also having this issue


prometheus_auth is not passed to config

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


cd: Create direct CDN deployment and NPM deprecation workflow

Description
This PR introduces two new workflows to help with the deprecation process for both CDN and NPM releases.
For CDN:

Previous method involved re-releasing the previous stable version to overwrite the 1.x versions. This had to rely on the existing CD workflow, which triggers smoke test and NPM release as well.
Instead, this PR creates a workflow dedicated to just re-releasing with a previous stable version without the smoke test or the NPM release. This helps reduce the time to deprecate for CDN releases. Verified in GAMMA stage.
To use this workflow, the contributor will create a release branch of a previously safe commit, trigger this workflow.

For NPM:

Previously to deprecate, we relied on manual work. This PR creates a workflow that requires two inputs, 1) deprecating version and 2) deprecating reason. With these inputs, the workflow will automatically deprecate the NPM version.
Tested with https://www.npmjs.com/package/aws-rum-web-experimental/v/1.9.0-alpha.0

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

question: In our rollback steps to deprecate CDN, the runbook indicates that we need to "Revert the affected major (M.x) and minor (M.m.x) auto-update versions to the previous good version by deleting the bad (most recent) version." Do we not have to still delete the bad (most recent) version? Or does deploying the previous stable version do this under the hood?

question: In our rollback steps to deprecate CDN, the runbook indicates that we need to "Revert the affected major (M.x) and minor (M.m.x) auto-update versions to the previous good version by deleting the bad (most recent) version." Do we not have to still delete the bad (most recent) version? Or does deploying the previous stable version do this under the hood?

Synced offline that the intended behavior should be to only release previous stable version to update 1.x or 1.x.x, not remove the bad version. LGTM.

Regarding the CDN workflow, is there any disadvantage to re-running the smoke tests? I feel like doing that validation after deployment can only be a good thing.
What do you think about merging these two scripts into a single "deprecate" workflow?


Handling of invalid requests results in infinite recursive call to the proxy

Describe the bug
While testing code for https://github.com/xvzc/SpoofDPI/issues/31 I accidentally discovered that if you make a request to the proxy as a regular webpage (e.g. omit --proxy from curl -v --proxy http://<IP_ADDRESS>:8080 *something*) SpoofDPI goes into endless recursion querying self again and again.
Expected behavior
SpoofDPI recognizes when request wants it to proxy connection SpoofDPI proxy itself.
Actual behavior
Infinite recursion, see below
How To Reproduce?

Run spoof-dpi --debug
curl -v http://<IP_ADDRESS>
Observe endless stream of

...
DEBU[2024-07-21T19:54:17+03:00] [HTTP] Start
DEBU[2024-07-21T19:54:17+03:00] [HTTP] Finished <IP_ADDRESS>:48844
DEBU[2024-07-21T19:54:17+03:00] [HTTP] Closing client Connection.. <IP_ADDRESS>:48844
DEBU[2024-07-21T19:54:17+03:00] [HTTP] New connection to the server <IP_ADDRESS> <IP_ADDRESS>:50862
DEBU[2024-07-21T19:54:17+03:00] [HTTP] Error reading from <IP_ADDRESS>:48844 timed out
DEBU[2024-07-21T19:54:17+03:00] [PROXY] Request from <IP_ADDRESS>:50862

GET / HTTP/1.1
Host: <IP_ADDRESS>:8080
User-Agent: curl/8.8.0
Accept: */*

DEBU[2024-07-21T19:54:17+03:00] [HTTP] Start
DEBU[2024-07-21T19:54:17+03:00] [HTTP] Closing server Connection.. <IP_ADDRESS> <IP_ADDRESS>:48858
DEBU[2024-07-21T19:54:17+03:00] [HTTP] Sent a request to <IP_ADDRESS>
DEBU[2024-07-21T19:54:17+03:00] [HTTP] Finished <IP_ADDRESS>:48858
DEBU[2024-07-21T19:54:17+03:00] [HTTP] New connection to the server <IP_ADDRESS> <IP_ADDRESS>:50876
DEBU[2024-07-21T19:54:17+03:00] [HTTP] Closing client Connection.. <IP_ADDRESS>:48858
DEBU[2024-07-21T19:54:17+03:00] [HTTP] Sent a request to <IP_ADDRESS>
DEBU[2024-07-21T19:54:17+03:00] [PROXY] Request from <IP_ADDRESS>:50876
...

Client information
OS: Arch linux
Version: latest git at the moment: 83326a0a663a0cad0ab62bbbb8863b09f937718e
How it could be fixed
Pass which port proxy is listening on to HandleHttp(s) methods, then right after this
https://github.com/xvzc/SpoofDPI/blob/83326a0a663a0cad0ab62bbbb8863b09f937718e/net/conn.go#L73-L76
Add (after the types are properly converted)
	// Avoid recursively querying self
	if port == proxyPort && isOurAddress(net.ParseIP(ip)) {
		log.Error("[HTTP] Invalid request: final target has the same IP and port as our proxy")
		lConn.Close()
		return
	}

Where isOurAddress() is
// Check if a given IPv4 assigned to one of our host's network interfaces
// Doesn't handle WAN addresses
// WARNING: needs access to AF_NETLINK address family to work on Linux.
// This is usually not a problem, unless you're filtering system calls or
// running it as systemd service with RestrictAddressFamilies.
func isOurAddress(ip net.IP) bool {
	// we don't handle IPv6 at all it seems
	if ip.To4() == nil {
		return false
	}

	if ip.IsLoopback() {
		return true
	}

	// Get list of available addresses
	// See `ip -4 addr show`
	addr, err := net.InterfaceAddrs() // needs AF_NETLINK on linux
	if err != nil {
		log.Error("[IP CHECK] Error while getting addresses of our network interfaces: ", err)
		return false
	}

	for _, addr := range addr {
		if ipnet, ok := addr.(*net.IPNet); ok {
			if ipnet.IP.To4() != nil && ipnet.IP.To4().Equal(ip) {
				return true
			}
		}
	}
	return false
}

so basically:
Patch

diff --git a/net/conn.go b/net/conn.go
index e152854..e7c202c 100644
--- a/net/conn.go
+++ b/net/conn.go
@@ -59,7 +59,37 @@ func (conn *Conn) ReadBytes() ([]byte, error) {
 	return ret, nil
 }

-func (lConn *Conn) HandleHttp(p *packet.HttpPacket, timeout int, resolver *dns.DnsResolver) {
+// Check if a given IPv4 assigned to one of our host's network interfaces
+// Doesn't handle WAN addresses
+func isOurAddress(ip net.IP) bool {
+	// we don't handle IPv6 at all it seems
+	if ip.To4() == nil {
+		return false
+	}
+
+	if ip.IsLoopback() {
+		return true
+	}
+
+	// Get list of available addresses
+	// See `ip -4 addr show`
+	addr, err := net.InterfaceAddrs()
+	if err != nil {
+		log.Error("[IP CHECK] Error while getting addresses of our network interfaces: ", err)
+		return false
+	}
+
+	for _, addr := range addr {
+		if ipnet, ok := addr.(*net.IPNet); ok {
+			if ipnet.IP.To4() != nil && ipnet.IP.To4().Equal(ip) {
+				return true
+			}
+		}
+	}
+	return false
+}
+
+func (lConn *Conn) HandleHttp(p *packet.HttpPacket, timeout int, resolver *dns.DnsResolver, proxyPort string) {
 	p.Tidy()

 	ip, err := resolver.Lookup(p.Domain())
@@ -75,6 +105,13 @@ func (lConn *Conn) HandleHttp(p *packet.HttpPacket, timeout int, resolver *dns.D
 		port = p.Port()
 	}

+	// Avoid recursively querying self
+	if port == proxyPort && isOurAddress(net.ParseIP(ip)) {
+		log.Error("[HTTP] Invalid request: final target has the same IP and port as our proxy")
+		lConn.Close()
+		return
+	}
+
 	rConn, err := DialTCP("tcp", ip, port)
 	if err != nil {
 		log.Debug("[HTTP] ", err)
@@ -105,7 +142,7 @@ func (lConn *Conn) HandleHttp(p *packet.HttpPacket, timeout int, resolver *dns.D

 }

-func (lConn *Conn) HandleHttps(p *packet.HttpPacket, timeout int, resolver *dns.DnsResolver) {
+func (lConn *Conn) HandleHttps(p *packet.HttpPacket, timeout int, resolver *dns.DnsResolver, proxyPort string) {
   ip, err := resolver.Lookup(p.Domain())

 	if err != nil {
@@ -120,6 +157,13 @@ func (lConn *Conn) HandleHttps(p *packet.HttpPacket, timeout int, resolver *dns.
 		port = p.Port()
 	}

+	// Avoid recursively querying self
+	if port == proxyPort && isOurAddress(net.ParseIP(ip)) {
+		log.Error("[HTTPS] Invalid request: final target has the same IP and port as our proxy")
+		lConn.Close()
+		return
+	}
+
 	rConn, err := DialTCP("tcp4", ip, port)
 	if err != nil {
 		log.Debug("[HTTPS] ", err)
diff --git a/proxy/proxy.go b/proxy/proxy.go
index 7831f41..dab6620 100644
--- a/proxy/proxy.go
+++ b/proxy/proxy.go
@@ -3,6 +3,7 @@ package proxy
 import (
 	"fmt"
 	"os"
+	"strconv"

 	log "github.com/sirupsen/logrus"
 	"github.com/xvzc/SpoofDPI/dns"
@@ -72,12 +73,13 @@ func (p *Proxy) Start() {
 				return
 			}

+			proxyPort := strconv.Itoa(p.Port())
 			if pkt.IsConnectMethod() {
 				log.Debug("[HTTPS] Start")
-				conn.HandleHttps(pkt, p.timeout, p.resolver)
+				conn.HandleHttps(pkt, p.timeout, p.resolver, proxyPort)
 			} else {
 				log.Debug("[HTTP] Start")
-				conn.HandleHttp(pkt, p.timeout, p.resolver)
+				conn.HandleHttp(pkt, p.timeout, p.resolver, proxyPort)
 			}
 		}()
 	}

which results the above request being handled like so:
...
DEBU[2024-07-21T20:18:01+03:00] [PROXY] Request from <IP_ADDRESS>:51484

GET / HTTP/1.1
Host: <IP_ADDRESS>:8080
User-Agent: curl/8.8.0
Accept: */*

DEBU[2024-07-21T20:18:01+03:00] [HTTP] Start
ERRO[2024-07-21T20:18:01+03:00] [HTTP] Invalid request: final target has the same IP and port as our proxy
*continuing as normal*

Another note
Also, while hunting down why it did so, I noticed that you don't close connections on errors.
E.g. connections here
https://github.com/xvzc/SpoofDPI/blob/83326a0a663a0cad0ab62bbbb8863b09f937718e/proxy/proxy.go#L66
https://github.com/xvzc/SpoofDPI/blob/83326a0a663a0cad0ab62bbbb8863b09f937718e/proxy/proxy.go#L71
and here (same with HandleHttps)
https://github.com/xvzc/SpoofDPI/blob/83326a0a663a0cad0ab62bbbb8863b09f937718e/net/conn.go#L66-L69
https://github.com/xvzc/SpoofDPI/blob/83326a0a663a0cad0ab62bbbb8863b09f937718e/net/conn.go#L79-L81
should be closed manually because the defer doesn't handle them as it is defined later (thus we return before it's defined), otherwise the client which made the request will hang until its timeout. But as far as I can tell it doesn't result in anything truly problematic.

Can not reproduce. Could you give me an exact case that occuring this issue?

command to run spoof-dpi
the website that raises this issue

The site is not important, because there is no site, because it happens when you try to http GET the proxy address itself (hence why invalid request).
I gave it in description above:

Run spoof-dpi --debug
curl -v http://<IP_ADDRESS>

I get that it's not a valid request to make in a first place (valid would be curl --proxy http://<IP_ADDRESS> example.com), but I think spoof-dpi should reject it and not serve it, as opposed to this:
https://github.com/user-attachments/assets/0a7eb9ce-54e9-489f-87f0-28cd3c6be425
Same thing will happen if you just open http://<IP_ADDRESS>:8080 in your browser. Of course, none of this is normal usage, but I think it shouldn't hammer the cpu for no reason and just reject such request.
Maybe it doesn't happen on Mac for some reason?

Yeah this makes sense. Thank you for reporting issue. I will fix this as soon as possible.

Resolved.

Thank you!
Also, I think it'll be good to note that, in README, this:

What SpoofDPI does to bypass this is to send the first 1 byte of a request to the server, and then send the rest

Is no longer true, and now client hello is fragmented in chunks of equal size, size of which is determined by new --window-size argument, and that users should experiment with its value, if default does not work for them.
For example, new 50 bytes chunks default doesn't bypass my ISP's DPI while old simpler split in 2 chunk, first 1 byte and the rest, works fine. The maximum size which still bypasses mine is as low as --window-size 2. Of course, that depends on DPI, maybe 50 works fine for someone.
What I propose is something like this in --help (and readme):
...
  -window-size int
    	chunk size, in number of bytes, for fragmented client hello,
        try lower values if default doesn't bypass the DPI (default 50)
...

Currerntly in progress


Node tests should test node bundle, not a separated bundle made just for testing

as dependencies resolution is handled differently.

Implemented in #244
Released in https://github.com/Palindrom/Palindrom/releases/tag/v6.2.0


Add separate resources files to make them accessible for tech writer

Changes were made directly on dev. Closing.


Software-writable field spanning multiple words should cause error message (accesswidth/regwidth problem)

Here is my example:
addrmap mymap_t {
  reg {
    accesswidth = 32;
    regwidth = 64;
    field {
      sw = rw;
      hw = r;
    } myfield[40];
  } myreg;
};

I expected it would fail to compile as the field is software writable and requires two accesses. As stated in the SystemRDL 2.0 spec. 10.6.1f:

Any field that is software-writable or clear on read shall not span multiple software accessible sub-words (e.g., a 64-bit register with a 32-bit access width may not have a writable field with bits in both the upper and lower half of the register).

Is this check missing in the compiler?

This seems an impractical requirement in the standard. The most common use case, where you would be in violation of the rule is having a 32-bit system bus and 64-bit counters timers. Writing the 64-bit register as a pair of HI/LO 32-bit registers is rather common, but it would also reduce practicality. In generated HDL code it would be nice to have 64 bit vectors without hand written glue logic, and in generated SW code it would be nice to be able to tell a 64-bit CPU, it can use 64-bit load/store operations.

Indeed this rule has been relaxed for exactly the reason jeras stated above.


Clearing input field after search

Describe the Bug 🐛
A clear and concise description of what the bug is
The input field clears when you search thus it is hard to know what was searched

Please add Hacktoberfest label to this issue.


USHIFT-1061: Move hack/config-gen into cmd/generate-config directory

Automatic configuration generation and verification is part of the development and CI procedures now.
Moving the sources and scripts out of hack directory.

/assign @eggfoobar

/retitle USHIFT-1061: Move hack/config-gen into cmd/generate-config directory

/label tide/merge-method-squash

/lgtm

/retest ci/prow/microshift-e2e-arm

/test ci/prow/microshift-e2e-arm

/retest


multiple line bracket pairs guides for jsdoc is not suitable

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.83.1
OS Version: win10

Maybe it's not a bug but a design flaw for bracket pairs guides?
But it is reasonable to treat the beginning “*” as a blank character in jsdoc for bracket pairs guides.
Which means bracket pairs guides' position should be right here.

This feature request is now a candidate for our backlog. The community has 60 days to upvote the issue. If it receives 20 upvotes we will move it to our backlog. If not, we will close it. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

@hediet Because it doesn't look like the vote on this issue is going to be successful, can I help solve this issue? This requires only a minor change, which is to ignore the characters marked as comments when drawing the bracket pairs guides.

This feature request has not yet received the 20 community upvotes it takes to make to our backlog. 10 days to go. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

:slightly_frowning_face: In the last 60 days, this feature request has received less than 20 community upvotes and we closed it. Still a big Thank You to you for taking the time to create this issue! To learn more about how we handle feature requests, please see our documentation.
Happy Coding!


better support for Win10 OpenSSH server

If msys2 bash is set as the DefaultShell for the Win10 OpenSSH server,
there is no way to pass -l to make it a login shell, so /etc/profile is
not sourced, PATH is not set, and the bashrc produces errors.
Detect this case in /etc/bash.bashrc, by checking for the SSH_CONNECTION
environment variable and PATH not having msys2 elements, and source
/etc/profile.

You need to update pkgver.

Updated pkgver and sha256sum for the file, thanks!


Restructured expand supernodes

PR for restructuring the expand supernodes so that the matrix will not re-render every time the user clicks on a chevron

Comment https://github.com/multinet-app/multimatrix/pull/159#discussion_r556205317
I had some problems with the matrix updating the nodes and links before I called expand and retract so I had to deep copy of the nodes and links to ensure they were not manipulated during the transitions and when the data was updated for the Vue component. @JackWilb @gotdairyya any suggestions for how to fix this without using deep copy? Thanks!

Comment https://github.com/multinet-app/multimatrix/pull/159#discussion_r556205317
I had some problems with the matrix updating the nodes and links before I called expand and retract so I had to deep copy of the nodes and links to ensure they were not manipulated during the transitions and when the data was updated for the Vue component. @JackWilb @gotdairyya any suggestions for how to fix this without using deep copy? Thanks!

Comment https://github.com/multinet-app/multimatrix/pull/159#discussion_r556204568
When constructing new neighbor nodes, the non-aggregated nodes neighbors do not get updated. I remove these nodes so the network reflects the aggregated nodes. However, there is a potential issue with this such if there is a node in the network that is disconnected from all the other nodes in the graph. @gotdairyya @JackWilb any thoughts about this? I can file an issue about this for the future since it does not currently relate to this PR

Comment https://github.com/multinet-app/multimatrix/pull/159#discussion_r556204568
When constructing new neighbor nodes, the non-aggregated nodes neighbors do not get updated. I remove these nodes so the network reflects the aggregated nodes. However, there is a potential issue with this such if there is a node in the network that is disconnected from all the other nodes in the graph. @gotdairyya @JackWilb any thoughts about this? I can file an issue about this for the future since it does not currently relate to this PR

@JackWilb, can you file issues to track these?

Opened #180 + #181

@JackWilb, can you file issues to track these?

Opened #180 + #181


finishTransaction should take purchase instead of transaction ID

The newly added finishTransaction takes "transaction ID" as its first argument (https://github.com/dooboolab/react-native-iap/blob/master/index.ts#L401), whereas in reality, the user should pass in "purchase token" in the case Android non-consumable (https://github.com/dooboolab/react-native-iap/blob/master/index.ts#L418). I'd recommend refactoring the method to take purchase object instead to make it truly platform-neutral.

Thanks! Sounds very reasonable. We'll refactor@


Remove outdated comment

save() now calls update() or create() behind the scenes, and hence should be the preferred method.

Retracting due to discussion in https://github.com/robmorgan/phinx/issues/851#issuecomment-223882883 as historic use of $this->table('new_table') instead of $this->createTable('new_table').


Write more tests for cassandra/workflows.go

What changed?
Adding unit tests for IsWorkflowExecutionExists and SelectTransferTasksOrderByTaskID

Why?
Improve coverage

Pull Request Test Coverage Report for Build 018e6216-56a8-45bc-bbcb-d8f2e8f82372
Details

0 of 0   changed or added relevant lines in 0 files are covered.
14 unchanged lines in 6 files lost coverage.
Overall coverage increased (+0.05%) to 65.409%

Files with Coverage Reduction
New Missed Lines
%

service/history/task/transfer_standby_task_executor.go
2
87.63%

common/task/weighted_round_robin_task_scheduler.go
2
89.05%

common/peerprovider/ringpopprovider/config.go
2
81.58%

service/history/task/transfer_active_task_executor.go
2
72.38%

common/util.go
2
91.78%

service/matching/taskListManager.go
4
79.7%

Totals

Change from base Build 018e61ef-a6f4-4969-adc1-af73494586b3:
0.05%

Covered Lines:
95614

Relevant Lines:
146178

💛  - Coveralls


[PUB] Refactor search views to allow multiple types

Remember to allow search on srn instead of external_id

Initial proposal
search-client-refactor.01.pdf

Done


closurecompiler: 20180716 -> 20180805

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/closure-compiler/versions.
meta.description for closurecompiler is: '"A tool for making JavaScript download and run faster"'.

Checks done (click to expand)

built on NixOS
/nix/store/8l8sqnc31ihrbwad8pcnc4mjwygp7ffh-closure-compiler-20180805/bin/closure-compiler passed the binary check.
1 of 1 passed binary check by having a zero exit code.
0 of 1 passed binary check by having the new version present in output.
found 20180805 with grep in /nix/store/8l8sqnc31ihrbwad8pcnc4mjwygp7ffh-closure-compiler-20180805
directory tree listing: https://gist.github.com/b8acd3f568254fb03b2c213bc9151b45
du listing: https://gist.github.com/28581eec4df978c897b9e36440c5a11f

Test this build with Cachix (click to expand)

One time setup in nixpkgs Git checkout:
cachix use r-ryantm
git remote add r-ryantm https://github.com/r-ryantm/nixpkgs.git

Test this build:
git fetch r-ryantm && git checkout d520aa638628cdc25a5fcfd54e8951c43b8d39c2 && nix-shell --pure -I nixpkgs=. -p closurecompiler

Success on x86_64-darwin (full log)
Attempted: closurecompiler
Partial log (click to expand)
configuring
no configure script, doing nothing
building
no Makefile, doing nothing
installing
post-installation fixup
strip is /nix/store/251fvx4mx9q4zarbca5cciasmn66p668-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/1j5fyc2sn67n5w40593zihdrbjljkz4x-closure-compiler-20180805/bin
patching script interpreter paths in /nix/store/1j5fyc2sn67n5w40593zihdrbjljkz4x-closure-compiler-20180805
/nix/store/1j5fyc2sn67n5w40593zihdrbjljkz4x-closure-compiler-20180805

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: closurecompiler
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

Success on x86_64-linux (full log)
Attempted: closurecompiler
Partial log (click to expand)
building
no Makefile, doing nothing
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/8l8sqnc31ihrbwad8pcnc4mjwygp7ffh-closure-compiler-20180805
strip is /nix/store/gpc2wld1s0c6qzx9326cwn1wcx29xzsj-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/8l8sqnc31ihrbwad8pcnc4mjwygp7ffh-closure-compiler-20180805/bin
patching script interpreter paths in /nix/store/8l8sqnc31ihrbwad8pcnc4mjwygp7ffh-closure-compiler-20180805
checking for references to /build in /nix/store/8l8sqnc31ihrbwad8pcnc4mjwygp7ffh-closure-compiler-20180805...
/nix/store/8l8sqnc31ihrbwad8pcnc4mjwygp7ffh-closure-compiler-20180805


Rejection route added incorrectly

Hi, I use dhcpcd with prefix delegation, which misconfigures the rejection route.
It adds two rejection routes. The first one was for the assigned /62, and the 2nd one was with exactly the same prefix as assigned to the interface but with a better metric, basically killing the network connectivity.
Logfile (prefix changed):
Jan 22 15:12:13 storage001 daemon.info dhcpcd[12149]: control command: /sbin/dhcpcd -h storage001 eth2
Jan 22 15:12:13 storage001 daemon.err dhcpcd[12149]: control_free: No such file or directory
Jan 22 15:12:16 storage001 daemon.info dhcpcd[12149]: eth2: carrier acquired
Jan 22 15:12:16 storage001 daemon.warn dhcpcd[12149]: eth2: config file changed, expiring leases
Jan 22 15:12:16 storage001 daemon.info dhcpcd[12149]: eth2: IAID 00:00:00:01
Jan 22 15:12:16 storage001 daemon.info dhcpcd[12149]: eth2: IA type 25 IAID 00:00:00:02
Jan 22 15:12:16 storage001 daemon.info dhcpcd[12149]: eth2: adding address fe80::3653:4897:7823:438a2
Jan 22 15:12:17 storage001 daemon.info dhcpcd[12149]: eth2: soliciting an IPv6 router
Jan 22 15:12:18 storage001 daemon.info dhcpcd[12149]: eth2: rebinding prior DHCPv6 lease
Jan 22 15:12:20 storage001 daemon.info dhcpcd[12149]: eth2: REPLY6 received from fe80::1231:2fff:fe32:4621
Jan 22 15:12:20 storage001 daemon.info dhcpcd[12149]: eth2: renew in 1800, rebind in 2880, expire in 7200 seconds
Jan 22 15:12:20 storage001 daemon.info dhcpcd[12149]: lo: adding reject route to 2001:db8:217f:7ff8::/62
Jan 22 15:12:20 storage001 daemon.info dhcpcd[12149]: eth2: delegated prefix 2001:9e8:217f:7ff8::/62
Jan 22 15:12:20 storage001 daemon.info dhcpcd[12149]: bond0: adding address 2001:db8:217f:7ff8:4e1e:1ca4:f56:492/64
Jan 22 15:12:20 storage001 daemon.info dhcpcd[12149]: bond0: adding route to 2001:db8:217f:7ff8::/64
Jan 22 15:12:22 storage001 daemon.info dhcpcd[12149]: eth2: Router Advertisement from fe80::1234:1df1:bb73:784e
Jan 22 15:12:22 storage001 daemon.info dhcpcd[12149]: eth2: Router Advertisement from fe80::1231:2fff:fe32:4621
Jan 22 15:12:22 storage001 daemon.info dhcpcd[12149]: eth2: adding address 2001:db8:217f:7f00:d614:80c1:1262:58a2/64
Jan 22 15:12:22 storage001 daemon.info dhcpcd[12149]: eth2: adding address fdac:e100:6c9f:2f39:5863:49ab:1234:1234/64
Jan 22 15:12:22 storage001 daemon.info dhcpcd[12149]: eth2: adding route to 2001:db8:217f:7f00::/64
Jan 22 15:12:22 storage001 daemon.info dhcpcd[12149]: eth2: adding route to fdac:e100:6c9f:2f39::/64
Jan 22 15:12:22 storage001 daemon.info dhcpcd[12149]: eth2: adding default route via fe80::1231:2fff:fe32:4621
Jan 22 15:12:25 storage001 daemon.warn dhcpcd[12149]: bond0: no IPv6 Routers available

Configuration:
duid 1E4918D2-83CC-42D5-9022-5735816A324C
persistent
vendorclassid
option domain_name_servers, domain_name, domain_search
option classless_static_routes
option interface_mtu
option rapid_commit
require dhcp_server_identifier
slaac private
noipv4
noarp
noipv4ll
ipv6rs
ipv6ra_fork
fqdn both
noalias
nohook 20-resolv.conf
interface bond0
  ipv6only
interface eth2
  waitip
  iaid 1
  ia_pd 2 bond0/0/64/0

ip -6 route:
2001:db8:217f:7f00::/64 dev eth2 proto ra metric 1004 mtu 1492 pref medium
2001:db8:217f:7ff8::/64 dev lo proto kernel metric 256 pref medium
2001:db8:217f:7ff8::/64 dev bond0 proto dhcp metric 1005 pref medium
unreachable 2001:db8:217f:7ff8::/62 dev lo proto dhcp metric 1001 pref medium
fdac:e100:6c9f:2f39::/64 dev eth2 proto ra metric 1004 mtu 1492 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
fe80::/64 dev eth1 proto kernel metric 256 pref medium
fe80::/64 dev bond0 proto kernel metric 256 pref medium
fe80::/64 dev eth2 proto kernel metric 256 pref medium
default via fe80::1231:2fff:fe32:4621 dev eth2 proto ra metric 1004 mtu 1492 pref medium

dhcpcd --version
dhcpcd 9.4.1
Copyright (c) 2006-2021 Roy Marples
Compiled in features: INET ARP ARPing IPv4LL INET6 DHCPv6 AUTH PRIVSEP

cat /etc/os-release
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.17.1
PRETTY_NAME="Alpine Linux v3.17"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://gitlab.alpinelinux.org/alpine/aports/-/issues"

Why is 2001:db8:217f:7ff8::/64 dev lo proto kernel metric 256 pref medium in the routing table?
When I delete this route and restart dhcpcd, it is re-added. However I don't see a line for it within the log (as seen above).

I don't think dhcpcd is adding it - the route has the kernel marking so the kernel added it.
If you remove the route and manually add an a different address but within the same prefix to bond0, does the kernel add the same route to lo again?

No, not to dev lo.
If I add another address from the same prefix to bond0 I get these routes:
2001:db8:217f:7ff8::/64 dev bond0 proto kernel metric 256 pref medium
2001:db8:217f:7ff8::/64 dev bond0 proto dhcp metric 1005 pref medium

So something that dhcpcd does triggers the kernel to add it to lo. Does dhcpcd first configure the delegated prefix on localhost before moving it to the correct interface? And thereby the kernel adds the route but doesn't remove it once the IP is removed from lo by dhcpcd again? (just speculating)

Wait a sec, are issues like this the reason why dhcpcd sets accept_ra=0? I'm asking because I've accept_ra=2 in sysctl.conf, as otherwise routes weren't configured properly (see #177)

This issue was caused by dhcpcd not implementing RFC4191 and me, therefore, having set accept_ra=2 in other places. But that, unfortunately, cause this interaction.
Going to close this ticket, as the request of implementing RFC4191 is within #177


Add docker stack, makefile & readme to setup the project

[x] Add docker-compose & docker file for build & run a symfony container.
[x] Add makefile and commands install and start
[x] Add a readme

Enhance Makefile


Breakage of connection harms others

Good connection helps

Do you have any more details on this?

A well connected link should receive and transmit valuable energy and displace waste particles to a sub connector that has a primary function of subbing energy


How can I run it when my cuda=10.2?

My cuda is 10.2,  so I can't install pytorch which is below 1.5.0. But it requires torch=0.4.1 and cuda=9.0 from readme.md. So is there any way to make me run this code?

You can try this code on COCO dataset https://github.com/fanq15/FewX.

@1786939789 Does it work with cuda 10.2?


NesPi 4 Case EEPROM Issue

Hello,
I dunno if it's right place to ask but I have updated to latest eeprom which dates to December 11th.. Now the HDD inside my NesPi 4 Case will no longer be detected.
I am running Batocera booting from SD card and ROMs on 2TB HDD. Batocera boots just fine but will no longer see my ROMs HDD. I checked in another OS I have on another SD card same issue drive no longer there. I have added the usb.quircks line to cmdline.txt to no avail. I am not booting from HDD. I just want HDD to be detected.
Tested the following :
Raspberry Pi OS
Twister OS
Batocera
Same result on all. Only EEPROM that works is Stable September 03 build anything beyond that renders my HDD useless...
Note : I updated controller firmware from your site. Not fixed

There are at least 4 people i know having issues with the JMicron controller (?) on nespi 4 case not booting  USB drives
as @TechXero some on older pi4 fw (sept 20 or older) can boot by disabling UASP by blacklisting it with quirks line mentioned above.

Yep here's what happens after firmware upgrade :
https://i.imgur.com/rYYmYD3.png


Fireox Sound echoes severely and freezes

Describe the bug
Sound echoes severely and freezes
To Reproduce
Steps to reproduce the behavior:

Go to 'Firefox
Click on 'YouTube
Scroll down to 'Play the video'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
no
https://github.com/helloSystem/ISO/assets/44593430/cdbbf1aa-b97c-445e-bd3b-4131874a11d2
Version (please complete the following information):

Live ISO or installed system to disk: [installed]
FreeBSD version (13.2]
helloSystem build (0I46 )
Running in a virtual machine: [no]

Computer (please complete the following information):

Device: [H55M-D2H]

Additional context
Add any other context about the problem here.

In other browsers such as Falkon Librium, there is no such problem.


Add Account and Corpus Management APIs to Playground

TY!  LGTM

Merging this now, as the blog is going out at 3a PT, and the links would be broken otherwise


next validator task

Hi, Ben! I want next validator task)

Hi Oleg! 😄
Do you want to take will-change? We need an isAnimateableFeature validator for this property. The syntax is quite simple - scroll-position | contents | <custom-ident> - check that it is either of those keywords or pass it through isCustomIdent.
More info on this property at https://developer.mozilla.org/en/docs/Web/CSS/will-change.

Implemented in https://github.com/ben-eb/css-values/pull/14. 😄


[release-1.2] Fix flaky mem hotplug

What this PR does
Manual backport of https://github.com/kubevirt/kubevirt/pull/12298#

Fixes #
Links to places where the discussion took place:
Special notes for your reviewer

Release note

NONE

/retest

/approve


[cxx-interop] Builtin functions should ignore return type nullability…

… to be compatible with C interop
In C interop mode, the return type of builtin functions like 'memcpy' drops any nullability specifiers from their return type, and preserves it on the declared return type. Thus, in C mode the imported return type of such functions is always optional. However, in C++ interop mode, the return type of builtin functions can preseve the nullability specifiers on their return type, and thus the imported return type of such functions can be non-optional, if the type is annotated with _Nonnull. The difference between these two modes can break cross-module Swift serialization, as Swift will no longer be able to resolve an x-ref such as 'memcpy' from a Swift module that uses C interop, within a Swift context that uses C++ interop. In order to avoid the x-ref resolution failure, normalize the return type's nullability for builtin functions in C++ interop mode, to match the imported type in C interop mode.
This fixed an issue when using 'memcpy' from the Android NDK in a x-module context, like between Foundation (that inlines it) and another user module.

@swift-ci please smoke test

@swift-ci please test

@swift-ci please test source compatibility

@swift-ci please test source compatibility

@swift-ci please test

@swift-ci please test source compatibility

@swift-ci please test

@swift-ci please test

@swift-ci please test source compatibility

@swift-ci please test windows platform

@swift-ci please test

@swift-ci please test

@swift-ci please test windows platform

@swift-ci please test windows platform

@swift-ci please test windows platform

@swift-ci please test


kubectl: rationalize which messages to stderr/stdout

A user reports to<EMAIL_ADDRESS>(https://groups.google.com/forum/#!topic/google-containers/qHDR-mvh7sM) the following.  I verified similar behavior in kubectl v0.9.1.
I am using Kubernetes v0.7.0, when I run "kubectl" to create pod, I found:
$ kubectl create -s http://<IP_ADDRESS>:8080 -f ./mariadb-pod.yaml
I0128 22:09:44.258267   30016 restclient.go:133] Waiting for completion of operation 19    --> this is to stderr
mariadb    --> this is to stdout
I do not know why the message "...Waiting for completion ..." is considered as an error, I think it should be to stdout too.

I've seen that log before, and actually, it should be log.V(4). Internal things like polling actions shouldn't appear in logs unless the command is run with --v=#

@smarterclayton @ghodss
I think another problem is actually that the kubectl outputs (at least the create.go one) is using fmt.Fprintf(...) which goes to stdout, while everything else is using glog... which goes to stderr.
What is the desired behavior?  I thought that logging to stderr, and user facing output to stdout was sort of the expected behavior.

Can sometimes be - in this case we're leveraging stdout for "things you might want to script" and stderr for "logs and pretty info".  So create returns only the name of the thing created, so you can (in theory)
kubectl get pods $(kubectl create -f something.json)

Create/Delete/Update return the names of the things - get does stdout.  I'm not sure these rules have to be followed (although get -q is something we need to add).
----- Original Message -----

@smarterclayton @ghodss
I think another problem is actually that the kubectl outputs (at least the
create.go one) is using fmt.Fprintf(...) which goes to stdout, while
everything else is using glog... which goes to stderr.
What is the desired behavior?  I thought that logging to stderr, and user
facing output to stdout was sort of the expected behavior.

Reply to this email directly or view it on GitHub:
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-71929420

I do want to preserve and improve scriptability. It's a line item in cli-roadmap.md.
All user-friendly messages should go to stderr.
I'd like full objects and other machine-readable info to go to stdout.
We should provided an option to simply output names, partial URLs (for objects of multiple kinds, namespaces, etc.), whatever that can be provided back to kubectl as command-line arguments.
I could also imagine a completely silent mode that just produces a useful exit code.

We also need -q or equiv for get so you can easily pipe to xargs.  That's on my list, as well as delete all.

On Jan 28, 2015, at 8:36 PM, Brian Grant<EMAIL_ADDRESS>wrote:
I do want to preserve and improve scriptability. It's a line item in cli-roadmap.md.
All user-friendly messages should go to stderr.
I'd like full objects and other machine-readable info to go to stdout.
We should provided an option to simply output names, partial URLs (for objects of multiple kinds, namespaces, etc.), whatever that can be provided back to kubectl as command-line arguments.
I could also imagine a completely silent mode that just produces a useful exit code.
—
Reply to this email directly or view it on GitHub.

My opinion: stderr is for errors, full stop.
kubectl help get -> stdout, user asked for it
kubectl gett -> stderr, user made a mistake
On Jan 28, 2015 5:45 PM, "Clayton Coleman"<EMAIL_ADDRESS>wrote:

We also need -q or equiv for get so you can easily pipe to xargs. That's
on my list, as well as delete all.

On Jan 28, 2015, at 8:36 PM, Brian Grant<EMAIL_ADDRESS>wrote:
I do want to preserve and improve scriptability. It's a line item in
cli-roadmap.md.
All user-friendly messages should go to stderr.
I'd like full objects and other machine-readable info to go to stdout.
We should provided an option to simply output names, partial URLs (for
objects of multiple kinds, namespaces, etc.), whatever that can be provided
back to kubectl as command-line arguments.
I could also imagine a completely silent mode that just produces a
useful exit code.
--
Reply to this email directly or view it on GitHub.

--
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-71954234
.

Time to research!
http://www.jstorimer.com/blogs/workingwithcode/7766119-when-to-use-stderr-instead-of-stdout
http://en.wikipedia.org/wiki/Standard_streams
One thing that stands out - stdout is what you pipe to the next program, full stop.  If you'd grep it, it would be on stdout.  If you'd pipe it to xargs, it's stdout.  If you need to see it, but don't care much past that... it's probably stderr.
The section on help docs in the first link matches my experience - usage "errors" are always stderr.  Help output is probably more accurately stdout (I asked you for help).

Also, if ordering matters, you probably want to keep it all on one stream.  I.e. grep errors can be out of band (and potentially be flushed later than message), but trying to deliver things out of order.

Printing anything the user asked for is stdout.  Printing the json of an
object can pipe to jq.  Printing lines of a table can pipe to awk.  Etc.
On Jan 28, 2015 7:13 PM, "Clayton Coleman"<EMAIL_ADDRESS>wrote:

Also, if ordering matters, you probably want to keep it all on one stream.
I.e. grep errors can be out of band (and potentially be flushed later than
message), but trying to deliver things out of order.
--
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-71962115
.

Yup, agree.

On Jan 28, 2015, at 10:35 PM, Tim Hockin<EMAIL_ADDRESS>wrote:
Printing anything the user asked for is stdout. Printing the json of an
object can pipe to jq. Printing lines of a table can pipe to awk. Etc.
On Jan 28, 2015 7:13 PM, "Clayton Coleman"<EMAIL_ADDRESS>wrote:

Also, if ordering matters, you probably want to keep it all on one stream.
I.e. grep errors can be out of band (and potentially be flushed later than
message), but trying to deliver things out of order.
--
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-71962115
.

—
Reply to this email directly or view it on GitHub.

Help output to stdout -- sure, fine.

Table headers should be to stderr.
----- Original Message -----

Help output to stdout -- sure, fine.

Reply to this email directly or view it on GitHub:
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-71973809

no.  They are not errors.
On Thu, Jan 29, 2015 at 7:37 AM, Clayton Coleman<EMAIL_ADDRESS>wrote:

Table headers should be to stderr.
----- Original Message -----

Help output to stdout -- sure, fine.

Reply to this email directly or view it on GitHub:

https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-71973809
--
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-72044965
.

I thought we just had the thing where the Unix way is to separate things that are output and part of the shell composition pattern from the things that are output and are not?  How does osc get | grep Run -> returns header help me?
----- Original Message -----

no.  They are not errors.
On Thu, Jan 29, 2015 at 7:37 AM, Clayton Coleman<EMAIL_ADDRESS>wrote:

Table headers should be to stderr.
----- Original Message -----

Help output to stdout -- sure, fine.

Reply to this email directly or view it on GitHub:

https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-71973809
--
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-72044965
.

Reply to this email directly or view it on GitHub:
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-72058513

Table headers are part of the output you asked for.
ps, df, netstate, etc all print table headers to stdout.  I have never
heard of any program printing headers to stderr and rows to stdout.  I
would find that INCREDIBLY annoying.
On Thu, Jan 29, 2015 at 9:45 AM, Clayton Coleman<EMAIL_ADDRESS>wrote:

I thought we just had the thing where the Unix way is to separate things
that are output and part of the shell composition pattern from the things
that are output and are not? How does osc get | grep Run -> returns header
help me?
----- Original Message -----

no. They are not errors.
On Thu, Jan 29, 2015 at 7:37 AM, Clayton Coleman <
<EMAIL_ADDRESS>wrote:

Table headers should be to stderr.
----- Original Message -----

Help output to stdout -- sure, fine.

Reply to this email directly or view it on GitHub:

https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-71973809

--
Reply to this email directly or view it on GitHub
<
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-72044965

.

Reply to this email directly or view it on GitHub:

https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-72058513
--
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-72070033
.

Because you want to grep / xargs / table headers amrite?  I'm not going to go to the wall for this one, but do you normally redirect things to files more often than you pipe them?  Since we can turn off table headers via flag I don't really care that much.

On Jan 29, 2015, at 1:08 PM, Tim Hockin<EMAIL_ADDRESS>wrote:
Table headers are part of the output you asked for.
ps, df, netstate, etc all print table headers to stdout. I have never
heard of any program printing headers to stderr and rows to stdout. I
would find that INCREDIBLY annoying.
On Thu, Jan 29, 2015 at 9:45 AM, Clayton Coleman<EMAIL_ADDRESS>wrote:

I thought we just had the thing where the Unix way is to separate things
that are output and part of the shell composition pattern from the things
that are output and are not? How does osc get | grep Run -> returns header
help me?
----- Original Message -----

no. They are not errors.
On Thu, Jan 29, 2015 at 7:37 AM, Clayton Coleman <
<EMAIL_ADDRESS>wrote:

Table headers should be to stderr.
----- Original Message -----

Help output to stdout -- sure, fine.

Reply to this email directly or view it on GitHub:

https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-71973809

--
Reply to this email directly or view it on GitHub
<
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-72044965

.

Reply to this email directly or view it on GitHub:

https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-72058513
--
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-72070033
.

—
Reply to this email directly or view it on GitHub.

Honestly, given a table with enough columns I actually DO want the header.
Likewise, if I am importing to CSV, etc.
I will go to the wall - dumping things that are not errors to stderr is
just wrong. :)
On Thu, Jan 29, 2015 at 10:41 AM, Clayton Coleman<EMAIL_ADDRESS>wrote:

Because you want to grep / xargs / table headers amrite? I'm not going to
go to the wall for this one, but do you normally redirect things to files
more often than you pipe them? Since we can turn off table headers via flag
I don't really care that much.

On Jan 29, 2015, at 1:08 PM, Tim Hockin<EMAIL_ADDRESS>wrote:
Table headers are part of the output you asked for.
ps, df, netstate, etc all print table headers to stdout. I have never
heard of any program printing headers to stderr and rows to stdout. I
would find that INCREDIBLY annoying.
On Thu, Jan 29, 2015 at 9:45 AM, Clayton Coleman <
<EMAIL_ADDRESS>wrote:

I thought we just had the thing where the Unix way is to separate
things
that are output and part of the shell composition pattern from the
things
that are output and are not? How does osc get | grep Run -> returns
header
help me?
----- Original Message -----

no. They are not errors.
On Thu, Jan 29, 2015 at 7:37 AM, Clayton Coleman <
<EMAIL_ADDRESS>wrote:

Table headers should be to stderr.
----- Original Message -----

Help output to stdout -- sure, fine.

Reply to this email directly or view it on GitHub:

https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-71973809

--
Reply to this email directly or view it on GitHub
<

https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-72044965

.

Reply to this email directly or view it on GitHub:

https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-72058513

--
Reply to this email directly or view it on GitHub
<
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-72070033>

.

--
Reply to this email directly or view it on GitHub.

--
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-72079504
.

Turning off headers is the solution for people who don't want them is fine. Or tail +2.

Turning off headers via flag is the usual solution. I find the thought of
it going to stderr really odd, and agree with Tim's general principal of
"you asked for it" (this especially applies to --help ... I find it so
aggravating when --help gets dumped to stderr just because someone was lazy
and didn't change their usage() routine to do the right thing).
On Thu, Jan 29, 2015 at 10:55 PM, Brian Grant<EMAIL_ADDRESS>wrote:

Turning off headers is the solution for people who don't want them is
fine. Or tail +2.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-72161669
.

Yeah, since we have --no-headers we're good.  Headers was maybe a bad example.
----- Original Message -----

Turning off headers via flag is the usual solution. I find the thought of
it going to stderr really odd, and agree with Tim's general principal of
"you asked for it" (this especially applies to --help ... I find it so
aggravating when --help gets dumped to stderr just because someone was lazy
and didn't change their usage() routine to do the right thing).
On Thu, Jan 29, 2015 at 10:55 PM, Brian Grant<EMAIL_ADDRESS>wrote:

Turning off headers is the solution for people who don't want them is
fine. Or tail +2.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-72161669
.

Reply to this email directly or view it on GitHub:
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-72231811

So are we in fact saying that glog going to stderr is fine, and therefore this issue can be closed? Or should we parse out some glog messages (like info) to go to stdout? Just not clear on what actually needs to be changed here given the above thread.

The glog prefixes on CLIs are terrible.  Unless you're at --v=3 or higher writing the standard glog gorp to the screen is abusive to new users.

On Feb 11, 2015, at 7:22 PM, Sam Ghods<EMAIL_ADDRESS>wrote:
So are we in fact saying that glog going to stderr is fine, and therefore this issue can be closed? Or should we parse out some glog messages (like info) to go to stdout? Just not clear on what actually needs to be changed here given the above thread.
—
Reply to this email directly or view it on GitHub.

This - all glog stuff < ERROR should be silent unless -v is set, though we
can debate the -v level impact
On Wed, Feb 11, 2015 at 6:02 PM, Clayton Coleman<EMAIL_ADDRESS>wrote:

The glog prefixes on CLIs are terrible. Unless you're at --v=3 or higher
writing the standard glog gorp to the screen is abusive to new users.

On Feb 11, 2015, at 7:22 PM, Sam Ghods<EMAIL_ADDRESS>wrote:
So are we in fact saying that glog going to stderr is fine, and
therefore this issue can be closed? Or should we parse out some glog
messages (like info) to go to stdout? Just not clear on what actually needs
to be changed here given the above thread.
--
Reply to this email directly or view it on GitHub.

--
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-74004518
.

Yeah, i just picked 3 randomly cause we said 2 is "normal" operation
----- Original Message -----

This - all glog stuff < ERROR should be silent unless -v is set, though we
can debate the -v level impact
On Wed, Feb 11, 2015 at 6:02 PM, Clayton Coleman<EMAIL_ADDRESS>wrote:

The glog prefixes on CLIs are terrible. Unless you're at --v=3 or higher
writing the standard glog gorp to the screen is abusive to new users.

On Feb 11, 2015, at 7:22 PM, Sam Ghods<EMAIL_ADDRESS>wrote:
So are we in fact saying that glog going to stderr is fine, and
therefore this issue can be closed? Or should we parse out some glog
messages (like info) to go to stdout? Just not clear on what actually needs
to be changed here given the above thread.
--
Reply to this email directly or view it on GitHub.

--
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-74004518
.

Reply to this email directly or view it on GitHub:
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-74299925

Seems like the specific info message mentioned at the top of this ticket was removed in #3921. Combined with the fact that we don't plan on changing the current stdout/stderr split seems means we can close this issue. If we want to change glog prefixes we can probably make a new issue for that.

Since the purpose of this issue was to decide what messages go to stdout/stderr, I think we can close this out and open separate bugs for specific output that isn't doing the right thing. Summary of decisions:

Only errors, (log.fatal, log.error) go to stderr
Anything the user requested goes to stdout. This includes table headers (they can be turned off via --no-headers), periodic updates ('waiting for blah...'), and crucially, --help output.
All glog output < ERROR should be silent unless -v is set.

glog prefixing
----- Original Message -----

Since the purpose of this issue was to decide what messages go to
stdout/stderr, I think we can close this out and open separate bugs for
specific output that isn't doing the right thing. Summary of decisions:

Only errors, (log.fatal, log.error) go to stderr
Anything the user requested goes to stdout. This includes table headers
(they can be turned off via --no-headers), periodic updates ('waiting for
blah...'), and crucially, --help output.
All glog output < ERROR should be silent unless -v is set.

Reply to this email directly or view it on GitHub:
https://github.com/GoogleCloudPlatform/kubernetes/issues/3894#issuecomment-77238672


Added package nim_telegram_bot

Builds on top of Telebot, usable just editing config.ini, but can also use plugins just putting it on the ./plugins/ folders.

This is awesome :O

Sadly I can't install it:
» nimble install https://github.com/juancarlospaco/nim-telegram-bot
Downloading https://github.com/juancarlospaco/nim-telegram-bot using git
       Tip: 6 messages have been suppressed, use --verbose to show them.
     Error: Could not read package info file in /tmp/nimble_43367/githubcom_juancarlospaconimtelegrambot/nim_telegram_bot.nimble;
        ...   Reading as ini file failed with:
        ...     Invalid section: .
        ...   Evaluating as NimScript file failed with:
        ...     private/tmp/nimble_43367/githubcom_juancarlospaconimtelegrambot/nim_telegram_bot.nimble(13, 27) Error: undeclared identifier: 'telebot'.

Fixed :+1:
$ nimble install https://github.com/juancarlospaco/nim-telegram-bot

Downloading https://github.com/juancarlospaco/nim-telegram-bot using git
  Verifying dependencies for<EMAIL_ADDRESS>      Info: Dependency on openexchangerates@any version already satisfied
  Verifying dependencies for<EMAIL_ADDRESS>      Info: Dependency on telebot@any version already satisfied
  Verifying dependencies for<EMAIL_ADDRESS> Installing<EMAIL_ADDRESS>   Success: nim_telegram_bot installed successfully.

$

Works now, but shouldn't you specify a bin in your .nimble file so that it gets compiled?

Fixed :+1:
$ nimble install https://github.com/juancarlospaco/nim-telegram-bot

Downloading https://github.com/juancarlospaco/nim-telegram-bot using git
  Verifying dependencies for<EMAIL_ADDRESS>      Info: Dependency on openexchangerates@any version already satisfied
  Verifying dependencies for<EMAIL_ADDRESS>      Info: Dependency on telebot@any version already satisfied
  Verifying dependencies for<EMAIL_ADDRESS> Installing<EMAIL_ADDRESS>   Building nim_telegram_bot/nim_telegram_bot using c backend
   Success: nim_telegram_bot installed successfully.

$

Thanks!,... :cat:


Changing links in the old interactive tutorials.

Fixes Step 1 for #20316
Reason to have it - the localization team doesn't check if the links are really working (spoiler they don't). So this Error travels to localizations too.

Karen, can you take a look? Thank you.
/assign @kbarnard10


Logo page & Brand Guidelines

There's a page on the old site that shows the logo and instructions to drag to desktop see here https://www.doi.org/logo.html
Do we need something like this on the new site ?  Note that if I copy the logo image on new website it is the full DOI Foundation one

working on it - have a page https://www.doi.org/the-identifier/resources/logos but need to actually add the logos. To come.


linkageGroupIdList in Genome Map Data

From @GuilhemSempere on July 28, 2016 13:15
The "Genome Map Data" call is illustrated as
/maps/mapDbId/positions?linkageGroupIdList=linkageGroupIdlinkageGroupId&pageSize=pageSize&pageNumber=pageNumber
I have several comments about it:

is there a reason for passing linkageGroupIdList as a CSV list whereas in other calls we use the more standard way of repeating the parameter?
in case we want to keep it as such, a coma is missing in the example ;-)
whatever method we use, the list is likely to be big (for GBS data for example), so I think there has to be a way to POST to this URL

Copied from original issue: plantbreeding/Documentation#8

From @sebastian-raubach on July 28, 2016 13:47
My understanding of a linkage group was that it's pretty much (but not quite) the same as a chromosome. Are you expecting thousands of those?

From @sebastian-raubach on July 28, 2016 13:18
I agree that it shouldn't be a comma-separated list but rather a repetition of the parameter.
However, @GuilhemSempere what number of linkage groups do you expect to have? In our case we'll have at most 10.

From @GuilhemSempere on July 28, 2016 13:45
I've seen GBS data with thousands of scaffolds

From @rfinkers on July 28, 2016 13:49
Depends on the context. A linkage group (genetic markers) will usually come to the expected number of chromosomes. A genome assembly (not ordered to pseudo molecules) can have many thousands of scaffolds.

From @sebastian-raubach on July 28, 2016 13:52
I really don't mind it being a POST. Just trying to figure out what the requirements are.

From @GuilhemSempere on July 28, 2016 13:57
I'll let you guys decide, I just wanted to point that out. I think the question is narrowed down to "is a scaffold a linkage group?".

From @GuilhemSempere on July 28, 2016 14:1
Just as mine does

From @sebastian-raubach on July 28, 2016 14:0
@GuilhemSempere Exactly. And that's pretty much where my biological knowledge ends...

From @GuilhemSempere on July 28, 2016 17:45
Having discussed with @rfinkers it seems that scaffolds are not "exactly" linkage groups, but that chromosomes aren't either. So I guess if the "language abuse" is acceptable for chromosome, it should also be for scaffold.
Something else though, this call supports a parameter named "pageNumber" where others expect it to be simply "page".

From @rfinkers on July 28, 2016 15:28
@sebastian-raubach @GuilhemSempere No, they are two different things!
Genetic study: Linkage map
Genomic study: Scaffolds
but both are key:value pairs; although the value's means different things (bp (genome) is different then cM (genetic). Let's sit down if this needs further clarification.

From @jeback1 on July 29, 2016 15:5
I suggest that have a parameter that would cover chromosomes, linkage groups, and scaffolds would be very useful.  Linkage group is perhaps the closest existing term but we could just use markerGroup or even featureGroup.  I also think we should convert it to POST.

From @sebastian-raubach on July 28, 2016 17:55
@GuilhemSempere I've updated the Apiary to reflect the "pageNumber" and the case of multiple separate linkageGroupIds. It still needs to be discussed if we should change the whole call to POST or add a separate POST.

From @lukasmueller on September 7, 2016 20:16
Hmmm... I thought we used linkage group so that we didn't have to worry about chromosomes or scaffolds...

From @dbolser-ebi on September 2, 2016 10:45

Having discussed with @rfinkers it seems that scaffolds are not "exactly" linkage groups, but that chromosomes aren't either. So I guess if the "language abuse" is acceptable for chromosome, it should also be for scaffold.

The point is that when people say 'linkage group' they are talking about chromosome-scale things, not scaffold-scale things (unless there are a few really big scaffolds ;-)
Not sure of the relevance of the discussion here, but just thought I'd weigh in :-)

From @GuilhemSempere on February 8, 2018 9:35
Current version doesn't allow multiple linkage group parameters anymore  :-(

V2.0 now has a /search/markerpositions call. This allows you to search by any number of Map Ids, Linkage Group Names, and Marker Ids, and returns objects which link Marker Ids to Map positions.
The "language abuse" is still present. BrAPI still vaguely defines Linkage Group as "generic term for a named section of a map"


docs(LAB_01_RBAC.md): Obtain .id parameter

Module: 01
Lab: 01
Fixes: The previous command tried to access the .objectId parameter which does not exist in a user object.
Changes proposed in this pull request:

The correct parameter is accessed with the .id parameter.

Resolves #263


Explain how to add HTTPS to XMLRPC

Author Name: OpenNebula Systems Support Team (OpenNebula Systems Support Team)
Original Redmine Issue: 5257, https://dev.opennebula.org/issues/5257
Original Date: 2017-07-20

To harden OpenNebula based clouds. Using an SSL proxy like nginx

Original Redmine Comment
Author Name: OpenNebula Systems Support Team (OpenNebula Systems Support Team)
Original Date: 2017-07-20T09:44:52Z

Notes for the docs:
$ cat /etc/nginx/conf.d/opennebula-ssl.conf
server {
    listen 2634 ssl;
    ssl_certificate           /etc/nginx/nginx.crt;
    ssl_certificate_key       /etc/nginx/nginx.key;
    location / {
        proxy_pass          http://localhost:2633;
    }
}

export ONE_XMLRPC=https://localhost:2634/RPC2
or one_endpoint

export ONE_DISABLE_SSL_VERIFY=true

Original Redmine Comment
Author Name: Strahinja Kustudic (Strahinja Kustudic)
Original Date: 2017-08-14T09:02:48Z

I'm willing to write this documentation, but I'm not sure where exactly to add it. I was thinking to add it to frontend_installation.rst similar like MySQL is added as option and than make a separate page about it, but I'm not sure if that is the best way to do it?

Original Redmine Comment
Author Name: Kristian Feldsam (Kristian Feldsam)
Original Date: 2017-09-11T18:24:43Z

Hii all, I like to share my config. I also use limit_req_zone to limit max connections to api
1. main server config /etc/nginx/nginx.conf
1. For more information on configuration, see:
1.   * Official English Documentation: http://nginx.org/en/docs/
1.   * Official Russian Documentation: http://nginx.org/ru/docs/

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;

events {
    worker_connections 1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;
    server_names_hash_bucket_size 64;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    client_max_body_size 20M;

    limit_req_zone $binary_remote_addr zone=perip:10m rate=24r/s; # one tenth of MAX_CONN from oned.conf
    limit_req_zone $server_name zone=perserver:10m rate=240r/s; # MAX_CONN from oned.conf

    # Load modular configuration files from the /etc/nginx/conf.d directory.
    # See http://nginx.org/en/docs/ngx_core_module.html#include
    # for more information.
    include /etc/nginx/conf.d/*.conf;
}

1. vhost config in /etc/nginx/conf.d/
server {
    listen 443;
    server_name  cloud-api.domain.tld;

    ssl on;
     ssl_certificate /etc/nginx/nginx.crt;
     ssl_certificate_key /etc/nginx/nginx.key;
     ssl_session_timeout 1d;
     ssl_session_cache shared:SSL:50m;

    # Diffie-Hellman parameter for DHE ciphersuites, recommended 2048 bits
     ssl_dhparam /etc/nginx/dhparams.pem;

    # intermediate configuration. tweak to your needs.
     ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
     ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
     ssl_prefer_server_ciphers on;

    # HSTS (ngx_http_headers_module is required) (15768000 seconds = 6 months)
     add_header Strict-Transport-Security max-age=15768000;

    # Load configuration files for the default server block.
    include /etc/nginx/default.d/*.conf;

    location / {
        limit_req zone=perip burst=48; # one tenth of MAX_CONN_BACKLOG from oned.conf
        limit_req zone=perserver burst=480;  #  MAX_CONN_BACKLOG from oned.conf

        proxy_set_header   Host             $host;
        proxy_set_header   X-Real-IP        $remote_addr;
        proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
        proxy_pass http://<IP_ADDRESS>:2633;
    }
}

Original Redmine Comment
Author Name: kvaps kvaps (kvaps kvaps)
Original Date: 2017-09-21T13:48:51Z

This is how to connect to opennebula remotely:
sudo gem install --no-user-install  opennebula-cli
echo "export ONE_XMLRPC=https://opennebula-backend.tld/RPC2" >> .bashrc
echo "export ONE_DISABLE_SSL_VERIFY=true" >> .bashrc

After re login to bash:
mkdir .one
touch .one/one_auth
oneuser login myuser


Add admin-ui to the staging environment

In order to have a staging environment that fully matches production as an SRE I want the admin-ui release to be deployed into the staging environment so that I can be sure that staging accurately reflects production.
Acceptance Criteria

[x] cg-deploy-admin-ui is updated to deploy into the staging environment as well as production

Ran into the stemcell issue 18F/cg-deploy-admin-ui#8

Still needs someone to actually log in and check it.

checked! works


[htaccess] certain routes not working

This is a follow-up to a question I asked on StackOverflow.
I have three files under the api directory
api/.htaccess
<IfModule mod_rewrite.c>
    <IfModule mod_negotiation.c>
        Options -MultiViews -Indexes
    </IfModule>

    RewriteEngine On
    RewriteBase /api/

    # Handle Authorization Header
    RewriteCond %{HTTP:Authorization} .
    RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]

    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_URI} (.+)/$
    RewriteRule ^ %1 [L,R=301]

    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteRule ^ index.php [QSA,L]
</IfModule>

api/index.php
<?php declare(strict_types=1);

use Dotenv\Dotenv;
use League\Route\Router;
use Laminas\Diactoros\ServerRequestFactory;
use Laminas\HttpHandlerRunner\Emitter\SapiEmitter;

define('BASE_DIR', dirname(__DIR__));

require BASE_DIR . '/vendor/autoload.php';

$dotenv = Dotenv::createImmutable(BASE_DIR);
$dotenv->load();

/** @var Router $router */
$router = require __DIR__ . '/routes/api.php';

$request = ServerRequestFactory::fromGlobals(
    $_SERVER,
    $_GET,
    $_POST,
    $_COOKIE,
    $_FILES
);
$response = $router->dispatch($request);

// send the response to the browser
(new SapiEmitter)->emit($response);

api/routes/api.php
<?php declare(strict_types=1);

use League\Route\Router;
use League\Route\RouteGroup;
use League\Route\Strategy\JsonStrategy;
use Laminas\Diactoros\ResponseFactory;
use App\Http\Controllers\DemoController;

$response_factory = new ResponseFactory();
$strategy = new JsonStrategy($response_factory);
/** @var Router $router */
$router = (new Router)->setStrategy($strategy);

$router->group('/api/', function (RouteGroup $route) {
    $route->get('/', [DemoController::class, 'index']);
    $route->get('/profile', [DemoController::class, 'profile']);
});

return $router;

The /api endpoint which points to the index method on the DemoController works but the /api/profile does not work.
What could be causing this?

I'm a bit rusty on htaccess rewrites but what is odd is that one route works and one doesn't.
would it be possible for you to dump out the request path before dispatch for me? $request->getUri()->getPath();
would be helpful to see what it is seeing on each route

Closing as stale


Differential Evolution Hyperband (DEHB)

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

I am all for a general benchmarking script, but it may be difficult, because it would come with many CL args.

OK. The definitions are somewhat different. What I can do is to extend MethodArguments in benchmark_automl, and then use the extended one in benchmark_dehb.

But I will not import Methods from benchmark_automl, because it is too different. For example, BOHB means async BOHB in automl, but sync BOHB in dehb.

OK: #321


API Tokens and docs/links to pull download stats

New homepage needs download data.
Lists JFrog, Conan, and vcpkg.
We can definitely track these individually, but personally I think showing one big total of downloads is more impressive.
Either way we need:

[ ] API / docs on how to retrieve it
[ ] API token / access information for it

There's no real way to know the actual number of downloads can we just put a ### Placeholder for now?


Potter/astradb updates

Keeping this as DRAFT for now.

@potter-potter starting work on this now! thank you!


Can you provide some help for putting a chart in adminlte-laravel.

https://almsaeedstudio.com/themes/AdminLTE/pages/charts/chartjs.html
I know adminlte-laravel only shows a minimum page...can you please detail the steps to add a chart like the one above to this project..thanks for any help

You can compare adminlte demo pages with current provided files and customize to your needs adding css/js and html code. So compare adminlte demo files with acacha adminlte files to find the differences. Maybe in the future this package will provide adminlte demo pages too.


The desktop application exhibits abnormal behavior

Describe the bug
The desktop application exhibits abnormal behavior and can no longer be used normally after experiencing issues. Initially, executing commands (such as select) results in very slow queries, often terminating due to exceeding 10 seconds. Reconnecting to the database within the application allows for successful queries after the first incident. However, during continued use, query timeouts occurred again. Following the previous steps, the application failed to reconnect to the database, showing a perpetual "connecting" status. Restarting both the application and the database temporarily resolved the issue until it became impossible to start the application normally, displaying the same error despite attempts to uninstall and reinstall.
To Reproduce
Steps to reproduce the behavior:

Execute database queries which then timeout after exceeding 10 seconds.
Disconnect and reconnect to the database within the application after the first query timeout.
On subsequent query timeouts, attempt to reconnect, leading to a failure to connect to the database.
Restart the application and the database to temporarily resolve the issue.
After multiple occurrences, the application fails to start normally, showing an error screen every time it's opened.
Deleting the surrealist.json file from the AppData\Roaming directory allows the application to start normally again, but all configuration data is lost and needs to be set up again.

Expected behavior
Queries should execute within a reasonable time frame without causing the application to crash or fail to reconnect to the database. The application should start normally without requiring the deletion of configuration files.
Environment:

Operating System: Windows 11 23H2
Surrealist Version: 1.11.4
SurrealDB Version: 1.2.0

Additional Information
I have also provided the configuration file (surrealist.json) found in the AppData\Roaming directory, which could help in reproducing the error interface and understanding the issue in detail.

By analyzing your config it appears you're trying to execute the following query
use ns test;
use db test;
select * from person
CREATE person CONTENT {    name: 'Tobie',    company: 'SurrealDB',   skills: ['Rust', 'Go', 'JavaScript']};

However a semicolon is missing after the select statement, causing parsing confusion in SurrealDB itself.
Here's a fixed version of the config which should hopefully allow Surrealist to start up again (surrealist.json)

By analyzing your config it appears you're trying to execute the following query通过分析您的配置，似乎您正在尝试执行以下查询
use ns test;
use db test;
select * from person
CREATE person CONTENT {    name: 'Tobie',    company: 'SurrealDB',   skills: ['Rust', 'Go', 'JavaScript']};

However a semicolon is missing after the select statement, causing parsing confusion in SurrealDB itself.然而，在选择语句之后缺少一个分号，导致SurrealDB本身的解析混乱。
Here's a fixed version of the config which should hopefully allow Surrealist to start up again (surrealist.json)这是一个修复后的配置版本，希望能让Surrealist再次启动起来（surrealist.json）。

It's amazing that it was such a basic error. The reason for doing this was to debug the statement below, which would undergo constant modifications. For the sake of convenience in comparing data within the database, I placed the select all statement before executing the debug statement.
What's speechless is that it worked normally several times after doing so, and I couldn't anticipate that such an occasional small habit could lead to a basic error.
Thank you for your time and handling of the configuration file; it's working properly now. 😄❤


Add Check for plugin updates option to the settings

Fixes https://github.com/sourcegraph/jetbrains/issues/68.
I checked it against IntelliJ 2022.1 - it works 🎉
I did runPluginVerifier - it works too 🟢
Demo
https://github.com/sourcegraph/jetbrains/assets/19799111/e99cc100-64b4-4fb9-81d9-36f3cb64a13e
Test plan

runIde with these changes
play with Nightly/Stable switch
(in the case when no updates appear you can try to modify pluginVersion in gradle.properties and re-run)

I dont know if it is important issue but i noticed that when there is new version of nightly and stable release im getting new notification whenever i switch between updating nightly and stable. Should it work this way? Maybe we can remove the previous popup 🤔
https://github.com/sourcegraph/jetbrains/assets/9321940/6b5fbaa8-c871-4171-81e5-acb83b997854

I dont know if it is important issue but i noticed that when there is new version of nightly and stable release im getting new notification whenever i switch between updating nightly and stable. Should it work this way? Maybe we can remove the previous popup 🤔
Screen.Recording.2023-11-07.at.13.04.50.mov

I'd say it's expected. It is possible that your currently installed version is older than available stable and nightly - in both cases the notification seems to be justified.


ImportError if entry_point is different from repo name

When adding a module to pythonista.cloud with the API, if the Github Repo is named differently to the entry_point then an Import Error is displayed. This is because pythonista.cloud is looking for the py file with the name of the Repo, not the entry_point
For Example:
from cloud import testPyModule where testPyModule is the Repo name and the entry point is
thisIsATestModule.py, the import fails.
http://db.pythonista.cloud/testPyModule
{
    _id: "testPyModule",
    _rev: "1-abced0386af3f17e7a2a6f25edba12f7",
    name: "testPyModule",
    url: "https://github.com/TheJJ100100/testPyModule",
    entry_point: "thisIsATestModule.py",
    py_versions: [
        2,
        3
    ]
}

Error Message:

Add [bug] tag?

Thanks. This is because while the module is installed correctly, the importme method looks for a module named name. This shouldn't be too hard a fix.

This should be solved by require all new submissions to pass a  name parameter as well.
Differentiating between the repo name and the module name would also allow submitting multiple modules that are in the same repo. (A lot of people have a pythonista-scripts repo where they collect small modules they wrote that aren't quite large enough to have their own repository).
It should also be considered to create some naming rules, as Python modules can't contain certain characters (hyphens, etc).

Going forward, I see a few ways to proceed. It depends on whether or not I want to use entry_point as a unique identifier for modules. If I did, the solution would just be to infer name from entry_point rather than url. If not, I would re-introduce name as a required attribute, and I would have to construct the modules with imp.create_module rather than importing them normally. Going forward, this way makes more sense, especially when modules will be moved to a different directory.
I like the second way, any opinions?


Update OceanSpy and Oceanography image on SciServer

At some point we should spend some time to update OceanSpy and the Oceanography image on SciServer. If you are using OceanSpy with the latest dependencies, let's use this issue to collect warnings/issues that need to be addressed.

This issue creates problems with the survey function: https://github.com/JiaweiZhuang/xESMF/issues/71
We need to add:
regridder._grid_in = None
regridder._grid_out = None

here:
https://github.com/malmans2/oceanspy/blob/40b0ff79e9f094e455d3e8c57faf3e49bc79d888/oceanspy/subsample.py#L983-L984

FutureWarning: dropping coordinates using drop is be deprecated; use drop_vars.
ds = self._to_temp_dataset().drop(labels, dim, errors=errors)

We need to add the combine argument in open_mfdataset:
combine ({'by_coords', 'nested'}, optional) – Whether xarray.combine_by_coords or xarray.combine_nested is used to combine all the data. If this argument is not provided, xarray.auto_combine is used, but in the future this behavior will switch to use xarray.combine_by_coords by default.

Done!


VSCode Pasting Issues with Multiple Displays

I've been struggled with this for a very long time, so I'm finally raising this as a bug.
I'm currently on v0.24.0.
If I have multiple VSCode windows opened across two displays, when I paste from the app, it pastes into the other window opened in the other display/monitor instead of the one which was focused earlier before the shortcut was executed.
I have a paid license for the product.

There is not much I can do about it, it's being working and not working for Electron-based apps and Chrome. Sometimes when you activate the Chrome window - it activates not the most recent one, but a different one - that's exactly what happens during paste. Closing this as it's duplicate of #534


createPlutusDataCbor doesn't accept certain acceptable structures to cardano-cli

This structure is not accepted:
import { encode } from 'cborg';
import {utils} from "@strictahq/typhonjs"

const pkh = Buffer.from(await getPubKeyHash());
const pkhCBORed = Buffer.from(encode(pkh));

utils.createPlutusDataCbor({
  "fields": [
    {
      "fields": [ pkhCBORed ],
      "constructor": 0
    }
  ],
  "constructor": 0
})

or with the bytes tag and the CBOR as a string (as it is accepted in cardano-cli)

the encode import looks different, are you trying to use @stricahq/cbors?
pkh doesn't need to be cbor encoded (unless you are trying to do something specific)
I think you need to do Buffer.from(value, 'hex'), unless you know what the value is

That was it. The data should just be the bytes of the PKH not CBOR encoded bytes. Hash comes out the same as cardano-cli with the CBOR string in JSON.


update: translate packages/react-router-redux/README.md &  packages/react-router-redux/README-en.md

任务详见#5993

Please stop submitting these PRs.