The Go SDK v1.31+ break for some reason

Title: The Go SDK v1.31+ breaks development by requiring the Envoy process to run
Description:
It appears that because of the newly added code in the Go SDK v1.31+ as shown in https://github.com/envoyproxy/envoy/blob/cc4a75482810de4b84c301d13deb551bd3147339/contrib/golang/common/go/api_impl/capi_impl.go#L64-L85.
Repro steps:
Including the package github.com/envoyproxy/envoy/contrib/golang/filters/http/source/go/pkg/http in a filter that requires unit testing will immediately cause it to break and display the following error:
SIGSEGV: segmentation violation
PC=0x0 m=0 sigcode=1 addr=0x0
signal arrived during cgo execution

goroutine 1 gp=0xc0000061c0 m=0 mp=0xca0260 [syscall, locked to thread]:
runtime.cgocall(0x7fead0, 0xc0000cbdb8)
	/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/runtime/cgocall.go:167 +0x4b fp=0xc0000cbd90 sp=0xc0000cbd58 pc=0x46c74b
github.com/envoyproxy/envoy/contrib/golang/common/go/api_impl._Cfunc_envoyGoFilterLogLevel()
	_cgo_gotypes.go:67 +0x47 fp=0xc0000cbdb8 sp=0xc0000cbd90 pc=0x7ee267
github.com/envoyproxy/envoy/contrib/golang/common/go/api_impl.init.0()
	/home/runner/go/pkg/mod/github.com/envoyproxy/envoy@v1.31.2/contrib/golang/common/go/api_impl/capi_impl.go:76 +0xdb fp=0xc0000cbe20 sp=0xc0000cbdb8 pc=0x7ee43b
runtime.doInit1(0xc60710)
	/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/runtime/proc.go:7278 +0xe8 fp=0xc0000cbf50 sp=0xc0000cbe20 pc=0x44b648
runtime.doInit(...)
	/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/runtime/proc.go:7245
runtime.main()
	/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/runtime/proc.go:254 +0x345 fp=0xc0000cbfe0 sp=0xc0000cbf50 pc=0x43d105
runtime.goexit({})
	/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0000cbfe8 sp=0xc0000cbfe0 pc=0x47adc1

It appears that the code required the envoy process to run, specifically on the following code L76:
https://github.com/envoyproxy/envoy/blob/e3ed5a7adcab0f519717c054bf88b664e4e2f5b1/contrib/golang/common/go/api_impl/capi_impl.go#L75-L77

Note: The Envoy_collect tool
gathers a tarball with debug logs, config and the following admin
endpoints: /stats, /clusters and /server_info. Please note if there are
privacy concerns, sanitize the data prior to sharing the tarball/pasting.

Admin and Stats Output:

Include the admin output for the following endpoints: /stats,
/clusters, /routes, /server_info. For more information, refer to the
admin endpoint documentation.

Note: If there are privacy concerns, sanitize the data prior to
sharing.

Config:

Include the config used to configure Envoy.

Logs:

Include the access logs and the Envoy logs.

Note: If there are privacy concerns, sanitize the data prior to
sharing.

[optional Relevant Links:]
Slack thread: https://envoyproxy.slack.com/archives/C04QNSXC7U0/p1727604052263089

cc @doujiang24

cc @spacewander

Including the package github.com/envoyproxy/envoy/contrib/golang/filters/http/source/go/pkg/http in a filter that requires unit testing will immediately cause it to break and display the following error

The break is because the unit test executes C.xx functions exported by Envoy outside the Envoy process. It's recommended to move the  github.com/envoyproxy/envoy/contrib/golang/filters/http/source/go/pkg/http outside the uniting test path if the test doesn't run on an Envoy. Because the golang/filters/http/source/go/pkg/http provides some functions that will be executed in Envoy, it is unavoidable to run into the C.xx.
In fact, import golang/filters/http/source/go/pkg/http will break the compile in some Mac environments even before 1.31. For example, in my dev machine I will get:
dyld[72971]: symbol not found in flat namespace (_envoyGoConfigHttpFinalize)
signal: abort trap

It's lucky if you don't mean any problem caused by this before Envoy 1.31.
There are some ways to solve this problem. For example, in HTNN we move the golang/filters/http/source/go/pkg/http in a separate package https://github.com/mosn/htnn/blob/main/api/tests/integration/libgolang/main.go so it won't affect the unit test. You can also find some useful skills in HTNN.
As you are running unit test... Moving this part to a new package also improves the coverage as you can't really run into it in your unit test.

As you are running unit test... Moving this part to a new package also improves the coverage as you can't really run into it in your unit test.

which means it is intended to be exactly like that, yes @spacewander?

CC @doujiang24

Sorry for not replying in time. We are on holiday these days.

As you are running unit test... Moving this part to a new package also improves the coverage as you can't really run into it in your unit test.

which means it is intended to be exactly like that, yes @spacewander?

Not intended, but it is recommended to do this. Putting this part on the same package doesn't have enough benefits, and moving it to a separate package can improve the modularization.

@ardikabs Not a real bug/break, just need a small tweak in tests, right?
Move http.RegisterHttpFilterFactoryAndConfigParser into another Golang package that won't be imported in tests.

yup @doujiang24 , i am following @spacewander suggestion.

There are some ways to solve this problem. For example, in HTNN we move the golang/filters/http/source/go/pkg/http in a separate package https://github.com/mosn/htnn/blob/main/api/tests/integration/libgolang/main.go so it won't affect the unit test. You can also find some useful skills in HTNN.


Extra  blocks around in-place markup

This may be user error.
I'm trying to set up a multi-column Bootstrap layout in Metalsmith with Handlebars. I have a page design that has
left-col        right-col
   bottom-wide-section

where left-col, right-col, and bottom-wide-section are all user-provided content (markdown).

Bootstrap requires me to wrap parts of my {{{content}}} block in row and column divs.
Handlebars returns content as a monolithic block and doesn't have provisions for splitting it (as best I can tell).
Metalsmith can't combine multiple .md files into a multi-part content block (as best I can tell), and even if it could, I doubt Handlebars could address it properly.

So, to get around these limitations, I was trying to use in-place to define rows and columns as follows (reduced test case, only defines row).
test.md
---
layout: standardPage.hbs
title: "TestPage"
---
{{{inplace.beginRow}}}

This is some test content.

Will there be a blank paragraph block above?

{{{inplace.endRow}}}

When my pipeline is markdown -> in-place -> layouts, I get the following output.

This is some test content.

Will there be a blank paragraph block above?

Note the
 blocks enclosing the
 and
. The browser automatically "fixes" the HTML.
Browser "Fixed" HTML

This is some test content.

Will there be a blank paragraph block above?

This creates layout problems by injecting extra paragraph margins and padding in the layout.
Any pipeline that runs markdown first will have the extra
 blocks because markdown wraps the in-place values in
 blocks as shown by the markdown -> layouts (no in-place) pipeline below.

{{{inplace.beginRow}}}

This is some test content.

Will there be a blank paragraph block above?

{{{inplace.endRow}}}

Running markdown after in-place prevents markdown from rendering content wrapped in divs and other block elements.
Is this expected behavior?
If so, how would one go about assembling a page that requires content to be broken into several pieces on the page? Or how would one inject HTML markup into the content block? (Coding it directly will cause markdown to skip wrapped content, and I'd rather not ask users to write HTML--that's the whole point of markdown.)
Or should in-place (optionally?) remove and immediately adjacent

 wrapper?

Well, metalsmith-layouts will pass file.contents to the selected layout as a single {{contents}} variable, so there's no way to have multiple content blocks. And metalsmith-in-place will process variables in your source files, but that's not the same as applying a layout (if you're using handlebars).
If you want to use handlebars then I'd suggest you use handlebars-layouts which allows you to have multiple content areas: https://github.com/shannonmoeller/handlebars-layouts.
Or, you could use a different templating language, like Jade, Jinja, Swig, or Twig, which already have support for these kinds of content blocks.

So, it sounds like this isn't necessarily by explicit design, but isn't surprising either.
Thanks for the tip on handlebars-layouts. After a quick look, my sense is I'll run into the same issues. Either Markdown will wrap the handlebars-layouts helpers in
 blocks or, after handlebars-layouts runs, I'll have trouble getting Markdown to process--or it will undo handlebars-layouts content blocks. I'll come back to this later.
I may investigate one of the other template engines in the long run, but I think I'm going to run into the same issue again unless I have users write content in HTML.
I think the issue is actually with how Markdown is defined. Markdown has no way to define named content blocks. (Even if it did, Handlebars probably couldn't address them). With the content it has, it either wraps everything in
 blocks or ignores content in HTML blocks (like
). What I really need is a way to tell Markdown, "Ignore these {{ }} or {{{ }}} things." There's also no way I'm aware of in Markdown or Metalsmith to say, "Bind some .md files into a single object with keys left, right, and bottom."
The latter may be a solution that has broader applications (though I'm not sure what yet). Front-matter defines the target file name and key for a given .md. A post-processor plugin runs immediately after markdown and rewrites the files array based on the metadata. But that's more than I have time to do right now.
So, I think I'll try the simple approach first--a plugin that will strip the
 tags from Handlebars-like expressions. This approach also avoids the disk file management issue of having index_a.md, index_b.md, etc.  I was expecting I'd need to go that route, so I've built a regex to identify
-wrapped Handlebars (except comments). I'll work on getting it plugin-ized and see how it works.

Well, it's easier than I thought. metalsmith-grep does what I need. I may try to put together a pull request for it to add a file filter argument, but that's all.
For Handlebars, run metalsmith-grep after markdown and before in-place with the regexp below. Wrap Handlebars in a 4-stache ({{{{ }}}}), and the regexp will turn it into a 3-stache when wrapped by a
 block with nothing else. That means each 4-stache needs to be in it's own paragraph.
metalsmith-grep call
	.use(markdown())
	.use(grep({
		subs: [
			{
				search: /
\s*\{((\s*\{\{\{\s*(~?([#/][^\s!"#;<=>@`%-,[-^{-~]*)?)\s*@?[^\s!"#;<=>@`%-,[-^{-~]*~?\s*}}}\s*)+)}\s*<\/p>/gi,
				replace: '$1'
			}
		]
	}))
	.use(inplace(...

test.md
---
layout: standardPage.hbs
title: "TestPage"
---
{{{{inplace.beginRow}}}}

This is some test content.

Will there be a blank paragraph block above?

{{{{inplace.endRow}}}}

Output HTML

This is some test content.

Will there be a blank paragraph block above?

I chose to use a 4-stache to ensure the regexp could never accidentally replace deliberate 3-stache markup, but given the requirement for a surrounding
 block with no other content but whitespace, this is belt-and-suspenders level safety.

Ok cool. I'll be closing this then, but let me know if there's anything else!


require explicit check for undefined?

This code has a runtime error. It will not use the 0 value.
Should Hegel force us to explicitly check undefined to avoid the runtime error?

This is the goal "Don't have a runtime error". But, I actually don't understand the example, because, I run it and didn't get an error. Could please you explain the problem?

Sorry, not runtime error, but a silent error: the n && useNum(n) does not pass the 0 into useNum because 0 is falsy. So if the dev meant to pass all numbers into the function, it will pass them all except zero.
If it forces us to check for undefined, then it could prevent such a mistake.

Maybe n && useNum(n) would still have a type error, and we would need to write something like n !== undefined && useNum(n).

I mean, maybe the fact that undefined was not checked could somehow be a type error?

Maybe https://github.com/JSMonk/hegel/issues/227 would help with eliminating verbosity.

Maybe n && useNum(n) would still have a type error, and we would need to write something like n !== undefined && useNum(n). Not sure though.

No, actually, if you will hove at the left part of the && you will see which values will be not passed into the function.

Oh yeah, the tooltip shows 0 | undefined, but it isn't a guarantee people are going to look at that.
So TLDR, this type of error checking is out of scope?

It seems like I can't show the error in this case. And I don't sure, that this is Type System responsibility (Actually, I think it's linter responsibility, because of this case is about coding style neither the type system issue).

i think that this would report error
const nums: Array<number> = [0,1,2,3,4,5]

for (let i=0; i<nums.length; i++) {
  const n = nums[i]
  n && useNum(n)// should report: Type "number | undefined" is incompatible with type "boolean"
}

function useNum(n: number) {
  n
}

like this does
const nums: Array<number> = [0,1,2,3,4,5]

for (let i=0; i<nums.length; i++) {
  const n = nums[i]
  if (n) useNum(n)// reports: Type "number | undefined" is incompatible with type "boolean"
}

function useNum(n: number) {
  n
}

i think && and || are boolean operators (and you should not be allowed to use it otherwise)

I think it would make sense to warn on Nullish | PossblyFalsy used in if, while, ternary, etc.
(where Nullish is either null or undefined or union of both, and PossiblyFalsy is any combination of string, number, bigint and boolean)
This is what Flow does (with sketchy-null lint enabled), and there's strict-boolean-expressions for typescript-eslint which also does that.

I don't think a linter could do a good job checking these things, otherwise the linter would become something like Hegel already is.


Python: specifying an effect is broken.

I'm using the package

[ ] iam-floyd
[x] cdk-iam-floyd

I'm using the package in language

[ ] TypeScript/JavaScript (npm)
[x] Python (pip)

Describe the problem
Works:
statement.Cloudwatch().to_describe_alarms()
Does not work:
statement.Cloudwatch().allow().to_describe_alarms()
Typing error: Cannot access member "to_describe_alarms" for type "PolicyStatementWithEffect". Member "to_describe_alarms" is not known.
Only thing that works with allow() is the to() method. No service-specific methods work.
I'm using the latest version of cdk-iam-floyd.

@udondan can you take a look?

Going to close this. The problem is already tracked in #99.


[STORM-1230] port backtype.storm.process-simulator to java.

From storm-core/src/clj/org/apache/storm/process-simulator.clj to storm-core/src/jvm/org/apache/storm/ProcessSimulator.java

Only two very minor comments, other even without them I am a +1

Hi @revans2, really appreciate your comments! I addressed your comment in Storm-1563 to change all the Shutdownable to be AutoClosable in storm.

upmerge

@redsanket thanks, upmerged.

Very minor comment, not significant LGTM +1


No copy/paste options in build 848

No option for copy paste. No ability to open multiple windows.
Information

Device : [iPad Gen 3]
OS : [e.g. iOS 15.4]
WebSSH Pro version : [17.0 (848)]

Screenshots
N/A
To Reproduce
Steps to reproduce the behavior:

Open ssh connection

Attempt any copy paste operation either by selecting text of from the buffer.

Open a connection

Attempt to open a second connection window

Hello @DustyBawls 👋

iOS terminal copy tracking is here : https://github.com/isontheline/pro.webssh.net/issues/150#issuecomment-1021656119
I don't understand "attempt to open a second connection window". Doesn't the multi window working on your device? Could you post a video?

My apologies,
Having reopened WebSSH the multiple window option is working fine. The copy paste is still not working.
Kind regards
-- Nigel

On 30 Jan 2022, at 21:16, Arnaud Mengus @.***> wrote:

Hello @DustyBawls 👋
iOS terminal copy tracking is here : #150 (comment)
I don't understand "attempt to open a second connection window". Doesn't the multi window working on your device? Could you post a video?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you were mentioned.


Tutorial Source Code

Can we see the tutorial source code ? Especially for unfinished yaml Kubernetes resources .

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 95aa8619-2154-00fb-5994-09b7a9673e03
Version Independent ID: 182cacc8-12d1-70ac-6005-71d4f6f420d5
Content: Build ASP.NET Core applications deployed as Linux containers into AKS/Kubernetes clusters
Content Source: docs/architecture/containerized-lifecycle/design-develop-containerized-apps/build-aspnet-core-applications-linux-containers-aks-kubernetes.md
Product: dotnet-architecture
Technology: containerized-lifecycle
GitHub Login: @nishanil
Microsoft Alias: nanil

Hi @sorryb, thank you for reaching out.
Unfortunately, the actual source code for the WebApi project is not available or maintained under any public repository. The article only talks about the necessary steps to build an ASP.NET Core application with Docker support which can be further
deployed to AKS/kubernetes. These steps are generic and can be referred as to build any such project.
Having said that, there is an entire application eShopOnContainers available for you to explore. It  has been developed following similar steps.
Hope this helps !

Hi @sorryb, I am closing this issue as of now. If you want to explore more on this topic, you may want to check out below MS learn modules -
Hello World Microservice tutorial
Create and deploy a cloud-native ASP.NET Core microservice
Please feel free to reopen it, if you have any pending question.
Thank you.


HT's Treechop Conflict with Enderio/EnderCore

Issue Description:
Creation of ghost blocks when simple machines are places then broken in survival mode
What happens:
Simple Alloy Smelter, SAG Mill, or Stirling Generator is placed, player attempts to break previously placed simple machine with pickaxe, simple machine is turned into ghost block and cannot be removed.
What you expected to happen:
Simple Alloy Smelter, SAG Mill, or Stirling Generator is placed, player attempts to break previously placed simple machine with pickaxe, simple machine is broken and drops in block form so player can move to optimal position
Steps to reproduce:
IN SURVIVAL MODE

craft a Simple Alloy Smelter, SAG Mill, or Stirling Generator
Place crafted simple machine in world
Break previously placed simple machine with pickaxe
...

Affected Versions:

EnderIO: 1.12.2-5.2.66
EnderCore: 1.12.2-0.5.76
HT's Treechop: 1.12.2-0.11.2
Minecraft: 1.12.2
Forge: <IP_ADDRESS>55
SpongeForge? no
Optifine? no
Single Player and/or Server? Single Player

Your most recent log file where the issue was present:
file:///Users/zac/Library/Application%20Support/gdlauncher_next/instances/Testing1/logs/latest.log

Thanks for the sweet report! I've found the issue and will have it fixed shortly.

Thank you so much! Glad I could help

The issue should be fixed in 0.11.4 (https://www.curseforge.com/minecraft/mc-mods/treechop/files/3193897)


mgo.v2 is unmaintained

Currently used mongo driver is not maintained anymore. Are there plans to move to something else?

Hi, @tomasol, I didn’t know it was not maintained anymore. Thanks for the information. Are you into any issues? There is a built container that must be working at “flaviostutz/schellar”.

Hi, not really. I am considering your project for conductor scheduling, I found no problems during my initial testing. Feel free to close the issue.

Great. If you need any assistance come here. PRs are welcome too!


[hplsql] syntax highlight for hplsql dialect

What changes were proposed in this pull request?

syntax highlight for hplsql keywords, built-in functions, datatypes.

How was this patch tested?

manually check with editor.

some places, we are using snippetviewsetting :
1 -> https://github.com/cloudera/hue/blob/master/desktop/libs/indexer/src/indexer/templates/importer.mako#L2960
2 -> https://github.com/cloudera/hue/blob/master/desktop/core/src/desktop/js/ko/components/ko.historyPanel.js#L179
3 -> https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/templates/editor2.mako#L1057
4 -> https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/templates/editor_m.mako#L180
so all files also need to update with 'hplsql' settings? @JohanAhlen

https://github.com/cloudera/hue/blob/master/desktop/libs/indexer/src/indexer/templates/importer.mako#L2960

some places, we are using snippetviewsetting : 1 -> https://github.com/cloudera/hue/blob/master/desktop/libs/indexer/src/indexer/templates/importer.mako#L2960 2 -> https://github.com/cloudera/hue/blob/master/desktop/core/src/desktop/js/ko/components/ko.historyPanel.js#L179 3 -> https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/templates/editor2.mako#L1057 4 -> https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/templates/editor_m.mako#L180
so all files also need to update with 'hplsql' settings? @JohanAhlen

No, I think it's enough with editor_components.mako, the other ones are probably just unused duplication.


Progress bar don't work?

It don't work for me. Can't enable it.

I hate that resource, never worked correctly
Maybe we should just go back to the original version and see if that helps

Or "get rid" of this resource :)

Restart seems to have fixed it


Automate release workflow

Automatically release when tags are pushed to master
Release automation will build wheel and sdist + upload to pypi
Added test pypi as initial target
Updated version update scripts to help with automatic version
management

tagged reviewers for awareness -- will be adding release documentation

I see it is uploading to test-pypi, is it a temporary solution or a standard workflow?

the test.pypi.org part will be a test to make sure the action works properly, my intention is to switch over to do real releases this way if everyone thinks it makes sense

It makes sense to me. I also tried the release workflow of pythonfinder, it works like a charm.

It makes sense to me. I also tried the release workflow of pythonfinder, it works like a charm.

I didn't even realize you made a new release, that's awesome


for () and while () loops

Linux 5.3 merged BPF bounded loops ( https://lwn.net/Articles/794934/ ). This may allow bpftrace to have implementations of for and while loops (not urgent, since 5.3 in production is some way away!).

( I've got a basic while POC here which supports: i:s:5 { $a = 1; while($a <= 100) { @=count(); $a++ }  exit() }. It looks like LLVM9 (fedora 31) does loop unrolling up until $a <= 15 so this might work on older kernels too )
How do we want this to work?
We can go for the C style loops, e.g:

while($a < 10) {@=count(); })
for($a=0; $a < 10; $a++) { @=count() }

Or we can do something like go does and only implement a for which acts like a while:

for ($a < 10) { @=count(); }
for($a=0; $a < 10; $a++) { @=count() }

We could also implement a loop like rust does:

loop { ...; if ($a == 10) break; }

Or...
I like the C ones but I don't have any strong preferences. In all cases I think it will be useful to implement the continue and break keywords

C style loops seem good to me. Everyone's going to be familiar with them too

break statement has its own issue: https://github.com/iovisor/bpftrace/issues/461, since it can also be used on unroll. continue would also make sense for unroll I think.
I'm wondering if we can also have a for ($a in @map) { ... } to iterate over map keys, arrays, etc. Not sure how viable that would be. This could also be implemented in the future if we want, as it wouldn't be a breaking change for the language.

Moving the discussion from #1066 here

ajor:
I'm not sure why anyone would want a regular loop inside an unrolled one but don't seem the harm in allowing it through (I'm assuming it would just work unless we explicitly block it).

My  "issue" with allowing it is that we must also keep it working, write useful semantic and codegen tests etc.

ajor:
Might as well allow jumps since they'll be forward only in an unrolled loop.

mmarchini
break statement has its own issue: #461, since it can also be used on unroll. continue would also make sense for unroll I think.

In my PR (#1066) jumps are currently not allowed (as in_loop is not incremented in unroll). Allowing those shouldn't be too much work so I'll give that a go in a a follow up PR to avoid "bloating" the current.

I'm wondering if we can also have a for ($a in @map) { ... } to iterate over map keys, arrays, etc. Not sure how viable that would be.

AFAIK  there is no way to iterate maps with bpf helpers now, you can only do that in userspace. And  from bpf we also have to deal with percpu maps, cpus cannot access each others values. It will probably just be a source of confusion/bugs.

wrt loops inside unroll, we can give a loops inside unrolls are currently not supported. It might be in the future message. We could even create a tracking issue and link it on the message, so people can chime in with examples of why they tried to use it, and if there is enough interest we implement it.
I don't have strong opinions about this though. There's a good chance that, with loops, unroll will be a very niche feature.

ajor:
I'm not sure why anyone would want a regular loop inside an unrolled one but don't seem the harm in allowing it through (I'm assuming it would just work unless we explicitly block it).

My "issue" with allowing it is that we must also keep it working, write useful semantic and codegen tests etc.

Sure it should be tested, but since mixing unroll and while is something that would just work otherwise, blocking it would actually add more complexity to bpftrace than allowing it, and would require testing too. Blocking it would also something we'd have to keep working - either that or we eventually just allow mixing the loop types, so we might as well just get it over with now.
I guess my thinking is that even though I don't have a use for it, someone might be doing something crazy one day that means it'd be useful to them and we shouldn't add restrictions to the language if it's reasonable (and easier!) not to.
(Maybe it could help in a program which is just over the BPF verifier's complexity limit while still being within the instruction limit?)
Allowing the loop types to be mixed means we should probably make break and continue work with unroll loops otherwise there could be some confusion about which loop is being broken out of. They shouldn't be too hard to implement though.

Implemented in #1066 .


Seal classes in shelf_router and prepare 0.7.3

@graphicbeacon I randomly stumbled into your video and figured it would be useful if package users could find it.

Thank you @jonasfj


Complete Unicode support for linguistic processing and indexing

It seems that vespa does not completely support Unicode indexing and retrieval right now, only alphabetic and numeric characters are considered to be isIndexable.
I've encountered this issue while playing with Emojis, searching for 😊 will simply get rejected with a 400 Bad Request. Searching 😊a is allowed but unfortunately the results are only about a since the Emoji code points were stripped.
Actually it is still unclear to me which code points are invalid as the comments in TokenType noted:

Note that a Token can be excluded from an index, even though the token type marks it as indexable.

Further testing revealed that besides Emojis, a wide range of codes points were excluded while indexing. I would suggest to preserve all Unicode code points throughout linguistic processing and indexing, as it would increase adoption of vespa in domains like conversation searching (which requires Emoji) and technical document indexing (which even the U+FFFD replacement character is meaningful).

The tokenizer in Vespa turns text into tokens that are indexed and used for queries. There are two aspects to this: Deciding what characters should be included as indexable tokens, and deciding the token boundaries. These decisions impact recall and precision, as well as performance.
For example, if we indexed every period sign as a separate token, we would create a much larger number of tokens, and any query which included a period would in effect retrieve the entire index. Or, if we indexed periods as part of the preceding word, we would not retrieve any document which happened to have that word as the last in a sentence. For this reason it is not in general a good idea to just preserve all code points (and it doesn't even have one unique meaning).
Regarding emoji's - the default tokenizer does not classify them as text tokens for indexing. This can be changed by plugging in a custom tokenizer, but should it be changed in the default? Do you know of a real use case where people want to retrieve only the documents that contains a specific emoji?

For example, if we indexed every period sign as a separate token, ... Or, if we indexed periods as part of the preceding word, we would not retrieve any document which happened to have that word as the last in a sentence.

For the specific case of period signs (and other ambiguous punctuations), I would suggest to follow the Word Boundary Rules in Unicode® Standard Annex #29.

... we would create a much larger number of tokens, and any query which included a period would in effect retrieve the entire index... For this reason it is not in general a good idea to just preserve all code points (and it doesn't even have one unique meaning).

We used to implement our in-house information retrieval system with index-time pruning based on Inverse document frequency. Basically we assume punctuations will have low IDF weights and cut them off just like stop words in large text fields (e.g. body), while preserving a limit amount of them in significant text fields like titles. This might lead to index format changes since we have to count the frequency of a token while not actually indexing it. I think we could take a simpler approach: maybe just allow the users to enable full Unicode in specific fields in their schemas?

... but should it be changed in the default? Do you know of a real use case where people want to retrieve only the documents that contains a specific emoji?

We've been testing with a dataset of social network dumps, the original platform allows users to use display names with arbitrary Unicode characters, so that Elon and Elon ♐ are two unique users.
Actually I think emoji is only the tip of the iceberg, adding support for all Unicode planes is more of a future-proof enhancement. Here's a real world example: The Unicode Consortium reserves code points for eras in the Japanese calendar, but the latest ㋿ U+32FF announced on 1 April 2019 is not a subsequent of the previously used block of ㍻ U+337B - ㍾ U+337E. Although I'm not Japanese but I'm aware that they use these Unicode graphemes in official documents and formal writings to distinguish ㍼24年 (AD 1949) vs ㍻24年 (AD 2012).
I understand that Unicode might not be a first class citizen in languages with latin-script writing systems, but languages of the world are so diverse, the Chinese are basically writing in "emojis". Vespa is such a wonderful piece of technology and enhancing Unicode support will definitely bring more diversity to the community.

For the specific case of period signs (and other ambiguous punctuations), I would suggest to follow the Word Boundary Rules in Unicode® Standard Annex #29.

I can't even understand if that recommends keeping "foo." together or not, but in any case this must be decided by the tokenizer (which by default is OpenNLP), not Vespa.
Using IDF to determine indexing would have a least three problems: We need to track it when not indexing (as you say), non-indexable characters can be low frequency (e.g ";"), and indexable may have high frequency (say, "the" - we still want to match "the the").

Elon and Elon ♐ are two unique users

For user names you should not use text matching but instead match:exact, which just matches all the characters exactly (including whitespace). If you want that, but not whitespace, you can use match:word instead. See https://docs.vespa.ai/en/reference/schema-reference.html#match
(You might also want to make these attributes instead of index so that you can group on username etc.)
FYI, all of this is not related to unicode in itself - Vespa uses unicode everywhere.
CJK characters are classified as text (in contrast to emojis), but here the problem is that you need a tokenizer that segments the text into probably-correct tokens. OpenNLP does not do that so you need to plug in another tokenizer for that.
However, segmentation of CJK in queries is unreliable due to the lack of context, so it will always cause recall issues. I recommend using n-gram instead for CJK, in combination with a rank function that incorporates position info (such as e.g nativeRank).


Pre-computed type system as a compiler input

apollo-compiler recently gained the ability handle multiple inputs. Each input has a string to be parsed with apollo-parser, and is optionally limited to only type definitions or only executable definitions.
To work around the limitation of Salsa being !Sync, in apollo-router we’d like first parse a schema (type system definitions), pre-compute what we can about it, extract all of that into read-only data that can be shared through Arc with multiple threads, and have each thread be able to use this in apollo-compiler without reparsing. "Pre-computed type definitions" a.k.a. "pre-computed schema" would be an additional kind of input that can be provided to a new compiler. However this single input in the new compiler could reference multiple string inputs from the original compiler, so FileId handling needs to be aware of this so that diagnostics print the correct context.
Option 1
A single compiler can have type definitions from any number of pre-computed inputs and any number of string inputs. This could be enabled by separating the concept of "source" from the concept of "input". API sketch:
struct SourceId(u64);
struct Source {
    filename: PathBuf,
    text: Arc<str>,
}

struct InputId(u64);
enum Input {
    Schema(SourceId),
    Query(SourceId),
    Document(SourceId),
    PrecomputedSchema(Arc<PrecomputedSchema>),
}

pub struct PrecomputedSchema {
    // Type definitions etc
    sources: IndexMap<SourceId, Source>
}

#[salsa::query_group(InputStorage)]
pub trait InputDatabase {
    #[salsa::input] fn input(&self, id: InputId) -> Input;
    #[salsa::input] fn source(&self, id: SourceId) -> Source;

Option 2
In order to reduce complexity from the above, we introduce a limitation : if a compiler has a pre-computed schema input, that must be the only input with type system definitions. (In other words, all other inputs must be "query" inputs with only executable definitions). That way we don’t need IDs for pre-computed inputs (the one is either there or not) and can stick with a single FileId type. Internally, HIR-construction queries would effectively have two separate "modes".
This is sufficient for current apollo-router plans. Since pre-computing a niche feature to begin with, it’s unlikely many other people will run into this limitation.

Thanks for writing this out!
At the moment I only expect apollo-router and @lennyburdette plugin work to be using a pre-computed type system, so it seems wise to keep the scope of it as small as possible and make other considerations should more use cases come up. That is, having either a pre-computed schema or schema str as input for compiler's schema is fine.


relatedProfiles not working when relatedProfilesKeywords is empty

if relatedProfilesKeywords : [] , the related profiles sections return as 0 profiles found

fixed at https://github.com/linkedtales/scrapedin-linkedin-crawler/commit/27e103c2b9481f7ebf3e577e287b6eb9bd96898d


Openwisp utils 0.3.2

Coverage decreased (-0.03%) to 96.907% when pulling 3ae6e572971ccc6c64df8df1e608daddb6490b83 on openwisp-utils-0.3.2 into b00072f04937a071c7c8e9b7098e292d65a7c5ec on master.


Bech32: Minor style fixes + remove usage of logger

Pretty simple stuff, see each commit.

TIL the order of @safe pure nothrow @nogc.

We don't really have an order, besides having type qualifiers first followed by safety (and simply because it's so omniscient).
It's not something I would flag in code review either, as I don't think it's productive (we would need something like dfmt for it to be something worth even talking about).


capnp build error

From @xet7 on February 7, 2018 21:41
@kentonv
When I try to build wordpress-sandstorm .spk file in vagrant with this script:
https://github.com/wekan/wordpress-sandstorm/blob/master/build-spk-in-vagrant.sh
I get this error when building capnp. How can this be fixed?
libtool: install: chmod 644 /usr/local/lib/libcapnpc.a
libtool: install: ranlib /usr/local/lib/libcapnpc.a
libtool: finish: PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin" ldconfig -n /usr/local/lib
----------------------------------------------------------------------
Libraries have been installed in:
   /usr/local/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the `LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the `LD_RUN_PATH' environment variable
     during linking
   - use the `-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to `/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
 /bin/mkdir -p '/usr/local/bin'
  /bin/bash ./libtool   --mode=install /usr/bin/install -c capnp capnpc-capnp capnpc-c++ '/usr/local/bin'
libtool: install: /usr/bin/install -c .libs/capnp /usr/local/bin/capnp
libtool: install: /usr/bin/install -c .libs/capnpc-capnp /usr/local/bin/capnpc-capnp
libtool: install: /usr/bin/install -c .libs/capnpc-c++ /usr/local/bin/capnpc-c++
make  install-exec-hook
make[3]: Entering directory '/home/vagrant/capnproto/c++'
ln -sf capnp /usr/local/bin/capnpc
ldconfig < /dev/null > /dev/null 2>&1 || true
make[3]: Leaving directory '/home/vagrant/capnproto/c++'
 /bin/mkdir -p '/usr/local/lib/cmake/CapnProto'
 /usr/bin/install -c -m 644 cmake/CapnProtoConfig.cmake cmake/CapnProtoConfigVersion.cmake cmake/CapnProtoMacros.cmake cmake/CapnProtoTargets.cmake '/usr/local/lib/cmake/CapnProto'
 /bin/mkdir -p '/usr/local/include/capnp'
 /usr/bin/install -c -m 644 src/capnp/c++.capnp src/capnp/schema.capnp src/capnp/rpc.capnp src/capnp/rpc-twoparty.capnp src/capnp/persistent.capnp src/capnp/compat/json.capnp '/usr/local/include/capnp'
 /bin/mkdir -p '/usr/local/include/capnp'
 /usr/bin/install -c -m 644 src/capnp/c++.capnp.h src/capnp/common.h src/capnp/blob.h src/capnp/endian.h src/capnp/layout.h src/capnp/orphan.h src/capnp/list.h src/capnp/any.h src/capnp/message.h src/capnp/capability.h src/capnp/membrane.h src/capnp/schema.capnp.h src/capnp/schema-lite.h src/capnp/schema.h src/capnp/schema-loader.h src/capnp/schema-parser.h src/capnp/dynamic.h src/capnp/pretty-print.h src/capnp/serialize.h src/capnp/serialize-async.h src/capnp/serialize-packed.h src/capnp/serialize-text.h src/capnp/pointer-helpers.h src/capnp/generated-header-support.h src/capnp/raw-schema.h src/capnp/rpc-prelude.h src/capnp/rpc.h src/capnp/rpc-twoparty.h src/capnp/rpc.capnp.h src/capnp/rpc-twoparty.capnp.h src/capnp/persistent.capnp.h src/capnp/ez-rpc.h '/usr/local/include/capnp'
 /bin/mkdir -p '/usr/local/include/capnp/compat'
 /usr/bin/install -c -m 644 src/capnp/compat/json.h src/capnp/compat/json.capnp.h '/usr/local/include/capnp/compat'
 /bin/mkdir -p '/usr/local/include/kj'
 /usr/bin/install -c -m 644 src/kj/common.h src/kj/units.h src/kj/memory.h src/kj/refcount.h src/kj/array.h src/kj/vector.h src/kj/string.h src/kj/string-tree.h src/kj/encoding.h src/kj/exception.h src/kj/debug.h src/kj/arena.h src/kj/io.h src/kj/tuple.h src/kj/one-of.h src/kj/function.h src/kj/mutex.h src/kj/thread.h src/kj/threadlocal.h src/kj/filesystem.h src/kj/async-prelude.h src/kj/async.h src/kj/async-inl.h src/kj/time.h src/kj/timer.h src/kj/async-unix.h src/kj/async-win32.h src/kj/async-io.h src/kj/main.h src/kj/test.h src/kj/windows-sanity.h '/usr/local/include/kj'
 /bin/mkdir -p '/usr/local/include/kj/compat'
 /usr/bin/install -c -m 644 src/kj/compat/gtest.h src/kj/compat/url.h src/kj/compat/http.h '/usr/local/include/kj/compat'
 /bin/mkdir -p '/usr/local/include/kj/parse'
 /usr/bin/install -c -m 644 src/kj/parse/common.h src/kj/parse/char.h '/usr/local/include/kj/parse'
 /bin/mkdir -p '/usr/local/include/kj/std'
 /usr/bin/install -c -m 644 src/kj/std/iostream.h '/usr/local/include/kj/std'
 /bin/mkdir -p '/usr/local/lib/pkgconfig'
 /usr/bin/install -c -m 644 pkgconfig/capnp.pc pkgconfig/capnp-rpc.pc pkgconfig/capnp-json.pc pkgconfig/kj.pc pkgconfig/kj-async.pc pkgconfig/kj-http.pc pkgconfig/kj-test.pc '/usr/local/lib/pkgconfig'
make[2]: Leaving directory '/home/vagrant/capnproto/c++'
make[1]: Leaving directory '/home/vagrant/capnproto/c++'
(cd sandstorm && make CXX=g++)
make[1]: Entering directory '/opt/app/sandstorm'
make[1]: *** No rule to make target '/opt/sandstorm/latest/usr/include/sandstorm/*.capnp', needed by 'tmp/genfiles'.  Stop.
make[1]: Leaving directory '/opt/app/sandstorm'
Makefile:29: recipe for target 'sandstorm/bin/getPublicId' failed
make: *** [sandstorm/bin/getPublicId] Error 2
Connection to <IP_ADDRESS> closed.
Command failed with a non-zero exit status (2).

BR,
xet7
Copied from original issue: wekan/wordpress-sandstorm#1

From @kentonv on February 11, 2018 6:18
It appears as if Sandstorm isn't installed inside the Vagrant image, which is weird. Maybe there's some more initialization that needs to be done by vagrant-spk? TBH I have never used vagrant-spk so I don't really know. Maybe @dwrensha (who created the Wordpress package) has an idea.

From @dwrensha on February 11, 2018 14:3
~Yeah I probably forgot to include the Sandstorm capnp files in the vagrant-spk build: https://github.com/dwrensha/wordpress-sandstorm/commit/17d397180e48282fe37d432aa8c9eb4d2786f4c8~
EDIT: hm. Those sandstorm files really should be there. When I do vagrant-spk up and vagrant-spk ssh, I see them in /opt/sandstorm/

From @dwrensha on February 11, 2018 14:31
When I try to build https://github.com/dwrensha/wordpress-sandstorm, it looks like I get farther than you did, but I still hit an error:
(cd sandstorm && make CXX=g++)
make[1]: Entering directory '/opt/app/sandstorm'
mkdir -p tmp
capnp compile --src-prefix=/opt/sandstorm/latest/usr/include -oc++:tmp /opt/sandstorm/latest/usr/include/sandstorm/*.capnp
*** Uncaught exception ***
kj/filesystem-disk-unix.c++:469: failed: mmap: Invalid argument
stack: 0x405b42 0x43a40e 0x43af2c 0x7f7d19cb38c5 0x7f7d19cb52c6
c++: plugin failed: exit code 1
Makefile:21: recipe for target 'tmp/genfiles' failed

I conjecture that this has to do with the fact that the /vagrant directory is synced between the host and guest.

From @kentonv on February 11, 2018 21:33
Ugh, apparently shared folders don't implement writable mmap(). I'm surprised they can get away with that. I guess maybe the capnp C++ output plugin needs to go back to using write(). Sad.
https://github.com/capnproto/capnproto/issues/635

This is the PR they used to fix their Vagrant 1.9.6 downloading the sandstorm version of the box: github.com/sandstorm-io/vagrant-spk/pull/209
Got that from @ocdtrekkie . I did change previously to probably broken box version.

From @JamborJan on March 6, 2018 19:38
Hi guys,
I have updated my fork of wordpress for Sandstorm (see https://github.com/JamborJan/wordpress-sandstorm/commit/cb93151f8b0901b6db1eb85aec7dbc28e230a4ef). Now I'm running into a different error:
(cd sandstorm && make CXX=g++)
make[1]: Entering directory '/opt/app/sandstorm'
generating capnp files...
/opt/sandstorm/latest/usr/include/sandstorm/payments.capnp:21:14-37: error: Import failed: /sandstorm/util.capnp
/opt/sandstorm/latest/usr/include/sandstorm/payments.capnp:28:29-42: error: Not defined: LocalizedText
/opt/sandstorm/latest/usr/include/sandstorm/payments.capnp:65:25-38: error: Not defined: LocalizedText
/opt/sandstorm/latest/usr/include/sandstorm/payments.capnp:73:17-30: error: Not defined: LocalizedText
/opt/sandstorm/latest/usr/include/sandstorm/payments.capnp:22:14-43: error: Import failed: /sandstorm/supervisor.capnp
/opt/sandstorm/latest/usr/include/sandstorm/payments.capnp:87:58-74: error: Not defined: SystemPersistent
/opt/sandstorm/latest/usr/include/sandstorm/payments.capnp:88:62-78: error: Not defined: SystemPersistent
/opt/sandstorm/latest/usr/include/sandstorm/payments.capnp:89:46-62: error: Not defined: SystemPersistent
Makefile:21: recipe for target 'tmp/genfiles' failed
make[1]: *** [tmp/genfiles] Error 1
make[1]: Leaving directory '/opt/app/sandstorm'
Makefile:26: recipe for target 'sandstorm/bin/getPublicId' failed
make: *** [sandstorm/bin/getPublicId] Error 2

I was not able yet to dig a bit deeper into these error messages but will do so as soon as possible. I wanted to leave a note here just in case someone of you can easily help or solve this.
PS: @xet7 as soon as we have figured out how we deal with the app key I will contribute to your repo. Until then I use my fork for testing reasons. Otherwise I would not be able to build packages and test them.
Thanks

@JamborJan
No problem, a moment ago I did fetch and merge changes from your repo to this repo, and also add some more modifications for php5 to php7. I will look at app key when bulding wp-sandstorm works, and there is something worth releasing.

From @ocdtrekkie on March 6, 2018 19:44
@JamborJan payments.capnp is a weird thing to show up here. I know that @kentonv just merged that back into Sandstorm proper, so it's possible something's missing there, and it's never been an issue with building before for that reason: https://github.com/sandstorm-io/sandstorm/pull/3042

From @JamborJan on March 7, 2018 8:3
@xet7
You also took my sandstorm-pkgdef.capnp. I can send you my private app key (over another secure channel) so that you are able tu build packages. If this is okay for you, I would be able to delete my fork and use yours instead.

@JamborJan
Yes it's OK. You can install desktop or mobile client of https://about.riot.im so we can enable end-to-end encryption, my Riot nickname is xet7wekan

Hey @ocdtrekkie, did I get your comment right and the payments.capnp will be analysed or is there something I need to do because I did something wrong when I tried to build the Wordpress app?

I am just suspecting it's a new problem cause by the fact that payments.capnp is new to the Sandstorm repo. Kenton should be around this weekend and can probably explain why it's showing up as an error.

It looks like there's a bug in the makefile:
capnp compile --src-prefix=/opt/sandstorm/latest/usr/include -oc++:tmp /opt/sandstorm/latest/usr/include/sandstorm/*.capnp

This line is missing an argument:
-I/opt/sandstorm/latest/usr/include

It fails on payments.capnp because that file contains absolute imports rather than relative, which is vestigial since it used to live in Blackrock rather than Sandstorm. I will change it to use relative imports in today's release, but it may also make sense to fix the Wordpress app's Makefile.

With latest changes in git I have this progress, after installing vagrant:
git clone git@github.com:wpsandstorm/wordpress-sandstorm.git

cd wordpress-sandstorm

./build-spk-in-vagrant.sh

Then inside VirtualBox VM window, where is username: vagrant password: vagrant
sudo apt install task-lxde-desktop

startx

Then on Firefox ESR http://local.sandstorm.io:6080

Related https://github.com/sandstorm-io/sandstorm/issues/3054

I guess it's about the new PHP7 version according to the error log. I tried to run and enter a grain to figure out what is wrong but I ran onto another issue (see https://github.com/sandstorm-io/vagrant-spk/issues/213).
I've realised that there are some php config files in the repo. There is one line which seems to be wrong for php 7, but it seems these config files are not used at all.
...** SANDSTORM SUPERVISOR: Starting up grain. Sandbox type: userns
[12-Mar-2018 06:48:59] NOTICE: [pool www] 'user' directive is ignored when FPM is not running as root
[12-Mar-2018 06:48:59] NOTICE: [pool www] 'group' directive is ignored when FPM is not running as root
[12-Mar-2018 06:48:59] ERROR: unable to bind listening socket for address '/run/php/php7.0-fpm.sock': No such file or directory (2)
[12-Mar-2018 06:48:59] ERROR: FPM initialization failed
started php-fpm. status code: 70
started nginx. status code: 0

@JamborJan
Yes, it did not work, it caused error about symlink to symlink, or something like that.

@JamborJan
I removed that symlink from repo.

We are able to build the app again. For details see https://github.com/wpsandstorm/wordpress-sandstorm/commit/fec80241dd53a8e70074b2706835e1bdbaeb62d9
Is there anything left we need to clean up or can we close this issue @xet7 ?

I am super excited that progress has been made here!


Old state 2 - test lint worflow

Codecov Report

Merging #232 (6dc2235) into master (33e495b) will decrease coverage by 2.54%.
The diff coverage is 66.36%.

@@            Coverage Diff             @@
##           master     #232      +/-   ##
==========================================
- Coverage   93.31%   90.76%   -2.55%
==========================================
  Files          35       37       +2
  Lines         942     1040      +98
==========================================
+ Hits          879      944      +65
- Misses         63       96      +33

Impacted Files
Coverage Δ

server/audit/migrations/0001_initial.py
100.00% <ø> (ø)

server/audit_template/migrations/0001_initial.py
100.00% <ø> (ø)

server/inventory_item/migrations/0001_initial.py
100.00% <ø> (ø)

server/organization/migrations/0001_initial.py
100.00% <ø> (ø)

server/user_account/migrations/0001_initial.py
100.00% <ø> (ø)

server/audit/utils.py
20.00% <20.00%> (ø)

server/inventory_item/updater.py
78.31% <73.91%> (+3.31%)
:arrow_up:

server/audit_template/views.py
97.61% <92.85%> (-2.39%)
:arrow_down:

server/audit/migrations/0002_audit_template_id.py
100.00% <100.00%> (ø)

server/audit/models.py
100.00% <100.00%> (ø)

... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 33e495b...6dc2235. Read the comment docs.

Codecov Report

Merging #232 (6dc2235) into master (33e495b) will decrease coverage by 2.54%.
The diff coverage is 66.36%.

@@            Coverage Diff             @@
##           master     #232      +/-   ##
==========================================
- Coverage   93.31%   90.76%   -2.55%
==========================================
  Files          35       37       +2
  Lines         942     1040      +98
==========================================
+ Hits          879      944      +65
- Misses         63       96      +33

Impacted Files
Coverage Δ

server/audit/migrations/0001_initial.py
100.00% <ø> (ø)

server/audit_template/migrations/0001_initial.py
100.00% <ø> (ø)

server/inventory_item/migrations/0001_initial.py
100.00% <ø> (ø)

server/organization/migrations/0001_initial.py
100.00% <ø> (ø)

server/user_account/migrations/0001_initial.py
100.00% <ø> (ø)

server/audit/utils.py
20.00% <20.00%> (ø)

server/inventory_item/updater.py
78.31% <73.91%> (+3.31%)
:arrow_up:

server/audit_template/views.py
97.61% <92.85%> (-2.39%)
:arrow_down:

server/audit/migrations/0002_audit_template_id.py
100.00% <100.00%> (ø)

server/audit/models.py
100.00% <100.00%> (ø)

... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 33e495b...6dc2235. Read the comment docs.


Land-sea masking takes 3 times longer to run

When running a recipe with land-sea masking, I find that it can take 3 (and sometimes even up to 5) times longer to run compared to without a mask. Is this normal?
I'm running with ESMValTool v2.1.1.

@hahsan1 could you please post the recipe you using? Masking a rather time-consuming operation so am not surprised it takes longer, probably not 3-5x more though, I'll go straight and try answer the other issue you opened, my apologies, I was away :+1:

I have attached two recipes. The global_diff_shp-10p-red has no mask applied to it and it took 0:17:53 to run. The other recipe is land_diff_shp-10p-red and it masks out the sea. The time it took to run was 0:56:12.
global_diff_shp-10p-red.txt
land_diff_shp-10p-red.txt

cheers! I'll have a look today :beer:

hi @hahsan1 I tried running your recipes but that data can not be found neither locally on the ESGF node on CEDA/JASMIN nor on any other remote ESGF node to be downloaded - is that your locally produced model? If you could supply me with a used case that shows the massive cliff in performance due to masking, using readily-available ESGF data, that would be great! Also note that the difference in timing might be due to the diagnostic taking longer for one case; also if the data is very high resolution then yes, you'd expect a significant drop in performace when doing masking (especially with shapefiles) since the number of operations per dataset increases by quite a bit when masking

I was able to reproduce it with data that can be found on ESGF, but on a smaller scale. So the global_test.yml recipe has no masking and it took 0:02:41, and the land_test.yml has sea masking and took 0:07:04, so about 2.5 times.
land_test.txt
global_test.txt
I installed the files using the wget command, as attached here:
test_files.txt
It may be, as you said, just the increased number of operations with masking. In that case, this is just an observation and no course of action is needed to be taken to remedy it.

We managed to work around the longer processing time by running recipes in parallel. This seems to work for us.

@hahsan1 excellent, cheers for the heads up! What are you guys working on with ESMValTool as a fork? it'd be nice if you dropped a brief description of the project either as an issue or in the Discussion section - that way we'd know more of your use of ESMValTool and we could offer assistance/pointers etc. Also we could add a link to the documentation for other users to be aware of your project (if it's not private) :beer:

We are working on a project called Emissions-MIP which focusses on evaluating the sensitivity of models to various emissions characteristics. We are still in the stage of verifying our experimental protocol, and we've essentially been using ESMValTool for all of our analysis. You and your team have been extremely helpful and we thank you for that!
We presented preliminary results at the AeroCOM meeting last month! (Our abstract is the second one down if you're interested)

awesome, excellent work and very happy we could (and will, in the future) help! Cheers, mate, would you be OK if I quoted your project summary (in the comment above) in our communications? I am thinking we will gather some basic info on projects/teams using ESMValTool for pride&joy on either the website esmvaltool.org or in the intro in the documentation :beer:

Yes that would be great, thanks!


Next.js errors less useful in Solito than vanilla Next.js? (Solito Source Map)

I'm working on a Solito project, and I observed that errors in the UI seem to be the compiled jsx code instead of the TypeScript raw code.
That makes it a little harder to debug than I'm used to when working with Next.js.
Is this an inherent limitation of cross-platform development with React Native, or am I missing something obvious?
Screenshot

Note the var ... instead of the actual TSX code, which looks like:
const {status, data, error} = useQuery( // ...

This is certainly not a solito-related issue. This would be related to Next.js, possibly if you're prebuilding your pages or something.


[OFO/flagd] Research and document flagd client support

With client spec changes and SDKs on the way, we need to decide what sort of support we want for the client/web in OFO (if any). We already have the required bulk-resolution functionality in flagd, as well as CORs options to support local development
@james-milligan already did some research here with a PoC deployment, I believe. That may feed into this discussion.
If we were to support this, we'd want enhancements to the operator that could facilitate the following:

Flagd must be exposed via a public ingress
Flagd must support a bi-directional HTTP2 connections
A client must be able to access flagd
Flagd must be horizontally scalable based on load or number of connections

Non functionally, we should consider:

updates to the threat model
load considerations: How many persistent connection can be supported?

POC
I have put together a POC using kubernetes gateway API (KGA) as means to ingest external traffic (intended as successor to an ingress).
The deployment pattern this POC is catered to:

Deployment of an application exposed by a Service
Gateway exposes the app's Service via a HTTPRoute and listener (with hostname)

Assuming that the above is true (application is readily deployed and ready to receive external traffic), one can create a ClientSideConfiguration CR (naming TBD).
The creation of a ClientSideConfiguration CR triggers its reconciliation loop which does the following:

Creates a deployment of flagd
Creates a service exposing the flagd deployment
Appends a listener to the Gateway resource (name, port defined by the CR)
Creates a HTTPRoute exposing the flagd deployment service to the gateway (hostname defined by the CR)

The end result is two exposed applications: the client application and flagd. Both are exposed by the Gateway, which routes the request to the appropriate service based on the matching of the Host header (this is checked against the hostname defined in the Gateway's listeners).
Note: To get this to work locally I had to add hostnames to my hosts file to route them to localhost. Without this the browser doesn't send the appropriate Host header to the Gateway, preventing it from being able to route the request.
Questions/Concerns

KGA and its implementors are currently in beta.
Can KGA and Ingress coexist or do we enforce anyone who wishes to use OFO to deploy flagd to use KGA?
Do we want to support creating a Gateway if it doesn't already exist or just return an error?
How can local development be improved to not require adding hostnames to hosts file?
Does KGA support bi-directional HTTP2 connections?
Can scalability be demonstrated?

Further Steps

Address the above questions/concerns
Decide on naming of the ClientSideConfiguration CR
Move shared code into their own packages and import to existing invokers (a lot of duplicated code for POC)
Flesh out POC to include other sync provider deployments (currently only kubernetes provider is available)

g that the above is true (application is readily deployed

Hi, it's me again, playing devil's advocate here.
What's the need to connect to the client side to flagd, what do we gain doing this over using the client side SDKs elsewhere?
If you think about the story behind this; someone is going to be using a browser-based experience but the flags are completely unexposed and only accessible through a custom resource; do you have a use case in mind for this? Are we thinking of also having a Web API accessible through the gateway to program flags?

What's the need to connect to the client side to flagd, what do we gain doing this over using the client side SDKs elsewhere?

I'm not sure I'm following, are you suggesting that the client connects to flagd indirectly via some existing server?

someone is going to be using a browser-based experience but the flags are completely unexposed and only accessible through a custom resource

The flags are exposed in the demonstrated POC, or are you referring to the mutation of them being unexposed?

do you have a use case in mind for this?

It could enable user interfaces to react dynamically to user context based on flags by going straight to the flag source (flagd).

Are we thinking of also having a Web API accessible through the gateway to program flags?

To alter flagd's internal state you mean? Interesting idea but I'd argue it's out of scope for the topic at hand

I am querying whether there is a desire to have flagD as a provider the client SDK's can connect to is the goal.
I was curious whether there are people interested in this pattern.
For example, the in K8s pattern makes sense when its workloads a backend dev might be controlling or a website you're running from a pod. It gets a bit more complex when you're an iOS developer that hits a KGA endpoint for flagD as a provider. That's where I'm trying to really identify what we're talking about.

Just so I'm clear, are you suggesting that there might not be enough value in exposing flagd for use by client side applications? For those use-cases, defer to a vendor built for this?

I am querying whether there is a desire to have flagD as a provider the client SDK's can connect to is the goal.

There's a definite interest internally at Dynatrace.
Anecdotally, in the talks I've been party to with @moredip doing interviews with vendors, we typically hear that client use-cases represent about half of the usage most vendors see.

See this OFEP comment for the current plan for flagd client support.

Addressed by https://github.com/open-feature/open-feature-operator/pull/633


Retire the v3 label?

@leehambley Seeing as every issue is v3 by default these days, any objection to me deleting the v3 GitHub label?
We can keep the v2 label around for the rare issue that targets 2.x.
Also, in an effort to clean up the labels, I would like to delete these as well if you have no objections:

breaking-change (assigned to 1 open issue)
documentation (2)
duplicate (0)
needs-rebase (2)
under-consideration (15)

Nearly all the tagged issues haven't seen activity in a year or more, so the labels probably aren't very meaningful at this point.

Agreed - let's kill it, all the labels if you like. I preferred your suggested scheme anyway.


Auto-generate WAFv2

Here is a first pass at auto-generating WAFv2. Wanted to get some input on this change as there were quite a few backward compatibility changes needed along with some removed properties and changes to required fields.
@dalgarin @jerry153fish @axelpavageau @marksidell

@sid21rane LabelMatchStatement is included in this PR via StatementOne, StatementTwo and StatementThree properties. These are used due to the circular definitions in WAFv2 and are implemented per AWS::WAFv2::RuleGroup Statement.

Thanks Mark,
Please can I request for LabelMatchStatement under StatementOne, Two and
Three for AWS::WAFv2::WebACL as well.
Regards,
Siddharth Rane
On Thu, 30 Dec 2021, 21:10 Mark Peek, @.***> wrote:

@sid21rane https://github.com/sid21rane LabelMatchStatement is included
in this PR via StatementOne, StatementTwo and StatementThree properties.
These are used due to the circular definitions in WAFv2 and are implemented
per AWS::WAFv2::RuleGroup Statement
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-wafv2-webacl-statement.html
.
—
Reply to this email directly, view it on GitHub
https://github.com/cloudtools/troposphere/pull/1996#issuecomment-1003080953,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AXCXQNNPET3Y4HUFCCSKIWTUTR4NDANCNFSM5K6UCCHA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID:
@.***>

@sid21rane yes, as I commented above LabelMatchStatement is already in each of those classes for this PR.

Oh, sorry, misunderstood your request. WebACL and RuleGroup use a common set of Statements so it is already there. There are additions to the WebACL version which I took care of this morning.

Great then. Thanks Mark. When can we expect the PR to go live?
Regards,
Siddharth Rane
On Thu, 30 Dec 2021, 23:58 Mark Peek, @.***> wrote:

@sid21rane https://github.com/sid21rane yes, as I commented above
LabelMatchStatement is already in each of those classes for this PR.
—
Reply to this email directly, view it on GitHub
https://github.com/cloudtools/troposphere/pull/1996#issuecomment-1003137843,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AXCXQNKCSW6QSMCLD23JQUDUTSQE5ANCNFSM5K6UCCHA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID:
@.***>

Thanks for the changes Mark, can see the attribute now on the doc.
Appreciate it.
regards,
Siddharth Rane
On Fri, Dec 31, 2021 at 12:52 AM Mark Peek @.***> wrote:

Merged #1996 https://github.com/cloudtools/troposphere/pull/1996 into
main.
—
Reply to this email directly, view it on GitHub
https://github.com/cloudtools/troposphere/pull/1996#event-5828911403,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AXCXQNNCC7BTDBRH4Q2FFGTUTSWOZANCNFSM5K6UCCHA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID:
@.***>


Not auto redirect to correct sitemap URL

Describe the bug
When using CLI with a domain only, ex. warmup domain.com, the Warmer did not redirect to correct sitemap URL: https://domain.com/sitemap_index.xml
To Reproduce
Steps to reproduce the behavior:
warmup domain.com/sitemap404.xml
or
warmup notexistsdomain.com
Expected behavior
Automatically redirect to https://domain.com/sitemap_index.xml
Logs
/usr/local/lib/node_modules/datuan-sitemap-warmer/node_modules/sitemap-xml-parser/lib/sitemap.js:106
resolve(body.toString());
^

TypeError: Cannot read property ‘toString’ of undefined
at Request._callback (/usr/local/lib/node_modules/datuan-sitemap-warmer/node_modules/sitemap-xml-parser/lib/sitemap.js:106:34)
at self.callback (/usr/local/lib/node_modules/datuan-sitemap-warmer/node_modules/request/request.js:185:22)
at Request.emit (events.js:198:13)
at Request.onRequestError (/usr/local/lib/node_modules/datuan-sitemap-warmer/node_modules/request/request.js:877:8)
at ClientRequest.emit (events.js:198:13)
at Socket.socketErrorListener (_http_client.js:401:9)
at Socket.emit (events.js:198:13)
at emitErrorNT (internal/streams/destroy.js:91:8)
at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)
at process._tickCallback (internal/process/next_tick.js:63:19)```

The issue due to https://github.com/shinkawax/sitemap-xml-parser lib did not catch the error (resolve(body.toString());) when the domain is not exist or sitemap URL is 404.
I think I will do a pre-check before sending to parser, then rewrite it later if need to. It's 2 year old code already.

The issue due to https://github.com/shinkawax/sitemap-xml-parser lib did not catch the error (resolve(body.toString());) when the domain is not exist or sitemap URL is 404.
I think I will do a pre-check before sending to parser, then rewrite it later if need to. It's 2 year old code already.

Fixed in 7c873565f65762a34fc4eb534721b75fe7d23eb1

Fixed in 7c873565f65762a34fc4eb534721b75fe7d23eb1


Cluster (on Tutum)

This image is easy to setup and use as a single node, but now I am trying to setup a simple two nodes (using Tutum) cluster without success.
Here is what I am doing.
Here is the stack file:
rabbitmq-master:
  image: tutum/rabbitmq
  environment:
    - RABBITMQ_PASS=admin

rabbitmq-slave:
  image: tutum/rabbitmq
  environment:
    - RABBITMQ_PASS=admin
    - CLUSTER_WITH=rabbitmq-master
  links:
    - rabbitmq-master

Here is the logs when I start the rabbitmq-master service:
[rabbitmq-master-1] 2015-04-04T01:07:09.108931477Z => Securing RabbitMQ with a preset password
[rabbitmq-master-1] 2015-04-04T01:07:09.115997647Z => Done!
[rabbitmq-master-1] 2015-04-04T01:07:09.117667468Z ========================================================================
[rabbitmq-master-1] 2015-04-04T01:07:09.117805754Z You can now connect to this RabbitMQ server using, for example:
[rabbitmq-master-1] 2015-04-04T01:07:09.117954901Z
[rabbitmq-master-1] 2015-04-04T01:07:09.119918215Z     curl --user admin:admin http://<host>:<port>/api/vhosts
[rabbitmq-master-1] 2015-04-04T01:07:09.120031422Z
[rabbitmq-master-1] 2015-04-04T01:07:09.120147312Z Please remember to change the above password as soon as possible!
[rabbitmq-master-1] 2015-04-04T01:07:09.120285922Z ========================================================================
[rabbitmq-master-1] 2015-04-04T01:07:10.515347752Z
[rabbitmq-master-1] 2015-04-04T01:07:10.515347752Z               RabbitMQ 3.5.0. Copyright (C) 2007-2014 GoPivotal, Inc.
[rabbitmq-master-1] 2015-04-04T01:07:10.515347752Z   ##  ##      Licensed under the MPL.  See http://www.rabbitmq.com/
[rabbitmq-master-1] 2015-04-04T01:07:10.515347752Z   ##  ##
[rabbitmq-master-1] 2015-04-04T01:07:10.515347752Z   ##########  Logs<EMAIL_ADDRESS>[rabbitmq-master-1] 2015-04-04T01:07:10.515347752Z   ######  ##<EMAIL_ADDRESS>[rabbitmq-master-1] 2015-04-04T01:07:10.515347752Z   ##########
[rabbitmq-master-1] 2015-04-04T01:07:11.427650638Z               Starting broker... completed with 6 plugins.

At this point I can link other containers to rabbitmq-master, and it works.
Now, here is the logs when I start the rabbitmq-slave service:
[rabbitmq-slave-1] 2015-04-04T01:08:27.969397065Z => Securing RabbitMQ with a preset password
[rabbitmq-slave-1] 2015-04-04T01:08:27.974141351Z => Done!
[rabbitmq-slave-1] 2015-04-04T01:08:27.974933700Z ========================================================================
[rabbitmq-slave-1] 2015-04-04T01:08:27.974999455Z You can now connect to this RabbitMQ server using, for example:
[rabbitmq-slave-1] 2015-04-04T01:08:27.975031083Z
[rabbitmq-slave-1] 2015-04-04T01:08:27.975097215Z     curl --user admin:admin http://<host>:<port>/api/vhosts
[rabbitmq-slave-1] 2015-04-04T01:08:27.975133605Z
[rabbitmq-slave-1] 2015-04-04T01:08:27.975176333Z Please remember to change the above password as soon as possible!
[rabbitmq-slave-1] 2015-04-04T01:08:27.975211085Z ========================================================================
[rabbitmq-slave-1] 2015-04-04T01:08:29.260962128Z
[rabbitmq-slave-1] 2015-04-04T01:08:29.260962128Z               RabbitMQ 3.5.0. Copyright (C) 2007-2014 GoPivotal, Inc.
[rabbitmq-slave-1] 2015-04-04T01:08:29.260962128Z   ##  ##      Licensed under the MPL.  See http://www.rabbitmq.com/
[rabbitmq-slave-1] 2015-04-04T01:08:29.260962128Z   ##  ##
[rabbitmq-slave-1] 2015-04-04T01:08:29.260962128Z   ##########  Logs<EMAIL_ADDRESS>[rabbitmq-slave-1] 2015-04-04T01:08:29.260962128Z   ######  ##<EMAIL_ADDRESS>[rabbitmq-slave-1] 2015-04-04T01:08:29.260962128Z   ##########
[rabbitmq-slave-1] 2015-04-04T01:08:30.240396506Z               Starting broker... completed with 6 plugins.
[rabbitmq-slave-1] 2015-04-04T01:08:38.477823523Z Stopping node 'rabbit@rabbitmq-slave-1' ...
[rabbitmq-slave-1] 2015-04-04T01:08:38.941879642Z Clustering node 'rabbit@rabbitmq-slave-1' with 'rabbit@rabbitmq-master' ...
[rabbitmq-slave-1] 2015-04-04T01:08:39.087411150Z Error: unable to connect to nodes ['rabbit@rabbitmq-master']: nodedown
[rabbitmq-slave-1] 2015-04-04T01:08:39.211067548Z
[rabbitmq-slave-1] 2015-04-04T01:08:39.211067548Z DIAGNOSTICS
[rabbitmq-slave-1] 2015-04-04T01:08:39.211067548Z ===========
[rabbitmq-slave-1] 2015-04-04T01:08:39.211067548Z
[rabbitmq-slave-1] 2015-04-04T01:08:39.211067548Z attempted to contact: ['rabbit@rabbitmq-master']
[rabbitmq-slave-1] 2015-04-04T01:08:39.211067548Z
[rabbitmq-slave-1] 2015-04-04T01:08:39.211067548Z rabbit@rabbitmq-master:
[rabbitmq-slave-1] 2015-04-04T01:08:39.211067548Z   * connected to epmd (port 4369) on rabbitmq-master
[rabbitmq-slave-1] 2015-04-04T01:08:39.211067548Z   * epmd reports node 'rabbit' running on port 25672
[rabbitmq-slave-1] 2015-04-04T01:08:39.211067548Z   * TCP connection succeeded but Erlang distribution failed
[rabbitmq-slave-1] 2015-04-04T01:08:39.211067548Z   * suggestion: hostname mismatch?
[rabbitmq-slave-1] 2015-04-04T01:08:39.211067548Z   * suggestion: is the cookie set correctly?
[rabbitmq-slave-1] 2015-04-04T01:08:39.211067548Z
[rabbitmq-slave-1] 2015-04-04T01:08:39.211067548Z current node details:
[rabbitmq-slave-1] 2015-04-04T01:08:39.211067548Z - node name: 'rabbitmq-cli-154@rabbitmq-slave-1'
[rabbitmq-slave-1] 2015-04-04T01:08:39.211067548Z - home dir: /var/lib/rabbitmq
[rabbitmq-slave-1] 2015-04-04T01:08:39.211067548Z - cookie hash: 5N9FZwsQ39vI73Pe0Lmgaw==
[rabbitmq-slave-1] 2015-04-04T01:08:39.211067548Z
[rabbitmq-slave-1] 2015-04-04T01:08:39.613897991Z Starting node 'rabbit@rabbitmq-slave-1' ...
[rabbitmq-slave-1] 2015-04-04T01:08:39.838592650Z
[rabbitmq-slave-1] 2015-04-04T01:08:39.838592650Z               RabbitMQ 3.5.0. Copyright (C) 2007-2014 GoPivotal, Inc.
[rabbitmq-slave-1] 2015-04-04T01:08:39.838592650Z   ##  ##      Licensed under the MPL.  See http://www.rabbitmq.com/
[rabbitmq-slave-1] 2015-04-04T01:08:39.838592650Z   ##  ##
[rabbitmq-slave-1] 2015-04-04T01:08:39.838592650Z   ##########  Logs<EMAIL_ADDRESS>[rabbitmq-slave-1] 2015-04-04T01:08:39.838592650Z   ######  ##<EMAIL_ADDRESS>[rabbitmq-slave-1] 2015-04-04T01:08:39.838592650Z   ##########
[rabbitmq-slave-1] 2015-04-04T01:08:40.061849849Z               Starting broker... completed with 6 plugins.
[rabbitmq-slave-1] 2015-04-04T01:08:40.169205690Z /usr/sbin/rabbitmq-server

At this point I can link other containers to rabbitmq-master and rabbitmq-slave, and they works, but they are independents, they do not replicate queues.
Here is the parts that bugs me:
In the rabbitmq-slave-1 container, when it stops and try to join the cluster, it clearly says that it is connected to epmd (port 4369) on rabbitmq-master and also that TCP connection succeeded, but Erlang distribution failed, and it gives me those suggestions:

suggestion: hostname mismatch?
suggestion: is the cookie set correctly?

The suggestion is the cookie set correctly?: this is obviously the case since the cookie is hardcoded in the container.
So, what about the suggestion: hostname mismatch? ?
Indeed, in the rabbitmq-slave service environment: I setup that it should - CLUSTER_WITH=rabbitmq-master. But when the rabbitmq-master service starts, the single container created is called rabbitmq-master-1. I guess this is where the hostname mismatch, but what can I do about it ?
Do I need to hardcode that the slave should - CLUSTER_WITH=rabbitmq-master-1 ?

Ok, I should have tried harder before posting this issue.
My little two nodes cluster is now working using the hack I previously mentioned: I tel the slave to CLUSTER_WITH=rabbitmq-master-1. Also, I exposed all the ports needed for the cluster, but I guess I did not needed them.
For the records, here is my updated and working stack file:
rabbitmq-master:
  image: tutum/rabbitmq
  environment:
    - RABBITMQ_PASS=admin
  ports:
    - "15672:15672"
  expose:
    - "5672"
    - "35197"
    - "4369"
    - "25672"

rabbitmq-slave:
  image: tutum/rabbitmq
  environment:
    - RABBITMQ_PASS=admin
    - CLUSTER_WITH=rabbitmq-master-1
  ports:
    - "15672:15672"
  expose:
    - "5672"
    - "35197"
    - "4369"
    - "25672"
  links:
    - rabbitmq-master

Hi @pierredavidbelanger
Thank you for posting this.
I think set the HOSTNAME on rabbitmq-master could solve the hostname mismatch error:
rabbitmq-master:
  image: 'tutum/rabbitmq:latest'
  environment:
    - HOSTNAME=rabbitmq-master
rabbitmq-slave:
  image: 'tutum/rabbitmq:latest'
  environment:
    - CLUSTER_WITH=rabbitmq-master
  links:
    - 'rabbitmq-master:rabbitmq-master'

As the container rabbitmq-master-1 could be changed if you scale up/down the rabbitmq-master service, I think using HOSTNAME is better.
Let me know if works for you, and then I can added this stack file to README.
Thank you

Thank you @tifayuki .
This is also working.
But now my problem is when rabbitmq-master is restarted, it is not able to resolve rabbitmq-slave.
And I can not link from rabbitmq-master to rabbitmq-slave because it creates a circular reference.
I also tried to cluster RabbitMQ with only one service:
rabbitmq:
  image: 'tutum/rabbitmq:latest'
  environment:
    - CLUSTER_WITH=rabbitmq-1
  sequential_deployment: true

I was told that with sequential_deployment set to true, subsequents containers are able to resolve the previous ones. So this cluster is also working when I scale up this service. (I have only one warning when the first node is started, it says that it cannot cluster with itself, though this can be safely ignored).
But the problem remains, when the first (or a previous) node do down, it will not be able to resolve other (or a subsequent) nodes.
This is not a problem from this image though, this is a Tutum issue.
You can close this issue.
Thank you.

We are changing DNS to allow services/containers within the same stack to resolve each other, and this could solve your problem I think.
Thank you

It will indeed solve the problem, and more!
This will be a great enhancement for Tutum users.
I am eager to test this :)


Jetpack Compose color variables export does not include latest Material 3 colors

Jetpack Compose color variables export does not include latest Material 3 colors:
surfaceBright
surfaceContainer
surfaceContainerHigh
surfaceContainerHighest
surfaceContainerLow
surfaceContainerLowest
surfaceDim

Same for xml too.
I guess they'll be added in the future as stated here.

They are now available!


Unified wording on titles

Unified wording on titles
NVSHAS-8016: (UI) Add a new operator in Policy > Admission Control > …


PyFunceble-docs-3

PyFunceble-docs-3 DOC: components

Fixed missing --syntax in "known issues"
Fixed bad anchor link in install & update #pypi-python-package-index
Added docs/use/components/.*
Fixed TOC in README.md + typos

Reviewed-by: @spirillen
Issue draft https://github.com/foreign-affairs/PyFunceble-docs/issues/3

By the import of the component, this isn't a "done" job, as there clearly are room for optimizations in the text and layout. But this will republish a huge section of missing information's to the public


Update sign script to work for track 1 libraries

eng/scripts/SignAndHash-MavenPackages.ps1 was making assumptions that libraries would have additional artifacts. For Client/Track 2 libraries this artifact would be the changelog, but for Data/Track 1 artifacts this doesn't have to be the case. Most Track 1 libraries don't have to have changelogs. For Track 2/Client libraries, the existence of changelog is verified before we even try to publish but for Track 1/Data libraries, there is no such check because they're not required.

/azp run java - template


Attached the links properly and coloured the icon as you specified

Issue  📐

I  have worked for

Guidelines 🔐
I accept the fact that i have followed the guidelines and have not copied the codes from around the internet

[x] Contribution Guidelines
[x] Code of Conduct

Issue to be closed 🛅

My pull request closes

Screenshots 📷
Here are the pictures of changes that i have made 🔽

Links have been attached properly  for twitter and removed facebook , github links as you mentioned.

After clicking on twitter icon it directs us to mentioned twitter handle

@IAmTamal  Please review the changes and accept by PR

@DhruvRathi2001 Please mention the issue number !

Hey @IAmTamal  , I have fixed the issue #346 now you can merge it.


Updates to work with 0.18 and added new function values

I did some updates to work with elm 0.18 and added one function.

Changed module declaration syntax
Removed map[2-5] function since they are in the elm-core since version 3
Run elm-format
Added values function

Hi @txgruppi, thanks for the PR but https://github.com/elm-community/maybe-extra is the new home for this package. You should submit PRs over there.


[Dashboard] Tooltips disappear on loading dashboards

In a dashboard, if you mouse over a chart while other charts on the same tab are loading, the tooltip is visible. However, once at least one other chart loads, mousing over the original chart no longer displays the tooltip, and it won't come back unless you reload the dashboard, force refresh the chart, or inspect element.
Expected results
Tooltip appears:

Actual results
Tooltip does not appear:

Screenshots
If applicable, add screenshots to help explain your problem.
How to reproduce the bug

Open any superset Dashboard (preferably with lots of charts to load)
While charts are loading, mouse over an "already loaded" chart to engage the tooltips
Once another chart has loaded, the tooltip for the "first-loaded" chart will no longer be visible.

Environment
(please complete the following information):

superset version: superset version: up-to-date as of 2021-04-02
python version: python --version: 3.7
node.js version: node -v: v14.16.0

Checklist
Make sure to follow these steps before submitting your issue - thank you!

[X] I have checked the superset logs for python stacktraces and included it here as text if there are any.
[X] I have reproduced the issue with at least the latest released version of superset.
[X] I have checked the issue tracker for the same issue and I haven't found one similar.

@serenajiang thanks for reporting! which charts are you seeing this issue? I assume it's NVD3 line?

We also face this issue with line charts.

Same here. Happens at least for line charts and time-series bar charts (currently only have these two types).

This is resolved in master now!


Subscribe to the Low-Code / No-Code Service in SAP BTP

Tutorials: https://developers.sap.com/tutorials/appgyver-subscribe-service.html
Hi,
the prerequisites for the tutorial: a BTP account in EU10 ist no longer possible with a Trial account, as announced here:
https://help.sap.com/doc/43b304f99a8145809c78f292bfc0bc58/Cloud/en-US/98bf747111574187a7c76f8ced51cfeb.html?from=2021-10-14&to=2021-10-14&search=SAP%2520BTP%2520Trial%2520Subaccounts%2520on%2520eu10
Only cf-us10 and cf-ap21 are possible. So, how can we test appgyver in the BTP trial account now?
Thank you, René

Hello - unfortunately the AppGyver service is not available with a trial account, it is a paid service. To test the same features and complete the other tutorials in this mission, you can sign up for free at www.appgyver.com. The color scheme and button locations change at times, but the steps should still be possible.


Make MessageParser mIRC Compatible

Not sure why, \r\n isnt the character pattern used by common irc clients. Everything seems to work after changing this.

.replace('\r\n', '\n')
.split('\n')


Post release automated changes for storagesync releases

Post release automated changes for azure-arm-storagesync

API change check
API changes are not detected in this pull request.


Fix sizing modes

It's hacky as hell but I think it works now. This PR:

Fixes "original" sizing (which I renamed). This sizing will display the original page size, as-is.
Fixes fit-height and fit-width sizing to have a scrollbar.
Adds "fit-both" sizing.

Closes: #18.

If someone can test it, especially on widescreens and/or phone-sizes, that would be great.

Not sure if it's just chrome's dev tool being weird, but everything seems to break when I'm testing out vertical monitors viewport (break as in like pages not being displayed at all in long strip). It works fine if I just use my normal viewport tho.

Fit Both

Fit Height

Hmm... it looks fine for me, not sure why:

Everything breaks now even in normal viewport for me, I'm getting this warning

Hmmm... I can't replicate it but I can guess where that's coming from.

No idea if that fixes it as I can't replicate.

Weird, guess we can ask if it's fine for the others. I'll also get the new page count thingy up in a bit

Added two possible fixes to where I suspect it might be being caused, lmk if it changes anything?

It's still the same, the problem only occurs for fit both and original size, also got this in the console

Original size

Fit both

Uhhh... that doesn't look correct. You're sure you're on the latest commit, right? As that was an error I had way back.

I think so? You added this in the latest commit right?

Hm, yeah, that's correct then. I can't reproduce what you're seeing at all though, and there's nowhere off the top of my head where I modify the width xor the height anymore...?

Can you give me the exact window size I guess?

1536 x 729.6px

Hmmm... ah, I have another idea where an issue might be regarding loading, let me try changing that...

Just as a check - do you have caching disabled when you open the dev console?

Yup

That explains it, it's doing some wonky stuff with how things are loading. I can try and fix that, but if you disable it, does it still cause problems?

Disabling the cache fixes it

Cool, and try re-enabling it with the newest commit?

Nice, it's working fine now

Fix for #34

Seems like that doesn't work for the end pages, though first page works fine now.

(resolution is a 1440 x 2560)

Trying a different method, but before that, I don't think my part messed with anything but everytime I refresh the page I have a never ending loading icon on the screen now

Trying a different method, but before that, I don't think my part messed with anything but everytime I refresh the page I have a never ending loading icon on the screen now

happens for fit width, long strip,

Ugh, I know why - it's because the loading icon for other pages is not being moved correctly.

K I think that fixes it, though hopefully not breaking other things in the process...

Okay I tested both fixes and I think we're fine, so I'm going to merge this branch for now since it's getting long.


Disconnection problem

when i run the example locally I can connect to keycloak without problem but when i want to disconnect, it redirect me to http://localhost:8080/camunda/app/welcome/default/#!/login
but I am still connected and there is no way to disconnect me.

For logout see https://github.com/camunda/camunda-bpm-identity-keycloak/issues/1 - this might help you.

Question is connected to the SSO-example. Not to the plugin itself.

I will provide the logout functionality integrated within the example shortly

Solved. See examples/sso-kubernetes.

Added Keycloak Logout Handler
Added custom logout module to Cockpit/Tasklist/Admin/Welcome app


Some feedback for SCM

I understand it's beneficial for VSCode to abstract away SCM to allow multiple source control systems.
However, my initial reaction was that it's taken away some of the functionality I had with the Git module, which was awesome.
Is there a flag or something we can change to bring back Git? I miss are being able to undo my commit.

Also one more thing I miss is that there used to be a button in the top right corner of the "diff" view that allowed you to switch to editor, and vice versa. It seems that button is missing now.

All the features will be preserved. You can track the progress here: #18615

We might introduce a temporary setting for switching between the two modes.


Merge back releases/1.3.0

Summary

1.3.0 not finalized, but merging intermediate state into develop

How to test

Checklist

[ ] I have added unit tests to cover my changes.
[ ] I have added integration tests to cover my changes.
[ ] I have added the description of my changes into CHANGELOG.
[ ] I have updated the documentation accordingly

License

[ ] I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.
[ ] I have updated the license header for each file (see an example below).

# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT

Codecov Report
Patch coverage: 85.22% and project coverage change: -0.03 :warning:

Comparison is base (565b95d) 78.59% compared to head (dbb72dd) 78.56%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1015      +/-   ##
===========================================
- Coverage    78.59%   78.56%   -0.03%
===========================================
  Files          237      237
  Lines        27046    27074      +28
  Branches      5396     5401       +5
===========================================
+ Hits         21256    21270      +14
- Misses        4526     4536      +10
- Partials      1264     1268       +4

Flag
Coverage Δ

macos-11_Python-3.8
?

ubuntu-20.04_Python-3.8
78.56% <85.22%> (-0.02%)
:arrow_down:

windows-2019_Python-3.8
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...ms/noisy_label_detection/loss_dynamics_analyzer.py
84.94% <66.66%> (-3.56%)
:arrow_down:

datumaro/components/visualizer.py
88.72% <75.00%> (-0.26%)
:arrow_down:

datumaro/plugins/transforms.py
75.31% <77.77%> (+0.02%)
:arrow_up:

datumaro/components/exporter.py
87.01% <100.00%> (+0.12%)
:arrow_up:

datumaro/plugins/data_formats/arrow/exporter.py
73.58% <100.00%> (-1.42%)
:arrow_down:

datumaro/plugins/data_formats/ava/ava.py
83.59% <100.00%> (ø)

datumaro/plugins/data_formats/camvid.py
87.14% <100.00%> (ø)

datumaro/plugins/data_formats/cifar.py
87.24% <100.00%> (ø)

datumaro/plugins/data_formats/cityscapes.py
88.54% <100.00%> (ø)

datumaro/plugins/data_formats/coco/exporter.py
85.74% <100.00%> (ø)

... and 26 more

... and 1 file with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

This PR should be merged by "Create a merge commit"


Fail to declare a typed.Dict inside numba.jit

Reporting a bug

[Yes] I have tried using the latest released version of Numba (most recent is
visible in the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
[Yes] I have included a self contained code sample to reproduce the problem.
i.e. it's possible to run as 'python bug.py'.

I want to create a dict inside jit function but gets
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Unknown attribute 'Tuple' of type Module(<module 'numba.core.types' from '/Users/benjaminchou/miniforge3/envs/tf2/lib/python3.8/site-packages/numba/core/types/__init__.py'>)

The code to reproduce is:
import numba
from numba import objmode, typed, types

@numba.jit(nopython=True)
def dict_test():
    d = typed.Dict.empty(
        key_type=types.Tuple((types.int64, types.int64)),
        value_type=types.int64,
    )
    elem = (1, 2)
    d[elem] = d.get(elem, 0) + 1
    return d

A workaround is:
import numba
from numba import objmode, typed, types

@numba.jit(nopython=True)
def dict_test():
    d = typed.Dict.empty(
        key_type=tuple((1, 1)),
        value_type=types.int64,
    )
    elem = (1, 2)
    d[elem] = d.get(elem, 0) + 1
    return d

Could anyone explain the correct way to use typed.Dict? So far, I don't find any detailed example in the docs.

Thanks you for reporting the issue. The dictionary support can be unintuitive sometimes because of limitations from type inference.
It does not work in the reproducer because Numba cannot handle the type spelling (types.Tuple((types.int64, types.int64))) in jit code. If you move that into a global, it will work. For example:
import numba
from numba import typed, types

key_type = types.Tuple((types.int64, types.int64))

@numba.jit(nopython=True)
def dict_test():
    d = typed.Dict.empty(
        key_type=key_type,  # this now references a global
        value_type=types.int64,
    )
    elem = (1, 2)
    d[elem] = d.get(elem, 0) + 1
    return d

In simpler cases, Numba can infer the type of the dict so you can spell it with the typical d = {}. But, your case here is a little bit challenging for type inference because d.get(elem, 0) is used before the d[elem] = ....
For example, the following case can be type inferred:
@numba.jit(nopython=True)
def dict_test():
    elem = (1, 2)
    d = {elem: 0}
    d[elem] = d.get(elem, 0) + 1
    return d

Lastly, I'm a bit surprised that the workaround spelling is accepted.
d = typed.Dict.empty(
        key_type=tuple((1, 1)),  # surprised this is accepted
        value_type=types.int64,
    )

Yes. I just find that python instances can be inferred for types in the jitted function. That's something not in the docs, which makes me quite confused. I am kind of lost in the usage of signatures. It's not a very consistent experience. As for your workaround moving it to global, it requires me to manage a global space for signatures, which is quite oblique.
I guess it's because most codes of types work in python-mode while not supported in non-python mode. I wonder if it is possible to provide types argument in jit to mimic the C++ template?

Yes, you can also pass types as parameters:
e.g.
import numba
from numba import typed, types

@numba.jit(nopython=True)
def dict_test(key_type):
    d = typed.Dict.empty(
        key_type=key_type,  # this is now from an arg
        value_type=types.int64,
    )
    elem = (1, 2)
    d[elem] = d.get(elem, 0) + 1
    return d

tup_type = types.Tuple((types.int64, types.int64))
r = dict_test(tup_type)
print(r)

Closing this now as the question has been answered. Please feel free to re-open in case the issue remains.