chore: move embark site into mono repo

@michaelsbradleyjr please double check if there's adjustments needed in lerna.json.
Right now, I think the most pragmatic step forward is to have lerna ideally ignore ./site.

I guess you put the site not inside packages/ because otherwise it would be deployed to npm or something?

Yes, at least we want to clearly differentiate between embark packages and the website. Probably easiest for now if lerna completely ignores ./site, waiting for @michaelsbradleyjr input on that.

@jrainville @michaelsbradleyjr addressed your comments:

Optimized some of the illustration assets which were couple MB. Please note that we still have big assets in this project. This is because we're using animated gifs for the documentation
Rewrote the deploy script in Node, added a check that the correct remote is configured
Added an deploy:site npm script to the root's package.json
Removed a lot of files within the site project that we don't use any more (patternlab stuff etc)
Re-added site/.gitignore as discussed.

@PascalPrecht @jrainville What if we setup a repo like embark-site-assets and could make its default branch gh-pages. Images and other bulky static things could be stored there and then we link to those assets from the site.
I don't want to overly complicate things, but hoping we can somehow avoid repo bloat with static assets.

What if we setup a repo like embark-site-assets and made its default branch gh-pages. Images and other bulky static things could be stored there and then we link to those assets from the site.

Certainly an option. Just comes with the side effect that, when running locally (and offline) those assets are missing unless the assets repo is setup and served locally as well.
This may or may not be a problem.
Is this something we want to decide before this PR gets merged? Or can we iterate and do that in another step

when running locally (and offline) those assets are missing unless the assets repo is setup and served locally as well.

when running locally (and offline) those assets are missing unless the assets repo is setup and served locally as well.

They would be available if we always use the remote url (eg embark.status.im/images/something.gif)
However, there still comes the problem that adding assets to the website is still quite a pain, because you need to add them to the other repo and publish.

Thanks for the suggestions @michaelsbradleyjr ! Updated accordingly :)


Fixed field declared dynamically

Thanks for your contribution, but I'm closing this; There's no hurry and it'll be fixed on the Threaded-Editing branch.


Run gradle functional task in pipeline

JIRA link (if applicable)
https://tools.hmcts.net/jira/browse/BPS-1247
Does this PR introduce a breaking change? (check one with "x")
[ ] Yes
[x] No

although it would be best to use built in gradle task


Can't find in docs, how do I define a top level namespace?  Also optional function args?

How do I define optional function arguments that include default values if not passed in?
Also, defining a top level namespace?

Top level namespaces are defined like the following:
(ns foo.bar)

(your-code-here)

Can you elaborate on what you mean by optional args? If you're looking for a way to do default argument values, it's like this:
(fn foo (arg1 [arg2 "default"])
  )

(foo "hello")

Notice the square brackets around the second argument. arg1 is required, but if no arg2 is provided it defaults to "default".

Thanks a ton @Scriptor - heavy user of PHP and Common Lisp, but it's been awhile since I did much/any clojure (not sure if that's what the default argument values are modeled after here or not).

Oh, one more question when you get a chance - I see the file generated by pharen requires an absolute path pharen file (based on where pharen was when the install.sh script was run).  Does the pharen binary support including a copy of this file (full of pharen functions) in a relative path?
For instance, it would be nice if the function file could be autoloaded or just composer required (so I don't have an absolute path reference in the php file generated from the phn file)

Also - does pharen have built in string access syntax at the moment?  Similar to using $string[0] or $string{0} to access the first character of a string stored in $string?

The default arguments is a custom syntax added for the sake of convenience. Clojure doesn't support default args, but since PHP does I figured it'd be nice to throw that in.
Unfortunately there's no way to use relative paths for files like lang.phn and the standard lib files. It simply wasn't something I considered when I first started working on this. Ideally it would be a simple Composer dependency. Hopefully I'll have time soon to fix the whole include path mess.
String access is done in the same what that array/list indexing is done:
(def s "foobar")
(:s 0) ; return 'f'


Should we get rid of (or archive) the issues-test repo

We still have the issues-test repo.
Can we just delete it? If not, maybe we should archive it.

We didn't see any reason to keep it so let's delete it.

We didn't see any reason to keep it so let's delete it.


Wishing well down?

Seems the link for the wishing well doc is down. I can't seem to find any alternatives for requests. Are those just no longer being taken?

Not really, no. Sorry!

Understood. Thanks for all of your hard work--your tool remains an invaluable resource, because god knows how long it's going to take Analogue to enable filters for openFPGA.

Understood. Thanks for all of your hard work--your tool remains an invaluable resource, because god knows how long it's going to take Analogue to enable filters for openFPGA.

I'm (@josejx) happy to take requests over at my github, if you'd like to make requests there.


Git adding a draft of a notebook for making ferrybox dataset.

…g extractor to extract all track not only good qc, upadting uuidmapper for nivaprod.

There is an error when storing the dataset (last cell in the notebook).
Anythink obvious @knl88 ?

In the second cell you have some dangling , so the values are interpreted as tuples.

I implemented manual QC to the notbook. Good to merge @knl88 ?


Old utxo simplification

remove protocol magic
properly verify signature using a new signature protocol and the old address witnessing

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@f35973a). Click here to learn what that means.
The diff coverage is 0%.

@@            Coverage Diff            @@
##             master     #543   +/-   ##
=========================================
  Coverage          ?   42.34%
=========================================
  Files             ?      181
  Lines             ?    15887
  Branches          ?        0
=========================================
  Hits              ?     6728
  Misses            ?     9159
  Partials          ?        0

Impacted Files
Coverage Δ

chain-impl-mockchain/src/transaction/witness.rs
62.85% <0%> (ø)

chain-impl-mockchain/src/legacy.rs
0% <0%> (ø)

chain-impl-mockchain/src/ledger.rs
0% <0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f35973a...ec0bf7c. Read the comment docs.


deno: fix CVE-2023-28446

Description of changes

Fix https://github.com/denoland/deno/security/advisories/GHSA-vq67-rp93-65qf on unstable with a patch for now until #221462 lands
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[x] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

cve details:
// Expected Prompt
┌ ⚠  Deno requests run access to "cat"
├ Requested by `Deno.Command().output()` API
├ Run again with --allow-run to bypass this prompt.
└ Allow? [y/n/A] (y = yes, allow; n = no, deny; A = allow all run permissions) >

// Actual Prompt
┌ ⚠  Deno requests run access to "echo"
├ Requested by `Deno.Command().output()` API
├ Run again with --allow-run to bypass this prompt.
└ Allow? [y/n/A] (y = yes, allow; n = no, deny; A = allow all run permissions) >

this build's output:
λ ./result/bin/deno run --unstable indeeex.ts
┌ ⚠  Deno requests run access to "cat".
├ Requested by `┌ ⚠  Deno requests run access to "echo"├ Requested by `Deno.Command().output()` API
├ Run again with --allow-run to bypass this prompt.
└ Allow? [y/n/A] (y = yes, allow; n = no, deny; A = allow all run permissions) >

Correctly lists cat for the first line. The Requested by section does still show the cheeky string that's trying to modify that first line which could cause some confusion but still good IMO. Will be an improvement from the current build & tide us over until #221462


Update 06-translate-text.md

The Endpoint as copied from Keys and Endpoint page has a "/" at the end by default so this broke my lab :)
Module: 00
Lab/Demo: 00
Fixes # .
Changes proposed in this pull request:

Changes here are already reflected in other PR. Closing this one in favor of the other


Upgrade aruba to 0.12.0

This hopefully fix the stylus issue. But let's see what's happening on travis. This is a fix for v3.
V4 is still on aruba 0.7.4. That needs some more changes.

Ok. Stylus now works with aruba 0.12.0.

Sweet

Good to merge then?

Yes. For v3 - this PR. Fortunately it's now working. Needed to fully rewrite the handler for ENV in aruba. Now all changes are recorded and applied on access - more or less.

To make it work for v4 i need to write a bit of code. Maybe I finish this within the next couple of hours. But now I'm going to have my evening meal.

And there are also some more nice changes to aruba in the pipeline. :-)

Heh, I know about long upgrade cycles :) Thanks!

No problem. Love to use middleman, so helping you with aruba is the best thing I can do to give back "some time". :-)

But we really need to talk about the article-helper :-P

Heh, will deal with helper.  I'll add that helper back :)

Thanks a lot! :-)

hen fight with nginx to install a new cert for the forum..

I switched back to apache. Although I like nginx's syntax apache has more functionality included/built in.


Directory.open error 31's when passed user:// path with /home being a symlink

Operating system or device - Godot version:
Athlon FX8320 (8-core) x64 16 GB RAM linux/X11 Godot v2.1.beta (HEAD at 6d5d23fa8de8673f4a2b312c05588fd4e7195543)
Issue description (what happened, and what was expected):
extends Node

func _ready():
    var loc="user://"
    var dir=Directory.new()
    var err=dir.open(loc)
    if err==OK:
        dir.list_dir_begin()
        var name=dir.get_next()
        while name!="":
            print(name)
            name=dir.get_next()
    else:
            print("Failed to list ",loc," due to: ",err)

gives output like:
Failed to list user:// due to 31
(ERR_INVALID_PARAMETER)
The output should perhaps be the contents at user://
If loc is changed to something like OS.get_data_dir() it starts working and properly outputs a listing at user:// (OS.get_data_dir() yields the absolute path of user://)
This could be rather troublesome in games that want to have multiple saves and provide a custom dialog to select a save game to continue from (since it'll get res:// on the above code rather than the saves) and wishes to peek into the saved games to show character, level, etc. like many console RPGs do.
Steps to reproduce:
see above.
Link to minimal example project (optional but very welcome):
Edit: I ended up making a small project to show the problem is not a project having no name: Project file link
That project is out of date.  Please see the above code.

PS: Please see #6016 it sounds like a related problem (Directory accessing something under user:// not working as expected)

PS: It doesn't matter where I try to specify on user:// (eg: user://some/save/dir) and it still outputs the contents at res:// instead!

did you set a project name? if not, then user:// will have the same place
as res://
On Fri, Aug 5, 2016 at 8:52 AM, gau-veldt<EMAIL_ADDRESS>wrote:

PS: It doesn't matter where I try to specify on user:// (eg:
user://some/save/dir) and it still outputs the contents at res:// instead!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/6047#issuecomment-237831515,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF-Z22Fi74Mg1uR4tbXidBdB7PLZRwliks5qcyPrgaJpZM4JdmqV
.

@reduz will that only affect Directory or would File be using res:// too in that case?  Since in my case File works properly and uses the dotdir off of ~ when specified user:// paths

PSS: Still does this with 6d5d23fa8de8673f4a2b312c05588fd4e7195543 (which is HEAD as of writing this).

Here's a small project to show the problem.  Application/name in Project Settings is "DirUserTest"
test.tar.gz
Output I get is:
** Debug Process Started **
Listing user://
  name=icon.png.flags
  name=icon.png
  name=engine.cfg
  name=.
  name=..
  name=test.gd
  name=test.tscn
** Debug Process Stopped **

I think while dir.open("user://")!=null: should be while dir.open("user://")==OK:
Directory.open returns Error enum.

extends Node

func _ready():
	var loc="user://"
	var dir=Directory.new()
	var err=dir.open(loc)
	if err==OK:
		dir.list_dir_begin()
		print("Listing ",loc)
		var name=dir.get_next()
		while name!="":
			print("  name=",name)
			name=dir.get_next()
	else:
		print("Didn't access ",loc," (",err,")")

** Debug Process Started **
Didn't access user:// (31)
** Debug Process Stopped **
31==ERR_INVALID_PARAMETER?!?
Do I need to use OS.shell("ls") to get a directory under user:// ?

Does get_data_dir return "res://", "user://" or something else?

@Sslaxx
dir.open("user://") gives me error 31
dir.open(OS.get_data_dir()) gives me the directory of "~/.godot/app-data/XXX/"  (where XXX is Application/name in the General tab of Project Settings) which is the absolute path of user://

This one and bug #6016 are among the last blocking bugs for 2.1 IMO. I'll have to check but I'm pretty sure they are regressions, and having an erratic user:// path would be pretty bad for the stable release.
It could be that those bugs are Linux X11 or Unix-specific, so if some of us can test on Windows and Mac OSX to confirm, it would be great.

Just tested on Windows 10, can confirm this to be a Linux related issue.

Assuming it affects OSX too as I think the bug is in drivers/unix.

I can't seem to reproduce, I wonder if there is anything else involved

Just tested on Mageia 6 x64 with a fresh build from the 2.1 branch, I can't reproduce either.

Same root issue as #6016 then, maybe?

System is Athlon FX8320 (8-core) 16-GB RAM
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cayman LE GL [FirePro V5900]
Operating system: X11 in Linux Mint 17.2 (etc/sources.list) #deb cdrom:[Linux Mint 17.2 _Rafaela_ - Release amd64 20150627]/ trusty contrib main non-free
git log -n 1 on my godot build reports: 6671c6bdc78864bbe1d27c508ec9528f3b683ca2
commit 6671c6bdc78864bbe1d27c508ec9528f3b683ca2
Author: Juan Linietsky<EMAIL_ADDRESS>Date:   Sat Aug 6 22:11:03 2016 -0300

    Added yield() signal smart autocompletion.

Fresh copy of the project I use to get the bug: test.tar.gz
Screenshot or it didn't happen (TM):
same deal as before if create a blank file in the userdir touch ~/.godot/app-data/DirUserTest/foo and change line 5 to var loc=OS.get_data_dir() this is the output:
** Debug Process Started **
Listing /home/gau/.godot/app_userdata/DirUserTest
  name=.
  name=..
  name=foo
** Debug Process Stopped **

What am I missing for folks to reproduce this?

Thanks for the screenshot and test project, that makes it very clear that there is a bug :) Sadly I still can't reproduce it on Mageia 6 x64, kernel 4.7.0.
Do you still reproduce the bug with the official 2.1 RC2 binaries and/or if you build the latest 2.1 branch?

Can't reproduce on Arch Linux :/
Listing user://
  name=.
  name=..

@akien-mga My last post was based on commit 6671c6bdc78864bbe1d27c508ec9528f3b683ca2 which is in master (2.2).
Changing branch to 2.1 and using 569966f3bbe82d768baf3fd26680462df22d29ad to scons-n-test:
I still get Didn't access user:// (31) when setting line 05 to var loc="user://" and lists the data dir (shows the foo I touched into the directory for testing to know I got that dir) properly when line 05 is var loc=OS.get_data_dir()

The only other thing I can think of is that my /home is a symlink (since you cannot hardlink to another drive).  The main / only has room for the OS (10 GB or so) so copied /home to the other drive with much more room (>100GB) and made a /home symlink.  Might be tripping up Directory.open("user://") since it accesses /home.  But it works with OS.get_data_dir() and File.open("user://") works as it should (or I'd be getting a TON of problems everywhere).

The symlink is likely to be the root of the issue yes. There might be a bug in DirAccess that makes it not resolve symlinks for user:// I guess.
Unrelated to the issue, but not that instead of making your /home a symlink, you could make it a real partition and mount it as /home. That's a common practice on Linux.

I'm using home as a partition BTW, and I can't reporo with them (as stated above). So it should be simlinks :smile: .

@akien-mga That drive already has data and no spare room to make a new partition to use for /home.

I've got the bug so I'll see if I can probe a bit dir_access_unix.cpp and see if I can backtrack the error 31 to what's not as expected.

Looks like the symlinks are being expanded by getcwd() and then the base no longer matches up, failing the new_dir.begins_with(base) check:

Can anyone still reproduce this on 3.2beta1 or later?

No answer, so closing.
Please comment if you can still reproduce the issue, and ensure that an up-to-date minimal reproduction project is attached here.

I have this issue in Godot 3.4.4

@desarrolladorldpf Please open a new issue with a minimal reproduction project attached.


彩虹屁机器人 开心每一天

title: 彩虹屁机器人     开心每一天
author: Livingbody
categories: article

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@wj-Mcat could you please take a look at this PR and see if you can help @livingbody to pass our CI test? Because it's based on Python Wechaty and seems that the CI is very close to being able to turn green.

Thank you very much for your contribution!
You are welcome to join Wechaty Contributor Program
1. Join Wechaty Organization

You've invited livingbody to Wechaty! They'll be receiving an email shortly. They can also visit https://github.com/wechaty to accept the invitation.

I have invited you to join our Wechaty GitHub Organization, please accept it by following the above message. (See also: wechaty/PMC#16)
2. Update Your Wechaty Contributor Profile

Please open Contributor Hall of Fame and add yourself to the end of the list, so that other contributors will know you better!
Please add yourself to our Website Contributors by creating a PR and refer to this PR link as well.

3. Join The Contributor Only WeChat Room
We also have a WeChat room for contributors only which can discuss Wechaty at a deeper level, you are welcome to join and if you are interested.
Please add @lijiarui wechat: ruirui_0914 and send her this pr link. She will invite you into Wechaty Contributor Room
Cheers!

Merge  this PR for preparint

#1501


Client.core: addition API for tabs controllers

chipmunk.client.components lib has TabController. It's possible to set active (selected tab) programmatically. But in addition, we have to have the event onTabReady or something like it to make users sure - the content of tab is rendered and ready. This API is needed for example to correct access to view's controls.

Should be used services and controllers (related to view)


flatpak: fix cross compilation

the 1.14.8 -> 1.14.10 update broke cross compilation, because the upstream build script now invokes runtime dependencies for feature detection. https://github.com/flatpak/flatpak/blob/5a2503f1e8ef94364e3060ec440546cc47af7fa5/configure.ac#L178
it seems the package will be more robust against changes if we simply emulate these checks rather than attempt to patch them out individually.
CC: @SuperSandro2000 and @getchoo for their familiarity with flatpak and with the original cross fix.
Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

not sure what is wrong with the icon validation but that was broken before.


fix: ensure File is pollyfilled for Node 18.11+

fixes #10946
Quick fix that ensures File is always polyfilled rather than just pre Node 18.11.0. The global was only added in Node 20, while buffer.File was only added in Node 18.13.0
https://nodejs.org/dist/latest-v20.x/docs/api/globals.html#class-file
Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

[x] It's really useful if your PR references an issue where it is discussed ahead of time. In many cases, features are absent for a reason. For large changes, please create an RFC: https://github.com/sveltejs/rfcs
[x] This message body should clearly illustrate what problems it solves.
[ ] Ideally, include a test that fails without this PR but passes with it.

Tests

[x] Run the tests with pnpm test and lint the project with pnpm lint and pnpm check

Changesets

[x] If your PR makes a change that should be noted in one or more packages' changelogs, generate a changeset by running pnpm changeset and following the prompts. Changesets that add features should be minor and those that fix bugs should be patch. Please prefix changeset messages with feat:, fix:, or chore:.

Mhhhhhm does this interfer with undici now again, as in "different versions are used and undici breaks", because File could come from our undici version and the other stuff could come from the built-in undici version?

Could but looks like File at least has some leeway by virtue of being based on Blob so undici has some quacks-like-a-duck logic to prevent such errors. Also on 18.13+ both undici and our polyfill try to use the File from buffer first so newer versions should not have any mismatches


Limiting maximum connection of node

Is it possible to limit maximum output to / input from of a node?
For example:
If A1 connected to B1 then A2 cannot connect to B1 since B1 maximum input is 1
If A1 connected to B1 and there is a B2, A1 cannot connect to B2 since A1 maximum output is 1
Thanks!

Hello @nodegin
Use events connectionCreated for remove connections.
You have to create your logic.
View examples to limit  sigle connection: https://github.com/jerosoler/Drawflow/issues/35
Jero

Thank you @jerosoler, the example worked for me!


Heteronym problem

Hello, thanks for your great work! I found that dp.phonemizer cannot handle heteronym problems well.
For example:

"We create the new record in the recording room"

turns into

"[W][IY] [K][R][IY][EY][T] [DH][AH] [N][UW] [R][AH][K][AO][R][D] [IH][N] [DH][AH] [R][AH][K][AO][R][D][IH][NG] [R][UW][M]"

while record should be [R], [EH], [K], [ER], [D]
Is there any suggestion? Thanks

Essentially the only way to differentiate the correct case is with part of speech information, something that is outside of the scope of the g2p model.
You can optionally train with this additional parameter but in my opinion is kind of pointless as English only has a finite defined set of heteronyms which can be manually replaced after doing part of speech tagging.
I have an implementation combining heteronym parsing with deep phonemizer for an example implementation:
https://github.com/ionite34/Aquila-Resolve

Essentially the only way to differentiate the correct case is with part of speech information, something that is outside of the scope of the g2p model.
You can optionally train with this additional parameter but in my opinion is kind of pointless as English only has a finite defined set of heteronyms which can be manually replaced after doing part of speech tagging.
I have an implementation combining heteronym parsing with deep phonemizer for an example implementation: https://github.com/ionite34/Aquila-Resolve

Thanks a lot for your great work! I will take a deep dive into your repo


Reminder notifications

Pull Request Template
1. Targeted Issue
(The first step to a good solution is knowing the problem. Please link the issue here.)
Issue #... or
Ticket #...
2. Overview of Solution
(You've put in the hard work, now's your chance to tell us about it. What's your solution in a nutshell?)
3. Tools Used
(Share the tools, libraries, or techniques you've used in your solution.)
4. Testing Strategy
(No code is complete without tests. Outline what you've tested and how.)
5. Future Implications
(Let's anticipate potential effects of your changes. Are there any potential issues down the line?)
6. Screenshots
(A picture is worth a thousand words. Include screenshots of your changes if relevant.)
7. Code Reviewers
(Who else is in the loop? Mention your team or other contributors here.)

Before you go...

[ ]  I have linked the relevant issue
[ ]  I have provided an overview of my solution
[ ]  I have shared the tools I've used
[ ]  I have outlined my testing strategy
[ ]  I have considered future implications of my changes
[ ]  I have included relevant screenshots
[ ]  I have kept my team in the loop

Big high-fives for your incredible pull request! We're thrilled to have you as a contributor! Our team will dig into your proposed changes and give you some feedback or merge them if they're good to go. Your contributions mean the world to us!

Need to merge some updates from the main branch. Please hold off on reviewing until noted.


middleware: changes wasm basic to use waPC

Description
Updates wasm basic middleware and simplifies some code inside of it,
notably by using waPC.
This changes the entrypoint function to be named "rewrite" and
uses a pool because known memory allocators are not goroutine
safe.
FYI wazero's first beta tag will be in two weeks
https://github.com/tetratelabs/wazero/issues/506#issuecomment-1162489846
Issue reference
N/A
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list:

[x] Code compiles correctly
[x] Created/updated tests
[n/a] Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

@daixiang0 can you review this since you know more about WASM and the Wazero library?
Also we want more information on the structure of the WASM that can be loaded. A compiled WASM sample is obviously not good enough :)

@codefromthecrypt thanks for your contribution! Since wazero is not stable now, it will continue to iterate.
Maybe we can get UUID from context. For the logging, I would like to make it detailed to help debug.
Adding more support is my plan, such as support handling HEADER/response changes, while not having much time to do it now.

@daixiang0 cool I can do a UUID like https://github.com/wapc/wapc-go/blob/master/engines/wazero/wazero.go#L303
Should we start by assuming the wasm binary changes per request? or should we assume it doesn't?

That is an interesting question, load once would get better performance. If wazero supports it, we can use it, and compare SHA256 of the wasm binary, if changes, we reload it.

OK I've refactored now, though feels there are some API glitches. Have a look at the impl and if there's any more stuff here I can do that before backfilling tests.

GetHandler isn't context aware, which can make things like tracing or profiling difficult (initialization logic)
GetHandler is a func, not an interface, which means what's returned can't be closed. This means some leaks

One way is to make Middleware closeable and then move the runtime to a field in that.

At least a README shoud be made in how to generate the wasm
one other thing I think would be helpful is that the "run" function isn't documented or linked in a README or something (at least not that I can find). I think the source that "hello.wasm" was compiled from should be checked in, with instructions on how it was compiled. (ex. tinygo build -o hello.wasm -scheduler=none --no-debug -target=wasi hello.go.
Alternative ABI
Right now, "run" is allocating memory for the URI using tinygo specific allocation. So, ex this won't work with other language compilers. Another way is to not do allocation externally.
WASI-only
Instead of passing memory in and out, you might consider using WASI instead. Doing so would be more portable as you can use other languages.
For example, you can set moduleConfig.args = "rewrite_request_uri", uri and moduleConfig.stdout = buf
then just instantiate the module and see if the stdout is empty or not, and if not assign it to SetRequestURIBytes
WASI+callback
reading stdout is easy, but could result in larger code depending on how people write to it. Another way is to have them call a custom host function if they need to change the URI. So, in this case a host module is instantiated per request which dispatches "set_request_uri(offset, len)" to req.SetRequestURIBytes. The wasm people generate import this function and use that instead of stdout.
Callback only
instead of having the wasm parse os.Args (ex that "rewrite_request_uri" == arg0), the entrypoint function could be like you have now.. "run" aka "rewrite_request_uri". It wouldn't accept any args, as it is hard to know which memory offsets are unused by the compiler. Instead, there's a host function to get the current uri (ex "get_request_uri"). So on each request "rewrite_request_uri" is invoked which might call "get_request_uri" if it was enabled, and then might call "set_request_uri" if it decided to rewrite it.
Your choice here
There's no perfect way, and also there are other tools that have callback flows like wapc. This is mainly to get the thinking down. Even the current approach is fine, if documented. Just it needs tinygo only for wasm generation. That may be fine!

Very cool! The basic WASM support is the first step, actually, I am thinking about Envoy WASM ABI implementation, that will create a bridge between service mesh and application runtime. For basic design, I think it is simple enough to start with.
Could you add an example in dapr/examples for this and update related documents?

cool. I'll polish this up where basic means URI rewrite. I'll go ahead and update the exported function name to "rewrite_request_uri", also, unless you've strong feelings towards "run"

That is ok, go ahead!

@daixiang0 I've updated the code here.

Could you add an example in dapr/examples for this and update related documents?

sorry if I'm a bit daft, but can you be precise where you want to update including both the repo and the directory names?

sorry if I'm a bit daft, but can you be precise where you want to update including both the repo and the directory names?

I mean https://github.com/dapr/samples, you can add a WASM sample.

@daixiang0 cool for the sample. is there any thing else to do in this repo? I would like to have this land before updating https://github.com/dapr/samples so that the instructions can say the right function name.

ok ready on my side, when this is merged I'll follow up on the sample no prob!

@daixiang0 feedback sounds good. I generally make a sub-module readme instead of making a large one for the whole directory. I'll give it a go and if we need another revision no problem.

Also please fix CI.

ok I've fixed everything. One thing I would like to say is that while we should merge this I think we should refactor before recutting the documentation.
Generally, I think it is too difficult to use this as generating the wasm is burdensome. People have to know too much for example memory offsets and such. I would recommend instead we either switch to a WASI approach as mentioned here or switch to wapc-go cc @pkedy. Either one I'm happy to do as a follow-up.
I also think that it is unlikely someone will be able to use something besides wazero, so allowing that configuration while seemingly open has limited usefulness. For example, if you add wasmtime or wasmer, you'd end up in the same situation of wapc-go where they cannot load at the same time due to shared library conflicts.
In summary, I think if the first goal is to show someone something easy, I think we should switch this to wapc-go+wazero and make it easy that way. This would also support languages besides tinygo. For future work, such as proxy-host, going straight to wazero makes more sense as it would be more efficient and the wasm side can leverage existing compilers. However, I'm fine to continue this course however you decide!

WASM can be considered as a single field although most program things can be brought from other languanges, like Golang/Rust, it still needs some extra knowledge.
In the first step, I introduce its support, which is really a very initial version needed to iterate. Replace request URI is one function, let's add more functions if you want.
Since now we do not want to enable CGO, the runtime config may be useless, but after CGO support, we can quickly add other WASM runtimes which need CGO.
wapc-go is really cool, you can optimize with it.

@daixiang0 cool so I will optimize for wapc-go and note that to use other runtimes is a build flag not a runtime one. wapc-go works for that (I implemented it!) I think this is a good path because it makes the first time experience less painful. I will add a commit to do this with wapc so we can validate.

ok I switched to wapc-go and i think you'll agree it is a lot simpler now, both in the host and also how users implement the guest side.

Since now we do not want to enable CGO, the runtime config may be useless, but after CGO support, we can quickly add other WASM runtimes which need CGO.

Concretely, WASM runtimes that use CGO (wasmer, wasmtime) import the same WASI C header file. This causes a conflict which prevents them from being compiled in the same binary. This wouldn't be the case if they used different namespace, but it is also unlikely one will change. So, later, keep this in mind as basically dapr would need to be rebuilt for each runtime that conflicts like this. Some may be fine doing that as they are using custom binaries anyway.

This shouldn't be a problem. We will not have CGO enabled in Dapr "ever". What we'll do is enable adding other components that communicate via gRPC. Because those will be in different processes, they can use CGO freely and don't risk conflicts.

thanks for the steering @ItalyPaleAle
FYI and more for the others on this thread... I bumped https://github.com/dapr/dapr/issues/3787#issuecomment-1186061176 with some thoughts as I think exactly this plugin could help people learn a bit about WebAssembly as a potential plugin model alongside gRPC (at least for things that can be handled locally). Agreed that gRPC for plugin arch is fairly common and well understood means to control dependencies, including build or platform libraries.

here's the doc PR https://github.com/dapr/docs/pull/2663

@Taction @ItalyPaleAle please help review and merge.

@codefromthecrypt Thanks for your great work! LGTM overall, just a nit change.

hi there. I'm wondering if there are any more steps needed. I'm attending a couple devops events (wellington and singapore) over the next couple weeks, and was hoping to point to this as a neat way to start learning about webassembly and extension. Any idea when this and the docs might be mergeable cc @msfussell

@codefromthecrypt Also please resolve the conflicts.

there's an API change under play so I'll update this again post merge of https://github.com/wapc/wapc-go/pull/35

@Taction I backfilled a bunch of tests. you can decide to merge before the wapc refactor because that refactor won't effect how we present configuration here to users anyway. Thanks for the help!

@codefromthecrypt - only components-contrib maintainers (like myself) can merge PRs.
You mentioned you will be updating this PR once https://github.com/wapc/wapc-go/pull/35 has been merged. So for the timing being I will convert this PR to a draft. Once you have it updated, feel free to convert it to a regular PR again and I can help get this reviewed again and then merged. Feel free to mention me or the contrib maintainers tag directly.

@Taction @daixiang0 I added e2e test dir as I think having this in place makes any bugfixes easier later, as well if we want to transition to some more serious (non-basic) it helps people understand the infrastructure needed. As this is new, you may want to PTAL
otherwise, once https://github.com/dapr/components-contrib/pull/1975 is merged I will rebase one more time and mark for review by a maintainer

https://github.com/dapr/components-contrib/pull/1975 is merged now.
Please note that after rebasing you must use Go 1.19 and if you want to use the linter locally, you need golangci-lint v1.48.0.

ok all rebased as of.... now! 🤞

I think the last commit should fix the windows fail (thanks again to @pkedy for the nudge ;). We were testing error messages too strictly and they can vary per OS, such as was the case.

Please fix conflicts. @codefromthecrypt

squashed and rebased @daixiang0!

Please update tests/certification/bindings/alicloud/dubbo/go.mod as well.

ok I think Dubbo maybe has some deps to update as I had to use go mod tidy -compat=1.17 to get it to work. Meanwhile 🤞

FYI, I'll update this from merge conflicts again tomorrow after wapc-go uses wazero's first tagged release (1.0.0-beta.1).
Meanwhile, we've also opened a gophers slack wazero channel if anyone wants to chat.

rebased again /cc https://github.com/orgs/dapr/teams/maintainers-components-contrib for merge review.

rebased :D

fossa scan looks like a timeout. could someone 🥾 it?

failure was Kafka maybe it can be kicked
=== FAIL: . TestKafka_with_retry (unknown)
[52](https://github.com/dapr/components-contrib/runs/8212909954?check_suite_focus=true#step:14:53)

@dapr/maintainers-components-contrib I know folks are busy, but I admit it is getting tiring to rebase or find unrelated build glitches every few days. I'd love to help in more productive ways than this, ex making a demo etc. can someone please help land this?

fyi @berndverst ps not sure how to tag the team as https://github.com/orgs/dapr/teams/maintainers-components-contrib is a dead link for me.

@daixiang0 ps since you added this comment, there's now an experimental compiler cache option https://github.com/dapr/components-contrib/pull/1833#issuecomment-1171959542
https://pkg.go.dev/github.com/tetratelabs/wazero/experimental#example-package-WithCompilationCacheDirName
note: this requires 5-7x the size of wasm to disk. The actual size is hard to say as it is more about the complexity of the code which we can't control.
If you want me to add this now I can, though I'd need to update docs etc. Otherwise, it can be done later. In any case, this only helps speed up restarts.

@daixiang0 ps since you added this comment, there's now an experimental compiler cache option #1833 (comment)
https://pkg.go.dev/github.com/tetratelabs/wazero/experimental#example-package-WithCompilationCacheDirName
note: this requires 5-7x the size of wasm to disk. The actual size is hard to say as it is more about the complexity of the code which we can't control.
If you want me to add this now I can, though I'd need to update docs etc. Otherwise, it can be done later. In any case, this only helps speed up restarts.

Thanks for your input! Let's do it in another pr.

fyi @berndverst ps not sure how to tag the team as https://github.com/orgs/dapr/teams/maintainers-components-contrib is a dead link for me.

Maybe you need to become a team member first, see dapr/community.

ps maybe the below error can be ignored until fixed elsewhere?
dockercompose.go:90: Starting kafka3 ...
[6462](https://github.com/dapr/components-contrib/runs/8219004661?check_suite_focus=true#step:14:6463)
        Host is already in use by another container
[6463](https://github.com/dapr/components-contrib/runs/8219004661?check_suite_focus=true#step:14:6464)
        Starting kafka3 ... error
[6464](https://github.com/dapr/components-contrib/runs/8219004661?check_suite_focus=true#step:14:6465)

[6465](https://github.com/dapr/components-contrib/runs/8219004661?check_suite_focus=true#step:14:6466)
        ERROR: for kafka3  Cannot start service kafka3: driver failed programming external connectivity on endpoint kafka-3 (c4ac5d642e0e70a8d524101dd817f3a408a977c8d1363d4085fda86102a703b0): Error starting userland proxy: listen tcp4 <IP_ADDRESS>:39092: bind: address already in use
[6466](https://github.com/dapr/components-contrib/runs/8219004661?check_suite_focus=true#step:14:6467)
        No containers to start
[6467](https://github.com/dapr/components-contrib/runs/8219004661?check_suite_focus=true#step:14:6468)
        1

I'm unsubscribing as it is just merge bot stuff and fuzz from other broke things. If someone needs my help, please mention me!

@codefromthecrypt I will merge this now. Can you please make any documentation updates and create the sample you were discussing previously in this issue.

@berndverst thanks for owning getting this in, and of course thanks to @daixiang0 @Taction @ItalyPaleAle for review help also.
Good news is we already completed the docs based on this PR before and nothing drifted since then. https://docs.dapr.io/reference/components-reference/supported-middleware/middleware-wasm/
I'll follow-up with the sample once things merge, no prob!

thanks for merging @artursouza! I opened this to bump deps and once in I'll do the sample mentioned in the desc https://github.com/dapr/dapr/pull/5169


feat(#35): use new features of jcabi-log 0.24.1

Now, in all the places where we use jcabi logger, we use %[file]s
Closes: #35.

PR-Codex overview
Detailed summary

The PR focuses on updating the format of log messages in the Compiler class.
The log messages now use the %[file]s format specifier to display file paths.

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

@rultor merge

@volodya-lombrozo OK, I'll try to merge now. You can check the progress of the merge here

@rultor merge

@volodya-lombrozo Done! FYI, the full log is here (took me 15min)


Table Content Discovery

Problem statement
Tables are used extensively across the ReportStream website, but use different styles and address different use cases. Prior to consolidating the table component into "One table to rule them all" (#8280), the design team needs to conduct an audit of all of the table instances to ensure that the new component will address the needs of tables across RS. Additionally, tables that don't yet exist (e.g, Programmer's Guide and the Data Dashboard) also have to be considered. Ultimately, this deeper understanding will better inform the subsequent synthesis and functionality work (captured in Develop rough requirements for RS tables #8231).
What you need to know

Table Component Design Brief
Sister Ticket: Table Design Discovery #8278

Acceptance criteria

[x] Current table content elements are documented in an accessible format
[x] Any preliminary recommendations/findings from the audit are captured and ready to inform subsequent ticket (Develop rough requirements for RS tables #8231)

To do

[ ] Audit current Admin & Admin Tool tables
[ ] Audit current Sender tables
[ ] Audit current Receiver tables
[ ] Audit public website tables
[ ] Audit future feature tables (e.g. Programmer's Guide, Data Dashboard)

@chris-kuryak mostly took this from @jillian-hammer's work on #8277 because that was solid!

closing @chris-kuryak
Doc of findings here


[Firefox]Dragging the button to the left/right end in slider ui, a blank page is loaded

Detailed description
Dragging the button to the left/right end in slider ui, a blank page is loaded.
Then a user cannot see the slider ui and do anything in this page.
Steps to reproduce the issue

Move to this site using latest Firefox
Drag the button to the left/right end
A blank page is loaded

Additional information

The initial state

Drag the button to right direction

A blank page is loaded!

Hi @laripappa thank you for reporting! I'm quickly trying your scenario to reproduce the problem, but no luck so far - Do you want to share the version of Firefox you are running? Thanks!

@asudoh I'm using Firefox 62.0

Running with Firefox 62.0 does not seem to reproduce the problem, either - Would you want to create en empty Firefox profile and run that again? Thanks!

Just in case, my environment is on macOS High Sierra 10.13.6/FF 62.0

Thank you for the details - Seems the same (for both) as me!

Hi all! Seems like we need some more steps to reproduce here?

Yup, I gonna update environmental info and reproducible step, so plz wait.

Sorry for a delay, I confirmed this defects still happened in my environment with empty Firefox profile.
Reproducible steps are just written as I wrote.

@asudoh please let me know if you will not be able to reproduce this.
I will demonstrate this face to face because I'm working at just your downstairs ^^

I updated the reproducible steps! Sorry for my incomplete description...


[Frontend] Add runtime warning for unexpected JAX version

Context: Sometimes it is useful to install in catalyst in debug mode. This enables one to checkout different versions of the project without having to pip install -e . again. However, the downside is that when one checks out versions of Catalyst that change compatibility with JAX it is necessary to install JAX again.
Description of the Change: This change adds a runtime warning in case catalyst is run against an unexpected version of JAX.
Benefits: Warning in case of running an unexpected version of JAX.
Possible Drawbacks: Another place to change whenever JAX version changes.

Thanks @erick-xanadu -- I have a quick question. Would catalyst work with versions of JAX outside of 0.4.13? Or is there ABI/API compatibility issues, and only JAX 0.4.13 will work with Catalyst?

There are only a few things that I believe would break between versions:

import locations / function call interfaces (API Changes)
the generated MHLO / StableHLO

If those two things remain stable then Catalyst should be able to work with JAX versions that are lower than the current one. As test, I ran the python tests we currently have and was able to get Catalyst to succeed for all versions of JAX until 0.4.7.
What happened here was a bit the opposite, I was trying to execute a benchmark with a JAX version of 0.4.13 when I was switching to an older version of Catalyst but it should still be relatively the same.
We can keep this as a WIP and I can take a look into what exactly the issue was that I encountered. I think maybe there was a difference in either how we compiled MHLO / StableHLO that led to a huge performance difference because I remember waiting for more than 5 minutes for a benchmark to execute when usually 3 minutes was more than enough.

Thanks @erick-xanadu! Ah this changes my thinking a bit. If <0.4.13 versions of JAX should work (within reason, e.g., not going down too far), I would not include this warning at runtime.
It is quite often that users find themselves in situations where they have different JAX versions installed because of 3rd party dependency conflicts -- in fact this happens regularly just internally at Xanadu, since tools like Optax may force you to downgrade JAX -- so would prefer not to raise a warning in that case


FR: Combine Tag and Link Folder pane into one?

I think the Link Folder feature is a great addition to the existing functionality offered by this plugin. I do hope these two can be worked together seamlessly in a single pane. One good use case is when I narrow down to one note using Tag Folder pane, if the note also links to other notes, it's much more natural and efficient to list them directly under that note, instead of having to switch to the Link Folder pane, finding the note there, then expand it to see those linked notes. Hope this is something you already did or will consider. Thx a lot.

Thank you for your idea! and sorry for being late!
I think that it might be so useful! However, the amount of computation might be much enlarged, without some tricks.


CLOUDSTACK-9305: Cloudstack Usage Breaks if DB HA enabled

With DB HA enabled in db.properties, the cloudstack-usage service restarts every 10 seconds.  Making the suggested change has fixed it for me.  Cloudstack 4.8 on Centos7

@kiwiflyer  Very good point, let me make a different change here so it is more generic!

If anyone still has issues, i also had to make the change from this PR: #1409

LGTM

Tested patch on a 4.7.1 lab.
Prior to patch:
INFO  [cloud.usage.UsageServer] (main:null) (logid:) UsageServer ready...
INFO  [context.support.ClassPathXmlApplicationContext] (main:null) (logid:) Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@70ea0c2a: startup date [Thu Mar 17 06:51:40 CDT 2016]; root of context hierarchy
INFO  [factory.xml.XmlBeanDefinitionReader] (main:null) (logid:) Loading XML bean definitions from class path resource [usageApplicationContext.xml]
INFO  [factory.annotation.AutowiredAnnotationBeanPostProcessor] (main:null) (logid:) JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
INFO  [context.support.ClassPathXmlApplicationContext] (main:null) (logid:) Bean 'transactionContextBuilder' of type [class com.cloud.utils.db.TransactionContextBuilder] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
INFO  [factory.support.DefaultListableBeanFactory] (main:null) (logid:) Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@33037ca5: defining beans [org.springframework.context.annotation.internalConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor,vmDiskUsageParser,networkOfferingUsageParser,VPNUserUsageParser,IPAddressUsageParser,VMSnapshotUsageParser,loadBalancerUsageParser,networkUsageParser,securityGroupUsageParser,storageUsageParser,portForwardingUsageParser,VMInstanceUsageParser,volumeUsageParser,usageManagerImpl,usageAlertManagerImpl,usageVmDiskDaoImpl,usageIPAddressDaoImpl,usageStorageDaoImpl,usageVolumeDaoImpl,usageDaoImpl,usagePortForwardingRuleDaoImpl,usageVPNUserDaoImpl,usageJobDaoImpl,usageLoadBalancerPolicyDaoImpl,usageNetworkDaoImpl,externalPublicIpStatisticsDaoImpl,usageSecurityGroupDaoImpl,usageVMInstanceDaoImpl,usageNetworkOfferingDaoImpl,usageVMSnapshotDaoImpl,usageEventDetailsDaoImpl,usageEventDaoImpl,eventDaoImpl,SSHKeyPairDaoImpl,userDaoImpl,vmDiskStatisticsDaoImpl,userAccountDaoImpl,accountDaoImpl,userStatsLogDaoImpl,userStatisticsDaoImpl,resourceCountDaoImpl,resourceLimitDaoImpl,alertDaoImpl,domainDaoImpl,configurationDaoImpl,quotaEmailTemplatesDaoImpl,userVmDetailsDaoImpl,quotaUsageDaoImpl,quotaBalanceDaoImpl,quotaTariffDaoImpl,quotaAccountDaoImpl,serviceOfferingDaoImpl,quotaCreditsDaoImpl,quotaAlertManagerImpl,quotaStatementImpl,quotaManagerImpl,transactionContextBuilder,instantiatePostProcessor,ComponentContext,org.springframework.context.annotation.ConfigurationClassPostProcessor.importAwareProcessor]; root of factory hierarchy
DEBUG [utils.crypt.EncryptionSecretKeyChecker] (main:null) (logid:) Encryption Type: file
INFO  [db.Transaction.Transaction] (main:null) (logid:) Is Data Base High Availiability enabled? Ans : true
INFO  [db.Transaction.Transaction] (main:null) (logid:) The slaves configured for Cloud Data base is/are : <IP_ADDRESS>,<IP_ADDRESS>

After patch:
INFO  [utils.component.ComponentContext] (main:null) (logid:) Starting com.cloud.usage.dao.UsageVMSnapshotDaoImpl_EnhancerByCloudStack_c20890bf
INFO  [utils.component.ComponentContext] (main:null) (logid:) Starting org.apache.cloudstack.quota.QuotaStatementImpl_EnhancerByCloudStack_5d0529b4
INFO  [cloudstack.quota.QuotaStatementImpl] (main:null) (logid:) Starting Statement Manager
INFO  [utils.component.ComponentContext] (main:null) (logid:) Starting org.apache.cloudstack.quota.QuotaManagerImpl_EnhancerByCloudStack_3ffc3462
INFO  [cloudstack.quota.QuotaManagerImpl] (main:null) (logid:) Starting Quota Manager
INFO  [utils.component.ComponentContext] (main:null) (logid:) Starting com.cloud.usage.UsageManagerImpl_EnhancerByCloudStack_93e4d889
INFO  [cloud.usage.UsageManagerImpl] (main:null) (logid:) Starting Usage Manager
INFO  [utils.component.ComponentContext] (main:null) (logid:) Starting com.cloud.usage.UsageAlertManagerImpl_EnhancerByCloudStack_8fa0a2a9
INFO  [utils.component.ComponentContext] (main:null) (logid:) Starting org.apache.cloudstack.quota.QuotaAlertManagerImpl_EnhancerByCloudStack_9535c670
INFO  [cloudstack.quota.QuotaAlertManagerImpl] (main:null) (logid:) Starting Alert Manager
INFO  [cloud.usage.UsageServer] (main:null) (logid:) UsageServer ready...

LGTM

@Slair1 can you please squash your commits.  I will get this into my CI queue so we can get this merged...  Thanks...

@swill Just FYI that this PR conflicts with https://github.com/apache/cloudstack/pull/1409#issuecomment-197771350.
They fix different issues, but the patch conflicts.

@kiwiflyer they conflict in what sense?  They change different lines of code, so I don't expect git to have an issue with them.  Can you clarify?

I applied 1433 and then tried to apply 1409, but 1433 changes prevent the later patch from matching previous lines.  One or the other probably needs to be based on top of the other patch so it applies cleanly.

@swill , i apologize, but this is my first PR.  I squashed it in Github Desktop on my local copy and did a push, but it sees like it it just added additional commits at github.com.  If could you let me know how to correct it i'll do so as soon as I can.

No worries.  :)  This should work: https://github.com/ginatrapani/todo.txt-android/wiki/Squash-All-Commits-Related-to-a-Single-Issue-into-a-Single-Commit
Also described here:  https://ariejan.net/2011/07/05/git-squash-your-latests-commits-into-one/
I have used this method in the past as well: http://stackoverflow.com/questions/616556/how-do-you-squash-commits-into-one-patch-with-git-format-patch?answertab=votes#tab-top
Let me know if you have problems and I will try to help you out.

@swill i seem to get an access denied when pushing my change:
C:\slair\Source\Github\cloudstack [pr/1433]> git push origin pr/1433 --force
remote: Permission to apache/cloudstack.git denied to Slair1.
fatal: unable to access 'https://github.com/apache/cloudstack.git/': The requested URL returned error: 403

No one is allowed to push code to apache/cloudstack, you have to push it to your fork where your PR is coming from.  Does that make sense?

@swill thanks, that makes sense.  I did it to my branch the first time, but for some reason when i tried to save to here it just created two more commits.  i'll try again

Once you get your local branch setup correctly with the squashed commit, you should be able to do a push - f to replace what is in the PR with what you have locally. Let me know if you have problems and I can do a test to validate the steps you need to take.

thanks @swill , it looks much better now, i needed the -f on the git push

No problem. Ya it took me a couple pull requests to figure out how they work. Thanks for taking the time to do this for me. :)

Running CI now.  Will have results later tonight or tomorrow...

CI RESULTS
Tests Run: 103
  Skipped: 0
   Failed: 3
   Errors: 2

Summary of the problem(s):
ERROR: Test Create/Delete a manual snap shot and verify
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/git/cs2/cloudstack/test/integration/component/test_project_usage.py", line 1376, in test_01_snapshot_usage
    snapshot = Snapshot.create(self.apiclient, volumes[0].id)
  File "/usr/lib/python2.7/site-packages/marvin/lib/base.py", line 1018, in create
    return Snapshot(apiclient.createSnapshot(cmd).__dict__)
  File "/usr/lib/python2.7/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py", line 797, in createSnapshot
    response = self.connection.marvinRequest(command, response_type=response, method=method)
  File "/usr/lib/python2.7/site-packages/marvin/cloudstackConnection.py", line 379, in marvinRequest
    raise e
CloudstackAPIException: Execute cmd: createsnapshot failed, due to: errorCode: 530, errorText:KVM Snapshot is not supported: 1
----------------------------------------------------------------------
Additional details in: /tmp/MarvinLogs/test_usage_A3LTTO/results.txt

ERROR: Test Create/Delete a manual snap shot and verify
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/git/cs2/cloudstack/test/integration/component/test_usage.py", line 1300, in test_01_snapshot_usage
    snapshot = Snapshot.create(self.apiclient, volumes[0].id)
  File "/usr/lib/python2.7/site-packages/marvin/lib/base.py", line 1018, in create
    return Snapshot(apiclient.createSnapshot(cmd).__dict__)
  File "/usr/lib/python2.7/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py", line 797, in createSnapshot
    response = self.connection.marvinRequest(command, response_type=response, method=method)
  File "/usr/lib/python2.7/site-packages/marvin/cloudstackConnection.py", line 379, in marvinRequest
    raise e
CloudstackAPIException: Execute cmd: createsnapshot failed, due to: errorCode: 530, errorText:KVM Snapshot is not supported: 1
----------------------------------------------------------------------
Additional details in: /tmp/MarvinLogs/test_usage_A3LTTO/results.txt

FAIL: Test redundant router internals
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/git/cs2/cloudstack/test/integration/smoke/test_routers_network_ops.py", line 290, in test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
    "Attempt to retrieve google.com index page should be successful!"
AssertionError: Attempt to retrieve google.com index page should be successful!
----------------------------------------------------------------------
Additional details in: /tmp/MarvinLogs/test_network_4ISAIJ/results.txt

FAIL: test_02_vpc_privategw_static_routes (integration.smoke.test_privategw_acl.TestPrivateGwACL)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/git/cs2/cloudstack/test/integration/smoke/test_privategw_acl.py", line 253, in test_02_vpc_privategw_static_routes
    self.performVPCTests(vpc_off)
  File "/data/git/cs2/cloudstack/test/integration/smoke/test_privategw_acl.py", line 324, in performVPCTests
    self.check_pvt_gw_connectivity(vm1, public_ip_1, vm2.nic[0].ipaddress)
  File "/data/git/cs2/cloudstack/test/integration/smoke/test_privategw_acl.py", line 559, in check_pvt_gw_connectivity
    "Ping to outside world from VM should be successful"
AssertionError: Ping to outside world from VM should be successful
----------------------------------------------------------------------
Additional details in: /tmp/MarvinLogs/test_network_4ISAIJ/results.txt

FAIL: test_03_vpc_privategw_restart_vpc_cleanup (integration.smoke.test_privategw_acl.TestPrivateGwACL)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/git/cs2/cloudstack/test/integration/smoke/test_privategw_acl.py", line 265, in test_03_vpc_privategw_restart_vpc_cleanup
    self.performVPCTests(vpc_off, True)
  File "/data/git/cs2/cloudstack/test/integration/smoke/test_privategw_acl.py", line 324, in performVPCTests
    self.check_pvt_gw_connectivity(vm1, public_ip_1, vm2.nic[0].ipaddress)
  File "/data/git/cs2/cloudstack/test/integration/smoke/test_privategw_acl.py", line 559, in check_pvt_gw_connectivity
    "Ping to outside world from VM should be successful"
AssertionError: Ping to outside world from VM should be successful
----------------------------------------------------------------------
Additional details in: /tmp/MarvinLogs/test_network_4ISAIJ/results.txt

Associated Uploads
/tmp/MarvinLogs/DeployDataCenter__Apr_28_2016_22_51_48_T2XTTQ:

dc_entries.obj
failed_plus_exceptions.txt
runinfo.txt

/tmp/MarvinLogs/test_network_4ISAIJ:

failed_plus_exceptions.txt
results.txt
runinfo.txt

/tmp/MarvinLogs/test_usage_A3LTTO:

failed_plus_exceptions.txt
results.txt
runinfo.txt

/tmp/MarvinLogs/test_vpc_routers_LS4J5A:

failed_plus_exceptions.txt
results.txt
runinfo.txt

Uploads will be available until 2016-06-29 02:00:00 +0200 CEST
Comment created by upr comment.

None of the failures above are related to this PR, they are all issues I see periodically in my environment.  All of the supported usage tests passed, so CI has passed for this PR.  Sorry the output makes that hard to understand.

That's ok, thanks!!

2 x LGTMs and CI complete. This is Ready to Merge.

Sorry, I missed the LGTMs.  :)  I will add this to my merge queue.  Thanks @kiwiflyer for following up.

@Slair1 if dh ha jar is missing for example there is no db-ha deb package, what happens?

Hi @rhtyd since it uses "ls" to get a list of packages, if the directory or .jar file does not exist it is no big deal.  It will output a No such file or directory message but still loads any packages that exist without issue.  the UCP variable still looks good when that occurs.

fair enough, LGTM
tag:mergeready


Lazy AllPackages

Defers reading a types/* directory until it’s asked for, and further defers analyzing the source files of a package (to get used/unused files, check that local references are of an allowed form, and get the full file list for content hash) until it’s asked for.

I left in some types/options for parallel parsing that are currently unused, thinking I might bring back sharded processing, but with other changes we’ve talked about, I don’t think it will be necessary. I’ll remove them now.

but now we request all packages in parallel

I don’t think we’re making any network requests in parallel; did you see a place where that’s happening? I tried this locally and noticed that it fired off ~9000 requests to download allowedPackageJsonDependencies.txt so I fixed that one with a quick hack, knowing we want to move it to a CI-only check later.

I didn't see anything specific; a lot of the Promise.alls are at a highlevel so I wasn't super sure one way or the other.

Yeah, they’re just doing file I/O


Render area:highway=* for roads

as requested in #6
Render area:highway=yes;primary;residential;service;etc as asphalt in separate layer: under the roads, not over them like on intersections or parkings.
For this I've added new area category and new z-offset.
Before:

After;

The reason why I hesitated to implement support for highway areas is because it looks a bit ugly (at least to me).
There's a better way of doing this though: render the actual roads as the first layer, then render the area:highway, and then render road markings as a separate and the final layer. So kinda like railways which are rendered using 3 layers. But there are the performance implications (i.e. overdraw), so this approach needs to be implemented wisely.
Thank you for the PR. It's better than nothing I think.


Server disappears from server list if player count is 0

Might be since version 1.1.1 or 1.1.2.
If you immediately join after server creation, it won't disappear and allows other players to join using the server list.

I could be running into this issue. I'm trying to run a server and it only stays on the master server list for around 10 seconds.

Only workaround I found so far:
Create an Autohotkey script that reads the window title and restart it every minute if it says "mp_lobby 0/16 "🤡
makes server visible 25% of the time all the time.

this has been fixed with this pull request, should be on main masterserver soon


Release 7.17.0

Release 7.17.0 of camunda-platform-7-keycloak

this is the first version after renaming the extension
the version from now on reflects the Camunda version used in tests and samples

Sorry, I haven't been able to add the team @camunda-community-hub/devrel as assignee. So I picked you. Please assign to an appropriate colleague.

Hmm...  is there really anything to do?
It's already available on Maven Central:

You are right @VonDerBeck - it is already released. You might have been lucky and somebody looked into staging artifacts anyway - as there is definitely a manual step involved :-)


Fixed failing tests: set user agent on load external content

For a while tests were failing on the build. Here is the simple fix

Tests are now passing, the only problem is CS issue, introduced in #363 and fixed in #364 which is not merged yet.


[3.1][runtime] Fix race in unknown weak variables.

The bug is a three-way race on a single weak variable after a
weakly-referenced pure-Swift object is deallocated:
T1: calls swift_weakLoadStrong() and sets WR_READING on the value
T2: calls swift_weakLoadStrong(), advances past the nil check, and
starts spinning waiting for WR_READING to clear
T1: completes swift_weakLoadStrong(), setting the variable to nil
T2: sees WR_READING clear and sets the variable to nil | WR_READING
T3: calls swift_unknownWeakLoadStrong(), sees a non-nil value with
WR_NATIVE not set, and calls objc_loadWeakRetained() because it thinks
this is a live ObjC weak variable
The fix is to keep WR_NATIVE set when setting the variable to nil.
Then swift_unknownWeakLoadStrong() will not see a non-nil value with
WR_NATIVE not set. It will not call into objc_loadWeakRetained(), so
objc_loadWeakRetained() will not trip over the unexpected WR_READING bit.
rdar://30404063

The other implementation (using a CAS loop to prevent storing nil | WR_READING) might be safer after all. I'm testing that one now.

Abandoned in favor of #7499.


Dimension Operator Values

Can you tell me what the valid dimension operator values are?  I see "Include" in all of the examples (see below).  Is that the only operator supported or are there others?
"dimensions": [
{
"name":"ApiName",
"operator": "Include",
"values": ["GetBlob"]
}
],

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1f2bd9d0-af11-d684-051b-7ac8e611cca9
Version Independent ID: f83d6e85-4d52-6b7f-a79e-f3ed3f3d0a95
Content: Create a metric alert with a Resource Manager template
Content Source: articles/azure-monitor/platform/alerts-metric-create-templates.md
Service: azure-monitor
Sub-service: alerts
GitHub Login: @snehithm
Microsoft Alias: snmuvva

Thanks for your comment. We are actively investigating and will get back to you shortly. Thanks for your patience.

@bdschaap Right now, that is the only operator supported.


ActionSheetIOS.showShareActionSheetWithOptions not able to share local image file

I am trying to share a local image file with ActionSheetIOS.showShareActionSheetWithOptions.
I can display image from the local file with following jsx:
<Image source={{uri: 'file:///' + this.path}} style={styles.png}/>
But with same path (i.e. 'file:///' + this.path) assigned to url key of the object parameter to ActionSheetIOS.showShareActionSheetWithOptions, does not share file. Only text assigned to message key of the parameter is shared.
react-native version: 0.27.2
I am running app on an attached iPhone 4S iOS 9.3.2 device.
Dev machine: Mac os x 10.11.5

You have a extra / in your file path.
Try file://' + this.path

@christopherdro, appreciate your help. I tried with file://, but it did not work either. The this.path is having value /var/mobile/Containers/Data/Application/60127A89-D21F-4125-881E-D7D145830455/Documents/test.png, if at all this helps.
And just a gentle reminder, I can display image with the same path and triple '/' after 'file'. So I was wondering, what is that makes Image to understand the path and not ActionSheetIOS.showShareActionSheetWithOptions?

I have the same issue. When emailing, only the subject/message carry through. There is no attachment.
If I share via Airdrop, the file transfers fine, but I get a strange file name. Example: (null)-257D026032E9-1.data
If I rename the file to the correct name and extension, the file works properly.
It is able to find the file just fine but not email it or airdrop with the name intact.
Any ideas?

Passing the file URL instead of the file contents to UIActivityViewController fixes the issue.
Created a PR with the fix: https://github.com/facebook/react-native/pull/9693

Looks like this is fixed now


Fixed Issue #201

Hey ,I have fixed the issue by implementing a alert dialog which restricts user if user gives same name as already existing file.Please review it. #201

@codingblazer
Hey, The code looks great. Don't you think It would be better to use AlertDialog box class instead of MaterialDialog box. It will actually make your PR independent of some external library:)
Thanks

@octacode Thanks for suggestion. Actually, MaterialDialog  has already been imported in the TemplateEditor.java and used at various places (also used in other files in project). Making another import for using AlertDialog in this file didn't looked good to me. Also using two different types of dialog in same file is not considered a good practice.Thanks

@codingblazer , can you please resolve the merge conflicts.
Instead of showing them in alert box, it would be great if we show them the error beside the textview only. Validation will be performed before the project is saved.

You can achieve this by textView.setError("message");

@opticod Thanks for review. I have updated the branch as per review.

@codingblazer, Thanks for your changes, I am reviewing now. I was travelling this week, so was away from this.

@opticod Its working fine on my device.

Can you please recheck it or send me a screenshot ?


Detect changes using crates.io index repository

Possible Improvements

Database errors are ignored just to match the previous implementation. It might be better to do as much as possible, but at least return the last seen error so something shows up in the logs. Alternatively, errors could just be logged.

Caveats

Even though the functionality itself is very well tested via the crates-index-diff and the crates-io-cli crates, I never saw the implementation in action.
Unless the crates.io-index is already checked out with the branch crates-index-diff_last-seen set to a recent commit, the first invocation of get_new_crates() will list (and possibly queue) all 40000 of them.
Previously a similar implementation might have caused file handle leakage. This should be watched carefully, as the leak could be in the outdated version of git2. If that should be the case,  crates-io-cli could be used to move the leakage into its own process.

Notes

All the relevant tests were #[ignore]d, so I hope the CI will catch potential issues.
This PR would fix #89 .

Awesome, and thank you very much again. There is only one more thing I'd like to mention since we are also updating all dependencies with this PR.
docs.rs is currently using an old version of cargo and until docs.rs starts using iron 0.4 we are basically stuck with an old version of cargo. Before merging this PR I'd like to update cargo to a recent revision that still uses openssl 0.7.
Cargo started depending on openssl 0.9 with this commit: https://github.com/rust-lang/cargo/commit/15acaa9c312422dc82d5046007f55f2b8765afa6
Last revision with openssl 0.7 is f6500c6b. I'll update cargo to this revison and merge this PR. This revision is using git2 0.5 but I think there isn't any compatibility problems between git2 0.4 and 0.5.

Great to hear!
If there are any problems, just let me know and I will put git2 back version 0.5 in the crates-index-diff crate - no problem.

@Byron this is working great but there is only one problem.
When I first run get_new_crates, it added every crate and every version (~38k row) into build queue. I have a clone of crates.io-index in self.options.crates_io_index_path and it was only 6 commits behind. Can you make it to get current head of the self.options.crates_io_index_path repository and compare it with the new head.

Ahh its using head of crates-index-diff_last-seen branch and comparing it with new head of master branch. I prefer comparison of old-master and new-master but this is also perfectly fine.

Awesome!
I will now battle-test it and publish 140 or so crates in very short succession :)!
Because Docs.rs is so awesome (!!), I have now switched all my crates (at least those that I remember;)) to use docs.rs automatically.
Those who are generated, i.e. the ones of google-apis-rs link to their specific version. Said link right now will only be available, in the worst case, after 10 minutes (I believe to have read that value somewhere here), which could already distract interested parties.
Something that would certainly help is to reduce the time it polls considerably - what do you think about that? Another way to help could be to indicate something is queued to be built.
What is your opinion about that?

Yes I was thinking the same yesterday. I already had a FIXME tag about this in daemon.rs. We can return length of changes vector from get_new_crates and build them right away when there is any. Reducing sleeping time between each request is also fine now.

Great you have it on your radar, as we are both dreaming of a world where docs.rs can build near-realtime and is the defacto standard facility for all crates documentation.
By the way, right now I am anxiously looking at the site...

...and am wondering if everything is alright. It seems to be about 50 minutes behind, and also doesn't seem to process the crates in the same order as returned by the find_changes() method. I know this thanks to the following invocation (and would have expected orders to be the same):

➜  docs.rs git:(master) ✗ crates recent-changes
google-tasks1 1.0.2+20141121 added
➜  docs.rs git:(master) ✗ crates recent-changes
google-blogger3 1.0.2+20150422 added
google-dfareporting2d1 1.0.2+20160323 added
google-dfareporting2d2 1.0.2+20160803 added
google-dfareporting2d3 1.0.2+20160803 added
google-groupsmigration1 1.0.2+20140416 added
google-manager1_beta2 1.0.2+20140915 added
google-proximitybeacon1_beta1 1.0.2+20160429 added
➜  docs.rs git:(master) ✗ crates recent-changes
google-adexchangebuyer1d3 1.0.2+20161020 added
google-adexchangebuyer1d4 1.0.2+20161020 added
google-adexchangeseller2 1.0.2+20160805 added
google-admin1_directory 1.0.2+20161124 added
google-admin1_reports 1.0.2+20160704 added
google-adsense1d4 1.0.2+20161206 added
google-adsensehost4d1 1.0.2+20161206 added
google-analytics3 1.0.2+20161004 added
google-androidenterprise1 1.0.2+20161207 added
google-androidpublisher2 1.0.2+20161212 added
google-appengine1 1.0.2+20161208 added
google-appengine1_beta4 1.0.2+20161208 added
google-appengine1_beta5 1.0.2+20161208 added
google-appsactivity1 1.0.2+20161202 added
google-appstate1 1.0.2+20161207 added
google-autoscaler1_beta2 1.0.2+20150629 added
google-bigquery2 1.0.2+20161130 added
google-blogger3-cli 1.0.2+20150422 added
google-books1 1.0.2+20161206 added
google-calendar3 1.0.2+20161211 added
google-classroom1 1.0.2+20161006 added
google-cloudbilling1 1.0.2+20151222 added
google-clouddebugger2 1.0.2+20160810 added
google-cloudlatencytest2 1.0.2+20160309 added
google-cloudmonitoring2_beta2 1.0.2+20161031 added
google-cloudresourcemanager1 1.0.2+20161212 added
google-cloudresourcemanager1_beta1 1.0.2+20161212 added
google-clouduseraccountsvm_beta 1.0.2+20160316 added
google-container1 1.0.2+20160421 added
google-content2 1.0.2+20161205 added
google-content2_sandbox 1.0.2+20161205 added
google-coordinate1 1.0.2+20150811 added
google-customsearch1 1.0.2+20160411 added
google-dataproc1 1.0.2+20161102 added
google-deploymentmanager2 1.0.2+20161209 added
google-deploymentmanager2_beta2 1.0.2+20160201 added
google-dfareporting2d1-cli 1.0.2+20160323 added
google-dfareporting2d2-cli 1.0.2+20160803 added
google-dfareporting2d3-cli 1.0.2+20160803 added
google-dfareporting2d4 1.0.2+20160803 added
google-dfareporting2d4-cli 1.0.2+20160803 added
google-dfareporting2d5 1.0.2+20161027 added
google-dfareporting2d6 1.0.2+20161027 added
google-dfareporting2d7 1.0.2+20161027 added
google-discovery1 1.0.2+00000000 added
google-dns1 1.0.2+20161130 added
google-doubleclickbidmanager1 1.0.2+20161010 added
google-doubleclicksearch2 1.0.2+20161108 added
google-drive2 1.0.2+20161212 added
google-drive3 1.0.2+20161212 added
google-firebasedynamiclinks1 1.0.2+20161118 added
google-fitness1 1.0.2+20161128 added
google-fusiontables2 1.0.2+20160526 added
google-games1 1.0.2+20161207 added
google-gamesconfiguration1_configuration 1.0.2+20161207 added
google-gamesmanagement1_management 1.0.2+20161207 added
google-gan1_beta1 1.0.2+20130205 added
google-genomics1 1.0.2+20160928 added
google-gmail1 1.0.2+20161206 added
google-groupsmigration1-cli 1.0.2+20140416 added
google-groupssettings1 1.0.2+20160525 added
google-iam1 1.0.2+20160915 added
google-identitytoolkit3 1.0.2+20161206 added
google-kgsearch1 1.0.2+20151215 added
google-licensing1 1.0.2+20150901 added
google-logging2 1.0.2+20161206 added
google-logging2_beta1 1.0.2+20161206 added
google-manager1_beta2-cli 1.0.2+20140915 added
google-manufacturers1 1.0.2+20161028 added
google-mirror1 1.0.2+20160616 added
google-ml1_beta1 1.0.2+20161212 added
google-monitoring3 1.0.2+20161212 added
google-pagespeedonline2 1.0.2+20161204 added
google-partners2 1.0.2+20151009 added
google-people1 1.0.2+20160210 added
google-playmoviespartner1 1.0.2+20160518 added
google-plus1 1.0.2+20161214 added
google-plusdomains1 1.0.2+20161214 added
google-prediction1d6 1.0.2+20160511 added
google-proximitybeacon1_beta1-cli 1.0.2+20160429 added
google-pubsub1 1.0.2+20161122 added
google-pubsub1_beta2 1.0.2+20161122 added
google-qpxexpress1 1.0.2+20160708 added
google-replicapool1_beta2 1.0.2+20160512 added
google-replicapoolupdater1_beta1 1.0.2+20161003 added
google-reseller1_sandbox 1.0.2+20160329 added
google-resourceviews1_beta2 1.0.2+20160512 added
google-safebrowsing4 1.0.2+20160520 added
google-serviceregistryalpha 1.0.2+20160401 added
google-siteverification1 1.0.2+20160228 added
google-slides1 1.0.2+20161213 added
google-spectrum1_explorer 1.0.2+20161116 added
google-sqladmin1_beta4 1.0.2+20161213 added
google-storage1 1.0.2+20161123 added
google-storagetransfer1 1.0.2+20150811 added
google-surveys2 1.0.2+20161103 added
google-tagmanager1 1.0.2+20160310 added
google-taskqueue1_beta2 1.0.2+20160428 added
google-tasks1-cli 1.0.2+20141121 added
google-translate2 1.0.2+20160627 added
google-urlshortener1 1.0.2+20150519 added
google-webfonts1 1.0.2+20160302 added
google-webmasters3 1.0.2+20160317 added
google-youtube3 1.0.2+20161202 added
google-youtubeanalytics1 1.0.2+20161213 added
google-youtubereporting1 1.0.2+20160719 added
➜  docs.rs git:(master) ✗ crates recent-changes
google-dfareporting2d5-cli 1.0.2+20161027 added
➜  docs.rs git:(master) ✗

Unfortunately each google crate is taking 6 minutes to build, so everything is fine they are in queue.
This is the order of crates docs.rs used for build:
2016/12/27 07:53:07 [INFO] cratesfyi::docbuilder::chroot_builder: Building package google-blogger3-1.0.2+20150422
2016/12/27 07:58:33 [INFO] cratesfyi::docbuilder::chroot_builder: Building package google-dfareporting2d1-1.0.2+20160323
2016/12/27 08:06:38 [INFO] cratesfyi::docbuilder::chroot_builder: Building package google-dfareporting2d2-1.0.2+20160803
2016/12/27 08:14:22 [INFO] cratesfyi::docbuilder::chroot_builder: Building package google-groupsmigration1-1.0.2+20140416
2016/12/27 08:20:27 [INFO] cratesfyi::docbuilder::chroot_builder: Building package google-manager1_beta2-1.0.2+20140915
2016/12/27 08:26:38 [INFO] cratesfyi::docbuilder::chroot_builder: Building package google-proximitybeacon1_beta1-1.0.2+20160429
2016/12/27 08:32:32 [INFO] cratesfyi::docbuilder::chroot_builder: Building package google-tasks1-1.0.2+20141121
2016/12/27 08:42:53 [INFO] cratesfyi::docbuilder::chroot_builder: Building package google-androidenterprise1-1.0.2+20161207
2016/12/27 08:49:21 [INFO] cratesfyi::docbuilder::chroot_builder: Building package google-customsearch1-1.0.2+20160411
2016/12/27 08:55:19 [INFO] cratesfyi::docbuilder::chroot_builder: Building package google-dfareporting2d3-1.0.2+20160803

And rest of the queue:
                     name                     |    version
----------------------------------------------+----------------
 google-dfareporting2d4                       | 1.0.2+20160803
 google-dfareporting2d5                       | 1.0.2+20161027
 google-dfareporting2d6                       | 1.0.2+20161027
 google-dfareporting2d7                       | 1.0.2+20161027
 google-adexchangebuyer1d3                    | 1.0.2+20161020
 google-adexchangebuyer1d4                    | 1.0.2+20161020
 google-adsense1d4                            | 1.0.2+20161206
 google-cloudmonitoring2_beta2                | 1.0.2+20161031
 google-dns1                                  | 1.0.2+20161130
 google-gamesconfiguration1_configuration     | 1.0.2+20161207
 google-gan1_beta1                            | 1.0.2+20130205
 google-groupssettings1                       | 1.0.2+20160525
 google-iam1                                  | 1.0.2+20160915
 google-manufacturers1                        | 1.0.2+20161028
 google-pagespeedonline2                      | 1.0.2+20161204
 google-partners2                             | 1.0.2+20151009
 google-prediction1d6                         | 1.0.2+20160511
 google-pubsub1                               | 1.0.2+20161122
 google-pubsub1_beta2                         | 1.0.2+20161122
 google-replicapool1_beta2                    | 1.0.2+20160512
 google-resourceviews1_beta2                  | 1.0.2+20160512
 google-safebrowsing4                         | 1.0.2+20160520
...

I am not sure why we got different orders but getting the right order with git2 diff comparison was also an issue with my old code.

Thanks for the clarification! But wouldn't that mean that on the landing page, the most recent google crate should at best be 6 minutes old? It seems odd that the most recent release is said to be done an hour ago. Maybe the release specifies the actual release date in crates.io, and not when the documentation was built on docs.rs?
The order reported by find_changes() is dependent only on the diff between two arbitrary branches - to reproduce correct ordering, I believe it would have to step along the history and diff each commit separately.
However, another point of confusion is that docs.rs seems to have yet another order, maybe it's related to this sorting? It's not that I would think this is an actual issue, but it certainly added to my initial fear that it might not build all the docs.

Yes, release date is actual release date in crates.io.
id is just an auto incremented number. We are adding new crates to bottom of que with each INSERT in get_new_crates. There might be some logical error in this since we are also checking new crates every 5 minutes.
I am not sure why you have any fears but docs.rs was only skipping some crates because they weren't added properly into queue and thanks to you this problem is solved now. I only manually removed google-* crates from que in the past once or twice, because they were taking too long to build and you always released all of them at once.
BTW bottom of the queue right now:
  id   |                     name                     |    version
-------+----------------------------------------------+----------------
 16271 | google-youtubeanalytics1-cli                 | 1.0.2+20161213
 16270 | google-doubleclicksearch2-cli                | 1.0.2+20161108
 16269 | google-coordinate1-cli                       | 1.0.2+20150811
 16268 | google-appengine1_beta5-cli                  | 1.0.2+20161208
 16267 | nanomsg-sys                                  | 0.6.1
 16266 | nanomsg                                      | 0.6.1
 16265 | google-taskqueue1_beta2-cli                  | 1.0.2+20160428
 16264 | google-tagmanager1-cli                       | 1.0.2+20160310
 16263 | google-serviceregistryalpha-cli              | 1.0.2+20160401
 16262 | google-games1-cli                            | 1.0.2+20161207
 16261 | google-drive3-cli                            | 1.0.2+20161212
 16260 | google-appengine1_beta4-cli                  | 1.0.2+20161208
 16259 | google-appengine1-cli                        | 1.0.2+20161208
 16258 | google-webfonts1-cli                         | 1.0.2+20160302
 16257 | google-storagetransfer1-cli                  | 1.0.2+20150811

I think there is definitely a logical error in my INSERT and SELECT statements and 5 min timeouts between each get_new_crates. But since every new crate is getting added to que, I am fine with it.

Thanks again for letting me know!
I think what confuses me is that it can take very long until there is an update on the release page, clearly longer than 6 minutes. Maybe it's waiting for the appveyor builds, so the total time till all docs are finished is a multiple of that?
I should probably look more closely on how it works, as my use-case seems to push some limits to the point where I am blocking docs.rs for ours all by myself. After all, one release means ~140 crates, of which only 70 generate meaningful docs, with a total of 500kLOC.
Something I could imagine is to make it easy to add additional workers and some scheduling logic that prevents crates known to take a long time to use all of them.
But I am rambling - let's just wait for now :).

@Byron you can see the list of crates in queue now: https://docs.rs/releases/queue
Since crate order in queue and actual release time is a bit different, its hard to see new crates in the main page.

This is so awesome! Thanks for sharing that. I never clicked the 'recent releases' link on the main page, but will surely do so more often now! 174 crates are queued at the time of writing, incredible.
Something that would be awesome for my case is if instead of showing this (for example when clicking the documentation link of some google crate...

...one would show that it is queued, maybe alongside the queue number to give a hint how long it might take till it shows up, maybe like that:

Doing this would solve my major problem, which is producing dead links right after uploading to crates.io.
If you think it would be nice to have, I could possibly contribute this feature, even though so far I have not brought up a server on my own, but would work to make that easier as well.


[WORKFLOWS-67] Create more granular budget alerts

I've adapted the budget template from aws-infra and added cost filters. This way, we can create more granular budget alerts.
I propose three budgets: compute costs, storage (S3) costs, and everything else. The compute costs will spike, so the threshold on that is higher and will probably need to be adjusted once we have a better sense of compute needs. The storage costs will gradually increase as more projects are onboarded and more workflows are run. The "other" category should capture basic operating costs, which are generally stable. This would have caught the NAT gateway traffic that happened in September. I've set a purposely low threshold on this.
The thresholds were estimated based on current costs in our nextflow-prod AWS account. We can tweak these later.
We could create a daily budget report in a separate PR. I wanted to tackle this first so I can report back to AWS Support.

LGTM. Will you have to continue adding to the "other" list or is this "other" list curated already to services commonly used at sage?

The "other" list is everything else that appeared in the Budgets console. Unfortunately, I cannot find a comprehensive list of services with the exact spelling that Budgets expects. Unfortunately, I had to scrape the current values from the "Create Budget" webpage (see docs for serviceFilters param for more details).
So, these services are probably the ones that we've already encountered in our account. If we wanted to be even more comprehensive, I would need access to scrape the Budgets page for the AWS account that has the most diverse costs (presumably the account that is at the head of the AWS Organization).

LGTM


fix(combine-api): fixed JSON serialization for Swagger app

[x] Closes #4292
[x] Closes #4291
[x] Addresses #4283
[x] Renames minPolicy to maxPolicy (substitution policy)

:tada: This PR is included in version 9.8.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Move out Storybook into it's own workspace

We've got a real issue with upgrading certain dependencies, looking at you webpack 👀 , because we have two different versions of webpack v4, and v5 running side by side, and the types are all over the place for the webpack plugins and libraries.
We have a stable build right now, but trying to upgrade dependencies #8793 #8548 are proving to be a massive time sink as we need to get the correct modules installed in the correct places for our typescript build to pass with skipLibCheck: false.
Not hoisting the packages from the storybook workspace has worked for now, but it would be really nice if we just had one version of webpack and all the dev-server dependencies in the project.
Until storybook starts to use v5 out of the box, there's little we can do, as you can use v5 with storybook, but it will also try and bring in v5 which makes the whole thing even worse.
Let's move out Storybook into it's own yarn workspaces, that requires a seperate yarn install to get up and running and would install things in storybook/node_modules seperate with it's own yarn.lock.
Maybe there's also a possibility to skip the build tooling of storybook entirely and just get it to generate something that we can hook into our own build system instead.

hi am interested to work on it but I need some help can I work  on it?

So I think it should be a case of removing storybook from packages and placing it at the root level under storybook/ and then making sure that all the stories still work, and that any workflows that run storybook need to do a yarn install in the storybook directory as well as a normal yarn install. Feel free to go ahead and see if you can get this working, maybe there could also be other dragons which make this a little tricky and not a good first issue. 😅

Fixed! Thanks @snehaljos!


Dracula theme for TTY

Redefines the 16 ANSI colors in a TTY to dracula. I have absolutely no idea what to put as the website hahaha. Link

Looks awesome! I love it  🎉 🎉 🎉
Would you mind transferring it?

One day I'll manage to submit a theme without needing one of your "Standardize
repository" commits... anyway thanks for adding it :tada:

Hehe that was a super tiny one ;)

Aaaand we're live: https://draculatheme.com/tty :)


[WIP] Internals rework

Opening in draft mode because it requires revisiting the documentation side of things and perhaps a little cleanup.
The essence of the changes is as follows:

The core module has been refactored to remove unnecessary abstraction between the SSE transport and the STDIO transport.
Tests have been improved and no longer complain about multiple subscriptions to single-subscriber Sink.
SSE transport layer is now assuming non-blocking execution and no longer makes thread hops when not needed.

Thanks @chemicL . Great refactoring.

Rebased, add some additional improvements:

Rename transport classes to better reflect client-side role (*ServerTransport -> *ClientTransport)
Introduce fluent builder pattern for client configuration
Simplify DefaultMcpSession by removing ObjectMapper dependency
Move unmarshal responsibility to transport layer
Update documentation and diagrams to reflect new architecture
Clean up test implementations
and merged at 7a98a935af83b4975f2b12955781a3946aa5707d


Update protect-documents-that-have-fci-or-other-properties.md

Update for Bug 1956254

@chrfox please review

@JohanFreelancer9 - kindly do the copy editor review for this PR. Thanks


malc segfault

Hi,
I get the below segfault when running logger-bench with both it's default commit of malc and with cpp11-dev branch. I'm unable to reproduce with a debug build, only release.
$ gdb logger-bench /apps/cores/logger-bench.18141.1577787354

GNU gdb (GDB) Red Hat Enterprise Linux 8.2-3.el7

Copyright (C) 2018 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law.

Type "show copying" and "show warranty" for details.

This GDB was configured as "x86_64-redhat-linux-gnu".

Type "show configuration" for configuration details.

For bug reporting instructions, please see:

<http://www.gnu.org/software/gdb/bugs/>.

Find the GDB manual and other documentation resources online at:

    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".

Type "apropos word" to search for commands related to "word"...

Reading symbols from logger-bench...done.

[New LWP 18141]

[New LWP 18142]

[Thread debugging using libthread_db enabled]

Using host libthread_db library "/lib64/libthread_db.so.1".

Core was generated by `./logger-bench 8MB 16 own 1 all'.

Program terminated with signal SIGSEGV, Segmentation fault.

#0  0x0000000000000030 in ?? ()

[Current thread is 1 (Thread 0x7f7c5f79f780 (LWP 18141))]

Missing separate debuginfos, use: debuginfo-install glibc-2.17-196.el7.x86_64 libgcc-4.8.5-16.el7.x86_64 libstdc++-4.8.5-16.el7.x86_64

(gdb) bt

#0  0x0000000000000030 in ?? ()

#1  0x00007f7c5e95c9ed in ?? () from /lib64/libgcc_s.so.1

#2  0x00007f7c5e95cf0b in _Unwind_Resume_or_Rethrow () from /lib64/libgcc_s.so.1

#3  0x00007f7c5eec3ccc in __cxa_rethrow () from /lib64/libstdc++.so.6

#4  0x00007f7c5eec5a1f in __gnu_cxx::__verbose_terminate_handler() () from /lib64/libstdc++.so.6

#5  0x00007f7c5eec3a36 in ?? () from /lib64/libstdc++.so.6

#6  0x00007f7c5eec3a63 in std::terminate() () from /lib64/libstdc++.so.6

#7  0x00007f7c5eec3c83 in __cxa_throw () from /lib64/libstdc++.so.6

#8  0x00000000004069fc in malc_base::terminate() [clone .cold.70] ()

#9  0x000000000040e47d in own_subcommand(int, char**, int, unsigned long, bool) ()

#10 0x000000000040a924 in main ()

Seems like a racy termination. Not sure if the bug is in the bench or in malc.
Can you run with malc-tls only? I suspect this:
https://github.com/RafaGago/mini-async-log-c/blob/96cc576b4aabe0e6353d50c20cc0c460b938bf52/src/malc/malc.c#L531

Btw thanks for reporting

From your trace it seems that you were just running 1 iteration of 16 messages. Does the bug happen always or very often?
Did you compile "logger-bench" only with malc or were another (potentially singleton) loggers enabled?
It seems that the termination function got inlined. "malc_base::terminate" is not a destructor and the only thing it does is to call malc_terminate (false).
I tried this while true; do ./logger-bench 8MB 16 own 1 all && rm *.log || break ; done for 30 minutes without luck. I compiled only with -DMALC=on. -DCMAKE_BUILD_TYPE=Release. Ubuntu 18-04. Vanilla gcc 7.4.0 both on an i5 laptop and on an old AMD Phenom x4.
Maybe you can compile with -DCMAKE_BUILD_TYPE=RelWithDebInfo, check if the bug still and see if you can load the core dump.

I was able to reproduce by installing gcc8.
I was starting to think that this was going to be a hard bug, but it's actually pretty trivial:
94a286bbb48f32d4837f5b0554c3faa3491704b2
This did the trick for me. It's very weird that gcc-7 was OK with this.
Say if it works for you and again, thanks for reporting :)

That fixed the issue for me, thanks.
On Thu, 2 Jan 2020, 6:30 pm RafaGago<EMAIL_ADDRESS>wrote:

I was able to reproduce by installing gcc8.
I was starting to think that this was going to be a hard bug, but it's
actually pretty trivial:
94a286b
https://github.com/RafaGago/logger-bench/commit/94a286bbb48f32d4837f5b0554c3faa3491704b2
This did the trick for me. It's very weird that gcc-7 was OK with this.
Say if it works for you and again, thanks for reporting :)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/RafaGago/logger-bench/issues/5?email_source=notifications&email_token=AL5LS3D5ZYPQ5MSCUQQ6GEDQ3X3B5A5CNFSM4KBUQ7EKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEH6O4FA#issuecomment-570224148,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AL5LS3A3U5NGIQZHPBBJOELQ3X3B5ANCNFSM4KBUQ7EA
.


CC Status Page

The CockroachCloud status page displays the current uptime status of the CC Console as well as other related components that affects a large amount of customers (e.g.,a region outage in a cloud provider). The status page also allows customers to subscribe to updates so they can be notified of unavailability. The status page is publicly available to all CC users - paid and free, and will be linked from the marketing website as well as the CC Console UI.
The status page can be found here: https://cockroachcloud.statuspage.io/ (this page is currently hidden)
Full feature epic can be found here: https://cockroachlabs.atlassian.net/browse/CC-399

Closing in favor of CC-3681