[Request] Decentralized Ecosystem Map

Let's create an easy to use, decentralized ecosystem map in the polkadot-developers org.

@shawntabrizi why do we need this now that we have projects listed on the official website? (https://polkadot.com/get-started/dapps)

That is fair, perhaps we don't need a third party site, but Distractive, the team that made the Polkadot site, directly referenced this third part site on their brochure:
https://distractive.notion.site/Polkadot-Booklet-26dbbb9392954fa1a00c66da9acb9f34
We need a way to make sure that any list is up to date.
It could also be that the Polkadot site uses a more specific list than a broad one like the third party.
In any case, out of date sites for Polkadot make the ecosystem look stale / bad.

I see your point. The only way I can think of solving this is having one central conversion funnel where any serious project in the ecosystem will list themselves and update their profile when there's a change - user ratings may be useful for ranking, but could be gameable.
Regarding, I think the main Polkadot.com list is the way to go. We would need distractive to make periodic tweets from the Polkadot account featuring community projects to encourage people to actively list their work.

@iammasterbrucewayne I think you are right about the source of truth being somewhere in Polkadot. I will update the issue accordingly.
i think i would like to take a "decentralized" approach.
Ideally we have a repo which has a JSON list of projects, their websites, their icon, their github etc...
Then each day/week/month, we run a script which checks:

all the sites are still active
somewhere on the site they have "powered by polkadot" or similar branding
their github has seen activity in the last 3 months

Then, each project would also have a Polkadot account, and we send everyone a super small, incentive, like 20 DOT per month, if this data works.
Fake projects or grifters could be called out by community, who would then earn their 20 DOT instead.
This would be funded by treasury, and even with 1,000 projects, not be super expensive, and at least DOT would go into the right hands.

I think this is amazing and I fully support it. This will help libraries and small teams to keep working. I have written a thread based on this scenario. We all know that Web3 Foundation has funded 1000s of projects and I can assure that half of the projects are not working or they have been archived. Awesomedot is a good repo and they have been working on this for a long time and if Fardeen can push this forward. We can create a good visible Decentralized Ecosystem Map for Polkadot ecosystem.

Relevant Forum Threads:

https://forum.polkadot.network/t/visibility-initiative-for-funded-projects/10604/15
https://forum.polkadot.network/t/awesomedot-central-library-of-polkadot-ecosystem/6529/4

@faultcore what would be needed for you to maintain the project and take it to the next level?
If you are passionate about making a project like this work, I would be happy to help find a way to make it happen :)

Also, I think you can probably reduce the work by using a no-code tool with a directory template given the goal is to maintain a list - this will free up your time and allow you to focus on the critical part i.e. maintaining the list and potentially marketing it.


Update 0000-01-02-Tamiren.md

made change to title

merge commits

merge commits

merge pull requests


chore: fixes compile error

What :computer:

Update dependencies to latest on zksync-era
Updates node.rs and zk.rs to return a BoxFuture type from the jsonrpc_core library

Why :hand:

To reflect the latest changes in zksync-era and fix errors with compiling and running cargo install

Evidence :camera:
Run: make rust-build

Notes :memo:
Closes #13 Closes #14

@mm-zk updated with requested changes.


Bind librechat config file in the docker-compose

Pull Request Template
⚠ Before Submitting a PR, read the Contributing Docs in full!
Summary
By default the librechat.yaml was not bound to the proper location, so my setup was erroring that it couldn't find this file.
Change Type
Please delete any irrelevant options.

[x] Bug fix (non-breaking change which fixes an issue)

Checklist
Please delete any irrelevant options.

[x] My code adheres to this project's style guidelines
[x] I have performed a self-review of my own code
[x] I have commented in any complex areas of my code
[x] I have made pertinent documentation changes
[x] My changes do not introduce new warnings
[x] I have written tests demonstrating that my changes are effective or that my feature works
[x] Local unit tests pass with my changes
[x] Any changes dependent on mine have been merged and published in downstream modules.
[x] New documents have been locally validated with mkdocs

You need to bind a librechat.yaml in docker-compose.override.
See https://github.com/danny-avila/LibreChat/blob/main/docker-compose.override.yml.example
IMO it's a little weird but the deployment works like this.

The reason for this to not make the librechat.yaml file a required step for spinning up the default compose file. Otherwise, if no file is present, there will be an error.

The reason for this to not make the librechat.yaml file a required step for spinning up the default compose file. Otherwise, if no file is present, there will be an error.

It would be good to have this documented somewhere. The docs for the config file do not mention where to mount it, and it took me like 30 minutes to figure out. Also I'm not using the compose override file since I deploy via Dockge (https://github.com/louislam/dockge) and I believe it doesn't support them.

(...) Also I'm not using the compose override file since I deploy via Dockge (https://github.com/louislam/dockge) and I believe it doesn't support them.

I'm using portainer, and to make my config file management easier I host it on GitHub and use the env var CONFIG_PATH to provide the URL that point to the raw file

The reason for this to not make the librechat.yaml file a required step for spinning up the default compose file. Otherwise, if no file is present, there will be an error.

It would be good to have this documented somewhere. The docs for the config file do not mention where to mount it, and it took me like 30 minutes to figure out. Also I'm not using the compose override file since I deploy via Dockge (https://github.com/louislam/dockge) and I believe it doesn't support them.

It’s in the docs, it shows how to bind it via override file, which can be applied like you did in the PR. I do think we could be clearer about the CONFIG_PATH option especially the http request option


Be careful if you are using this package!

Here is an example of why:
X = X.matrix()
rf.fit(X, y, feature_names=features)

The function 'fit' accepts a matrix which converts you integers to floats. Then if you have a column with integer value, for example, gender (0, 1) then the code will create the rules that don't make sense. For instance, it will create a rule 'when gender  > 0.5'. But it should create 'when gender  == 1' or 'when gender == 0'.

That's common behaviour of decision trees and random forests for binary trees.
I agree it would be nicer to directly encode binary features as factors and not as numeric feature.
I'd argue that if you know how your data is encoded (e.g. gender 0 = male, gender 1 =female) and how trees work, that you will correctly interpret gender > 0.5 as "if female"
But it would be nice to directly encode the binary feature as a factor.
Do you know of a good solution for treating factors in Python?

Currently there's no good solution for categorical data in decision trees.
sklearn is working on something (see here) but so far it hasn't been merged.
It is generally accepted that one-hot encoding and the >0.5 or <= 0.5 splits will produce a comparable model in terms of prediction accuracy, but maybe not as interpretable.

One possible solution for this package is to pass X as a data frame but not as a matrix. Whenever you need to access the column, then instead
X[:, i_col]
you could use
X.iloc[:, icol]
This will pass integers. sklearn accepts dataframes so you package would be consistent with it. Then it would be sklearn's responsibility how to treat integers.

These 0.5 splits come from sklearn not from rulefit.
It's the implementation of their trees that create the floats.
As far as I am aware sklearn does not work with DataFrames, they even try to avoid the extra dependancy on pandas, see the readme. The reason why dataframes 'work' in sklearn is that they convert it to numpy arrays behind the scenes (lists of lists also work because of that). At any rate the integer datatype is stored in the numpy array metadata (dataframes are just numpy arrays with index, anyway).
In practice this is a non-issue for binary variables once you convince yourself that x > 0.5  is true only when x==1. Some postprocessing on the resulting rules might be needed for visualisation, but as far as the ML task the representation is equivalent.
It's only a concern for non-ordinal factors (e.g, weather = {'sunny', 'snowy', 'rainy'}). In this case you would need to do one-hot encoding and this is when some proper handling of categorical variables might be better. But to be fair, you would already need to have your data encoded for the 'linear' bit of rulefit anyway as regression does not work with categories either.
Unless there is some categorical random tree implementation in python that I am not aware of, I think this is better discused in sklearn repo, for instance Issue 5442, PR 4899 as there is little that can be done here besides maybe a FAQ point in the documentation.


Do something with the private JSON data

Is there any plan yet for using the private JSON data?
The obvious way would be to display the exact same map than the public one, but with all private information on it.
The question is how to to authenticate access to this private map.  It can either be done at the webserver level, or enforced directly in wifi-with-me.  Maybe a simple list of login/password would do?  Or it there a generic way to do authentication with bottlepy?

First thing I see would be to display a count of how many private records are stored, just in order not to forget them.
Maybe a second map, requiring auth would do the job then. I think not implementing accounts system is a good thing. We all have many means to authenticate our users already... A proposal :

define in WWM conf a key (think api-key)
create two endpoints, both of them require the api-key to be set in conf and a named header (ex: X-API-Key) on protected request to match it.

/map-private/ a leaflet map with both private.json and public.json
private.json

this key can then either be used directly by client, or, far better, configure the httpd to add this header and to do auth/authz at HTTP level

(simple) related idea in #26

here is a stub of a djago admin web interface for wifi-with-me
https://gitlab.com/vindarel/wifi-with-django
it's supposed to become a full django rewrite of wifi-with-me someday

Cool :-)

@bikepunk @vindarel when it will come to be a full rewrite, I'm ok if it's a replacement of the current wifi-with-me. I see no real reason to keep bottle and I'm ok to drop it (@opi @zorun  what do you think ?).
To put it short, @vindarel do as you fancy, but if you want to start working on a branch here, you're more than welcome :-).
(We could also take this opportunity to move to https://code.ffdn.org/ ?)

https://code.ffdn.org/FFDN/wifi-with-me/issues/13


disco_l475_iot1 don't confirm MCUBoot slot-1 image

Describe the bug
When build UpdateHub for disco_l475_iot1 the app don't confirm the image.
To Reproduce
Steps to reproduce the behavior:

clone https://github.com/UpdateHub/zephyr/commits/topic/uhu_wifi
set variables at prj.conf
CONFIG_UPDATEHUB_WIFI_SSID="wifi network"
CONFIG_UPDATEHUB_WIFI_PSK="password"
west build -b disco_l475_iot1 samples/net/updatehub
sign image
~/<zephyr_home_dir>/bootloader/mcuboot/scripts/imgtool.py sign --key ~/<zephyr_home_dir>/bootloader/mcuboot/root-rsa-2048.pem --header-size 0x200 --align 8 --version 1.0.0 --slot-size 0x6C000 ~/<zephyr_home_dir>/zephyr/build/zephyr/zephyr.hex ~/<zephyr_home_dir>/zephyr/build/zephyr/zephyr-1.0.0.hex
west flash --hex-file build/zephyr/zephyr-1.0.0.hex
See error

Expected behavior
boot_write_img_confirmed() executes properly on main.c
Impact
MCUBoot applications don't work.
Screenshots or console output
*** Booting Zephyr OS build zephyr-v2.2.0-2058-g1e3fd042505e  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.007,000] <inf> mcuboot: Primary image: magic=good, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.007,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.007,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.015,000] <inf> mcuboot: Swap type: none

uart:~$ *** Booting Zephyr OS build zephyr-v2.2.0-2358-gf071c82bc516  ***

[00:00:00.539,000] <inf> main: UpdateHub sample app started
[00:00:00.539,000] <inf> main: Confirming the boot image
[00:00:00.539,000] <err> main: Error to confirm the image  <<<<<<< PROBLEM
uart:~$ Connected <<< WIFI got connected >>>>
*** Booting Zephyr OS build zephyr-v2.2.0-2058-g1e3fd042505e  *** <<<<< AUTO REBOOT
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.007,000] <inf> mcuboot: Primary image: magic=good, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.007,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.007,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.015,000] <inf> mcuboot: Swap type: none

Environment (please complete the following information):

OS: Linux
Toolchain Zephyr SDK 0.11.2
master

CC @otavio @erwango

^^@ABOSTM

@nandojve Is this TLS enabled?

Hi @parthitce, if you are referring to UpdateHub, yes, it works with/without DTLS.
Take look at https://github.com/zephyrproject-rtos/zephyr/tree/master/lib/updatehub and https://github.com/zephyrproject-rtos/zephyr/tree/master/samples/net/updatehub.
You can get more info at https://updatehub.io/
About MCUBoot, you must use your own certificate to sign images.
https://github.com/JuulLabs-OSS/mcuboot/blob/master/docs/readme-zephyr.md
To get MCUboot working you need SoC flash stuff working properly, as far I know. With that, you can get UpdateHub working if you have a network interface. You can use a wifi shield like ESP_8266 or any other available. The shield is basically wifi config + board overlay.
https://github.com/zephyrproject-rtos/zephyr/tree/master/boards/shields/esp_8266
This way, you don't need a full ethernet available to test it, only a serial 115200 with RX/TX.

Hi @nandojve,
It would be interresting to determined whether issue comes from wifi as opposed to ethernet.
So I tried to run the samples\net\updatehub\ from original zephyr on STM32F429 which support Ethernet.
I followed instructions from samples\net\updatehub\README.rst
Unfortunately after spending about 1 day and a half, I am unable to have updateHub image running with docker.

$ docker run -i -p 8080:8080 -p 5683:5683/udp --rm updatehub/updatehub-ce:latest
Unable to find image 'updatehub/updatehub-ce:latest' locally
C:\Program Files\Docker\Docker\resources\bin\docker.exe: Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers).

I suspect Proxy issue, I tried to set proxy environnement variables, and also tried to add proxy to config.json, but it doesn't help.
I looked for precompiled updateHub image for windows to download, (so that it could be available locally) but I could not found it.
I also tried to recompile it, but I face again connection issue when installing Packr utility:

$ go get -u github.com/gobuffalo/packr/packr
go<EMAIL_ADDRESS>Get<EMAIL_ADDRESS>dial tcp [2a00:1450:4007:816::2011]:443: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

Would you be able to test on your side the same: From original Zephyr repo, on any STM32 board with Ethernet (instead of Wifi) ?

Hi @ABOSTM,
Docker procedures:
docker pull updatehub/updatehub-ce
# get container_id if updatehub docker is running
docker ps
docker stop container_id
# run       local-port:docker-port
docker run -it -p 8081:8080 -p 5683:5683/udp --rm updatehub/updatehub-ce:latest
# How to access (localhost == CONFIG_UPDATEHUB_SERVER)
http://localhost:8081/
# login admin
# pass admin

Current master need update timer variable because timer changes, we'll open issue and send PR:
diff --git a/lib/updatehub/updatehub.c b/lib/updatehub/updatehub.c
index e50b4503cf..a61450f7f3 100644
--- a/lib/updatehub/updatehub.c
+++ b/lib/updatehub/updatehub.c
@@ -32,7 +32,7 @@ LOG_MODULE_REGISTER(updatehub);
 #include <net/tls_credentials.h>
 #endif

-#define NETWORK_TIMEOUT K_SECONDS(2)
+#define NETWORK_TIMEOUT 2000
 #define UPDATEHUB_POLL_INTERVAL K_MINUTES(CONFIG_UPDATEHUB_POLL_INTERVAL)
 #define MAX_PATH_SIZE 255
 /* MAX_PAYLOAD_SIZE must reflect size COAP_BLOCK_x option */

Update prj.conf file for convenience:
diff --git a/samples/net/updatehub/prj.conf b/samples/net/updatehub/prj.conf
index 603ea45e41..776ed4b05b 100644
--- a/samples/net/updatehub/prj.conf
+++ b/samples/net/updatehub/prj.conf
@@ -18,3 +18,10 @@ CONFIG_UPDATEHUB_SHELL=y
 # Debug helpers
 CONFIG_LOG=y
 CONFIG_UPDATEHUB_LOG_LEVEL_INF=y
+
+#Example of product_uid, you must set you product uid.
+CONFIG_UPDATEHUB_PRODUCT_UID="e4d37cfe6ec48a2d069cc0bbb8b078677e9a0d8df3a027c4d8ea131130c4265f"
+CONFIG_UPDATEHUB_POLL_INTERVAL=1
+CONFIG_UPDATEHUB_CE=y
+# Local network IP, the machine that will run docker
+CONFIG_UPDATEHUB_SERVER="<IP_ADDRESS>"

Build for frdm_k64f board
# Just to make sure everything is on good shape
rm -rf build
rm -rf .uhu

# build
west build -b frdm_k64f samples/net/updatehub

# sign image that you will flash using JTAG, for instance:
~/zephyros/mainline/bootloader/mcuboot/scripts/imgtool.py sign --key ~/zephyros/mainline/bootloader/mcuboot/root-rsa-2048.pem --header-size 0x200 --align 8 --version 1.0.0 --slot-size 0x60000 /home/gfbudke/zephyros/mainline/zephyr/build/zephyr/zephyr.bin /home/gfbudke/zephyros/mainline/zephyr/build/zephyr/zephyr-1.0.0.bin

# sign image that you will use to upgrade using UpdateHub:
~/zephyros/mainline/bootloader/mcuboot/scripts/imgtool.py sign --key ~/zephyros/mainline/bootloader/mcuboot/root-rsa-2048.pem --header-size 0x200 --align 8 --version 2.0.0 --slot-size 0x60000 /home/gfbudke/zephyros/mainline/zephyr/build/zephyr/zephyr.bin /home/gfbudke/zephyros/mainline/zephyr/build/zephyr/zephyr-2.0.0.bin

# note: for frdm_k64f --slot-size is 0x60000, check flash slot partition:
# for disco_l475_iot1.dts
#		slot0_partition: partition@20000 {
#			label = "image-0";
#			reg = <0x00020000 0x0006C000>;
#		};

# create uhu pakage:
# CONFIG_UPDATEHUB_PRODUCT_UID from prj.conf
uhu product use "e4d37cfe6ec48a2d069cc0bbb8b078677e9a0d8df3a027c4d8ea131130c4265f"

# make sure put full relative path (build/zephyr):
uhu package add build/zephyr/zephyr-2.0.0.bin -m zephyr

# set package version
uhu package version 2.0.0

# you can use other output dir
uhu package archive --output build/zephyr/zephyr-2.0.0.pkg

Flash initial image:
west flash --bin-file build/zephyr/zephyr-1.0.0.bin
Server not found, no network, etc:
uart:~$ updatehub run
Starting UpdateHub run...
Invalid response
[00:01:38.354,000] <err> updatehub: Could not receive data
uart:~$

OK, now run docker container and login:
docker run -it -p 8081:8080 -p 5683:5683/udp --rm updatehub/updatehub-ce:latest
Server running and communication is OK:
uart:~$ updatehub run
Starting UpdateHub run...
No update found
[00:02:03.067,000] <inf> updatehub: No update available

Check that machine is connected at devices:
Setup update:
Use UpdateHub CE web interface to upload build/zephyr/zephyr-2.0.0.pkg at packages
Program a upload:
Goto rollouts and create a new one:
Wait or force upload at board's terminal:
uart:~$ updatehub run
Starting UpdateHub run...
[00:01:18.844,000] <inf> updatehub: Probe metadata received
[00:01:20.353,000] <inf> updatehub: Firmware downloaded successfully
[00:01:20.391,000] <inf> updatehub: Image flashed successfully, you can reboot now
uart:~$ kernel reboot cold *** Booting Zephyr OS build zephyr-v2.2.0-2058-gc2f118723970  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=good, swap_type=0x2, copy_done=0x1, image_ok=0x1
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: none
[00:00:00.006,000] <inf> mcuboot: Swap type: test
*** Booting Zephyr OS build v2.3.0-rc1  ***

uart:~$ updatehub info
Unique device id: ffffffff4e454130000b000f
Firmware Version: 2.0.0
Product uid: e4d37cfe6ec48a2d069cc0bbb8b078677e9a0d8df3a027c4d8ea131130c4265f
UpdateHub Server: <IP_ADDRESS>
[00:00:03.000,000] <inf> eth_mcux: ETH_0 enabled 100M full-duplex mode.
uart:~$

If you have network running and flash works with MCUboot you will be able to reproduce above steps. If you have any trouble, please, let me know!

@nvlsianpu is this related to the confirmation bug in mcumgr that you detected the other day?

@carlescufi ^^ not related to that one (was issue caused by different NCS partition manager configuration).
@nandojve Is this bug is regression to something which used to work before, or it was newer tested on this SoC? Confirmation problems suggest me an issue with the flash write operation to write area, as confirmation is just the write of proper data to certain flash addresses.

Hi @nandojve ,
After you provided me the package I could not generate, I continued on Ethernet use case on nucleo_f429zi.
I finally get IP address with DHCP (I had issue with my PLC to connect to my internet box)
But unfortunately I spent a lot of time and I am stuck again:
my updateHub server is accessible from web browser running on the same PC running the server,
but the server is not accessible from another PC even if running on the same local network.
In other word, updatehub server is isolated and cannot be accessible by any external device.
As I understand, generally speaking, updatehub is working with ESP_8266 shield.
Would it be possible for you to test updateHub disco_l475_iot1 with this ESP shield on ?
(Deactivating disco_l475_iot1 wifi module, and enabling ESP shield)

Hi @ABOSTM, by the docker command line you must have UDP port 5683 open to work.
If for some reason this port is blocked you can't communicate with docker service.
Anyway, my problem to advance with disco_iot is related to the logs:
[00:00:00.539,000] <inf> main: Confirming the boot image
[00:00:00.539,000] <err> main: Error to confirm the image  <<<<<<< PROBLEM
uart:~$ Connected <<< WIFI got connected >>>>

That message is printed by the following code at samples/net/updatehub/main.c
	/* The image of application needed be confirmed */
	LOG_INF("Confirming the boot image");
	ret = boot_write_img_confirmed();
	if (ret < 0) {
		LOG_ERR("Error to confirm the image");
	}

So, my problem is, boot_write_img_confirmed need be OK to system work. It seems something related to flash on disco_iot. This issue you can check without UpdateHub-CE be present. Be sure trhat NET_CONFIG_SETTINGS option is false. Depending the code you are looking my be necessary adjust it on zephyr/lib/updatehub/Kconfig.
https://github.com/UpdateHub/zephyr/commit/6db1c5a874e9bedf6cffe6d5b93ebd60cd6dcc8e
About WIFI, I have both es-WIFI and ESP_8266 working, so yes, I can test and switch if necessary.

Hi @nandojve ,
I intend to put a debugger to understand why boot_write_img_confirmed() return is not OK, to know which step fails with which error code ... This is why I tried to get updateHub up and running.
If you know a way to reproduce the issue without the server, please let me know .
For example, would it be possible to flash the update image ( which one exactly?? including probably the necessary header describing the image ...) to the flash, so that I could be in the same state than after updateHub would have written flash with the update image? and thus be able to reproduce the issue.
This suppose writting the update image was fully successful ..

@nandojve Is this bug is regression to something which used to work before, or it was newer tested on this SoC? Confirmation problems suggest me an issue with the flash write operation to write area, as confirmation is just the write of proper data to certain flash addresses.

I never got UpdateHub working on that board. I've been raising issues related with hope to get disco_iot working at some moment. My feeling is that solving the boot_write_img_confirmed (this issue) and assuming flash write is OK probably UpdateHub will start to works.

Hi @nandojve ,
I intend to put a debugger to understand why boot_write_img_confirmed() return is not OK, to know which step fails with which error code ... This is why I tried to get updateHub up and running.
If you know a way to reproduce the issue without the server, please let me know .
For example, would it be possible to flash the update image ( which one exactly?? including probably the necessary header describing the image ...) to the flash, so that I could be in the same state than after updateHub would have written flash with the update image? and thus be able to reproduce the issue.
This suppose writting the update image was fully successful ..

@ABOSTM it is mandatory apps confirm the image on first boot otherwise MCUBoot will must roolback on next reboot. I think this is not fully addressed on current MCUBoot samples on Zephyr. The consequence is we can't see regressions or even know that MCUBoot is 100% functional and supported.
My suggestion is add boot_write_img_confirmed at hello word targeting to MCUBoot (CONFIG_BOOTLOADER_MCUBOOT=y) and debug it. If boot_write_img_confirmed fail you can experiment board enter on a infinite reset loop because there isn't a plan B image. Take that on consideration because isn't a bug.
@nvlsianpu Is there a regression test to check boot_write_img_confirmed requirement?

@nandojve ,
Following your proposal, I test disco_l475_iot1 ,
I erase full flash, then flash MCUBOOT, then flash Helloworld example (signed image) with call to boot_write_img_confirmed()
But I did not face issue, function returns 0 wich is good:

*** Booting Zephyr OS build zephyr-v2.2.0-2072-g6102ba19f1ee  ***
[00:00:00.005,000]  mcuboot: Starting bootloader
[00:00:00.007,000]  mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x1
[00:00:00.007,000]  mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.007,000]  mcuboot: Boot source: primary slot
[00:00:00.015,000]  mcuboot: Swap type: none
*** Booting Zephyr OS build zephyr-v2.2.0-2072-g6102ba19f1ee  ***
Hello World! disco_l475_iot1
Confirming the boot image
Confirm the image return 0

Hi @ABOSTM thank you for the feedback!
Could you confirm SHA that you tested?

@nandojve
SHA1:5321c4288203c113abac5eab9eaca918e9b4f8f9

@ABOSTM I continue saw flash not be confirmed on your SHA and on the last one.
The only difference that raise a flag for me was that:
Your Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x1
mine Primary image: magic=good, swap_type=0x1, copy_done=0x3, image_ok=0x3
Below the procedure that I believe get last version of everything to make sure environment is not poisoned.
Please, add #25333 on top of master if not merged yet.
git checkout master
git fetch upstream
git rebase upstream/master
git lg -5
* 25a2f6c175 - (HEAD -> master) lib: updatehub: Add missing include for FLASH_AREA (34 minutes ago) <Gerson Fernando Budke>
* a0ae53410e - (upstream/master) sanitylib: Remove hardcoded sample.yaml & testcase.yaml in TestSuite class (5 hours ago) <Aastha Grover>
* 5496ea2d9b - soc: x86: apollo_lake: Fix incorrect DTS gpio nodelabel references (10 hours ago) <Kumar Gala>
* 845abb04cf - boards: qemu_xtensa: enable icount mode (14 hours ago) <Wentong Wu>
* bb80d3528e - boards: hifive1: enable icount mode (14 hours ago) <Wentong Wu>

After that, I always make sure that all modules are on the last version:
gfbudke@Capricornio:~/zephyros/mainline/zephyr$ west update
=== updating cmsis (modules/hal/cmsis):
HEAD is now at 542b229 DSP: Integrate CMSIS-DSP 1.8.0 (CMSIS 5.7.0)
=== updating hal_atmel (modules/hal/atmel):
HEAD is now at 1fe96f0 samv71: patch: Fix GMAC priority queue register offsets
=== updating hal_altera (modules/hal/altera):
HEAD is now at 23c1c1d hal_altera: Conditionally include the Altera HAL
=== updating canopennode (modules/lib/canopennode):
HEAD is now at 5c6b056 stack: remove all GNU GPL licensed platform drivers
=== updating ci-tools (tools/ci-tools):
HEAD is now at da9a2df labels: change test suite label
=== updating civetweb (modules/lib/civetweb):
HEAD is now at 99129c5e Merge pull request #778 from antmicro/zephyr-filtering
=== updating esp-idf (modules/hal/esp-idf):
HEAD is now at 6835bfc74 module: add zephyr integration
=== updating fatfs (modules/fs/fatfs):
HEAD is now at 9ee6b9b fs: Expose _USE_LFN, _MAX_LFN, and _CODE_PAGE as kconfig options.
=== updating hal_cypress (modules/hal/cypress):
HEAD is now at a12d928 zephyr: move kconfig to the zephyr tree
=== updating hal_infineon (modules/hal/infineon):
HEAD is now at f1fa824 hal: infineon: initial commit of XMCLib
=== updating hal_nordic (modules/hal/nordic):
HEAD is now at e5a1a6f nrf_radio_802154: Use chosen zephyr,entropy to get entropy device
=== updating hal_openisa (modules/hal/openisa):
HEAD is now at 3b54187 hal: openisa: pass MIC status from CAUv3 on stack
=== updating hal_microchip (modules/hal/microchip):
HEAD is now at aad89bf modules: Fix MEC1501 OOB RX/TX transfer length masks
=== updating hal_silabs (modules/hal/silabs):
HEAD is now at 78da967 hal_silabs: patch: rename ramfunc section to match Zephyr's
=== updating hal_st (modules/hal/st):
HEAD is now at 5b3ec3e hal_st: CMakelists: Update coding style to main repo
=== updating hal_stm32 (modules/hal/stm32):
HEAD is now at d1bc80d hal_stm32: rename HAS_STLIB in HAS_STM32LIB
=== updating hal_ti (modules/hal/ti):
HEAD is now at c398cc7 cc13x2_cc26x2: drivers: fix build warnings in PowerCC26X2.c
=== updating libmetal (modules/hal/libmetal):
HEAD is now at 3c3c9ec lib: update libmetal to release v2020.04.0
=== updating lvgl (modules/lib/gui/lvgl):
HEAD is now at 74fc2e75 Merge lvgl v6.1.1 into zephyr branch
=== updating mbedtls (modules/crypto/mbedtls):
HEAD is now at 8211541 config-tls-generic: Add CTR cipher mode
=== updating mcuboot (bootloader/mcuboot):
HEAD is now at e88113b scripts/assemble: Rework to use EDT library to get devicetree data
=== updating mcumgr (modules/lib/mcumgr):
HEAD is now at 1f5234c zephyr: Convert to new FLASH_AREA macros
=== updating net-tools (tools/net-tools):
HEAD is now at 1c4fdba docker: Add dante SOCKS 5 proxy
=== updating hal_nxp (modules/hal/nxp):
HEAD is now at 80a337d mcux: build LPC Entropy drivers
=== updating open-amp (modules/lib/open-amp):
HEAD is now at 724f7e2 lib: update open-amp lib to release v2020.04.0
=== updating loramac-node (modules/lib/loramac-node):
HEAD is now at 29e516ec radio: sx1276: allow overriding Buffer reand/write functions
=== updating openthread (modules/lib/openthread):
HEAD is now at a83d18cf cmake: Make sure libc is linked after OT libs
=== updating segger (modules/debug/segger):
HEAD is now at 6fcf616 ext: segger: update to SystemView v2.52h
=== updating tinycbor (modules/lib/tinycbor):
HEAD is now at 40daca9 zephyr: Remove TINYCBOR from interface libraries
=== updating tinycrypt (modules/crypto/tinycrypt):
HEAD is now at 3e9a49d cmake: Fix conditional in root CMakeLists.txt
=== updating littlefs (modules/fs/littlefs):
HEAD is now at 0aefdda Merge tag 'v2.2.0' into zephyr
=== updating mipi-sys-t (modules/debug/mipi-sys-t):
HEAD is now at 957d46b Delete initialization related code from mipi-sys-t tree
=== updating nrf_hw_models (modules/bsim_hw_models/nrf_hw_models):
HEAD is now at fec6970 Convert into a west module
=== updating hal_xtensa (modules/hal/xtensa):
HEAD is now at e7a57d0 hal: Add README for information on updating HAL versions.
=== updating edtt (tools/edtt):
HEAD is now at c39888f Update unsupported commands test to handle command status response
=== updating trusted-firmware-m (modules/tee/tfm):
HEAD is now at 7de2daa zephyr: cmake: TF-M libraries to be used by the ns-app
gfbudke@Capricornio:~/zephyros/mainline/zephyr$

Then, apply changes on hello_world
git stash pop
On branch master
Your branch is ahead of 'origin/master' by 162 commits.
  (use "git push" to publish your local commits)

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   samples/hello_world/prj.conf
	modified:   samples/hello_world/src/main.c

no changes added to commit (use "git add" and/or "git commit -a")
Dropped refs/stash@{0} (0c8e0a6f0547cf8acf2ba2c81e4b9f04aefe9636)

diff:
git diff
diff --git a/samples/hello_world/prj.conf b/samples/hello_world/prj.conf
index b2a4ba5910..08ea5c1f9a 100644
--- a/samples/hello_world/prj.conf
+++ b/samples/hello_world/prj.conf
@@ -1 +1,5 @@
 # nothing here
+CONFIG_BOOTLOADER_MCUBOOT=y
+CONFIG_IMG_MANAGER=y
+CONFIG_FLASH=y
+CONFIG_MPU_ALLOW_FLASH_WRITE=y
diff --git a/samples/hello_world/src/main.c b/samples/hello_world/src/main.c
index 6c5c8a27dc..4abe9174a6 100644
--- a/samples/hello_world/src/main.c
+++ b/samples/hello_world/src/main.c
@@ -7,7 +7,22 @@
 #include <zephyr.h>
 #include <sys/printk.h>

+#include <dfu/mcuboot.h>
+
 void main(void)
 {
+       int ret = -1;
+
        printk("Hello World! %s\n", CONFIG_BOARD);
+
+       /* The image of application needed be confirmed */
+       printk("Confirming the boot image\n");
+       ret = boot_write_img_confirmed();
+       if (ret < 0) {
+               printk("Error to confirm the image\n");
+       } else {
+               printk("Image is OK\n");
+       }
+
+       for(;;);
 }

build:
west build -b disco_l475_iot1 samples/hello_world
sign with --slot-size 0x6C000
~/zephyros/mainline/bootloader/mcuboot/scripts/imgtool.py sign --key ~/zephyros/mainline/bootloader/mcuboot/root-rsa-2048.pem --header-size 0x200 --align 8 --version 1.0.0 --slot-size 0x6C000 /home/gfbudke/zephyros/mainline/zephyr/build/zephyr/zephyr.hex /home/gfbudke/zephyros/mainline/zephyr/build/zephyr/zephyr-1.0.0.hex
I'm using west to flash image:
west flash --hex-file build/zephyr/zephyr-1.0.0.hex
Then, the unfortunately result:
*** Booting Zephyr OS build v2.3.0-rc1-102-g1383170029db  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=good, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.013,000] <inf> mcuboot: Swap type: none
*** Booting Zephyr OS build v2.3.0-rc1-100-g25a2f6c175e3  ***
Hello World! disco_l475_iot1
Confirming the boot image
Error to confirm the image

Hi @nandojve ,
I reinstalled Zephyr from scratch (e550302a61a088abe549e7bf4d3a2fd51d8971f3),
I merge your modification of hello world  example.
I build and flash MCUBoot , then build and flash example,
and again it is working fine:
*** Booting Zephyr OS build v2.3.0-rc1-122-ge550302a61a0  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x1
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.013,000] <inf> mcuboot: Swap type: none
*** Booting Zephyr OS build v2.3.0-rc1-122-ge550302a61a0  ***
Hello World! disco_l475_iot1
Confirming the boot image
Image is OK

There is 1 thing you did not mention in your procedure: recompiling MCUBoot and flashing it?
Did you just omite to mention it or you didn't do it?
I may be worth to give it  a try.

@ABOSTM is there any lower level debugging information which might help to spot what is different from your setup?

Hi @ABOSTM

There is 1 thing you did not mention in your procedure: recompiling MCUBoot and flashing it?
Did you just omite to mention it or you didn't do it?
I may be worth to give it a try.

I simple omit, but I built MCUboot and flash it first same week.
I'll give another try with latest.
Anyway, could you help to understand why the difference?
Your version
[00:00:00.006,000]  mcuboot: Primary image: magic=good, swap_type=0x1, copy_done=0x3, image_ok=0x3
Mine
[00:00:00.006,000]  mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x1
What make that image_ok=0x3 instead 0x1? Could be this the reason I can confirm image?

I am not an MCUBoot expert, but I found those definitions
#define BOOT_FLAG_SET       1
#define BOOT_FLAG_BAD       2
#define BOOT_FLAG_UNSET     3
#define BOOT_FLAG_ANY       4  /* NOTE: control only, not dependent on sector */

Then I figure out that after a fresh flashing of images, the 1st boot displays image_ok=0x3 and then all further reboots (manual reset) display image_ok=0x1
So the fact you have image_ok=0x3 is good.
And It makes sense to me as the change is done by the call to boot_write_img_confirmed(), which will set image_ok=1 ... after confirmation.

*** Booting Zephyr OS build v2.3.0-rc1-122-ge550302a61a0  ***
[00:00:00.005,000]  mcuboot: Starting bootloader
[00:00:00.006,000]  mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000]  mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000]  mcuboot: Boot source: primary slot
[00:00:00.013,000]  mcuboot: Swap type: none
*** Booting Zephyr OS build v2.3.0-rc1-122-ge550302a61a0  ***
Hello World! disco_l475_iot1
Confirming the boot image
Image is OK
*** Booting Zephyr OS build v2.3.0-rc1-122-ge550302a61a0  ***
[00:00:00.005,000]  mcuboot: Starting bootloader
[00:00:00.006,000]  mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x1
[00:00:00.006,000]  mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000]  mcuboot: Boot source: primary slot
[00:00:00.013,000]  mcuboot: Swap type: none
*** Booting Zephyr OS build v2.3.0-rc1-122-ge550302a61a0  ***
Hello World! disco_l475_iot1
Confirming the boot image
Image is OK

But it still doesn't explain why you are not able to confirm the image.

Can you start by a fullerase of the flash (if needed you can use CubeProgrammer https://www.st.com/en/development-tools/stm32cubeprog.html)
And then flash both image, It should work !!!
Then you can try to mix:
My MCUBoot image + Your application
and then Your MCUBoot + My application
It is important to make a full erase of the flash between each test.

zephy_MCUBoot.zip
zephyr-1.0.0_Signed_HelloWorld.zip

@ABOSTM , sure, I'll do!
Thank you for helping.
@utzig any comment here can help a lot!

Well, it worked for me as well:
*** Booting Zephyr OS build v2.3.0-rc1-138-gc46b138453a2  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.013,000] <inf> mcuboot: Swap type: none
*** Booting Zephyr OS build v2.3.0-rc1-138-gc46b138453a2  ***
Hello World! disco_l475_iot1
Confirming the boot image
Image is OK
*** Booting Zephyr OS build v2.3.0-rc1-138-gc46b138453a2  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x1
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.013,000] <inf> mcuboot: Swap type: test
*** Booting Zephyr OS build v2.3.0-rc1-138-gc46b138453a2  ***
Hello World! disco_l475_iot1
Confirming the boot image
Image is OK

I am using latest MCUBoot and fairly recent Zephyr tree (updated this afternoon). This error you're having usually happens if you have built the bootloader and the app with partition configurations that are not the same. Otherwise I would suggest attaching a debugger at boot_write_img_confirmed to check why it is failing.

Hi @ABOSTM,
At first, I did another try with lastest Zephyr version and your signed image, both without success.
So, move to install stm32cubeprog, update st-link version from 18 to 26, first erase full + flash with error. I repeat the process and got success. Than, write my hello image using west and got the image confirmed.
*** Booting Zephyr OS build v2.3.0-rc1-138-g1d06829ef97b  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.013,000] <inf> mcuboot: Swap type: none
*** Booting Zephyr OS build v2.3.0-rc1-138-g1d06829ef97b  ***
Hello World! disco_l475_iot1
Confirming the boot image
Image is OK
*** Booting Zephyr OS build v2.3.0-rc1-138-g1d06829ef97b  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x1
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.013,000] <inf> mcuboot: Swap type: none
*** Booting Zephyr OS build v2.3.0-rc1-138-g1d06829ef97b  ***
Hello World! disco_l475_iot1
Confirming the boot image
Image is OK
*** Booting Zephyr OS build v2.3.0-rc1-138-g1d06829ef97b  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x1
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.013,000] <inf> mcuboot: Swap type: none
*** Booting Zephyr OS build v2.3.0-rc1-138-g1d06829ef97b  ***
Hello World! disco_l475_iot1
Confirming the boot image
Image is OK

I remember that I lost the board sometimes, usully ctrl+C when using west. The west was not able to write anymore. The solution was cp *hex or bin to mounted unity and get board running again.
Not sure if that can cause flash write problem.
I think we discover the way to make board works, need flash MCUboot using stm32cubeprog with full chip erase. With that, I can move forward and I think this issue can be closed.
Question: Is there a way to flash with full erase from command line on linux?
Just for record: I give some UpdateHub runs and get below:
When I put UpdateHub to run I got some connection instability and was need increase TCP/IP buffers. After some attempts I got
Using es-WIFI
[00:00:00.538,000] <inf> main: UpdateHub sample app started
[00:00:00.538,000] <inf> main: Confirming the boot image
[00:00:03.635,000] <inf> main: WIFI Connected
[00:00:03.641,000] <inf> main: Starting UpdateHub polling mode
[00:00:04.200,000] <inf> updatehub: Probe metadata received
[00:02:41.766,000] <err> flash_stm32: Write range invalid. Offset: 845824, len: 92
[00:02:41.766,000] <err> STREAM_FLASH: flash_write error -22 offset=0x000ce800
[00:02:41.766,000] <err> updatehub: Error to write on the flash

And
Using ESP_8266
[00:00:00.570,000] <inf> main: UpdateHub sample app started
[00:00:00.570,000] <inf> main: Confirming the boot image
[00:00:04.436,000] <inf> main: WIFI Connected
[00:00:04.478,000] <inf> main: Starting UpdateHub polling mode
[00:00:05.391,000] <inf> updatehub: Probe metadata received
[00:00:53.641,000] <err> flash_stm32: Write range invalid. Offset: 880640, len: 140
[00:00:53.641,000] <err> STREAM_FLASH: flash_write error -22 offset=0x000d7000
[00:00:53.641,000] <err> updatehub: Error to write on the flash

Hi @nandojve ,
Good news !!
I don't know exactly what appends in your flash, but maybe previousoperations leave some images header in the secondary slot or in scratch area which make MCUBoot confused.

Question: Is there a way to flash with full erase from command line on linux?
It is possible to use CubeProgrammer with command line (CLI).
I nerver use it by you can found information in the documentation:
https://www.st.com/resource/en/user_manual/dm00403500-stm32cubeprogrammer-software-description-stmicroelectronics.pdf

It think it should also be possible to use OpenOCD or PyOCD do full erase the flash. But I am not sure Zephyr propose such option.

I don't know exactly what appends in your flash, but maybe previousoperations leave some images header in the secondary slot or in scratch area which make MCUBoot confused.

Confirming does actually write the confirmed flag to the primary slot, so probably there was some garbage there already (not previously erased!?).

Question: Is there a way to flash with full erase from command line on linux?

I use: https://github.com/stlink-org/stlink. st-flah erase for mass erase, and st-flash write <bin-file> <address> for write (preceeded by sector erase).

Thank you guys!
Now I need to do some homework.

Hi @ABOSTM , I open this issue again because I got same problem with zephyr-v2.3.0-1218.
If I do full chip erase and flash zephy_MCUBoot.zip and zephyr-1.0.0_Signed_HelloWorld.zip using cubeprogrammer on Windows, the system works.
This process is valid on Linux using the Utzig suggestion https://github.com/stlink-org/stlink and st-flash erase command followed by west flash zephy_MCUBoot and west flash --hex-file zephyr-1.0.0_Signed_HelloWorld.
Changing MCUBoot for the current version still system working. The system stop to work when I flash any application from main repository. To put system running I need execute again st-flash erase but only your app zephyr-1.0.0_Signed_HelloWorld confirming the boot image.
Note: Once the problem happen even your image don't works without a full erase.

Hi @nandojve ,
If I understand correctly, it seems there is a regression.
Regression is not in MCUBoot but in application.
Regression occurs between the time I created zephyr-1.0.0_Signed_HelloWorld.zip and now.
So I suggest to make dichotomy (git bisect) to determine which commit cause the regression.

@nandojve Is this issue still valid?

@nandojve testsuite for mcuboot interface functions are here: /tests/subsys/dfu/mcuboot.

@nandojve Is this issue still valid?

I'll check it again.

Hi @nvlsianpu , After https://github.com/zephyrproject-rtos/mcuboot/commit/bdcfc859d4b8b4c000a2a46906080e086d048686 I can't even build MCUboot for disco_l475_iot1. The commit message doesn't explain if there are something that should be done on my environment to allow build.
Any tip?

Hi @nvlsianpu , After zephyrproject-rtos/mcuboot@bdcfc85 I can't even build MCUboot for disco_l475_iot1. The commit message doesn't explain if there are something that should be done on my environment to allow build.
Any tip?

@nandojve please raise a dedicated issue for this new problem, it will help tracking.

@nandojve @erwango We are looking into the build problem anyway.

^^ Fix https://github.com/zephyrproject-rtos/mcuboot/pull/32

Hi @nvlsianpu thank you for help us with this. I'll try again later.

@nandojve would you mind rebasing https://github.com/UpdateHub/zephyr/commits/topic/uhu_wifi so we can have a test on recent zephyr version ?
Txs

Hi @erwango , sure! My apologies about delays from my side, it has been very busy these days.

Hi folks, some updates.
First test: Write files from ABOSTM and confirm board is Ok: Pass
st-flash erase
flash old bootloader
*** Booting Zephyr OS build v2.3.0-rc1-122-ge550302a61a0  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.013,000] <inf> mcuboot: Swap type: none
[00:00:00.013,000] <err> mcuboot: Unable to find bootable image
flash old firmware
*** Booting Zephyr OS build v2.3.0-rc1-122-ge550302a61a0  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.013,000] <inf> mcuboot: Swap type: none
*** Booting Zephyr OS build v2.3.0-rc1-122-ge550302a61a0  ***
Hello World! disco_l475_iot1
Confirming the boot image
Image is OK

Second test: Confirm that new MCUboot works as expected only flashing new MCUBoot without erase all flash content. The MCUBoot at a06884a4e6 doesn't run the current v2.3.0-rc1 on the flash: Fail
flash new bootloader
*** Booting Zephyr OS build v2.4.0-rc1-264-g44ac9d33e69c  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.015,000] <inf> mcuboot: Swap type: none
[00:00:00.015,000] <err> mcuboot: Unable to find bootable image

Third test: Confirm that new MCUboot works as expected with full flash erase before upload new MCUBoot and firmware. The MCUBoot at a06884a4e6 works and the current v2.3.0-rc1 run on the flash: Pass
st-flash erase
flash new bootloader
*** Booting Zephyr OS build v2.4.0-rc1-264-g44ac9d33e69c  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.015,000] <inf> mcuboot: Swap type: none
[00:00:00.015,000] <err> mcuboot: Unable to find bootable image
flash old firmware
*** Booting Zephyr OS build v2.4.0-rc1-264-g44ac9d33e69c  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.015,000] <inf> mcuboot: Swap type: none
*** Booting Zephyr OS build v2.3.0-rc1-122-ge550302a61a0  ***
Hello World! disco_l475_iot1
Confirming the boot image
Image is OK

Fourth test: Flash a new firmware with new MCUboot: Fail
flash new firmware
*** Booting Zephyr OS build v2.4.0-rc1-264-g44ac9d33e69c  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=good, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.015,000] <inf> mcuboot: Swap type: none
*** Booting Zephyr OS build v2.4.0-rc1-264-g44ac9d33e69c  ***
Hello World! disco_l475_iot1
Confirming the boot image
Error to confirm the image

Fifth test: Re flash the old firmware with new MCUboot and confirm that works: Fail
flash old firmware
*** Booting Zephyr OS build v2.4.0-rc1-264-g44ac9d33e69c  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=good, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.015,000] <inf> mcuboot: Swap type: none
*** Booting Zephyr OS build v2.3.0-rc1-122-ge550302a61a0  ***
Hello World! disco_l475_iot1
Confirming the boot image
Error to confirm the image

Sixth test: Flash old MCUboot and new firmware with full chip erase: Fail
st-flash erase
flash old bootloader
*** Booting Zephyr OS build v2.3.0-rc1-122-ge550302a61a0  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.013,000] <inf> mcuboot: Swap type: none
[00:00:00.013,000] <err> mcuboot: Unable to find bootable image
flash new firmware
*** Booting Zephyr OS build v2.3.0-rc1-122-ge550302a61a0  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=good, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.013,000] <inf> mcuboot: Swap type: none
*** Booting Zephyr OS build v2.4.0-rc1-264-g44ac9d33e69c  ***
Hello World! disco_l475_iot1
Confirming the boot image
Error to confirm the image

Seventh test: Revert to old firmware without full chip erase: Fail
*** Booting Zephyr OS build v2.3.0-rc1-122-ge550302a61a0  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=good, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.013,000] <inf> mcuboot: Swap type: none
*** Booting Zephyr OS build v2.3.0-rc1-122-ge550302a61a0  ***
Hello World! disco_l475_iot1
Confirming the boot image
Error to confirm the image

Eighth test: Full erase, new bootloader, old firmware, bad image, old firmware: Ok
st-flash erase
flash new bootloader
*** Booting Zephyr OS build v2.4.0-rc1-264-g44ac9d33e69c  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.015,000] <inf> mcuboot: Swap type: none
[00:00:00.015,000] <err> mcuboot: Unable to find bootable image
flash old firmware
*** Booting Zephyr OS build v2.4.0-rc1-264-g44ac9d33e69c  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.015,000] <inf> mcuboot: Swap type: none
*** Booting Zephyr OS build v2.3.0-rc1-122-ge550302a61a0  ***
Hello World! disco_l475_iot1
Confirming the boot image
Image is OK
flash BAD firmware
*** Booting Zephyr OS build v2.4.0-rc1-264-g44ac9d33e69c  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x1
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.015,000] <inf> mcuboot: Swap type: none
[00:00:00.015,000] <err> mcuboot: Image in the primary slot is not valid!
[00:00:00.015,000] <err> mcuboot: Unable to find bootable image
flash old firmware
*** Booting Zephyr OS build v2.4.0-rc1-264-g44ac9d33e69c  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x1
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.015,000] <inf> mcuboot: Swap type: none
*** Booting Zephyr OS build v2.3.0-rc1-122-ge550302a61a0  ***
Hello World! disco_l475_iot1
Confirming the boot image
Image is OK

Nineth test: Continuing eighth test: new image, old firmware: Fail
flash new firmware
*** Booting Zephyr OS build v2.4.0-rc1-264-g44ac9d33e69c  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=good, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.015,000] <inf> mcuboot: Swap type: none
*** Booting Zephyr OS build v2.4.0-rc1-264-g44ac9d33e69c  ***
Hello World! disco_l475_iot1
Confirming the boot image
Error to confirm the image
flash old firmware
*** Booting Zephyr OS build v2.4.0-rc1-264-g44ac9d33e69c  ***
[00:00:00.005,000] <inf> mcuboot: Starting bootloader
[00:00:00.006,000] <inf> mcuboot: Primary image: magic=good, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Scratch: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.006,000] <inf> mcuboot: Boot source: primary slot
[00:00:00.015,000] <inf> mcuboot: Swap type: none
*** Booting Zephyr OS build v2.3.0-rc1-122-ge550302a61a0  ***
Hello World! disco_l475_iot1
Confirming the boot image
Error to confirm the image

It seems that Zephyr-2.4.0-rc1 images doesn't confirm and corrupt the whole system. When flash a bad image (eighth test) the system run normally when revert to old firmware.
I confirm that after rebase to 2.4.0-rc1 updatehub doesn't confirm the image and stay rebooting. When I disable image confirmation boot_write_img_confirmed() the system still rebooting. It seems another regression but need more investigation. Because of all this I not able to run UpdateHub using esWiFi on disco_l475_iot1. I'll try run more tests and expects report early next week.

@nandojve would you mind rebasing https://github.com/UpdateHub/zephyr/commits/topic/uhu_wifi so we can have a test on recent zephyr version ?
Txs

Rebased on top of 2.4.0-rc2. The 28b9617f5ee1123413ef92d1a74469402272ba81 is full functional here at ETH/WIFI/MODEM.
Don't forget to change configs:
at prj.conf
CONFIG_UPDATEHUB_SERVER="<IP>"

IP: can be on local network or internet. DNS names will be resolved.
This way a docker machine can run updatehub-ce at any place.

at overlay-wifi.conf
CONFIG_UPDATEHUB_WIFI_SSID="<SSID>"
CONFIG_UPDATEHUB_WIFI_PSK="<PASSWD>"

Can you describe clear operation sequence required to reproduce the file: eg:

flash mcuboot of version xxx and application
upload image of , set it to test...
...

I have only nRF52xxdks boards so want to reproduce yours results using smp_svr sample. So fare I can't.

Hi @nvlsianpu , the project that I try run is https://github.com/UpdateHub/zephyr/commit/74ae362d7b806c2957c57c2203b189141a33a9a1
I sign the image and upload with west flash --hex.... All that I reported on the old post is related to the image at slot-0 after flash, and the above commit is the new firmware. I think you don't need use smp_svr because problem shows with west flash.
old MCUboot(bootloader)
zephy_MCUBoot.zip
old firmware
zephyr-1.0.0_Signed_HelloWorld.zip
Originally posted by @ABOSTM in https://github.com/zephyrproject-rtos/zephyr/issues/25010#issuecomment-630280091
Sequence is inside old post, try resume the first test, eighth and nine:
st-flash erase
flash old bootloader
flash old firmware
result: OK
Eighth test: Full erase, new bootloader, old firmware, bad image, old firmware: Ok
st-flash erase
flash new bootloader
flash old firmware
flash BAD firmware (image without sign)
flash old firmware
result: OK
Nineth test: Continuing eighth test: new image, old firmware: Fail
flash new firmware
flash old firmware
So, when I flash the new firmware at slot-0 using west it fail to confirm the image and somehow compromise the whole system. I mean, a old firmware (that always works) only confirm after full chip erase. I believe it can be something related only to ST SoCs.
I start to think this is the key related to the problem that I've been facing with ST SoC that MCUboot don't swap slot-1 to slot-0 keeping updatehub on a infinite upgrade loop.

@nandojve What you men by new bootloader, old bootloader, new firmware and old firmware?

@nvlsianpu it is about versions
old: v2.3.0-rc1-122
new: v2.4.0-rc1-264


doc: describe the default behaviour

Because for someone new, it might not be clear what it does.
The idea is also to modify the welcome page on mptcpd.mptcp.dev with the same description.
Note that I also modified the main wiki page with the same description.

LGTM. Thanks @matttbe!

Thank you for the review @ossama-othman! I was going to merge this PR, but then I realised that while at it, it might be good to describe how to change the behaviour. So I just added a new paragraph about that if that's OK with you :)

@matttbe Sure!  No problem at all.  Thanks!


Add some more to README.md

look at this for an example: https://gist.github.com/PurpleBooth/109311bb0361f32d87a2
we need to include the authors, versioning, contributing, license, acknowledgements, and add some hyperlinks for the things used.
also we should maybe add an explanation of how to run the automated tests?

completed in pull request #134 , thanks Zach!


Status bar height not excluded from screen height in android

Hi.
I found an issue in height calculation for android.
There is a known issue for android height definition.
Reference: https://qiita.com/uutarou10/items/1aad5fd2d03fb54e0dbf
For iOS, height of window and screen is the same.

But for android it isn't. See picture below.

So, for android, we must exclude statusbar height from screenHeight.
I'll soon send a PR to fix this issue.

Thanks a lot for your contribution @Hinaser . Will have a look and we 'll proceed with that.

Any news about that ?
Thanks !


Set minimum fee to $1

Rationale
The gas prices on rollups have changes quite a lot since we initially created the fee model. We should set the minimum fee lower than $5. Currently 1 tx on rollups costs around $0.10 and an agent does 3 per fill. So have some room for fluctuating gas prices, it is suggested that the minimum fee is set to $1.
Implementation

Update the minimum fee in the mainnet deployment to be 1 USDC.
Add "The minimum fee for a transfer is 1 USDC" to the popup text.

Hey team! Please add your planning poker estimate with Zenhub @fredo @istankovic

@GabrielBuragev please be aware that I added a component for this issue after aligning with Jacob.
We need the sentence "The minimum fee for a transfer is 1 USDC"  at the end of the popup (when you hover the mouse over "?")

Please add your planning poker estimate with Zenhub @GabrielBuragev

@andfletcher Should we create 2 separate issues and add them in this issue description and estimate them separately?
Since this issue covers work on both frontend & backend.

@andfletcher Should we create 2 separate issues and add them in this issue description and estimate them separately? Currently this issue covers work on both frontend & backend.

i dont think its worth it. Its a good practice on how we can swarm on issues together to finish them since both acceptance criteria are unrelated but still go towards the same value.

Where are the fees defined?

defined

This is a good point. But currently, we only support USDC, so it should not matter for now.

Where are the fees defined?

I dont know but why do you need to know where the fees are defined (just asking)

They are defined in the contract. So you should be able to pull the number from there. Especially since we might need to manually update it in the future based on gas prices.

@andfletcher As Jacob explained. Need to know where to fetch them from on the frontend.

Update Fee
Protocol: 0 PPM
Agent: 3000 PPM
Fixed Fee: 1000000

Fees updated on Optimism: https://optimistic.etherscan.io/tx/0xf2aaf2c93e41061fc106ea6aa62f186c258d865da31ea26b0e7f6b4468791212
Fees updated on Boba: https://bobascan.com/tx/0xa5b9119ea5a903961352661606b2009d8ab96d334b70e54767a59f333e850d94


File folder path and @wip/@ignore provision for spec files

Is your feature request related to a problem? Please describe.

@wip/@ignore tags should be provided to ignore the set of scenarios in Specmatic
In specmatic.json, we have "test" and "stubs", where we define the filePath. In the case of 1xxx API contracts, JSON will become huge.

Describe the solution you'd like

@wip/@ignore tags should be supported to comment out the scenarios
Instead of mentioning the path for all 1xxx APIs, better to mention the folder path in "test" and "stub".
eg:
/** Current structure /
/ NOTE: Consider n = 1xxx **/
"test":
[
"/projectName/name1/specFolder1/module1/fileName1.spec" ,
"/projectName/name1/specFolder1/module1/fileName2.spec",
"/projectName/name1/specFolder1/module2/fileName1.spec"
"/projectName/name1/specFolder1/module3/fileName1.spec"
"
"
"
"/projectName/name1/specFolder1/module/fileName.spec"
],
"stub":
[
"/projectName/name2/stubFolder1/stubServiceName1.spec"
"/projectName/name2/stubFolder1/stubServiceName1.spec"
"
"
"
"/projectName/name2/stubFolder1/stubServiceName.spec"
]

/** Suggested Solution /
/ NOTE: Consider n = 1xxx **/
"test":
[
"/projectName/name1/specFolder1/.."
]
"stub":
[
"/projectName/name1/stubFolder1/..."
]

https://specmatic.in/documentation/Versioning.html#handling-contracts-in-progress
Check the above documentation for use of WIP tag

The second point is a duplicate of https://github.com/znsio/specmatic/issues/113


Add omniscidb 5.6.4

Checklist

[x] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=366800&view=results

CI is green (see the previous comment).


additional scalar functions for array type

Inbuilt functions - Scalar funtions for array columns
select array_concat_int(multi_value_int_field, multi_value_int_field) from airlineStats;
select array_concat_string(multi_value_string_field, multi_value_string_field) from airlineStats;
select array_position_int(multi_value_int_field, 2) from airlineStats;
select array_position_string(multi_value_string_field, 'a') from airlineStats;
select array_union_int(multi_value_int_field, multi_value_int_field) from airlineStats;
select array_union_string(multi_value_string_field, multi_value_string_field) from airlineStats;
select array_remove_int(multi_value_int_field, 3) from airlineStats;
select array_remove_string(multi_value_string_field, 'bar') from airlineStats;
select array_distinct_int(multi_value_int_field) from airlineStats;
select array_distinct_string(multi_value_string_field) from airlineStats;
select array_slice_int(multi_value_int_field, 1, 2) from airlineStats;
select array_slice_string(multi_value_string_field, 1, 2) from airlineStats;

Codecov Report

Merging #6446 (8771e4e) into master (1beaab5) will decrease coverage by 1.19%.
The diff coverage is 56.80%.

@@            Coverage Diff             @@
##           master    #6446      +/-   ##
==========================================
- Coverage   66.44%   65.25%   -1.20%
==========================================
  Files        1075     1330     +255
  Lines       54773    64943   +10170
  Branches     8168     9469    +1301
==========================================
+ Hits        36396    42376    +5980
- Misses      15700    19536    +3836
- Partials     2677     3031     +354

Flag
Coverage Δ

unittests
65.25% <56.80%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...e/pinot/broker/api/resources/PinotBrokerDebug.java
0.00% <0.00%> (-79.32%)
:arrow_down:

...ot/broker/broker/AllowAllAccessControlFactory.java
71.42% <ø> (-28.58%)
:arrow_down:

.../helix/BrokerUserDefinedMessageHandlerFactory.java
33.96% <0.00%> (-32.71%)
:arrow_down:

...ker/routing/instanceselector/InstanceSelector.java
100.00% <ø> (ø)

...ava/org/apache/pinot/client/AbstractResultSet.java
66.66% <0.00%> (+9.52%)
:arrow_up:

.../main/java/org/apache/pinot/client/Connection.java
35.55% <0.00%> (-13.29%)
:arrow_down:

...inot/client/JsonAsyncHttpPinotClientTransport.java
10.90% <0.00%> (-51.10%)
:arrow_down:

...not/common/assignment/InstancePartitionsUtils.java
73.80% <ø> (+0.63%)
:arrow_up:

...common/config/tuner/NoOpTableTableConfigTuner.java
100.00% <ø> (ø)

...ot/common/config/tuner/RealTimeAutoIndexTuner.java
100.00% <ø> (ø)

... and 1195 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 33de6dc...8771e4e. Read the comment docs.

Codecov Report

Merging #6446 (8771e4e) into master (1beaab5) will decrease coverage by 1.19%.
The diff coverage is 56.80%.

@@            Coverage Diff             @@
##           master    #6446      +/-   ##
==========================================
- Coverage   66.44%   65.25%   -1.20%
==========================================
  Files        1075     1330     +255
  Lines       54773    64943   +10170
  Branches     8168     9469    +1301
==========================================
+ Hits        36396    42376    +5980
- Misses      15700    19536    +3836
- Partials     2677     3031     +354

Flag
Coverage Δ

unittests
65.25% <56.80%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...e/pinot/broker/api/resources/PinotBrokerDebug.java
0.00% <0.00%> (-79.32%)
:arrow_down:

...ot/broker/broker/AllowAllAccessControlFactory.java
71.42% <ø> (-28.58%)
:arrow_down:

.../helix/BrokerUserDefinedMessageHandlerFactory.java
33.96% <0.00%> (-32.71%)
:arrow_down:

...ker/routing/instanceselector/InstanceSelector.java
100.00% <ø> (ø)

...ava/org/apache/pinot/client/AbstractResultSet.java
66.66% <0.00%> (+9.52%)
:arrow_up:

.../main/java/org/apache/pinot/client/Connection.java
35.55% <0.00%> (-13.29%)
:arrow_down:

...inot/client/JsonAsyncHttpPinotClientTransport.java
10.90% <0.00%> (-51.10%)
:arrow_down:

...not/common/assignment/InstancePartitionsUtils.java
73.80% <ø> (+0.63%)
:arrow_up:

...common/config/tuner/NoOpTableTableConfigTuner.java
100.00% <ø> (ø)

...ot/common/config/tuner/RealTimeAutoIndexTuner.java
100.00% <ø> (ø)

... and 1195 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 33de6dc...8771e4e. Read the comment docs.


metronome 0.6.18 bump on dcos 1.11

High-level description
What features does this change enable? What bugs does this change fix?
Corresponding DC/OS tickets (obligatory)
These DC/OS JIRA ticket(s) must be updated (ideally closed) in the moment this PR lands:

DCOS_OSS-4994 Release Metronome 1.6.18 on Master.

Related tickets (optional)
Other tickets related to this change:

DCOS_OSS-4636  Failure when restart policy is ON_FAILURE. This bug was introduced through the fix of another bug regarding stopping invalid extra instances of a job run. Metronome should not check the launch queue when a restart is invoked.
DCOS_OSS-4636  Allow using IS operator when creating jobs. This was broken since introduction of the IS operator, which replaced EQ but was not a valid schema value.
New Feature:  Added new metric metronome.uptime.gauge.seconds

Checklist for all PRs

[x] Added a comprehensible changelog entry to CHANGES.md or explain why this is not a user-facing change:
[x] Included a test which will fail if code is reverted but test is not. If there is no test please explain here:
[x] Read the DC/OS contributing guidelines
[x] Followed relevant code rules Rules for Packages and Systemd

Checklist for component/package updates:
If you are changing components or packages in DC/OS (e.g. you are bumping the sha or ref of anything underneath packages), then in addition to the above please also include:

[x] Change log from the last version integrated (this should be a link to commits for easy verification and review): diff
[x] Test Results: CI
[ ] Code Coverage (if available): N/A

@mesosphere-mergebot label ready for review


Ticket importer

Added a new project - importer of tickets from GitHub.
Added a new controller, service, tests etc. for comments on backlog items.
Many additions related to user references (mentioned users on the ticket, modified users, etc.).
Fixed numerous bugs I picked when testing seeding the database.


Remove Microsoft.AspNetCore.Razor.Design

We're no longer producing the package in 3.0.

@pranavkm - Can you tell me if there is a related issue/PR to update the templates or should I open an issue?  When I dotnet new web with 3.0.100-preview-009734, the resulting project fails to restore
<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>netcoreapp3.0</TargetFramework>
    <AspNetCoreHostingModel>InProcess</AspNetCoreHostingModel>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.App" />
    <PackageReference Include="Microsoft.AspNetCore.Razor.Design" Version="3.0.0-alpha1-10663" PrivateAssets="All" />
  </ItemGroup>

</Project>

@MichaelSimons completely missed that. I'll send an PR to fix the templates


Make it possible to set the localization from the /connect/authorize request

Adding culture=it-CH&ui-culture=it-CH parameters to the URL of the /connect/authorize request and setting the localization using QueryStringRequestCultureProvider is ignored from IdentityServer4
It would be nice if IdentityServer4 supported optional parameters to set the localization from the /connect/authorize request.

There's a standard OIDC parameter for that ui_locale.

Using the ui_locales in the /connect/authorize request does not set the culture either

of course not - this is your job.

CultureInfo.CurrentCulture = new CultureInfo(culture);
CultureInfo.CurrentUICulture = new CultureInfo(culture);
Gets ignored, because the IdentityServer4 middleware is doing something

Also this is not a question but a feature request, because I don't see how IdentityServer4 supports culture switches from the /connect/authorize request

I don't follow - the authorize endpoint is a protocol endpoint.
We redirect to your UI and there you can read the standard ui_locales parameter from there - and do whatever you need to do for localization.

No matter how I set the localization from this method, the culture is being used, set from somewhere else. It is not possible to set the culture using the /connect/authorize  which redirects to my login in the AccountController login. Identityserver4 redirects with a returnUrl parameter and the culture is incorrect.

No matter how I set the localization from this method

From which method?

Calling /connect/authorize from an Implicit flow client.
Using localization middlerware, it is not possible to set the IdentityServer4 host localization using query parameters or filters. It is only possible using a cookie.
I'll document this with an example.
Greetings Damien

I guess I don't know enough about how localization works - but we are not doing anything special for it.

I will close this - because I think this is the job of the UI - not the protocol endpoint.
Please open a new issue - maybe someone else knows more about the localization middleware.

OK, When I find the reason why, I'll let you know. Reason I opened the issue here, is because I suspect it's something to do with the IdentityServer4 middleware redirects.
I still think it is not possible to use the ui_locales parameter to set the localization in as ASP.NET Core application using IdentityServer4.
If I open a view/controller in the same host (ASP.NET Core) which does not use the Identityserver4 middleware, the query string localization works.
Greetings Damien

QS is the wrong approach for this type of application. I thought we agree on that.
Dropping a cookie from the account controller is the much better way.

@leastprivilege I found a solution. The problem is that IdentityServer4 redirects the oidc endpoint and packs the query in a returnUrl and the culture is no longer set. I created a custom provider which checks for this redirect and the culture is set correctly for the host.
I'll document how to do this

Why is the QS the wrong solution for this?

I found a solution. The problem is that IdentityServer4 redirects the oidc endpoint and packs the query in a returnUrl and the culture is no longer set. I created a custom provider which checks for this redirect and the culture is set correctly for the host.

I'll document how to do this
@damienbod what is the need to parse returnUrl for ui_locales? can't you simple create a cookie with culture from ui_locales ?


Bump marathon for MWT

High-level description
What features does this change enable? What bugs does this change fix?
Corresponding DC/OS tickets (obligatory)
These DC/OS JIRA ticket(s) must be updated (ideally closed) in the moment this PR lands:

DCOS_OSS- Foo the Bar so it stops Bazzing.

Related tickets (optional)
Other tickets related to this change:

DCOS_OSS- Foo the Bar so it stops Bazzing.

Checklist for all PRs

[ ] Added a comprehensible changelog entry to CHANGES.md or explain why this is not a user-facing change:
[ ] Included a test which will fail if code is reverted but test is not. If there is no test please explain here:
[ ] Read the DC/OS contributing guidelines
[ ] Followed relevant code rules Rules for Packages and Systemd

Checklist for component/package updates:
If you are changing components or packages in DC/OS (e.g. you are bumping the sha or ref of anything underneath packages), then in addition to the above please also include:

[ ] Change log from the last version integrated (this should be a link to commits for easy verification and review): example
[ ] Test Results: [link to CI job test results for component]
[ ] Code Coverage (if available): [link to code coverage report]

PLEASE FILL IN THE TEMPLATE ABOVE / DO NOT REMOVE ANY SECTIONS ABOVE THIS LINE
Instructions and review process
What is the review process and when will my changes land?
All PRs require 2 approvals using GitHub's pull request reviews.
Reviewers should be:

Developers who understand the code being modified.
Developers responsible for code that interacts with or depends on the code being modified.

It is best to proactively ask for 2 reviews by @mentioning the candidate reviewers in the PR comments area. The responsibility is on the developer submitting the PR to follow-up with reviewers and make sure a PR is reviewed in a timely manner. Once a PR has 2 ship-it's, no red reviews, and all tests are green it will be included in the next train.

@mesosphere-mergebot label Ready For Review

Hi @alenkacz  - this bump will go into  master branch if it is not already in there.
Also, please share the JIRA with details on what this brings specifically for MWT and do we need this update for 1.13.0-alpha (which is under soak for > 1 week).

@orsenthil I created this PR just to have the build of dcos with this marathon for MWT for testing. We'll do a proper bump against master later.

Hi @alenkacz - so, this should not be merged. The reason I state this is, we have already provided 1.13.0-alpha to soak, and that will be artifact that is being shared with the customers as Alpha build. This branch is temporary and will be deleted and converted to tag today.  I hope this helps.

Closing as per https://github.com/dcos/dcos/pull/4913#issuecomment-477222666


[Adoption metrics] Provide real-time up to date info about Packit adoption

TODO:

[x] #1566
[x] #1567
[ ] And make the metrics available to public via dashboard#185

The main benefit: Being able to track how much packit is used
Who will benefit: us, possibly also new users (they can see that packit is used and not some proof-of-concept)

Replaced by the usage page on dashboard.


kub401 workshop code

Issue #, if available:
Description of changes:
Adding code files related to KUB401 workshop
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Closing this PR. This repo should be used to share materials with re:Invent attendees after the session, not for day-of execution.


[SQL] [Minor] Tests the data type conversion code paths

In SimpleTextRelation, we specified needsConversion to true, indicating that values produced by this testing relation should be of Scala types, and need to be converted to Catalyst types when necessary. However, we also used Cast to convert strings to expected data types. And Cast always produces values of Catalyst types, thus no conversion is done at all. This PR makes SimpleTextRelation produce Scala values so that data conversion code paths can be properly tested.

cc @JoshRosen @yhuai

Merged build triggered.

Merged build started.

Test build #33522 has started for   PR 6414 at commit 9968fba.

Test build #33522 has finished for   PR 6414 at commit 9968fba.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33522/
Test FAILed.

Test build #859 has started for   PR 6414 at commit 9968fba.

Test build #859 has finished for   PR 6414 at commit 9968fba.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Nice, it looks like the test failure reproduces #6400:
Lost task 0.0 in stage 405.0 (TID 974, localhost): java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.spark.sql.types.UTF8String  at org.apache.spark.sql.execution.SparkSqlSerializer2$$anonfun$createSerializationFunction$1.apply(SparkSqlSerializer2.scala:319)  at org.apache.spark.sql.execution.SparkSqlSerializer2$$anonfun$createSerializationFunction$1.apply(SparkSqlSerializer2.scala:212)  at org.apache.spark.sql.execution.Serializer2SerializationStream.writeKey(SparkSqlSerializer2.scala:65)  at org.apache.spark.storage.DiskBlockObjectWriter.write(BlockObjectWriter.scala:206)  at org.apache.spark.util.collection.WritablePartitionedIterator$$anon$3.writeNext(WritablePartitionedPairCollection.scala:104)  at org.apache.spark.util.collection.ExternalSorter.spillToPartitionFiles(ExternalSorter.scala:375)  at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:208)  at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:62)  at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:70)  at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)  at org.apache.spark.scheduler.Task.run(Task.scala:70)  at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  at java.lang.Thread.run(Thread.java:745)  Driver stacktrace: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 405.0 failed 1 times, most recent failure: Lost task 0.0 in stage 405.0 (TID 974, localhost): java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.spark.sql.types.UTF8String  at org.apache.spark.sql.execution.SparkSqlSerializer2$$anonfun$createSerializationFunction$1.apply(SparkSqlSerializer2.scala:319)  at org.apache.spark.sql.execution.SparkSqlSerializer2$$anonfun$createSerializationFunction$1.apply(SparkSqlSerializer2.scala:212)  at org.apache.spark.sql.execution.Serializer2SerializationStream.writeKey(SparkSqlSerializer2.scala:65)  at org.apache.spark.storage.DiskBlockObjectWriter.write(BlockObjectWriter.scala:206)  at org.apache.spark.util.collection.WritablePartitionedIterator$$anon$3.writeNext(WritablePartitionedPairCollection.scala:104)  at org.apache.spark.util.collection.ExternalSorter.spillToPartitionFiles(ExternalSorter.scala:375)  at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:208)  at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:62)  at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:70)  at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)  at org.apache.spark.scheduler.Task.run(Task.scala:70)  at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  at java.lang.Thread.run(Thread.java:745)  Driver stacktrace:  at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1266)  at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1257)  at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1256)  at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)  at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)  at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1256)  at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:730)  at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:730)  at scala.Option.foreach(Option.scala:236)  at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:730)  at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1450)  at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1411)  at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)

How do you want to handle merging this and #6400?  I think that #6400 needs to go first so that we don't commit broken code.  Should I rebase my branch to include this commit and remove the regression test that I added?  Or should we leave both tests in place?

@JoshRosen I think we can add this change to #6400 and also keep your regression test.

@liancheng's commit message is really nice, so I'll quote it in my PR so that we don't lose it.

Closing this as it's already merged into #6400.


PTP timestamps, published by HSB, aren't synchonized with the host time.

The user guide hardware setup instructions show how to configure the phc2sys tool, which ensures that the NIC time-of-day clock (which is published in PTP network messages) is synchronized with the host real-time clock. The setup instructions rely on the default ntpdate configuration to initialize the system clock with the current real-world time.  As written in the user guide, the phc2sys startup doesn't properly wait for ntpdate to complete, so that when ntpdate does finish, phc2sys sees a large jump in the system time.  Because phc2sys slowly adjusts the NIC clock, the time published by ptp4l will then not be synchronized with the system clock, and could take a very long time to do so.  This has the effect of publishing a time with ingress data that is out-of-sync with the system time.
To verify you're in this condition, observe the output in the "sys offset" column from the command systemctl status phc2sys-*.service; large absolute values are an indication of this condition.
To work around this, run sudo systemctl restart phc2sys-*.service after ntpdate is synchronized. There is some anecdotal evidence that adding "-w" to the phc2sys command line may fix this problem but the documentation for this option doesn't address the configuration in use here.

Note that the earlier fix (setting After=timemaster.service) isn't correct; instead, the correct fix is to add "-S 0.001" to the phc2sys command line, so that it reads
ExecStart=/usr/sbin/phc2sys -c $EN0 -s CLOCK_REALTIME -O 0 -S 0.001


Arrays inside AnyMap can not be accessed from Swift (AnyMapHolder)

AnyMap represents any JSON-like type.
A value inside an AnyMap looks like this:
public enum AnyValue {
  case null
  case number(Double)
  case bool(Bool)
  case bigint(Int64)
  case string(String)
  case array([AnyValue]) // <-- ❌ This does not work
  case object(Dictionary<String, AnyValue>)
}

Due to a Swift compiler bug (see https://github.com/swiftlang/swift/issues/75994), an enum does not support cases with recursive arrays inside.
Due to this limitation, AnyMapHolder (in Swift) cannot get or set arrays inside the AnyMap. They will still exist if created from JS or C++, but just cannot be modified from Swift.
Once this Swift issue is resolved, I can add arrays to AnyMap.

Fixed in https://github.com/mrousavy/nitro/pull/146


nz-cascader support borderless

What problem does this feature solve?
nz-cascader provide borderless style like nz-select
What does the proposed API look like?
Add @Input() @WithConfig<boolean>() @InputBoolean() nzBorderless property to NzCascaderComponent
The style file of nz-cascader support borderless style, but not be exposed. Adding  nzBorderless property to expose it is easy.

https://github.com/NG-ZORRO/ng-zorro-antd/pull/7115

cascader component had been refactored and now it's supported to customize render template via nzLabelRender


Add MSBuildProjectBuilder

This is an initial implementation of an MSBuild project builder. It includes a custom asset that can reference and configure an MSBuild project (MSBuildProjectReference), a custom asset importer (creates MSBuildProjectReferences from .csproj/.sln files), and logic for manually and automatically building referenced MSBuild projects (MSBuildProjectBuilder).
See https://github.com/microsoft/MSBuildForUnity/blob/user/ryantrem/MSBuildProjectBuilder/README.md for more details.

Eek! Very exciting!

Nit: When I create an MSBuildProjectReference I can build/rebuild it and get the below esoteric error. Perhaps we want to disable the build buttons until something buildable is actually referenced? Or perhaps log an error about needing to specify one?
ArgumentException: Invalid path
System.IO.Path.GetDirectoryName (System.String path) (at <df7127ba07dc446d9f5831a0ec7b1d63>:0)
MSBuildForUnity.MSBuildProjectBuilder+<RunDotNetAsync>d__15.MoveNext () (at Assets/MSBuildProjectBuilder/Editor/MSBuildProjectBuilder.cs:222)
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () (at <df7127ba07dc446d9f5831a0ec7b1d63>:0)
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) (at <df7127ba07dc446d9f5831a0ec7b1d63>:0)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) (at <df7127ba07dc446d9f5831a0ec7b1d63>:0)
System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) (at <df7127ba07dc446d9f5831a0ec7b1d63>:0)
System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[TResult].GetResult () (at <df7127ba07dc446d9f5831a0ec7b1d63>:0)
MSBuildForUnity.MSBuildProjectBuilder+<BuildProjectsAsync>d__9.MoveNext () (at Assets/MSBuildProjectBuilder/Editor/MSBuildProjectBuilder.cs:149)
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () (at <df7127ba07dc446d9f5831a0ec7b1d63>:0)
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) (at <df7127ba07dc446d9f5831a0ec7b1d63>:0)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) (at <df7127ba07dc446d9f5831a0ec7b1d63>:0)
System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) (at <df7127ba07dc446d9f5831a0ec7b1d63>:0)
System.Runtime.CompilerServices.TaskAwaiter`1[TResult].GetResult () (at <df7127ba07dc446d9f5831a0ec7b1d63>:0)
MSBuildForUnity.MSBuildProjectBuilder+<BuildProjectsAsync>d__8.MoveNext () (at Assets/MSBuildProjectBuilder/Editor/MSBuildProjectBuilder.cs:108)
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () (at <df7127ba07dc446d9f5831a0ec7b1d63>:0)
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) (at <df7127ba07dc446d9f5831a0ec7b1d63>:0)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) (at <df7127ba07dc446d9f5831a0ec7b1d63>:0)
System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) (at <df7127ba07dc446d9f5831a0ec7b1d63>:0)
System.Runtime.CompilerServices.TaskAwaiter`1[TResult].GetResult () (at <df7127ba07dc446d9f5831a0ec7b1d63>:0)
MSBuildForUnity.MSBuildProjectReference+MSBuildProjectReferenceEditor+<OnInspectorGUI>d__1.MoveNext () (at Assets/MSBuildProjectBuilder/Editor/MSBuildProjectReferenceEditor.cs:24)
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () (at <df7127ba07dc446d9f5831a0ec7b1d63>:0)
System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c.<ThrowAsync>b__6_0 (System.Object state) (at <df7127ba07dc446d9f5831a0ec7b1d63>:0)
UnityEngine.UnitySynchronizationContext+WorkRequest.Invoke () (at C:/buildslave/unity/build/Runtime/Export/UnitySynchronizationContext.cs:115)
UnityEngine.UnitySynchronizationContext:ExecuteTasks()

Nit: Since a solution dropped in seems to have AutoBuild forced to true I think it'd make more sense to default new MSBuildProjectReference AutoBuild properties to true as well.

Things seem to be working rather well! Really cool! Can't wait to pull this into our projects!

Nit: When I create an MSBuildProjectReference I can build/rebuild it and get the below esoteric error. Perhaps we want to disable the build buttons until something buildable is actually referenced? Or perhaps log an error about needing to specify one?
ArgumentException: Invalid path
System.IO.Path.GetDirectoryName (System.String path) (at :0)
MSBuildForUnity.MSBuildProjectBuilder+d__15.MoveNext () (at Assets/MSBuildProjectBuilder/Editor/MSBuildProjectBuilder.cs:222)
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () (at :0)
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) (at :0)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) (at :0)
System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) (at :0)
System.Runtime.CompilerServices.ConfiguredTaskAwaitable1+ConfiguredTaskAwaiter[TResult].GetResult () (at <df7127ba07dc446d9f5831a0ec7b1d63>:0) MSBuildForUnity.MSBuildProjectBuilder+<BuildProjectsAsync>d__9.MoveNext () (at Assets/MSBuildProjectBuilder/Editor/MSBuildProjectBuilder.cs:149) --- End of stack trace from previous location where exception was thrown --- System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () (at <df7127ba07dc446d9f5831a0ec7b1d63>:0) System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) (at <df7127ba07dc446d9f5831a0ec7b1d63>:0) System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) (at <df7127ba07dc446d9f5831a0ec7b1d63>:0) System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) (at <df7127ba07dc446d9f5831a0ec7b1d63>:0) System.Runtime.CompilerServices.TaskAwaiter1[TResult].GetResult () (at :0)
MSBuildForUnity.MSBuildProjectBuilder+d__8.MoveNext () (at Assets/MSBuildProjectBuilder/Editor/MSBuildProjectBuilder.cs:108)
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () (at :0)
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) (at :0)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) (at :0)
System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) (at :0)
System.Runtime.CompilerServices.TaskAwaiter`1[TResult].GetResult () (at :0)
MSBuildForUnity.MSBuildProjectReference+MSBuildProjectReferenceEditor+d__1.MoveNext () (at Assets/MSBuildProjectBuilder/Editor/MSBuildProjectReferenceEditor.cs:24)
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () (at :0)
System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c.b__6_0 (System.Object state) (at :0)
UnityEngine.UnitySynchronizationContext+WorkRequest.Invoke () (at C:/buildslave/unity/build/Runtime/Export/UnitySynchronizationContext.cs:115)
UnityEngine.UnitySynchronizationContext:ExecuteTasks()

Updated, now we show an error instead of build/rebuild buttons in this case: