apt borked after installing ubuntu desktop?

Windows Version
Microsoft Windows [Version 10.0.19045.3086]
WSL Version
<IP_ADDRESS>
Are you using WSL 1 or WSL 2?

[X] WSL 2
[ ] WSL 1

Kernel Version
<IP_ADDRESS>
Distro Version
Ubuntu 22.04
Other Software
No response
Repro Steps
apt install ubuntu-desktop
Expected Behavior
to still be able to use apt commands as normal after installing ubuntu-desktop
Actual Behavior
apt currently gives errors on updates
I can use ubuntu, but updating seems broken, & certain aliases like zshrc to launch .zshrc in code editor now give zsh: permission denied
I'm assuming it's because of running apt install ubuntu-desktop , but I'm not 100% sure. Maybe its a $PATH issue?
can this be fixed without a reinstall?
Failed to start apt-news.service: Transport endpoint is not connected See system logs and 'systemctl status apt-news.service' for details.
Failed to start esm-cache.service: Transport endpoint is not connected See system logs and 'systemctl status esm-cache.service' for details.
Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB]
Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease
Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB]
Hit:4 https://cli.github.com/packages stable InRelease
Get:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [108 kB]
Hit:6 https://ppa.launchpadcontent.net/daniel-milde/gdu/ubuntu jammy InRelease
Hit:8 https://ppa.launchpadcontent.net/neovim-ppa/stable/ubuntu jammy InRelease Hit:7 https://packagecloud.io/github/git-lfs/ubuntu jammy InRelease

Fetched 337 kB in 1s (236 kB/s)  <-- hangs here

I've tried sudo dpkg --configure -a
Setting up xserver-xorg-input-all (1:7.7+23ubuntu2) ...
Setting up whoopsie (0.2.77) ...
Failed to preset unit: Transport endpoint is not connected /usr/bin/deb-systemd-helper: error: systemctl preset failed on whoopsie.path: No such file or directory Failed to reload daemon: Transport endpoint is not connected
[process exited with code 1 (0x00000001)]

Diagnostic Logs
No response

Thank you for reporting this @Hifilo. Is systemd enabled in your distribution ?
What's the output of cat /etc/wsl.conf ?

Thank you for reporting this @Hifilo. Is systemd enabled in your distribution ?
What's the output of cat /etc/wsl.conf?

I did enable it for the distro before the one I had this issue with. However I can't confirm for this issue because I backed up & reset.
It is currently disabled & I haven't enabled it yet. I can see whether running apt install ubuntu-desktop causes the same problem on a fresh install if necessary.

Ok. Closing for now but feel free to reopen if the issue happens again.

I do have the same error today.  All apt commands will hang..

a long clueless google leads me to a simple fix as below.
 sudo rm -rf /etc/acpi/events

I found this by searching the error messages I was getting trying to update 22.04.
I ran 'sudo apt install ubuntu-desktop' command in a fresh new install of 18.04 yesterday and first noticed this behaviour in my 22.04 (default) distro today but that may be a coincidence.  I checked my 22.04 dpkg.log and I had installed ubuntu-desktop on 22.04 in April of 2022.  I haven't run the 22.04 distro for about 4 months but I know I've done updates periodically ever since the initial install - dpkg.log has 34 unique dates in it.
I don't have a problem with 18.04, just 22.04.  Renaming the /etc/acpi/events folder did indeed allow me to perform the usual update procedures but I first had to kill the 'unattended-upgrade' process which seemed to have hung. Renaming the events folder back to the original name brings back the errors but allows the apt command to proceed.  I'd be happy to provide additional input if needed.  I do have systemd enabled.
C:\Users\allen>wsl --version
WSL version: <IP_ADDRESS>
Kernel version: <IP_ADDRESS>
WSLg version: 1.0.51
MSRDC version: 1.2.3770
Direct3D version: 1.608.2-61064218
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22621.2134

I have the same problem.  I run "apt install ubuntu-desktop"  but it failed. After that, every apt command will hang as well other commands like systemctl, reboot... I solved this probelm using the method from @pwang2 .
The cat /etc/wsl.conf outpus:

And the system details:
OS: Ubuntu 22.04.3 LTS on Windows 10 x86_64
Kernel: <IP_ADDRESS>-microsoft-standard-WSL2
Shell: zsh 5.8.1

I reproduced the same issue while following the official documentation https://docs.docker.com/desktop/wsl/
WSL version: <IP_ADDRESS>
Kernel version: <IP_ADDRESS>-1
Windows version: 10.0.23619.1000
Ubuntu 22.04.3 LTS
Docker Engine: 24.0.7

a long clueless google leads me to a simple fix as below.
 sudo rm -rf /etc/acpi/events

Thanks, you saved my life! It works in my situation.
WSL version: <IP_ADDRESS>
Kernel version: <IP_ADDRESS>-1
WSLg version: 1.0.59
MSRDC version: 1.2.4677
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22631.3085
Docker Desktop 4.27.1 (136059)

a long clueless google leads me to a simple fix as below.
 sudo rm -rf /etc/acpi/events

This error has gone for me:

/usr/bin/deb-systemd-helper: error: systemctl preset failed on xxxxx: No such file or directory

WSL version: <IP_ADDRESS>
Kernel version: <IP_ADDRESS>-1
WSLg version: 1.0.59
MSRDC version: 1.2.4677
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22631.3085

same problem.

a long clueless google leads me to a simple fix as below.
 sudo rm -rf /etc/acpi/events

save my day.'
how it works?

same problem! I couldn't find a solution until I found the fix from @pwang2
This gave me a big scare.
Linux Distribution: Ubuntu-22.04
WSL version: <IP_ADDRESS>
Kernel version: <IP_ADDRESS>-2
WSLg version: 1.0.60
MSRDC version: 1.2.5105
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22635.3286

a long clueless google leads me to a simple fix as below.
 sudo rm -rf /etc/acpi/events

works pretty good on ubuntu22.04
WSL version: <IP_ADDRESS>
Kernel version: <IP_ADDRESS>-2
WSLg version: 1.0.60
MSRDC version: 1.2.5105
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22631.3296

It is related to ACPI support.  More details can be found here. https://github.com/microsoft/WSL/issues/10059
It is suggested apt-mark hold the acpi related package before we run apt update.   Still I found it is an unwise thing to install all the Bloatware from Ubuntu-desktop package to WSL.
sudo apt-mark hold acpid acpi-support

a long clueless google leads me to a simple fix as below.
 sudo rm -rf /etc/acpi/events

resolve my problem
Setting up whoopsie (0.2.77) ...
Failed to preset unit: Transport endpoint is not connected
/usr/bin/deb-systemd-helper: error: systemctl preset failed on whoopsie.path: No such file or directory

a long clueless google leads me to a simple fix as below.
sudo rm -rf /etc/acpi/events

tested, very useful


feat: merge user-provided --{disable,enable}-features in args

Currently user-provided features are completely ignored in args, being overridden with default ones.

Currently user-provided features are completely ignored in args, being overridden with default ones.

Is it the case? I would expect that user-provided args override the default ones.

Is it the case? I would expect that user-provided args override the default ones.

Yes. If you want to test it out locally, run this script:
#!/usr/bin/env node
import puppeteer from 'puppeteer';

(async () => {
  const browser = await puppeteer.launch({
    devtools: true,
    headless: false,
    args: ['--enable-features=WebBluetoothNewPermissionsBackend'],
  });
  const mainPage = await browser.newPage();
  await mainPage.goto(
    'http://googlechrome.github.io/samples/web-bluetooth/get-devices-async-await.html'
  );
  await mainPage.waitForNetworkIdle();
  await mainPage.waitForDevicePrompt();
})();

Then observe that a log message will appear in the page that sayd getDevices is not a function.
When the feature is enabled, this message does not appear.

Done


Index out of bounds (while executing)

Hello,
We have an error related to the latest version of ArangoDB 3.8.0. (the related query worked with 3.7).
For your information, we have not upgraded from 3.7 but we made a fresh install directly in 3.8.
Best regards,
Antoine
My Environment

ArangoDB Version:        3.8.0
Storage Engine:             RocksDB
Deployment Mode:       Single Server
Deployment Strategy:   Manual Start
Configuration:
Infrastructure:               own
Operating System:        Debian 9
Total RAM in your machine:        128Gb
Disks in use:        HDD
Used Package:              Debian or Ubuntu .deb

Component, Query & Data
Affected feature:
AQL query
AQL query (if applicable):
let genera = [{ genus: 'Acer', referential: 'weurope' }]
				let data = (
					for g in genera
						let counts = (
							for tm in z_taxa_meta
								filter tm.images_count > 0
								filter tm.referential == g.referential
								and tm.genus_without_author == g.genus
								collect aggregate
									images_count = sum(tm.images_count),
									observations_count = sum(tm.observations_count),
									species_count = sum(1)
								return {
									images_count: images_count,
									observations_count: observations_count,
									species_count: species_count
								}
						)[0]
						let images = (
							for tm in z_taxa_meta
								filter tm.images_count > 0
								filter tm.referential == g.referential
								and tm.genus_without_author == g.genus
								for i in tm.tmp_best_images
									sort i.order, i.score desc, i.date desc
									limit 4
									return i
						)
						let best_images_ids = (
							(length(images) > 0)
							? (
								for b in images
									return b.id
							)
							: null
						)
						return {
							referential: g.referential,
							genus: g.genus,
							genus_species_count: counts.species_count,
							genus_images_count: counts.images_count,
							genus_observations_count: counts.observations_count,
							genus_best_images: best_images_ids
						}
				)
				for tm in z_taxa_meta
					for d in data
						filter tm.genus_without_author == d.genus
						and tm.referential == d.referential
						and (
							tm.genus_species_count != d.genus_species_count
							or tm.genus_images_count != d.genus_images_count
							or tm.genus_observations_count != d.genus_observations_count
							or tm.genus_best_images != d.genus_best_images
						)
						update tm with {
							genus_species_count: d.genus_species_count,
							genus_images_count: d.genus_images_count,
							genus_observations_count: d.genus_observations_count,
							genus_best_images: d.genus_best_images
						} in z_taxa_meta

AQL explain and/or profile (if applicable):
Query String (1977 chars, cacheable: false):
 let genera = [{ genus: 'Acer', referential: 'weurope' }]
 				let data = (
 					for g in genera
 						let counts = (
 							for tm in z_taxa_meta
 								filter tm.images_count > 0
 								filter tm.referential == g.referential
 								and tm.genus_without_author == g.genus
 								collect aggregate
 									images_count = sum(tm.images_count),
 									observations_count = sum(tm.observations_count),
 									species_count = sum(1)
 								return {
 									images_count: images_count,
 									observations_count: observations_count,
 									species_count: species_count
 								}
 						)[0]
 						let images = (
 							for tm in z_taxa_meta
 								filter tm.images_count > 0
 								filter tm.referential == g.referential
 								and tm.genus_without_author == g.genus
 								for i in tm.tmp_best_images
 									sort i.order, i.score desc, i.date desc
 									limit 4
 									return i
 						)
 						let best_images_ids = (
 							(length(images) > 0)
 							? (
 								for b in im...

Execution plan:
 Id   NodeType            Est.   Comment
  1   SingletonNode          1   * ROOT
 14   CalculationNode        1     - LET #35 = 1   /* json expression */   /* const assignment */
  2   CalculationNode        1     - LET genera = [ { "genus" : "Acer", "referential" : "weurope" } ]   /* json expression */   /* const assignment */
  5   EnumerateListNode      1     - FOR #65 IN genera   /* list iteration */
 58   SubqueryStartNode      1       - LET #62 = ( /* subquery begin */
 53   IndexNode              3         - FOR tm IN z_taxa_meta   /* hash index scan, projections: `images_count`, `tmp_best_images` */    FILTER (tm.`images_count` > 0)   /* early pruning */
 26   CalculationNode        3           - LET #43 = tm.`tmp_best_images`   /* attribute expression */   /* collections used: tm : z_taxa_meta */
 27   EnumerateListNode    300           - FOR i IN #43   /* list iteration */
 28   CalculationNode      300             - LET #45 = i.`order`   /* attribute expression */
 29   CalculationNode      300             - LET #47 = i.`score`   /* attribute expression */
 30   CalculationNode      300             - LET #49 = i.`date`   /* attribute expression */
 31   SortNode             300             - SORT #45 ASC, #47 DESC, #49 DESC   /* sorting strategy: constrained heap */
 32   LimitNode              4             - LIMIT 0, 4
 59   SubqueryEndNode        1             - RETURN  i ) /* subquery end */
 56   SubqueryStartNode      1       - LET #61 = ( /* subquery begin */
 36   EnumerateListNode    100         - FOR b IN #62   /* list iteration */
 37   CalculationNode      100           - LET #51 = b.`id`   /* attribute expression */
 57   SubqueryEndNode        1           - RETURN  #51 ) /* subquery end */
 54   SubqueryStartNode      1       - LET #64 = ( /* subquery begin */
 52   IndexNode              3         - FOR tm IN z_taxa_meta   /* hash index scan, projections: `images_count`, `observations_count` */    FILTER (tm.`images_count` > 0)   /* early pruning */
 12   CalculationNode        3           - LET #31 = tm.`images_count`   /* attribute expression */   /* collections used: tm : z_taxa_meta */
 13   CalculationNode        3           - LET #33 = tm.`observations_count`   /* attribute expression */   /* collections used: tm : z_taxa_meta */
 15   CollectNode            1           - COLLECT AGGREGATE images_count = SUM(#31), observations_count = SUM(#33), species_count = SUM(#35)   /* sorted */
 50   LimitNode              1           - LIMIT 0, 1
 16   CalculationNode        1           - LET #37 = { "images_count" : images_count, "observations_count" : observations_count, "species_count" : species_count }   /* simple expression */
 55   SubqueryEndNode        1           - RETURN  #37 ) /* subquery end */
 19   CalculationNode        1       - LET #63 = #64[0]   /* simple expression */
 40   CalculationNode        1       - LET #60 = ((LENGTH(#62) > 0) ? #61 : null)   /* simple expression */
 48   CalculationNode        1       - LET #57 = { "genus_species_count" : #63.`species_count`, "genus_images_count" : #63.`images_count`, "genus_observations_count" : #63.`observations_count`, "genus_best_images" : #60 }   /* simple expression */
 51   IndexNode             12       - FOR tm IN z_taxa_meta   /* hash index scan */
 46   CalculationNode       12         - LET #55 = (((tm.`genus_without_author` == #65.`genus`) && (tm.`referential` == #65.`referential`)) && ((((tm.`genus_species_count` != #63.`species_count`) || (tm.`genus_images_count` != #63.`images_count`)) || (tm.`genus_observations_count` != #63.`observations_count`)) || (tm.`genus_best_images` != #60)))   /* simple expression */   /* collections used: tm : z_taxa_meta */
 47   FilterNode            12         - FILTER #55
 49   UpdateNode             0         - UPDATE tm WITH #57 IN z_taxa_meta

Indexes used:
 By   Name              Type      Collection    Unique   Sparse   Selectivity   Fields                                      Ranges
 53   idx_11017026002   hash      z_taxa_meta   false    false        26.92 %   [ `genus_without_author`, `referential` ]   ((tm.`genus_without_author` == #65.`genus`) && (tm.`referential` == #65.`referential`))
 52   idx_11017026002   hash      z_taxa_meta   false    false        26.92 %   [ `genus_without_author`, `referential` ]   ((tm.`genus_without_author` == #65.`genus`) && (tm.`referential` == #65.`referential`))
 51   idx_11017026002   hash      z_taxa_meta   false    false        26.92 %   [ `genus_without_author`, `referential` ]   ((tm.`genus_without_author` == #65.`genus`) && (tm.`referential` == #65.`referential`))
 51   idx_11017026002   hash      z_taxa_meta   false    false        26.92 %   [ `genus_without_author`, `referential` ]   ((tm.`genus_without_author` == #65.`genus`) && (tm.`referential` == #65.`referential`))
 51   idx_11017026002   hash      z_taxa_meta   false    false        26.92 %   [ `genus_without_author`, `referential` ]   ((tm.`genus_without_author` == #65.`genus`) && (tm.`referential` == #65.`referential`))
 51   idx_11017026002   hash      z_taxa_meta   false    false        26.92 %   [ `genus_without_author`, `referential` ]   ((tm.`genus_without_author` == #65.`genus`) && (tm.`referential` == #65.`referential`))
 49   primary           primary   z_taxa_meta   true     false       100.00 %   [ `_key` ]                                  tm

Functions used:
 Name     Deterministic   Cacheable   Uses V8
 LENGTH   true            true        false

Optimization rules applied:
 Id   RuleName
  1   inline-subqueries
  2   simplify-conditions
  3   move-calculations-up
  4   move-filters-up
  5   remove-redundant-calculations
  6   remove-unnecessary-calculations
  7   optimize-subqueries
  8   move-calculations-up-2
  9   move-filters-up-2
 10   remove-data-modification-out-variables
 11   use-indexes
 12   remove-filter-covered-by-index
 13   remove-unnecessary-calculations-2
 14   move-calculations-down
 15   sort-limit
 16   reduce-extraction-to-projection
 17   move-filters-into-enumerate
 18   splice-subqueries

Optimization rules with highest execution times:
 RuleName                                    Duration [s]
 use-indexes                                      0.00052
 inline-subqueries                                0.00027
 remove-filter-covered-by-index                   0.00019
 remove-redundant-calculations                    0.00018
 remove-unnecessary-calculations-2                0.00018

43 rule(s) executed, 1 plan(s) created

Write query options:
 Option                   Value
 waitForSync              false
 skipDocumentValidation   false
 keepNull                 true
 mergeObjects             true
 ignoreRevs               true
 isRestore                false
 ignoreErrors             false
 ignoreDocumentNotFound   false
 readCompleteInput        true
 consultAqlWriteFilter    false
 exclusive                false

Dataset:
Here is an example of a document from the collection:
{
  "species": "Silene vivianii Steud.",
  "referential": "bdtfx_sync_v1",
  "species_without_author": "Silene vivianii",
  "genus": "Silene L.",
  "genus_without_author": "Silene",
  "family": "Caryophyllaceae",
  "family_without_author": "Caryophyllaceae",
  "observations_count": 1,
  "images_count": 2,
  "images_count_flower": 1,
  "images_count_fruit": 1,
  "images_count_leaf": 0,
  "images_count_bark": 0,
  "images_count_habit": 0,
  "images_count_branch": 0,
  "images_count_scan": 0,
  "images_count_sheet": 0,
  "images_count_other": 0,
  "best_images": [
    "dc8da0c6a9c532cb4575cd2681a574e0fb403acd",
    "c3d216aabd144128908f7bffbb8889e5b6ec9867"
  ],
  "tmp_best_images": [
    {
      "id": "dc8da0c6a9c532cb4575cd2681a574e0fb403acd",
      "order": 0,
      "score": 3,
      "date":<PHONE_NUMBER>000
    },
    {
      "id": "c3d216aabd144128908f7bffbb8889e5b6ec9867",
      "order": 1,
      "score": 3,
      "date":<PHONE_NUMBER>000
    }
  ],
  "genus_observations_count": 2311,
  "genus_images_count": 4431,
  "genus_species_count": 65,
  "genus_best_images": [
    "294ad592cba741e4d65922871b629c37bf43f8ad",
    "4896947a989e20e94e742ede9dffb9a2edd02676",
    "d2b4bb7017e4ba80e6211c5b2a627cd8c7ecb684",
    "87a1d9fe783de0a7b991f48e6944cca6679c59f4"
  ],
  "family_species_count": 280,
  "family_genera_count": 37,
  "family_best_images": [
    "294ad592cba741e4d65922871b629c37bf43f8ad",
    "4896947a989e20e94e742ede9dffb9a2edd02676",
    "a559e92dbee0d193b9a5a06611b5437477d74603",
    "d2b4bb7017e4ba80e6211c5b2a627cd8c7ecb684"
  ],
  "family_observations_count": 9252,
  "family_images_count": 16875,
  "images_count_anatomy": 0,
  "images_count_exclude": 0,
  "images_count_seed": 0,
  "images_count_bud": 0,
  "images_count_drawing": 0,
  "images_count_landscape": 0
}

Size of your Dataset on disk:
Replication Factor & Number of Shards (Cluster only):
Steps to reproduce

Run the query

Problem:
{
	code: 500,
	error: true,
	errorMessage: 'Index out of bounds (while executing) (exception location: /work/ArangoDB/arangod/RestHandler/RestCursorHandler.cpp:289). Please report this error to arangodb.com',
	errorNum: 4
}

ArangoError: Index out of bounds (while executing) (exception location: /work/ArangoDB/arangod/RestHandler/RestCursorHandler.cpp:289). Please report this error to arangodb.com
    at new ArangoError (/home/admin/test-data-access-layer-read/node_modules/arangojs/error.js:115:21)
    at Object.resolve (/home/admin/test-data-access-layer-read/node_modules/arangojs/connection.js:410:37)
    at callback (/home/admin/test-data-access-layer-read/node_modules/arangojs/connection.js:192:26)
    at IncomingMessage.<anonymous> (/home/admin/test-data-access-layer-read/node_modules/arangojs/lib/request.node.js:108:21)
    at IncomingMessage.emit (node:events:406:35)
    at endReadableNT (node:internal/streams/readable:1331:12)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)

Expected result:

Can you provide a minimal example to reproduce the issue, or potentially share the dataset (in private) so that we can try to reproduce it on our end?

Hi,
Yes, I can share the dataset with you, what is the proper way to do it?

Hi @AntoineAA,
As Simran-B mentioned, you can send us a mail to<EMAIL_ADDRESS>attaching a dump of your dataset (using arangodump) and referencing the GitHub issue number. In case your dataset is too big for an attachment, please still send us a mail and we will provide you with an upload location.

The email has been sent!


FR: Container Registration Validation

Feature Request: Container Registration Validation.

Validate all registrations can be resolved.
Validate that all IHttpControllers can be resolved.

Really don't think this could be done or worth the effort.

A lot of other IoC containers have the feature, and we have currently implemented our own unit tests that simply just tries to resolve all registrations and controllers inheriting from IHttpController. Just thought it would be great to have the feature included in the container with awesome error messages, but I guess not.

The validation is only possible when entire dependency graph is created and available at the time of validation. Unity does not create nodes for the type until type is requested to be crated. So, to put it simply, it would be as if you enumerate registered types and create them all to verify.
You could do it already, it does not require any additional implementation.


Namespaces / Modules - the preferred way

Hi all,
I'm not sure if this is a small typo/mistake or if I'm not getting it right. On the Namespaces and Modules page there seems to be 2 preferred ways of architecting the code.
The top of the document says:

A note about terminology: It’s important to note that in TypeScript 1.5, the nomenclature has changed. “Internal modules” are now “namespaces”. “External modules” are now simply “modules”, as to align with ECMAScript 2015’s terminology, (namely that module X { is equivalent to the now-preferred namespace X {).

Then later under the "Modules" sections the documents says the opposite:

Starting with ECMAScript 2015, modules are native part of the language, and should be supported by all compliant engine implementations. Thus, for new projects modules would be the recommended code organization mechanism.

So my guess is that the mistake is probably in the heading.

Sorry, I should have opened this in the handbook repo. Closing and creating a new issue there.


FAILURE: Build failed with an exception.

When I run flutter build apk get the following information:
* Where:
Build file 'C:\Users\SILENCE\flutter\.pub-cache\hosted\pub.flutter-io.cn\screen-0.0.5\android\build.gradle' line: 11

* What went wrong:
A problem occurred evaluating root project 'screen'.
> Flutter SDK not found. Define location with flutter.sdk in the local.properties file.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 2s

The plugin screen could not be built due to the issue above.

flutter doctor
Doctor summary (to see all details, run flutter doctor -v):
[√] Flutter (Channel stable, v1.12.13+hotfix.9, on Microsoft Windows [Version 10.0.18363.778], locale zh-CN)

[√] Android toolchain - develop for Android devices (Android SDK version 29.0.3)
[√] Android Studio (version 3.6)
[√] VS Code (version 1.44.2)
[√] Connected device (1 available)

• No issues found!

I'm getting this error too.
Latest stable branch with Flutter Add-to-app.

yep. it appears that this is because of these lines in the build.gradle of this project.
i think these are unnecessary and should be removed:
https://github.com/clovisnicolas/flutter_screen/blob/3e9c4e3cc5281d5ea6804f468e55793deb7d1b08/android/build.gradle#L9-L12
@clovisnicolas i'm happy to open a PR to address this issue. are you still maintaining this project?

it looks like a work-around is to pin your version to 0.0.4 before this change was added https://github.com/clovisnicolas/flutter_screen/commit/3e9c4e3cc5281d5ea6804f468e55793deb7d1b08


Improve logging

I want to improve logging in the following ways:
First of all, I want to introduce a Winston to have a customizeable and better logging framework. I want to by default have less verbose logging to the console so you can follow what the bot does, and more verbose logging to a file to debug issues. In a second step, I want to add code so you can configure the logging, but I won't prioritize that from the start.
I also want to expand on the logging inside the bot, so it's easier to follow what the bot is doing. What kind of feed is it currently following, how much is left of it, whose feed is it, and so on.
I'll make these changes and submit a PR.

really cool ! guess this is really important :)
you could also just use -v command (in the config.ts add a property called verbose: boolean)
and depending on that boolean verbose logging is enabled :)
btw about the log file, right now the user has to pass a session.json path to the program, we could change that, so the user has to pass a 'data' dir, and in that dir we could save the session and the log file :)


Put the CC3220SF.ccxml back into the ti CC3220SF vendors folder.

This file is needed not only for debug in the CCS IDE, but it is also needed to just flash the device post building with Uniflash.
All files needed for flashing with the resulting build from CMAKE should be under the vendors/ folder.

Description

Checklist:

[ ] I have tested my changes. No regression in existing tests.
[ ] My code is Linted.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/bot run checks


initialValue not working as expected

Hello CHB61
Thanks for this great flutter lib.
Seems the initialValue property is not working correctly. Based on your example I added all animals as an initial value:
MultiSelectBottomSheetField(     initialValue: _animals,     .....                 ),
As a result the selected animals are not shown inside the form. But all animals are pre-selected on the bottom selection area (see screenshot).

Hi @jjoschyy,
Thank you for submitting this issue. I can see that the chips do not get displayed on the default chipDisplay when an initialValue is set on the MultiSelectBottomSheetField. This will be fixed.
I want to point out that the selected values in the bottomsheet will be set to initialValue every time you open it, even if you changed the selection!
By setting initialValue: _animals, it will have all options selected every time you open it and this is probably not the behavior you want. ex:

To provide background on this param:
The initialValue param is required on MultiSelectBottomSheet and MultiSelectDialog and should be set to the list of selected values so that they are highlighted when the bottomsheet or dialog is re-opened.
In the case of MultiSelectBottomSheetField/MultiSelectDialogField, the initialValue param is not required as there is an internal list of selected items that is being passed from MultiSelectBottomSheetField to MultiSelectBottomSheet every time it is
opened.
However, this param becomes useful when your MultiSelect widget is being re-inserted into the widget tree. For example if you have a MultiSelectDialogField inside of your own custom BottomSheet, you might:

open the bottomsheet
select values in the MultiSelectDialogField
close the bottom sheet
open the bottomsheet again and expect the MultiSelectDialogField to have the same values selected

Setting MultiSelectDialogField.initialValue to the list of selected items would achieve this.
Basically the initialValue should always be set to _selectedValues.
But what if you just want some pre-selected items the first time it is opened?
I recommend following these steps to pre-select values before initial interaction:

Set initialValue as the list of _selectedValues
In initState, set _selectedValues to the pre-selected values that you want
Add onConfirm param with setState that updates the _selectedItems

Hope this helps :)

Hi @jjoschyy,
This issue is fixed in version 3.1.0.
Please take a look at the changelog and let me know if this solves your issue.

Hello CHB61
Thanks a lot for your detailed answer. This helped me a lot. Also thanks for the fix. I can confirm the inital values are now shown correctly.
Nevertheless, one issue seems still existing. After adding inital values, the remove action is not working anymore. Example:

On this action, the selected value is removed from the _selected list. This works, the value will be removed from the _selected list. But there is no refresh on the MultiSelectChipDisplay.
But if I manually select any other item from the list, the remove action works.
I follwed your recommondation:

Set initialValue as the list of _selectedValues
In initState, set _selectedValues to the pre-selected values that you want
Add onConfirm param with setState that updates the _selectedValues

Thanks again for your help!

@jjoschyy Hi! Sorry for the delay! Been so busy last couple weeks but I'll take a look into this problem ASAP.

Hi @jjoschyy ,
Sorry for the delay on this one. I was able to reproduce the issue.
The reason you can only delete from the chip display after doing a confirm is because in the onConfirm function,
I am passing a reference of the internal list of selected values.
This means that only when you obtain a reference to the list that is provided in the onConfirm, can you manipulate it from onTap and then the changes will be updated.
onConfirm: (values) {
    _selectedAnimals = values; // gets reference to internal list of selected items
}

So it also means that if you were to change the code in the onConfirm to something like:
onConfirm: (values) {
    _selectedAnimals = [];
    _selectedAnimals.addAll(values);
}

Deleting from the chip display won't work at all now because there is no reference to the list that MultiSelectDialogField uses
to determine the selected items.
With onTap(V) all we know is which item was tapped, and there's no guarantee that the list is going to change.
So in order to delete without getting a reference to the internal list, the fix I've applied is to have MultiSelectDialogField check if
the onTap() function returned anything, and if it returned a List, update the state.
All you would need to do to is simply return the updated list in the onTap function. Example:
onTap: (item) {
    _selectedAnimals.remove(item);
    return _selectedAnimals;
}

You don't even need to set state anymore.
I was considering not passing by reference at all in the onConfirm since it wasn't intentional but I decided to leave that for now.
This leaves both options available to delete from chipDisplay:

Get a reference to internal list of selected items in onConfirm, and setState when doing the remove operation in onTap.
(Only works if you don't need to delete before confirm)
Return the updated list in onTap.
(No need to set state here and works any time)

This change is available in version 3.1.4.
Hope this helps :)

@CHB61 Please open this issue again  The issue is still exist for me, initialValue can't be shown

This was not working for me neither because I followed the example of pub.dev that is using the class without the Generics, like:
MultiSelectBottomSheetField(

instead of:
MultiSelectBottomSheetField<Animal>(

following the repo example generics use, my initialValue code started to work. I hope this helps.

@CHB61 I'm having issues having initial values because I'm setting them asynchronously, but these values are being set on MultiSelectDialogField's initState callback only. That means it won't rerender as soon as the initial values are ready or they get updated. For now, I'm working around this by using a loading flag and rendering the widget when the data is ready... but this is far from ideal. Any plans to get this sorted?
BTW, thanks for the awesome lib!

I have the same problem.

check this please it may help
https://stackoverflow.com/questions/68780670/initialvalue-isnt-working-properly-in-multi-select-package-flutter/70680978#70680978

Hi @jjoschyy ,
Sorry for the delay on this one. I was able to reproduce the issue.
The reason you can only delete from the chip display after doing a confirm is because in the onConfirm function, I am passing a reference of the internal list of selected values.
This means that only when you obtain a reference to the list that is provided in the onConfirm, can you manipulate it from onTap and then the changes will be updated.
onConfirm: (values) {
    _selectedAnimals = values; // gets reference to internal list of selected items
}

So it also means that if you were to change the code in the onConfirm to something like:
onConfirm: (values) {
    _selectedAnimals = [];
    _selectedAnimals.addAll(values);
}

Deleting from the chip display won't work at all now because there is no reference to the list that MultiSelectDialogField uses to determine the selected items.
With onTap(V) all we know is which item was tapped, and there's no guarantee that the list is going to change.
So in order to delete without getting a reference to the internal list, the fix I've applied is to have MultiSelectDialogField check if the onTap() function returned anything, and if it returned a List, update the state.
All you would need to do to is simply return the updated list in the onTap function. Example:
onTap: (item) {
    _selectedAnimals.remove(item);
    return _selectedAnimals;
}

You don't even need to set state anymore.
I was considering not passing by reference at all in the onConfirm since it wasn't intentional but I decided to leave that for now. This leaves both options available to delete from chipDisplay:

Get a reference to internal list of selected items in onConfirm, and setState when doing the remove operation in onTap.
(Only works if you don't need to delete before confirm)
Return the updated list in onTap.
(No need to set state here and works any time)

This change is available in version 3.1.4.
Hope this helps :)

Hi,it helps for me , but now I am unable to get values from list, it shows instances.

@MuhammadSufyanMalik could you help me in this one
onTap does not update initial values
onTap: (value) {
              selected.removeWhere((element) => element.getId==value.getId);
              pprint(selected);
              ///my code

              return selected;
          },


Enhance YARN app state judgment

:mag: Description
Issue References 🔗

This pull request aims to enhance the YARN app state judgment.
The combination of FINISHED and FAILED is valid, but Kyuubi now print ERROR logs

[ERROR] YarnApplicationOperation#94 - Unknown YARN state FINISHED for app application_1700805876733_11637 with final status FAILED.

Describe Your Solution 🔧
The fix just suppresses the ERROR logs, does not change the behavior, because the previous implementation treats all invalid combinations as FAILED too.
Types of changes :bookmark:

[x] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Test Plan 🧪
Behavior Without This Pull Request :coffin:
Behavior With This Pull Request :tada:
Related Unit Tests

Checklists
📝 Author Self Checklist

[x] My code follows the style guidelines of this project
[x] I have performed a self-review
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[x] This patch was not authored or co-authored using Generative Tooling

📝 Committer Pre-Merge Checklist

[ ] Pull request title is okay.
[ ] No license issues.
[ ] Milestone correctly set?
[ ] Test coverage is ok
[ ] Assignees are selected.
[ ] Minimum number of approvals
[ ] No changes are requested

Be nice. Be informative.

Codecov Report
Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (86bec4d) 61.61% compared to head (2ba30ce) 61.48%.

Files
Patch %
Lines

...pache/kyuubi/engine/YarnApplicationOperation.scala
0.00%
0 Missing and 2 partials :warning:

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #5922      +/-   ##
============================================
- Coverage     61.61%   61.48%   -0.13%
  Complexity       23       23
============================================
  Files           616      616
  Lines         36387    36388       +1
  Branches       4978     4979       +1
============================================
- Hits          22419    22374      -45
- Misses        11557    11586      +29
- Partials       2411     2428      +17

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Thanks, merged to master/1.8.1/1.7.4


Field errors and description

Love that conform handle errors for me in an accessible way. But one question, should it use aria-errormessage instead of aria-describedby?
I would also like to have conform handle field descriptions for me. Adding aria-describedby with a unique id if description is set to true. Maybe the api could look something like this:
<input {...conform.input({ ...email, description: true }, { type: "email" })} />

Company email

Thanks for the question. My knowledge on accessibility topic is quite limited, so the main reason I go for aria-describedby was simply based on suggestion from articles like this which has concern on the limited support of aria-errormessage.
But what you suggested sounds fair to me too. How about putting both id on aria-describedby as suggested on the article above too?
<input {...conform.input(email, { type: "email" })} />
// aria-describedby={`${errorId}`}

<input {...conform.input(email, { type: "email", description: true })} />
// aria-describedby={`${errorId} ${descriptionId}`}

I'm no accessibility export, so this is something I'm unsure about (didn't even think about checking support for aria-errormessage). Yes, sounds like that's the way to go by adding both id's in aria-describedby.
One thing I came across trying to read up on the topic, it looks like some only include the "errorId" in aria-describedby if there's an error (https://www.a11yproject.com/posts/how-to-write-accessible-forms/#provide-error-messages). Unsure about the difference of leaving the "errorId" element empty vs toggling the id in aria-describedby depending on if there's an error. What do you think?

I got the same question as you. It was tempting to not provide the extra description option and just have the descriptionId added for always.
I couldn't find much information on this topic except this question on stackoverflow. I think I am convinced it's always a good idea to ensure the HTML to be semantically correct. As I could not guarantee the error element to be always rendered (i.e. there is no element with the errorId when no error), it seems better to add the errorId only if there is an actual error.
Like this:
<input {...conform.input(email, { type: "email" })} />
// If valid, no aria attributes will be set
// If invalid, aria-invalid="true" aria-describedby="errorId"

<input {...conform.input(email, { type: "email", description: true })} />
// If valid, aria-describedby="descriptionId"
// If invalid, aria-invalid="true" aria-describedby="errorId descriptionId"

Based on that stackoverflow post this looks perfect!

This is now supported on v0.6.1 🚀


Render video in window?

Is it possible to play a video as a texture or otherwise within some part of the window?

It's possible and has been done already afaik (https://github.com/unpatched/internet-broadcasting-service)
The basic idea would be to copy every frame into an upload buffer and copy it to the texture. The texture will be drawn with a simple fullscreen shader pipeline.
Starting points:

Upload buffer creation and writing: https://github.com/gfx-rs/gfx/blob/v0.16/examples/instancing/main.rs#L118-L122
Copy buffer to a texture (non-immutable!): https://github.com/gfx-rs/gfx/blob/v0.16/src/render/src/encoder.rs#L263
blend, flowmap or other examples for setting up a pipeline to draw a texture to the screen.

Video encoding/decoding and other related operations are out of scope for gfx-rs!

A preferable way to render a decoded video streams, when using hardware video decoding, is to use the GL_TEXTURE_EXTERNAL_OES target in GL or equivalents in other APIs and sample from decoded frames as "regular" textures.

Understood that encoding/decoding are outside the scope of gfx-rs.  That technique sounds reasonable, and I appreciate the examples.  Will do the naive copies and check if that's performant enough.  Thanks!


Add support for redirect after calling endsession

We have identity server in our company. Our process of logout is following:

client calls endsession with parameter id_token_hint and post_logout_redirect_uri.
In success case, the call ends up with HTPP status 302 and redirects client to the browser with logout page which removes all proper cookies and so on.
Then the user is redirected to the post_logout_redirect_uri.

Is it possible to add the support for redirect in case the endsession endpoint requires that?

Ok so currently we just issue a POST request to the end session endpoint, mainly because i thought it would be stupid to open a browser just for a logout.
I know AppAuth does this differently and opens a browser for the logout.
My problem with that is that some IDPs may not respect the post_logout_redirect_uri so the browser stays open.
Maybe we could implement this feature in a way such that if post_logout_redirect_uri is provided, logout will be carried out be browser. If not, we just issue a POST as before.

It would be good to have it there. I don't know what is the specification of AppAuth (I'm not an expert :-)) but I think this is common requirement.
JFYI:
Here is our example from different app with different library. As you can see there is a GET call which ends with Redirect. User is redirected to proper page and then back.


Deprecate experimental folder

Pull Request Functional Test Report for #3483 / 8bf7340bfbae847bc25b7621182a4e76c815f3ef
No tests identified for validation.
Help

Pull Request Test Coverage Report for Build<PHONE_NUMBER>7
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 55.268%

Totals

Change from base Build<PHONE_NUMBER>2:
0.0%

Covered Lines:
1983

Relevant Lines:
3588

💛  - Coveralls

Closing this in favor of https://github.com/openconfig/featureprofiles/pull/3469#issuecomment-2383789662 which should add this change to it.  @self-maurya FYI


Decide whether ids need to be globally unique or not

Since all entities share the same entity log, right now entity ids need to be unique across all entity types. Either the queries could be improved not to make this assumption (add another WHERE clause everywhere) or this assumption needs to be documented.

Hmm ... so change UNIQUE (entity_id, version) to UNIQUE (entity_id, type_name, version)?

Yes that, and also add an extra constraint on the type to all queries.

I actually already ran into this problem because you can just load the entity of a certain id with the wrong entity type and it will happily recover that entity (which somewhat likely fails during deserializing the wrong event types but might not depending on lucky circumstances).

Done in #38.


Express 5 router strange behavior (parameters)

Hi.
I have pretty simple code to compare express route parameters (wich uses "router" package) and path-match, both use path-to-regexp under the hood.
import Pathmatch from 'path-match'

const routeMatch = Pathmatch({
	sensitive: false,
	strict: false,
	// end: false
})('/api/:module/:s*');

app.get('/api/:module/:s*', (req, res) => {
	const pr = {};
	pr['srv'] = req.params;
	pr['lib'] = routeMatch(req.path);

	res.json(pr);
});

and i get strange results with path "/api/about/first/second":
{
  "srv":{
    "0": "/second",
    "module": "about",
    "s":"first"
  },
  "lib":{
    "module": "about",
    "s":["first","second"]
  }
}

Also express doesnt handle path "/api/about" in that case.
Thats a bug or my missunderstanding of concepts?
--
Express 5 alpha-6
Router 1.3.2
Path-match 1.2.4

It looks like this is because of the version of path-to-regexp Express is currently using. That version defines the * character in the path as it's own expression, not part of the :s name. In other words, it see the route /api/:module/:s* as the pieces /api/, :module, /, :s, and *, so you get :module as a named match in the second segment, :s as a named mated in the third segment, and all the rest of the URL is slurped up into a positional match (0 since it's the first positional one).

Oh, i've forgot that there may be "local" node_modules that "replaces" the project's node_modules. I thought it uses project global scope.


gogs payload validation failed

I'm trying to use the webhook with Gogs (Go Git Server) and the payload validation is failing:
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | [webhook] 2020/05/19 23:25:29 [c84770] incoming HTTP GET request from <IP_ADDRESS>:52448
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | [webhook] 2020/05/19 23:25:29 [c84770] gogs got matched
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | [webhook] 2020/05/19 23:25:29 [c84770] error parsing JSON payload EOF
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | [webhook] 2020/05/19 23:25:29 [c84770] error evaluating hook: invalid payload signatures [76cda40505e58837211ee37fe7e8c819e31f33a1221a686fbebbe5829002ea01]
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | > [c84770] GET /hooks/gogs
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | > [c84770] Accept-Encoding: gzip
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | > [c84770] Content-Type: application/json
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | > [c84770] Referer: http://nixie.home.honig.net:18140//hooks/gogs
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | > [c84770] User-Agent: GogsServer
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | > [c84770] X-Github-Delivery: f45348ae-7fdf-4697-8230-45b07e55e056
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | > [c84770] X-Github-Event: push
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | > [c84770] X-Gogs-Delivery: f45348ae-7fdf-4697-8230-45b07e55e056
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | > [c84770] X-Gogs-Event: push
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | > [c84770] X-Gogs-Signature: 76cda40505e58837211ee37fe7e8c819e31f33a1221a686fbebbe5829002ea01
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | > [c84770] id:
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | < [c84770] Internal Server Error
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | < [c84770] Error occurred while evaluating hook rules.
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k |
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | [webhook] 2020/05/19 23:25:29 [c84770] 500 | 43 B | 133.025µs | nixie.home.honig.net:18140 | GET /hooks/gogs

I'm using a version of https://github.com/camptocamp/docker-g10k-webhook where I've upgraded to 2.7.0 and swapped -verbose for -debug to try to debug this.
Now the funny thing is that I have python flask implementation of a hook that verifies the signatures that has been working with Gogs webhooks for a long time.
The template file looks to be derrived from the example in the git repo:
- id: gogs
  execute-command: /push-to-g10k.sh
  command-working-directory: /
  pass-arguments-to-command:
  - source: payload
    name: ref
  - source: payload
    name: repository.ssh_url
  trigger-rule:
    and:
      - match:
          type: payload-hash-sha256
          secret: "{{ getenv "HOOK_SECRET" | js }}"
          parameter:
            source: header
            name: X-Gogs-Signature

I've verified that HOOK_SECRET in the environment is correct.  I even replaced

with
                secret: FRED

(which is the secret I'm using so that I can share stuff here).
Any tips to help debug this?
Gogs gives me the following information about the hook call:
Request URL: http://nixie.home.honig.net:18140//hooks/gogs
Request method: POST
Content-Type: application/json
X-Github-Delivery: 9a37cb8c-151c-4801-9f38-f722e93c85fa
X-Github-Event: push
X-Gogs-Delivery: 9a37cb8c-151c-4801-9f38-f722e93c85fa
X-Gogs-Event: push
X-Gogs-Signature: 76cda40505e58837211ee37fe7e8c819e31f33a1221a686fbebbe5829002ea01

{
  "ref": "refs/heads/production",
  "before": "0d63846da1e52542b34ec1cd7f63598a42e4a9fe",
  "after": "0d63846da1e52542b34ec1cd7f63598a42e4a9fe",
  "compare_url": "",
  "commits": [
    {
      "id": "0d63846da1e52542b34ec1cd7f63598a42e4a9fe",
      "message": "fixup! Fix trim_ssd errors with FUSE% variables\n",
      "url": "https://git.home.honig.net/Sysadmin/puppet/commit/0d63846da1e52542b34ec1cd7f63598a42e4a9fe",
      "author": {
        "name": "Jeffrey C Honig",
        "email"<EMAIL_ADDRESS>        "username": "jch"
      },
      "committer": {
        "name": "Jeffrey C Honig",
        "email"<EMAIL_ADDRESS>        "username": "jch"
      },
      "added": null,
      "removed": null,
      "modified": [
        "site/profile/files/bin/trim_ssd"
      ],
      "timestamp": "0001-01-01T00:00:00Z"
    }
  ],
  "repository": {
    "id": 10,
    "owner": {
      "id": 3,
      "username": "Sysadmin",
      "login": "Sysadmin",
      "full_name": "Sysadmin repos",
      "email": "",
      "avatar_url": "https://git.home.honig.net/avatars/3"
    },
    "name": "puppet",
    "full_name": "Sysadmin/puppet",
    "description": "Server puppet configs",
    "private": true,
    "fork": false,
    "parent": null,
    "empty": false,
    "mirror": false,
    "size": 23244800,
    "html_url": "https://git.home.honig.net/Sysadmin/puppet",
    "ssh_url": "git@git-new.home.honig.net:Sysadmin/puppet.git",
    "clone_url": "https://git.home.honig.net/Sysadmin/puppet.git",
    "website": "",
    "stars_count": 0,
    "forks_count": 0,
    "watchers_count": 2,
    "open_issues_count": 0,
    "default_branch": "production",
    "created_at": "2017-04-22T23:31:30Z",
    "updated_at": "2020-05-19T13:02:51Z"
  },
  "pusher": {
    "id": 1,
    "username": "jch",
    "login": "jch",
    "full_name": "Jeffrey Honig",
    "email"<EMAIL_ADDRESS>    "avatar_url": "https://secure.gravatar.com/avatar/b25e2ae1f07ec30647f346df81d2d841"
  },
  "sender": {
    "id": 1,
    "username": "jch",
    "login": "jch",
    "full_name": "Jeffrey Honig",
    "email"<EMAIL_ADDRESS>    "avatar_url": "https://secure.gravatar.com/avatar/b25e2ae1f07ec30647f346df81d2d841"
  }
}

Any other suggestions for debugging this?
Thanks.
Jeff

Payload signature validation is failing.  Can you test it with 2.6.11?

Same failure:
[webhook] 2020/05/20 14:29:27 version 2.6.11 starting
[webhook] 2020/05/20 14:29:27 setting up os signal watcher
[webhook] 2020/05/20 14:29:27 attempting to load hooks from /etc/webhook/g10k.yaml.tmpl
[webhook] 2020/05/20 14:29:27 os signal watcher ready
[webhook] 2020/05/20 14:29:27 found 4 hook(s) in file
[webhook] 2020/05/20 14:29:27 	loaded: github
[webhook] 2020/05/20 14:29:27 	loaded: gogs
[webhook] 2020/05/20 14:29:27 	loaded: gitlab
[webhook] 2020/05/20 14:29:27 	loaded: payload
[webhook] 2020/05/20 14:29:27 serving hooks on http://<IP_ADDRESS>:9000/hooks/{id}
[webhook] 2020/05/20 14:29:38 301 | 57.593µs | nixie.home.honig.net:18140 | POST //hooks/gogs
[webhook] 2020/05/20 14:29:38 [41f1fb] incoming HTTP request from <IP_ADDRESS>:44810
[webhook] 2020/05/20 14:29:38 [41f1fb] gogs got matched
[webhook] 2020/05/20 14:29:38 500 | 302.084µs | nixie.home.honig.net:18140 | GET /hooks/gogs
[webhook] 2020/05/20 14:29:38 [41f1fb] error parsing JSON payload EOF
[webhook] 2020/05/20 14:29:38 [41f1fb] error evaluating hook: invalid payload signature 76cda40505e58837211ee37fe7e8c819e31f33a1221a686fbebbe5829002ea01
[webhook] 2020/05/20 14:31:57 301 | 11.945µs | nixie.home.honig.net:18140 | POST //hooks/gogs

I've validated that the other webhook listener I wrote receives the exact same payload and signature.  For reference, the code I have that validates the hook successfully:
def verify_signature(request):
    """ Verify the signature if provided """

    # Validate signature if required
    if 'GOGS_SECRET' in WEBHOOK.config:
        gogs_secret = WEBHOOK.config['GOGS_SECRET'].encode('utf-8')
        if gogs_secret:
            signature = request.headers.get('X-Gogs-Signature')
            if not signature:
                logging.error("Request has no signature")
                return False

            valid_signature = hmac.HMAC(gogs_secret,
                                        request.data,
                                        digestmod=hashlib.sha256).hexdigest()
            if not hmac.compare_digest(signature, valid_signature):
                logging.error("Signature %s expected %s", signature, valid_signature)
                return False

    return True

I added a test case for this, and it works as expected.  Hmm.
One thing that's throwing me off is the error parsing JSON payload EOF message you're seeing.  For some reason, webhook is failing to parse the payload even though it appears to be valid JSON.  I'm not seeing that message in my testing.
So, what are we missing?

What Headers is the gogs sending to webhook?

Can you run nc -l -v -p WEBHOOK_PORT instead of webhook, trigger the hook and post the raw output of the request that you get. (nc is the netcat)

This part totally looks out of place:
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | > [c84770] id:

root@nixie:/vol/services/puppetserver# nc -l -v -p 18140
Listening on <IP_ADDRESS> 18140
Connection received on <IP_ADDRESS> 42140
POST //hooks/gogs HTTP/1.1
Host: nixie.home.honig.net:18140
User-Agent: GogsServer
Content-Length: 2191
Content-Type: application/json
X-Github-Delivery: 1139bd01-a5af-449a-9233-8662c90a1393
X-Github-Event: push
X-Gogs-Delivery: 1139bd01-a5af-449a-9233-8662c90a1393
X-Gogs-Event: push
X-Gogs-Signature: 00ad1632fdd477da3af0a69bd6c526b5c1c97f3933e3a80d1b8f692cbc3f6a2b
Accept-Encoding: gzip

{
  "ref": "refs/heads/production",
  "before": "0d63846da1e52542b34ec1cd7f63598a42e4a9fe",
  "after": "0d63846da1e52542b34ec1cd7f63598a42e4a9fe",
  "compare_url": "",
  "commits": [
    {
      "id": "0d63846da1e52542b34ec1cd7f63598a42e4a9fe",
      "message": "fixup! Fix trim_ssd errors with FUSE% variables\n",
      "url": "https://git.home.honig.net/Sysadmin/puppet/commit/0d63846da1e52542b34ec1cd7f63598a42e4a9fe",
      "author": {
        "name": "Jeffrey C Honig",
        "email"<EMAIL_ADDRESS>        "username": "jch"
      },
      "committer": {
        "name": "Jeffrey C Honig",
        "email"<EMAIL_ADDRESS>        "username": "jch"
      },
      "added": null,
      "removed": null,
      "modified": [
        "site/profile/files/bin/trim_ssd"
      ],
      "timestamp": "0001-01-01T00:00:00Z"
    }
  ],
  "repository": {
    "id": 10,
    "owner": {
      "id": 3,
      "username": "Sysadmin",
      "login": "Sysadmin",
      "full_name": "Sysadmin repos",
      "email": "",
      "avatar_url": "https://git.home.honig.net/avatars/3"
    },
    "name": "puppet",
    "full_name": "Sysadmin/puppet",
    "description": "Server puppet configs",
    "private": true,
    "fork": false,
    "parent": null,
    "empty": false,
    "mirror": false,
    "size": 23244800,
    "html_url": "https://git.home.honig.net/Sysadmin/puppet",
    "ssh_url": "git@git-new.home.honig.net:Sysadmin/puppet.git",
    "clone_url": "https://git.home.honig.net/Sysadmin/puppet.git",
    "website": "",
    "stars_count": 0,
    "forks_count": 0,
    "watchers_count": 2,
    "open_issues_count": 0,
    "default_branch": "production",
    "created_at": "2017-04-22T23:31:30Z",
    "updated_at": "2020-05-19T13:02:51Z"
  },
  "pusher": {
    "id": 1,
    "username": "jch",
    "login": "jch",
    "full_name": "Jeffrey Honig",
    "email"<EMAIL_ADDRESS>    "avatar_url": "https://secure.gravatar.com/avatar/b25e2ae1f07ec30647f346df81d2d841"
  },
  "sender": {
    "id": 1,
    "username": "jch",
    "login": "jch",
    "full_name": "Jeffrey Honig",
    "email"<EMAIL_ADDRESS>    "avatar_url": "https://secure.gravatar.com/avatar/b25e2ae1f07ec30647f346df81d2d841"
  }
}```

This part totally looks out of place:
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | > [c84770] id:

That was 2.7.0 with -debug flag.

This part totally looks out of place:
May 19 19:25:29 nixie docker-compose[3608844]: puppet_g10k | > [c84770] id:

That looks like a bug in the debug logger.  Doesn't look to be related to this issue.  I'll fix it separately.

@jchonig, are you sure that nc sample is correct?  It looks like the same body as before but the header has a new signature.

@jchonig, are you sure that nc sample is correct? It looks like the same body as before but the header has a new signature.

I updated the comment above with the correct output.  I'd changed the password to compare it to the hook that worked and didn't do all the steps to switch it back. Or screwe up the copy and paste.  Sorry about that.

The Content-Length header differs from the actual content length, that's why it fails to parse the payload.

Hmm, I managed to reproduce the error parsing JSON payload unexpected EOF  message by specifying Content-Length header value to something less or more than the actual payload size. Could it be an encoding issue? Weird locale?

That would certainly explain it.
I updated my working python web hook as listed above and added:
            logging.warning("Content-Length: %s data length %d",
                            request.headers.get('Content-Length'),
                            len(request.data))

I get:
May 21 07:29:07 mittens r10kd[4640]: Content-Length: 2176 data length 2176

So it appears that Gogs is sending the correct data.

Perhaps we should try to rule out the environment first, could you try launching a webhook instance on your host, outside of docker, and see if the signature is still being miscalculated?
Unfortunately(?) my local setup calculates the correct signature for this scenario, so my guess would be that either there is something that gogs is sending that is being lost in the copy/paste, or the environment in which webhook is running is making it misbehave.

Using the version in the Ubuntu 20.04 repos:
jch@nixie[535]:docker-g10k-webhook$ webhook --hooks ./g10k.yaml.tmpl -template -verbose --port 18140
[webhook] 2020/05/21 08:57:57 version 2.6.9 starting
[webhook] 2020/05/21 08:57:57 setting up os signal watcher
[webhook] 2020/05/21 08:57:57 attempting to load hooks from ./g10k.yaml.tmpl
[webhook] 2020/05/21 08:57:57 os signal watcher ready
[webhook] 2020/05/21 08:57:57 found 4 hook(s) in file
[webhook] 2020/05/21 08:57:57 	loaded: github
[webhook] 2020/05/21 08:57:57 	loaded: gogs
[webhook] 2020/05/21 08:57:57 	loaded: gitlab
[webhook] 2020/05/21 08:57:57 	loaded: payload
[webhook] 2020/05/21 08:57:57 serving hooks on http://<IP_ADDRESS>:18140/hooks/{id}
[webhook] 2020/05/21 08:58:12 Started POST //hooks/gogs
[webhook] 2020/05/21 08:58:12 Completed 301 Moved Permanently in 59.25µs
[webhook] 2020/05/21 08:58:12 Started GET /hooks/gogs
[webhook] 2020/05/21 08:58:12 [98d411] incoming HTTP request from <IP_ADDRESS>:41840
[webhook] 2020/05/21 08:58:12 [98d411] gogs got matched
[webhook] 2020/05/21 08:58:12 [98d411] error parsing JSON payload EOF
[webhook] 2020/05/21 08:58:12 [98d411] error evaluating hook: invalid payload signature 76cda40505e58837211ee37fe7e8c819e31f33a1221a686fbebbe5829002ea01
[webhook] 2020/05/21 08:58:12 Completed 500 Internal Server Error in 148.516µs

Let me see if I can build from master and add some debugging.

Could you also try one more thing before doing that, fix the request url so that it's hooks/gogs instead of /hooks/gogs in the gogs?

I'm starting to wonder if gogs webhook sending logic can handle redirects properly :/

Could you also try one more thing before doing that, fix the request url so that it's hooks/gogs instead of /hooks/gogs in the gogs?

D'oh!  That was it!
Here's some debugging that I added:
diff --git a/webhook.go b/webhook.go
index 3af8ab4..840bd4d 100644
--- a/webhook.go
+++ b/webhook.go
@@ -246,6 +246,9 @@ func hookHandler(w http.ResponseWriter, r *http.Request) {
 		if err != nil {
 			log.Printf("[%s] error reading the request body. %+v\n", rid, err)
 		}
+//		fmt.Printf("BODY: %d %d '%s'\n", len(body), len(string(body)), string(body))
+		fmt.Printf("REQUEST: %#v\n", r)
+		fmt.Printf("REQUEST: %#v\n", r.Body)

 		// parse headers
 		headers := valuesToMap(r.Header)

Which shows:
REQUEST: &http.Request{Method:"GET", URL:(*url.URL)(0xc00012a200), Proto:"HTTP/1.1", ProtoMajor:1, ProtoMinor:1, Header:http.Header{"Accept-Encoding":[]string{"gzip"}, "Content-Type":[]string{"application/json"}, "Referer":[]string{"http://nixie.home.honig.net:18140//hooks/gogs"}, "User-Agent":[]string{"GogsServer"}, "X-Github-Delivery":[]string{"9a37cb8c-151c-4801-9f38-f722e93c85fa"}, "X-Github-Event":[]string{"push"}, "X-Gogs-Delivery":[]string{"9a37cb8c-151c-4801-9f38-f722e93c85fa"}, "X-Gogs-Event":[]string{"push"}, "X-Gogs-Signature":[]string{"76cda40505e58837211ee37fe7e8c819e31f33a1221a686fbebbe5829002ea01"}}, Body:http.noBody{}, GetBody:(func() (io.ReadCloser, error))(nil), ContentLength:0, TransferEncoding:[]string(nil), Close:false, Host:"nixie.home.honig.net:18140", Form:url.Values(nil), PostForm:url.Values(nil), MultipartForm:(*multipart.Form)(nil), Trailer:http.Header(nil), RemoteAddr:"<IP_ADDRESS>:42716", RequestURI:"/hooks/gogs", TLS:(*tls.ConnectionState)(nil), Cancel:(<-chan struct {})(nil), Response:(*http.Response)(nil), ctx:(*context.valueCtx)(0xc0000f8360)}
REQUEST: http.noBody{}

Might I suggest checking for an empty body.  And also bailing if the json.Decode() fails?
Thanks for your help finding my typo!

Dang it.  I just noticed that your original post has the debug output and the body isn't printed.  Duh!  I should  have caught that.

Might I suggest checking for an empty body. And also bailing if the json.Decode() fails?

Well, empty bodies are acceptable in some scenarios. And a failed JSON decode may be acceptable, too, since we attempt to decode it even if you're not referencing it in the trigger rules.
I think more applicable to your issue would have been a warning that you requested a signature validation on an empty body.  That may be easy to add.
It would require more work (I think) but be even more helpful to add warnings if you use any payload references when an empty body is sent.
Braindump:
I don't have time to flesh this out right now, but wanted to leave a note (at least for myself).  Maybe we add a Hook property such as RequireBody bool that we set to true if we find any payload references in the hook definition.  We can bail early in the request processing if we get an empty body.


Remove git requirement for distribution testing

(and ideally wider than that)
For distribution at least, it seems possible to mount the artifacts, test scripts directories directly within docker. This would drastically reduce cloning time and increase our confidence that we're not accidentally taking dependencies.
Later, for regular builds, this seems desirable also. Way lower priority.

I think for testing this is going to form a hard requirement: multiple clones coming out of the same directory don't work, so we'll be limited to a single test environment at a time.

Actually, what's causing trouble is the with multiple parallel clones is git branch -f jenkins-docker.   Git clone can't really cause any trouble because the source repo is mounted as readonly, so there's no way for cloners to lock anything.


BigQuery: PDO-compatible layer for running BigQuery queries

Feature request from StackOverflow: https://stackoverflow.com/q/49789524/101923
Note: In the Python package, we do provide a DB-API layer, so it might also make sense to do something similar for PHP.

I'd categorize this as a low priority FR.  I'd love to see more feedback from the community on PDO usage before we engage here.  History from implementing similar DB abstractions for the other libraries/languages would indicate this is a pretty significant amount of work.  Are there heavily leveraged analytics/visualization frameworks in PHP that depend on PDO support, for example?
A quick perusal of https://www.php.net/manual/en/book.pdo.php would indicate that there's a lot of functionality related to OLTP-based relational databases, which are not relevant for an OLAP system such as BigQuery.

Going to close this out as a "Won't Do" for now.


AspNetCore extensions

Hi!
Is it possible to use this together with the Quartz.Extensions.DependencyInjection?
I was thinking of something like this:
                q.UsePersistentStore(ps =>
                {
                    ps.UseMongoDb(connectionString: MongoDbConfig.ConnectionString, prefix: "quartz");
                });

How would the extensionmethod UseMongoDb look like?

            services.AddQuartz(q =>
            {
                q.UseMicrosoftDependencyInjectionScopedJobFactory(options =>
                {
                    // if we don't have the job in DI, allow fallback to configure via default constructor
                    options.AllowDefaultConstructor = true;
                });

                q.UseSimpleTypeLoader();
                q.UseInMemoryStore();

                q.UseDefaultThreadPool(tp =>
                {
                    tp.MaxConcurrency = 5;
                });
            });

            // ASP.NET Core hosting
            services.AddQuartzServer(options =>
            {
                // when shutting down we want jobs to complete gracefully
                options.WaitForJobsToComplete = true;
            });

            var properties = new NameValueCollection();
            properties["quartz.scheduler.instanceId"] = $"{Environment.MachineName}-{Guid.NewGuid()}";
            properties["quartz.jobStore.type"] = typeof(MongoDbJobStore).AssemblyQualifiedName;
            properties["quartz.jobStore.collectionPrefix"] = "quartz_local";
            properties["quartz.jobStore.connectionString"] = MongoDbConfig.ConnectionString;
            properties["quartz.serializer.type"] = "json";

I've done as the code example before. Is this approach bad?

I think you would do it like this:
services.AddQuartz(cfg =>
{
    cfg.SetProperty(StdSchedulerFactory.PropertyJobStoreType, typeof (MongoDbJobStore).AssemblyQualifiedName!);
    cfg.SetProperty(
        $"{StdSchedulerFactory.PropertyJobStorePrefix}.{StdSchedulerFactory.PropertyDataSourceConnectionString}",
        "mongodb://localhost/quartz");
    // optional prefix
    cfg.SetProperty($"{StdSchedulerFactory.PropertyJobStorePrefix}.collectionPrefix", "quartz");
});
services.AddQuartzHostedService();

I'm not sure if trying to integrate it with UsePersistentStore would bring any benefits. I think it was more meant for relational databases and not document databases see https://github.com/quartznet/quartznet/issues/988

Refactor persistence API to be more document database friendly, #814 probably contributes to this


Add functionality to TestSecurityManager.

The current implementation of TestSecurityManager doesn't really do anything. I added functionality to make it useful in testing.

I should also note that most users will be using Spring Boot to build their Spring applications.  In this case, Spring Boot for Apache Geode (SBDG) also offers a TestSecurityManager implementation that is not that dissimilar to the proposed changes in this PR.