Move builds from browserify to webpack

Needed it we are to move to three-to-ammo 1.0.0, as it uses ES6-style imports, which are unsupported by browserify.
(Could in theory patch up with babel, but better long-term to move to webpack).
Note, this also solves:
https://github.com/n5ro/aframe-physics-system/issues/201
(and maybe other problems reported compiling with webpack)
https://github.com/n5ro/aframe-physics-system/issues/151
https://github.com/n5ro/aframe-physics-system/issues/88

This seems to have broken the Cannon worker driver completely - moving back to draft for now...

Error with worker driver is this:

And physics doesn't work (many examples didn't work anyway with worker driver, but the sandbox example did work, and isn't working now).

Problems with this PR are with cannon worker driver only.  Comparing the built files, it looks like a big chunk of code is being omitted in the new builds, but I didn't understand why yet.
I'm not sure how much effort it's worth expending to maintain cannon worker driver.  Lots of the function already doesn't work, and I don't see much need for a 4th physics option on top of cannon, ammo and PhysX.
I think a better way forwards is probably to deprecate cannon worker driver, but quite a bit of work to do so cleanly.

what work is needed to deprecate the cannon worker driver?

what work is needed to deprecate the cannon worker driver?

Docs updates; remove various bits of code & tidy up; remove examples; test to ensure non-deprecated function is not regressed.
Worth noting that I am curious about whether an A-Frame physics system built on PHY might be a better solution, particularly in terms of maintainability.
That would give us 3 drivers (Oimo, Ammo.js and PhysX), with both worker drivers and direct drivers available, and an actively developed foundation to build on.  My intention is to check out the feasibility of that option as a priority, before investing further in either of aframe-physics-system, or aframe-physx.

Need to watch out for this effect, of course :-)
https://xkcd.com/927/

PHY looks very interesting; however, single maintainer, and looks like it was untouched fo 2 years? I hadn't heard of it, but from the README and demo it certainly looks promising. Would of course be good to leverage the work of another dev in the space.
Even if just for "full support of physx" (is that for real???), getting the extra oimo support as well--definitely should be looked into. Very cool find.

Need to watch out for this effect, of course :-) https://xkcd.com/927/

I knew exactly which xkcd this was going to be without clicking, lol.

@kylebakerio You may have overlooked the commits? There were commits the last two weeks. There is support for Physx 5.1. The maintainer shared the news the 3rd february 2023 in this tweet that I shared in the #physics channel on slack.

Not sure if my english was unclear, but my tone was of excited surprise and appreciation, not so much doubt--but some sense of 'need to confirm this, but that is amazing news if so'.

I've spent a bit of time looking at PHY, and although there is some potential there, I don't see a way forwards there that I'm keen to pursue at the moment...
PHY is more of a game engine, than a physics engine, with a 3 layer architecture

Main.js, which is responsible for everything you see here takes care of orchestrating a wide range of different demos, engines etc.
Motor (aka Phy) is the main game engine.  It provides the main game engine API, which the demos use to set things up, and takes care of fan-out to the appropriate physics engine + also the rendering of the objects (using Object3Ds or Instanced Meshes)
There are then wrappers for the various physics engines (PhysX, Ammo, Oimo etc.) that Motor can use to drive the actual physics.  All the physics engines preset the same clean, standardized API to Motor, either via direct function calls, or running as workers.

The API between Motor & the Engines is fairly simple & clean.  It's not documented, but you can see how it is used by searching across the codebase for:

root.post( for messages from the Motor to the Engine
engine.post( for messages from the Engine to the Motor

This API provides support for rigid bodies (including compound shapes), joints + drivers, plus more.  I didn't see how collision events work (only the Rapier engine seemed to have code to generate collision events), but I may_ have overlooked something.
How could we make use of any of this in A-Frame?
It looks to me like Motor wraps together too many concerns to fit into A-Frame's ECS architecture as a physics engine.  I wouldn't want an A-Frame physics engine to take responsibility for setting up Meshes, Materials etc.  I want to handle that separately using standard pre-existing A-Frame components.
I think it would be possible to create a set of A-Frame components that take the place of Motor, sitting on top of the Motor-Engine API, and provide a standardized component-based way to set up any of the PHY Engines.
Potentially this could give us a physics engine for A-Frame that provides a standard set of components, and can easily be switched between any of the physics engines that PHY supports (at this time, that's Oimo, Ammo & Physx).
The PHY Motor-Engine interface is not documented or versioned.  There's no guarantee it changes over time, although it's likely to be quite stable.  The individual physics Engines are built to per-engine JS modules in the build folder of the repo.
I think there's the potential to build a strong, flexible physics system for A-Frame on top of the PHY Engines.  But it also looks like quite a lot of work just to get to parity with where we are with PhysX, Ammo & Cannon today.
I'm also unconvinced that the A-Frame community needs multiple physics engines available.  Excellent, well-documented support for one really good Physics Engine is probably sufficient.
The question remains in my mind, whether that should be based on PhysX or Ammo.  My personal inclination at this time is to lean into PhysX, to try to bring it up to feature parity with Ammo/aframe-physics-engine.  However that's probably not going to happen fast, so in the meantime, tidying up aframe-physics engine to remove the deprecated Cannon Worker option (as per the topic of this issue) is probably a worthwhile thing to do.

Thanks for looking into that.
Is it possible to use PHY as a reference for building out our own interface for PhysX?

We already have PhysX here:
https://github.com/c-frame/physx
PHY may be a useful reference for filling some gaps, e.g. compound shapes.

Sure, filling in the gaps is what I mean; from the description I had the impression that their implementation of a PhysX to JS wrapper was much more complete.


sd_mscerror

Hello, I encountered an error when using the tinyusb example "sd_msc.ino". An error will occur if the file size of the SD card exceeds 512 bytes. Can you teach me how to repair it

Hi,
i dont understand what problem do you have, sorry.
If you describe it more, maybe i can help.

Thank you for your reply.
Preparation for problem recurrence:
1.esp32s3
2. Text file larger than 512 bytes (000.txt)
3.sd_ Msc sample code
Step:

Modify the example to read and write the SD card correctly.
#define SD_ MISO  37
#define SD_ MOSI  39
#define SD_ SCK   38
#define SD_ CS    40
Burn the program to esp32s3
Copy "000. TXT" to esp32disk.
Unplug the esp32 and disconnect it from the computer
Insert esp32 into the computer and connect it to the computer, and open the USB flash drive
Open "000.TXT" in the USB flash drive, and some contents of the file are lost.

Its because you did not unmount device under system properly.
Im not sure, but this small snippet in your .ino may help to fix it:
#include "ff.h"
#include "diskio.h"
void tud_msc_write10_complete_cb(uint8_t lun)
{
    if (disk_ioctl(0, CTRL_SYNC, NULL) != RES_OK)
    {
        Serial.printf("failed to sync\n");
    }
}

Ok, thank you very much for your help


Abort getUpdates when stopping runner

When .stop()ing runner it doesn't abort getUpdates API call. As a result, it hangs until long polling interval passes (e.g. up to 30 seconds) even if there are no updates to process.
This delay is annoying while developing and also when running a bot in Docker. Not sure it's a good idea to shutdown a bot using SIGKILL each time.

The runner is supposed to terminate pending calls:
https://github.com/grammyjs/runner/blob/b1da579c981e1070508d6c7c352ebe7d3a886b98/src/source.ts#L181
is called by
https://github.com/grammyjs/runner/blob/b1da579c981e1070508d6c7c352ebe7d3a886b98/src/runner.ts#L330
but if you observe a different behaviour, then there is a bug somewhere. Could you provide a minmal repro?

Sorry, seems like it's not related to grammY but to bun.

Holy crap, is this another case where it executes JS incorrectly?

Seems like that. As far as I understand, the problem was with workers and/or signals controller.


Error while merging coverage files

Hey.
I have the following error after all my tests finish executing and spoon is generating the report:
error while merging coverage files
Caused by: java.io.FileNotFoundException .../coverage/[somehash]/coverage.ec (No such file or directory)
I'm running a single test class, on a single physical device. I didn't try running everything or a single test method.
I've ran clean first, didn't help.
Version 1.2.0.

Hi @pawelkw
Does it happen even if you disable test coverage?

Nope. Works correctly without the codeCoverage = true flag.

It looks like there some problems obtaining the coverage file from your device. It's not pulled and coverage merger then fails to open it.
@pawelkw Would you describe your device environment (real device or emulator, Android version)?
/cc @jainsahab

Real Device. Samsung Galaxy S6 (SM-G920F). Android 6.0.1. Samsung build number: MMB29K.G920FXXU3DPDP.

Same problem on Nexus 5 (not 5X).

Also on Huawei P8 Lite.

My config:
spoon {
    debug = true
    // Enable setting test class/method-to-be-run from command line. E.g.:
    // $> ../gradlew spoonFreeDebugTest -PspoonClassName=com.stanfy.spoon.example.test.MainActivityTest -PspoonMethodName=testSetText
    if (project.hasProperty('spoonClassName')) {
        className = project.spoonClassName

        if (project.hasProperty('spoonMethodName')) {
            methodName = project.spoonMethodName
        }
    }

    adbTimeout = 60 * 3
    codeCoverage = true
}

I'm running:
./gradlew clean spoon[MyVariant] -PspoonClassName=my.test.Class

Also on emulator.

This is the issue with Spoon itself on which this plugin depends on. I have raised the request with that repo #346. Hopefully fixing that should also fix this issue

I've commented on that https://github.com/square/spoon/issues/346. I will pick that up soon.

Awesome. I've seen that threre's a PR already. I'll re-test and let you know if it fixed the issue for me once  it's merged.

I've put a fix for this issue.


Tags to default spans

Purpose
Addition of tags to the default spans that are started automatically when the resources are serving to a request.
Goals
Ability to include user defined tags to the default spans by passing -1 as the span id argument to the addTagtoSpan function as mentioned below.

_ = observe:addTagToSpan(-1, "tot_requests",   req_count);

Approach
Checks if  a default span is created when a resource starts serving and if it is present, the observercontext is returned and the tag is added to it.

@Shairam Can you reword the commit messages according to the standard we follow https://chris.beams.io/posts/git-commit/.


Fix module and program execution order

Purpose

This PR changes the program execution in-line with the spec. With this change, a program can have both a main() and listeners. The execution order is

Module init functions
main() function if there is one. If the main() returns abnormally, the program will terminate.
If the main() finishes execution successfully, the listen phase executes. If there are any listeners registered, they get started.

Fixes #14375

Approach

Refactored the APIs for program execution by unifying the previously separate APIs provided for main program execution and services execution.

Remarks

This PR disables the OAuthConfigTest since it is written in a manner which makes it impossible to run with the new execution order. It should be refactored to comply with the changes introduced by this PR.

Check List

[x] Read the Contributing Guide
[ ] Required Balo version update
[ ] Updated Change Log
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

Codecov Report

Merging #15030 into master will increase coverage by <.01%.
The diff coverage is 52.99%.

@@             Coverage Diff              @@
##             master   #15030      +/-   ##
============================================
+ Coverage     63.07%   63.08%   +<.01%
+ Complexity      560      557       -3
============================================
  Files          2336     2336
  Lines        103753   103719      -34
  Branches      13830    13828       -2
============================================
- Hits          65447    65426      -21
+ Misses        33228    33227       -1
+ Partials       5078     5066      -12

Impacted Files
Coverage Δ
Complexity Δ

...llerinalang/net/http/serviceendpoint/Register.java
81.25% <ø> (ø)
0 <0> (ø)
:arrow_down:

...in/java/org/ballerinalang/bre/bvm/BVMExecutor.java
89.21% <ø> (+0.72%)
0 <0> (ø)
:arrow_down:

...java/org/ballerinalang/launcher/LauncherUtils.java
7.73% <0%> (+0.48%)
0 <0> (ø)
:arrow_down:

...ballerinalang/test/agent/BallerinaServerAgent.java
0% <0%> (ø)
0 <0> (ø)
:arrow_down:

...ang/compiler/tree/expressions/BLangInvocation.java
70% <0%> (-1.25%)
0 <0> (ø)

...g/compiler/tree/expressions/BLangSimpleVarRef.java
89.79% <0%> (-2.05%)
0 <0> (ø)

...java/org/ballerinalang/test/util/BServiceUtil.java
57.14% <100%> (-7.57%)
4 <0> (ø)

.../org/ballerinalang/launcher/util/BServiceUtil.java
64.28% <100%> (-6.31%)
0 <0> (ø)

...java/org/ballerinalang/launcher/util/BRunUtil.java
48.38% <40%> (-6.05%)
0 <0> (ø)

...java/org/ballerinalang/test/util/BCompileUtil.java
59.81% <50%> (+1.67%)
32 <0> (ø)
:arrow_down:

... and 20 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3e1adc8...2cf65bc. Read the comment docs.


Issue #1579 TLSRoute Passthrough Conformance Test (normative)

Issue #1579 TLSRoute Passthrough Conformance Test (normative)
What type of PR is this?
/area conformance
What this PR does / why we need it:
This is the first test specified in the TLSRoute Passthrough Conformance Test suite, the normative test.  This test may be used as reference to learn more about designing conformance tests, particularly for TLSRoute Passthrough.
Which issue(s) this PR fixes:
This addresses part of Issue #1579
Does this PR introduce a user-facing change?:
NONE

/ok-to-test

Did you test this on any implementations?

Yes, I tested on Istio, but had to tweak it to make it get past some of the condition naming updates in Gateway API.

JFYI, I was able to run this test and have it pass against Contour's implementation which has support for TLS passthrough. I haven't reviewed the code yet but will try to do so soon. Thanks!

Thanks @candita! This change LGTM, but would like someone else to sign off on this as well.
/cc @skriss @arkodg @shaneutt
Do any of you have time to run through this as well?
/approve

@robscott @arkodg thanks for the review.  I made some more changes and responded to your comments on changes I didn't make.  Could you PTAL when you get a chance ?

@skriss @shaneutt this has been /approved by @robscott and LGTM by @arkodg.  It passes for Istio and Contour.  Could one of you please give it the /lgtm and allow to merge?

/label tide/merge-method-squash


ccruntime: Set INSTALL_OFFICIAL_CONTAINERD to false

The majority of the managed kubernetes solutions are already relying on
a new enough (v1.7+) version of containerd.
/cc @wainersm, we'd need to adapt CI for this, and I'd need your help to do so.

I hit an error testing this out locally:
# kubectl logs pod/cc-operator-pre-install-daemon-8xwt9 -n confidential-containers-system
/opt/confidential-containers-pre-install-artifacts/scripts/reqs-deploy.sh: line 208: INSTALL_COCO_CONTAINERD: unbound variable

Are you sure you'be been testing the latest version of this PR?  Asking as looking at the reqs script I don't easily see it happening, as we set those at the beginning of the file.

I hit an error testing this out locally:
# kubectl logs pod/cc-operator-pre-install-daemon-8xwt9 -n confidential-containers-system
/opt/confidential-containers-pre-install-artifacts/scripts/reqs-deploy.sh: line 208: INSTALL_COCO_CONTAINERD: unbound variable

Are you sure you'be been testing the latest version of this PR? Asking as looking at the reqs script I don't easily see it happening, as we set those at the beginning of the file.

I'm not sure, but given all the changes you've made since then I've re-tested in a 'clean' environment

@wainersm, I need your help here for changing the tests.

@wainersm, I need your help here for changing the tests.

ok, looking at it.

@fidencio I'm thinking in tackle CI like this:

Current jobs that are run on Ubuntu 20.04 (or CentOS Stream) got containerd 1.6 (containerd.io package from docker). Those I am testing a change so that the operator's pre-reqs script will install the official containerd 1.7.
Add jobs for kata-qemu and kata-clh on Ubuntu 22.04 which will have containerd 1.7.2, so to test the default installation behavior. I am still trying to test this will work out with current tests/e2e scripts but I've faced networking issues when building the pre-reqs image....

sound as a good plan?

@fidencio I'm thinking in tackle CI like this:
1. Current jobs that are run on Ubuntu 20.04 (or CentOS Stream) got containerd 1.6 (`containerd.io` package from docker). Those I am [testing a change](https://github.com/confidential-containers/operator/pull/276) so that the operator's pre-reqs script will install the official containerd 1.7.

CI pass with that change: http://jenkins.katacontainers.io/job/confidential-containers-operator-main-ubuntu-20.04-x86_64-containerd_kata-qemu-PR/328/console

2. Add jobs for kata-qemu and kata-clh on Ubuntu 22.04 which will have containerd 1.7.2, so to test the default installation behavior. I am still trying to test this will work out with current tests/e2e scripts but I've faced networking issues when building the pre-reqs image....

Here we have a problem... the CI scripts leverage docker to build images and service with a local registry, but docker depends on its containerd.io  bundle which install containerd 1.6. So although Ubuntu 22.04 provides the packaged 1.7.2 containerd, docker seems to not like it. I'm looking for a solution...

sound as a good plan?

@fidencio I'm thinking in tackle CI like this:
1. Current jobs that are run on Ubuntu 20.04 (or CentOS Stream) got containerd 1.6 (`containerd.io` package from docker). Those I am [testing a change](https://github.com/confidential-containers/operator/pull/276) so that the operator's pre-reqs script will install the official containerd 1.7.

CI pass with that change: http://jenkins.katacontainers.io/job/confidential-containers-operator-main-ubuntu-20.04-x86_64-containerd_kata-qemu-PR/328/console

2. Add jobs for kata-qemu and kata-clh on Ubuntu 22.04 which will have containerd 1.7.2, so to test the default installation behavior. I am still trying to test this will work out with current tests/e2e scripts but I've faced networking issues when building the pre-reqs image....

Here we have a problem... the CI scripts leverage docker to build images and service with a local registry, but docker depends on its containerd.io bundle which install containerd 1.6. So although Ubuntu 22.04 provides the packaged 1.7.2 containerd, docker seems to not like it. I'm looking for a solution...

sound as a good plan?

Managed to fix the execution on Ubuntu 22.04. @fidencio you will need to pick my two commits from https://github.com/confidential-containers/operator/pull/276
Here is the final coverage:
a) Jobs running on Ubuntu 20.04, containerd 1.6, INSTALL_OFFICIAL_CONTAINERD=true
http://jenkins.katacontainers.io/job/confidential-containers-operator-main-ubuntu-20.04-x86_64-containerd_kata-clh-PR
http://jenkins.katacontainers.io/job/confidential-containers-operator-main-ubuntu-20.04-x86_64-containerd_kata-qemu-PR
If I'm not mistaken SEV/SNP is Ubuntu 20.04 and TDX is CentOS Stream 8 then they will fall on this case.
b) Jobs running on Ubuntu 22.04, containerd 1.7, INSTALL_OFFICIAL_CONTAINERD=false (default behavior)
http://jenkins.katacontainers.io/job/confidential-containers-operator-main-ubuntu-22.04-x86_64-containerd_kata-clh-PR
http://jenkins.katacontainers.io/job/confidential-containers-operator-main-ubuntu-22.04-x86_64-containerd_kata-qemu-PR

/test

/test

TDX CI is failing due to:
17:16:13 not ok 4 [cc][agent][kubernetes][containerd] Test can pull an unencrypted image inside the guest

SEV CI is failing due to:
17:06:06 not ok 2 [cc][kubernetes][containerd][sev] Test SEV encrypted container launch failure with INVALID measurement

/test

/test

/test-s390x


Folder mismatch on Repair (is verify EXACT_PATH based on a sanitized API value instead of the actual install location?)

Prerequisites

[X] I've looked through existing issues to make sure that this bug hasn't been reported before.
[X] I've read the GitHub Community Guidelines and know netiquette.

Playnite version
<IP_ADDRESS>68
Plugin version
<IP_ADDRESS>976
Nile Launcher version
1.1.2
Bug description
I noticed this issue with a game that had been installed prior with Amazon Games. Endling: Extinction is Forever.
It was installed to Endling - Extinction is Forever, I did a Repair on it to test the functionality of both Playnite and Nile. It was taking forever- I found out that it was actually pointing the verify operation to a different folder, Endling Extinction is Forever, and since the game files didn't exist there, it was redownloading them all as per verify's rules. I migrated the games over with the plugin's functionality, and Nile's installed.json correctly pointed to Endling - Extinction is Forever... so I have no clue how it determined the folder to be different than the current install location. I imagine some logic assigning a game folder name for newly installed games is getting used incorrectly in this operation?
Relevant log
15-11 11:06:53.229|DEBUG|NileLibrary#CliWrapCommandExtensions:Executing command: C:\Users\Adam\scoop\apps\amazon-nile\current\nile.exe verify amzn1.adg.product.21965245-d668-4a8f-818a-38133698587b --path "D:\AmazonLibrary\Endling Extinction is Forever" --max-workers 4
Screenshots
No response

Yeah, I see the problem. Will be fixed in next update.


workflow: only verify commit message on main branch

For other branches, the commit message is set by the PR title before merging.

I think it's still nice to enforce consistency since some commits on minor don't go through PRs.


🚀 Feature Request: Support for Astro

🎤 Tell us your idea
Is there any plan to support for Astro framework?

Hi @hckhanh, Astro supports React, and it should work fine integrating CopilotKit.
You may need a little modification to the endpoint.

Hi @NathanTarbert
How can I modify the endpoint correctly?

@hckhanh, you can try something like this.
// src/pages/api/copilotkit.ts
import type { APIContext } from 'astro';

import {
  CopilotRuntime,
  OpenAIAdapter,
  copilotRuntimeNextJSAppRouterEndpoint,
} from "@copilotkit/runtime";
import OpenAI from "openai";

const openai = new OpenAI();
const serviceAdapter = new OpenAIAdapter({ openai });

const runtime = new CopilotRuntime({
  remoteActions: [
    {
      url: process.env.REMOTE_ACTION_URL || "http://localhost:8000/copilotkit",
    },
  ],
});

export const POST = async ({ request }: APIContext) => {
  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter,
    endpoint: "/api/copilotkit",
  });

  return handleRequest(request);
};


[build] Add optional build-id

Build-ids are required for doing certain kinds of performance tracing.
For example build-ids let Linux perf find the correct objects and
transfer them to other machines via "perf archive".
This adds an option to add -Wl,--build-id to the linker. It can be
enabled by passing -Dbuild_id=true to meson or --build-id to
package-release.sh.
Signed-off-by: Nicholas Fraser<EMAIL_ADDRESS>

Would there be any sort of downside to always enabling it? Could get rid of the config option if not, but if this blows up binary size or creates extra files or whatever, we should keep it.

I don't think so, apparently this just adds something to the ELF header?
How does this work given we do PE builds? Does it replace the PDB dir or something?

It does the same thing for PE builds, just adds a 53 byte .buildid section. The reason I didn't turn it on by default is because it's a random GUID so it could interfere with reproducible builds. (This isn't the case for ELF build-ids because they are (by default) an SHA-1 hash of the .text section so they are reproducible.) I'm not sure if you or Valve are doing reproducible builds or planning on it and I figure most people doing performance tracing are going to be building from master anyway so they can turn this on themselves.

@doitsujin

Would there be any sort of downside to always enabling it?

It potentially can brake some debug tools which rely on CRC. As opposite it will unbrake those tools which rely on build-id only (AutoFDO ?). See also: https://sourceware.org/gdb/current/onlinedocs/gdb/Separate-Debug-Files.html

Fair enough, gonna keep it as an option then.


Should the vacuum command add an entry to the transaction log?

The vacuum() command doesn’t add a JSON file to the _delta_log/ directory.  I’m getting this error message when trying to display a version that’s no longer accessible after a vacuum (I’m doing this intentionally to see the error message):
[info]   org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1645.0 failed 1 times, most recent failure: Lost task 0.0 in stage 1645.0 (TID 21123, localhost, executor driver): java.io.FileNotFoundException: File file:/Users/powers/Documents/code/my_apps/yello-taxi/tmp/vacuum_example/part-00000-618d3c69-3d1b-402c-818f-9d3995b5639f-c000.snappy.parquet does not exist
[info] It is possible the underlying files have been updated. You can explicitly invalidate the cache in Spark by running 'REFRESH TABLE tableName' command in SQL or by recreating the Dataset/DataFrame involved.
[info]   Cause: java.io.FileNotFoundException: File file:/Users/powers/Documents/code/my_apps/yello-taxi/tmp/vacuum_example/part-00000-618d3c69-3d1b-402c-818f-9d3995b5639f-c000.snappy.parquet does not exist
[info] It is possible the underlying files have been updated. You can explicitly invalidate the cache in Spark by running 'REFRESH TABLE tableName' command in SQL or by recreating the Dataset/DataFrame involved.

Maybe we could have the vacuum command write out a JSON file to the transaction log and give a descriptive error message when the user tries to access a version that’s no longer accessible?  Something like “Version 0 of the Delta lake is no longer accessible because XYZ file was deleted from the filesystem with the vacuum command”.

@mukulmurthy - In the Delta Slack channel you mentioned "VACUUM garbage collects old files; it doesn’t actually change the state of the table, so it’s a bit odd to have it write to the transaction log".  It kind of depends on what we consider the state of the table.
If the state of the table is the current version of the table, then yes, vacuum doesn't change anything.  But if the state of the table is all versions of the table, then vacuum arguably does change the table state.
I don't think it'd really hurt to have vacuum write out a log file...

What if there were a separate vacuum log to track this?
As long as the user isn't vacuuming with tremendous frequency, then this log shouldn't grow too large either. If that is a concern perhaps you can do a "hard vacuum" if you want to clear the vacuum logs as well, but default to a "soft vacuum". @mukulmurthy

The other option is to (when throwing that error) scan the transaction log for a RemoveFile event so we can give a slightly better error message like "Cannot create snapshot at version 15 because a data file ___ was deleted in version 5. If VACUUM was run after that version without a long enough retention period, that data file has been deleted from disk".

@mukulmurthy - I like the readability of that proposed error message 👍

@nvander1 - I like the idea of different types of vacuums... maybe not so much for how we deal with records in log files, but more for what happens with the files.  A hard vacuum could mean deleting data files and a soft vacuum could be moving data files from S3 to Glacier.
Delta Lake should stay file system independent so maybe the softVacuum command could take a callback function as an argument and give the users a list of all the files that need to be vacuumed and let them apply custom logic on what to do with the files.
In any case, I'm getting off track here.  TL;DR feel like it's pretty much "free" to persist info on what's be vacuumed and don't think we should "throw away" any transaction data when it could easily be persisted.

Great discussion here!
A better error message when a file is missing.
This makes a lot of sense. The current message is pretty non-sensical as it suggests that you REFRESH TABLE, which cannot actually help. I fully support improving this.
A complication is that this error message is being thrown by Spark. In DBR we just fork to improve this message, but to improve things for users of Apache Spark we are going to have to open up some new APIs.
Logging when vacuums occur
I'm not against this in principle, but its not clear to me what additional information would come from this log. If a file is missing, it was vacuumed (or someone deleted it manually). You do not need a log to tell you this.
What would you log here? When vacuum is run? Which files are in each vacuum? Note that to list the version where a file was originally logically deleted, as Mukul suggest, we do not need any additional logging.
It is also going to be pretty difficult to say anything authoritative in this log.  Deleting many files is not atomic. So whether you log before or after the vacuum there is a chance of the log differing from reality. You could have a WAL with retries... but now you are getting pretty complicated.
Using Glacier
We actually have several customers that do this today.  You can put parts of a delta table into glacier (usually at date partition boundaries). Since partition pruning prevents these files from being read for queries on newer data, everything works fine. If you want to query these older dates then you need to move them out of glacier.


Fix broken link in README.md

Signed-off-by: knn<EMAIL_ADDRESS>What this PR does / why we need it :
This PR fixes the broken link Get Started in README.md.
Which issue this PR fixes :
Issue #233

Thorough PR, thank you!

@saraclay  You are welcome!


Can't boot ramdisk

3 and 4 steps from readme complete without errors, but on the device it stops with something like this:
Bootstrap path: path = /AppleInternal/Library/LaunchDaemons, error = 2: No such file or directory.
Hence iProxy also doesn't work: in SSH terminal it says, "connection reset by peer", and in iProxy terminal -- "No connected device found. Terminating"
I'm using LiveUSB with Ubuntu-22.04, installed libimobiledevice-utils, libirecovery-common, libirecovery-1.0-3, irecovery and libusbmuxd-tools as dependences, trying to boot iPhone SE on iOS 15.4.1, as ipsw for step 3 using iOS 15.6.1 (also tried with iOS 15.4.1 link). Before step 3 iPhone successfully paired with PC using "idevicepair pair".

The phone is fine, its a usbmuxd issue, stop the usbmuxd daemon and compile usbmuxd2 and run that


Issue sh: -c: line 1: syntax error: unexpected end of file

We made a plugin that uses your library to download  videos from youtube vut suddely got this error.... and this looks like some bug in code. Can you assist please and direct us to the right way ? we really need it to be fixed.
It looks like some quotes are missin for $process value...
2016/06/16 00:14:09 [error] 3963#0: *182310768 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Uncaught exception 'Youtubedl\Exceptions\YoutubedlException' with message 'sh: -c: line 0: unexpected EOF while looking for matching ``'
sh: -c: line 1: syntax error: unexpected end of file
' in /var/www/site.com/http/wp-content/plugins/yt-downloader/vendor/labzone/youtube-downloader/src/Youtubedl/Youtubedl.php:73
Stack trace:
#0 /var/www/site.comhttp/wp-content/plugins/yt-downloader/yt-downloader.php(204): Youtubedl\Youtubedl->execute()
#1 [internal function]: download_youtube_video('')
#2 /var/www/site.com/http/wp-includes/plugin.php(525): call_user_func_array('download_youtub...', Array)
#3 /var/www/site.com/http/wp-admin/admin-ajax.php(89): do_action('wp_ajax_downloa...')
#4 {main}
thrown in /var/www/site.com/http/wp-content/plugins/yt-downloader/vendor/labzone/youtube-downloader/src/Youtubedl/Youtubedl.php on line 73"

solved


Invalid Biome ID 71 and 64

What is the output link of /viaversion dump? https://dump.viaversion.com/f0e794cb85597775940fdf876cd5b3405d958987c71f4988d899a77b45c1b296
Are you using any additional software like BungeeCord? If so, what software and version? (Please list your plugins as well)
No just Spigot
Plugins (31): AsyncWorldEditInjector, LuckPerms, EmptyWorldGenerator, Skript, MinePacks, RelaAutoBroadcast, Votifier, Multiverse-Core, OpenTerrainGenerator, AdvancedHelp, VotingPlugin, AsyncWorldEdit, ViaVersion, Multiverse-Portals, Vault, ViaBackwards, ViaRewind, WorldEdit, Essentials, ProtocolLib, WorldGuard, DiscordSRV, EssentialsChat, CrazyAuctions, SuperVanish, PyrCore, GriefPrevention, EnjinMinecraftPlugin, HolographicDisplays, PyrParticles, BossShop
How does this error happen? login? Using an item?
In 1.13 only (I think) while I am in the world Biome Bundle, ViaVersion spams the message Received invalid Biome id 65 or Received invalid Biome id 71
Is there an error in the console? Use pastebin.com. Is there a kick message?
https://pastebin.com/xyVCyKiv
I just was wondering if there is a way for me to stop the spam as of this moment because it can be very annoying. Thanks!

Follow #963 for progress


Mobile sidebar doesn't hide on clicking outside of it or on clicking on an item

Mobile sidebar should hide when clicking on one of the links or clicking outside of it. Currently it stay open and you need to hide it by clicking on the hamburger icon again.

On hold until we solve #1201

Solved with #1201


When used with vue-at, the edit event could not be captured

I try to use vue-medium-editor with vue-at. When the text is updated, no edit event was emitted, and so I couldn't update the content.

Did you miss
If you are reporting a bug, please provide a minimal working example.

I can't tell if it's a bug or an enhancement request.

<template>
<div class="text">
    <at :members="friendlist" name-key="nickname" @at="handleAt">
        <medium-editor
            :text='content'
            v-on:edit='applyTextEdit' />
    </at>
</div>
</template>
<script>
import At from 'vue-at'
import editor from 'vue2-medium-editor'
export default {
    data () {
        return {
            content: '',
        };
    },
    methods: {
        applyTextEdit (ev) {
          this.text = ev.event.target.innerHTML
        },
    },
    components: {
        At,
        'medium-editor': editor
    }
}
</script>

I'm not even sure if this can be done.

Actually I made a small hack to use vue-at, as vue-at emits an at event and thus I could handle the case.
But in my experience, changes that made to DOM may not be captured by the editableInput, therefore couldn't be handled directly.

Would you mind sharing this hack?


Minor fix request for the workshop

I have found a couple of minor bugs.
[1] File name fix required
Guide link: https://catalog.us-east-1.prod.workshops.aws/v2/workshops/56ef6f79-74e2-4710-aefb-10b9807057a9/en-US/data-types/basics/declaring
Step 6. the file link should be fixed.
services/shared/apps/tenant-registration/src/registration/registration-service.ts
-> services/shared/apps/tenant-registration/src/registration/registration.service.ts
[2]
Guide link: https://catalog.us-east-1.prod.workshops.aws/v2/workshops/e04c0885-830a-479b-844b-4c7af79697f8/en-US/lab-4/6-apply-network-policy#step-10-install-tigera-calico
Step 10.
When I installed calico, the command did not work, having 404 error.
kubectl apply -f https://raw.githubusercontent.com/aws/amazon-vpc-cni-k8s/master/config/v1.6/calico.yaml
I tried below two commands instead and worked well.
kubectl apply -f https://raw.githubusercontent.com/aws/amazon-vpc-cni-k8s/master/config/master/calico-operator.yaml
kubectl apply -f https://raw.githubusercontent.com/aws/amazon-vpc-cni-k8s/master/config/master/calico-crs.yaml

I refered to the official doc: https://docs.aws.amazon.com/eks/latest/userguide/calico.html
Thank you for great contents. Cheers!

Thanks for reporting @jimini55 . We will check and get this addressed.

@jimini55 Thank you. This worked for me.

We removed Calico entirely from the workshop in lieu of Istio. Closing this comment as not applicable anymore.


Website Url Field In Profile Settings EditProfile.tx

Website Links Show
const [website, setWebsite] = useState(profile.website || '');
JSX
<input
type="text"
placeholder="Website"
value={website}
onChange={(e) => setWebsite(e.target.value)}
/>
const saveProfile = () => {
// Your existing code to update other profile fields
// Add code to update the website in the profile
// Assuming you have an updateProfile function that sends data to the server
updateProfile({ description, website });
};
Describe the solution you'd like

Describe alternatives you've considered

Additional context

Closing as a duplicate


⚠ Turnos 24x7 has degraded performance

In f91e1f7, Turnos 24x7 (https://reservas-dev.alternativasinteligentes.com/) experienced degraded performance:

HTTP code: 200
Response time: 971 ms

Resolved: Turnos 24x7 performance has improved in a2c6810.


Reorganize user details page

Under John Allison (name)
Name and affiliation should be at top
Admin Info - is unclear - is this available to all users (scroll isn't working there)
Some description of API key and Globus User Name would be helpful

Re-arranged as specified above and added explanations to apikey and globus sections.
The admin tab is only available to admin users and is meant to help with managing the system.


50.07 Settings-Audio：Cannot change

Discussed in https://github.com/dfint/df-steam-hook/discussions/11

Originally posted by dyjpxc February 27, 2023
Master Volume：
50.04-50.05no problem

@dyjpxc, please check if the problem is still relevant. If so, please describe the problem in more detail.

Close as stale


Generated create_import_object contains many spaces

Although the code is syntactically correct, it looks broken due to the spacebars:
fn create_import_object(store: &Store, env: &RuntimeInstanceData) -> ImportObject {
    imports! {
       "fp" => {
           "__fp_host_resolve_async_value" => Function :: new_native_with_env (store , env . clone () , resolve_async_value) ,
           "__fp_gen_count_words" => Function :: new_native_with_env (store , env . clone () , _count_words) ,
           "__fp_gen_log" => Function :: new_native_with_env (store , env . clone () , _log) ,
           "__fp_gen_make_request" => Function :: new_native_with_env (store , env . clone () , _make_request) ,
           "__fp_gen_my_async_imported_function" => Function :: new_native_with_env (store , env . clone () , _my_async_imported_function) ,
           "__fp_gen_my_complex_imported_function" => Function :: new_native_with_env (store , env . clone () , _my_complex_imported_function) ,
           "__fp_gen_my_plain_imported_function" => Function :: new_native_with_env (store , env . clone () , _my_plain_imported_function) ,
        }
    }
}

Thanks you for the issue report @joepio!
We currently run the generated code through rustfmt however since this code is inside a macro it doesn't get formatted. The spaces themselves are an unfortunate consequence of  how quote!() works as far as I know.
When time permits I'll have a look at fixing this by switching to https://crates.io/crates/prettyplease for the formatting


[Substream Proposal] List of Tokens

Goal
Retrieve a list of tokens that are available on the chain.
Models
Briefly describe the models that are used as input / output for this Substream
Maps
Briefly list the maps including their input / output for this Substream
Stores
Briefly list the stores used in this Substream
Consumers
Briefly explain how to consume this substream and whether this is a custom consumer or a default sink

Implemented via https://github.com/EOS-Nation/substreams-antelope/tree/develop/substreams/eosio.token
StreamingFast team needs to implement kV Sink to make it useful


Does pandas-profiling work in Jupyter Notebooks on AWS?

Does pandas-profiling work in Jupyter Notebooks on AWS? I understand there are a lot of configuration differences that can lead to issues but whenever I try to produce a profiling report, I get the following errors when I run:
profile = ProfileReport(df, 'myreport')
profile.to_file('s3://myfolder')

Summarize dataset:  97%|█████████▋| 427/438 [01:14<00:01,  8.03it/s, Calculate auto correlation]                    /home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/multimethod/__init__.py:315: FutureWarning: In a future version, `df.iloc[:, i] = newvals` will attempt to set the values inplace instead of always setting a new array. To retain the old behavior, use either `df[df.columns[i]] = newvals` or, if columns are non-unique, `df.isetitem(i, newvals)`
  return func(*args, **kwargs)
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:112: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
  warnings.warn("The input array could not be properly "
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:4881: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined.
  warnings.warn(stats.ConstantInputWarning(warn_msg))
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/pandas_profiling/model/correlations.py:67: UserWarning: There was an attempt to calculate the auto correlation, but this failed.
To hide this warning, disable the calculation
(using `df.profile_report(correlations={"auto": {"calculate": False}})`
If this is problematic for your use case, please report this as an issue:
https://github.com/ydataai/pandas-profiling/issues
(include the error message: 'No data; `observed` has size 0.')
  warnings.warn(
Summarize dataset:  98%|█████████▊| 428/438 [28:20<32:48, 196.80s/it, Calculate spearman correlation]/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/multimethod/__init__.py:315: FutureWarning: The default value of numeric_only in DataFrame.corr is deprecated. In a future version, it will default to False. Select only valid columns or specify the value of numeric_only to silence this warning.
  return func(*args, **kwargs)
Summarize dataset:  98%|█████████▊| 430/438 [30:55<21:07, 158.47s/it, Calculate kendall correlation] /home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:5218: RuntimeWarning: overflow encountered in long_scalars
  (2 * xtie * ytie) / m + x0 * y0 / (9 * m * (size - 2)))
/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/scipy/stats/_stats_py.py:5219: RuntimeWarning: invalid value encountered in sqrt
  z = con_minus_dis / np.sqrt(var)
Summarize dataset:  99%|█████████▊| 432/438 [45:40<00:38,  6.34s/it, Calculate phi_k correlation]
---------------------------------------------------------------------------
_RemoteTraceback                          Traceback (most recent call last)
_RemoteTraceback:
"""
Traceback (most recent call last):
  File "/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/joblib/externals/loky/backend/queues.py", line 125, in _feed
    obj_ = dumps(obj, reducers=reducers)
  File "/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/joblib/externals/loky/backend/reduction.py", line 211, in dumps
    dump(obj, buf, reducers=reducers, protocol=protocol)
  File "/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/joblib/externals/loky/backend/reduction.py", line 204, in dump
    _LokyPickler(file, reducers=reducers, protocol=protocol).dump(obj)
  File "/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/joblib/externals/cloudpickle/cloudpickle_fast.py", line 632, in dump
    return Pickler.dump(self, obj)
  File "/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/joblib/_memmapping_reducer.py", line 446, in __call__
    for dumped_filename in dump(a, filename):
  File "/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/joblib/numpy_pickle.py", line 553, in dump
    NumpyPickler(f, protocol=protocol).dump(value)
  File "/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/pickle.py", line 487, in dump
    self.save(obj)
  File "/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/joblib/numpy_pickle.py", line 352, in save
    wrapper.write_array(obj, self)
  File "/home/ec2-user/SageMaker/.envs/mykernel/lib/python3.9/site-packages/joblib/numpy_pickle.py", line 134, in write_array
    pickler.file_handle.write(chunk.tobytes('C'))
OSError: [Errno 28] No space left on device
"""

The above exception was the direct cause of the following exception:

PicklingError                             Traceback (most recent call last)
<ipython-input-9-34649000e9e9> in <module>
      1 profile = ProfileReport(df_perf_18, title="MyReport")
----> 2 profile.to_file(f"s3://sf-puas-prod-use1-pc/fire/research/home_telematics/adt/analysis/MyReport.html")

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/typeguard/__init__.py in wrapper(*args, **kwargs)
   1031         memo = _CallMemo(python_func, _localns, args=args, kwargs=kwargs)
   1032         check_argument_types(memo)
-> 1033         retval = func(*args, **kwargs)
   1034         try:
   1035             check_return_type(retval, memo)

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/pandas_profiling/profile_report.py in to_file(self, output_file, silent)
    307                 create_html_assets(self.config, output_file)
    308
--> 309             data = self.to_html()
    310
    311             if output_file.suffix != ".html":

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/typeguard/__init__.py in wrapper(*args, **kwargs)
   1031         memo = _CallMemo(python_func, _localns, args=args, kwargs=kwargs)
   1032         check_argument_types(memo)
-> 1033         retval = func(*args, **kwargs)
   1034         try:
   1035             check_return_type(retval, memo)

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/pandas_profiling/profile_report.py in to_html(self)
    418
    419         """
--> 420         return self.html
    421
    422     def to_json(self) -> str:

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/typeguard/__init__.py in wrapper(*args, **kwargs)
   1031         memo = _CallMemo(python_func, _localns, args=args, kwargs=kwargs)
   1032         check_argument_types(memo)
-> 1033         retval = func(*args, **kwargs)
   1034         try:
   1035             check_return_type(retval, memo)

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/pandas_profiling/profile_report.py in html(self)
    229     def html(self) -> str:
    230         if self._html is None:
--> 231             self._html = self._render_html()
    232         return self._html
    233

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/typeguard/__init__.py in wrapper(*args, **kwargs)
   1031         memo = _CallMemo(python_func, _localns, args=args, kwargs=kwargs)
   1032         check_argument_types(memo)
-> 1033         retval = func(*args, **kwargs)
   1034         try:
   1035             check_return_type(retval, memo)

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/pandas_profiling/profile_report.py in _render_html(self)
    337         from pandas_profiling.report.presentation.flavours import HTMLReport
    338
--> 339         report = self.report
    340
    341         with tqdm(

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/typeguard/__init__.py in wrapper(*args, **kwargs)
   1031         memo = _CallMemo(python_func, _localns, args=args, kwargs=kwargs)
   1032         check_argument_types(memo)
-> 1033         retval = func(*args, **kwargs)
   1034         try:
   1035             check_return_type(retval, memo)

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/pandas_profiling/profile_report.py in report(self)
    223     def report(self) -> Root:
    224         if self._report is None:
--> 225             self._report = get_report_structure(self.config, self.description_set)
    226         return self._report
    227

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/typeguard/__init__.py in wrapper(*args, **kwargs)
   1031         memo = _CallMemo(python_func, _localns, args=args, kwargs=kwargs)
   1032         check_argument_types(memo)
-> 1033         retval = func(*args, **kwargs)
   1034         try:
   1035             check_return_type(retval, memo)

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/pandas_profiling/profile_report.py in description_set(self)
    205     def description_set(self) -> Dict[str, Any]:
    206         if self._description_set is None:
--> 207             self._description_set = describe_df(
    208                 self.config,
    209                 self.df,

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/pandas_profiling/model/describe.py in describe(config, df, summarizer, typeset, sample)
     93         pbar.total += len(correlation_names)
     94
---> 95         correlations = {
     96             correlation_name: progress(
     97                 calculate_correlation, pbar, f"Calculate {correlation_name} correlation"

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/pandas_profiling/model/describe.py in <dictcomp>(.0)
     94
     95         correlations = {
---> 96             correlation_name: progress(
     97                 calculate_correlation, pbar, f"Calculate {correlation_name} correlation"
     98             )(config, df, correlation_name, series_description)

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/pandas_profiling/utils/progress_bar.py in inner(*args, **kwargs)
      9     def inner(*args, **kwargs) -> Any:
     10         bar.set_postfix_str(message)
---> 11         ret = fn(*args, **kwargs)
     12         bar.update()
     13         return ret

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/pandas_profiling/model/correlations.py in calculate_correlation(config, df, correlation_name, summary)
    105     correlation = None
    106     try:
--> 107         correlation = correlation_measures[correlation_name].compute(
    108             config, df, summary
    109         )

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/multimethod/__init__.py in __call__(self, *args, **kwargs)
    313         func = self[tuple(func(arg) for func, arg in zip(self.type_checkers, args))]
    314         try:
--> 315             return func(*args, **kwargs)
    316         except TypeError as ex:
    317             raise DispatchError(f"Function {func.__code__}") from ex

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/pandas_profiling/model/pandas/correlations_pandas.py in pandas_phik_compute(config, df, summary)
    152         from phik import phik_matrix
    153
--> 154         correlation = phik_matrix(df[selected_cols], interval_cols=list(intcols))
    155
    156     return correlation

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/phik/phik.py in phik_matrix(df, interval_cols, bins, quantile, noise_correction, dropna, drop_underflow, drop_overflow, verbose, njobs)
    254         verbose=verbose,
    255     )
--> 256     return phik_from_rebinned_df(
    257         data_binned,
    258         noise_correction,

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/phik/phik.py in phik_from_rebinned_df(data_binned, noise_correction, dropna, drop_underflow, drop_overflow, njobs)
    164         ]
    165     else:
--> 166         phik_list = Parallel(n_jobs=njobs)(
    167             delayed(_calc_phik)(co, data_binned[list(co)], noise_correction)
    168             for co in itertools.combinations_with_replacement(

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/joblib/parallel.py in __call__(self, iterable)
   1096
   1097             with self._backend.retrieval_context():
-> 1098                 self.retrieve()
   1099             # Make sure that we get a last message telling us we are done
   1100             elapsed_time = time.time() - self._start_time

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/joblib/parallel.py in retrieve(self)
    973             try:
    974                 if getattr(self._backend, 'supports_timeout', False):
--> 975                     self._output.extend(job.get(timeout=self.timeout))
    976                 else:
    977                     self._output.extend(job.get())

~/SageMaker/.envs/mykernel/lib/python3.9/site-packages/joblib/_parallel_backends.py in wrap_future_result(future, timeout)
    565         AsyncResults.get from multiprocessing."""
    566         try:
--> 567             return future.result(timeout=timeout)
    568         except CfTimeoutError as e:
    569             raise TimeoutError from e

~/SageMaker/.envs/mykernel/lib/python3.9/concurrent/futures/_base.py in result(self, timeout)
    436                     raise CancelledError()
    437                 elif self._state == FINISHED:
--> 438                     return self.__get_result()
    439
    440                 self._condition.wait(timeout)

~/SageMaker/.envs/mykernel/lib/python3.9/concurrent/futures/_base.py in __get_result(self)
    388         if self._exception:
    389             try:
--> 390                 raise self._exception
    391             finally:
    392                 # Break a reference cycle with the exception in self._exception

PicklingError: Could not pickle the task to send it to the workers.

I'm on the latest version of pandas-profiling (just installed it today).

@JohnTravolski Pandas profiling does not provide a pickable object.
After generating your report I suggest you save it as an HTML and use the html itself, instead of the report, as it is not pickable.
In order to provide better support I might need to have more details on what you aim to achieve. Join us at the data-centric AI community (https://datacentricai.community/) and I'll be happy to further discuss the topic.

@JohnTravolski Pandas profiling does not provide a pickable object.
After generating your report I suggest you save it as an HTML and use the HTML itself, instead of the report, as it is not pickable.
You might also want to test whether the machine you're working with has space for the process, given the error that is being prompted: OSError: [Errno 28] No space left on device
In order to provide better support I might need to have more details on what you aim to achieve. Join us at the data-centric AI community (https://datacentricai.community/) and I'll be happy to further discuss the topic.

I apologize, I updated my input. I did attempt to save it as an HTML and this was the error that was returned to me. I am simply trying to generate the typical html report and this is the error I receive. I am not very familiar with the AWS environment, but I will try to figure out the no space left on device issue. I was hoping somebody else who has used this with AWS had run into this before.

@JohnTravolski can you please provide more detail on the flow you are building?
So I can better support you.


Test Request: Kong Gateway EE version <IP_ADDRESS>-rc2

What component needs testing with KIC?
Kong Gateway EE
Container image
kong/kong-gateway-dev:<IP_ADDRESS>-rc2
Additional information
No response

@outsinre can you try this one again? There was something we broke in the test manifests around the time this started, and the error output (one of the containers never coming online) looks like that.

@rainest I do not have permission to trigger or re-run tests in this repo.
@randmonkey could you help re-run?


upgrades to php7.4

Why to force 7.4 that early?

because we use arrow functions.

I saw your commit but it's not used yet, is it?

We will probably publish enso 3.8.0 ~ 28 Dec. it won’t be merged until then.


New Docker instance, [CrashPlanEngine] starting... infinite loop

I ran into some issues with my crashplan docker instance so I've started completely from scratch and am setting it up like its a brand new install, but crashplan never actually starts. Here's the log output for the docker container,
`[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 00-app-niceness.sh: executing...
[cont-init.d] 00-app-niceness.sh: exited 0.
[cont-init.d] 00-app-script.sh: executing...
[cont-init.d] 00-app-script.sh: exited 0.
[cont-init.d] 00-app-user-map.sh: executing...
[cont-init.d] 00-app-user-map.sh: exited 0.
[cont-init.d] 00-clean-logmonitor-states.sh: executing...
[cont-init.d] 00-clean-logmonitor-states.sh: exited 0.
[cont-init.d] 00-clean-tmp-dir.sh: executing...
[cont-init.d] 00-clean-tmp-dir.sh: exited 0.
[cont-init.d] 00-set-app-deps.sh: executing...
[cont-init.d] 00-set-app-deps.sh: exited 0.
[cont-init.d] 00-set-home.sh: executing...
[cont-init.d] 00-set-home.sh: exited 0.
[cont-init.d] 00-take-config-ownership.sh: executing...
[cont-init.d] 00-take-config-ownership.sh: exited 0.
[cont-init.d] 00-xdg-runtime-dir.sh: executing...
[cont-init.d] 00-xdg-runtime-dir.sh: exited 0.
[cont-init.d] 10-certs.sh: executing...
[cont-init.d] 10-certs.sh: exited 0.
[cont-init.d] 10-cjk-font.sh: executing...
[cont-init.d] 10-cjk-font.sh: exited 0.
[cont-init.d] 10-nginx.sh: executing...
[cont-init.d] 10-nginx.sh: exited 0.
[cont-init.d] 10-vnc-password.sh: executing...
[cont-init.d] 10-vnc-password.sh: exited 0.
[cont-init.d] 10-web-index.sh: executing...
[cont-init.d] 10-web-index.sh: exited 0.
[cont-init.d] crashplan-pro.sh: executing...
[cont-init.d] crashplan-pro.sh: generating machine-id...
[cont-init.d] crashplan-pro.sh: handling initial run...
[cont-init.d] crashplan-pro.sh: exited 0.
[cont-init.d] validate_max_mem.sh: executing...
[cont-init.d] validate_max_mem.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] starting s6-fdholderd...
[services.d] starting CrashPlanEngine...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
s6-svwait: fatal: timed out
[services.d] starting xvfb...
[xvfb] starting...
The XKEYBOARD keymap compiler (xkbcomp) reports:

Warning:          Could not resolve keysym XF86BrightnessAuto
Warning:          Could not resolve keysym XF86DisplayOff
Warning:          Could not resolve keysym XF86Info
Warning:          Could not resolve keysym XF86AspectRatio
Warning:          Could not resolve keysym XF86DVD
Warning:          Could not resolve keysym XF86Audio
Warning:          Could not resolve keysym XF86ChannelUp
Warning:          Could not resolve keysym XF86ChannelDown
Warning:          Could not resolve keysym XF86Break
Warning:          Could not resolve keysym XF86VideoPhone
Warning:          Could not resolve keysym XF86ZoomReset
Warning:          Could not resolve keysym XF86Editor
Warning:          Could not resolve keysym XF86GraphicsEditor
Warning:          Could not resolve keysym XF86Presentation
Warning:          Could not resolve keysym XF86Database
Warning:          Could not resolve keysym XF86Voicemail
Warning:          Could not resolve keysym XF86Addressbook
Warning:          Could not resolve keysym XF86DisplayToggle
Warning:          Could not resolve keysym XF86SpellCheck
Warning:          Could not resolve keysym XF86ContextMenu
Warning:          Could not resolve keysym XF86MediaRepeat
Warning:          Could not resolve keysym XF8610ChannelsUp
Warning:          Could not resolve keysym XF8610ChannelsDown
Warning:          Could not resolve keysym XF86Images
Warning:          Could not resolve keysym XF86NotificationCenter
Warning:          Could not resolve keysym XF86PickupPhone
Warning:          Could not resolve keysym XF86HangupPhone
Warning:          Could not resolve keysym XF86Fn
Warning:          Could not resolve keysym XF86Fn_Esc
Warning:          Could not resolve keysym XF86FnRightShift
Warning:          Could not resolve keysym XF86Numeric0
Warning:          Could not resolve keysym XF86Numeric1
Warning:          Could not resolve keysym XF86Numeric2
Warning:          Could not resolve keysym XF86Numeric3
Warning:          Could not resolve keysym XF86Numeric4
Warning:          Could not resolve keysym XF86Numeric5
Warning:          Could not resolve keysym XF86Numeric6
Warning:          Could not resolve keysym XF86Numeric7
Warning:          Could not resolve keysym XF86Numeric8
Warning:          Could not resolve keysym XF86Numeric9
Warning:          Could not resolve keysym XF86NumericStar
Warning:          Could not resolve keysym XF86NumericPound
Warning:          Could not resolve keysym XF86NumericA
Warning:          Could not resolve keysym XF86NumericB
Warning:          Could not resolve keysym XF86NumericC
Warning:          Could not resolve keysym XF86NumericD
Warning:          Could not resolve keysym XF86CameraFocus
Warning:          Could not resolve keysym XF86WPSButton
Warning:          Could not resolve keysym XF86CameraZoomIn
Warning:          Could not resolve keysym XF86CameraZoomOut
Warning:          Could not resolve keysym XF86CameraUp
Warning:          Could not resolve keysym XF86CameraDown
Warning:          Could not resolve keysym XF86CameraLeft
Warning:          Could not resolve keysym XF86CameraRight
Warning:          Could not resolve keysym XF86AttendantOn
Warning:          Could not resolve keysym XF86AttendantOff
Warning:          Could not resolve keysym XF86AttendantToggle
Warning:          Could not resolve keysym XF86LightsToggle
Warning:          Could not resolve keysym XF86ALSToggle
Warning:          Could not resolve keysym XF86Buttonconfig
Warning:          Could not resolve keysym XF86Taskmanager
Warning:          Could not resolve keysym XF86Journal
Warning:          Could not resolve keysym XF86ControlPanel
Warning:          Could not resolve keysym XF86AppSelect
Warning:          Could not resolve keysym XF86Screensaver
Warning:          Could not resolve keysym XF86VoiceCommand
Warning:          Could not resolve keysym XF86Assistant
Warning:          Could not resolve keysym XF86BrightnessMin
Warning:          Could not resolve keysym XF86BrightnessMax
Warning:          Could not resolve keysym XF86KbdInputAssistPrev
Warning:          Could not resolve keysym XF86KbdInputAssistNext
Warning:          Could not resolve keysym XF86KbdInputAssistPrevgroup
Warning:          Could not resolve keysym XF86KbdInputAssistNextgroup
Warning:          Could not resolve keysym XF86KbdInputAssistAccept
Warning:          Could not resolve keysym XF86KbdInputAssistCancel
Warning:          Could not resolve keysym XF86RightUp
Warning:          Could not resolve keysym XF86RightDown
Warning:          Could not resolve keysym XF86LeftUp
Warning:          Could not resolve keysym XF86LeftDown
Warning:          Could not resolve keysym XF86RootMenu
Warning:          Could not resolve keysym XF86MediaTopMenu
Warning:          Could not resolve keysym XF86Numeric11
Warning:          Could not resolve keysym XF86Numeric12
Warning:          Could not resolve keysym XF86AudioDesc
Warning:          Could not resolve keysym XF863DMode
Warning:          Could not resolve keysym XF86NextFavorite
Warning:          Could not resolve keysym XF86StopRecord
Warning:          Could not resolve keysym XF86PauseRecord
Warning:          Could not resolve keysym XF86VOD
Warning:          Could not resolve keysym XF86Unmute
Warning:          Could not resolve keysym XF86FastReverse
Warning:          Could not resolve keysym XF86SlowReverse
Warning:          Could not resolve keysym XF86Data
Warning:          Could not resolve keysym XF86OnScreenKeyboard
Warning:          Could not resolve keysym XF86PrivacyScreenToggle
Warning:          Could not resolve keysym XF86SelectiveScreenshot
Warning:          Could not resolve keysym XF86Macro1
Warning:          Could not resolve keysym XF86Macro2
Warning:          Could not resolve keysym XF86Macro3
Warning:          Could not resolve keysym XF86Macro4
Warning:          Could not resolve keysym XF86Macro5
Warning:          Could not resolve keysym XF86Macro6
Warning:          Could not resolve keysym XF86Macro7
Warning:          Could not resolve keysym XF86Macro8
Warning:          Could not resolve keysym XF86Macro9
Warning:          Could not resolve keysym XF86Macro10
Warning:          Could not resolve keysym XF86Macro11
Warning:          Could not resolve keysym XF86Macro12
Warning:          Could not resolve keysym XF86Macro13
Warning:          Could not resolve keysym XF86Macro14
Warning:          Could not resolve keysym XF86Macro15
Warning:          Could not resolve keysym XF86Macro16
Warning:          Could not resolve keysym XF86Macro17
Warning:          Could not resolve keysym XF86Macro18
Warning:          Could not resolve keysym XF86Macro19
Warning:          Could not resolve keysym XF86Macro20
Warning:          Could not resolve keysym XF86Macro21
Warning:          Could not resolve keysym XF86Macro22
Warning:          Could not resolve keysym XF86Macro23
Warning:          Could not resolve keysym XF86Macro24
Warning:          Could not resolve keysym XF86Macro25
Warning:          Could not resolve keysym XF86Macro26
Warning:          Could not resolve keysym XF86Macro27
Warning:          Could not resolve keysym XF86Macro28
Warning:          Could not resolve keysym XF86Macro29
Warning:          Could not resolve keysym XF86Macro30
Warning:          Could not resolve keysym XF86MacroRecordStart
Warning:          Could not resolve keysym XF86MacroRecordStop
Warning:          Could not resolve keysym XF86MacroPresetCycle
Warning:          Could not resolve keysym XF86MacroPreset1
Warning:          Could not resolve keysym XF86MacroPreset2
Warning:          Could not resolve keysym XF86MacroPreset3
Warning:          Could not resolve keysym XF86KbdLcdMenu1
Warning:          Could not resolve keysym XF86KbdLcdMenu2
Warning:          Could not resolve keysym XF86KbdLcdMenu3
Warning:          Could not resolve keysym XF86KbdLcdMenu4
Warning:          Could not resolve keysym XF86KbdLcdMenu5
Errors from xkbcomp are not fatal to the X server
[services.d] starting nginx...
The XKEYBOARD keymap compiler (xkbcomp) reports:
Warning:          Could not resolve keysym XF86BrightnessAuto
Warning:          Could not resolve keysym XF86DisplayOff
Warning:          Could not resolve keysym XF86Info
Warning:          Could not resolve keysym XF86AspectRatio
Warning:          Could not resolve keysym XF86DVD
Warning:          Could not resolve keysym XF86Audio
Warning:          Could not resolve keysym XF86ChannelUp
Warning:          Could not resolve keysym XF86ChannelDown
Warning:          Could not resolve keysym XF86Break
Warning:          Could not resolve keysym XF86VideoPhone
Warning:          Could not resolve keysym XF86ZoomReset
Warning:          Could not resolve keysym XF86Editor
Warning:          Could not resolve keysym XF86GraphicsEditor
Warning:          Could not resolve keysym XF86Presentation
Warning:          Could not resolve keysym XF86Database
Warning:          Could not resolve keysym XF86Voicemail
Warning:          Could not resolve keysym XF86Addressbook
Warning:          Could not resolve keysym XF86DisplayToggle
Warning:          Could not resolve keysym XF86SpellCheck
Warning:          Could not resolve keysym XF86ContextMenu
Warning:          Could not resolve keysym XF86MediaRepeat
Warning:          Could not resolve keysym XF8610ChannelsUp
Warning:          Could not resolve keysym XF8610ChannelsDown
Warning:          Could not resolve keysym XF86Images
Warning:          Could not resolve keysym XF86NotificationCenter
Warning:          Could not resolve keysym XF86PickupPhone
Warning:          Could not resolve keysym XF86HangupPhone
Warning:          Could not resolve keysym XF86Fn
Warning:          Could not resolve keysym XF86Fn_Esc
Warning:          Could not resolve keysym XF86FnRightShift
Warning:          Could not resolve keysym XF86Numeric0
Warning:          Could not resolve keysym XF86Numeric1
Warning:          Could not resolve keysym XF86Numeric2
Warning:          Could not resolve keysym XF86Numeric3
Warning:          Could not resolve keysym XF86Numeric4
Warning:          Could not resolve keysym XF86Numeric5
Warning:          Could not resolve keysym XF86Numeric6
Warning:          Could not resolve keysym XF86Numeric7
Warning:          Could not resolve keysym XF86Numeric8
Warning:          Could not resolve keysym XF86Numeric9
Warning:          Could not resolve keysym XF86NumericStar
Warning:          Could not resolve keysym XF86NumericPound
Warning:          Could not resolve keysym XF86NumericA
Warning:          Could not resolve keysym XF86NumericB
Warning:          Could not resolve keysym XF86NumericC
Warning:          Could not resolve keysym XF86NumericD
Warning:          Could not resolve keysym XF86CameraFocus
Warning:          Could not resolve keysym XF86WPSButton
Warning:          Could not resolve keysym XF86CameraZoomIn
Warning:          Could not resolve keysym XF86CameraZoomOut
Warning:          Could not resolve keysym XF86CameraUp
Warning:          Could not resolve keysym XF86CameraDown
Warning:          Could not resolve keysym XF86CameraLeft
Warning:          Could not resolve keysym XF86CameraRight
Warning:          Could not resolve keysym XF86AttendantOn
Warning:          Could not resolve keysym XF86AttendantOff
Warning:          Could not resolve keysym XF86AttendantToggle
Warning:          Could not resolve keysym XF86LightsToggle
Warning:          Could not resolve keysym XF86ALSToggle
Warning:          Could not resolve keysym XF86Buttonconfig
Warning:          Could not resolve keysym XF86Taskmanager
Warning:          Could not resolve keysym XF86Journal
Warning:          Could not resolve keysym XF86ControlPanel
Warning:          Could not resolve keysym XF86AppSelect
Warning:          Could not resolve keysym XF86Screensaver
Warning:          Could not resolve keysym XF86VoiceCommand
Warning:          Could not resolve keysym XF86Assistant
Warning:          Could not resolve keysym XF86BrightnessMin
Warning:          Could not resolve keysym XF86BrightnessMax
Warning:          Could not resolve keysym XF86KbdInputAssistPrev
Warning:          Could not resolve keysym XF86KbdInputAssistNext
Warning:          Could not resolve keysym XF86KbdInputAssistPrevgroup
Warning:          Could not resolve keysym XF86KbdInputAssistNextgroup
Warning:          Could not resolve keysym XF86KbdInputAssistAccept
Warning:          Could not resolve keysym XF86KbdInputAssistCancel
Warning:          Could not resolve keysym XF86RightUp
Warning:          Could not resolve keysym XF86RightDown
Warning:          Could not resolve keysym XF86LeftUp
Warning:          Could not resolve keysym XF86LeftDown
Warning:          Could not resolve keysym XF86RootMenu
Warning:          Could not resolve keysym XF86MediaTopMenu
Warning:          Could not resolve keysym XF86Numeric11
Warning:          Could not resolve keysym XF86Numeric12
Warning:          Could not resolve keysym XF86AudioDesc
Warning:          Could not resolve keysym XF863DMode
Warning:          Could not resolve keysym XF86NextFavorite
Warning:          Could not resolve keysym XF86StopRecord
Warning:          Could not resolve keysym XF86PauseRecord
Warning:          Could not resolve keysym XF86VOD
Warning:          Could not resolve keysym XF86Unmute
Warning:          Could not resolve keysym XF86FastReverse
Warning:          Could not resolve keysym XF86SlowReverse
Warning:          Could not resolve keysym XF86Data
Warning:          Could not resolve keysym XF86OnScreenKeyboard
Warning:          Could not resolve keysym XF86PrivacyScreenToggle
Warning:          Could not resolve keysym XF86SelectiveScreenshot
Warning:          Could not resolve keysym XF86Macro1
Warning:          Could not resolve keysym XF86Macro2
Warning:          Could not resolve keysym XF86Macro3
Warning:          Could not resolve keysym XF86Macro4
Warning:          Could not resolve keysym XF86Macro5
Warning:          Could not resolve keysym XF86Macro6
Warning:          Could not resolve keysym XF86Macro7
Warning:          Could not resolve keysym XF86Macro8
Warning:          Could not resolve keysym XF86Macro9
Warning:          Could not resolve keysym XF86Macro10
Warning:          Could not resolve keysym XF86Macro11
Warning:          Could not resolve keysym XF86Macro12
Warning:          Could not resolve keysym XF86Macro13
Warning:          Could not resolve keysym XF86Macro14
Warning:          Could not resolve keysym XF86Macro15
Warning:          Could not resolve keysym XF86Macro16
Warning:          Could not resolve keysym XF86Macro17
Warning:          Could not resolve keysym XF86Macro18
Warning:          Could not resolve keysym XF86Macro19
Warning:          Could not resolve keysym XF86Macro20
Warning:          Could not resolve keysym XF86Macro21
Warning:          Could not resolve keysym XF86Macro22
Warning:          Could not resolve keysym XF86Macro23
Warning:          Could not resolve keysym XF86Macro24
Warning:          Could not resolve keysym XF86Macro25
Warning:          Could not resolve keysym XF86Macro26
Warning:          Could not resolve keysym XF86Macro27
Warning:          Could not resolve keysym XF86Macro28
Warning:          Could not resolve keysym XF86Macro29
Warning:          Could not resolve keysym XF86Macro30
Warning:          Could not resolve keysym XF86MacroRecordStart
Warning:          Could not resolve keysym XF86MacroRecordStop
Warning:          Could not resolve keysym XF86MacroPresetCycle
Warning:          Could not resolve keysym XF86MacroPreset1
Warning:          Could not resolve keysym XF86MacroPreset2
Warning:          Could not resolve keysym XF86MacroPreset3
Warning:          Could not resolve keysym XF86KbdLcdMenu1
Warning:          Could not resolve keysym XF86KbdLcdMenu2
Warning:          Could not resolve keysym XF86KbdLcdMenu3
Warning:          Could not resolve keysym XF86KbdLcdMenu4
Warning:          Could not resolve keysym XF86KbdLcdMenu5
Errors from xkbcomp are not fatal to the X server
[services.d] starting certsmonitor...
[nginx] starting...
[services.d] starting x11vnc...
[services.d] starting statusmonitor...
[certsmonitor] disabling service: secure connection not enabled.
[services.d] starting openbox...
[x11vnc] starting...
[openbox] starting...
[statusmonitor] starting...
24/02/2022 21:07:48 passing arg to libvncserver: -rfbport
24/02/2022 21:07:48 passing arg to libvncserver: 5900
24/02/2022 21:07:48 passing arg to libvncserver: -rfbportv6
24/02/2022 21:07:48 passing arg to libvncserver: -1
24/02/2022 21:07:48 passing arg to libvncserver: -httpportv6
24/02/2022 21:07:48 passing arg to libvncserver: -1
24/02/2022 21:07:48 passing arg to libvncserver: -desktop
24/02/2022 21:07:48 passing arg to libvncserver: CrashPlan for Small Business
24/02/2022 21:07:48 x11vnc version: 0.9.14 lastmod: 2015-11-14  pid: 953
24/02/2022 21:07:48 Using X display :0
24/02/2022 21:07:48 rootwin: 0x43 reswin: 0x200001 dpy: 0x72ee040
24/02/2022 21:07:48
24/02/2022 21:07:48 ------------------ USEFUL INFORMATION ------------------
24/02/2022 21:07:48 X DAMAGE available on display, using it for polling hints.
24/02/2022 21:07:48   To disable this behavior use: '-noxdamage'
24/02/2022 21:07:48
24/02/2022 21:07:48   Most compositing window managers like 'compiz' or 'beryl'
24/02/2022 21:07:48   cause X DAMAGE to fail, and so you may not see any screen
24/02/2022 21:07:48   updates via VNC.  Either disable 'compiz' (recommended) or
24/02/2022 21:07:48   supply the x11vnc '-noxdamage' command line option.
24/02/2022 21:07:48 X COMPOSITE available on display, using it for window polling.
24/02/2022 21:07:48   To disable this behavior use: '-noxcomposite'
24/02/2022 21:07:48
24/02/2022 21:07:48 Wireframing: -wireframe mode is in effect for window moves.
24/02/2022 21:07:48   If this yields undesired behavior (poor response, painting
24/02/2022 21:07:48   errors, etc) it may be disabled:
24/02/2022 21:07:48    - use '-nowf' to disable wireframing completely.
24/02/2022 21:07:48    - use '-nowcr' to disable the Copy Rectangle after the
24/02/2022 21:07:48      moved window is released in the new position.
24/02/2022 21:07:48   Also see the -help entry for tuning parameters.
24/02/2022 21:07:48   You can press 3 Alt_L's (Left "Alt" key) in a row to
24/02/2022 21:07:48   repaint the screen, also see the -fixscreen option for
24/02/2022 21:07:48   periodic repaints.
24/02/2022 21:07:48 GrabServer control via XTEST.
24/02/2022 21:07:48
24/02/2022 21:07:48 Scroll Detection: -scrollcopyrect mode is in effect to
24/02/2022 21:07:48   use RECORD extension to try to detect scrolling windows
24/02/2022 21:07:48   (induced by either user keystroke or mouse input).
24/02/2022 21:07:48   If this yields undesired behavior (poor response, painting
24/02/2022 21:07:48   errors, etc) it may be disabled via: '-noscr'
24/02/2022 21:07:48   Also see the -help entry for tuning parameters.
24/02/2022 21:07:48   You can press 3 Alt_L's (Left "Alt" key) in a row to
24/02/2022 21:07:48   repaint the screen, also see the -fixscreen option for
24/02/2022 21:07:48   periodic repaints.
24/02/2022 21:07:48
24/02/2022 21:07:48 XKEYBOARD: number of keysyms per keycode 7 is greater
24/02/2022 21:07:48   than 4 and 51 keysyms are mapped above 4.
24/02/2022 21:07:48   Automatically switching to -xkb mode.
24/02/2022 21:07:48   If this makes the key mapping worse you can
24/02/2022 21:07:48   disable it with the "-noxkb" option.
24/02/2022 21:07:48   Also, remember "-remap DEAD" for accenting characters.
24/02/2022 21:07:48
24/02/2022 21:07:48 X FBPM extension not supported.
Xlib:  extension "DPMS" missing on display ":0".
24/02/2022 21:07:48 X display is not capable of DPMS.
24/02/2022 21:07:48 --------------------------------------------------------
24/02/2022 21:07:48
24/02/2022 21:07:48 Default visual ID: 0x21
24/02/2022 21:07:48 Read initial data from X display into framebuffer.
24/02/2022 21:07:48 initialize_screen: fb_depth/fb_bpp/fb_Bpl 24/32/5120
24/02/2022 21:07:48
24/02/2022 21:07:48 X display :0 is 32bpp depth=24 true color
24/02/2022 21:07:48
24/02/2022 21:07:48 Listening for VNC connections on TCP port 5900
24/02/2022 21:07:48
24/02/2022 21:07:48 Xinerama is present and active (e.g. multi-head).
24/02/2022 21:07:48 Xinerama: number of sub-screens: 1
24/02/2022 21:07:48 Xinerama: no blackouts needed (only one sub-screen)
24/02/2022 21:07:48
24/02/2022 21:07:48 fb read rate: 1494 MB/sec
24/02/2022 21:07:48 fast read: reset -wait  ms to: 10
24/02/2022 21:07:48 fast read: reset -defer ms to: 10
24/02/2022 21:07:48 The X server says there are 10 mouse buttons.
24/02/2022 21:07:48 screen setup finished.
24/02/2022 21:07:48

The VNC desktop is:      b806a8fc61e0:0
PORT=5900

Have you tried the x11vnc '-ncache' VNC client-side pixel caching feature yet?
The scheme stores pixel data offscreen on the VNC viewer side for faster
retrieval.  It should work with any VNC viewer.  Try it by running:
x11vnc -ncache 10 ...

One can also add -ncache_cr for smooth 'copyrect' window motion.
More info: http://www.karlrunge.com/x11vnc/faq.html#faq-client-caching
[services.d] starting logmonitor...
[services.d] starting app...
[logmonitor] starting...
[app] starting CrashPlan for Small Business...
[services.d] done.
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...
[CrashPlanEngine] starting...

Your log is saying
"Listening for VNC connections on TCP port 5900"
but your command shows you specify port 5800? Maybe unrelated, just had me wondering.

To check why the engine is not starting, you can try to look at the following log files, located under /saltlake/var/crashplan-data/log:

engine_error.log
engine_output.log
service.log.0

I made an ext4 volume on my ZFS array and made that the /config volume, now the docker instance starts right up. So it seems theres a file system incompatibility perhaps.
Here are the log files form the previous attempt (ZFS file system)
engine_error.log
WARNING: Illegal reflective access by com.code42.crypto.jce.ec.EcCurveLookup (file:/usr/local/crashplan/lib/c42-crypto-impl-15.2.3.jar) to method sun.security                                                                                .util.CurveDB.lookup(java.security.spec.ECParameterSpec) WARNING: Please consider reporting this to the maintainers of com.code42.crypto.jce.ec.EcCurveLookup WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release
engine_output.log
Initializing the JVM arguments. Creating the Java virtual machine. Number of arguments: 20 Java virtual machine created. Starting service. [02.24.22 21:12:19.314 INFO  main             com.code42.utils.ClassFinder] Loaded classpaths in 1261 ms Service is complete. Launching as root Paths:   cwd:      /usr/local/crashplan   appData:  /usr/local/crashplan   jvm_args: /usr/local/crashplan/conf/jvm_args JVM arguments in use: -Dsun.jnu.encoding=UTF-8 -Dfile.encoding=UTF-8 -Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -Dnetworkaddress.cache.negative.ttl=0 -Dapp=Code42Service -DappBaseName=Code42 -Djna.nosys=true -Djava.awt.headless=true -DCode42=Code42 -Dnashorn.args=--no-deprecation-warning -Dc42.installation.enforcement.enabled=true -XX:-ShrinkHeapInSteps -XX:MaxHeapFreeRatio=30 -XX:MinHeapFreeRatio=15 -Djava.library.path=/usr/local/crashplan/nlib -Djna.library.path=/usr/local/crashplan/nlib -Dc42.log.config=/usr/local/crashplan/conf/service.log.xml Arguments in use:
service.log.0
`"Reference Handler" #6 daemon prio=10 tid=0x0000000000000002 nid=0x6 runnable [0x0000000065c3d39f]
java.lang.Thread.State: RUNNABLE
at java.base@11.0.12/java.lang.ref.Reference.waitForReferencePendingList(Native Method)
at java.base@11.0.12/java.lang.ref.Reference.processPendingReferences(Unknown Source)
at java.base@11.0.12/java.lang.ref.Reference$ReferenceHandler.run(Unknown Source)
"Finalizer" #5 daemon prio=8 tid=0x0000000000000003 nid=0x5 in Object.wait() [0x000000000e83e8fb]
java.lang.Thread.State: WAITING (on object monitor)
at java.base@11.0.12/java.lang.Object.wait(Native Method)
at java.base@11.0.12/java.lang.ref.ReferenceQueue.remove(Unknown Source)
at java.base@11.0.12/java.lang.ref.ReferenceQueue.remove(Unknown Source)
at java.base@11.0.12/java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)
"Signal Dispatcher" #4 daemon prio=9 tid=0x0000000000000004 nid=0x4 runnable [0x000000001ec37dc1]
java.lang.Thread.State: RUNNABLE
"Common-Cleaner" #3 daemon prio=8 tid=0x000000000000000a nid=0x3 in Object.wait() [0x000000003494183d]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.base@11.0.12/java.lang.Object.wait(Native Method)
at java.base@11.0.12/java.lang.ref.ReferenceQueue.remove(Unknown Source)
at java.base@11.0.12/jdk.internal.ref.CleanerImpl.run(Unknown Source)
at java.base@11.0.12/java.lang.Thread.run(Unknown Source)
at java.base@11.0.12/jdk.internal.misc.InnocuousThread.run(Unknown Source)
"Thread-0" #2 prio=5 tid=0x000000000000000d nid=0x2 runnable [0x000000003bb04b53]
java.lang.Thread.State: RUNNABLE
at java.base@11.0.12/java.lang.Thread.dumpThreads(Native Method)
at java.base@11.0.12/java.lang.Thread.getAllStackTraces(Unknown Source)
at app//com.code42.logging.ThreadDumpLogger.threadDump(ThreadDumpLogger.java:99)
at app//com.code42.logging.ThreadDumpLogger.dumpThreads(ThreadDumpLogger.java:57)
at app//com.code42.logging.ThreadDumpLogger.dumpThreadsToComAndThreadLogs(ThreadDumpLogger.java:91)
at app//com.backup42.service.CPService.dumpThreads(CPService.java:1419)
at app//com.backup42.service.CPService.cleanup(CPService.java:1291)
at app//com.backup42.service.CPService.lambda$start$0(CPService.java:423)
at app//com.backup42.service.CPService$$Lambda$206/0x000000080043c840.run(Unknown Source)
at java.base@11.0.12/java.lang.Thread.run(Unknown Source)
"process reaper" #1 daemon prio=10 tid=0x000000000000000e nid=0x1 waiting on condition [0x000000004fb06a1b]
java.lang.Thread.State: TIMED_WAITING (parking)
at java.base@11.0.12/jdk.internal.misc.Unsafe.park(Native Method)
at java.base@11.0.12/java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
at java.base@11.0.12/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
at java.base@11.0.12/java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
at java.base@11.0.12/java.util.concurrent.SynchronousQueue.poll(Unknown Source)
at java.base@11.0.12/java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
at java.base@11.0.12/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base@11.0.12/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base@11.0.12/java.lang.Thread.run(Unknown Source)`

Unfortunately I don't see anything obvious from what you provided.
But you are right, CrashPlan is probably using an operation not supported by the ZFS filesystem...


aspect error messages reference bazel

When an info / warn / error message is thrown by bazel, these hint to the user to run bazel, however we likely want to hint to the user to run aspect instead as this may lead to confusion.
eg:
$ bazel-bin/cmd/aspect/aspect_/aspect build
WARNING: Usage: bazel build <options> <targets>.
Invoke `bazel help build` for full description of usage and options.
Your request is correct, but requested an empty set of targets. Nothing will be built.
INFO: Analyzed 0 targets (0 packages loaded, 0 targets configured).
INFO: Found 0 targets...
INFO: Elapsed time: 0.155s, Critical Path: 0.04s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action

To be precise I think devx teams will commonly install under another name, so we should use $0 everywhere and/or a product_name config setting. Some might actually install as 'bazel' like we do with bazelisk

Yah good point, $0 or configure a product name is better here.


Hooks API

TODO:

[x] Support transaction hooks & context passing to sql and query
[x] Catch hook errors as a noop and pass previous context
[x] Tests for transaction hooks catching errors
[x] Clean up api (duplication in hooks lifecycle)
[x] Documentation

Good point!

On Dec 27, 2018, at 5:00 PM, Alex Munda<EMAIL_ADDRESS>wrote:
@alexmunda commented on this pull request.
In README.md:

@@ -113,6 +113,125 @@ function ApiRequestHandler(request, reply) {

In the above example isolated_db is the same instance of TinyPg except with an overridden events property and dispose method to remove all listeners. The UserService can create other services and pass its reference to isolated_db to other services. In doing so, you can track all database queries executed as the result of every API request.
+## Hooks
Ah ok sure, preSql and preRawQuery (the only transformative hooks) should throw but the others should not. I like that idea
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.


⚠ Moon has degraded performance

In baea726, Moon (http://moon.sb) experienced degraded performance:

HTTP code: 200
Response time: 5370 ms

Resolved: Moon performance has improved in 05608ed.


Replace react & bootstrap with Astro & Tailwind

Workflow might need some tweaking as I don't have access for trial and error
Can be tested through:
cd homepage
npm i
npm run dev

Tackles #83

[x] use dev.json to get the developer names and info for the page
[x] enable dark/light mode based on user machine preference.
[x] it should be responsive on mobile
[ ] not sure if I would be able to do Arabic/English versions without affecting SEO.
[ ] Maybe a quick search and filter textarea to search for developers or repo names.

Just gave you admin access to the workflow

Thanks. I would attempt merging and making it work, if something goes wrong I will revert back.


fix lint warnings in pilot

I see the following lint warnings in pilot. This PR fixes those
trivial lint warnings.
pilot/cmd/pilot-discovery/main.go:176::warning: declaration of "client" shadows declaration at pilot/cmd/pilot-discovery/main.go:103 (vetshadow)
pilot/platform/kube/conversion_test.go:208:13:warning: if block ends with a return statement, so drop this else and outdent its block (golint)

What this PR does / why we need it:
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:
Release note:

none

/assign @kyessenov

@jmuk: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-presubmit.sh
e33ee98412bd195c8df9232f30f7b0f42a3e4bcc
link
/test istio-presubmit

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/lgtm