Firefox webgl and performance

I am getting this error on firefox with my project:
Error when getting WebGL context:  Error: Cannot create a canvas in this context 2 bundle.worker.js:40131:13
Initialization of backend webgl failed bundle.worker.js:53628:15
MathBackendWebGL@http://localhost:3000/static/js/bundle.worker.js:15334:13
../audio-ml-plugin/node_modules/@tensorflow/tfjs-backend-webgl/dist/base.js/<@http://localhost:3000/static/js/bundle.worker.js:16354:96
In the human demo Firefox is half the FPS of Chrome.
Any ideas?
Thanks

Web worker version of human probably doesn't work on Firefox because OffscreenCanvas is required, which is still a behind-a-flag feature on Firefox.

Web worker version of human probably doesn't work on Firefox because OffscreenCanvas is required, which is still a behind-a-flag feature on Firefox.

First, http://localhost:3000/static/js/bundle.worker.js:15334:13 is clearly your custom bundle - so I cannot correlate it to an actual source code. Please post actual error backtrace and I'll take a look.
But, if you're using web workers, Human currently does not support Firefox due to lack of OffscreenCanvas() support so there is no efficient way to handle image data inside a worker thread. It could be done with some workaround, but it would be much slower and I'd rather not do special-case just for Firefox - they should really enable the feature.
(@ButzYung you're correct & thanks for jumping in)
Second, regarding Firefox performance using webgl backend, yes, it is slower primarily because Firefox performs very aggressive GPU memory garbage collection so uploaded GL shaders do not stay cached and need to be re-uploaded frequently. Unfortunately, there is no way to modify this Firefox behavior.
But in my test cases, difference is ~20% and not 50%. Can you provide more details on this?

First, http://localhost:3000/static/js/bundle.worker.js:15334:13 is clearly your custom bundle - so I cannot correlate it to an actual source code. Please post actual error backtrace and I'll take a look.
But, if you're using web workers, Human currently does not support Firefox due to lack of OffscreenCanvas() support so there is no efficient way to handle image data inside a worker thread. It could be done with some workaround, but it would be much slower and I'd rather not do special-case just for Firefox - they should really enable the feature.
(@ButzYung you're correct & thanks for jumping in)
Second, regarding Firefox performance using webgl backend, yes, it is slower primarily because Firefox performs very aggressive GPU memory garbage collection so uploaded GL shaders do not stay cached and need to be re-uploaded frequently. Unfortunately, there is no way to modify this Firefox behavior.
But in my test cases, difference is ~20% and not 50%. Can you provide more details on this?

I am actually not using Web Worker but the strange thing is that I always get this error but it still works...
BTW you were right, it works well with the UI and all without Web Worker. On your demo I see on my Mac 14 fps for Chrome and 7 fps for Firefox.
Anyway I am trying to run some instances on remote browsers controlled by Microsoft Remote Desktop and on Machine without GPU card Chrome failed to use WebGL and nothing works while Firefox works on the same machine. On another machine with a GPU card Chrome works and even get a performance boost (compared to my Mac). but Firefox does not get the same boost.
Anyway I am generally very happy with the library.

I am actually not using Web Worker but the strange thing is that I always get this error but it still works...
BTW you were right, it works well with the UI and all without Web Worker. On your demo I see on my Mac 14 fps for Chrome and 7 fps for Firefox.
Anyway I am trying to run some instances on remote browsers controlled by Microsoft Remote Desktop and on Machine without GPU card Chrome failed to use WebGL and nothing works while Firefox works on the same machine. On another machine with a GPU card Chrome works and even get a performance boost (compared to my Mac). but Firefox does not get the same boost.
Anyway I am generally very happy with the library.

the strange thing is that I always get this error but it still works...

Like I said, I need an actual backtrace to investigate.
One possibility is that registration of WebGL2 plus extensions in src/tfjs/backend.js goes wrong which is a non-fatal error since basic WebGL1 is still operational.
Also, could you copy&paste browser console message regarding GL version and TF flags when using Chrome and Firefox?
It should look like
11:35:22.243 Human: gl version:WebGL 2.0 (OpenGL ES 3.0 Chromium) renderer:WebKit WebGL
11:35:22.243 Human: tf flags: ...

on my Mac 14 fps for Chrome and 7 fps for Firefox.

This is just a general note - MacOS is fairly closed platform, any hardware integration is really difficult. Chrome uses Blink which is a fork of Apple's version of WebKit so it's likely much closer to hardware optimizations. Firefox on the other hand uses variations of Gecko engine which doesn't have same level of Mac integration.

the strange thing is that I always get this error but it still works...

Like I said, I need an actual backtrace to investigate.
One possibility is that registration of WebGL2 plus extensions in src/tfjs/backend.js goes wrong which is a non-fatal error since basic WebGL1 is still operational.
Also, could you copy&paste browser console message regarding GL version and TF flags when using Chrome and Firefox?
It should look like
11:35:22.243 Human: gl version:WebGL 2.0 (OpenGL ES 3.0 Chromium) renderer:WebKit WebGL
11:35:22.243 Human: tf flags: ...

on my Mac 14 fps for Chrome and 7 fps for Firefox.

This is just a general note - MacOS is fairly closed platform, any hardware integration is really difficult. Chrome uses Blink which is a fork of Apple's version of WebKit so it's likely much closer to hardware optimizations. Firefox on the other hand uses variations of Gecko engine which doesn't have same level of Mac integration.

Chomre:
11:58:36.696 Human: version: 0.9.24 TensorFlow/JS version: 2.8.2
0.chunk.js:287641 11:58:36.696 Human: setting backend: webgl
0.chunk.js:287641 11:58:36.720 Human: gl version:WebGL 2.0 (OpenGL ES 3.0 Chromium) renderer:WebKit WebGL
0.chunk.js:287641 11:58:36.720 Human: tf flags: ObjectDEBUG: falseHAS_WEBGL: trueIS_BROWSER: trueIS_NODE: falseIS_TEST: falsePROD: trueTENSORLIKE_CHECK_SHAPE_CONSISTENCY: falseWEBGL_BUFFER_SUPPORTED: trueWEBGL_CHECK_NUMERICAL_PROBLEMS: falseWEBGL_CONV_IM2COL: trueWEBGL_CPU_FORWARD: trueWEBGL_DELETE_TEXTURE_THRESHOLD: -1WEBGL_DOWNLOAD_FLOAT_ENABLED: trueWEBGL_FENCE_API_ENABLED: trueWEBGL_FORCE_F16_TEXTURES: trueWEBGL_LAZILY_UNPACK: trueWEBGL_MAX_TEXTURES_IN_SHADER: 16WEBGL_MAX_TEXTURE_SIZE: 16384WEBGL_PACK: trueWEBGL_PACK_ARRAY_OPERATIONS: trueWEBGL_PACK_BINARY_OPERATIONS: trueWEBGL_PACK_DEPTHWISECONV: trueWEBGL_PACK_IMAGE_OPERATIONS: trueWEBGL_PACK_UNARY_OPERATIONS: trueWEBGL_RENDER_FLOAT32_ENABLED: falseWEBGL_SIZE_UPLOAD_UNIFORM: 0WEBGL_VERSION: 2__proto__: Object

Chomre:
11:58:36.696 Human: version: 0.9.24 TensorFlow/JS version: 2.8.2
0.chunk.js:287641 11:58:36.696 Human: setting backend: webgl
0.chunk.js:287641 11:58:36.720 Human: gl version:WebGL 2.0 (OpenGL ES 3.0 Chromium) renderer:WebKit WebGL
0.chunk.js:287641 11:58:36.720 Human: tf flags: ObjectDEBUG: falseHAS_WEBGL: trueIS_BROWSER: trueIS_NODE: falseIS_TEST: falsePROD: trueTENSORLIKE_CHECK_SHAPE_CONSISTENCY: falseWEBGL_BUFFER_SUPPORTED: trueWEBGL_CHECK_NUMERICAL_PROBLEMS: falseWEBGL_CONV_IM2COL: trueWEBGL_CPU_FORWARD: trueWEBGL_DELETE_TEXTURE_THRESHOLD: -1WEBGL_DOWNLOAD_FLOAT_ENABLED: trueWEBGL_FENCE_API_ENABLED: trueWEBGL_FORCE_F16_TEXTURES: trueWEBGL_LAZILY_UNPACK: trueWEBGL_MAX_TEXTURES_IN_SHADER: 16WEBGL_MAX_TEXTURE_SIZE: 16384WEBGL_PACK: trueWEBGL_PACK_ARRAY_OPERATIONS: trueWEBGL_PACK_BINARY_OPERATIONS: trueWEBGL_PACK_DEPTHWISECONV: trueWEBGL_PACK_IMAGE_OPERATIONS: trueWEBGL_PACK_UNARY_OPERATIONS: trueWEBGL_RENDER_FLOAT32_ENABLED: falseWEBGL_SIZE_UPLOAD_UNIFORM: 0WEBGL_VERSION: 2__proto__: Object

Firefox:
12:02:56.115 Human: version: 0.9.24 TensorFlow/JS version: 2.8.2 log.js:6
12:02:56.115 Human: setting backend: webgl log.js:6
12:02:56.146 Human: gl version:WebGL 2.0 renderer:Mozilla
12:02:56.147 Human: tf flags:
DEBUG: false

HAS_WEBGL: true

IS_BROWSER: true

IS_NODE: false

IS_TEST: false

PROD: true

TENSORLIKE_CHECK_SHAPE_CONSISTENCY: false

WEBGL_BUFFER_SUPPORTED: true

WEBGL_CHECK_NUMERICAL_PROBLEMS: false

WEBGL_CONV_IM2COL: true

WEBGL_CPU_FORWARD: true

WEBGL_DELETE_TEXTURE_THRESHOLD: -1

WEBGL_DOWNLOAD_FLOAT_ENABLED: true

WEBGL_FENCE_API_ENABLED: true

WEBGL_FORCE_F16_TEXTURES: true

WEBGL_LAZILY_UNPACK: true

WEBGL_MAX_TEXTURES_IN_SHADER: 16

WEBGL_MAX_TEXTURE_SIZE: 8192

WEBGL_PACK: true

WEBGL_PACK_ARRAY_OPERATIONS: true

WEBGL_PACK_BINARY_OPERATIONS: true

WEBGL_PACK_DEPTHWISECONV: true

WEBGL_PACK_IMAGE_OPERATIONS: true

WEBGL_PACK_UNARY_OPERATIONS: true

WEBGL_RENDER_FLOAT32_ENABLED: false

WEBGL_SIZE_UPLOAD_UNIFORM: 0

WEBGL_VERSION: 2

Firefox:
12:02:56.115 Human: version: 0.9.24 TensorFlow/JS version: 2.8.2 log.js:6
12:02:56.115 Human: setting backend: webgl log.js:6
12:02:56.146 Human: gl version:WebGL 2.0 renderer:Mozilla
12:02:56.147 Human: tf flags:
DEBUG: false

HAS_WEBGL: true

IS_BROWSER: true

IS_NODE: false

IS_TEST: false

PROD: true

TENSORLIKE_CHECK_SHAPE_CONSISTENCY: false

WEBGL_BUFFER_SUPPORTED: true

WEBGL_CHECK_NUMERICAL_PROBLEMS: false

WEBGL_CONV_IM2COL: true

WEBGL_CPU_FORWARD: true

WEBGL_DELETE_TEXTURE_THRESHOLD: -1

WEBGL_DOWNLOAD_FLOAT_ENABLED: true

WEBGL_FENCE_API_ENABLED: true

WEBGL_FORCE_F16_TEXTURES: true

WEBGL_LAZILY_UNPACK: true

WEBGL_MAX_TEXTURES_IN_SHADER: 16

WEBGL_MAX_TEXTURE_SIZE: 8192

WEBGL_PACK: true

WEBGL_PACK_ARRAY_OPERATIONS: true

WEBGL_PACK_BINARY_OPERATIONS: true

WEBGL_PACK_DEPTHWISECONV: true

WEBGL_PACK_IMAGE_OPERATIONS: true

WEBGL_PACK_UNARY_OPERATIONS: true

WEBGL_RENDER_FLOAT32_ENABLED: false

WEBGL_SIZE_UPLOAD_UNIFORM: 0

WEBGL_VERSION: 2

unfortunately, there is nothing strange in the flags.
and you didn't post full backtrace of error Error when getting WebGL context: Error: Cannot create a canvas in this context, only first two lines.
but i did notice one thing - backtrace is from tfjs-backend-webgl which is a module dependency of audio-ml-plugin.
how are you using Human, is it bundled version or non-bundled? if it's bundled, you most likely have two versions of tfjs and not sure if they match.
anyhow, i just pushed new version of Human, can you try setting backend to my custom version of webgl backend which has extra diagnostics in src/tfjs/backend.js?
(also updated tfjs to 2.8.3)
for example:
const userConfig = { backend: 'humangl' };

the log produced should look like:
09:25:55.719 Browser: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36 Edg/87.0.664.75
09:25:55.743 Human: version: 0.9.24 TensorFlow/JS version: 2.8.3
09:25:55.744 Human: setting backend: humangl
09:25:55.744 Human: backend registration: humangl
09:25:55.771 Human: backend registered: humangl

unfortunately, there is nothing strange in the flags.
and you didn't post full backtrace of error Error when getting WebGL context: Error: Cannot create a canvas in this context, only first two lines.
but i did notice one thing - backtrace is from tfjs-backend-webgl which is a module dependency of audio-ml-plugin.
how are you using Human, is it bundled version or non-bundled? if it's bundled, you most likely have two versions of tfjs and not sure if they match.
anyhow, i just pushed new version of Human, can you try setting backend to my custom version of webgl backend which has extra diagnostics in src/tfjs/backend.js?
(also updated tfjs to 2.8.3)
for example:
const userConfig = { backend: 'humangl' };

the log produced should look like:
09:25:55.719 Browser: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36 Edg/87.0.664.75
09:25:55.743 Human: version: 0.9.24 TensorFlow/JS version: 2.8.3
09:25:55.744 Human: setting backend: humangl
09:25:55.744 Human: backend registration: humangl
09:25:55.771 Human: backend registered: humangl

We have audio stuff that uses tfjs so I use the non bundled version and making everything is using the same tfjs version. I will try to do what you propose later. Will humangl have the same performance as webgl? What is the difference or should I use it only for the logs?

We have audio stuff that uses tfjs so I use the non bundled version and making everything is using the same tfjs version. I will try to do what you propose later. Will humangl have the same performance as webgl? What is the difference or should I use it only for the logs?

right now, humangl backend uses custom canvas to register backend, but actual operations are taken from webgl backend.
reason for having it is so I can have custom GL hooks into canvas and monitor & profile GL operations as they are executed which I can't if I can't reach canvas that is internally used by default webgl backend. If profiling is disabled (which it is by default), there is no functional difference between humangl and webgl.

right now, humangl backend uses custom canvas to register backend, but actual operations are taken from webgl backend.
reason for having it is so I can have custom GL hooks into canvas and monitor & profile GL operations as they are executed which I can't if I can't reach canvas that is internally used by default webgl backend. If profiling is disabled (which it is by default), there is no functional difference between humangl and webgl.

Cool I will use it. Do you want me to turn on profiling for debugging this thing?

Cool I will use it. Do you want me to turn on profiling for debugging this thing?

No, profiling will slow down your code 20x and not likely to show anything. It's useful for me when I'm optimizing models so I can see which operations are taking the most time.

No, profiling will slow down your code 20x and not likely to show anything. It's useful for me when I'm optimizing models so I can see which operations are taking the most time.

Firefox (didn't see any new info):
18:00:41.015 Human: version: 0.9.24 TensorFlow/JS version: 2.8.2 log.js:6
18:00:41.015 Human: setting backend: humangl log.js:6
18:00:41.015 Human: registering humangl backend
Error when getting WebGL context:  Error: Cannot create a canvas in this context 2 bundle.worker.js:40131:13
Initialization of backend webgl failed
MathBackendWebGL@http://localhost:3000/static/js/bundle.worker.js:15334:13
../audio-ml-plugin/node_modules/@tensorflow/tfjs-backend-webgl/dist/base.js/<@http://localhost:3000/static/js/bundle.worker.js:16354:96
initializeBackend@http://localhost:3000/static/js/bundle.worker.js:53586:44
initializeBackendsAndReturnBest@http://localhost:3000/static/js/bundle.worker.js:53682:16
get backend@http://localhost:3000/static/js/bundle.worker.js:53468:16
makeTensor@http://localhost:3000/static/js/bundle.worker.js:54059:5
zeros@http://localhost:3000/static/js/bundle.worker.js:89121:57
apply@http://localhost:3000/static/js/bundle.worker.js:116124:79
addWeight@http://localhost:3000/static/js/bundle.worker.js:109865:35
build@http://localhost:3000/static/js/bundle.worker.js:117423:24
apply/<@http://localhost:3000/static/js/bundle.worker.js:109586:14
nameScope@http://localhost:3000/static/js/bundle.worker.js:106537:17
apply@http://localhost:3000/static/js/bundle.worker.js:109571:69
add@http://localhost:3000/static/js/bundle.worker.js:125157:15
fromConfig@http://localhost:3000/static/js/bundle.worker.js:125693:13
deserializeKerasObject@http://localhost:3000/static/js/bundle.worker.js:126493:35
deserialize@http://localhost:3000/static/js/bundle.worker.js:123560:93
loadLayersModelFromIOHandler@http://localhost:3000/static/js/bundle.worker.js:124991:90

Firefox (didn't see any new info):
18:00:41.015 Human: version: 0.9.24 TensorFlow/JS version: 2.8.2 log.js:6
18:00:41.015 Human: setting backend: humangl log.js:6
18:00:41.015 Human: registering humangl backend
Error when getting WebGL context:  Error: Cannot create a canvas in this context 2 bundle.worker.js:40131:13
Initialization of backend webgl failed
MathBackendWebGL@http://localhost:3000/static/js/bundle.worker.js:15334:13
../audio-ml-plugin/node_modules/@tensorflow/tfjs-backend-webgl/dist/base.js/<@http://localhost:3000/static/js/bundle.worker.js:16354:96
initializeBackend@http://localhost:3000/static/js/bundle.worker.js:53586:44
initializeBackendsAndReturnBest@http://localhost:3000/static/js/bundle.worker.js:53682:16
get backend@http://localhost:3000/static/js/bundle.worker.js:53468:16
makeTensor@http://localhost:3000/static/js/bundle.worker.js:54059:5
zeros@http://localhost:3000/static/js/bundle.worker.js:89121:57
apply@http://localhost:3000/static/js/bundle.worker.js:116124:79
addWeight@http://localhost:3000/static/js/bundle.worker.js:109865:35
build@http://localhost:3000/static/js/bundle.worker.js:117423:24
apply/<@http://localhost:3000/static/js/bundle.worker.js:109586:14
nameScope@http://localhost:3000/static/js/bundle.worker.js:106537:17
apply@http://localhost:3000/static/js/bundle.worker.js:109571:69
add@http://localhost:3000/static/js/bundle.worker.js:125157:15
fromConfig@http://localhost:3000/static/js/bundle.worker.js:125693:13
deserializeKerasObject@http://localhost:3000/static/js/bundle.worker.js:126493:35
deserialize@http://localhost:3000/static/js/bundle.worker.js:123560:93
loadLayersModelFromIOHandler@http://localhost:3000/static/js/bundle.worker.js:124991:90

I guess you're using Human from NPM? I pushed update on Git, but didn't update actual NPM so this output is without diagnostics. I just uploaded new NPM, version 0.9.25.

I guess you're using Human from NPM? I pushed update on Git, but didn't update actual NPM so this output is without diagnostics. I just uploaded new NPM, version 0.9.25.

Thanks, I am afraid I wasted your time, the error comes even if I disable the human related code in the app and leave only the audio stuff, so its probably comes from there. Somehow when I see webgl I always thing about visual and forget it is used as a backend :) Do you still need me to run with the new version?

Thanks, I am afraid I wasted your time, the error comes even if I disable the human related code in the app and leave only the audio stuff, so its probably comes from there. Somehow when I see webgl I always thing about visual and forget it is used as a backend :) Do you still need me to run with the new version?

No worries.
And yes, it should still help to pinpoint the issue, even if it's not in Human*.

No worries.
And yes, it should still help to pinpoint the issue, even if it's not in Human*.

Ok thanks
18:30:45.279 Human: version: 0.9.25 TensorFlow/JS version: 2.8.3 log.js:6
18:30:45.280 Human: setting backend: humangl log.js:6
18:30:45.281 Human: backend registration: humangl log.js:6
18:30:45.291 Human: backend registered: humangl
I don't see any new information. Anyway I always get (and got in the past) those warnings in the beginning (this is only a part of the list). Maybe some tfjs conflict?
Platform browser has already been set. Overwriting the platform with [object Object]. environment.ts:57
cpu backend was already registered. Reusing existing backend factory. engine.ts:238
The kernel '_FusedMatMul' for backend 'cpu' is already registered kernel_registry.ts:142
The kernel 'Abs' for backend 'cpu' is already registered kernel_registry.ts:142
The kernel 'Acos' for backend 'cpu' is already registered kernel_registry.ts:142
The kernel 'Acosh' for backend 'cpu' is already registered kernel_registry.ts:142
The kernel 'Add' for backend 'cpu' is already registered kernel_registry.ts:142
The kernel 'AddN' for backend 'cpu' is already registered kernel_registry.ts:142

Ok thanks
18:30:45.279 Human: version: 0.9.25 TensorFlow/JS version: 2.8.3 log.js:6
18:30:45.280 Human: setting backend: humangl log.js:6
18:30:45.281 Human: backend registration: humangl log.js:6
18:30:45.291 Human: backend registered: humangl
I don't see any new information. Anyway I always get (and got in the past) those warnings in the beginning (this is only a part of the list). Maybe some tfjs conflict?
Platform browser has already been set. Overwriting the platform with [object Object]. environment.ts:57
cpu backend was already registered. Reusing existing backend factory. engine.ts:238
The kernel '_FusedMatMul' for backend 'cpu' is already registered kernel_registry.ts:142
The kernel 'Abs' for backend 'cpu' is already registered kernel_registry.ts:142
The kernel 'Acos' for backend 'cpu' is already registered kernel_registry.ts:142
The kernel 'Acosh' for backend 'cpu' is already registered kernel_registry.ts:142
The kernel 'Add' for backend 'cpu' is already registered kernel_registry.ts:142
The kernel 'AddN' for backend 'cpu' is already registered kernel_registry.ts:142

it doesn't pinpoint the problem exactly, but does provide some info.
first of all, there are no errors between: Human: backend registration: humangl and Human: backend registered: humangl. which performs all canvas creation and registration operations like webgl normally would - it all works just fine on the main thread.
actual warnings about already registered means there is more than one instance of each backend that tries to register itself. and you get them for cpu backend - so it's multiple copies of tfjs-core (or tfjs metapackage), not just tfjs-backend-webgl that's loading twice.
actual error Error when getting WebGL context: Error: Cannot create a canvas in this context comes from inside tfjs code during initialization of webgl backend:
if (typeof OffscreenCanvas !== 'undefined' && webGLVersion === 2) {
  return new OffscreenCanvas(300, 150);
} else if (typeof document !== 'undefined') {
  return document.createElement('canvas');
} else {
  throw new Error('Cannot create a canvas in this context');
}

which pretty much can only fail if executed within a web worker as Firefox doesn't supprot OffscreenCanvas and DOM is only undefined inside web worker (or NodeJS that doesn't have DOM to start with, but that's a totally offtopic).
you said earlier you're not using web workers yourself - but it seems that audio-ml-plugin is using them and it tries to initialize backend inside a worker instead of just using it (which would work). But since backend is already initialized, error becomes non-fatal since worker can use it regardless of the initialization failure.
hope this helps...

it doesn't pinpoint the problem exactly, but does provide some info.
first of all, there are no errors between: Human: backend registration: humangl and Human: backend registered: humangl. which performs all canvas creation and registration operations like webgl normally would - it all works just fine on the main thread.
actual warnings about already registered means there is more than one instance of each backend that tries to register itself. and you get them for cpu backend - so it's multiple copies of tfjs-core (or tfjs metapackage), not just tfjs-backend-webgl that's loading twice.
actual error Error when getting WebGL context: Error: Cannot create a canvas in this context comes from inside tfjs code during initialization of webgl backend:
if (typeof OffscreenCanvas !== 'undefined' && webGLVersion === 2) {
  return new OffscreenCanvas(300, 150);
} else if (typeof document !== 'undefined') {
  return document.createElement('canvas');
} else {
  throw new Error('Cannot create a canvas in this context');
}

which pretty much can only fail if executed within a web worker as Firefox doesn't supprot OffscreenCanvas and DOM is only undefined inside web worker (or NodeJS that doesn't have DOM to start with, but that's a totally offtopic).
you said earlier you're not using web workers yourself - but it seems that audio-ml-plugin is using them and it tries to initialize backend inside a worker instead of just using it (which would work). But since backend is already initialized, error becomes non-fatal since worker can use it regardless of the initialization failure.
hope this helps...

Thanks!
I said I don't use Web Worker for human because of the cons but for audio we do you web worker. I heard about Firefox limitations but didn't think it will be a problem with audio but I guess the model uses spectrograms which are images so it needs a canvas.
Not sure what it means to initialize a backend since this is the code that runs on the worker to initialize the plugin:
import * as tf from '@tensorflow/tfjs';
loadModel: async () => {
model = await tf.loadLayersModel('../../models/audio/model.json');
const arr = [makeArray(INPUT_SHAPE[1], INPUT_SHAPE[0], 0)];
const input = tf.tensor(arr, [1, ...INPUT_SHAPE]);
model.predict(input);
return NUM_FRAMES;
},
No sure what to do and if the audio stuff is using webgl or some fallback

Thanks!
I said I don't use Web Worker for human because of the cons but for audio we do you web worker. I heard about Firefox limitations but didn't think it will be a problem with audio but I guess the model uses spectrograms which are images so it needs a canvas.
Not sure what it means to initialize a backend since this is the code that runs on the worker to initialize the plugin:
import * as tf from '@tensorflow/tfjs';
loadModel: async () => {
model = await tf.loadLayersModel('../../models/audio/model.json');
const arr = [makeArray(INPUT_SHAPE[1], INPUT_SHAPE[0], 0)];
const input = tf.tensor(arr, [1, ...INPUT_SHAPE]);
model.predict(input);
return NUM_FRAMES;
},
No sure what to do and if the audio stuff is using webgl or some fallback

Canvas is not used just for graphics.  How it works is that tfjs creates tiny canvas just to get WebGL context to it - without context, there is no way to upload shaders and commands to GPU (that's one of the reasons why I've created humangl so I can control creation of that canvas and context and re-use it as I want and not rely on internal creation inside tfjs).
And initialization is implicit if you're using tfjs metapackage as import * as tf from '@tensorflow/tfjs';
What does initialization to? Checks capabilities and registers actual operations. Since they are already registered, nothing bad happens (regardless if error is 'already registered' or `cannot create context).

Canvas is not used just for graphics.  How it works is that tfjs creates tiny canvas just to get WebGL context to it - without context, there is no way to upload shaders and commands to GPU (that's one of the reasons why I've created humangl so I can control creation of that canvas and context and re-use it as I want and not rely on internal creation inside tfjs).
And initialization is implicit if you're using tfjs metapackage as import * as tf from '@tensorflow/tfjs';
What does initialization to? Checks capabilities and registers actual operations. Since they are already registered, nothing bad happens (regardless if error is 'already registered' or `cannot create context).

Ok thanks for your help!

Ok thanks for your help!


Added Taskqueue sample code.

Code snippets for:
https://cloud.google.com/appengine/docs/php/taskqueue/overview-push

LGTM.

Thanks!


boTTT issue

Had an issue where using the .vab command made boTTT disconnect from the server entirely. http://prntscr.com/640qeq here is a screenshot of it. I had checked in multiple rooms, and boTTT had left all of them. I tried .vab in both THP and in Studio, both wielded the same results.

That sounds like a bug with .vab. boTTT's not disconnecting; it's crasihng. I'll take a look into it.

I could not get this to trigger in either of the rooms you mentioned.

It was more like a temporary issue, but it only affected .vab not .vbw which was a very weird behaviour

Forgot about this issue. The bot would crash if it could not connect to Hastebin, which I'm guessing is what was happening here. This was solved in ea2bece4a56139b18ae19f6e7943669ebd4de0b1


rpc: use with_ prefix for AuthServerConfig setter methods

since this is a builder type only we don't necessarily need this prefix, I usually only add with_ for types that kinda are its own builder

@mattsse aaah ok I did that because by reading the code I found a with_ prefix here in similar functions
https://github.com/paradigmxyz/reth/blob/a908f977736ebd0d609fc3d2f4a6904a14f0bd73/crates/rpc/rpc-builder/src/auth.rs#L135
https://github.com/paradigmxyz/reth/blob/a908f977736ebd0d609fc3d2f4a6904a14f0bd73/crates/rpc/rpc-builder/src/auth.rs#L149
Is it normal then or should we remove the prefix here?


从右往左选择字符串的时候，会额外选择到右边的引号

I can't reproduce that, probably it's made by other extensions?

I can't reproduce that, probably it's made by other extensions?

I'm not sure, but after I upgraded to v0.0.5, the problem didn't reappear, maybe some commit fixed it


Command Denied When Running takeoff_and_land

Hello,
I am trying to run takeoff_and_land from my windows computer to get my px4 drone to takeoff and land. It is connected via the telemetry radio to the PC. The drone prints the following
-- Connected to drone!
-- Arming
mavsdk.action.ActionError: COMMAND_DENIED: 'Command Denied'; origin: arm(); params: ()
So the drone connects but it will not arm with the code. Not sure what is happening.
I eventually want to control the drone via a joystick connected to my PC. I am not sure what is going on. Can anyone help. In Q Ground Control it arms when I use the virtual joystick but not the physical joystick with the RC mode set to joystick.
Thanks.

Try to get some more log information using this change: https://github.com/mavlink/MAVSDK-Python/pull/370#pullrequestreview-729712188
Hopefully that will tell you why it doesn't arm.
Note to self: we should add docs how to do that.

There was no follow up, closing.
And here is a PR that should help with this: https://github.com/mavlink/MAVSDK-Python/pull/467.


Add .mvn/jvm.config

On Mac with various JDKs I could not do mvn compile. Ultimately found solution to have .mvn/jvm.config with following contents:
--add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.model=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.processing=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED
--add-opens jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED
--add-opens jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED

Please include this in the repository

Can you give some more explanation on when this happens (ie which JDKs, what part of build process)
Normally anything com.sun.* should be hidden from the users, and I have not seen the same issue so I am reluctant to add this to the repo for all users...

I am using openjdk@21 installed using brew. Here is the complete output on Mac laptop:
% mvn clean generate-resources compile assembly:single
[INFO] Scanning for projects...
[INFO]
[INFO] ------------< com.google.cloud.solutions:spanner-ddl-diff >-------------
[INFO] Building spanner-ddl-diff 1.0
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- clean:3.3.2:clean (default-clean) @ spanner-ddl-diff ---
[INFO] Deleting /Users/gurminder/spanner-schema-diff-tool/target
[INFO]
[INFO] --- exec:3.2.0:exec (jjt-2-concat) @ spanner-ddl-diff ---
[INFO]
[INFO] --- ph-javacc:4.1.5:jjtree-javacc (jjt-2-compile) @ spanner-ddl-diff ---
ParserGeneratorCC Version 1.1.4 (Tree Builder)
(type "jjtree" with no arguments for help)
Warning: Bad option value in "-JDK_VERSION=8" will be ignored.
Reading from file /Users/gurminder/spanner-schema-diff-tool/target/generated-sources/jjtree-src/DdlParser.jjt . . .
Warning: Bad option name "STATIC".  Option setting will be ignored.
opt:java
File "Node.java" does not exist.  Will create one.
File "SimpleNode.java" does not exist.  Will create one.
File "ASTany_reserved_word.java" does not exist.  Will create one.
File "ASTany_string_literal.java" does not exist.  Will create one.
File "ASTany_bytes_literal.java" does not exist.  Will create one.
File "ASTexpression.java" does not exist.  Will create one.
File "ASTgooglesql_operator.java" does not exist.  Will create one.
File "ASTgooglesql_punctuation_no_paren_without_dot.java" does not exist.  Will create one.
File "ASTgooglesql_punctuation_no_paren.java" does not exist.  Will create one.
File "ASTgooglesql_statement_token_no_paren_without_dot.java" does not exist.  Will create one.
File "ASTgooglesql_statement_token_no_paren.java" does not exist.  Will create one.
File "ASTpart.java" does not exist.  Will create one.
File "ASTidentifier_list.java" does not exist.  Will create one.
File "ASTidentifier.java" does not exist.  Will create one.
File "ASTqualified_identifier_list.java" does not exist.  Will create one.
File "ASTpath.java" does not exist.  Will create one.
File "ASTint_value.java" does not exist.  Will create one.
File "ASTstring_value.java" does not exist.  Will create one.
File "ASTddl_statement.java" does not exist.  Will create one.
File "ASTcreate_or_replace_statement.java" does not exist.  Will create one.
File "ASTalter_index_statement.java" does not exist.  Will create one.
File "ASTname.java" does not exist.  Will create one.
File "ASTadd.java" does not exist.  Will create one.
File "ASTdrop.java" does not exist.  Will create one.
File "ASTcolumn_name.java" does not exist.  Will create one.
File "ASToptions_clause.java" does not exist.  Will create one.
File "ASTcreate_database_statement.java" does not exist.  Will create one.
File "ASTdb_name.java" does not exist.  Will create one.
File "ASTcreate_table_statement.java" does not exist.  Will create one.
File "ASTcolumn_def.java" does not exist.  Will create one.
File "ASTnot_null.java" does not exist.  Will create one.
File "ASThidden.java" does not exist.  Will create one.
File "ASTcolumn_def_alter_attrs.java" does not exist.  Will create one.
File "ASTcolumn_def_alter.java" does not exist.  Will create one.
File "ASTset_not_null.java" does not exist.  Will create one.
File "ASTdrop_column_default.java" does not exist.  Will create one.
File "ASTdrop_not_null.java" does not exist.  Will create one.
File "ASTcolumn_type.java" does not exist.  Will create one.
File "ASTlength.java" does not exist.  Will create one.
File "ASTpgtype.java" does not exist.  Will create one.
File "ASTstruct_fields.java" does not exist.  Will create one.
File "ASTstruct_field.java" does not exist.  Will create one.
File "ASTcolumn_length.java" does not exist.  Will create one.
File "ASTgeneration_clause.java" does not exist.  Will create one.
File "ASTstored.java" does not exist.  Will create one.
File "ASTcolumn_default_clause.java" does not exist.  Will create one.
File "ASTcolumn_default_expression.java" does not exist.  Will create one.
File "ASTprimary_key.java" does not exist.  Will create one.
File "ASTforeign_key.java" does not exist.  Will create one.
File "ASTconstraint_name.java" does not exist.  Will create one.
File "ASTreferencing_columns.java" does not exist.  Will create one.
File "ASTreferenced_table.java" does not exist.  Will create one.
File "ASTreferenced_columns.java" does not exist.  Will create one.
File "ASTon_delete.java" does not exist.  Will create one.
File "ASTreferential_action.java" does not exist.  Will create one.
File "ASTno_action.java" does not exist.  Will create one.
File "ASTcascade.java" does not exist.  Will create one.
File "ASTstatement_token_no_paren.java" does not exist.  Will create one.
File "ASTcheck_constraint.java" does not exist.  Will create one.
File "ASTcheck_constraint_expression.java" does not exist.  Will create one.
File "ASTtable_interleave_clause.java" does not exist.  Will create one.
File "ASTinterleave_in.java" does not exist.  Will create one.
File "ASTrow_deletion_policy_clause.java" does not exist.  Will create one.
File "ASTrow_deletion_policy_expression.java" does not exist.  Will create one.
File "ASTrow_deletion_policy_function.java" does not exist.  Will create one.
File "ASTrow_deletion_policy_column.java" does not exist.  Will create one.
File "ASTinterval_expression.java" does not exist.  Will create one.
File "ASTon_delete_clause.java" does not exist.  Will create one.
File "ASTcreate_index_statement.java" does not exist.  Will create one.
File "ASTunique_index.java" does not exist.  Will create one.
File "ASTnull_filtered.java" does not exist.  Will create one.
File "ASTtable.java" does not exist.  Will create one.
File "ASTcolumns.java" does not exist.  Will create one.
File "ASTindex_interleave_clause.java" does not exist.  Will create one.
File "ASTcreate_change_stream_statement.java" does not exist.  Will create one.
File "ASTchange_stream_for_clause.java" does not exist.  Will create one.
File "ASTall.java" does not exist.  Will create one.
File "ASTchange_stream_tracked_tables.java" does not exist.  Will create one.
File "ASTchange_stream_tracked_tables_entry.java" does not exist.  Will create one.
File "ASTexplicit_columns.java" does not exist.  Will create one.
File "ASTcolumn.java" does not exist.  Will create one.
File "ASTcreate_role_statement.java" does not exist.  Will create one.
File "ASTprivilege.java" does not exist.  Will create one.
File "ASTprivileges.java" does not exist.  Will create one.
File "ASTtarget_type.java" does not exist.  Will create one.
File "ASTtable_function.java" does not exist.  Will create one.
File "ASTfunction.java" does not exist.  Will create one.
File "ASTchange_stream.java" does not exist.  Will create one.
File "ASTview.java" does not exist.  Will create one.
File "ASTsequence.java" does not exist.  Will create one.
File "ASTprivilege_target.java" does not exist.  Will create one.
File "ASTgrantees.java" does not exist.  Will create one.
File "ASTgrant_statement.java" does not exist.  Will create one.
File "ASTrevoke_statement.java" does not exist.  Will create one.
File "ASTcreate_sequence_statement.java" does not exist.  Will create one.
File "ASTkey_part.java" does not exist.  Will create one.
File "ASTdesc.java" does not exist.  Will create one.
File "ASTstored_column.java" does not exist.  Will create one.
File "ASTstored_column_list.java" does not exist.  Will create one.
File "ASToption_key_val.java" does not exist.  Will create one.
File "ASTkey.java" does not exist.  Will create one.
File "ASTnulll.java" does not exist.  Will create one.
File "ASTbool_true_val.java" does not exist.  Will create one.
File "ASTbool_false_val.java" does not exist.  Will create one.
File "ASTinteger_val.java" does not exist.  Will create one.
File "ASTstr_val.java" does not exist.  Will create one.
File "ASTstr_val_list.java" does not exist.  Will create one.
File "ASTor_replace.java" does not exist.  Will create one.
File "ASTsql_security.java" does not exist.  Will create one.
File "ASTcreate_view_statement.java" does not exist.  Will create one.
File "ASTview_definition.java" does not exist.  Will create one.
File "ASTstatement_tokens.java" does not exist.  Will create one.
File "ASTif_not_exists.java" does not exist.  Will create one.
File "ASTif_exists.java" does not exist.  Will create one.
File "ASTcreate_schema_statement.java" does not exist.  Will create one.
File "ASTcreate_model_statement.java" does not exist.  Will create one.
File "ASTinput.java" does not exist.  Will create one.
File "ASToutput.java" does not exist.  Will create one.
File "ASTremote.java" does not exist.  Will create one.
File "ASTdrop_statement.java" does not exist.  Will create one.
File "ASTindex.java" does not exist.  Will create one.
File "ASTrole.java" does not exist.  Will create one.
File "ASTmodel.java" does not exist.  Will create one.
File "ASTschema.java" does not exist.  Will create one.
File "ASTalter_database_statement.java" does not exist.  Will create one.
File "ASTalter_table_statement.java" does not exist.  Will create one.
File "ASTtable_name.java" does not exist.  Will create one.
File "ASTdrop_constraint.java" does not exist.  Will create one.
File "ASTdrop_row_deletion_policy.java" does not exist.  Will create one.
File "ASTdrop_column.java" does not exist.  Will create one.
File "ASTalter_column.java" does not exist.  Will create one.
File "ASTset_on_delete.java" does not exist.  Will create one.
File "ASTadd_row_deletion_policy.java" does not exist.  Will create one.
File "ASTadd_column.java" does not exist.  Will create one.
File "ASTreplace_row_deletion_policy.java" does not exist.  Will create one.
File "ASTalter_change_stream_statement.java" does not exist.  Will create one.
File "ASTdrop_for_all.java" does not exist.  Will create one.
File "ASTalter_sequence_statement.java" does not exist.  Will create one.
File "ASTalter_schema_statement.java" does not exist.  Will create one.
File "ASTalter_model_statement.java" does not exist.  Will create one.
File "ASTanalyze_statement.java" does not exist.  Will create one.
File "DdlParserTreeConstants.java" does not exist.  Will create one.
File "JJTDdlParserState.java" does not exist.  Will create one.
Annotated grammar generated successfully in /Users/gurminder/spanner-schema-diff-tool/target/javacc-1714486997275/node/DdlParser.jj
ParserGeneratorCC Version 1.1.4 (ParserGeneratorCC)
(type "pgcc" with no arguments for help)
Warning: Bad option value in "-JDK_VERSION=8" will be ignored.
Reading from file /Users/gurminder/spanner-schema-diff-tool/target/javacc-1714486997275/node/DdlParser.jj ...
Warning: Line 49, Column 3: Bad option name "STATIC".  Option setting will be ignored.
Note: UNICODE_INPUT option is specified. Please make sure you create the parser/lexer using a Reader with the correct character encoding.
Warning: Choice conflict in [...] construct at line 918, column 3.
         Expansion nested within construct and expansion following construct
         have common prefixes, one of which is: "bool"
         Consider using a lookahead of 2 or more for nested expansion.
File "TokenMgrException.java" does not exist.  Will create one.
File "ParseException.java" does not exist.  Will create one.
File "Token.java" does not exist.  Will create one.
File "CharStream.java" does not exist.  Will create one.
File "AbstractCharStream.java" does not exist.  Will create one.
File "SimpleCharStream.java" does not exist.  Will create one.
Parser generated with 0 errors and 2 warnings.
[INFO] Processed 1 grammar
[INFO]
[INFO] --- build-helper:3.5.0:add-source (add-source) @ spanner-ddl-diff ---
[INFO] Source directory: /Users/gurminder/spanner-schema-diff-tool/target/generated-sources/jjtree added.
[INFO]
[INFO] --- exec:3.2.0:exec (jjt-2-concat) @ spanner-ddl-diff ---
[INFO]
[INFO] --- ph-javacc:4.1.5:jjtree-javacc (jjt-2-compile) @ spanner-ddl-diff ---
ParserGeneratorCC Version 1.1.4 (Tree Builder)
(type "jjtree" with no arguments for help)
Warning: Bad option value in "-JDK_VERSION=8" will be ignored.
Reading from file /Users/gurminder/spanner-schema-diff-tool/target/generated-sources/jjtree-src/DdlParser.jjt . . .
Warning: Bad option name "STATIC".  Option setting will be ignored.
opt:java
File "Node.java" does not exist.  Will create one.
File "SimpleNode.java" does not exist.  Will create one.
File "ASTany_reserved_word.java" does not exist.  Will create one.
File "ASTany_string_literal.java" does not exist.  Will create one.
File "ASTany_bytes_literal.java" does not exist.  Will create one.
File "ASTexpression.java" does not exist.  Will create one.
File "ASTgooglesql_operator.java" does not exist.  Will create one.
File "ASTgooglesql_punctuation_no_paren_without_dot.java" does not exist.  Will create one.
File "ASTgooglesql_punctuation_no_paren.java" does not exist.  Will create one.
File "ASTgooglesql_statement_token_no_paren_without_dot.java" does not exist.  Will create one.
File "ASTgooglesql_statement_token_no_paren.java" does not exist.  Will create one.
File "ASTpart.java" does not exist.  Will create one.
File "ASTidentifier_list.java" does not exist.  Will create one.
File "ASTidentifier.java" does not exist.  Will create one.
File "ASTqualified_identifier_list.java" does not exist.  Will create one.
File "ASTpath.java" does not exist.  Will create one.
File "ASTint_value.java" does not exist.  Will create one.
File "ASTstring_value.java" does not exist.  Will create one.
File "ASTddl_statement.java" does not exist.  Will create one.
File "ASTcreate_or_replace_statement.java" does not exist.  Will create one.
File "ASTalter_index_statement.java" does not exist.  Will create one.
File "ASTname.java" does not exist.  Will create one.
File "ASTadd.java" does not exist.  Will create one.
File "ASTdrop.java" does not exist.  Will create one.
File "ASTcolumn_name.java" does not exist.  Will create one.
File "ASToptions_clause.java" does not exist.  Will create one.
File "ASTcreate_database_statement.java" does not exist.  Will create one.
File "ASTdb_name.java" does not exist.  Will create one.
File "ASTcreate_table_statement.java" does not exist.  Will create one.
File "ASTcolumn_def.java" does not exist.  Will create one.
File "ASTnot_null.java" does not exist.  Will create one.
File "ASThidden.java" does not exist.  Will create one.
File "ASTcolumn_def_alter_attrs.java" does not exist.  Will create one.
File "ASTcolumn_def_alter.java" does not exist.  Will create one.
File "ASTset_not_null.java" does not exist.  Will create one.
File "ASTdrop_column_default.java" does not exist.  Will create one.
File "ASTdrop_not_null.java" does not exist.  Will create one.
File "ASTcolumn_type.java" does not exist.  Will create one.
File "ASTlength.java" does not exist.  Will create one.
File "ASTpgtype.java" does not exist.  Will create one.
File "ASTstruct_fields.java" does not exist.  Will create one.
File "ASTstruct_field.java" does not exist.  Will create one.
File "ASTcolumn_length.java" does not exist.  Will create one.
File "ASTgeneration_clause.java" does not exist.  Will create one.
File "ASTstored.java" does not exist.  Will create one.
File "ASTcolumn_default_clause.java" does not exist.  Will create one.
File "ASTcolumn_default_expression.java" does not exist.  Will create one.
File "ASTprimary_key.java" does not exist.  Will create one.
File "ASTforeign_key.java" does not exist.  Will create one.
File "ASTconstraint_name.java" does not exist.  Will create one.
File "ASTreferencing_columns.java" does not exist.  Will create one.
File "ASTreferenced_table.java" does not exist.  Will create one.
File "ASTreferenced_columns.java" does not exist.  Will create one.
File "ASTon_delete.java" does not exist.  Will create one.
File "ASTreferential_action.java" does not exist.  Will create one.
File "ASTno_action.java" does not exist.  Will create one.
File "ASTcascade.java" does not exist.  Will create one.
File "ASTstatement_token_no_paren.java" does not exist.  Will create one.
File "ASTcheck_constraint.java" does not exist.  Will create one.
File "ASTcheck_constraint_expression.java" does not exist.  Will create one.
File "ASTtable_interleave_clause.java" does not exist.  Will create one.
File "ASTinterleave_in.java" does not exist.  Will create one.
File "ASTrow_deletion_policy_clause.java" does not exist.  Will create one.
File "ASTrow_deletion_policy_expression.java" does not exist.  Will create one.
File "ASTrow_deletion_policy_function.java" does not exist.  Will create one.
File "ASTrow_deletion_policy_column.java" does not exist.  Will create one.
File "ASTinterval_expression.java" does not exist.  Will create one.
File "ASTon_delete_clause.java" does not exist.  Will create one.
File "ASTcreate_index_statement.java" does not exist.  Will create one.
File "ASTunique_index.java" does not exist.  Will create one.
File "ASTnull_filtered.java" does not exist.  Will create one.
File "ASTtable.java" does not exist.  Will create one.
File "ASTcolumns.java" does not exist.  Will create one.
File "ASTindex_interleave_clause.java" does not exist.  Will create one.
File "ASTcreate_change_stream_statement.java" does not exist.  Will create one.
File "ASTchange_stream_for_clause.java" does not exist.  Will create one.
File "ASTall.java" does not exist.  Will create one.
File "ASTchange_stream_tracked_tables.java" does not exist.  Will create one.
File "ASTchange_stream_tracked_tables_entry.java" does not exist.  Will create one.
File "ASTexplicit_columns.java" does not exist.  Will create one.
File "ASTcolumn.java" does not exist.  Will create one.
File "ASTcreate_role_statement.java" does not exist.  Will create one.
File "ASTprivilege.java" does not exist.  Will create one.
File "ASTprivileges.java" does not exist.  Will create one.
File "ASTtarget_type.java" does not exist.  Will create one.
File "ASTtable_function.java" does not exist.  Will create one.
File "ASTfunction.java" does not exist.  Will create one.
File "ASTchange_stream.java" does not exist.  Will create one.
File "ASTview.java" does not exist.  Will create one.
File "ASTsequence.java" does not exist.  Will create one.
File "ASTprivilege_target.java" does not exist.  Will create one.
File "ASTgrantees.java" does not exist.  Will create one.
File "ASTgrant_statement.java" does not exist.  Will create one.
File "ASTrevoke_statement.java" does not exist.  Will create one.
File "ASTcreate_sequence_statement.java" does not exist.  Will create one.
File "ASTkey_part.java" does not exist.  Will create one.
File "ASTdesc.java" does not exist.  Will create one.
File "ASTstored_column.java" does not exist.  Will create one.
File "ASTstored_column_list.java" does not exist.  Will create one.
File "ASToption_key_val.java" does not exist.  Will create one.
File "ASTkey.java" does not exist.  Will create one.
File "ASTnulll.java" does not exist.  Will create one.
File "ASTbool_true_val.java" does not exist.  Will create one.
File "ASTbool_false_val.java" does not exist.  Will create one.
File "ASTinteger_val.java" does not exist.  Will create one.
File "ASTstr_val.java" does not exist.  Will create one.
File "ASTstr_val_list.java" does not exist.  Will create one.
File "ASTor_replace.java" does not exist.  Will create one.
File "ASTsql_security.java" does not exist.  Will create one.
File "ASTcreate_view_statement.java" does not exist.  Will create one.
File "ASTview_definition.java" does not exist.  Will create one.
File "ASTstatement_tokens.java" does not exist.  Will create one.
File "ASTif_not_exists.java" does not exist.  Will create one.
File "ASTif_exists.java" does not exist.  Will create one.
File "ASTcreate_schema_statement.java" does not exist.  Will create one.
File "ASTcreate_model_statement.java" does not exist.  Will create one.
File "ASTinput.java" does not exist.  Will create one.
File "ASToutput.java" does not exist.  Will create one.
File "ASTremote.java" does not exist.  Will create one.
File "ASTdrop_statement.java" does not exist.  Will create one.
File "ASTindex.java" does not exist.  Will create one.
File "ASTrole.java" does not exist.  Will create one.
File "ASTmodel.java" does not exist.  Will create one.
File "ASTschema.java" does not exist.  Will create one.
File "ASTalter_database_statement.java" does not exist.  Will create one.
File "ASTalter_table_statement.java" does not exist.  Will create one.
File "ASTtable_name.java" does not exist.  Will create one.
File "ASTdrop_constraint.java" does not exist.  Will create one.
File "ASTdrop_row_deletion_policy.java" does not exist.  Will create one.
File "ASTdrop_column.java" does not exist.  Will create one.
File "ASTalter_column.java" does not exist.  Will create one.
File "ASTset_on_delete.java" does not exist.  Will create one.
File "ASTadd_row_deletion_policy.java" does not exist.  Will create one.
File "ASTadd_column.java" does not exist.  Will create one.
File "ASTreplace_row_deletion_policy.java" does not exist.  Will create one.
File "ASTalter_change_stream_statement.java" does not exist.  Will create one.
File "ASTdrop_for_all.java" does not exist.  Will create one.
File "ASTalter_sequence_statement.java" does not exist.  Will create one.
File "ASTalter_schema_statement.java" does not exist.  Will create one.
File "ASTalter_model_statement.java" does not exist.  Will create one.
File "ASTanalyze_statement.java" does not exist.  Will create one.
File "DdlParserTreeConstants.java" does not exist.  Will create one.
File "JJTDdlParserState.java" does not exist.  Will create one.
Annotated grammar generated successfully in /Users/gurminder/spanner-schema-diff-tool/target/javacc-1714486997934/node/DdlParser.jj
ParserGeneratorCC Version 1.1.4 (ParserGeneratorCC)
(type "pgcc" with no arguments for help)
Warning: Bad option value in "-JDK_VERSION=8" will be ignored.
Reading from file /Users/gurminder/spanner-schema-diff-tool/target/javacc-1714486997934/node/DdlParser.jj ...
Warning: Line 49, Column 3: Bad option name "STATIC".  Option setting will be ignored.
Note: UNICODE_INPUT option is specified. Please make sure you create the parser/lexer using a Reader with the correct character encoding.
Warning: Choice conflict in [...] construct at line 918, column 3.
         Expansion nested within construct and expansion following construct
         have common prefixes, one of which is: "bool"
         Consider using a lookahead of 2 or more for nested expansion.
File "TokenMgrException.java" does not exist.  Will create one.
File "ParseException.java" does not exist.  Will create one.
File "Token.java" does not exist.  Will create one.
File "CharStream.java" does not exist.  Will create one.
File "AbstractCharStream.java" does not exist.  Will create one.
File "SimpleCharStream.java" does not exist.  Will create one.
Parser generated with 0 errors and 2 warnings.
[INFO] Processed 1 grammar
[INFO]
[INFO] --- build-helper:3.5.0:add-source (add-source) @ spanner-ddl-diff ---
[INFO] Source directory: /Users/gurminder/spanner-schema-diff-tool/target/generated-sources/jjtree added.
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ spanner-ddl-diff ---
[INFO] skip non existing resourceDirectory /Users/gurminder/spanner-schema-diff-tool/src/main/resources
[INFO]
[INFO] --- compiler:3.13.0:compile (default-compile) @ spanner-ddl-diff ---
[INFO] Recompiling the module because of changed source code.
[INFO] Compiling 158 source files with javac [debug target 8] to target/classes
An exception has occurred in the compiler (21.0.2). Please file a bug against the Java compiler via the Java bug reporting page (https://bugreport.java.com) after checking the Bug Database (https://bugs.java.com) for duplicates. Include your program, the following diagnostic, and the parameters passed to the Java compiler in your report. Thank you.
java.lang.IllegalAccessError: class com.google.errorprone.BaseErrorProneJavaCompiler (in unnamed module @0xf95d64d) cannot access class com.sun.tools.javac.api.BasicJavacTask (in module jdk.compiler) because module jdk.compiler does not export com.sun.tools.javac.api to unnamed module @0xf95d64d
	at com.google.errorprone.BaseErrorProneJavaCompiler.addTaskListener(BaseErrorProneJavaCompiler.java:92)
	at com.google.errorprone.ErrorProneJavacPlugin.init(ErrorProneJavacPlugin.java:34)
	at jdk.compiler/com.sun.tools.javac.api.BasicJavacTask.initPlugin(BasicJavacTask.java:256)
	at jdk.compiler/com.sun.tools.javac.api.BasicJavacTask.initPlugins(BasicJavacTask.java:230)
	at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.prepareCompiler(JavacTaskImpl.java:204)
	at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.lambda$doCall$0(JavacTaskImpl.java:101)
	at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.invocationHelper(JavacTaskImpl.java:152)
	at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:100)
	at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:94)
	at org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess(JavaxToolsCompiler.java:126)
	at org.codehaus.plexus.compiler.javac.JavacCompiler.performCompile(JavacCompiler.java:214)
	at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:1228)
	at org.apache.maven.plugin.compiler.CompilerMojo.execute(CompilerMojo.java:215)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:126)
	at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2(MojoExecutor.java:328)
	at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:316)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:174)
	at org.apache.maven.lifecycle.internal.MojoExecutor.access$000(MojoExecutor.java:75)
	at org.apache.maven.lifecycle.internal.MojoExecutor$1.run(MojoExecutor.java:162)
	at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute(DefaultMojosExecutionStrategy.java:39)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:159)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:105)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:73)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:53)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:118)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:261)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:173)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:101)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:906)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:283)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:206)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:283)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:226)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:407)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:348)
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] An unknown compilation problem occurred
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.922 s
[INFO] Finished at: 2024-04-30T07:23:18-07:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.13.0:compile (default-compile) on project spanner-ddl-diff: Compilation failure
[ERROR] An unknown compilation problem occurred
[ERROR]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Ok, so its ErrorProne causing the issue.
Can you remove or comment out lines 121 and 129-130 from pom.xml:
            <!-- <arg>-Xplugin:ErrorProne -XepExcludedPaths:${project.build.directory}/generated-sources/.* -XepDisableWarningsInGeneratedCode</arg> -->

...

            <!-- <path>
              <groupId>com.google.errorprone</groupId>
              <artifactId>error_prone_core</artifactId>
              <version>${error-prone.version}</version>
            </path> -->

And indeed, Errorprone docs mention the issue in Maven and JDK 16+

Can you check to see if the change in #100 fixes it for you.
(and yes I can now repro locally - while I had JDK 17 installed, JDK 11 was the version being used by default due to PATH shenanigans!)

(PS thanks for raising these issues!)

Pulled the latest code from main branch:

The main branch did not have the fix in at the time - it was in my PR -  because at the time I posted, I could not test the fix - but since then I was able to repro and verify that #100 fixes the issue.
now that #100 is merged, main should include the fix (by updating pom.xml with the javac args)

Verified that it is working


bacon_ls do nothing

Hello,
bacon works fine and it is running, but bacon_ls does nothing, no error log and it is attached:
Client: bacon_ls (id: 1, bufnr: [3])
.bacon-locations, bacon-ls.log and bacon.toml files are present, please help me, I do not want to return to rust-analizer.

Hello, I'd be very happy to help you debug this. I think I will need a little bit more info:

Do you have actual errors / warning in your code that bacon is picking up?
When it does, can you paste the content of your .bacon-locations?
What version of bacon are you running?
Can you share a crate I can access to where bacon-ls doesn't work for you so I can try to troubleshoot it?

Hello, I'd be very happy to help you debug this. I think I will need a little bit more info:

Do you have actual errors / warning in your code that bacon is picking up?
No errors neither warnings.
When it does, can you paste the content of your .bacon-locations?

[export]
enabled = true
path = ".bacon-locations"
line_format = "{kind}:{path}:{line}:{column}:{message}"

What version of bacon are you running?
bacon 3.1.1
Can you share a crate I can access to where bacon-ls doesn't work for you so I can try to troubleshoot it?

A full working example would be good, is there any way to watch logs? (bacon-ls.log is empty!)

Hello, I'd be very happy to help you debug this. I think I will need a little bit more info:

Do you have actual errors / warning in your code that bacon is picking up?

No errors neither warnings.

If there are no errors / warnings in your code, bacon-ls will not report anything to the LSP client. Any chance you can paste a screenshot of bacon properly working in your repository?

When it does, can you paste the content of your .bacon-locations?

[export] enabled = true path = ".bacon-locations" line_format = "{kind}:{path}:{line}:{column}:{message}"

What I mean is the content of the file .bacon-locations when it is populated by bacon, not the settings file.

What version of bacon are you running?

bacon 3.1.1

Can you share a crate I can access to where bacon-ls doesn't work for you so I can try to troubleshoot it?

No, and a full working example would be better, is there any way to watch logs? (bacon-ls.log is empty!)

Hello, as I say before bacon works well:

This is the .bacon-locations content:
(populated by bacon --export-locations)
warning /home/realtica/projects/roll-api/src/main.rs:4:5 unused import: dotenvy::dotenvwarning /home/realtica/projects/roll-api/src/main.rs:78:43 unused variable:site_idwarning /home/realtica/projects/roll-api/src/main.rs:80:9 variable does not need to be mutable warning /home/realtica/projects/roll-api/src/main.rs:141:5 fieldusernameis never read warning /home/realtica/projects/roll-api/src/main.rs:88:5 unusedResultthat must be used warning /home/realtica/projects/roll-api/src/main.rs:105:5 unused implementer ofFuturethat must be used warning /home/realtica/projects/roll-api/src/main.rs:113:5 unused implementer ofFuture that must be used

I don't understand bacon-ls depends of rust-analyzer? This don't any sense.

Rust analyzer is still required to do anything that is not error reporting. Things like code completion, snippets, refactoring, etc.
Can you paste how did you configure bacon-ls and rust analyzer in neovim? Do you have some dotfiles I can use to try to debug your issues?

I am working on a new version that will allow better troubleshooting of issues.

I'm encountering similar. .bacon-locations is never created.
I can introduce artificial issues into my code, and nothing. bacon-ls is running, and I've disabled checkOnSave and diagnostics in my rust-analyzer config.
bacon_ls Capabilities:

{
  diagnosticProvider = {
    interFileDependencies = false,
    workspaceDiagnostics = true
  },
  positionEncoding = "utf-16",
  textDocumentSync = {
    change = 1,
    openClose = true,
    save = {
      includeText = false
    },
    willSave = false,
    willSaveWaitUntil = false
  }
}

bacon_ls Config:

{}

Very curious why bacon-ls links against image libraries as well:
$ lsof -p 84595
COMMAND    PID   USER   FD   TYPE             DEVICE SIZE/OFF                NODE NAME
bacon-ls 84595 dsully  txt    REG               1,14  6338054             7977671 /Users/dsully/.cargo/bin/bacon-ls
bacon-ls 84595 dsully  txt    REG               1,14    72816             2228631 /opt/homebrew/Cellar/giflib/5.2.2/lib/libgif.7.2.0.dylib
bacon-ls 84595 dsully  txt    REG               1,14   185136             2227262 /opt/homebrew/Cellar/xz/5.6.3/lib/liblzma.5.dylib
bacon-ls 84595 dsully  txt    REG               1,14   209568             2215979 /opt/homebrew/Cellar/libpng/1.6.44/lib/libpng16.16.dylib
bacon-ls 84595 dsully  txt    REG               1,14   526512             2199371 /opt/homebrew/Cellar/libtiff/4.7.0/lib/libtiff.6.dylib
bacon-ls 84595 dsully  txt    REG               1,14   473216             2247028 /opt/homebrew/Cellar/jpeg-turbo/3.0.4/lib/libjpeg.8.3.2.dylib
bacon-ls 84595 dsully  txt    REG               1,14  2423888<PHONE_NUMBER>312498005 /usr/lib/dyld
bacon-ls 84595 dsully  txt    REG               1,14   669440             2238572 /opt/homebrew/Cellar/zstd/1.5.6/lib/libzstd.1.5.6.dylib
bacon-ls 84595 dsully    0u  unix 0xe79e30871654fd03      0t0                     ->0xc94d658203e9ee46
bacon-ls 84595 dsully    1u  unix 0xce38c75dbb583e60      0t0                     ->0xbb3a72b6f8bf2b54
bacon-ls 84595 dsully    2u  unix 0x2acb7fdf2c75c68c      0t0                     ->0x8b17029a0ae013ea


tsdb: [feature request]: detect and stop compacting blocks that result with index file size >64Gi

We're actually running into this on our largest cluster and it's brought thanos compaction to a standstill. I'm whittling away at cardinality, sharding our scrape targets where possible and backing down our retention policies.  Hopefully all of the above will do the trick in the near-term, but it would be great to not have to worry about this in the future.
https://github.com/prometheus/tsdb/blob/e7436e13f07511ee669876905f5147ae1713f5bd/index/index.go#L233

Fix for this is here https://github.com/prometheus/prometheus/pull/5884

During our bug scrub, we realized that #5884 was caught in review limbo for years now. We are wondering how much this feature is actually needed and if the benefits outweigh the complications. See here for a line of argument why we shouldn't do this.
We would like to get an idea about the relevance of the problem, so if you still run into this problem, feel free to comment on this issue.

My personal opinion:
Once you're big enough to hit this (outrageously high) limit, all mitigations are complicated or expensive. Scale is a multiplier of complexity. As you know, at this scale, the necessary rearchitecture to work around something like this is months of work. That's compounded by the fact that it's a hidden limitation - with no warning or info level logs suggesting that you may be approaching a limit - so it ends up saddling teams with a significant, unplanned lift to prevent critical, production-level impacts. That's, like, the trifecta of "uh oh."
Back in my day, we were lucky to have 100 MiB of storage, and my day wasn't that long ago. Arbitrarily deciding "that ought to be enough bytes for anybody" has been consistently proven to show lack of foresight. Maybe Prometheus stacks reaching 64GiB blocks are rare today, but will they be rare next year? The year after? If Prometheus continues to hold the spot as the industry leading platform, the number of teams impacted will increase rapidly as a result of two simple factors: "we're at scale, so we should be using the industry leader" + "our observability is working, let's do more of it."
This limit will be patched out at some point. It will have to be. I'd urge maintainers to opt for "sooner" rather than "later" given the severity and suddenness of impact. If you wait until some critical mass of users have hit this limit, you'll be too late, because nobody runs latest in prod. Better to have been fixed a couple major versions ago than need to bump a major and a dozen minors for remediation.

Looks like this is still in limbo.
We are getting hit by this limitation. Granted, we have some huge prometheus servers, however this issue happens without warning (like @rtksamsmith said), and when it happens, it's kinda too late.
The machines that are hitting this limitation are kinda big, 8TB of storage, 256G RAM, 2xXeon 6234 CPUs with HT (32 logical cores), 20M timeseries stored in the TSDB.
Can this be mitigated with lowering the retention ratio? (probably not) Other than that the TSDB doesn't seem to have too many tunables.

(Revisiting this during bug scrub)
Thanks for those new use cases. So fixing the index size limitation (due to some int32 used) is one way of solving this, but if we fix that (with intrusive and complex index change) we will like hit some other limit soon. One obvious solution which would help you is to shard Prometheus or use systems like Cortex, Thanos, Mimir etc, but obviously it takes time and effort to migrate.

Granted, we have some huge prometheus servers, however this issue happens without warning

This is good point, warning and self-healing would be nice in Prometheus.
And probably this is the quickest fix we can do in Prometheus code--detect this case and stop compacting those certain blocks that contribute to too big index further. This is what we did in Thanos (https://github.com/thanos-io/thanos/pull/3410) and it worked well there. Most likely easier than the new index format.
We also notice this issue was created for Thanos use case specifically which has been fixed since then (as I mentioned above). Since we saw a few users @nixfloyd @rtksamsmith hitting this on Prometheus, let's convert this issue title to "detect and stop compacting too big blocks further". WDYT?

Acceptance Criteria:

Port similar behaviour of https://github.com/thanos-io/thanos/pull/3410 to Prometheus compaction

20M timeseries stored in the TSDB

I've seen much bigger, without hitting this limit.  So I suspect you hit it when Prometheus was trying to compact smaller blocks into larger ones - by default it goes bigger and bigger until it hits 31 days or 10% of the retention period.  You can configure this with --storage.tsdb.max-block-duration

@bboreham

I've seen much bigger, without hitting this limit

Sorry, I used the wrong term, we hit the issues when we got above 22-25M in prometheus_tsdb_head_series
In the TSDB I assume we have well over 400M timeseries (based on prometheus_tsdb_head_series_created_total since the last restart)
We moved some target groups away to a different prometheus pair (with faster hardware as well) and haven't noticed this issue since.


Add pyarrow to conda install in https://docs.dask.org/en/latest/develop.html

I tried following https://docs.dask.org/en/latest/develop.html and ran the tests but get failures as I need pyarrow which I conda installed but this should be added in this page?

@joy-rosie thanks for raising this issue! All tests that require pyarrow should be skipped if pyarrow is not installed. I've opened up #5217 to fix this issue

@joy-rosie with the updates in #5217, you should now no longer need pyarrow to be installed to run the tests suite. Is it safe to close this issue?

Yes thank you!

Great, thanks again for raising this issue


Fix #1505, accept "NULL" as entry point

Describe the contribution
Recognize the special string "NULL" to indicate no entry point should be called for the library.  Equivalent to leaving the
field empty.
Fixes #1505
Testing performed
Build and sanity check CFE, and use a startup script that loads a CFE_LIB with a NULL entry point
Run all unit tests.
Expected behavior changes
Specifying the special string NULL as the entry point in a startup script should work as it did in previous versions - skips the entry point.
System(s) tested on
Ubuntu
Contributor Info - All information REQUIRED for consideration of pull request
Joseph Hickey, Vantage Systems, Inc.

CCB:2021-05-19 APPROVED


first addition of _embedder.yaml

Configuring our analyzer world through a file.

cc @pq please take a look
fyi @abarth and @Hixie  for what we are doing. There will be many more options here, including lints. With this file, we can bundle and distribute our configuration of the analyzer.
also FYI @johnmccutchan

LGTM

Neat!  LGTM

Thanks!


Create Google ngrams chart for "artificial intelligence" in intro?

I noticed that in the introduction that there are several references to a google N-grams chart for "artificial intelligence". Has this chart been generated? If not, I am happy to generate clean version of it and provide appropriate raw data / documentation as suggested in the figs section.
For reference, here is the raw version from Google that I would plan to replicate (with better labels & in the book's style):

If this sounds good, I will go ahead and generate a clean visual and open a PR. If there's anything else I should keep in mind, please let me know. Thanks.

Thank you for this!

@Fossj117 hey! just set up a pre-styled notebook for you to help with this if you are still able to.

@ccerv1 howdy sir! yes will do!


Deserializing model with custom metric

I've defined an F0.5 score:
def fhalf(ypred, ytrue):
    return fbeta_score(ypred, ytrue, beta=0.5)

which I supply in model.compile and train before saving as an h5:
    model.compile(optimizer=SGD(lr=0.01, momentum=0.95, nesterov=True),
                  loss='categorical_crossentropy',
                  # TODO: f_beta_score broken during deserialization?
                  metrics=[fhalf, 'precision', 'recall'])

    print("Training model...")
    early_stopping = EarlyStopping(monitor='val_fhalf', patience=32)
    reduce_lr = ReduceLROnPlateau(monitor='val_fhalf', factor=0.25, patience=4, min_lr=1e-7)
    tensor_board = TensorBoard(log_dir=thoth_home + '/logs', histogram_freq=1)
    model.fit([org_as, org_bs, xs], labels, class_weight={1: 4, 0: 1, 2: 2, 3: 2},
              batch_size=128, nb_epoch=1000,
              validation_split=0.03, callbacks=[early_stopping, reduce_lr, tensor_board])

    print("Saving model...")
    model.save(thoth_home + "/models/org_comparator_keras.h5")

and the trained model successfully evalutes and computes F0.5 on a validation data set.
However, when I attempt to load the model with model = load_model(thoth_home + "models/org_comparator_keras.h5"), I get the following error:
  File "/Users/ben/workspace/diffbot/Thoth2/bin/keras/server.py", line 11, in <module>
    class KerasServer(BaseHTTPRequestHandler):
  File "/Users/ben/workspace/diffbot/Thoth2/bin/keras/server.py", line 21, in KerasServer
    model = load_model(thoth_home + "models/org_comparator_keras.h5")
  File "/usr/local/lib/python2.7/site-packages/keras/models.py", line 170, in load_model
    sample_weight_mode=sample_weight_mode)
  File "/usr/local/lib/python2.7/site-packages/keras/engine/training.py", line 715, in compile
    metric_fn = metrics_module.get(metric)
  File "/usr/local/lib/python2.7/site-packages/keras/metrics.py", line 184, in get
    return get_from_module(identifier, globals(), 'metric')
  File "/usr/local/lib/python2.7/site-packages/keras/utils/generic_utils.py", line 125, in get_from_module
    str(identifier))
ValueError: Invalid metric: fhalf

The error sounds as if it is trying to invoke the custom metric instance by its string name to some mapping provided in metric.py, but complains when it doesn't exist. Is there a way for me to make use of the custom metric without modifying metric.py to include it?

model = load_model(thoth_home + "models/org_comparator_keras.h5", custom_objects={'fhalf': fhalf})

Thanks @patyork


⚠ Bofrost.es has degraded performance

In 9a8c0e1, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1449 ms

Resolved: Bofrost.es performance has improved in 5f36de1 after 7 minutes.


Integration Test - Release - 2020/08/12 17:51:02

Auto create for integration test. AnotherAuthorUpdatedArticleFlow

@PRMergerTest3 : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Can't use this with Node v22

We're using wagmi which uses this library under the hood.
It looks like this new release has broken our build since this package is now locked in Node v22.
 ERR_PNPM_UNSUPPORTED_ENGINE  Unsupported environment (bad pnpm and/or Node.js version)

This error happened while installing the dependencies of<EMAIL_ADDRESS> at<EMAIL_ADDRESS> at<EMAIL_ADDRESS> at<EMAIL_ADDRESS>
Your Node version is incompatible with<EMAIL_ADDRESS>
Expected version: 20.x
Got: v22.5.1

This is happening because the package's manifest has an engines.node field specified.
To fix this issue, install the required Node version.

Thanks @maximebonhomme, I am looking into it 👀

@maximebonhomme you can find fix in v1.7.2


Update iOS Client to show Route time predictions

@bikeoid has completed the route predictions on the server and is currently responding to polling with an array of trolleys and time to the current stop.
Implement once a user selects a route, show them the ETA from the nearest trolley

To consider: is this something that should be subscribed to (via WebSocket or equivalent) and to be updated when the server recalculates the time (1/minute),  or is this something that will be polled as the user pops open the app to check or recheck arrival time?


NavigationView: Fix footer items scrollbar being visible without footer items to scroll

Description
This PR fixes a NavigationView issue where the scrollbar for the footer items would be visible even if there are no footer items to scroll.
Motivation and Context
Fixes https://github.com/microsoft/microsoft-ui-xaml/issues/3771.
How Has This Been Tested?
Tested manually.

Before
After

/azp run

Thanks @Felix-Dev. Could you update the v2.5 styles as well ?

Closed in https://github.com/microsoft/microsoft-ui-xaml/pull/4193


Form Checkbox: Switch does not properly slide right on checked

I am trying to use <b-form-checkbox switch>. However, when I click on switch, it stops sliding right on the middle. See image here
Here is my usage:
<b-form-checkbox switch @change="change(b)" :disabled="isDisabled(b)" size="lg" v-model="b.check"></b-form-checkbox>
Everything works properly except css part.
I am using yarn dev, i.e. vite to run program on localhost.
Here is my package.json for versions of the dependencies.
"dependencies": {
    "@dafcoe/vue-collapsible-panel": "^0.2.0",
    "@popperjs/core": "^2.11.5",
    "@tailwindcss/aspect-ratio": "^0.2.0",
    "@tailwindcss/forms": "^0.2.1",
    "@tailwindcss/typography": "^0.4.0",
    "@tailwindcss/ui": "^0.7.2",
    "bootstrap": "^5.1.3",
    "bootstrap-vue-3": "^0.1.10",
    "vue": "^3.2.32",
    "vue-router": "4",
    "yarn": "^1.22.18"
  }

Problem might be related to using tailwindcss together with bootstrap but I am not sure and I cannot find anything similar.

Works fine in current version. Please provide a minimal reproduction, excluding tailwindcss

<template>
  <b-form-checkbox
    v-model="check"
    :value="1"
    :unchecked-value="0"
    switch
  >
    switchTest
  </b-form-checkbox>
</template>
<script setup lang="ts">
const check = ref(0)
</script>

@mariusa
here is my minimal reproduction, when switch to true,  switch won't slide to right

ok, making a computed value that returns based on the vmodel value. can be help.
i will open a new issue for this problem.
thanks for reply


Multiple drop targets

I'm working on a project that I think Slip would be perfect for, except I'm really looking for the ability to use multiple lists and to move items between them. Any suggestions on how to go about accomplishing that using Slip?

@y-lohse Did you ever get round to porting your code over to the latest release?

Nope, sorry! I don't have the code anymore either.


Warnings for apply_numpy_specials depend on order

Using pysis 0.5.2
I'm having an issue with Cubefile.apply_numpy_specials.
Apparently the order matters for receiving a warning or not.
Possibly, because your hardcoded Min/Max values don't fit to what I find in this Cassini ISS image?
See screenshot:

My RingCube is just a wrapper around your CubeFile for Saturn Ring images, doesn't do anything with your CubeFile inner stuff.

I believe the warning is because it is running the comparisons on the special values. This shouldn't cause any problems however it does proabally mean there is a better way to do this.

Where did you get those special values from? Are they defined in a standard document somewhere? Possibly either those values need to be updated, or the ISIS routine creating those numbers in the cube, I guess one of them is not consistent with current standards/expectations.

They're the numpy special values that replace the cube file special values like np.NINF, np.inf and np.nan.

I think this is only happening because numpy.less does not like to see the np.nans from the 'Null' replacement.
As I wrote above, if we do the 'Null' to np.nan replacement at the end, the warning goes away.
Do you see a problem with doing the replacement of Null->np.nan at the end? If not, I can just submit a simple PR to change the order of this.

Ok, no, obviously I need to set NANs first, because the minimum is also a negative special pixel value.
But because we know that this operation is working, one should suppress the numpy warning in this case, using np.errstate(invalid='ignore') for those 2 operations. I'll send in a PR.


bootstrap 4.0.0

Hi!
After update the bootstrap to 4.0.0 version, the markup on the "serenity" demo has broken.
normal
broken
The "Serenity" is latest version (3.5.2)

It's not supported


The latest version does not support compression

No response to click compression in the latest version

Latest build supports bundle compression now.

How to use complress?  When i click on menu , i didn't get anything.


Waifu Log Watcher

Describe the user story
As a developer, I want my waifu to watch the logs for me, and alert me whenever a phrase shows up on my logs. This would be useful whenever an application starts slow and just waiting for "Started" or building a project and waiting for "Build successful" or can use it with debugging and look for "beep boop"
Describe the solution you'd like
Just a simple solution to watch the logs printed, and show an alert whenever it matches my list of phrases.

Blocked by: #273


Import error when following governance level 4 training

When creating a new project from resume_summarization_project.zip error importing Resume Summarization prompt template, see screenshot.

When creating the prompt template manually using the data in the JSON file and then exporting I see that it has double backslashes in the example text like "Nerissa G. McCloud-Pearcy\\n(205<EMAIL_ADDRESS>Magnolia Drive West, Birmingham, instead of singles like are in the provided .zip file.

I thought this might be a regional issue and changed that. I now have the following error log when creating the project: Failed to create an attachment. Status code: 404


Issues with Wink display on Front end.

Make sure you are running the latest version of Home Assistant before reporting an issue.
You should only file an issue if you found a bug. Feature and enhancement requests should go in the Feature Requests section of our community forum:
Home Assistant release (hass --version):
0.35.3
Python release (python3 --version):
Python 3.4.2
Component/platform:
Wink
Description of problem:
Front end display of wink devices is unreliable.  Sometimes the wink devices on my front end will not update properly, and show off when on, or unavailable when they are all working fine in the wink app.
Expected:
Wink devices update as desired per the state of the devices.
Problem-relevant configuration.yaml entries and steps to reproduce:
wink:
  email: <EMAIL ADDRESS>
  password: <PASSWORD>
  client_id: <CLIENT_ID>
  client_secret: <CLIENT_SECRET>

@smaggard We are discussing this over here https://community.home-assistant.io/t/ha-stops-reporting-wink-state-changes/5884 if you could provide some details on that post with what devices you are using, and also look to see if you are seeing the following in your logs.
"pubnub: take message interrupted: 'NoneType' object has no attribute 'loop'"
Also take a look at my https://community.home-assistant.io/t/ha-stops-reporting-wink-state-changes/5884/82 if you have the ability to run dev and test out the change I suggest in that post it would be helpful.

Installing the dev branch, I will reply as top whether it helps or not.

On Dec 31, 2016, at 11:49 AM, William Scanlon<EMAIL_ADDRESS>wrote:
@smaggard https://github.com/smaggard We are discussing this over here https://community.home-assistant.io/t/ha-stops-reporting-wink-state-changes/5884 https://community.home-assistant.io/t/ha-stops-reporting-wink-state-changes/5884 if you could provide some details on that post with what devices you are using, and also look to see if you are seeing the following in your logs.
"pubnub: take message interrupted: 'NoneType' object has no attribute 'loop'"
Also take a look at my https://community.home-assistant.io/t/ha-stops-reporting-wink-state-changes/5884/82 https://community.home-assistant.io/t/ha-stops-reporting-wink-state-changes/5884/82 if you have the ability to run dev and test out the change I suggest in that post it would be helpful.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/home-assistant/home-assistant/issues/5116#issuecomment-269875396, or mute the thread https://github.com/notifications/unsubscribe-auth/AV_aD3VFaVKbB9istggH-WbrpLv0umEaks5rNpWngaJpZM4LYdfj.

@smaggard I think your issue should go away with the next release 0.37 please let me know if you run into any issues.


Fix type in decimal_avg and decimal_sum while spilling

We have been playing around with spill in partial aggregate when we noticed velox crashes when it read data back from spill in partial aggregate. This happens when input type is of ShortDecimal like decimal(12,4).
Both these agg function produces intermediate sum of LongDecimal irrespective of input. So when input is of LongDecimal it works but in case of ShortDecimal this respective dynamic type cast fails producing null pointer causing crash.

cc: @zhouyuan

cc @liujiayi771

@rui-mo For sum agg, this probloem has been fixed in upstream PR. But for avg agg, I think the intermediate type will always been int128_t, as discussed in https://github.com/facebookincubator/velox/pull/6020/files#r1302694957, it seems that TSumResultType in this modification will always be int128_t.

@mskapilks could you please help to review this patch? https://github.com/facebookincubator/velox/pull/6020
@rui-mo actually has a new branch based on meta/velox (https://github.com/oap-project/velox/tree/update) which will become the new "main" branch soon.
thanks, -yuan


Support to kube-router CNI

Rancher versions:
rancher/server or rancher/rancher: v2.x
Is it possible to add support to kube-router CNI?

Merging into https://github.com/rancher/rke/issues/312

Uhh this request was specifically for RANCHER to support kube-router, and for me, the distinction between that and RKE is that Rancher, the webGUI, supporting kube-router instead of calico/etc... (or in parallel to?) Closing this ticket in favour of the RKE thread does not address the webGUI functional need for kube-router support, which appears to be absent as of Rancher v2.6 as of this writing.
If @superseb I am just incorrect, there is a way to install, manage, and such for kube-router within Rancher v2.6/etc, then please point me in the correct direction. But thus-far, my search for it turns up nothing.
I am particularly interested in kube-router as it looks to be a possible solution for retaining "Source IP" since I'm using Metal LB Layer 2 loadbalaner . This is, naturally, all on-prem.

@BloodyIron Without support in RKE for this, there is no need to create changes in Rancher.
The only way to do it in RKE today is to disable all conflicting components in RKE and deploy it manually through add-ons or outside RKE. There are examples how other users have done it in https://github.com/rancher/rke/issues/312.

@superseb what is the roadmap for kube-router then? As that ticket looks roughly dead in the water from RKE/Rancher bringing kube-router in to be a main feature. My vote would be for it to be a selection in the rancher UI, naturally as an alternative to Calico/Cilium, the like, and all necessary steps taken by Rancher, as is the point.
I mean so much is already automated by Rancher, even if kube-router isn't in main for RKE, clearly the method for what needs to be automated is, as you say, in that thread.


How to handle error  in Angular 12 HTTP request

Proposed title of article
How to handle error  in Angular HTTP request
Proposed article introduction
In Angular 12, a new way has been introduces to handle errors that do occur when HTTP requests are made. In this article, we discuss how we can intercept such errors to prevent the app from crashing and render error page appropriately.
Key takeaways

New updates to HTTP error handling in version 12
Discuss in-depth the HTTP interceptors and how they are used to catch errors
Build a sample project to showcase our error handling using the Angular services.

closing this since i already other one open!

I'm ready to work on this!


🔭 Add space under jupyter outputs

@kolibril13 this is something that you mentioned, this should improve it slightly!
I am also moving figure --> div as many times outputs are not "figures".

Theme isn't deployed yet! Sorry for the confusion. This PR is only going to solve (1) not the scrollbars.