GateCalculator issue in face_landmark_front_cpu.pbtxt

when prev_has_enough_faces is true, don't passes the incoming image through to trigger a new round of face detection in FaceDetectionFrontCpu，i want to know after this, how to stop face detection? I do not understand the code (stop to trigger a new round of face detection in FaceDetectionFrontCpu) clearly ? can you help me please?
i am expecting a reply. thank you very much!
Drops the incoming image if FaceLandmarkCpu was able to identify face presence
in the previous image. Otherwise, passes the incoming image through to trigger
a new round of face detection in FaceDetectionFrontCpu.
node {
calculator: "GateCalculator"
input_stream: "image"
input_stream: "DISALLOW:prev_has_enough_faces"
output_stream: "gated_image"
options: {
[mediapipe.GateCalculatorOptions.ext] {
empty_packets_as_allow: true
}
}
}

when don't passes the incoming image through to trigger a new round of face detection in FaceDetectionFrontCpu, what is output_stream: "gated_image" ?is previous  image or null?
node {
calculator: "GateCalculator"
input_stream: "image"
input_stream: "DISALLOW:prev_has_enough_faces"
output_stream: "gated_image"
options: {
[mediapipe.GateCalculatorOptions.ext] {
empty_packets_as_allow: true
}
}
}

I modify the ComputeShader (TfLiteConverterCalculator in face_landmark_gpu.pbtxt) as follows. I want to pass a grayscale image(0-255pixels) to my model(160*160). But the inference result is always wrong, could you help me aboat the problem? I work in ubuntu system with gpu.Thank you very much
// Shader to convert GL Texture to Shader Storage Buffer Object (SSBO),
// with normalization to either: [0,1] or [-1,1].
const std::string shader_source = absl::Substitute(
    R"( #version 310 es
  layout(local_size_x = $0, local_size_y = $0) in;
  layout(binding = 0) uniform sampler2D input_texture;
  layout(std430, binding = 1) buffer Output {float elements[];} output_data;
  ivec2 width_height = ivec2($1, $2);

  void main() {
    ivec2 gid = ivec2(gl_GlobalInvocationID.xy);
    if (gid.x >= width_height.x || gid.y >= width_height.y) return;
    vec4 pixel = texelFetch(input_texture, gid, 0);
    int linear_index = gid.y * width_height.x + gid.x;
    pixel = pixel * 255.0;
    output_data.elements[linear_index + 0] = 0.299*pixel.x + 0.587*pixel.y + 0.114*pixel.z;
  })",
    /*$0=*/kWorkgroupSize, /*$1=*/input.width(), /*$2=*/input.height(),
);
RET_CHECK_CALL(GlShader::CompileShader(GL_COMPUTE_SHADER, shader_source,
                                       &gpu_data_out_->shader));
RET_CHECK_CALL(GlProgram::CreateWithShader(gpu_data_out_->shader,
                                           &gpu_data_out_->program));

// ivec4 ipixel = ivec4(pixel * 255.0);
// output_data.elements[linear_index + 0] = float((19595*ipixel.x + 38469*ipixel.y + 7472*ipixel.z)>> 16);
return ::mediapipe::OkStatus();

}));

see https://github.com/google/mediapipe/issues/978


feat: cherry pick req extentions

Changes

#738  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @MasterPtato and the rest of your teammates on  Graphite


[JN-706] notification editing

DESCRIPTION
This enables very rough-and-ready editing of notification emails from the admin tool.    While I was initially worried about security for this feature, my online research indicates emails are a less dangerous attack vector in many ways than website content, since email clients are typically more robustly sandboxed than websites.  So this, combined with the fact that we still control publishing, gives me confidence that we can expose this feature to our current limited user base without thorough safeguards around, e.g. script tag inclusion.

Along the way, this upgrades the notification list page to match our latest api fetching practices, so it's no longer reliant on a portal 'god object'.
This also includes some quality-of-life improvements for the test email sender, automatically inserting the admin user's name into the "to" field, and defaulting to the<EMAIL_ADDRESS>address if an email address hasn't been configured yet for the portal.
Not yet supported:
deletion/deactivation of configs
library of default templates
sensible guards like checks on whether or not duplicate configs exist for a single event
user-useful error handling
sending test emails for templates that aren't yet saved
TO TEST:  (simple manual steps for confirming core behavior -- used for pre-release checks)

repopulate hearthive ./scripts/populate_portal.sh hearthive
redploy apiAdminApp
go to https://localhost:3000/hearthive/studies/cmyop/env/sandbox/notificationContent
Click "Add"
enter "consent" for the name, and choose "consent" for the event type
Confirm the config is created, and you are taken to the screen shown above, where you can edit and save the email text

What is the difference between an email and a notification? If all notifications are email, perhaps we can get rid of the term "notification"?
Yup, notifications may also include text messages, but we're not supporting those yet, so can simplify to Emails
It might be less confusing if this page just allowed me to configure automated emails sent to participants. (And perhaps that is all it does now?)
Yup, that's what this page does
In that case events might be best listed as "Study enrollment complete".
To me the 'complete' is redundant. the event is "a study enrollment", and adding 'complete' might imply they've also completed all study activities.  I could see that going either way
"Task reminder" might be confusing since we have admin tasks. Are these "Participant task reminders"?
yeah, I had thought it was obvious this was configuring participant reminders, but I suspect that's my own bias.  Will clarify
Does it make sense to use a unit like hours to increase usability? (We could always add a choice of time unit later.)
Yup, the time controls were implemented in a hurry, and no one is using them yet.
What is "Ad-hoc"?
This could use some docs.  that's the template that gets used when you send a custom email.   It lets you include a custom message, but still have study logo, header and footer

@devonbush Thx for your explanation on the UI.
Any chance I can talk you into titling the page "Participant emails", and changing the use of "notification" to "email"?
We can always change this when we offer text message notifications, but it seems like there are enough differences in text notifications that we might have a different approach anyway.

It's getting late and so I don't trust my typing to convert the 3 fields to hours from minutes without errors.  So if it's ok with you, I'll take an IOU and we can convert those time units to something more user-friendly the next time we touch this page


If cocoapods-stats is triggered for a machine before pod setup is ran and there is no master repo it crashes.

Whilst fixing this https://github.com/jcampbell05/cocoapods-deploy/issues/3
I discovered that cocoapods-stats will crash if CocoaPods hasn't had pod setup ran. As a workaround in my plugin I set the environmental variable to disable cocoapods-stats unfortunately losing the statistics for CI builds (I will get around to fixing this on my side).
Expected behaviour should be that cocoapods-stats should abort uploading any statistics rather than crashing (perhaps log out the error).

This ought to be fixed usin CocoaPods beta 5.


Bug 1872784: Bumping Buildah

Using buildah version 1.15.1, given #169 is merged, we can use newest buildah.

@adambkaplan, I might be late to open this PR to bump buildah, so please let me know. (/cc @gabemontero, @coreydaley, @nalind)

/bugzilla refresh

LGTM

LGTM
/approve

/assign @smarterclayton

/unassign @smarterclayton
/assign @adambkaplan

@otaviof we need a new BZ ID if we want to merge this in 4.6. the prior BZ was related to the CVE patch in golang/x/text.
IMO it is worthwhile to add one, as each OpenShift release should incorporate the latest buildah bugfixes and enhancements.

/bugzilla refresh

/bugzilla refresh

/lgtm

/hold
This actually regresses a fix in buildah - https://github.com/containers/buildah/pull/2502

/lgtm cancel

We just tagged<EMAIL_ADDRESS>which includes the fix we would have regressed on if we'd used v1.15.1.  @otaviof if you want to pull that in, I think we'll be okay.  Thanks!

We just tagged<EMAIL_ADDRESS>which includes the fix we would have regressed on if we'd used v1.15.1. @otaviof if you want to pull that in, I think we'll be okay. Thanks!

Sure. Thanks for the heads up. Please consider latest changes.

/test e2e-aws-builds

/bugzilla cc-qa
Add this, just test "DevExp verified bugs before PR merge" workflow.

/bugzilla cc-qa
Add the label, just try to test "Bug Fix PR Pre-Merge confirmation testing" workflow

/bugzilla refresh

/retitle Bug 1872784: Bumping Buildah

/bugzilla cc-qa

LGTM


Server w/ Cubic Chunks eats all memory and dies

Environment:

1.17.1 fabric server, loader version 0.11.7
mods:

fabric api 0.41.3
cubic chunks 1.17.1-0.0.667.0-SNAPSHOT, compiled from latest commit at default branch

Java 17
JVM args: -Xmx24G -XX:+UseZGC

A server with all default settings and nothing but Cubic Chunks and fabric API eventually consumes all memory given and dies from out of memory exception/oom kill.
With no mods except cubic chunks and fabric api, with all default settings, it either generates the world in several minutes then idles at 12 gb usage, or crash from no memory during generation.
Example of idling:

Example of last moments before OOM crash:

logs for server OOM killed on generation:
latest.log

sometimes it does generate beyond 0%, but its very rare if it fully prepares world spawn and then goes to idle.

1.17 is not at all ready to use for 1.17. Please don't open issues for a very very unpolished project.


RancherOS kernel issue?

On RancherOS (v0.4.1, Linux 4.2.3-rancher, Docker 1.9.1-rc1), I'm getting this error inside a (Debian based) container when trying to tail -f:
tail: unrecognized file system type 0x794c7630 for '/var/log/messages'. please report this to<EMAIL_ADDRESS>reverting to polling

Type 0x794c7630 seems to be OverlayFS. Since this issue does not occur when running the SAME container image on another host OS. This makes be believe it's got something to do with RancherOS or its kernel (configuration).
For example, running on Boot2Docker (v1.9.1, Linux 4.1.13-boot2docker, Docker 1.9.1) with the same container image does not produce the error.

Still having this issue, just tested with v0.4.4 (released today).

Actually was unaware that both my Ubuntu and Boot2Docker machines were using aufs instead of overlay. Switching them to overlay results in the same message
It's an issue in GNU coreutils, which has been fixed in a later release (but it's still using polling because inotify is broken for overlayfs, a separate issue). Anyway, closing the issue here.


@emotion/babel plugin not working with experimentalStyled as styled

[x] The issue is present in the latest release.
[x] I have searched the issues of this repository and believe that this is not a duplicate.

Current Behavior 😯
The emotion babel plugin does not seem to be working i.e. labels are not applied to classNames.
Expected Behavior 🤔
Labels should be applied to classNames as it is with @emotion/styled
Steps to Reproduce 🕹

clone this repo
yarn install in root
go into /examples/create-react-app-with-typescript
yarn install
yarn eject
yarn add -D @emotion/babel-plugin
open ./config/webpack.config.js
go into line 391 and add '@emotion', there as babel plugin for the test: /\.(js|mjs|jsx|ts|tsx)$/, (= babel-loader)
open src/App.tsx and add

import { experimentalStyled as styled } from '@material-ui/core';

const Frame = styled('div')`
  background-color: green;
`

modify default export App by wrapping the <Container>...</Container> inside a <Frame>...</Frame>
yarn start
open browser and check class name for the App's div, should be something like css-xyz123

--> There is no labels appended to css class names for better developer experience.
Now replace
import { experimentalStyled as styled } from '@material-ui/core';
with
import styled from '@emotion/styled'
And check class names again! There is now the -Frame appended to the class name.
Question: Will this experimentalStyled ever be replaced with proper @emotion/styled. And is it safe to use already now if I don't plan on using styled-components ?
Context 🔦
Improve Developer Experience
Your Environment 🌎

`npx @material-ui/envinfo`
  System:
    OS: macOS 11.3.1
  Binaries:
    Node: 14.15.1 - ~/.nvm/versions/node/v14.15.1/bin/node
    Yarn: 1.22.10 - ~/.yarn/bin/yarn
    npm: 7.7.0 - ~/.nvm/versions/node/v14.15.1/bin/npm
  Browsers:
    Chrome: 90.0.4430.212
    Edge: 90.0.818.62
    Firefox: Not Found
    Safari: 14.1
  npmPackages:
    @emotion/react: latest => 11.4.0
    @emotion/styled: latest => 11.3.0
    @material-ui/core: next => 5.0.0-alpha.34
    @material-ui/private-theming:  5.0.0-alpha.33
    @material-ui/styled-engine:  5.0.0-alpha.34
    @material-ui/styles:  5.0.0-alpha.33
    @material-ui/system:  5.0.0-alpha.34
    @material-ui/types:  6.0.0
    @material-ui/unstyled:  5.0.0-alpha.34
    @material-ui/utils:  5.0.0-alpha.33
    @types/react: latest => 17.0.6
    react: latest => 17.0.2
    react-dom: latest => 17.0.2
    typescript: latest => 4.2.4

Thanks for the report. We are currently working on adding documentation for the experimentalStyled() (will be soon exported as a styled() from @material-ui/core/styles.) The difference between the MUI's experimentalStyled(), is that it provides:

default theme when none is available in the context
sx prop on the new component
support for theme's styleOverrides & variants

If you don't need any of these features, you may use emotion's styled(). I haven't looked into how the emotion's babel plugin is working, I suspect that it doesn't even consider the styled() from @material-ui/core/styles

@passionkind have you tried defining the importMap in the babel plugin?
See https://emotion.sh/docs/@emotion/babel-plugin#importmap - you can define @material-ui/core/styles there. Let me know if this works, or please provide a repository where we can debug the issue.

Tip: I could make the styled work with :
[
    "@emotion",
    {
      "importMap": {
        '@material-ui/core/styles': {
          'styled': {
            "canonicalImport": ["@emotion/styled", "default"],
            "styledBaseImport": ["@material-ui/core/styles", "styled"]
          }
        }
      }
    }
  ]

Protip, no need to eject, you can use customize-cra for adding the plugin.

I was able to get it to work with something similar:
    [
      "@emotion",
      {
        "importMap": {
          "@material-ui/core": { // <-- only "/core", not "/core/styles"
            "styled": {
              "canonicalImport": ["@emotion/styled", "default"],
              "styledBaseImport": ["@material-ui/core/styles", "styled"]
            }
          }
        }
      }
    ],

Looks like the issue has been resolved. In the last two comments, you may see how you can configure the plugin to work with the styled exported from @material-ui/core or @material-ui/core/styles based on how you import the utility in your codebase.

Hi @mnajdova the emotion babel plugin does not work anymore with mui, especially the the label is not applied for styled components.
For a vite-react-ts based project.
// vite.config.ts
export default defineConfig({
  plugins: [react({
    jsxImportSource: '@emotion/react',
    babel: {
      plugins: [
        [
          "@emotion",
          {
            "sourceMap": true,
            "autoLabel": "always",
            "labelFormat": "[local]",
            "cssPropOptimization": true,
            "importMap": {
              '@mui/system': {
                styled: {
                  canonicalImport: ['@emotion/styled', 'default'],
                  styledBaseImport: ['@mui/system', 'styled'],
                },
              },
              '@mui/material/styles': {
                styled: {
                  canonicalImport: ['@emotion/styled', 'default'],
                  styledBaseImport: ['@mui/material/styles', 'styled'],
                },
              },
            },
          }
        ]
      ]
    }
  })],
})

// component
import {styled} from "@mui/material/styles"

const StyledBox = styled(Box)(({theme}) => {
  return {
    height: '100px',
    width: '100px',
    backgroundColor: 'red'
  }
})

inside react-dev tool I see the Box3 instead of proper variable name StyledBox

I'd really appreciate if you helped here, if you need more details let me know please

@nichita-pasecinic How did you solve the issue? Have the same problem

@andrew-stupchuk use name property, e.g.:
const StyledComponent = styled(`div`, { name: 'StyledComponent' }, (theme) => ({
  ...
}))

I am working on NextJs App router and, it seems that the current solution don't apply there:
https://codesandbox.io/p/devbox/frosty-lake-v6ymx4


Social app

Description
example SSX-powered app that has a universal post feed, and lets signed-in users add a post. This version is configured to run locally and does not persist state
Type

[ ] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Diligence Checklist
(Please delete options that are not relevant)

[X] This change requires a documentation update
[X] I have performed a self-review of my code
[X] I have made corresponding changes to the documentation
[X] My changes generate no new warnings

Codecov Report
Base: 90.34% // Head: 90.34% // No change to project coverage :thumbsup:

Coverage data is based on head (5bd63b0) compared to base (297b42d).
Patch has no changes to coverable lines.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@           Coverage Diff           @@
##             main      #71   +/-   ##
=======================================
  Coverage   90.34%   90.34%
=======================================
  Files          25       25
  Lines        3314     3314
  Branches      267      267
=======================================
  Hits         2994     2994
  Misses        320      320

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Closing - will be consolidated under a repository - ssx-examples


USGS Config Fails to Load on Initialization

Describe the bug
On initializing eodag without using any custom config, usgs doesn't get initialized. It doesn't show up in list of providers as well. Updating provider configuration using a yaml formatted string fails as well for usgs since it is not a part of the providers list.
Code To Reproduce
CLI commands or Python code snippet to reproduce the bug. Please use maximum verbosity using:
from eodag import EODataAccessGateway
from eodag.utils.logging import setup_logging
setup_logging(verbose=3)

dag = EODataAccessGateway()
dag.available_providers()
# ['astraea_eod', 'cop_ads', 'cop_cds', 'cop_dataspace', 'creodias', 'earth_search', 'earth_search_cog', 'earth_search_gcs', 'ecmwf', 'mundi', 'onda', 'peps', 'sara', 'theia', 'usgs_satapi_aws']
dag.update_providers_config(open('/eodag_config.yaml').read())
dag.available_providers()
# ['astraea_eod', 'cop_ads', 'cop_cds', 'cop_dataspace', 'creodias', 'earth_search', 'earth_search_cog', 'earth_search_gcs', 'ecmwf', 'mundi', 'onda', 'peps', 'sara', 'theia', 'usgs_satapi_aws']

Output
Compete output obtained with maximal verbosity.
2023-03-28 16:28:43,998 eodag.config                     [INFO    ] (config           ) Loading user configuration from: /home/varisht/.config/eodag/eodag.yml
2023-03-28 16:28:44,013 eodag.config                     [INFO    ] (config           ) sobloo: unknown provider found in user conf, trying to use provided configuration
2023-03-28 16:28:44,014 eodag.config                     [WARNING ] (config           ) sobloo skipped: could not be loaded from user configuration
2023-03-28 16:28:44,014 eodag.config                     [DEBUG   ] (config           ) Traceback (most recent call last):
File "/home/varisht/anaconda3/envs/arosics/lib/python3.7/site-packages/eodag/config.py", line 382, in override_config_from_mapping
config[provider] = ProviderConfig.from_mapping(new_conf)
File "/home/varisht/anaconda3/envs/arosics/lib/python3.7/site-packages/eodag/config.py", line 127, in from_mapping
mapping[key] = PluginConfig.from_mapping(mapping[key])
File "/home/varisht/anaconda3/envs/arosics/lib/python3.7/site-packages/eodag/config.py", line 203, in from_mapping
c.dict.update(mapping)
TypeError: 'NoneType' object is not iterable
2023-03-28 16:28:44,161 eodag.core                       [INFO    ] (core             ) usgs: provider needing auth for search has been pruned because no crendentials could be found
2023-03-28 16:28:44,161 eodag.core                       [INFO    ] (core             ) aws_eos: provider needing auth for search has been pruned because no crendentials could be found
2023-03-28 16:28:44,161 eodag.core                       [INFO    ] (core             ) meteoblue: provider needing auth for search has been pruned because no crendentials could be found
2023-03-28 16:28:44,296 eodag.core                       [DEBUG   ] (core             ) Opening product types index in /home/varisht/.config/eodag/.index
2023-03-28 16:28:44,304 eodag.core                       [INFO    ] (core             ) Locations configuration loaded from /home/varisht/.config/eodag/locations.yml
Environment:

Python version: python 3.7.8
EODAG version: eodag (Earth Observation Data Access Gateway): version 2.9.1

Additional context
Add any other context about the bug here.
Have encountered this bug on Python 3.9 as well. I don't see this bug when using a custom yaml config when initializing the EODataAccessGateway.

Hello @varisht-tathya , this is the actual expected behaviour:
see https://eodag.readthedocs.io/en/stable/notebooks/api_user_guide/2_providers_products_available.html

If a provider is configured to need authentication for search, and has no crendentials set, it will be pruned on EODAG initialization, and will not appear in available providers list.

Then, if usgs has already been pruned during initialization, it will not be restored.
Is there a particular reason to not use /eodag_config.yaml on EODataAccessGateway init ?

The reason for not using /eodag_config.yaml on init was that the file would have to be stored locally, which is not great for security considering it contains passwords.

ok thanks. We can plan to change this mechanism and restore pruned providers when configuration is updated


RTVE plugin needs an update

RTVE channels always return "error: The program is not available due to rights restrictions"
It looks like stream URLs have been changed, here's their new addresses (HLS streams) :
La 1:
http://rtve-live.hds.adaptive.level3.net/hls-live/rtvegl7-la1lv3aomgl7/_definst_/live.m3u8
La 2:
http://rtve-live.hds.adaptive.level3.net/hls-live/rtvegl0-la2lv3aomgl0/_definst_/live.m3u8
Canal 24 Horas:
http://rtve-live.hds.adaptive.level3.net/hls-live/rtvegl8-24hlv3aomgl8/_definst_/live.m3u8
Teledeporte:
http://rtve-live.hds.adaptive.level3.net/hls-live/rtvegl1-tdplv3aomgl1/_definst_/live.m3u8
However it seems the server needs to authorize the IP first by loading the webpage, haven't identified the way it's done yet. Hope this helps!

Correction : La 1 is actually<EMAIL_ADDRESS>

Buenas @nonoche2; if you're interested, Streamlink (a fork of Livestreamer) provides an updated RTVE plugin with VOD support as well.


fix log category

Problem
Quill's logs are using the user's package. It should be possible to configure a log category for all quill messages.
Solution
Use a fixed class to create the loggers.
Notes
I've removed the custom sql classpath because it breaks the eclipse project. It was necessary to setup h2 for compile-time query probing, that's now disabled.
Checklist

[x] Unit test all changes
[x] Update README.md if applicable
[x] Add [WIP] to the pull request title if it's work in progress
[x] Squash commits that aren't meaningful changes
[x] Run sbt scalariformFormat to make sure that the source files are formatted

@getquill/maintainers

cc/ @onilton

@fwbrasil I found sources log sql in INFO level. But DEBUG may be better.
And we may logger the sql for failed actions,
Because we often see such error message when we run sql like

sql

select t.x from t where t.id in ()

error

You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '(' at line 1

And because that was reported by a failed Future, it's totally use less to find where error occurs

:+1:
I agree with @jilen regarding logging level , but that might be changed in another PR since this is a bugfix PR.

:+1:

Amazing. Looks perfect to me! :D
Also, I agree with the DEBUG level, I will probably only look for this info when debugging the code.

I'm merging the fix, let's discuss other changes separately.


Missing Assemblies?

When I try to build the solution, I get this error:
The type or namespace name 'BaseViewModel' could not be found (are you missing a using directive or an assembly reference?)	MyShopAdmin	C:\Users\Erick Orlando\Documents\GitProjects\MyShoppe\MyShopAdmin\ViewModels\FeedbackListViewModel.cs	11	Active
Thanks

Sorry, I mistake with references.


Feature: Refactor imageAddNewView.js

This effects the connextCMS/public/js/app/views/imageAddNewView.js:
imgOrigWidth  and imgOrigHeight is redundent with imgWidth[] and imgHeight[]. Refactor to eliminate imgOriginWidth and imgOrigHeight.

Completed this task 5/3/16


[WIP] Get IISIntegration CI working

@natemcmaster I just went ahead and removed matrix. Duplication wise it isn't bad.


Cursor color in Xresources

Would it be possible to add an option to Xresources for which color the cursor points to?

So you want something separate from st.(foreground|background)? Right now if you modify those, the cursor changes too.

Oh, so if I get this right, the cursor is tied to foreground currently? Yea if we could change the cursor independent of that it would be awesome!

More specifically, the default config.h file has them all set to color slots 256/257. Then, st.(foreground|background) will change the values of slots 256/257, which are linked to foreground/background/cursorfg/cursorbg. It shouldn't be hard to check if st.(cursorfg|cursorbg) exist, and if so set the colors independently. I can get a pull request later today with that.

Much appreciated!

I was about to file the exact same feature request. Adding a "mee too" since I can't upvote.

@ccrusius why can't you upvote? :worried:

Oh right, I thought you meant you couldn't thumbs up, so I curious as to why not. But yeah, thumbs ups and downs are basically voting now on github. :+1:

Finally got some time to work on this, will update later today.

Sorry, been struggling with my distribution for a while now, when I get the environment correct it's a five minute fix. Keep going down rabbit holes along the way right now.

Code is done - if you want, give it a spin. I could go ahead and merge it but I want independent verification of it working. My setup is GuixSD and it sometimes displays strange stuff - no guarantee that since it works on my machine, it will work on yours. Assuming you are fine building it?

Not a problem just point me to the patch or patched source and I can give it a go.

Works for me, thanks!

Might be worth adding it to the example Xresources in the repo. I only stumbled upon this option in the issues.


[SPARK-32350][CORE] Add batch-write on LevelDB to improve performance of HybridStore

What changes were proposed in this pull request?
The idea is to improve the performance of HybridStore by adding batch write support to LevelDB. #28412  introduces HybridStore. HybridStore will write data to InMemoryStore at first and use a background thread to dump data to LevelDB once the writing to InMemoryStore is completed. In the comments section of #28412 , @mridulm mentioned using batch writing can improve the performance of this dumping process and he wrote the code of writeAll().
Why are the changes needed?
I did the comparison of the HybridStore switching time between one-by-one write and batch write on an HDD disk. When the disk is free, the batch-write has around 25% improvement, and when the disk is 100% busy, the batch-write has 7x - 10x improvement.
when the disk is at 0% utilization:

log size, jobs and tasks per job
original switching time, with write()
switching time with writeAll()

133m, 400 jobs, 100 tasks per job
16s
13s

265m, 400 jobs, 200 tasks per job
30s
23s

1.3g, 1000 jobs, 400 tasks per job
136s
108s

when the disk is at 100% utilization:

log size, jobs and tasks per job
original switching time, with write()
switching time with writeAll()

133m, 400 jobs, 100 tasks per job
116s
17s

265m, 400 jobs, 200 tasks per job
251s
26s

I also ran some write related benchmarking tests on LevelDBBenchmark.java and measured the total time of writing 1024 objects. The tests were conducted when the disk is at 0% utilization.

Benchmark test
with write(), ms
with writeAll(), ms

randomUpdatesIndexed
213.06
157.356

randomUpdatesNoIndex
57.869
35.439

randomWritesIndexed
298.854
229.274

randomWritesNoIndex
66.764
38.361

sequentialUpdatesIndexed
87.019
56.219

sequentialUpdatesNoIndex
61.851
41.942

sequentialWritesIndexed
94.044
56.534

sequentialWritesNoIndex
118.345
66.483

Does this PR introduce any user-facing change?
No
How was this patch tested?
Manually tested.

Can one of the admins verify this patch?

Can one of the admins verify this patch?

cc @HeartSaVioR @mridulm @tgravescs ^^

ok to test

Test build #126129 has started for PR 29149 at commit a12e178.

add to whitelist

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/30741/
Test PASSed.

Test build #126130 has started for PR 29149 at commit a12e178.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/30742/
Test PASSed.

Test build #126130 has finished for PR 29149 at commit a12e178.

This patch fails PySpark pip packaging tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126130/
Test FAILed.

Test build #126129 has finished for PR 29149 at commit a12e178.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126129/
Test PASSed.

@mridulm @tgravescs
I'm planning to merge this in tomorrow. Please comment if you'd like to have time to review. Thanks!

I likely won't have time for a review so go ahead without mine

OK I'll go ahead merging. To be sure I'll trigger test once again.

retest this, please

Test build #126278 has started for PR 29149 at commit a12e178.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/30894/
Test PASSed.

Test build #126278 has finished for PR 29149 at commit a12e178.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126278/
Test FAILed.

retest this, please

Test build #126284 has started for PR 29149 at commit a12e178.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/30899/
Test PASSed.

Test build #126284 has finished for PR 29149 at commit a12e178.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126284/
Test FAILed.

retest this, please

Test build #126290 has started for PR 29149 at commit a12e178.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/30905/
Test PASSed.

Test build #126290 has finished for PR 29149 at commit a12e178.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126290/
Test PASSed.

Thanks! Merging to master.

Thanks for the review!

Sorry for the delay in getting to this.
The only additional change I had made in my version was to batch the batch update.
For example, if the list is very large, the write will block all other threads for a very long time - which has fairness issues; in addition to the serialization consuming too much memory, etc.
Instead, I batched updates to some K values (In my case K was 128).
This can be trivially done with an inner loop doing for (List<?> batchList : Iterables.partition(entry.getValue(), batchSize)) { ... }

This seems an important improvement. Should I put up a followup PR to include this change?

That would be great, thanks @baohe-zhang !


Create: 0983-minimum-cost-for-tickets.kt

Language(s) Used: kotlin
Submission URL: https://leetcode.com/problems/minimum-cost-for-tickets/submissions/914351172/

Create: 0983-minimum-cost-for-tickets.kt
Language(s) Used: kotlin
Submission URL: _https://leetcode.com/problems/minimum-cost-for-tickets/submissions/914351172/_

Important
Please make sure the file name is lowercase and a duplicate file does not already exist before merging.

I corrected your filename and separeted the solutions by class. Thanks for the contribution

Thanks for merging!


Expanding ModalSheet, the navigation bar color does not respect theme with Material3

Using Material 3, when setting the theme where the navigation is dark and views are displayed above the bar, the ModalSheet still expands with a white navigation bar instead. See demo video. I modified the demo app from the code base to use Material 3 and added an to demonstrate how a Material 3 Modal Bottom Sheet behaves.
modelsheet_issue.webm
I've created a fork with my changes here.  The changes in this fork is that I added "android:windowTranslucentStatus = true" to themes.xml along with M3 ModalBottomSheet.
Thanks!

Thanks for the report!
I am afraid you forgot to push changes to your fork 🙈
But anyway, I got the point from the video.
Drawing over the navigation bar is one of the features of the ModalSheet, it is useful when you want to draw some image background in the sheet content.
As for now I would recommend to change navigation bar icons color with system UI controller from accompanist. You may change to darkIcons = true when sheet is visible and change back to darkIcons = false in onDispose lambda. If you do support dark mode, I guess this will not be needed.
ModalSheet(
    visible = visible,
    onVisibleChange = onVisibleChange
) {
    val sysUiController = rememberSystemUiController()
    DisposableEffect(sysUiController) {
        sysUiController.setNavigationBarColor(Color.Black, darkIcons = true)
        onDispose {
            sysUiController.setNavigationBarColor(Color.Black, darkIcons = false)
        }
    }

    ...
}

https://user-images.githubusercontent.com/20944869/235344648-081c224a-7935-4a15-ae4d-8dd7286b87f3.mp4
In the meanwhile I am thinking about adding a new parameter to ModalSheet to support drawing above (on y axis) navigation bar to match M3 behavior 🤔

Thanks, I pushed the code to my fork.  I think if you do add a new
parameter to support the drawing above the navigation bar to match M3
behavior, then this would help with my case. 🙏😃
On Sun, Apr 30, 2023 at 2:05 AM Oleksandr Balan @.***>
wrote:

Thanks for the report!
I am afraid you forgot to push changes to your fork 🙈
But anyway, I got the point from the video.
Drawing over the navigation bar is one of the features of the ModalSheet,
it is useful when you want to draw some image background in the sheet
content.
As for now I would recommend to change navigation bar icons color with system
UI controller https://google.github.io/accompanist/systemuicontroller/
from accompanist. You may change to darkIcons = true when sheet is
visible and change back to darkIcons = false in onDispose lambda. If you
do support dark mode, I guess this will not be needed.
ModalSheet(
visible = visible,
onVisibleChange = onVisibleChange
) {
val sysUiController = rememberSystemUiController()
DisposableEffect(sysUiController) {
sysUiController.setNavigationBarColor(Color.Black, darkIcons = true)
onDispose {
sysUiController.setNavigationBarColor(Color.Black, darkIcons = false)
}
}
...

}
https://user-images.githubusercontent.com/20944869/235344648-081c224a-7935-4a15-ae4d-8dd7286b87f3.mp4
In the meanwhile I am thinking about adding a new parameter to ModalSheet
to support drawing above (on y axis) navigation bar to match M3 behavior
🤔
—
Reply to this email directly, view it on GitHub
https://github.com/oleksandrbalan/modalsheet/issues/27#issuecomment-1528976093,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABA742H7DOYXFWIDTXZI3VDXDYTMBANCNFSM6AAAAAAXP4MLTM
.
You are receiving this because you authored the thread.Message ID:
@.***>

--
Duong (Yoong) Tran
Email: @.***

Yea, you could check PR #28, it should do the trick I think 👌

Awesome, thanks. This PR looks like it is working for me. Thank you so
much! 🙏😃
On Sun, Apr 30, 2023 at 10:32 AM Oleksandr Balan @.***>
wrote:

Yea, you could check PR #28
https://github.com/oleksandrbalan/modalsheet/pull/28, it should do the
trick I think 👌
—
Reply to this email directly, view it on GitHub
https://github.com/oleksandrbalan/modalsheet/issues/27#issuecomment-1529088522,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABA742HSWZCO3JYNXIG4UNLXD2O3DANCNFSM6AAAAAAXP4MLTM
.
You are receiving this because you authored the thread.Message ID:
@.***>

--
Duong (Yoong) Tran
Email: @.***


Custom CSS

Dear Support
I want to add some custom css for my app. How can i do this?

Hi @tohaitrieu. Thanks for your suggestion. We'll think about the best way to add custom css in our apps.


List save

I added a modal to the html and created the list on the modal.

Clear to move forward.


Tmi2 364/add grant advert controller and service

add grant advert controller and service, needed for the new mandatory question conditional redirection journey

Can you run an organise imports inside Intellij for the project pls. There's some unused imports from copying code over.


  I suspect this should be against the data socket like so:
  .finally(() => when.try(dataSocket.destroy.bind(dataSocket)));

Edit: To clarify I'm using a Duplex PassThrough stream for which destroy is not defined.  I see now that the Writable stream has a non-public destroy method.  It doesn't look like the Duplex stream implements that until an update added in May (not in the version I have).
If this is intended to called be on the writable stream, I can shim it for the time being.

The stream is what is meant to be destroyed as the dataSocket is ended by calling this.connector.end() here.
The reason I added the stream destroy was to clean up in the event of an error, I'm not sure if it is completely necessary or not.
I've added a check for the existence of destroy for these cases.


high-scores: Regenerate tests

Very simple PR to bump the test version now that https://github.com/exercism/problem-specifications/pull/1378 has been merged.

Thanks @pgaspar


totalDisc field is null

Hi :wave:
It appears that the totalDisc field is null

in this specific case i am very sure that it is not null in the mp3 :)
Thank you and best regards

Hi @Feichtmeier ,
I have pushed a new version fixing this bug (normally). Feel free to try it out :)


Error: "failed to write entire file" in .unpack()

I keep getting failed to write entire file errors when calling the .unpack() and .unpack_in() methods from Entry.
I saw that the destination file was correctly being created and filled with 4096 bytes of data, then the error appears.
I have no idea why this happens - could you please enlighten me?
Thanks in advance for your help!

I saw that the destination file was correctly being created and filled with 4096 bytes of data, then the error appears.

Maybe there's some problem with the system you are running this on?
Perhaps it runs out of the space?
Or the admin has set an artifical limit?

No as I'm able to extract the tarball with the tar command.

I tried to copy the Entry's content to a buffer directly, and saw that zero bytes had been copied.
So that's not a filesystem issue at all, even in memory the problem appears. And the 4096 allocated bytes on disk seem to be when the file was created, before any data was actually written to it.

That's because tar is a streaming format.
When you iterate over them, tar::Archive calls the underlying Reader which implements Read to read the data, which changes the cursor.

I see! I feel like this isn't explicited enough in the documentation, I didn't understand that at all. Maybe this could added in the docs of Entries and/or Entry to avoid other peoples such problems :) ?

I see! I feel like this isn't explicited enough in the documentation, I didn't understand that at all. Maybe this could added in the docs of Entries and/or Entry to avoid other peoples such problems :) ?

That will certainly help.
I remember similar things happen in zip-rs/zip where Entries of zip are stored.
Since they both refers to the same underlying File, using one invalidates others.


A passthrough for JSON

I have some topics sending the info in a JSON. I want to receive the JSON and send it without changes to the MQTT client. A passthrough...
How should I configure it?
Thanks!

YOu could use functions

YOu could simply create a map and set the Wildecard from and then set the payload to keep original. Then add this map to the client get


Pre-requisite of User Role for AWS RDS MYSQL to Azure MYSQL Migration

Hello experts,
I would like to know the role the user requires for both source and target needed for DB migration.
Case 1 : Source DB - AWS RDS MYSQL   Target - Azure MYSQL
Case 2 : Source DB - AWS RDS PostGresSQL   Target - Azure PostGresSQL
Both these needs to be migrated using Azure DMS services.
Can you please clearly call out the user privilege required for Data Migration. ( both for source and target )
Thanks!
[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b14c5b08-b543-b9d5-88f5-bfe846bd57d4
Version Independent ID: 4004d227-1c90-e4ec-4c14-77a6db6c8a75
Content: Tutorial: Migrate MySQL to Azure Database for MySQL offline using DMS - Azure Database Migration Service
Content Source: articles/dms/tutorial-mysql-azure-mysql-offline-portal.md
Service: dms
GitHub Login: @sumitgaurin
Microsoft Alias: sgaur

@chaisen2006 Thank you for the feedback. We are actively investigating and will get back to you soon.

@chaisen2006 Thanks for your patience. As per the MySQL documentation, "mysqldump requires at least the SELECT privilege for dumped tables, SHOW VIEW for dumped views, TRIGGER for dumped triggers, LOCK TABLES if the --single-transaction option is not used, and (as of MySQL 5.6.49) PROCESS if the --no-tablespaces option is not used. Certain options might require other privileges as noted in the option descriptions."
The same article provides more details if we have procedure or other type of objects in the schema.
For Database migration service, "The user must have the privileges to read data on the source database".
For target database, you can use server admin user name and password.

@AnuragSharma-MSFT ; thanks for your reply. For target using DMS , we would be only using to load data , do we really need server admin. ? ( or only read/write would not work? )

@chaisen2006 Thanks for relying back. Before we start importing data through DMS we need to import schema to our Azure MySQL Database and for that users need to have proper privileges for creating routine, view, tables and other object. If we assign these special privileges to users then it would be fine, otherwise using server admin will be easy way out as they already have all privileges.
Summary of Available Privileges

@AnuragSharma-MSFT : This is clear thanks!

@chaisen2006 We will now proceed to close this thread. If there are further questions regarding this matter, please comment and we will gladly continue the discussion.


request: role

Description
reward system to the counter.
Solution
please upgrade the counting code to add/remove role automatically as a last counter reward
Code of Conduct

[x] I agree to follow this project's Code of Conduct.

There already is one such system, though it is not here and a bit more complex.
See https://github.com/BlackWolfWoof/yagpdb-cc/tree/master/Counting.
Closing, we wouldn't want to duplicate existing efforts.

but the check mark emoji... not there so please add that option or help me with it


Support ESBuild Code splitting

Currently, these are the only options supported by js.Build https://godoc.org/github.com/gohugoio/hugo/resources/resource_transformers/js#Options
However, ESBuild exposes all of these. There's a lot of good stuff in there, including control over the output format (presently defaulting to IIFE), code-splitting and sourcemap support. Code-splitting is especially useful since it prevents users from re-downloading the same code twice. Sourcemaps are another important one for debugging.
@remko Thanks for the new feature BTW, it's really great!

Just a quick note that the above is not about just adding some more options -- which is the reason they're not there in the first place.

For source maps we could probably replicate what we do in ToCSS.
For code splitting, If I understand it correctly, it will produce multiple output files where the imports will be stable, so to speak.

The only thing that could be 'just adding some options' is inline sourcemaps (which would still leave one output file). However, this is only sensible in a conditional in development mode, so I left it out of the initial patch as well.

I have been looking at the code spitting "thing" (which I assume will make this really interesting).
The challenge with this one is that it requires multiple entry points to be usable/interesting. But I'm not totally sure how to express this.
Thinking out loud:
{{ $entryPoints := resources.Match "**/index.js" }}
{{ $js := resources.Get "home/index.js" }}

{{ $js = $js | js.Build (dict "entryPoints" $entryPoints) }}

Would the above be clear enough? /cc @regisphilibert

I think I got a bit confused in regard to code-splitting. From what I now understand, Code-splitting is when you use async imports, e.g.
const Home = lazy(() => import('./routes/Home'));
const About = lazy(() => import('./routes/About'));

This is nice for web apps, but for blogs and documentation the use cases are more limited.
Chunking is the idea of having more than one file in your bundle output, so that you get better caching / less code duplication.
Code-splitting in ESBuild still seems experimental and limited to the ESM output format. Chunking is something you can do manually with re-exporting e.g.
// my-utils.js
export { range, take } from 'lodash-es'

becomes something like
// my-utils.bundle.js
// <big bundle of code>
//  export const range = ...
// export const take = ...

However it's a bit unclear how one would make that work in the present context, when all your JS source files are in your assets folder. You need to tell ESBuild where to split your code, which if I understand correctly is what "entry point" alludes to (if that's the case, then I propose we use a different terminology because something like `my-utils.bundle.js` is clearly not an entry point into my program).

Since chunks / entry points are gonna be shared across your project, maybe it would make sense to declare them globally in `config.toml`.

I'm fairly limited in the JS realm, so my input albeit naming and UX might not be too acute.

Thought about this some more:
You can mix both patterns together e.g.
// my-utils.js
export { range, take } from 'lodash-es'

and then in your consumer code
const { range, take } = await import('./my-utils')

The advantage here is that you only need to declare your entry points / chunks in your source files. The disadvantage is that you're using async/await for something that doesn't really need to be.

@Vincent-Carrier Re. your last post, given that you have more than a few external dependencies (which I I see is not uncommon in the crazy world of JavaScript), I think you quickly will get a "management problem" doing this in Hugo (keeping track and building the chunks ...).
I don't think it's very uncommon, even for a documentation site, to have page or section specific JS code, and to be able to just import directly from the NPM dependencies without having to worry (too much) about the bundle size would be golden (to me).
But I think this problem needs to mature a little bit (both here and on the ESBuild side) before we attack it.

@bep Completely agree.
My website only has two dependencies at the moment (lit-html and rough-notation), and so far I've been shipping them unbundled without any performance problems. The problem with the JavaScript ecosystem at the moment is that so many packages are either straight up impossible to use without a bundler or they hurt performance by deeply chaining imports. It's been a fun challenge to work around that constraint (kind of like the JavaScript equivalent of going vegan), but eventually there are times when a bundler is the only solution that makes sense.
@remko If ever you'd like to walk me through your implementation, I'd be really excited to try and build a proof of concept myself.

@Vincent-Carrier I have split these "options tasks" into separate issues. The Format part is merged, for source maps, see #7504

FWIW, my news site has run into a "chunking" issue already. I have two JS entrypoints: main.js (which is Babel'd for IE11) and enhancements.js (which requires a modern browser). Both of them end up importing metrics.js, which does some localStorage stuff to figure out when you were last on the site if ever. It's not the end of the world in my case, but in general, it would be better if metrics.js were only executed once, so you're not wasting time doing the same calculations twice. (Downloading once would also be nice, but not if means that two requests become three.)
Anyway, all this is to say that even for a "simple" site, JS dependencies can get hairy pretty quickly.

I have been looking at the code spitting "thing" (which I assume will make this really interesting).
The challenge with this one is that it requires multiple entry points to be usable/interesting. But I'm not totally sure how to express this.
Thinking out loud:
{{ $entryPoints := resources.Match "**/index.js" }}
{{ $js := resources.Get "home/index.js" }}

{{ $js = $js | js.Build (dict "entryPoints" $entryPoints) }}

Would the above be clear enough? /cc @regisphilibert

This usage seems odd to me, since the esbuild doesn't distinguish between the first and subsequent entrypoints. I think it would also be most useful if this function with multiple entry points was capable of returning all of the files back as resources, so that we can fingerprint them properly.
If possible, it would be excellent is js.Build could simply take and return a resource.Resources, just like it currently does with resource.Resource.
{{/* baseof.html */}}

{{ $entryPoints := resources.Match `ts/pages/**` }}
{{ $opts := dict `minify` true  `sourcemap` `external`  `target` `es2016`  `targetPath` `js` }}
{{ $js := $entryPoints | js.Build $opts }}

{{ range $js }}

  {{/* we need to "publish" the files so they are able to be imported by each other */}}
  {{/* note these aren't fingerprinted, but esbuild chunk files are already named with a hash */}}
  {{ .Publish }}

  {{/* example: "/js/account-settings.js" specified in the page front matter */}}
  {{ if eq $.Params.js .Name  }}
    {{ with . | fingerprint }}
      <script src="{{ .RelPermalink }}" type="module" integrity="{{ .Data.Integrity }}"></script>
    {{ end }}
  {{ end }}

{{ end }}

Note: "type" has to be "module" here because esbuild only supports output mode "esm"  when using code splitting. Source


Notes on replacing Jasmine

Since you have a request about it in the readme, you can see where custom test framework support was added to Atom here, with a Mocha runner implemented here.

Oh cool! Thanks @Arcanemagus! 💃

Looks like it's been in Atom since v1.2.0.
Also I haven't seen anything about coverage testing, but if you find something I'd be interested to!

👍 I tried using nyc, but couldn’t get it to work, as it uses eval for some reason, and atom’s security settings don’t allow that... I believe the reason for that, in nyc, was to get a global scope or something, which should probably be fixable with a PR to nyc. Anyway, too much work for now! 😬


ValueError: The reference state does not converge for Northern Hemisphere.

Hello again! I was doing a trial run with some data I had and ran into the given error. The interpolated fields look good to me but  qgfield_object.compute_reference_states(northern_hemisphere_results_only=False) fails. I will take a closer look at the values tomorrow but please let me know if I should look for something specific for debugging this. Thanks in advance.

Hi @hs2112 , this behavior is observed sometimes with fine resolution data and where equatorial field changes sign. May you possibly share a snapshot of u, v and temperature field at the instance it fails (maybe in netCDF file - you may upload it on https://gofile.io/ and send me the link to the file here)? Please also share the python code you use here. Thanks!

Hi @csyhuang. Is it not possible to compute LWA for a subset of NH instead of the entire globe?
Here is the untreated data (u, v, t combined) on the filtered timestep - https://gofile.io/d/Wqurnq. But I tried on some other timesteps and it throws the same error every time. Note that my pressure levels are spaced by as much as 100hPa in mid troposphere.
I am running the following lines in python -
tstep=0
uu = u_file.variables['u'][tstep, ::-1, ::-1, :].data
vv = v_file.variables['v'][tstep, ::-1, ::-1, :].data
tt = t_file.variables['t'][tstep, ::-1, ::-1, :].data
qgfield_object = QGField(xlon, ylat, plev, uu, vv, tt, kmax=17)
qgpv[tstep, :, :, :], interpolated_u[tstep, :, :, :], interpolated_v[tstep, :, :, :], \
    interpolated_theta[tstep, :, :, :], static_stability = qgfield_object.interpolate_fields()
qref[tstep, :, :], uref[tstep, :, :], ptref[tstep, :, :] = \
    qgfield_object.compute_reference_states(northern_hemisphere_results_only=False)

Hello @hs2112,
I had a quick look at the dataset and can reproduce your issue:
>>> import xarray as xr
>>> from hn2016_falwa.xarrayinterface import QGDataset
>>> data = xr.open_dataset("export_tstep_0.nc")
>>> qgd = QGDataset(data, qgfield_kwargs={
...     "kmax": 17
... })
>>> qgd.interpolate_fields()
<xarray.Dataset>
>>> qgd.compute_reference_states()
 Maxits exceeded
           0       32764<PHONE_NUMBER>  converged at n =       100001
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../python3.9/site-packages/hn2016_falwa/xarrayinterface.py", line 245, in compute_reference_states
    out_fields = _map_collect(
  File ".../python3.9/site-packages/hn2016_falwa/xarrayinterface.py", line 45, in _map_collect
    for name, y in zip(names, f(x)):
  File ".../python3.9/site-packages/hn2016_falwa/xarrayinterface.py", line 246, in <lambda>
    lambda field: field.compute_reference_states(northern_hemisphere_results_only),
  File ".../python3.9/site-packages/hn2016_falwa/oopinterface.py", line 534, in compute_reference_states
    raise ValueError("The reference state does not converge for Northern Hemisphere.")
ValueError: The reference state does not converge for Northern Hemisphere.

I noticed that the data has 0.25° horizontal spacing, which is a very high resolution and might be an issue for the numerical solver. If I coarsen the data to 1°, the reference state solver converges:
>>> coarsened = data.isel({
...     "latitude":  slice(None, None, 4),
...     "longitude": slice(None, None, 4)
... })
>>> qgd = QGDataset(coarsened, qgfield_kwargs={
...     "kmax": 17
... })
>>> qgd.interpolate_fields()
<xarray.Dataset>
>>> qgd.compute_reference_states()
<PHONE_NUMBER>       32764<PHONE_NUMBER>  converged at n =          716
           0           0           0  converged at n =          721
<xarray.Dataset>
Dimensions:  (height: 17, ylat: 181)
Coordinates:
  * height   (height) float64 0.0 1e+03 2e+03 3e+03 ... 1.4e+04 1.5e+04 1.6e+04
  * ylat     (ylat) float64 -90.0 -89.0 -88.0 -87.0 ... 87.0 88.0 89.0 90.0
Data variables:
    qref     (height, ylat) float64 -0.0 -0.0 -0.0 -0.0 -0.0 ... 0.0 0.0 0.0 0.0
    uref     (height, ylat) float32 0.0 0.0 0.0 0.0 ... 11.65 8.511 4.507 0.503
    ptref    (height, ylat) float32 208.3 209.4 211.6 ... 443.1 443.4 443.5
Attributes: (12/13)
    ...

I'm only using every 4th gridpoint here, which is generally not a good way to coarsen data. It's just for testing.
Is moving to a coarser horizontal resolution, e.g. 1°, an option for you?

I see! The coarser resolution might not be a problem given that I'm trying to find high amplitude Rossby wave propagation/ breaking events. I will implement this tomorrow and let you know how it goes. Thank you so much for the timely responses.

Thanks @chpolste for the quick solution! Someone emailed me about the same issue with 0.25 deg resolution dataset before, and ended up with the solution of coarse sampling too. @hs2112 please keep us posted if that works on your side. Thanks!

@chpolste @csyhuang Confirming that the reference state converged for 1 degree resolution. Could you also confirm if it is possible to compute LWA for a subset of NH, say beyond subtropics, instead of the entire globe?

Hi @hs2112 , yes it is possible to compute LWA for a subset of NH. In Neal et al 2022, we compute LWA using a reference state computed using data points from 5 deg N to the pole. You can refer to scripts/nhn_grl2022/sample_run_script.py for the usage. Here I use the data you provided to compute Qref and Uref:
import numpy as np
import xarray as xr
import matplotlib.pyplot as plt
from hn2016_falwa.oopinterface import QGField
data = xr.open_dataset("export_tstep_0.nc")
coarsened = data.isel({
    "latitude":  slice(None, None, 4),
    "longitude": slice(None, None, 4)})

uu = coarsened.variables['u'][:, ::-1, :]
vv = coarsened.variables['v'][:, ::-1, :]
tt = coarsened.variables['t'][:, ::-1, :]
xlon = np.arange(360)
ylat = np.arange(-90, 91, 1)
plev = coarsened.variables['level']
kmax = 17
dz = 1000.
height = np.arange(0, kmax)*dz

eq_boundary_index = 5  # use domain from 5N to North pole
qgfield_object = QGField(xlon, ylat, plev, uu, vv, tt, kmax=kmax, dz=dz, eq_boundary_index=eq_boundary_index)

qgpv_temp, interpolated_u_temp, interpolated_v_temp, interpolated_avort_temp, interpolated_theta_temp, \
static_stability_n, static_stability_s, tn0, ts0 = qgfield_object._interpolate_field_dirinv()

qref, uref, tref, fawa, ubar, tbar = qgfield_object._compute_qref_fawa_and_bc()

fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 4))
cs1 = ax1.contourf(ylat[90:], height, np.swapaxes(qref, 0, 1), 30, cmap='rainbow')
ax1.set_title('Qref')
cbar1 = fig.colorbar(cs1)
cs2 = ax2.contourf(ylat[90+eq_boundary_index:], height, np.swapaxes(uref, 0, 1), np.arange(-20, 30, 5), cmap='rainbow')
ax2.set_title('Uref')
cbar2 = fig.colorbar(cs2)
plt.show()

The resultant Qref and Uref (only available from 5N to the North Pole) looks like:

Is that something you want to do?

Great! Sorry I go caught up with some things; was not able to reply. So the reference states are a function of the horizontal extent of data provided?
Had another quick query -
Currently I'm inputting the entire vertical extent upto a certain level, so that the library can interpolate to peudoheight levels.
But I'm interested in vertical levels of 10km to 12km and the pressure levels corresponding to these vertical levels are between 200 and 300hPa. So can I just upload these levels instead of the entire vertical extent upto 20hPa?

Hi @hs2112 ,
The current version of package only supports LWA budget analysis over the whole vertical column(so the entire vertical extent of data is necessary, since the vertical integral function is written for this configuration).
What analysis do you want to with levels between 200 and 300hPa? If you just want to LWA for those levels, you can use those levels of QGPV to compute the wave activity via the eqvlat_fawa function and the lwa function (a bugfix version is available on the release0.7.0 branch):
https://github.com/csyhuang/hn2016_falwa/blob/c6df73fd142f1a18beccff8137410046857ab9c5/hn2016_falwa/basis.py#L11
Is this something you're interested in computing? In any case, let me know.
-Clare

Hi @csyhuang ,
So, compute_lwa_and_barotropic_fluxes outputs LWA at various pressure levels; does it give vertically integrated values up to each level?
I am looking to compute LWA corresponding to any given pressure level(s) only. I will eventually look at the vertical structure of wave activity itself.
Can I use QGPV for inputting to eqvlat_lwa, and is there a function to calculate QGPV using u, v, z, t fields with minimum hard-coding involved? Can't I use qgpv_eqlat_lwa directly?

@hs2112 Sure. You can use qgpv_eqlat_lwa as shown in Example_qgpv.ipynb to compute LWA. qgpv_eqlat_lwa is just a wrapper calling eqlat_lwa. You can look at the code base to see what it does: https://github.com/csyhuang/hn2016_falwa/blob/9acd8106cf8a359979d4fb6c763ced1cd91d666b/hn2016_falwa/wrapper.py#L197
Can you state clearly what you want to calculate? If you only want to look at LWA without looking at the fluxes (which requires computation of Uref, and that requires data for all pressure level), this is what you can do:

Put in the data among pressure levels just below and above 200-300hPa into QGField to make sure static stability is properly calculated for layers between 200-300hPa.

Still do the 3 steps:

qgfield_object.interpolate_fields(return_named_tuple=False)
qgfield_object.compute_reference_states(return_named_tuple=False)
qgfield_object.compute_lwa_and_barotropic_fluxes(return_named_tuple=False)

and access the QGPV(z, lat, lon) and LWA(z, lat, lon) calculated: qgfield.qgpv and qgfield.lwa. The values are on regular psudoheight grid. You can get the values for your desired pressure levels my interpolation back to your original grid.
(Note that if you do what I suggested above, Uref Tref and other flux quantities are not right and shall not be analyzed.)

"If you only want to look at LWA without looking at the fluxes (which requires computation of Uref, and that requires data for all pressure level)," - How do you calculate LWA without Uref?  Since it is a function of (y,z) I thought it should be readily calculable for each pressure level separately (NH16).
This is exactly what I want to do - I want to compute LWA corresponding to an array of pressure levels between 900hPa and 200hPa. I want to analyze wave activity at different pressure levels (main interest in higher levels for establishing stationary wave activity during a certain period), instead of vertically averaged. Hence, I was asking about the LWA field that is an output of compute_lwa_and_barotropic_fluxes, which is available at pressure levels.
So, basically, I just want to use the QGPV-based LWA formulation per pressure level established in NH16. Of course, I would like to analyze the contributing fluxes too.

I do currently have daily average 1x1 outputs from compute_lwa_and_barotropic_fluxes, and have been looking at the lwa field between 5N and 35N over and around South Asia (largely between Africa and west Pacific), where the signals are low amplitude and the field is rather noisy. Would the 276 hPa field be accurately represented, or largely bear the low level signature due to the vertical averaging? It would be helpful if you could comment on this.
My concern is that the vertical structure of the wave field might not be barotropic, so wanted to implement LWA ideas from NH16, to each pressure level separately. Does that make sense?

@hs2112 : Thanks for explaining your use case. Now I get your concern.
First of all, the functionalities available in our python package were only the ones we have used so far in our publications. My PhD thesis focuses on blocking over the ocean (blocking is the main subject of HN16, NH18 and NHN22 as well), and the structure of blocking is largely barotropic (an example is shown in HN16 Fig. 10), so it suffices to look at the barotropic component of fluxes for those studies. It does not mean that other studies of different focus shall look at only barotropic components.
Your case involves grid points over the continents, so the lower level fields are likely noisy or unreliable. (Are you using reanalysis data? It also depends on the treatment of topography in data assimilation process.) Computation of LWA per pressure level only requires Qref but not Uref:

Since layerwise LWA (not vertical average) is also computed within the method compute_lwa_and_barotropic_fluxes, you still have to run it but you can simply take qgfield.qgpv and qgfield.lwa from qgfield object, which are 3D variables in (z,y,x), and discard the rest that you don't need. you can then analyze QGPV and LWA over only the upper levels.
Would that work?

@csyhuang
Yes, I am using ERA5 reanalysis that uses a seemingly advanced Land Data Assimilation system, but I agree and am happy looking at upper levels.
I am already using qgpv and lwa generated by calling interpolate_fields and compute_lwa_and_barotropic_fluxes on the qgfield object, and following the 3 steps you suggested as pasted below. So, no need to make any changes (not using qgpv_eqlat_lwa separately) for layerwise LWA.

Did you say that the fluxes and some other fields may not be reliable to analyze layerwise?
Would you recommend trying out any other changes like using actual geopotential heights instead of the pseudoheight hardcoding?

qgfield_object = QGField(xlon, ylat, plev, uu, vv, tt, kmax=kmax)

qgpv[tstep, :, :, :], interpolated_u[tstep, :, :, :], interpolated_v[tstep, :, :, :], \
    interpolated_theta[tstep, :, :, :], static_stability = qgfield_object.interpolate_fields()

qref[tstep, :, :], uref[tstep, :, :], ptref[tstep, :, :] = \
    qgfield_object.compute_reference_states(northern_hemisphere_results_only=False)

adv_flux_f1[tstep, :, :], \
adv_flux_f2[tstep, :, :], \
adv_flux_f3[tstep, :, :], \
adv_flux_conv[tstep, :, :], \
divergence_eddy_momentum_flux[tstep, :, :], \
meridional_heat_flux[tstep, :, :], \
lwa_baro[tstep, :, :], \
u_baro[tstep, :, :], \
lwa[tstep, :, :, :] \
    = qgfield_object.compute_lwa_and_barotropic_fluxes(northern_hemisphere_results_only=False)

Thanks!

I think my main problem is that the LWA field looks noisy. When I dig to see why, it roughly reflects the noisy QGPV field computed in compute_lwa_and_barotropic_fluxes, and the meridional extent also looks larger than indicated in QGPV. Pasting the QGPV and LWA plots along with ERA5 PV which is much more coherent.


Division by 0

Describe the bug
If a player haven't died or haven't killed anyone there is a couple of cases with division by 0 for me in stats.py at lines 204 and 221. I fixed it for myself by adding an if at the end of the rows like this:

204: player_statistics[player]["kills"] / player_statistics[player]["deaths"] if player_statistics[player]["deaths"] != 0 else player_statistics[player]["kills"],
221: player_statistics[player]["hs"] / player_statistics[player]["kills"] if player_statistics[player]["kills"] != 0 else 0, 2

To Reproduce
Steps to reproduce the behavior:

Record a session where the gamer doesn't die or doesn't kill (often only happens with very few rounds recorded)
Analyze the recorded .dem
demo_parser = DemoParser(...)
data = demo_parser.parse()
See error

Expected behavior
Not a division by 0 error, either nothing is calculated or a proper value is chosen based on what is looked for like in the example solution

Hi @nicodemuz-zz, thanks for spotting and reporting this error!
How crucial is this error to your workflow? If it isn't crucial, I'll just wrap it up in the next release. If it's important, I can make a separate release just for this.

Not crucial at all, so wrap it up for a future release. :)

Keeping open so I don't forget!

Fixed in #160


[Semantic API] Incorrect ModuleID for typedesc types

Description:
Subject
TargetType link is not working in api docs due to this issue.
https://docs.central.ballerina.io/ballerina/http/1.1.0-beta.1/clients/FailoverClient#post
Steps to reproduce:

Have the following code

# The types of data values that are expected by the HTTP `client` to return after the data binding operation
public type TargetType typedesc<aClass|State>;

public function testFunc(TargetType targ) {
}

Get the symbol of TargetType in function parameter targ in testFunc.
The ModuleID will be lang.annotations

Affected Versions:
sl-beta1
OS, DB, other environment details and versions:
Related Issues (optional):

Suggested Labels (optional):

Suggested Assignees (optional):

I believe this is due to #26284

Checked in 2201.0.0 and this now returns the correct module ID. Hence closing this.


Rest swagger specification : Specify mixed type for executions input

An execution in CARMIN can take any number of key/value pairs with different type possible for the value.
There is a problem to express this loose typing in the swagger specification. In theory JSON schema allow this (with the "oneOf" keywork or by multi-typing a property) but swagger override the standard and forbid those solution.
I've looked a lot about this issue and here are the solutions I have found.
1. use the additionalProperties flag
This flag in a json schema allow the free extension of an object and to put anything in it.
The schema could be
MultiValue:
    type: object
    additionalProperties: true

That would allow
{ "param1" : "string value", "param2" : 42, "param3": ["array item1", "array item2"] }

But would also allow unwanted things like nested objects.
2. use discriminator keyword
"discriminator" is a keyword added by swagger that do not exist in the JSON schema standard. They added it to allow inheritance and polymorphism to cover the fact that they do not support the standard JSON schema solutions (fine) (see http://swagger.io/specification/#schemaObject after the tables).
This allow this schema :
  MultiValue:
    type: object
    discriminator: type
    required:
    - type
    properties:
      type:
        type: string
  StrValue:
    allOf:
    - $ref: '#/definitions/MultiValue'
    - type: object
      properties:
        value:
          type: string
  IntValue:
    allOf:
    - $ref: '#/definitions/MultiValue'
    - type: object
      properties:
        value:
          type: integer

And the type MultiValue could accept these json :
{ "type": "StrValue", "value":"any string value" }
{ "type": "IntValue", "value": 42 }

3. use base object
We could use the schema to accept any value type :
MultiValue:
    type: object
    properties:
      valueStr:
        type: string
      valueInt:
        type: integer
      valueArray:
        type: array
        items:
          $ref: '#/definitions/MultiValue'

that could accept these json :
{"valueStr": "string value"}
{"valueInt": 42}

But could not ensure that one and only one of the property could be set, which is not desired.
Conclusion
The 1st solution is the best for the syntax : schema very easy, json very descriptive. But the schema is too open.
The 2nd solution describe exactly what we want. But both the schema and the json are cumbersome.
The 3rd solution is a compromise. It put responsibility on the client (and the server) to validate the data.
I think the choice should be made relative to the reason why we chose swagger to describe the API :

standard format to describe the api in a human format
standard format to describe the api to be used by tools

For human comprehension, 1 and 3 are clearly OK but 2 seems out of the game
For tooling, these need looking into depth at swagger tools but my early conclusions are:

3 is very basic and will always be supported (but allow setting bad data)
2 is the swagger solution but it seems that no swagger tool support it (which is understandable given the complexity of the solution but looks like a design flaw)
1 is very open but the swagger editor (http://editor.swagger.io/) supports it ! I dont know about the other tools.

My choice would depend of the last element. I will look into the swagger tools that we may use (the Codegen and UI) if it supports 1 or 2.
1 is my favorite, so if it is well supported i think we should use it. If not i think we should choose 3.
Well that was long and quite complex, so dont hesitate to ask clarification on some points !

I vote the solution 1 for the mixed type as well. Hope it could be supported by the plugin pyramid_swagger which is used in CATIDB.
There is another problem that I would like to let users know the key could be chosen from keywords "Play", "Kill", "Start" according to the carmin.yaml. I think we need to define different several key/values pairs for the different webservices?

I also vote for solution 1 if it can be properly supported in swagger. Proper input validation probably requires a separate JSON schema anyways, more specific to the pipeline.

After a meeting, a consensus has been reached on the solution 1, and it will be integrated into the swagger specification.
We will look into several tooling (swagger Codegen, swagger UI and pyramid_swagger) to verify its support.


adding both 32 and 64 bit cross walk do not work

i tried a lot to implement it for both 32 and 64 bit in the android studio but could not get success
basically, i have added both 32 and 64 bit code in the gradle file like below
implementation 'org.xwalk:xwalk_core_library:21.51.546.7'
implementation 'org.xwalk:xwalk_core_library:21.51.546.7-64bit'
and have implement all the logic to add both .aar files by following the procedure explained in the like below
url for adding both 32 and 64 bit
i also tried by adding both in the dependencies manually
but it is working in the debug mode but do not in the release mode, when i run the project in the release mode it give me blow error
Program type already present: SevenZip.CRC

is there any one to help ?

The Crosswalk Project is no longer maintained: https://github.com/crosswalk-project. You might consider using something else.


WIP: add support for controling powerful time

not yet tested
not sure about the stats that where added (should I add powerful too?)

@swolix when you have some time could you please resolve the conflicts with main and add the property to the device as I've done for other features like in this one: https://github.com/cjaliaga/aioaquarea/pull/33/?
Happy new year!!

Hey @swolix, I continued your work on #38 and added you as co-author. Apologies for the rush but I needed to do it as I'm not sure how much time I'll have in the next few weeks.

@cjaliaga thanks for your effort, and happy new year to you too!


Permission to use material and brand for a workshop

We are organizing a workshop to raise some awareness around Elixir, it's not going to be monetized but there will be some amount of "commitment fee" that the attendants need to pay. Something around $5 usd.
We will be keeping it true to ElixirSchool and brand it as ElixirSchool Kuala Lumpur.. Kuala Lumpur being the city.
The idea is to mimic the ruby community's Ruby Girl's movement.

@doomspork is the owner and only person who can comment on if that is truly fair game.
@imranismail IMHO any beginner event shouldn't have a pay-wall for entry, its just silly. The point of these materials are to give everyone a free resource; and that doesn't accomplish that goal at all. But I'm not the owner/creator so my opinion is just that, some random guy's opinion.

@ybur-yug it's not a paywall really, just a commitment fee because of the limited slots available. I've held past Elixir events that are free but those are more on talks instead of workshop style.

Hi @imranismail!  I'm excited to hear you're interested in making in-person training available to people in your area.  I am more than happy to have you make use of the materials, but I must say I am not a fan of branding this an "Elixir School" organized a event when it is not at all affiliated with us.
If you were actively involved in Elixir School through translations or contributions I think I might feel a bit differently, but right now this kind of comes off as someone looking to benefit from the hard work of other people and I can't get onboard with that.
If you'd like to chat about this more, feel free to ping me on Elixir Slack @ seanc.

I've talked to @doomspork and resolved to branding it something differently.
For those interested, the small community is here: https://www.facebook.com/groups/elixirmy/
I'll update this issue once we've set up the event.
Thank you!


⚠ Bofrost.be has degraded performance

In 7b9e684, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1338 ms

Resolved: Bofrost.be performance has improved in f289d77.


Roll Flutter Engine from ae38c9585a61 to 510ecfa93a11 (1 revision)

https://github.com/flutter/engine/compare/ae38c9585a61...510ecfa93a11
2023-02-02<EMAIL_ADDRESS>Don't rely on timings for dimension_provider unit test. (flutter/engine#39343)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC bdero@google.com,rmistry<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

Commit queue failed; closing this roll.


Johnson2011 Alaska Deep soil carbon

[x]  Evaluate dataset
[x]  merge study data
[ ]  clean up profile data
[ ]  clean up layer data
[ ]  make graphics
[ ]  Write up summary report narrative
[ ]  Good commenting
[ ]  External review by someone else

First pass here, but needs to be double checked. da3b1b8f2985244247db8a7c8a63e01e00b0b72e

duplicate issue, #77