NETCDF HDf Error on CEDA JASMIN

On JASMIN when I try to run ESMValTool, I run into a problem with opening NETCDF files. I think it could be a problem with the HDF5 library. I load the esmvaltool module on JASMIN and tried to run the following test recipe:
#ESMValTool
#recipe_ukesm_test.yml
---
documentation:
  description:
    Recipe for testing ESMValTool with UKESM and other CMIP6 data
  authors:
    - swaminathan_ranjini

maintainer:
  - swaminathan_ranjini

projects:
  -ukesm

preprocessors:
  pp_global_mean:
    area_statistics:
      operator: mean

diagnostics:
  diag_test_netcdf:
    description: test netcdf
    variables:
      tas: #surface temperature
        preprocessor: pp_global_mean
        mip: Amon
        additional_datasets:
          - {dataset: UKESM1-0-LL,    project: CMIP6, exp: historical,  ensemble: r1i1p1f2,  start_year: 1850,  end_year: 1900, grid: gn}

    scripts: null

and end up with the following error message:
 File "/apps/jasmin/community/esmvaltool/miniconda3/envs/esmvaltool/lib/python3.9/site-packages/esmvalcore/_recipe.py", line 481, in _read_attributes     with Dataset(filename, 'r') as dataset:   File "src/netCDF4/_netCDF4.pyx", line 2330, in netCDF4._netCDF4.Dataset.__init__   File "src/netCDF4/_netCDF4.pyx", line 1948, in netCDF4._netCDF4._ensure_nc_success OSError: [Errno -101] NetCDF: HDF error: b'/badc/cmip6/data/CMIP6/CMIP/MOHC/UKESM1-0-LL/historical/r1i1p1f2/Amon/tas/gn/latest/tas_Amon_UKESM1-0-LL_historical_r1i1p1f2_gn_185001-194912.nc'
When I do a  ncdump -h  on the above file without loading the esmvaltool environment I am able to read the file. ESMValTool and ESMValCore are at version 2.2.0.
@valeriupredoi - I was wondering if you had seen something like this on JASMIN before? Any suggestions on how to fix this?
Thanks!

cheers @rswamina - I'll have a look at this today, it smells to me like I have to update the environment on JASMIN, thanks for raising this :beer:

Thanks @valeriupredoi . I just want to add that I did a fresh creation of the esmvaltool-private environment after a conda update and have the same issue. Which is why I thought to test with the module load as well.

cheers @rswamina - I have just replicated the hdf5 error on JASMIN with the central install, I am going to see what's the hap :+1:

OK this is a localized issue with UKESM1 only, other models/OBS sets work fine, am on it

right, it seems that that exactly particular file is corrupt, running this
import iris

from netCDF4 import Dataset

filenames = [
    "/badc/cmip6/data/CMIP6/CMIP/MOHC/UKESM1-0-LL/historical/r1i1p1f2/Amon/tas/gn/files/d20190406/tas_Amon_UKESM1-0-LL_historical_r1i1p1f2_gn_185001-194912.nc",
    "/badc/cmip6/data/CMIP6/CMIP/MOHC/UKESM1-0-LL/historical/r1i1p1f2/Amon/tas/gn/files/d20190406/tas_Amon_UKESM1-0-LL_historical_r1i1p1f2_gn_195001-201412.nc",
    "/badc/cmip6/data/CMIP6/CMIP/MOHC/UKESM1-0-LL/historical/r1i1p1f2/Amon/ta/gn/latest/ta_Amon_UKESM1-0-LL_historical_r1i1p1f2_gn_195001-201412.nc",
    "/badc/cmip6/data/CMIP6/CMIP/MOHC/UKESM1-0-LL/historical/r1i1p1f2/Amon/pr/gn/latest/pr_Amon_UKESM1-0-LL_historical_r1i1p1f2_gn_185001-194912.nc"
]

for filename in filenames:
    try:
        with Dataset(filename, 'r') as dataset:
            print("Dataset() OK", filename)
    except OSError as ex:
        print("Dataset() NOT OK", filename)
        print(ex)

c = iris.load_cube("/badc/cmip6/data/CMIP6/CMIP/MOHC/UKESM1-0-LL/historical/r1i1p1f2/Amon/tas/gn/files/d20190406/tas_Amon_UKESM1-0-LL_historical_r1i1p1f2_gn_185001-194912.nc")

it's the only file that returns a Dataset() error and doesn't load in iris either. I'll see if this happens with newer netCDF4 versions

So with the latest netCDF4 =1.5.6 this is an issue
import iris

from netCDF4 import Dataset

# This is the latest netCDF4
# Name                    Version                   Build  Channel
# netcdf4                   1.5.6           nompi_py39h36800e2_100    conda-forge

filenames = [
    "/badc/cmip6/data/CMIP6/CMIP/MOHC/UKESM1-0-LL/historical/r1i1p1f2/Amon/tas/gn/files/d20190406/tas_Amon_UKESM1-0-LL_historical_r1i1p1f2_gn_185001-194912.nc",  # problem file
    "/badc/cmip6/data/CMIP6/CMIP/MOHC/UKESM1-0-LL/historical/r1i1p1f2/Amon/tas/gn/files/d20190406/tas_Amon_UKESM1-0-LL_historical_r1i1p1f2_gn_195001-201412.nc",
    "/badc/cmip6/data/CMIP6/CMIP/MOHC/UKESM1-0-LL/historical/r1i1p1f2/Amon/ta/gn/latest/ta_Amon_UKESM1-0-LL_historical_r1i1p1f2_gn_195001-201412.nc",
    "/badc/cmip6/data/CMIP6/CMIP/MOHC/UKESM1-0-LL/historical/r1i1p1f2/Amon/pr/gn/latest/pr_Amon_UKESM1-0-LL_historical_r1i1p1f2_gn_185001-194912.nc"
]

for filename in filenames:
    try:
        with Dataset(filename, 'r') as dataset:
            print("Dataset() OK", filename)
    except OSError as ex:
        print("Dataset() NOT OK", filename)
        print(ex)

c = iris.load_cube("/badc/cmip6/data/CMIP6/CMIP/MOHC/UKESM1-0-LL/historical/r1i1p1f2/Amon/tas/gn/files/d20190406/tas_Amon_UKESM1-0-LL_historical_r1i1p1f2_gn_185001-194912.nc")

I am going to report this now

What will the solution be? Seems unusual that this happens with just that one variable.

they will have to regenerate the file or kludge it by adding a _dataset attribute in cdo or something, no idea :man_shrugging:

Have you confirmed in any way that the file is actually ok? Imho ncdump -h is not enough for this purpose since it really only reads a very limited set of metadata.

here are the comprehensive checks that @matthew-mizielinski performed on the file and his conclusion (cheers, Matt!):

 copied the file back from JASMIN to the MO and I can then load it fine using the netCDF4 library.

 Retrieved the file from MASS and I can also load it fine

 Confirmed two separate checksums (adler32 and md5) and the file is identical to the one we have here that was submitted originally.

 Ncview on JASMIN loads this file fine

 ncdump (no -h argument) fails straight away on JASMIN, but gives the expected stream of data at the MO

 Next file in the dataset<PHONE_NUMBER>12) seems unaffected.

 Copied the file to my home directory on JASMIN and the file can be read fine.

Given the last point (copying the file elsewhere on JASMIN allows the file to be read) suggests to me that something funny is going on with the storage.

I can confirm that I am able to use this file with ESMValTool on JASMIN now - both under the module loaded environment and with my installation of esmvaltool-private. @valeriupredoi - if you are ok after checking, we can close this issue. Thanks for your help!

@agstephens wrote:
This is indeed very strange. I have _fixed_ the problem by copying the file to another file system, where it has the IDENTICAL md5sum, and then re-ingesting it into our archive (where it has the IDENTICAL md5sum).
Many thanks, Ag! This solved the file problem. I propose we close this now and keep an eye on these occurrences in the future :+1:


Move iteration onto their own traits in algorithms?

It would be nice to decouple the GeometryArrayTrait from its associated types. And maybe one way to do that would be to create traits to handle iteration and conversion to geo/geos?

Iteration is already on their own traits... https://github.com/geoarrow/geoarrow-rs/pull/235 does a lot of work to remove associated types from GeometryArrayTrait


GhostText can't handle multiple file tabs in Codecademy

Codecademy's HTML & CSS track lets the user switch between separate files for HTML and CSS.  I'm not sure whether the same textarea is used, but the file contents are replaced when switching tabs, and the same part of the screen stays highlighted as if it were connected to GhostText.  However, the new file contents are not shown in Sublime Text.
Once GhostText is connected, it binds to the active file, and won't edit the other file unless the page is entirely reloaded.  For instance, disconnecting, switching files and reconnecting just connects to the original file.
This also means GhostText can keep updating a file which is no longer visible (eg it keeps editing HTML after the user switches the view to CSS).  (When this happens, Codecademy doesn't notice the changes, unless the user switches to the file again and makes another change in Sublime Text.)

Thanks for the bug report.
It seems they reuse the same Ace editor instance for all tabs, I can't promise you that this is fixable, but we will give our best.

Moved to https://github.com/GhostText/GhostText/issues/84


chore: deliver styles to distro

Closes #2
This delivers all needed styles for the separate Modelers. The styles can be imported separately or together via [type]-modeler.css.
import 'camunda-bpmn-js/dist/assets/base-modeler.css';

import 'camunda-bpmn-js/dist/assets/camunda-cloud-modeler.css';

import 'camunda-bpmn-js/dist/assets/camunda-platform-modeler.css';

I'd vote against simply bundling all the sheets together, but importing them into several available style sheets. That's why I decided on this direction. I'm open to discussions to handle it differently.

@camunda/modeling-dev any chance to get a review here?


Enabling to add watermarks on pdf.js pages

This would be a nice additional feature:
Enabling to add watermarks on pdf.js pages especially printing.
Or perhaps is there a variable where we can get references to all the canvas pages that will be printed? so we can add our own watermarks?

This is out-of-scope for the PDF.js library itself, but may indeed be useful for custom solutions. Recently the printing code has been refactored, so you should be able to adapt the printService to add any watermarks to printed canvases.

thank you for this tim. i have one tiny question. i am using the stable version of pdf.js yet when i search through the code, there is no string of printservice that can be found.
is this not found in viewer.js?

It's introduced in #7697, so it's not in the stable version yet. In the stabl version I think there is beforePrint.

hi tim. thank you. i have checked beforePrint before but i am not quite familiar since it returns a PDFPageView but when beforePrint() is called, it does not seem to return anything looking at the current codebase.
i am planning to also listen to the beforeprint event, is it possible to have it return that pdfpageView object in the next version? i think adding canvas with the watermark inside every canvasWrapper can do the trick.
please advise.

hi tim. i have a question. i have managed to add watermarks to the pages during print preview and have verified during printing as well. but this is in chrome.
when i used firefox and IE  that do not have print preview panes, beforePrint does not get called nor is there any beforeprint event that gets fired. for this scenario, where should i look at in order to add watermarks to the pages?
please advise

Strange, because beforePrint should always be called to prepare the pages for printing. If not, the pages cannot be printed properly. I have done testing in Firefox/IE before and beforePrint was always called, most notably because changes in beforePrint (e.g., https://github.com/mozilla/pdf.js/commit/29947b5a570e07ca2ea2c2f14d5a3c7bcb838d4c) caused different printing in IE. If it's not called, probably something else is wrong, but it's not possible from the presented information to say for sure.
You could try using the current master branch with refactored printing code and see if that improves things. There is also https://github.com/mozilla/pdfjs-dist, a generic build of PDF.js that gets updated everytime we push a commit to the master branch, which you may consider.

in your end, does firefox and ie show no print preview pane as well? and beforeprint is still dispatched?
i just did it like this
window.addEventListener('beforeprint', function() {
console.log('beforeprint');
});
works in chrome, print preview pane shows up. but both firefox and ie shows the print popup window directly, no print preview.

Firefox uses mozPrintCallback, so that is a difference compared to Chrome/IE. Recently this has been changed to use mozPrintCallback only for the Firefox add-on and not for the regular viewer in Firefox anymore, because of problems with mozPrintCallback. If you have not applied #7697, mozPrintCallback will also be used for the regular viewer in Firefox, so you can try to see if that is used. I'm not too familiar with mozPrintCallback, unfortunately, and the new patch reduced its usage to make it more uniform for all browsers.

hi tim. thank you. i'll check that out and give an update after getting some findings.

hi tim. apologies, firefox does call beforeprint. not used to using firefox all the time so i was looking at the wrong tab window in dev console. instead of logging, i was looking at JS. tsk.
case closed. thank you again.

hi tim. got another question regarding this. does firefox work differently compared to IE and chrome because of the mozPrintCallback? What I did was to get hold of the canvas inside the printContainer and place my watermarks there.
It works in Chrome and IE (no print preview but i confirmed it works when i printed it). But in firefox, it does not include my watermarks.
Also, I noticed in Edge browser it does not print preview the pdf pages correctly. Sometimes it works, sometimes it does not. is this an ongoing issue yet? please advise

Yes, Firefox may behave differently because of mozPrintCallback. Recently we fixed a lot of printing issues (including IE/Edge problems) and made the printing code more the same in all browsers. Now, only the Firefox add-on uses mozPrintCallback, whereas the regular viewer does not anymore, to make it more in line with other browsers. I would recommend you to use the latest PDF.js master branch or, better yet, pdfjs-dist to get all these fixes. The current release version does not contain them yet.

is there be a plan to streamline all browsers to use the same including firefox? i figure i don't need to add code specifically for firefox if this is the plan going forward.
and also because i still could not figure out when to add the watermarks in all the canvas of printContainer in Firefox. the beforeprint does show that printContainer has all canvas inside but i believe it has not yet been rendered so my function to display watermarks gets wiped out when pdf.js renders the pages.

Everything is already streamlined, except for the Firefox add-on version, but you'll probably not use that when developing a custom solution (the add-on is only intended for inclusion in Firefox itself).
For mozPrintCallback, I think you can check https://github.com/mozilla/pdf.js/blob/master/web/firefox_print_service.js#L70, but, as said, this should now only be for the add-on version which is probably not relevant for a custom solution.

ok. guess ill just disable the print button for firefox.
before i tried the ads.on, i tried printing it and i got no watermarks unlike ie and chrome.
thank you.

Hi @chitgoks Have you find any workaround for enabling watermark?

@GJ2511 yes. i have my own solution.

COuld you please provide some reference for help,
As per my understanding mozPrintCallback doesn't get fired beforeprint( as Firefox don't provide print preview ).

you can install the print preview plugin in firefox so you will be able see your print preview in firefox.
what i did is get hold of the canvas of pdf.js and draw my watermarks there.

At what place you did that?
This is what I did so far (it's working for IE and Chrome)
function applyWatermarkToPrintedDocument(event) {
        //Remove listener so that the function will not cause problems when mozilla
        //attempts to use the real mozPrintCallback
        window.removeEventListener('beforeprint', this);

        var canvases = document.querySelectorAll('canvas');
        for (var count = 0; count < canvases.length; count++) {
            if (canvases[count].mozPrintCallback && !canvases[count].renderedWatermark) {
                //Override the default moz callback with new one.
                canvases[count].mozPrintCallback = mozCallbackOverride(canvases[count].mozPrintCallback);
                canvases[count].renderedWatermark = true;
            }
        }
    }
window.addEventListener('beforeprint', applyWatermarkToPrintedDocument, false);

function mozCallbackOverride(defaultMozPolyfillCallback) {
        var originalCallback = defaultMozPolyfillCallback;

        function newCallback(obj) {
            //Override the done callback to we are able to determine when rendering is complete. Use closure to retain
            //reference.
            obj.done = (function() {
                //grab the done object, for later calling
                //grab the contextObj
                var contextObj = obj;
                contextObj.done1 = obj.done;
                return function() {
                    isPrinting = true;

                    //Render watermark onto printing canvas
                    var watermarkSizes = calculateWatermarkSizes(contextObj.context.canvas);
                    watermarkSizes.canvas.context.save();
                    layoutRegion(watermarkSizes.canvas.context, watermarkSizes.top);//Here I am adding watermark
                    layoutRegion(watermarkSizes.canvas.context, watermarkSizes.middle);
                    layoutRegion(watermarkSizes.canvas.context, watermarkSizes.bottom);
                    watermarkSizes.canvas.context.restore();
                    isPrinting = false;

                    //Callback done object
                    contextObj.done1();
                };
            })();

            //fire original callback with new overridden object.
            originalCallback(obj);
        }

        return newCallback;
    }

i dont know how to override those functions in pdf.js since i dont understand their code. the only way that i did was to modify and add code in viewer.js. this would mean manually adding the changes for every new version of pdf.js
look at renderPage in viewer.js. it returns the canvas context. use that to draw your watermarks.
listening to beforeprint and afterprint didnt work out for me.

Oh ok, thanks for your help man :)

@GJ2511 can you email me at gmail. username is same. i wish to ask you something.

@GJ2511 did you manage to make it work in firefox?

Hello, could you explain me how to add watermarks? I am using last version of pdf.js
Thank you!


Parse error if imported module contains private instance fields

For example, fetch-blob uses private instance fields (available in Node.js v12+):
https://github.com/node-fetch/fetch-blob/blob/4747497b52c0d2b7e360b2281e1b51b779fc333c/index.js#L38
So if you try to lint a module importing it, with import/default, import/namespace, etc. rules:
import Blob from 'fetch-blob';

You get ESLint errors like:
Parse errors in imported module 'fetch-blob': Unexpected character '#'

For example:

@jaydenseric can you share the relevant code form 'fetch-blob' and what your ESLint config looks like? We use private fields in only one place in our project, but it works fine for us.

Try using @babel/eslint-parser as your parser with the following plugins configured:
parser: '@babel/eslint-parser',
  parserOptions: {
    sourceType: 'module',
    ecmaVersion: 2021,
    ecmaFeatures: {
      jsx: true,
      impliedStrict: true,
    },
    plugins: ['classPrivateProperties', 'classPrivateMethods'],
  },

And make sure your Babel config includes @babel/plugin-proposal-private-methods. I assume it does already since you're presumably able to compile your code - if you're using Babel.
In any case, I don't think this is related to this project per se, but I'm happy to try to help you get this working if what I've mentioned isn't enough.

The default parser doesn’t yet support ES2022 syntax, which includes private fields.

src/App.js
Line 1:23:  Parse errors in imported module './components/Stopwatch': Line 105: Unexpected token
103 | }
104 |

105 | default export Stopwatch


Plugin list update

Not sure where to post this so I'll just place it here.  Please update the plugin list with the new locations on github.
Case insensitive: https://github.com/adigitalife/yourls-case-insensitive
Allow aliases: https://github.com/adigitalife/yourls-allow-aliases
Check URL: https://github.com/adigitalife/yourls-check-url
Change error messages: https://github.com/adigitalife/yourls-change-error-messages
Fix long URLs: https://github.com/adigitalife/yourls-fix-long-url
HTTP Proxy: https://github.com/adigitalife/yourls-http-proxy

All updated ! Thanks


add SwiftyXML to README.md

Project URL

Description

Why it should be included to awesome-ios (optional)
Checklist

[ ] Only one project/change is in this pull request
[ ] Addition in chronological order (bottom of category)
[ ] Supports iOS 9 or later
[ ] Supports Swift 3
[ ] Has a commit from less than 2 years ago
[ ] Has a clear README in English

          1 Error

:no_entry_sign:
Found 2 link issues

Link issues by awesome_bot

Line
Status
Link

956
301
https://github.com/chenyunguiMilook/SwiftyXML.git redirects tohttps://github.com/chenyunguiMilook/SwiftyXML

2082
301
https://buddybuild.com/ redirects tohttps://www.buddybuild.com/

  Generated by :no_entry_sign: danger

Hey @chenyunguiMilook, thanks for contributing and good job on the library.
I'd ask you to fix the following issue before we can approve.

@lfarah now SwiftyXML support cocoapods, carthage, swift package manager~

You're fast! Thanks for contributing 🚀 🚀 🚀 🚀


update to v0.15.5-beta

Any backwards incompatible changes upstream?

Any backwards incompatible changes upstream?

no. very minor upstream bump... mostly bugfixes


Simpler syntax

Hi,
This is such a great idea that I've decided to port it to rust because I need it as a library rather than a command-line tool (BTW, let me know if you want me to rename my version of the library)
The question is why there are semicolons? All javascript interpreters allow to skip them in all the cases in this subset of a javascript. I've decided to omit them, to have less visual noise, and make it easier to use cut and awk for the output. Is there any strong reason for your choice?

Cool! I've not got much past the "hello world" stage with rust so it'll be really interesting to look at your source.
I think it would be a good idea to have a slightly different name for the package - not because I'm precious about it, but to make googling easier for users. Perhaps 'rust-gron' as per your license? :)
As for the semicolons: I've always been fairly strict about using them when writing JavaScript, mostly because I learnt JS in the days of IE5. That is to say: it's mostly habit. I'll look at maybe adding a flag to disable them if it gets enough interest.
Thanks!

I haven't really had any interest in this so I'm going to close the issue.
I'm not saying never, but certainly not for now.


[Bug] Actual package required

I am running the latest cargo debstatus on testing for rnote. The expected behavior would be t
for it run as usual. Instead, I get the following error:
cargo debstatus
Error: this command requires running against an actual package in this workspace

It worked fine before for other packages. I don't know what the root cause is here.
Cheers

It's because the root Cargo.toml doesn't define a crate, only the workspace:
[workspace]
members = [
    "rnote-compose",
    "rnote-engine",
    "rnote-ui",
    "rnote-fileformats",
]
exclude = [ "./piet" ]

[profile.dev]
opt-level = 2
debug = true

[profile.release]
opt-level = 3

The main crate seems to be in rnote-ui/, running it from that subfolder works for me. :)


Test Issue created at Mon, 22 Mar 2021 11:09:44 GMT

Test issue body  Mon, 22 Mar 2021 11:09:44 GMT

Comment on issue at Mon, 22 Mar 2021 11:09:46 GMT


Getting BadRequest (ExtendedCode: 51008) with adding Connection Strings to WebApp. Possible custom policy issues

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Versions

terraform: v1.3.5 (also tested with v1.4.5)
azure provider: v3.53.0
module: 3.3.0
Description

When trying to add Connection Strings to the Web App we are getting Status=400 Code="BadRequest".
{
  "Code": "BadRequest",
  "Message": "The parameter properties has an invalid value.",
  "Target": null,
  "Details": [
    {
      "Message": "The parameter properties has an invalid value."
    },
    {
      "Code": "BadRequest"
    },
    {
      "ErrorEntity": {
        "ExtendedCode": "51008",
        "MessageTemplate": "The parameter {0} has an invalid value.",
        "Parameters": [
          "properties"
        ],
        "Code": "BadRequest",
        "Message": "The parameter properties has an invalid value."
      }
    }
  ],
  "Innererror": null
}

As soon as the TLS policy AppService append sites with minimum TLS version to enforce. is disabled the Connection Strings gets created.
Steps to Reproduce

Create WebApp with Connection Strings:

resource "azurerm_windows_web_app" "web_app" {
  name                = "web_app_62"
  resource_group_name = azurerm_resource_group.rg.name
  location            = azurerm_service_plan.web_app_service.location
  service_plan_id     = azurerm_service_plan.web_app_service.id
  https_only          = true
  app_settings        = { "minTlsVersion" = "1.2" }

  site_config {
    minimum_tls_version     = 1.2
    ftps_state              = "Disabled"
    http2_enabled           = true
    scm_minimum_tls_version = 1.2
    always_on               = false
  }

  lifecycle {
    ignore_changes = [
      tags["cost_centre"], site_config
    ]
  }

  connection_string {
    name  = "ConnStr"
    type  = "SQLServer"
    value = "key: value; key1: value"
  }
}

Screenshots

Additional context

Landing Zone deployed with archetype_id: es_corp_custom_lz.
For testing we removed the policy Append-AppService-latestTLS from the initiative and it works . It also works when the "effect" parameter is set to "Disabled" instead of "Append" during the assignment of the policy.

Hi @har-singh
This sometimes happens when a definition is recently created.
This is caused by eventual consistency in the platform.
You can either retry the apply or try increasing the create delay.
We are working to see how we can address this in the next version.

Hi @har-singh
Let me see if I can reproduce this.

Hi, I have been testing this my work colleague. Whilst testing noticed that the labelling in the assignment maybe incorrect:


[Bug] Dependency injection not working for HEAD requests

Describe the bug
Sanic automatically creates HEAD endpoints, but dependencies are not injected into these endpoints, causing 500 errors when users HEAD these endpoints.
Screenshots

To Reproduce
from sanic import Sanic
from sanic.response import text

app = Sanic('test')

class Foo:
    pass

app.ext.add_dependency(Foo, lambda *_: None)

@app.route('/download')
async def handler(req, foo: Foo):
    return text('file')

if __name__ == '__main__':
    app.run(port=4321, debug=True)

And then curl -I http://<IP_ADDRESS>:4321/download.
It shows an HTTP 500 response. Sanic reports the following error:
TypeError: handler() missing 1 required positional argument: 'foo'
Expected behavior
It should receive an HTTP 200 response.
Environment (please complete the following information):

OS: Windows-10-10.0.19045-SP0
Browser: curl 7.83.1
Version: Sanic 22.9.0, sanic-ext 22.9.0, sanic-routing 22.8.0

Additional context
Many downloaders send a HEAD request before actually downloading. This bug breaks these downloaders.

Are there any update on this issue? #148 and this issue are causing many errors on my website, so currently I have to opt-out sanic-ext to avoid the website being broken.
Since both issues have not received any comment for >1 week, I would like to know whether sanic-ext is still actively maintained.

sanic-ext is actively maintained by the sanic organization. It is recommended for production use you stick with the LTS releases which have a higher priority. We will attempt to address this before the 22.12 LTS release.
https://sanic.dev/en/org/policies.html#release-schedule

Yes, I am using the LTS release but both this issue and #148 are found in sanic-ext==21.12.3.

This will be resolved in the December 2022 release.

@xmcp gotcha, please make sure to provide accurate info - you indicated using this version:
Version: Sanic 22.9.0, sanic-ext 22.9.0, sanic-routing 22.8.0
In the ticket. We do try and prioritize fixing issues in the LTS releases.


Change color of links on hover to theme blue

Currently, links on hover have a green color, by the default MDL theme. It has been changed to WDL theme blue.

Live Demo

@sumedh123 Please review


BazelUpgradeFixes

This picks up where @kramasamy left off, these are the initial set of changes for getting the c++ targets working

Fixed by #3209


[rt.openssl.org #4032] [PATCH]  Fast 1536-bit modular exponentiation with the new VPMADD52 instructions

Migrated from rt.openssl.org#4032 (status was 'new')
Requestors:

<EMAIL_ADDRESS>
Attachments:

rsaz-1356-vpmadd.patch

From<EMAIL_ADDRESS>on 2015-09-10 10:42:40:
Hello everyone,

This patch is a contribution to OpenSSL.

It extends the patch "Id 3590" (from Nov 04, 2014; by Gueron and Krasnov) entitled "Fast modular exponentiation with the new VPMADD52 instructions". This contribution includes 1536-bit modular exponentiation (constant time) with the RSA fix to use these functions.

An efficient 1536-bit modular exponentiation is useful for speeding up RSA3072 (decrypt/sign).
RSA3072 provides 128 bit equivalent security (compared to 112 bits offered by RSA2048).

Significant performance gains can be expected on future processors that will support VPMADD52.
Details:

The underlying method is VNRMM which explained in [1].

VPMADD52 instructions (VPMADD52LUQ and VPMADD52HUQ) were announced in
https://software.intel.com/sites/default/files/managed/0d/53/319433-022.pdf
(see also the Intel(r) Software Development Emulator at
https://software.intel.com/en-us/articles/intel-software-development-emulator)

(currently, building the patch requires "binutils" version 2.24 (at least)., which can be downloaded from
http://ftp.gnu.org/gnu/binutils/)

Reference:
[1] S. Gueron, V. Krasnov: "New CPU instructions for speeding up modular exponentiation"
(to be published)

Developers and authors:
***************************************************************************
Shay Gueron (1, 2), Nir Drucker (1)
(1) Intel Corporation, Israel Development Center, Haifa, Israel
(2) University of Haifa, Israel
***************************************************************************
Copyright (c) 2015, Intel Corp.

---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

@dot-asm, comment or close?

No comments, so closing.


Maven pom with dependencies and version templating

Closes #37
Adds a basic Maven pom with main-class definition, templates the version for the Config class, and adds the org.json dependency.

Ahh thank you looks good so far =)


Consider changing governance voting rules to be private

Right now the GOVERNANCE.md section for core contributor induction says this about voting:

New core contributors will be decided based on a vote conducted in the #contributors-private Discord channel.

Upon further reflection, having the vote done in public can result in people not wanting to voice their dissent or disapproval.
Should we change this to instead require messaging a steward (only me right now) to place a vote? This should be for both yes and no. This also adds some anonymity if the vote fails.

I agree that votes should be made in private. I also think that the subjective aspect of messaging a steward vs. using a private voting bot adds a necessary human element.

@romefrontend/core-contributors Anyone want to draft a PR? Final decision/approval can be made there.


[ie/qqmusic] Fix QQMusic extractors for new URLs

IMPORTANT: PRs without the template will be CLOSED
Description of your pull request and other information

Overhauled extractors to support new urls of the site.

Replaced broken old API
Changed RegEx to match current URL patterns

(side note: API endpoint musicu.fcg works the same as musics.fcg but does not requires a signature, so I removed signing algorithm from earlier commit of this patch to https://gist.github.com/c-basalt/9bc9feeed8e39e928b6607ddf01840b5)
Fixes #9336
Template

Before submitting a pull request make sure you have:

[x] At least skimmed through contributing guidelines including yt-dlp coding conventions
[x] Searched the bugtracker for similar pull requests
[x] Checked the code with flake8 and ran relevant tests

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

[x] I am the original author of this code and I am willing to release it under Unlicense
[ ] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

[x] Fix or improvement to an extractor (Make sure to add/update tests)
[ ] New extractor (Piracy websites will not be accepted)
[ ] Core bug fix/improvement
[ ] New feature (It is strongly recommended to open an issue first)

Excuse me, how long time can merge?

@bashonly Thank you for your hard work these days. If there is any need to test this PRs, please feel free to mention me


helm get values missing output in v3

v3 is currently missing the --output flag and code for helm get values. This functionality needs to be re-enabled

cc @endzyme

@thomastaylor312 I'm able to run helm get values release-name and get the values in dev-v3 branch latest version. Do you mean only the code for output flag is missing?

Yep - what's missing is the availability to output values as json.

@endzyme Are you working on this or can I give it a try?

Just as an FYI for whoever does work on this, #6550 moves a few of the output things around. Shouldn't impact it too much, just a few line difference

Unfortunately I've not been able to make the time. Feel free! Would love to help review too if you think of it

Cool, I'll take a shot at this tomorrow or the day after and open a PR if I'm happy.
I'll be sure to request reviews then. That'd be a great help!

Feel free to ping me in the PR @Dean-Coakley! Also, I just merged #6550, so you should be good to go with that

fixed in #6625


How to draw multiple parallel links between nodes?

I'm having a difficult time getting multiple parallel links defined between two nodes. I'm trying to do something similar to what is show in the examples between nodes A and B. I'm using prometheus as a datasource and already worked past the queries needing a label issue. I can get two links to work by moving the anchors to top, bottom, or whatever. But that won't work when I try with four or more links.
Without changing the anchors, the links just stack on one another between nodes when added.
Help!

Any way to do this?


Is Flutter compatible with Pillarbox?

Detailed question
As a Flutter developer, can I use the Pillarbox player?
Can your answer be found elsewhere?

[X] I have searched existing issues and found no similar question.

[X] I have browsed the available documentation and found no answer to my question.

Quick answer: Yes it is!
Firstly, here is the Flutter documentation to follow.
Additionally, you will find attached a quick and dirty example.


$Skip query option not working

The $skip option is not woriking as a standalone option. I have used the demo heroku app:
https://odata-v4-server-demo.herokuapp.com/api-docs/
Filled it with some sample data just to test it. Response of:

is:

$top option works well:

but $skip does not work at all:

Is there something I am missing?

Hello @danielkamin
Thanks for your detail comments, yes, its a bug in fact, if you don't specify the $top value, the default 0 will be used in  sql LIMIT expression, as a result, the response will be empty.
just try the url https://odata-v4-server-demo.herokuapp.com/Teachers?$skip=1&$top=1000 (with $top parameter with a fixed value`
this bug will be fixed in the next patch
Regards
Theo

fixed, after 0.2.48 released

Hello,
I have come onto problem with functions such as:
contains,any,
tolower
startswith
Is there existing support for these options or is there plan to support
them?
Sincerely,
Daniel K.
wt., 8 cze 2021 o 21:54 Daniel Kamiński @.***>
napisał(a):

Thank you very much for this quick response and let alone this quick fix.
You are awesome.
pon., 7 cze 2021 o 13:19 Theo Sun @.***> napisał(a):

fixed, after 0.2.48 released
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Soontao/odata-v4-server/issues/340#issuecomment-855840550,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKRWCXXXZLAXHX2MUIEAEXDTRSTMPANCNFSM454ONWDA
.

Hi @danielkamin
Sorry, we don't have the plan to implement the  Canonical Functions in the odata standard.
Regards,
Theo


Annotation server config fix

Annotation server deployed and its configs updated. After merging this PR. Redeployment will fix the issues

I have reviewed the annotation container and annotation creation.
The text annotation creation appears to be compliant with the WADM vocabulary and LDP protocol. It can be integrated.


Deep copying a datatree somehow duplicates data onto child nodes

What happened?
Deep copying a DataTree object somehow duplicates data onto a child node (!)
What did you expect to happen?
Copying to return an identical subtree.
Minimal Complete Verifiable Example
In [17]: ds = Dataset(
    ...:             data_vars={
    ...:                 "foo": (["x", "y"], np.random.randn(2, 3)),
    ...:             },
    ...:             coords={
    ...:                 "x": ("x", np.array([-1, -2], "int64")),
    ...:                 "y": ("y", np.array([0, 1, 2], "int64")),
    ...:                 "a": ("x", np.array([4, 5], "int64")),
    ...:                 "b": np.int64(-10),
    ...:             },
    ...:         )
    ...: dt = DataTree(data=ds)
    ...: dt["child"] = DataTree()
    ...:
    ...: actual = dt.copy(deep=True)
    ...: print(dt)
    ...: print(actual)

Relevant log output
<xarray.DataTree>
Group: /
│   Dimensions:  (x: 2, y: 3)
│   Coordinates:
│     * x        (x) int64 16B -1 -2
│     * y        (y) int64 24B 0 1 2
│       a        (x) int64 16B 4 5
│       b        int64 8B -10
│   Data variables:
│       foo      (x, y) float64 48B 1.368 -0.8423 0.2045 0.7884 -0.5859 -1.201
└── Group: /child
<xarray.DataTree>
Group: /
│   Dimensions:  (x: 2, y: 3)
│   Coordinates:
│     * x        (x) int64 16B -1 -2
│     * y        (y) int64 24B 0 1 2
│       a        (x) int64 16B 4 5
│       b        int64 8B -10
│   Data variables:
│       foo      (x, y) float64 48B 1.368 -0.8423 0.2045 0.7884 -0.5859 -1.201
└── Group: /child
        Dimensions:  (x: 2, y: 3)
        Coordinates:
          * x        (x) int64 16B -1 -2
          * y        (y) int64 24B 0 1 2
            a        (x) int64 16B 4 5
            b        int64 8B -10
        Data variables:
            *empty*

Anything else we need to know?

What's weird is that they don't appear in the child node when we print(dt)! This implies some bug with coordinate inheritance / handling of empty nodes (cc @shoyer).

I think I know what's going on... stay tuned for a fix


Home button isn't active in alarm state

After hard reset GRBL indicate alarm and waitig for home sequence, if enabled.
Home sequence can be started in any time, bur GUI Home button isn't active in alarm state and confusing users.
Also bright red Reset button looks rezer like alarm sign, which should be hendeled before any next operation.

Sender version please? You may want to try the latest edge version.
Are you running grbHAL on your controller, if so which build date? I have made a few changes lately to improve user feedback and make startup more robust. These are available in the test branch. If for another controller then it is possible that some changes has to be made, I have to admit that I do not spend much testing with legacy grbl.

Also bright red Reset button looks rezer like alarm sign, which should be hendeled before any next operation.

Not sure what you mean by this. You want another color? I have chosen red because it works a bit like an E-Stop button, and is the same color as the Mach3 namesake.

Sender version please? You may want to try the latest edge version.
Are you running grbHAL on your controller, if so which build date? I have made a few changes lately to improve user feedback and make startup more robust. These are available in the test branch. If for another controller then it is possible that some changes has to be made, I have to admit that I do not spend much testing with legacy grbl.

Also bright red Reset button looks rezer like alarm sign, which should be hendeled before any next operation.

Not sure what you mean by this. You want another color? I have chosen red because it works a bit like an E-Stop button, and is the same color as the Mach3 namesake.

IO sender Beta-8
; Grbl
; 1.1h.20190825
; [OPT:V,15,128]

Just after progrem started.
Home button disabled, but $H from the MDI works normally and Home become enabled after Unlock.
As you can see, ALARM indicator and Unlock button looks very similar

My suggestion change the button design to only red frame, or red text to visually separate them.
also playing with houming I found strange behavior - after error during houming "Alarm 9" blink on state indicator for a moment and then sender reset GRBL and clean alarm number. With different types of alarm it seems no auto-reset.

I found also some issues with jogging etc. What is your prefer - create separate issue for every new, or add all to the single thread?

IO sender Beta-8
; Grbl
; 1.1h.20190825
; [OPT:V,15,128]

Just after progrem started.
Home button disabled, but $H from the MDI works normally and Home become enabled after Unlock.
As you can see, ALARM indicator and Unlock button looks very similar

My suggestion change the button design to only red frame, or red text to visually separate them.
also playing with houming I found strange behavior - after error during houming "Alarm 9" blink on state indicator for a moment and then sender reset GRBL and clean alarm number. With different types of alarm it seems no auto-reset.

I found also some issues with jogging etc. What is your prefer - create separate issue for every new, or add all to the single thread?

Home button disabled, but $H from the MDI works normally and Home become enabled after Unlock.

I have changed the behaviour for legacy Grbl to enable the Home button on startup. This may not always work though as it is impossible to find out if homing is actually enabled in all circumstances. And Grbl suggests $H to unlock even if homing is not enabled... I have uploaded a new edge version with this and other changes for you to test.

As you can see, ALARM indicator and Unlock button looks very similar

I am sorry, they do not look similar to me. Changing the button style to put a red frame around them would be IMO odd. So I'll leave it as is.

playing with houming I found strange behavior - after error during houming "Alarm 9" blink on state indicator for a moment and then sender reset GRBL and clean alarm number.

This is another legacy Grbl oddity - at least my Mega version resets itself on alarm 9 and then the following status reports does not indicate which alarm is active. I have added a workaround that stores that last alarm number and adds that as long as alarm state is reported. Note that legacy Grbl controllers should be reset on connect, most by toggling DTR, some by toggling RTS. Many grblHAL controllers does not support reset on connect so the default is No action.

I found also some issues with jogging etc. What is your prefer - create separate issue for every new, or add all to the single thread?

Separate issues is often the best as otherwise they may become hard to follow.

Home button disabled, but $H from the MDI works normally and Home become enabled after Unlock.

I have changed the behaviour for legacy Grbl to enable the Home button on startup. This may not always work though as it is impossible to find out if homing is actually enabled in all circumstances. And Grbl suggests $H to unlock even if homing is not enabled... I have uploaded a new edge version with this and other changes for you to test.

As you can see, ALARM indicator and Unlock button looks very similar

I am sorry, they do not look similar to me. Changing the button style to put a red frame around them would be IMO odd. So I'll leave it as is.

playing with houming I found strange behavior - after error during houming "Alarm 9" blink on state indicator for a moment and then sender reset GRBL and clean alarm number.

This is another legacy Grbl oddity - at least my Mega version resets itself on alarm 9 and then the following status reports does not indicate which alarm is active. I have added a workaround that stores that last alarm number and adds that as long as alarm state is reported. Note that legacy Grbl controllers should be reset on connect, most by toggling DTR, some by toggling RTS. Many grblHAL controllers does not support reset on connect so the default is No action.

I found also some issues with jogging etc. What is your prefer - create separate issue for every new, or add all to the single thread?

Separate issues is often the best as otherwise they may become hard to follow.

I have changed the behaviour for legacy Grbl to enable the Home button on startup.
Thanks, now it's works from the start. Beta-8.8

This is another legacy Grbl oddity - at least my Mega version resets itself on alarm 9 and then the following status reports does not indicate which alarm is active.
My Mega328 version also resets on this alarm.
I have added a workaround that stores that last alarm number and adds that as long as alarm state is reported.
It's create another issue - ALARM state cannot be differenciated from Locked state. After reset nothing changed on the screen and looks like reset not issued.
It's probable better store last alarm, error and last command before it in the pop-up window or status bas report.
Note that legacy Grbl controllers should be reset on connect, most by toggling DTR, some by toggling RTS. Many grblHAL controllers does not support reset on connect so the default is No action.
I tryed both - with and without DTR - all works the same for my board

It's create another issue - ALARM state cannot be differenciated from Locked state. After reset nothing changed on the screen and looks like reset not issued.

Handling messages from the controller and presenting them in the status bar is problematic. Now and then I try to improve this but I am not yet happy with how it works. I may change when messages are sent in grblHAL to make this easier. A part of this is deciding on when a message should be removed from the status bar. IMO the controller should send an empty message on the relevant state changes...
If unsure use the console to see what has been sent.
On a reset then the message "Waiting for the controller (portparams)..." is briefly shown in the status bar. Note that some UI elements are not updated on a reset (coolant, spindle status) due to the Grbl bug mentioned in #86. Again this is fixed in grblHAL.

According to current situation with messages,
I propose to revert back Betta 8 behaviour and add additional popup window, for example on the click to status indicator or behind it, with short event log, for example
Time, event, description (helpful for alarms and errors)
For 2-3 latest events.
Only events, which changing state indication.
Looking console can be confusing for some users, especially traffic - saving communication shortening in GRBL


Error on generation of nested objects in array

Found a little issue on substructures in nested objects in array.
For example :
[
  {
    "a": 1,
    "b": {
	"b1": 1,
	"b2": "B2"
    }
  },
  {
    "b": {
  	"b3": 3,
        "b4": "B4"
    }
  }
]

will generate:
type AutoGenerated []struct {
	A int `json:"a,omitempty"`
	B struct {
		B1 int    `json:"b1"`
		B2 string `json:"b2"`
	} `json:"b,omitempty"`
	B struct {
		B3 int    `json:"b3"`
		B4 string `json:"b4"`
	} `json:"b,omitempty"`
}

Thanks!
Does anyone have time to submit a patch? I'm a bit swamped this week and next.

I take a look at the code, and I don't know which "merge/duplicate"strategy you want to use :
should B be a struct with B1,B2,B3,B4 or should we have two structures BFirst {B1, B2} BSecond {B3,B4} ?
The second strategy (name it "duplicate") seems easier to implement as we could image B struct and B_uuidv4() struct as you already do. For my case I wanted the first strategy ("merge") but that would mean a deep merge. This would end up with weird behaviour if we have nested array in the nested objects.
In my case I was on a structure like this:
{
	"stores": [{
			"sign": "wallmart",
			"address": {
				"street": "1st street",
				"zipcode": "123456",
				"city": "randomCity1",
				"phone": "1234567890"
			}
		},
		{
			"sign": "bestbuy",
			"address": {
				"street": "2nd street",
				"zipcode": "654321",
				"city": "randomCity2"
			}
		}
	]
}

and it seems pretty obvious that in this case, addresses structs should be merged. In other less well-structured jsons, we would like to end up with two structures
So basically in which direction would you like to go?

Similar issue with key-value in json. Ex:
    "packages":[
        {
            "name":"p1",
            "version":""
        }
    ],
    "vendorPackages":{
        "vp1":{
            "name":"",
            "version":"123"
        },
        "vp2":{
            "name":"456",
            "version":""
        }
    }
}

Should be
type Packages struct {
	Name    string `json:"name"`
	Version string `json:"version"`
}

type GMF struct {
	Packages       []Packages          `json:"packages"`
	VendorPackages map[string]Packages `json:"vendorPackages"`
}

Instead of
type Autogenerated struct {
	Packages []struct {
		Name    string `json:"name"`
		Version string `json:"version"`
	} `json:"packages"`
	Vendorpackages struct {
		Vp1 struct {
			Name    string `json:"name"`
			Version string `json:"version"`
		} `json:"vp1"`
		Vp2 struct {
			Name    string `json:"name"`
			Version string `json:"version"`
		} `json:"vp2"`
	} `json:"vendorPackages"`
}


Mekanism Gas conduits will likely be broken against version 8 of Mekanism

I am fairly certain your gas conduits will be broken against mekanism 8's gas transfer. I think they changed the way it works, or at least the requisite function's location in their code. (I have been testing it, and enderio's gas conduits for mekanism crash against the development version).

incidentally, it wouldn't hurt to have the ability to transfer mekanism gases through the transceiver (i haven't checked if you have a tab for that, as i JUST added mekanism to my mods.)

I would also like to add that this is *world-corrupting, as once the conduit is placed it crashes the world every time the block is loaded.
*assuming you don't have the knowledge&power to use MCEDIT to remove the tiles.

@TheFireRodan Mekanism can simply be downgraded to fix the issue.

Yes Mekanism can be downgraded but is that really a fix? I don't want to sacrifice one mod in a modpack for another.

downgrading is not a fix

you CAN turn off the enderio mekanism gas conduits in the config prior to the release of the fixed version

Though turning off the configuration does nothing for pre existing machine setups in the world.

depends on how enderio works internally, if it doesn't provide for those items to exist, they might auto-delete. You could always just remove the offending blocks with mcedit or temporarily downgrade and remove them prior to upgrading. It could be part of a fix, but the point is that downgrading is the best solution to this problem given the user wants to use the updated mekanism version.

its also marked as "code complete" so you might want to check if they have any jenkins builds or see if you can pull the sources and dependencies and do a custom build yourself ;p

has anything been done on this? i would love a world with a cleaer interface that ender IO gives.

EnderIO 2.2.X will be compatible with Mekanism 7. EnderIO 2.3.X will be compatible with Mekanism 8. End of story.

thanks :) hope 2.3 comes out soon :)


Grammar mistake

@ffyuanda plz make a PR to remove "s"

Sorry I opened the issue in the wrong place, and the PR is made @hsluoyz


Hard tabs in a file cause a render glitch after entering insert mode and then returning to normal mode.

When working in a project where the convention was to use hard tabs, every time I enter insert mode and then return to normal mode all of the text becomes illegible. The first few non whitespace characters of each line replaced with dots.

To reproduce try pasting this into a file and make sure that rust analyzer isn't auto formating to soft spaces. contents of this file
fn main() {
	println!("Hello, world!");
	// This is a comment.
	// compute fib 10
	//
	//
	let mut a = 0;
	let mut b = 1;
	let mut c = 0;

	for _ in 0..10 {
		c = a + b;
		a = b;
		b = c;
	}
	println!("fib 10 = {}", c);
}

This was due to a setting in my init.vim where I change the list mode  similar to what's described in this stack overflow
Any changing of the lcs tab seem to break visuals and syntax highlighting in VS code for files that use tabs.
The workaround is to disable this setting when in vscode:
if exists('g:vscode')
    " VSCode extension. Don't mess with list mode
else
  function MyInsertListMode()
    set list lcs=tab:\ \ ,eol:\ ,trail:\ " Spaces only while inserting
  endfunction
  function MyNormalListMode()
    set list lcs=tab:⇥.,eol:\ ,trail:⑀
  endfunction
  au InsertEnter * :call MyInsertListMode()
  au InsertLeave * :call MyNormalListMode()
  call MyNormalListMode()
endif

I would understand if this bug were closed as supporting list mode custom white space characters is probably beyond the scope of what this plugin is trying to do.


Angular compiler warns about CommonJS or AMD

When trying to build an Angular app that depends on unleash-proxy-client, it warns that "CommonJS or AMD dependencies can cause optimization bailouts. For more info see: https://angular.io/guide/build#configuring-commonjs-dependencies"
Is it possible to make an ES Module build?

Hey, @snowfrogdev! I'm sorry, but it seems we missed this issue when you opened it 🙇🏼  Is this still an issue? I remember you gave us some very nice contributions around that time, so I'm not sure whether this is resolved or not.

@thomasheartman this is still an issue, any update on a possible solution/workaround?

Hey, @timothylombrana! Thanks for reporting in 😄
Is it actually stopping the build in any way?
If I understand correctly, it's just a warning saying that it's unable to do optimizations when bundling (tree-shaking, I suppose), and it shouldn't cause any build errors. If it's preventing you from building, then you might need to allow the build to proceed with warnings. This SDK has a fairly small amount of dependencies, so it shouldn't be that big a deal, I think.
That said, I don't we've done anything to address this since then. I'll ask around if anyone on the team knows anything more, though. We don't get a lot of input from Angular users, so we see this asked about very infrequently.

@thomasheartman it is not stopping the build in any way. I did add all the dependencies([tiny-emitter,uuid]) to the angular.json under allowedCommonJsDependencies as suggested by NG. But it doesn't resolve the warning. I wanted to share my finding, hoping we could fix the warning.


Handle cases where a route's prefix is a nested URL

Nested prefixes (any prefix containing a '/' character) prior to this commit would incorrectly get tagged as the first segment of the URL prefix.
E.g. a route like "api/beta/v1/patients" would incorrectly get tagged as "api" instead of "patients"

Coverage increased (+0.0007%) to 99.331% when pulling f553276c4193df6f938a61f5d691564806be3be3 on SimonKaluza:handle-route-prefixes into 445d2d471d70183e96b0f0e539c7bf2ba3e10278 on ruby-grape:master.

mmh for me the changes are reasonable, but please adept the specs and/or add one which documents the behaviour

Added a RSpec example with tags that breaks without the adjustments made to endpoint.rb to support nested prefix and rebased off of the current master (hence the force-pushes)

thanks @SimonKaluza


PhpStorm isn't opened when managed by Toolbox

Duplicate of #343 on the other repository.

Standalone PhpStorm without Toolbox will have no issue with the phpstorm:// only when managed by Toolbox.
https://youtrack.jetbrains.com/issue/IDEA-283631
At this point maybe the user should be more involved in choosing the IDE, or there should be a dropdown in the UI about that where they can toggle between IDEs:

Auto
JetBrain PhpStorm
JetBrain Toolbox + PhpStrom

⚠ This also breaks when the .idea directory is not present.

Hi @DarkGhostHunter,
Thanks for reminding us. I'd love to get the JetBrains Toolbox to work in Ignition. If you, or anyone else, feels up for the task to port the previous PR to this version of Ignition, I'll happily merge it.
I'll leave this open for anyone else looking for Toolbox support, answers or a challenge :)


Help on WinAPI Beep leads to wrong (this) pag

In Visual Studio, when pressing F1 on the WinAPI Beep in a cpp-file in a i C++ Windows Desktop project, it opens up the documentation for SystemSounds.Beep Property of the System.Media.SystemSounds Class in .NET Framework 4.7.2.
This is what the prototype for the WinAPI Beep looks like:
WINBASEAPI BOOL WINAPI Beep(
In DWORD dwFreq,
In DWORD dwDuration );
The correct page should probably be:
https://msdn.microsoft.com/en-us/library/windows/desktop/ms679277(v=vs.85).aspx
Visual-Studio: Help -> About
Microsoft Visual Studio Professional 2017
Version 15.8.8
VisualStudio.15.Release/15.8.8+28010.2048
Microsoft .NET Framework
Version 4.7.03056
Installed Version: Professional
Visual C++ 2017   00369-60000-00001-AA714
Microsoft Visual C++ 2017
Add New File   3.5.134
The fastest and easiest way to add new files to any project - including files that start with a dot
ADL Tools Service Provider   1.0
This package contains services used by Data Lake tools
ASP.NET and Web Tools 2017   15.8.05085.0
ASP.NET and Web Tools 2017
ASP.NET Core Razor Language Services   15.8.31590
Provides languages services for ASP.NET Core Razor.
ASP.NET Web Frameworks and Tools 2017   5.2.60618.0
For additional information, visit https://www.asp.net/
Azure App Service Tools v3.0.0   15.8.05023.0
Azure App Service Tools v3.0.0
Azure Data Lake Node   1.0
This package contains the Data Lake integration nodes for Server Explorer.
Azure Data Lake Tools for Visual Studio   2.3.5001.4
Microsoft Azure Data Lake Tools for Visual Studio
Azure Functions and Web Jobs Tools   15.9.02046.0
Azure Functions and Web Jobs Tools
Azure Stream Analytics Tools for Visual Studio   2.3.5001.4
Microsoft Azure Stream Analytics Tools for Visual Studio
Browser Reload on Save   1.1.27
A Visual Studio extension for ASP.NET projects that leverages Browser Link to to reload all connected browsers when files are saved in Visual Studio.
Bundler & Minifier   2.8.396
Adds support for bundling and minifying JavaScript, CSS and HTML files in any project.
C# Tools   2.9.0-beta8-63208-01
C# components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.
Common Azure Tools   1.10
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.
Cookiecutter   15.8.18241.1
Provides tools for finding, instantiating and customizing templates in cookiecutter format.
CSS Tools   1.1.23
Provides additional features to the CSS editor in Visual Studio.
Dotfuscator Community Edition   <IP_ADDRESS>50-e77ce80a6
PreEmptive Protection - Dotfuscator CE
EditorConfig Language Service   1.17.246
Language service for .editorconfig files.
EditorConfig helps developers define and maintain consistent coding styles between different editors and IDEs.
Extensibility Message Bus   1.1.49 (remotes/origin/d15-8@ee674f3)
Provides common messaging-based MEF services for loosely coupled Visual Studio extension components communication and integration.
Fabric.DiagnosticEvents   1.0
Fabric Diagnostic Events
File Icons   2.7
Adds icons for files that are not recognized by Solution Explorer
File Nesting   2.6.85
Automatically nest files based on file name and enables developers to nest and unnest any file manually
GitHub.VisualStudio   <IP_ADDRESS>65
A Visual Studio Extension that brings the GitHub Flow into Visual Studio.
HTML Tools   1.0.3
Productivity tools for the HTML editor
Image Optimizer   4.0.132
Uses industry standard tools to optimize any JPEG, PNG and Gifs - including animated Gifs. Can do both lossy and lossless optimization.
Image Sprites   1.4.58
Boost your website's performance by creating image sprites to reduce the amount of HTTP requests needed.
JavaScript Language Service   2.0
JavaScript Language Service
JavaScript Project System   2.0
JavaScript Project System
JavaScript Transpiler   0.9.28
The simplest way to transpile JS and JSX to EcmaScript 5 without any complicated node.js tools in your project. It uses the TypeScript compiler behind the scenes.
Markdown Editor   1.12.236
A full featured Markdown editor with live preview and syntax highlighting. Supports GitHub flavored Markdown.
Microsoft Azure HDInsight Azure Node   2.3.5001.4
HDInsight Node under Azure Node
Microsoft Azure Hive Query Language Service   2.3.5001.4
Language service for Hive query
Microsoft Azure Service Fabric Tools for Visual Studio   2.3
Microsoft Azure Service Fabric Tools for Visual Studio
Microsoft Azure Stream Analytics Language Service   2.3.5001.4
Language service for Azure Stream Analytics
Microsoft Azure Stream Analytics Node   1.0
Azure Stream Analytics Node under Azure Node
Microsoft Azure Tools   2.9
Microsoft Azure Tools for Microsoft Visual Studio 2017 - v2.9.10730.2
Microsoft Continuous Delivery Tools for Visual Studio   0.4
Simplifying the configuration of Azure DevOps pipelines from within the Visual Studio IDE.
Microsoft JVM Debugger   1.0
Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines
Microsoft Library Manager   1.0
Install client-side libraries easily to any web project
Microsoft MI-Based Debugger   1.0
Provides support for connecting Visual Studio to MI compatible debuggers
Microsoft Visual C++ Wizards   1.0
Microsoft Visual C++ Wizards
Microsoft Visual Studio Tools for Containers   1.1
Develop, run, validate your ASP.NET Core applications in the target environment. F5 your application directly into a container with debugging, or CTRL + F5 to edit & refresh your app without having to rebuild the container.
Microsoft Visual Studio VC Package   1.0
Microsoft Visual Studio VC Package
MLGen Package Extension   1.0
MLGen Package Visual Studio Extension Detailed Info
Mono Debugging for Visual Studio   4.11.11-pre (8fb558f)
Support for debugging Mono processes with Visual Studio.
Node.js Tools   1.4.20907.4 Commit Hash:f048d52e417a8902dea9e42828b7fdab92d4aaaa
Adds support for developing and debugging Node.js apps in Visual Studio
NuGet Package Manager   4.6.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit http://docs.nuget.org/.
Office Developer Tools for Visual Studio 2017 ENU   15.0.27907.01
Microsoft Office Developer Tools for Visual Studio 2017 ENU
Open Command Line   2.1.218
Opens a command line at the root of the project. Support for all consoles such as CMD, PowerShell, Bash etc. Provides syntax highlighting, Intellisense and execution of .cmd and .bat files.
Package Security Alerts   1.0.22
Identifies npm and Bower packages that contains known security vulnerabilities to ensure your project is always using the most secure package versions.
PowerShell Pro Tools for Visual Studio   1.0
A set of tools for developing and debugging PowerShell scripts and modules in Visual Studio.
Project File Tools   1.0.1
Provides Intellisense and other tooling for XML based project files such as .csproj and .vbproj files.
ProjectServicesPackage Extension   1.0
ProjectServicesPackage Visual Studio Extension Detailed Info
Python   15.8.18241.1
Provides IntelliSense, projects, templates, debugging, interactive windows, and other support for Python developers.
Python - Django support   15.8.18241.1
Provides templates and integration for the Django web framework.
Python - IronPython support   15.8.18241.1
Provides templates and integration for IronPython-based projects.
Python - Profiling support   15.8.18241.1
Profiling support for Python projects.
Python - UWP support   15.8.18241.1
Provides templates and integration for the UWP framework.
R Tools for Visual Studio   1.3.40517.1016
Provides project system, R Interactive window, plotting, and more for the R programming language.
ResourcePackage Extension   1.0
ResourcePackage Visual Studio Extension Detailed Info
ResourcePackage Extension   1.0
ResourcePackage Visual Studio Extension Detailed Info
SQL Server Data Tools   15.1.61808.07020
Microsoft SQL Server Data Tools
SQLite & SQL Server Compact Toolbox   4.7
SQLite & SQL Server Compact Toolbox adds scripting, import, export, rename, query execution and much more to SQL Server Compact & SQLite Data Connections.
Syntax Highlighting Pack   2.8.134
Adds syntax highlighting and snippet support for a wide variety of programming languages such as Clojure, Go, Jade, Lua, Swift, Ruby and many more...
ToolWindowHostedEditor   1.0
Hosting json editor into a tool window
TypeScript Tools   15.8.20822.2001
TypeScript Tools for Microsoft Visual Studio
Web Compiler   1.12.394
Compiler for LESS, Sass and CoffeeScript files
Web Essentials 2017   2.0.44
The easiest way to set up Visual Studio for the ultimate web development experience. Requires Visual Studio 2017 Update 8 or newer
Visual Basic Tools   2.9.0-beta8-63208-01
Visual Basic components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.
Visual C++ for Cross Platform Mobile Development (Android)   15.0.27924.00
Visual C++ for Cross Platform Mobile Development (Android)
Visual C++ for Cross Platform Mobile Development (iOS)   15.0.27825.00
Visual C++ for Cross Platform Mobile Development (iOS)
Visual C++ for Linux Development   <IP_ADDRESS>924
Visual C++ for Linux Development
Visual F# Tools 10.2 for F# 4.5   <IP_ADDRESS>.  Commit Hash: 55a3dc3231c95c77f81ee53f7d29152029da7408.
Microsoft Visual F# Tools 10.2 for F# 4.5
Visual Studio Code Debug Adapter Host Package   1.0
Interop layer for hosting Visual Studio Code debug adapters in Visual Studio
Visual Studio Comparison Tools   1.0
Information about my package
Visual Studio Tools for Apache Cordova   15.123.7408.1
Visual Studio Tools for Apache Cordova
Visual Studio Tools for CMake   1.0
Visual Studio Tools for CMake
Visual Studio Tools for Containers   1.0
Visual Studio Tools for Containers
Visual Studio Tools for Unity   <IP_ADDRESS>
Visual Studio Tools for Unity
Visual Studio Tools for Universal Windows Apps   15.0.28010.2046
The Visual Studio Tools for Universal Windows apps allow you to build a single universal app experience that can reach every device running Windows 10: phone, tablet, PC, and more. It includes the Microsoft Windows 10 Software Development Kit.
VisualStudio.Mac   1.0
Mac Extension for Visual Studio
Workflow Manager Tools 1.0   1.0
This package contains the necessary Visual Studio integration components for Workflow Manager.
Vue.js Pack 2017   1.1.8
Contains HTML Intellisense and code snippets for the Vue.js JavaScript library
Xamarin   <IP_ADDRESS>6 (d15-8@1ae9b59d7)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.
Xamarin Designer   4.15.9 (d000f568b)
Visual Studio extension to enable Xamarin Designer tools in Visual Studio.
Xamarin Templates   1.1.116 (9619170)
Templates for building iOS, Android, and Windows apps with Xamarin and Xamarin.Forms.
Xamarin.Android SDK   <IP_ADDRESS> (HEAD/a8a3b0ec7)
Xamarin.Android Reference Assemblies and MSBuild support.
Xamarin.iOS and Xamarin.Mac SDK   <IP_ADDRESS> (84552a4)
Xamarin.iOS and Xamarin.Mac Reference Assemblies and MSBuild support.
ZenCoding   1.2.12
Provides ZenCoding for the HTML Editor - full support for static HTML, Razor and WebForms.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5db29198-3bb6-03f8-1e79-96e093c4ab30
Version Independent ID: 21bf9289-b3eb-2ced-5b8f-21335d992506
Content: SystemSounds.Beep Property (System.Media)
Content Source: xml/System.Media/SystemSounds.xml
Product: dotnet-api
GitHub Login: @dotnet-bot
Microsoft Alias: dotnetcontent

@mairaw Who do we contact about F1 help?

@Thraka we can log an issue internally for this. It seems like this is a re-occurring problem that needs to be addressed by the engineering team.

Thanks Den.


HOPS-141

[HOPS-141]  Basic testing with JMX
[HOPS-141]  Tests for getting CertificateLocalization service state via JMX
[HOPS-141]  JMX operation to force remove crypto material from store

https://hopshadoop.atlassian.net/browse/HOPS-141

Jenkins test this please.

Jenkins test this please.


Does it Support on MacOS Thanks

never fail or success thanks
I have added this code my self
def set_cmd_window_title(GENNED, LOCKED):
    title = f"[Exploited7 justmanooo]  |  [ Generated : {GENNED} ]  [ Failed : {LOCKED} ]"
    current_os = platform.system()
    if current_os == "Windows":
        ctypes.windll.kernel32.SetConsoleTitleW(title)
    else:
        print(f"Cannot set console title on {current_os}. Title would be: {title}")

After I set everything up, I run the script and it keeps showing this page. Is the script broken?


Publishing a nested node package

Our application has the following structure after being built:
/dist
|    |__/lib
|          |__package.json
|
/misc folders
|
misc files
|
package.json

We need to publish the package under the dist/lib folder not the one at the root. I don't see any way in the documentation to point to a specific package.json when using npm-publish. later we might decide to add more libs, ex. (dist/lib1, dist/lib2). how can we handle this?

@bstaley,
When running the npm-publish command from within the dist/lib folder, does the publish work properly?
If so, do you see an issue with this solution?

@bstaley,
I'm closing this issue for now, but please let me know if you'd like to continue discussing this.

@eyalbe4,
I was finally able to test this out and it worked. We just cd ./dist/lib right before calling npm-publish and presto we get the results we expect. Thanks for the suggestion!


deregister and re-register, ClosedChannelException is thrown

Hi,
I'm doing a project on middle-ware for distributed database, and I want to use Netty as client connecting to MySQL reading data. But a problem gets me stuck for such a long time so I have to ask for help. The scenario is as below:

Netty dispatches SQL query to MySQL;

MySQL returns a bunch of resultset which might be very large so I have to prevent it from OOM;
When the data received exceed a user-defined high water mark threshold, I use channel.deregister() to stop the incoming data for a while;
The business logic consumes the data received, when it reaches a  low water mark threshold, I use eventloop.register(channel) to continue to receive the remaining data;
Do the same thing until all data needed is received and processed.

But here the strange thing is when I run a benchmark tool with N concurrent clients executing the same SQL constantly, and the NioEventLoopGroup is with M threads created(N>M), after a while the N-M connections are shown closed, leaving M connections active to the end.
After debugging, I find during the deregister and re-register, the channel bounded to the later failed connection will be closed(seems like the SelectionKey is set invalid) , so when trying to re-register,  ClosedChannelException  will be thrown.
And it should be mentioned that, if I remove the deregister and re-register logic it works fine, so obviously my usage of deregister and re-register could probably be wrong in some place, could you do me a favor instructing me to locate the reason? Thank you so much in advance.
Bootstrap bootstrap = new Bootstrap();
bootstrap
    .group(MySQLConnectorConfig.group)
    .channel(NioSocketChannel.class)
    .handler(new ChannelInitializer<SocketChannel>() {
        @Override
        protected void initChannel(SocketChannel ch)
                    throws Exception {
            ch.pipeline().addLast(new MySQLClientDecoder(conn));
            ch.pipeline().addLast(new MySQLClientHandler(conn));
            ch.config().setAllocator(PooledByteBufAllocator.DEFAULT);
           }
     })
    .option(ChannelOption.SO_KEEPALIVE, true)
    .option(ChannelOption.TCP_NODELAY, true)
    .option(ChannelOption.SO_REUSEADDR, true)
    .option(ChannelOption.SO_RCVBUF, MySQLConnectorConfig.DEFAULT_SO_RCVBUF);

I deregister the channel in MySQLClientDecoder, and re-register in my business logic when consuming the data received.

the channel bounded to the later failed connection will be closed(seems like the SelectionKey is set invalid)

Can you provide some details as to why you say the SelectionKey is "set invalid", and what state is it in? Also is it possible to provide a reproducer for this scenario?

When the data received exceed a user-defined high water mark threshold, I use channel.deregister() to stop the incoming data for a while;

Sounds like this can be accomplished using a combination of primitives provided by Netty, and deregister/register could potentially be avoided:

channelWritabilityChanged and configure the high/low write water mark
isWritable
turn off autoRead and/or manually call read

First thanks very much for your in-time reply.
The reason I say the SelectionKey is set invalid is when debugging the program goes into the following logic in NioEventLoop:
private static void processSelectedKey(SelectionKey k, AbstractNioChannel ch) {
        final NioUnsafe unsafe = ch.unsafe();
        if (!k.isValid()) {
            // close the channel if the key is not valid anymore
            unsafe.close(unsafe.voidPromise());
            return;
        }
...

channel close is triggered because of the invalidity of the SelectionKey, that's what I saw.
Actually, I just want to control the flow of the incoming data, so I deregister the channel from eventloop when overflow and re-register it back to the same eventloop when underflow. I don't know if it's the typical usage of deregister/re-register paradigm, you said this could potentially be avoided by the way you suggest, but here Netty acts as a client reading data from MySQL Server, what I should control is the read buffer I think, not the  write buffer, so why it's all channel write related methods you offer? Could you plz explain? (maybe a naive question,:-D)
And also the high/low water mark I mention is not simply the accumulated bytes coming, but the accumulated bytes for complete packets I define in my process logic

What is the state of the Channel and the associated selector at this point? The SelectionKey will not be valid under the following circumstances:

A key is valid upon creation and remains so until it is cancelled,
* its channel is closed, or its selector is closed.

The SelectionKey is cancelled on deregister and an new SelectionKey is created on register ... can you tell me the state of ch.isRegistered() when the channel is closed in this condition (on the unsafe.close(...) line you pasted above)?

but here Netty acts as a client reading data from MySQL Server, what I should control is the read buffer I think, not the write buffer, so why it's all channel write related methods you offer? Could you plz explain?

The typical way to control how much you are reading is to disable autoRead and instead manually call read(). You can use w/e condition makes sense for your use case but it is also common that you want to stop reading if you are unable to write data to somewhere (for example stop reading requests if you are unable to write responses). This is what the writability stuff is useful for, but if you have other application semantics that is fine too.

Yes, I really appreciate your sincere help on my problem. And glad to tell you it's been solved with your constructive suggestions.
I have dived deeply into the autoRead paradigm and found it's what I'm looking for. Actually here in my scenario I just want to control the incoming flow rate, and set autoRead is enough(remove the OP_READ and add it back later) , and it works. On the other hand, the water mark is still controlled by the application semantics, not the built-in writability control.
Thank you again.

@purpleknightleo - Let me reopen this issue and I think you uncovered a bug. Would you mind keeping your current code base around so you can verify a PR I will submit?

Ok, no problem. The versions I tried include 4.0.32/33/35, I don't know if it has been fixed in later version. As I mentioned, my need should be done with autoRead, but I think deregister/re-register can also satisfy though It's too heavy-weight.
I don't really know what you want me to do now, keep code base around so to verify a PR? Could you plz explain? I'm a newbie in github and open-source projects participation. Thank you.

my need should be done with autoRead, but I think deregister/re-register can also satisfy though It's too heavy-weight.

Sorry for the confusion...
I would still recommend going the autoRead and controlling when you call read. I would like you to run your code which caused you to create this issue against https://github.com/netty/netty/pull/5167 to verify this PR fixes the original issue.

@Scottmitch Sorry to tell you your fix doesn't solve my previous issue, and I think the code logic you changed has a problem. You try to judge whether the channel is still registered to the eventloop by:
if (eventLoop != this || eventLoop == null) {
                return;
}

But in my case, when I set one thread in NioEventLoopGroup, and two clients connect to my middleware, the final result is one client succeeds and the other dies due to the reason you have known. On the other hand, after deregistered from eventloop, we can still use ch.eventloop() to get the previously bound eventloop(of course, their real binding is dismissed due to deregistration). So when the selectionKey is invalid, it goes into the code logic pasted above, but actually here it satisfies eventloop !=null && eventloop == this(because here we have only one eventloop, it can't be another one), the close process still gets in as we don't expect.
Thus, I think why we can't use the following logic to take a judge:
if (!k.isValid()) {
            final EventLoop eventLoop;
            try {
                eventLoop = ch.eventLoop();
            } catch (Throwable ignored) {
                // If the channel implementation throws an exception because there is no event loop, we ignore this
                // because we are only trying to determine if ch is registered to this event loop and thus has authority
                // to close ch.
                return;
            }
            // Only close ch if ch is still registerd to this EventLoop. ch could have deregistered from the event loop
            // and thus the SelectionKey could be cancelled as part of the deregistration process, but the channel is
            // still healthy and should not be closed.
            // See https://github.com/netty/netty/issues/5125
            if (eventLoop != this || eventLoop == null || (eventloop == this && !ch.isRegistered())) {
                return;
            }
            // close the channel if the key is not valid anymore
            unsafe.close(unsafe.voidPromise());
            return;
        }

@purpleknightleo - Thanks for reporting back. The issue is that the eventloop and isRegistered will not be updated until the next event loop run. Let me think about this a bit. If you could provide a unit test which reproduces the behavior that would be much appreciated ... seems non-trivial to get into this state.

Ok, after I try the newer version, I will give you a replay ASAP. Thanks.

Ok, after I try the newer version, I will give you a replay ASAP. Thanks.

Ok, after I try the newer version, I will give you a replay ASAP. Thanks.

Sorry again, your fix still can't  solve my problem and sorry for not offering the unit test for some reason.
if (eventLoop != this || !ch.isRegistered() || ch.selectionKey() != k) {
                  return;
 }

From you fixed code, I think since you said "isRegistered() will not be updated until the next event loop run", the !ch.isRegistered() judgement is of no use. On the other hand, ch.selectionKey() != k, why this? ch is got from k.attachment(), so I think these two objects are linked that ch.selectionKey() will always return k.

On the other hand, ch.selectionKey() != k, why this?

The AbstractNioChannel.selecitonKey is re-assigned under a few conditions ... one of these is when the channel is registered. This was added in case the channel is re-registered to the same EventLoop but an "old selection key" is being processed (and is now inactive) ... this should not result in the channel being closed.

!ch.isRegistered() judgement is of no use

In summary I can't reproduce the issue you are experiencing so I'm including things that should not be logically wrong ... but not sure it will impact the scenario you are running. I will need more information to debug this more:

Can you re-confirm you hit the line in NioEventLoop.processSelectedKey which calls unsafe.close(unsafe.voidPromise()); in this case?
Can you enumerate the state of the k, ch, and provide a stack trace in this state?
If you could relate when the NioEventLoop.processSelectedKey method is called relative to your call to register/dergister that would be helpful (e.g. I call degregister, processSelectedKey is called, then I call register).

sorry for not offering the unit test for some reason.

I also tried to create a unit test to recreate the sequence of events which I think would repro this issue (deregister, processSelectedKey, register) but was not able to get processSelectedKey to run in the middle. If you can confirm this is the sequence of events I should be able to figure out a way to get the state updated so we can see it "immediately" instead of waiting until the next event loop run.

@purpleknightleo - ping

@purpleknightleo - any update?

@purpleknightleo - ping.

No update so closing


Fix global escape of variable in TermMap.shrink()

for (foo in bar)
should be
for (var foo in bar)

This was independently fixed in the 4.0.0 release since it uses the new Map


Script attribute value with quotes is escaped

Have you read the Contributing Guidelines on issues?

[X] I have read the Contributing Guidelines on issues.

Prerequisites

[X] I'm using the latest version of Docusaurus.
[X] I have tried the npm run clear or yarn clear command.
[X] I have tried rm -rf node_modules yarn.lock package-lock.json and re-installing packages.
[X] I have tried creating a repro with https://new.docusaurus.io.
[X] I have read the console error message carefully (if applicable).

Description
I want to use Cloudflare Web Analytics on my Docusaurus site.
To enable this I need to add a script to every page. Note the data-cf-beacon value is a JSON object.

In docusaurus.config.ts I added the Cloudflare script to scripts:
const config: Config = {
  scripts: [
    {
      // Cloudflare Web Analytics
      defer: true,
      async: false,
      src: "https://static.cloudflareinsights.com/beacon.min.js",
      "data-cf-beacon": "{\"token\": \"1234\"}",
    },
  ]
}

However, Docusaurus escapes the quotes in the JSON value, meaning that the JSON is invalid.
<!doctype html>

  <meta charset="UTF-8">
  <meta name="generator" content="Docusaurus v3.3.2">

  [...]

Reproducible demo
https://stackblitz.com/edit/github-sqjruf?file=docusaurus.config.ts
Steps to reproduce
See above
Expected behavior
Either:

The quotes are not escaped, and the literal value is rendered.

There is some way to disable escaping.
I see that Eta has a function for modifying escaping, perhaps it could be used?
/** Function to XML-sanitize interpolations */
escapeFunction: (str: unknown) => string

I would also like it if the solution was documented.
Actual behavior
Docusaurus escapes the quotes in the JSON value, meaning that the JSON is invalid.
It's not easy to figure out a workaround. I think manually updating the ssrTemplate works:
// docusaurus.config.ts

import defaultSSRTemplate from<EMAIL_ADDRESS>
const config: Config = {

  ssrTemplate: defaultSSRTemplate
  .replace(
      "",
      "\n"
  ),

Your environment

Public source code: https://github.com/adamko-dev/dokkatoo/blob/ba9fe26aaa0add74e78f7527282f5e46fd24999e/modules/docs/site/docusaurus.config.ts#L36-L43
Public site URL: https://adamko-dev.github.io/dokkatoo/
Docusaurus version used: 3.3.2
Environment name and version (e.g. Chrome 89, Node.js 16.4): n/a
Operating system and version (e.g. Ubuntu 20.04.2 LTS): n/a

Self-service

[ ] I'd be willing to fix this bug myself.

I'm also running into this broken JSON object when using Cloudflare Analytics on Docusaurus 3.3.2. Unfortunately, I can't share our source code until the site is made public later this year.
It seems strange that I only find one other person (@aSemy) with this issue when I search for instances in docusaurus.config.ts. It seems like it would be more common than that. However, I found a few more examples when expanding the search from Typescript to Javascript. Some people ignored the issue and published sites with broken JSON objects. Maybe Cloudflare Analytics isn't as popular as I thought, or maybe people are using automatic mode, which can directly inject the beacon for sites proxied through Cloudflare.
Other attempts
I see other people trying various approaches to fix this JSON quoting issue:

Escape with backticks.

It doesn't work, view source line 11 of the site.

HTML-encode the double quotes.

It doesn't work, view source line 10 of the site.

Inject postBodyTags into the (soon to be deprecated) Google Analytics plugin, while escaping the double quotes.

This does work, view source line 16 of the site. It doesn't seem like the best approach.

Things I tried
I tried escaping with backticks (point 1 above) because it was a quick test. I can confirm it doesn't work. I also tried swizzling (wrap mode) the footer component to insert the link:
// theme/footer/index.tsx

import React from 'react';
import Footer from '@theme-original/Footer';
import type FooterType from '@theme/Footer';
import type { WrapperProps } from '@docusaurus/types';

type Props = WrapperProps<typeof FooterType>;

export default function FooterWrapper(props: Props): JSX.Element {
  return (
    <>

    </>
  );
}

This swizzle attempt didn't work.
My conclusion
At this point, I think the "least worst" option for hosting the beacon script in Docusaurus is @aSemy's workaround. I'll keep monitoring this issue and hope there's a cleaner solution someday.
Other options might be:

Create a new behavior plugin for Cloudflare Analytics, but I don't know if it's worth the extra effort for a uni-tasking solution.
@aSemy's suggestion to explore Eta's escapeFunction, which could be useful for other things, if I'm understanding that possibility correctly.

Cloudflare's automatic injection might be how most people do it. 🤷🏻

It is possible there is a bug in the way we escape HTML attribute values. It is quite old historical code that's not really related to ETA https://github.com/facebook/docusaurus/blob/61f71f6b69a36350dc0c28fb291dfc591904ee27/packages/docusaurus/src/server/htmlTags.ts#L69
However, are we even sure the output of Docusaurus is wrong.

<!doctype html>

  <meta charset="UTF-8">
  <meta name="generator" content="Docusaurus v3.3.2">

body

For browsers, the following HTML looks valid and gets decoded properly to a JSON value.

Can you prove that the output of Docusaurus is wrong, and that it's not a bug in the script you load?
Is there anything written in the HTML spec mentioning that " encoding is invalid as an attribute?

For browsers, the following HTML looks valid and gets decoded properly to a JSON value.

True, it looks correct, but I think that is Chrome trying to be friendly. When I right click and edit the element, I see that the value is not valid JSON.

Can you prove that the output of Docusaurus is wrong, and that it's not a bug in the script you load?

I think I'm confused about what you're asking, but I'll try and answer. The attribute value that I configured is valid JSON {"token": "1234"}. However, Docusaurus produces invalid JSON {"token": "1234"}.

This problem occurs even before the external script is loaded, so it's not a bug in the script.

Is there anything written in the HTML spec mentioning that " encoding is invalid as an attribute?

What?

Please show me the problem outside of Docusaurus:

show me a manually crafted HTML file with the bad encoding

show me a manually crafted HTML file with the encoding you expect

show me how you figure out the encoding is bad, do you see any error?

When the attribute quotes are escaped as ", I get zero hits in Cloudflare Web Analytics. After I applied a workaround, I see hits in Cloudflare.

What produces that error? Show me the JS script part reading the data attribute for example

There is no error.

TLDR I need a minimal repro, probably not even involving Docusaurus

I can reproduce the issue as follows:

In docusaurus.config.ts I added the Cloudflare script to scripts:
const config: Config = {
  scripts: [
    {
      // Cloudflare Web Analytics
      defer: true,
      async: false,
      src: "https://static.cloudflareinsights.com/beacon.min.js",
      "data-cf-beacon": "{\"token\": \"1234\"}",
    },
  ]
}

However, Docusaurus escapes the quotes in the JSON value, meaning that the JSON is invalid.
<!doctype html>

  <meta charset="UTF-8">
  <meta name="generator" content="Docusaurus v3.3.2">

  [...]

I don't understand what you mean about trying not to reproduce the problem without Docusaurus.
If I don't use Docusaurus then I don't use docusaurus.config.ts, and so I can add  manually. But that's not a reproducer for the issue...

What I mean is: how can you be sure Docusaurus is the problem, and not Cloudflare Web Analytics?
I believe what we do is fine, and that the bug might be somewhere else.
Yes, I'm asking you to troubleshoot the script Cloudflare provides you.
https://gist.github.com/slorber/bec62fcfb278d5a569e2d2c9c716504a

There might be other places in that script that break, but the code above found in the script seems to work fine.
If something is broken, please tell me what exactly by isolating things. You shouldn't need to depend on Docusaurus nor a third-party cloudflare script to prove your point, and a simple HTML file should be enough to reproduce the problem, if there is a problem in the first place.

What I mean is: how can you be sure Docusaurus is the problem, and not Cloudflare Web Analytics?

Docusaurus is the problem because it is escaping a value, meaning that Docusaurus converts valid JSON into invalid JSON. Cloudflare Web Analytics is not involved with how Docusaurus generates the HTML.

If something is broken, please tell me what exactly by isolating things.

I am trying to provide this details, but I'm really really not sure what is not clear about. If you need more details, then I would be happy to provide them, but I'm really not sure how else I can explain it.

I configure Docusaurus to add a


feature: default-run included in cargo metadata

Describe the problem you are trying to solve
I'm writing a subcommand that mirrors cargo, but records things under a time-travelling debugger instead of running them. When someone types cargo rr run I need to get whatever binary cargo would have run.
In #9491 I requested a solution for this specific use case (something like cargo build --default-bin). When looking into replicating the algorithm cargo uses as a workaround I learned cargo doesn't expose the field default_run in cargo metadata.
Describe the solution you'd like
The default_run field under [package] in Cargo.toml is included in cargo metadata
Notes
Based on a cursory look I think this would just require adding a field to SerializedPackage and changing Package::serialized to propagate the field value from Manifest. I'd be happy to file a PR for this.

We discussed this in today's @rust-lang/cargo meeting, and we'd be entirely in favor.

@danielzfranklin  @ehuss I would like to try to add it.

@rustbot claim


NeuralynxRawIO reader.parse_header() error

This is my code:
from neo.rawio import NeuralynxRawIO

reader = NeuralynxRawIO(dirname = r'filename')
reader.parse_header()

I want to load Neuralynx data, but for all my recordings I get the same Error:

AssertionError: ncs files do not have the same gaps

I am using the newest version of neo together with Python 3.6 (Spyder).
My recordings should not have any gaps, because they are continous recordings over several hours and even if there were gaps they should be the same for all channels. Can somebody help me and tell me what is going on?
Thank you,
Philipp

Hi @dieKrone!
The NeuralynxIO requires a directory for your recording session and assumes all contained files with Neuralynx extensions belong to the same session (and therefore have the same gaps). Do you have files of multiple recording sessions in the same folder? For testing purposes you can remove files from the directory to identify which of your ncs files is causing the problem.
A small note note on the side: Is there a reason you want to use the NeuralynxRawIO? This is providing fast, but very basic access to the data. If you want to load the data fully annotated into a Neo structure you should use from neo import NeuralynxIO.

Hi @JuliaSprenger
Thank you for your help. You are right one channel seems to have different gaps than the others. In all recordings it is always the same. It is the channel we fed analog triggers. It has exactly the same length but is 1.6e-05 s off compared to the other channels.
Do you have a suggestion how to deal with this? We have a lot of recordings that are like this and I rather would not like to put this channel in a different directory for every recording to be able to read the data.
thanks so much for your help!
PS: And I am using NeuralynxIO now, because there was no reason not to.

@dieKrone: When you say that one channel is 1.6e-05 s off compared to the other channels, is there actually a gap in the recording of this single channel or is the complete recording trace shifted with respect to the others which is (falsely) detected as a gap?
Would it be possible for you to provide an example file for this so I can have a look at the original data causing the problem or is the data too sensitive?

No there is no gap. The recording somehow starts at 1.6e-05 s in channels 1-16 and in channel 32 the signal starts at 0 and when I check the segment start and and times this is persistent. Yes I could definitely give you an example file, no problem. How would you like to receive the example files?
thanks again for your help!

@dieKrone Depending on the file size you can either send them directly via mail (one correct channel + the shifted one should be sufficient) or use your favourite data exchange platform, e.g. gin, gigamove if your institute is providing access there, or even dropbox or similar would work.

Sorry to jump in. Is there a way to set NeuralynxIO to ignore the gaps? It looks like the NeuralynxIO is removing the indices with gaps. When we read the same CSC file into Matlab there is 6444 sample difference (neo is less). Thanks!

Hi @neuropil! There is no flag turning off the gap detection as this would result in a the loaded data being inconsistently aligned. But, as described above, there is always the option to separate your files into different folders, that then only contain files with consistent gaps. These sets of files can then be loaded individually.
Regarding the number of samples, the IO is discarding data from incomplete data blocks as these are probably corrupted. This might explain the different number of samples you get with Matlab.

Hello @JuliaSprenger! First off, brilliant work on building this extension for reading nlx data - thank you!
Also, thank you for the helpful response! I'll definitely give the separate folder idea a try.
Best,
John

Hello @JuliaSprenger! First off, brilliant work on building this extension for reading nlx data - thank you!
Also, thank you for the helpful response! I'll give the separate folder idea a try.
Best,
John

Dear John
I've been working with NLX routinely and even contributed twice to neo. Inconsistent recordings are a bad sign for Cheetah. If it happens once - I'd handle it manually, like, moving to a different folder. If several records behave similarly, I'd first check the setup. What we had that affected the recordings

Working antivirus that constantly monitored network activity
Hard drive bottleneck in IO operations, e.g. you write and read too much.
Beta version of Cheetah.

In general, I'd prefer to record on a dedicated PC with a clean setup, if possible.
Sincerely,
Mike

I am closing this issue due to inactivity.
Suggested Solutions:

For rare occasions of this error, identify files with inconsistent gaps and separate these in different folders.
For frequent occasions of this error check the recording setup as described by @sin-mike above.