Chinese characters render garbled in iOS Simulator

Steps to Reproduce
The chinese text 中文 is garbled:
final Widget indexItem = new DrawerItem(
    icon: new Icon(Icons.home),
    child: new Row(
        children: <Widget>[
          new Flexible(child: new Text('中文')),
        ],
        ),
    );

Logs
// ignore
Flutter Doctor
[✓] Flutter (on Mac OS, channel master)
• Flutter at /Users/kuloud/Dev/flutter
• Framework revision 9c1a24fa72 (6 hours ago), 2016-11-18 21:22:32
• Engine revision cea5ed2b9b
• Tools Dart version 1.21.0-dev.6.0
[✓] Android toolchain - develop for Android devices (Android SDK 25.0.0)
• Android SDK at /usr/local/opt/android-sdk/
• Platform android-25, build-tools 25.0.0
• ANDROID_HOME = /usr/local/opt/android-sdk/
• Java(TM) SE Runtime Environment (build 1.8.0_92-b14)
[✓] iOS toolchain - develop for iOS devices (Xcode 8.1)
• XCode at /Applications/Xcode.app/Contents/Developer
• Xcode 8.1, Build version 8B62
[✓] IntelliJ IDEA Ultimate Edition (version 2016.2.5)
• Dart plugin installed
• Flutter plugin installed
[✓] Connected devices
• iPhone 7 • 3DC11AAB-EF8F-4810-B104-0C756CC86323 • ios • iOS 10.1 (simulator)

Here is a picture of new Text('中文') rendered in iPhone simulator:

Here is Genymotion emulating a Nexus 5X 6.0:

This may be iOS specific and possibly related to https://github.com/flutter/flutter/issues/4273.
FYI @chinmaygarde in case he has theories.

Probably a font issue?

I did a test with embedded fonts (based on the current documentation) for Western/English, Chinese (simplified) and Korean/Hangul. Custom fonts only get rendered for Western character set. Full project source code can be found here:
https://github.com/raju-bitter/flutter-unicode-test

Did a test run on iOS, here are my observations:

Western glyphs render ok, both for default font and custom font.
Chinese or Korean with default font, behavior depends on value of inherit property.

inherit=true: For each glyph 6 solid horizontal bars are rendered.
inherit=false: empty rectangles with black border are rendered.

Chinese or Korean with custom font: same rendering as for default font with inherit=false

The behavior is identical both for debug and release mode on iOS.

On Android, all East Asian glyphs render fine, only the custom fonts are not used.

Dart source code used: https://github.com/raju-bitter/flutter-unicode-test/blob/master/lib/main.dart

@jason-simmons or @chinmaygarde are the right folks to ask here.

Thanks for the reproducible test case. Investigating.

Thanks @raju-bitter 's help

For the record, verified that this also fails with Japanese: Hiragana 'あ', Katakana 'ア', Kanji '亜', full-width alpha-numeric characters 'ａ' and '１' and punctuation '「」【】', though some symbolics such as '○' seem to render.

Some brain dump (taking this opportunity to test a bit as well):
Most written scripts work (Arabic, Hebrew, Cyrillic, Braille, as well as Asian scripts like Mongolian, Tibetan, Manchu, Thai, Tamil, Shahmukhi)
chữ Hán doesn't work since it's basically Chinese. Curiously Devanagari doesn't work either.
Suspect it's not just a fullwidth halfwidth issue since halfwidth Japanese and Korean characters like ﾇ and ￅ fail too, though in slightly different ways.
@chinmaygarde previously checked and suspects it's failing somewhere in skia

Some partial info.
Testing with these characters:
a 97
ا 1575
א 1488
ཀ 3904
中 20013 (this doesn't work)
Some control flows in Font.cpp
Rendering character 97
  Page number 0
    No node in fallback list at page number 0
  Node level 1
    Node has page
  Page has glyph for 97 with font Helvetica
Rendering character 1575
  Page number 6
    No node in fallback list at page number 6
  Node level 1
    Node doesn't have page. Falling back.
  Node level 2
    Node has page
  Node is system fallback
  Looking for system font for character 1575
    Found system font GeezaPro
  Substituted for system font Geeza Pro
Rendering character 1488
  Page number 5
    No node in fallback list at page number 5
  Node level 1
    Node has page
    Node doesn't have page. Falling back.
  Node level 2
    Node has page
  Node is system fallback
  Looking for system font for character 1488
    Found system font ArialHebrew
  Substituted for system font Arial Hebrew
Rendering character 3904
  Page number 15
    No node in fallback list at page number 15
  Node level 1
    Node doesn't have page. Falling back.
  Node level 2
    Node has page
  Node is system fallback
  Looking for system font for character 3904
    Found system font Kailasa
  Substituted for system font Kailasa
Rendering character 20013
  Page number 78
    No node in fallback list at page number 78
  Node level 1
    Node doesn't have page. Falling back.
  Node level 2
    Node has page
  Node is system fallback
  Looking for system font for character 20013
    Found system font PingFangSC-Regular
  Can't find glyph for 20013

Seems like the problem is in the creation of the FontPlatformData for the CJK fonts in FontCacheSkia

Slight difference between the font files. All the other fonts are in
iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator.sdk/System/Library/Fonts/Core/ or
/System/Library/Fonts/Core/.
But the Chinese font is in
iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator.sdk/System/Library/Fonts/LanguageSupport/ or
/System/Library/Fonts/LanguageSupport/.
Fails in Skia's SkFontHost_mac.cpp with CoreText's call to CTFontCreateWithGraphicsFont which doesn't return a CoreGraphics's CGFontRef. Not sure how to debug into it.

In case it's useful, there are other implementations of font fallback from the projects we forked from.  e.g.
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/fonts/mac/FontPlatformDataMac.mm
https://github.com/WebKit/webkit/blob/master/Source/WebCore/platform/graphics/ios/FontCacheIOS.mm
or similar files in either repository can be helpful at times.  I'm not sure the chromium repo has iOS support, but the WebKit one definitely does.  Both of their font systems are likely different from ours, but of the same lineage at least.

Actually may not be just font locations.
Korean fails to create the font at /System/Library/Fonts/Core/AppleSDGothicNeo.ttc.
Devanagari fails at /System/Library/Fonts/Core/Kohinoor.ttc
Hiragana and Katakana (somewhat surprisingly) uses the same font as chinese at /System/Library/Fonts/LanguageSupport/PingFang.ttc

Update: discussed offline with Skia and @chinmaygarde.
We're losing the ttc index in the data chain of unicode->CTFont->CTFontDescriptor->PlatformFallbackFont->FontFaceCreationParams->filename->CGDataProvider->CGFont->CTFont->SkTypeface between the filename and CGDataProvider for mac platforms.
Instead of shoehorning that step of the existing pipeline, will try to simplify and use Skia's newer API. Remove as much of the platform specific handlings in the sky engine and use the generic APIs Skia already provides and hopefully do unicode->SkTypeface directly.
Catalog existing Blink features wrt fallback mechanisms and remove everything below FontCacheSkia::fallbackFontForCharacter and call Skia with matchFamilyStyleCharacter directly.

Sounds good to me.

I suspect the above is fixed in the current flavor of the blink pipeline (which also uses skia, but has diverged significantly from ours):
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/fonts/mac/FontPlatformDataMac.mm
However, I also think your instinct to consider re-thinking from first principles is the correct one, since we have so many fewer constraints on our engine than Blink does.  Doing so may be difficult and you might want to try and take a series of related fixes in this area as warm-ups as you slowly move us towards a simpler design (and with luck add any amount of testing along the way?). :)

Yes, agreed. I don't want to spend too much time refactoring everything.
Comparing the various implementations for different platforms on different 'branches', https://chromium.googlesource.com/chromium/blink.git/+/master/Source/platform/fonts/android/FontCacheAndroid.cpp is probably the leanest and platform agnostic. Though there is still some redundancies in data wrangling going from
unicode->SkTypeface->string->FontFaceCreationParams->SkTypeface. But inside that chain, there are still 2 layers of caching that I'll probably want to reproduce anyway if I wanted to reduce it to unicode->SkTypeface.
Will start by just removing FontCacheIOS, removing some parts of FontCacheSkia, updating FontCacheAndroid and making it FontCacheMobile. Will create a separate optimization issue for deleting more from FontCacheSkia, and reimplementing a single layer of cache.

@eseidelGoogle, I think we're talking about some kind of pixel golden test right? Can you point me to some examples? I didn't find any in the framework repo.

We removed all pixel tests from flutter/engine.  We don't have a solution at this time. :(  FYI @abarth

With #3418:

iOS looks mostly on par with Android pre-fix (well the fix merges the fallback mechanisms between the platforms and pushes it to Skia). Doesn't seem to have any regressions and mostly responds well to font family/size/weight/various decoration changes. Both iOS and Android are still missing some extended math symbols and arrows etc but that's unrelated to this bug.
One remaining fix needed is to get serif and cursive Chinese fonts working. Will create a separate bug for it but it's lower priority. I think serious Chinese app developers will probably want to bundle their own fonts anyway.

Nice! For Japanese, I believe the system fonts you'll want are HiraKakuProN and HiraMinProN for the gothic and mincho variants, respectively.

So did this fix emoji too? https://github.com/flutter/flutter/issues/4273

Will test emojis in the morning

The emojis were the same issue. Fixed as well. Turns out the apple emoji font family "Apple Color Emoji" has a few more than Android does though composite multi-codepoint emojis like profession+skin colour type emojis don't seem to render right but it's not even possible to type or paste them in Atom or emacs so probably lower priority.
iOS
Android


Header defining working the wrong way

I have been making pretty complex navigation logics in my RN-apps and everytime I'm met the same annoying feature of react-navigation: parent navigators deciding what children's headers look like.
For example in Java when you extend other class you can redefine every rule a parent class has defined and the child always has the last word in it's functions and this pretty much goes for all different situation in different coding languages and libraries.
But in react-navigation let's say I have this kind of hierarchy:
StackNavgiator1
  screen1
  DrawerNavigator
    screen2
    StackNavigator2
      screen3
      screen4

I'd like to define in DrawerNavigator's NavigationOptions that I have a default header with a button for DrawerOpen-action on the left. Then in the StackNavigator2 I have screen3 which leads to screen4 and I could navigate back with a back button overriding the DrawerOpen-button, but in the end I'll end up seeing the header defined in DrawerNavigator.
Is there anyway to patch this so that the children have the power to define their NavigationOptions? Right now the only way about this is define header at every screen which ends up showing on the screen.

Could you provide a small running example of this issue? Snack is a great tool for that

Hi! In an effort to get the hundreds of issues on this repo under control I'm closing issues tagged as questions. Please don't take this personally - it's simply something we need to do to get the issues to a manageable point. If you still have a question I encourage you to re-read the docs, ask on StackOverflow, or ask on the #react-navigation Reactiflux channel. Thank you!


Add preferences to Emoji, and print nicer names

Description

Add preferences to Emoji extension to support Unicode versions, and search by shortcodes (example: ⚡ was "high voltage" only, now also "zap")
Capitalize names for a nicer view

Type of change

Improvement for somebody else's extension. Extension author: @FezVrasta

Screencast
Before:

After:

Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing
[x] I checked that extension's icon has size of 512x512

👍

I was missing this because I can't find the pray emoji -> 🙏. Good job 😉.

@victor-falcon note that this behavior is disabled by default.
We may want to opt for enabling that by default, but that's a core change, making the extension require interned connection from time to time.

I was missing this because I can't find the pray emoji -> 🙏. Good job 😉.

Exactly the same 😂

Hey @fe9lix - made it optional.
Thanks for the feedback, very good point :)


fixed children routes to be a jsArray instead of a single route

Simple change to match the expected javascript value for children routes of a route.

LGTM, thanks!


Add remill and gap portfiles

Add remill and gap ports.
remill port allows to specify features cxx-common-llvm-14 or system-llvm to build with local install llvm or uses cxx-common provided LLVM.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.2 out of 3 committers have signed the CLA.:white_check_mark: xlauko:white_check_mark: ekilmer:x: lkorencYou have signed the CLA already but the status is still pending? Let us recheck it.


Fix issue where non-standard media types were not showing requestData

Previously, perfectly valid media types that are JSON but not an
exact match for 'application/json' e.g 'application/json; charset=charset=utf-8'
were causing requestData not to show

LGTM, except one comment, happy to merge after it is addressed

@noahdietz my bad I'll add that back.

@noahdietz - any danger of a minor NPM release with this in?

@baynezy I made a patch release: 1.4.2

Thanks so much @noahdietz


Dropdown menu has no output [Help Request]

Lighthouse is executing commands, but the dropdown is not displaying results.
I am running and building this on gentoo. The build was created without any errors and the config file seems fine.  The only conclusion I have is that either I am missing one of the required libraries, or that my path is somehow incorrect.

.. The dropdown shows one result in your screenshot.
???
Are you sure you're not just generating extra newlines? Lighthouse reads up to the next newline to get the list of options.

You are using the example shell script "cmd". It just outputs "look!" followed by the input. To get some more features change
cmd=~/.config/lighthouse/cmd

in the rc file to
cmd=~/.config/lighthouse/cmd.py

And make sure that cmd.py is executable.


Allow helm chart to be used as subchart

Hi,
As this chart has an enforced values schema, it cannot be deployed as a subchart. This is because global is not an allowed property in the schema.
Solution:
Add global property in the value schema:
Add this in line 11:
  "properties": {
    "global": {
      "type": "object"
    },

How to reproduce this:
Create the following folder structure (or just type helm create parentChart ):

parentChart

charts

Chart.yaml (see below)
values.yaml (contain general values)

values:

param1.yaml (param1 specific values)
param2.yaml (param2 specific values)

Chart.yaml contains:
apiVersion: v2
name: parentChart
version: 1.0.0
dependencies:
  - name: splunk-otel-collector
    version: 0.39.0
    repository: https://signalfx.github.io/splunk-otel-collector-chart

Value files have this format:
splunk-otel-collector:
  - clusterName: myCluster
  - rest config here....

run helm install parentChart charts --dry-run --values values/param1.yaml
Output:
splunk-otel-collector:
- (root): Additional property global is not allowed

@illrill thanks for taking care of it


Improve unset pool error

This enhancement was in reaction to a user's experience in a community Slack thread. In their own words:

zed log isn’t working for me right now … dunno if I’m just missing something or what. I’ve tried a couple of things to troubleshoot this … (still on 1.16.0)
❯ zed init lake
lake created: file:///Users/chrismo/dev/ds6/ops/platform/aws/tf/kms/lake

❯ export ZED_LAKE=lake

❯ zed create keys
pool created: keys 2k157GhxTaVVMCOc0YGjaxgacMN

❯ aws kms list-keys --profile staging --region us-east-2 | zq -z 'over this.Keys' - | zed load -use keys -
(11/1) 792B 792B/s
2k158CfFyGJfCBFlwsXNAKFPwhf committed

❯ zed log
empty pool and branch

They were informed of the role of zed use here and that got them up and running. But their experience did highlight that the error message could be improved to help new users bumping into this for the first time.


test files which are not  in the 'test' dir than seems won't work

good one:

bad one:

test with same file: power.js
var assert = require('assert');

describe('Array', function(){
    beforeEach(function(){
        this.ary = [1,2,3];
    });
    describe('#indexOf()', function(){
        it('should return index when the value is present', function(){
            var zero = 0, two = 2;
            assert(this.ary.indexOf(zero) === two);
        });
    });
});

I have one project that the test files are not in the test dir .
please help me. thanks!

node version 4.4.0
intelli-espower-loader": "^1.0.1",
"mocha": "^3.2.0",

Maybe, You could require power-assert insteadof assert and resolve this issue.
var assert = require('assert');

should be
var assert = require('power-assert');

@kaidiren Thank you for your question. If you are using intelli-espower-loader and having tests in directory other than test, you have to set test directory in your package.json explicitly.

@twada  thanks!

I close this issue since this issue seems to be fixed by my answer. Please feel free to reopen this issue if you have any problems.


[General Inquiry]: Cloudwatch CDN NextJS

What is your general inquiry?
I am using Nextjs v10. I want to use CDN script tag approach to log and monitor the application.
To log the JS error, I build Error boundary class and I wanna log JS error to cloudwatch RUM
declare function cwr(operation: string, payload: any): void;
class App extends Component {
...
componentDidCatch(error, info) {
console.log(error);
cwr('recordError', error);
};
...
}
But, I am getting error while building the application. It throws error at declare
Syntax error: Unexcepted token, expected ";"
### Tasks

componentDidCatch(error, info) {
console.log(error);
cwr('recordError', error);
}; // remove this semicolon


GPS support

Describe at a high level the solution you're providing
Supporting GPS in-house through TinyGPS++ with an intuitive API.
Is this a patch, a minor version change, or a major version change
minor.
Is this related to an open issue?
#311
Additional context
WIP right now

Decided to bring the printData() helper function internal and use the DuckLogger interface since there seemed to be some confusion about its purpose. It was originally intended for the developer to use at will without having to use debug flags. It now uses the debug log level and people will just have to enable debugging for everything if they want to see the printout.

#if defined(CDPCFG_GPS_RX) && defined(CDPCFG_GPS_TX)
#include <DuckGPS.h>
#endif

This is how the preprocessor guards are going to be used in cdpcfdg.h after the board definition section. It'll look for the receive and transmit pin definitions to determine whether to auto-include GPS stuff. I also added a preprocessor guard on one of the initializers that take from the board config. This way people can still use the DuckGPS API if they define the pins themselves.


MARP-947 re structure product module content in product

Please handle this case: When clicking Download button, choose version dropdown, content tabs and version of information column disappear.


format yaml and use dnf as example

Format the YAML files
Use DNF instead of YUM in the examples
Update documentation links

Closes #9348
Signed-off-by: Thomas Sjögren<EMAIL_ADDRESS>

@JohanFreelancer9 please do the copy editor review for this PR.

https://github.com/MicrosoftDocs/windows-itpro-docs/blob/public/windows/security/threat-protection/microsoft-defender-atp/linux-install-with-ansible.md has been deleted?

Actually, it is the directory /microsoft-defender-atp/ which has been renamed or deleted. I am still trying to trace the changes in https://github.com/MicrosoftDocs/windows-itpro-docs/tree/public/windows/security/threat-protection/ to see if I can find if the file exists at all. Maybe the file needs to be recreated by using the WayBack Machine (https://web.archive.org)?

Looks like all the linux-install-* files have been removed along with the directory. I don't know how others react to this event, but it seems a bit too destructive to me.

As mentioned in the issue ticket:
"... all of the MDE (formerly known as Defender ATP) documentation [has been moved] to the microsoft-365-docs repo and also [...] updated all of the branding."
https://github.com/MicrosoftDocs/windows-itpro-docs/issues/9348#issuecomment-807608222

Thanks @illfated.


Added some missing functions to $log

I find myself using these console functions a lot, so I decided to add them to $log. This makes it easier to turn off or find/remove all my debug stuff before deployment and looks more consistent in the code.

I signed it!

The methods $log provides are not meant to mirror every method console provides. $log only writes by default to console, but if you have another implementation, for example writing to a remote log service, methods like table etc. would not really make a lot of sense. On the other hand, log, info, warn, error and debug are common error levels. I suggest you use a decorator to add the missing methods if you use them often.


Synced blocks wrongly produce indented Markdown output

Hello,
Thanks for the good work, this package is very useful to me !
General context
I'm trying to build a Notion to Hugo process using notion-to-md, which I simply called notion-to-hugo. I want to use it to build online courses in the most generic way I can, allowing content creators to leverage the full potential of Notion while ensuring that the final website works in our course setup.
I want to be able to customize the rendering process to be able to adapt the generated content for the particular Hugo settings that I use, including shortcodes for instance.
I use Notion Synced Block to reuse content across pages.
I started to work on a generic pre/post-processing pipeline here. I was very happy to see setCustomTransformer emerging, as I like the direction it's heading !
The issue
You can see that in this Notion page there is a Notion synced block. When processed with notion-to-md, the Markdown MdBlock output is the following :
	This block is shared with other pages and editing one changes all other instances.

	> 💡 This is a callout which turns into a `<div class="notices note"></div>` block.

As you can see, the content is indented, leading to the paragraph being converted as a code block by Hugo, which you can see here on the generated HTML page produced by notion-to-hugo through notion-to-md.
Analysis
The block structure produced by notion-to-md is the following :
  {
    "type": "synced_block",
    "parent": "",
    "children": [
      {
        "type": "paragraph",
        "parent": "This block is shared with other pages and editing one changes all other instances.",
        "children": []
      },
      {
        "type": "callout",
        "parent": "> 💡 This is a callout which turns into a `<div class=\"notices note\"></div>` block.",
        "children": []
      }
    ]
  }

We can see that the actual content blocks are children of the synced block. The synced block is only a container for actual content, and I don't see any reason why we should treat its children as real children.
The indent is made here :
      if (mdBlocks.parent) {
        if (
          mdBlocks.type !== "to_do" &&
          mdBlocks.type !== "bulleted_list_item" &&
          mdBlocks.type !== "numbered_list_item"
        ) {
          // add extra line breaks non list blocks
          mdString += `\n${md.addTabSpace(mdBlocks.parent, nestingLevel)}\n\n`;
        } else {
          mdString += `${md.addTabSpace(mdBlocks.parent, nestingLevel)}\n`;
        }
      }

My guess is that we should not addTabSpace to children of blocked whose type is synced_block.
Attempts at solving the issue
I tried the following things :

remove the leading spaces in the generated md output with one of my own post-processor : I seem to lack information about the blocks if I plug it after generating the Markdown output ;
use setCustomTransformer to deal with the particular case : I did not find a proper way to use it for my case ;
fiddle with toMarkdownString to deal with synced_block : I can't get it to work as I could find no way of determining the parent's type here.

I'm feeling stuck, probably by my lack of understanding of some of the processing flows... Would you be able to help me sort it out ? I hope I provided enough information for that.
Thanks a lot !

Hi there. Thanks for reporting the issue and the PR #44 .
For the time your PR will fix the issue so I'll merge it, however I believe this may be the issue with other blocks as well so I'll release a fix in coming days.

Hi,
Thanks for the quick answer and merge ! I think that this would need to be investigated further as well.
Best regards

The fix is now live in v2.5.3.
Thanks for the contribution :)

Hi,
Thanks for the quick answer and merge ! I think that this would need to be investigated further as well.
Best regards

yes, I'm on it.


c-inline ECL patch

You mention that ECL has to be patched to keep intermediate files. Doesn't runtime
(setf compiler::*delete-files* nil)
work OK for that?

Haha.  Yeah, not sure why I couldn't figure that out and felt like I needed to create a patch.  But that works perfectly.    I've updated the document to use the special valiable rather than the patch.


fix: reconciler Pooler RoleBinding

This patch ensures that we reconcile the RoleBinding in case of differences.
Previously, only the RoleBinding creation was executed.

/test limit=local

/test limit=local


Library can't be installed

I created the project using react-native init.
I can't be able to install the library.
react-native info
System:
  OS: Linux 4.15 Ubuntu 18.04.2 LTS
  CPU: (6) x64 AMD FX(tm)-6300 Six-Core Processor
  Memory: 794.44 MB / 3.59 GB
  Shell: 4.4.19 -
Binaries:
  Node: 8.16.0
  npm: 6.4.1
npmPackages:
  react: 16.6.3 => 16.6.3
  react-native: 0.58.3 => 0.58.3
npmGlobalPackages:
  react-native-cli: 2.0.1

npm install @react-native-community/image-editor --save
Not working, generates the same error that @renishdeveloper got.

Will fix it

Should be fixed in v1.3.0


Question about pwd logic when loading mhd

Hi! Thank you for the library.
My question is about https://github.com/yanlend/mhd_utils/blob/master/mhd_utils/init.py#L84.
In case when meta_dict['ElementDataFile'] contains an absolute filepath, and we do load_raw_data_with_mhd(another_absolute_filepath) => an error is raised because we concatenate both absolute filepaths: data_file = pwd + '/' + meta_dict['ElementDataFile']. Could we make this logic with pwd optional or remove it at all? I can prepare a PR, if you want.
All the best

In my opinion, absolute file paths don't make much sense for the this format. Usually, the raw file and the mhd file are in the same folder. With absolute paths, you cannot move the image to a different path without editing the mhd

Technically, it would not be an issue to implement this.

Would it be possible to make a flag argument that switches this behavior?
def load_raw_data_with_mhd(filename, pwd=True):

By default it would be the existing behavior but with pwd=False we can turn it off.

I updated the package, it should work now. Can you try it please?

Yep, it works. Thank you!


Interface improvements

CLI and Node/Browser options should match

CLI
Node/Browser

--depth
maxDepth

Node/Browser JSON option, since is mandatory and the other aren't we should make it the first argument and the second should be all the non mandatory, it will make more closer to the behaviour of the CLI
Node/Browser JSON option shouldn't be called JSON or we should convert it not in the cli but in the Node/Browser library code and add there the validation
We should rename the cli (in the package) from json-analyzer.js to json-analyzer

Would be useful to do this before doing the asciicinema/svg-term-cli animations in the README and gh-page

work in progress: https://github.com/Fetz/json-analyzer/tree/feature/issue-49-interface-improvement


'fs' is no longer "requireable" and throws errors

The modules 'fs' is now in node core. Lines like
[this] (https://github.com/visionmedia/debug/blob/39ecd87bcc145de5ca1cbea1bf4caed02c34d30a/node.js#L163)
throw error:

Module not found: Error: Can't resolve 'fs'

it throws error even after

npm install --save fs

I've cleaned cache and upgraded my npm to 3.10.7 from 3.8 and tested again.
With

var fs=require('fs');
new version of npm doesn't throw error so existing, works without changes to it.


CI/CD workflows for spec repositories

Proposal sketched up with @DylanVanAssche for CI/CD workflows for the spec repositories, to keep specs and ontology/shapes in line, and also do some automatic validations / generation where applicable.
shapes coverage test
Create a GitHub action to be used by all spec repo's that contains generic CI/CD functionality.
As input the action 0 or more other spec-repositories can be specified on which this spec-repo is dependent.
The action then collects the necessary artefacts from the repo's and executes the following steps:

Syntax-validation/parse ontologies (if error is encountered give clear msg with offending repo and file)
Validate / Test shapes
Validate examples

All spec repo's get the following directory structure
[spec-repo]
├── model/
│   ├── ontology/       # Ontology reflecting the spec ( this will be combined with rml-core and possibly other specs )
│   └── shapes/         # Shapes reflecting the spec
│       └── tests/      # Test cases which cover the shapes
└── examples/           # Examples used in the spec

Under model/shapes/tests we have test cases that cover the shapes reflecting the spec developed in the current repo.
This helps us ensure that our shapes and ontology are valid and stay up to date with the spec.
Under examples/ we have all examples that are used in the text of the spec. We place them in this standard location
so that they can also be validated against the shapes used in the previous step.
On each push to any branch, these tests will be run.
PR templates
We will introduce PR templates including a checklist which reminds us to:

make sure that the model is in sync with the spec.

Generate combined model
We can introduce a merge action which triggers a commit or PR to update the full model (in rml-core repo?)

Update: CI/CD is merged with SHACL shapes and test-cases so we're almost there for all specs.


Add editing actions to ImageViewer

Add editing actions:

This pull request is under development.
If someone feels this useful, please share your ideas!

The features you're adding are very interesting and we needed similar logic to this tool :)
Are you going to put those actions inside a sub-menu later on? (eg. Edit menu)

For what it's worth, I would prefer these buttons outside of the menu in this instance. Having them easily presented and accessible for such a simple tool is ideal in my opinion. Otherwise, so far it looks like a great addition to have!

This time I have put editors under Edit menu.

In the future, it will be better if it can implement thumbnail list (and action menu too) like the following design sample:

This will be better

Thanks for reviewing this!
I'll try to fix it by using assembly reference.


Script to generate ATC list

Only ATCs (and TSC-approved ATCs) are allowed to run for and vote in elections. According to the Community Governance doc, an ATC is defined as:

<IP_ADDRESS>.1 Active Technical Contributor (ATC)
Active contribution to the TF project by contributing code/patches, which are approved by the project committers. The contribution is not limited to code, it could be documentation or other contributions. At least one approved/merged patch in one release is required in the preceding 12 months prior to the election.

We need a script to generate this list so we don't have to rely on someone with Gerrit/GitHub fu to do it manually.
This script should look not only at review.opencontrail.org (for now; will be tungstenfabric in time instead of opencontrail) but also at this documentation repository. It should allow for passing date ranges rather than simply defaulting to the 12 months prior to when the script is run.
The output should be, at a minimum, the community member's name and the email address they used for their merged contribution.
Please commit the script to the Governance directory of this repository.

Migrated to Jira: https://jira.tungsten.io/browse/TFB-1393


concurrentAll/mapConcurrentAll

methods that attempt every input function/item, and do not bail on the first error encountered.

See #12


Configmap fix env from change

Description:
This PR modifies

The deployment template to use envFrom for the secret and configmap.
Removes annotation_JSON and will use the suggestion [here] (https://github.com/hashgraph/hedera-json-rpc-relay/pull/522#discussion_r971487256)
Updates the configmap to quote all the values.  During deployments this error [debug] ConfigMap in version "v1" cannot be handled as a ConfigMap: v1.ConfigMap.Data: ReadString: expects " or n, but found 2, error found in and kubectl apply -f templates/test.yaml  The Deployment "test" is invalid: spec.template.spec.containers[0].env[15].valueFrom: Invalid value: "": may not be specified when value is not empty.  Both of these error occurrence suggest that quoting values is needed for the pods to accept them correctly.  Here is one of the issues being referenced.

Notes for reviewer:

WIP - rollout testing will happen 2022-12-13 and 2022-12-14
Checklist

[ ] Documented (Code comments, README, etc.)
[ ] Tested (unit, integration, etc.)

Codecov Report
Base: 72.87% // Head: 72.89% // Increases project coverage by +0.02% :tada:

Coverage data is based on head (6b8b8d0) compared to base (75dc43f).
Patch coverage: 100.00% of modified lines in pull request are covered.

:exclamation: Current head 6b8b8d0 differs from pull request most recent head 3809ca0. Consider uploading reports for the commit 3809ca0 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #767      +/-   ##
==========================================
+ Coverage   72.87%   72.89%   +0.02%
==========================================
  Files          16       16
  Lines        1246     1247       +1
  Branches      231      232       +1
==========================================
+ Hits          908      909       +1
  Misses        280      280
  Partials       58       58

Impacted Files
Coverage Δ

packages/relay/src/lib/eth.ts
83.75% <100.00%> (+0.02%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


feature: Support stap

Contact Details
<EMAIL_ADDRESS>Feature Description
Add stap to the managed packages
Why?
It provides a frontend for interacting with the systemtap tool for probing
What-should happen?
stap should be a package in the index.json
Steps to Test

jq '.packages[] | select(.name = "stap")' resources/install/index.json should return the details for managing the stap package

Code of Conduct

[X] I agree to follow this project's Code of Conduct

https://sourceware.org/systemtap/wiki/SystemtapOnUbuntu


add functions CRUD api & model

this simply plumbs the function API model and CRUD endpoints, without doing
any removal of existing constructs in fn. that is, it is possible to
create/read/update/delete functions without the ability to invoke them at all,
whatsoever. the hope is that this reiterates that we have too many test
suites, reiterates that my karma is far more negative than originally thought,
and this allows the CLI and other datastore implementers to get unblocked on
this, and we can then begin the re-plumbing of agent to invoke functions
rather than routes on top of this.
note that this does not reflect extremely recent discussions around certain
fields on a func, it should be rather easy to modify this code to change/add
any fields that we want to. we need to put them on an app as we've decided,
which is pretty straightforward from this, too. The other thing that sticks
out is that PUT behaves very much like PATCH and there is maybe a discussion
to have around that (esp. keeping in mind that we're adding revisions).
we can quickly rename this to fns too, nothing final here.

one thing to figure out maybe while we're at this point is do we want to use function ids in the backend (datastore) or names? currently this uses names, and app id is, well, app id, if we want to do things that way. Not clear the best path forward there.

@mantree @riconnon do we want to merge this to fn-v2.0? we can use it as a WIP branch to stage work into if we'd like.
afaict the immediate work to do here:

rename to fns
put on app
maybe use id in same vein as app id?

with maybe now or a little bit later:

revisions (across apps, funcs, and triggers - maybe do all at once)
status (across funcs and triggers - maybe do all at once)
change config behavior
docker image id - we want the client to provide this, probably sep discussion item, not urgent

we can close, assume #1062 supercedes


[SPARK-50031][SQL][FOLLOW-UP] Add tests for try_parse_url in the connect module

What changes were proposed in this pull request?
Add tests in PlanGenerationTestSuite for try_parse_url function introduced in this PR and generate the corresponding golden files.
Why are the changes needed?
The added tests ensure the new function behaves as expected.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Tests added.
Was this patch authored or co-authored using generative AI tooling?
No.

+1, LGTM. Merging to master.
Thank you, @jovanm-db and @LuciferYang for review.


Validation rules for Client Controlled Nullability operators

Here is a prroposed list of validations with nullability operators. I am sure there are cases that I missed here so please comment below if you see any cases missed. The current changeset on graphql-js was made with this list in mind, but we need to audit and make sure they behave per this list.
Schema + Operation validation
When types are not mutually exclusive (same parent / interface)
When a field is nullable
With, or without aliases

!
?
not-marked

!
valid
(duplicate)
(duplicate)

?
invalid
valid
(duplicate)

not-marked
valid
valid
valid

When a field is non-nullable
With, or without aliases

!
?
not-marked

!
valid
(duplicate)
(duplicate)

?
invalid
valid
(duplicate)

not-marked
valid
valid
valid

When types are exclusive
Here, with aliases, all combinations are valid.

[ ] Do a double pass to ensure that marking a nullable field non-nullable stays backward compatible.

Fields are both nullable

!
?
not-marked

!
valid
(duplicate)
(duplicate)

?
invalid
valid
(duplicate)

not-marked
invalid
valid
valid

Fields are both non-nullable

!
?
not-marked

!
valid
(duplicate)
(duplicate)

?
invalid
valid
(duplicate)

not-marked
valid
invalid
valid

Field A is nullable and Field B is non-nullable

! on B
? on B
Not-marked on B

! on A
valid
invalid
valid

? on A
invalid
valid
invalid

Not-marked A
invalid
valid
invalid

Operation only validation
Given fragment A and fragment B (one of them could just be a selection set)

! on B
? on B
Not-marked on B

! on A
valid
invalid
valid

? on A
invalid
valid
valid

Not-marked A
valid
valid
valid

🎉 thanks for this!
I think I'm following along the gist, but could we add example schema + queries / fragments and label the axes? My brain is having a little trouble keeping this in order :P

When a field is nullable
With, or without aliases

!
?
not-marked

!
valid
(duplicate)
(duplicate)

?
invalid
valid
(duplicate)

not-marked
valid
valid
valid

ok here's my stab - maybe something like:
When a field is nullable
Schema
type Query {
  greeting: String
}

Query
{
  greeting
}

With, or without aliases

Query.greeting! (schema)
Query.greeting? (schema)
Query.greeeting not-marked (schema)

greeting! (query)
valid
(duplicate)
(duplicate)

greeting? (query)
invalid
valid
(duplicate)

greeting not-marked (query)
valid
valid
valid

Ok this is pretty gross, maybe there's a better way to actually format this - but I think some sort of visualization aid might be helpful here?
Thanks again, I think these tables to enumerate all possibilities arre super helpful to work out the subtleties

Actually I'm being dumb aren't I - the two axes are both for fragments? (What even is Query.greeting?, that's not a thing in SDL or being proposed right?)

Definitely, I will add more labels to the tables. I also liked your suggestion to add some examples to ground the cases in the tables so I will do that as well.

Updated the issue with examples and added the current status of tests we have. I marked open questions with the checkboxes.


Research perf improvements for "contains" searches

contains currently uses a regex. I don't remember precisely why I did it that way in the past, but I believe I researched other options back then and found that they lacked some functionality. However, regex searches are obviously not the most performant ones with large data volumes.
This StackOverflow discussion mentions the option of using $indexOfCP:
db.test.aggregate([{$match:
    {$expr: { $gt: [{ $indexOfCP: [ "$A", "Star Wars" ] }, -1]}}
    }, {$project: {A:1}}])

There is also a suggestion of adding $toUpper as needed for case insensitive searches. Current MongoDB also supports $substrBytes and $substrCP.

Strangely, the docs do not mention whether any of these functions take advantage of existing indexes. The regex based query does not, understandably, but do these other functions? Is there a MongoDB docs page that describes index use per operator?

A simple $regex query with a basic collection, unchanged after a data import, looks like this:
> db.data.explain("executionStats").aggregate([{"$match": { "ProductName": { "$regex": "^contoso battery charger", "$options": "i" } } }])
{
	"queryPlanner" : {
		"plannerVersion" : 1,
		"namespace" : "demo.data",
		"indexFilterSet" : false,
		"parsedQuery" : {
			"ProductName" : {
				"$regex" : "^contoso battery charger",
				"$options" : "i"
			}
		},
		"queryHash" : "0D240FCD",
		"planCacheKey" : "0D240FCD",
		"optimizedPipeline" : true,
		"winningPlan" : {
			"stage" : "COLLSCAN",
			"filter" : {
				"ProductName" : {
					"$regex" : "^contoso battery charger",
					"$options" : "i"
				}
			},
			"direction" : "forward"
		},
		"rejectedPlans" : [ ]
	},
	"executionStats" : {
		"executionSuccess" : true,
		"nReturned" : 1563,
		"executionTimeMillis" : 543,
		"totalKeysExamined" : 0,
		"totalDocsExamined" : 1000000,
		"executionStages" : {
			"stage" : "COLLSCAN",
			"filter" : {
				"ProductName" : {
					"$regex" : "^contoso battery charger",
					"$options" : "i"
				}
			},
			"nReturned" : 1563,
			"executionTimeMillisEstimate" : 1,
			"works" : 1000002,
			"advanced" : 1563,
			"needTime" : 998438,
			"needYield" : 0,
			"saveState" : 7812,
			"restoreState" : 7812,
			"isEOF" : 1,
			"direction" : "forward",
			"docsExamined" : 1000000
		}
	},
	"serverInfo" : {
		"host" : "b1c70a1e87a3",
		"port" : 27017,
		"version" : "4.2.0",
		"gitVersion" : "a4b751dcf51dd249c5865812b390cfd1c0129c30"
	},
	"ok" : 1
}

A search for a specific field also doesn't use an index - obviously, we don't have one.
> db.data.explain("executionStats").aggregate([{"$match": { "ProductName": "Contoso Battery charger - bike E200 Black" } }])
{
	"queryPlanner" : {
		"plannerVersion" : 1,
		"namespace" : "demo.data",
		"indexFilterSet" : false,
		"parsedQuery" : {
			"ProductName" : {
				"$eq" : "Contoso Battery charger - bike E200 Black"
			}
		},
		"queryHash" : "C070B184",
		"planCacheKey" : "C070B184",
		"optimizedPipeline" : true,
		"winningPlan" : {
			"stage" : "COLLSCAN",
			"filter" : {
				"ProductName" : {
					"$eq" : "Contoso Battery charger - bike E200 Black"
				}
			},
			"direction" : "forward"
		},
		"rejectedPlans" : [ ]
	},
	"executionStats" : {
		"executionSuccess" : true,
		"nReturned" : 492,
		"executionTimeMillis" : 426,
		"totalKeysExamined" : 0,
		"totalDocsExamined" : 1000000,
		"executionStages" : {
			"stage" : "COLLSCAN",
			"filter" : {
				"ProductName" : {
					"$eq" : "Contoso Battery charger - bike E200 Black"
				}
			},
			"nReturned" : 492,
			"executionTimeMillisEstimate" : 1,
			"works" : 1000002,
			"advanced" : 492,
			"needTime" : 999509,
			"needYield" : 0,
			"saveState" : 7812,
			"restoreState" : 7812,
			"isEOF" : 1,
			"direction" : "forward",
			"docsExamined" : 1000000
		}
	},
	"serverInfo" : {
		"host" : "b1c70a1e87a3",
		"port" : 27017,
		"version" : "4.2.0",
		"gitVersion" : "a4b751dcf51dd249c5865812b390cfd1c0129c30"
	},
	"ok" : 1
}

With an index, the specific query obviously uses it:
> db.data.createIndex({ProductName:1})
{
	"createdCollectionAutomatically" : false,
	"numIndexesBefore" : 1,
	"numIndexesAfter" : 2,
	"ok" : 1
}
> db.data.explain("executionStats").aggregate([{"$match": { "ProductName": "Contoso Battery charger - bike E200 Black" } }])
{
	"queryPlanner" : {
		"plannerVersion" : 1,
		"namespace" : "demo.data",
		"indexFilterSet" : false,
		"parsedQuery" : {
			"ProductName" : {
				"$eq" : "Contoso Battery charger - bike E200 Black"
			}
		},
		"queryHash" : "C070B184",
		"planCacheKey" : "55C5EB1A",
		"optimizedPipeline" : true,
		"winningPlan" : {
			"stage" : "FETCH",
			"inputStage" : {
				"stage" : "IXSCAN",
				"keyPattern" : {
					"ProductName" : 1
				},
				"indexName" : "ProductName_1",
				"isMultiKey" : false,
				"multiKeyPaths" : {
					"ProductName" : [ ]
				},
				"isUnique" : false,
				"isSparse" : false,
				"isPartial" : false,
				"indexVersion" : 2,
				"direction" : "forward",
				"indexBounds" : {
					"ProductName" : [
						"[\"Contoso Battery charger - bike E200 Black\", \"Contoso Battery charger - bike E200 Black\"]"
					]
				}
			}
		},
		"rejectedPlans" : [ ]
	},
	"executionStats" : {
		"executionSuccess" : true,
		"nReturned" : 492,
		"executionTimeMillis" : 4,
		"totalKeysExamined" : 492,
		"totalDocsExamined" : 492,
		"executionStages" : {
			"stage" : "FETCH",
			"nReturned" : 492,
			"executionTimeMillisEstimate" : 0,
			"works" : 493,
			"advanced" : 492,
			"needTime" : 0,
			"needYield" : 0,
			"saveState" : 3,
			"restoreState" : 3,
			"isEOF" : 1,
			"docsExamined" : 492,
			"alreadyHasObj" : 0,
			"inputStage" : {
				"stage" : "IXSCAN",
				"nReturned" : 492,
				"executionTimeMillisEstimate" : 0,
				"works" : 493,
				"advanced" : 492,
				"needTime" : 0,
				"needYield" : 0,
				"saveState" : 3,
				"restoreState" : 3,
				"isEOF" : 1,
				"keyPattern" : {
					"ProductName" : 1
				},
				"indexName" : "ProductName_1",
				"isMultiKey" : false,
				"multiKeyPaths" : {
					"ProductName" : [ ]
				},
				"isUnique" : false,
				"isSparse" : false,
				"isPartial" : false,
				"indexVersion" : 2,
				"direction" : "forward",
				"indexBounds" : {
					"ProductName" : [
						"[\"Contoso Battery charger - bike E200 Black\", \"Contoso Battery charger - bike E200 Black\"]"
					]
				},
				"keysExamined" : 492,
				"seeks" : 1,
				"dupsTested" : 0,
				"dupsDropped" : 0
			}
		}
	},
	"serverInfo" : {
		"host" : "b1c70a1e87a3",
		"port" : 27017,
		"version" : "4.2.0",
		"gitVersion" : "a4b751dcf51dd249c5865812b390cfd1c0129c30"
	},
	"ok" : 1
}

... and so does the regex query:
> db.data.explain("executionStats").aggregate([{"$match": { "ProductName": { "$regex": "^contoso battery charger", "$options": "i" } } }])
{
	"queryPlanner" : {
		"plannerVersion" : 1,
		"namespace" : "demo.data",
		"indexFilterSet" : false,
		"parsedQuery" : {
			"ProductName" : {
				"$regex" : "^contoso battery charger",
				"$options" : "i"
			}
		},
		"queryHash" : "0D240FCD",
		"planCacheKey" : "829CC92A",
		"optimizedPipeline" : true,
		"winningPlan" : {
			"stage" : "FETCH",
			"inputStage" : {
				"stage" : "IXSCAN",
				"filter" : {
					"ProductName" : {
						"$regex" : "^contoso battery charger",
						"$options" : "i"
					}
				},
				"keyPattern" : {
					"ProductName" : 1
				},
				"indexName" : "ProductName_1",
				"isMultiKey" : false,
				"multiKeyPaths" : {
					"ProductName" : [ ]
				},
				"isUnique" : false,
				"isSparse" : false,
				"isPartial" : false,
				"indexVersion" : 2,
				"direction" : "forward",
				"indexBounds" : {
					"ProductName" : [
						"[\"\", {})",
						"[/^contoso battery charger/i, /^contoso battery charger/i]"
					]
				}
			}
		},
		"rejectedPlans" : [ ]
	},
	"executionStats" : {
		"executionSuccess" : true,
		"nReturned" : 1563,
		"executionTimeMillis" : 703,
		"totalKeysExamined" : 1000000,
		"totalDocsExamined" : 1563,
		"executionStages" : {
			"stage" : "FETCH",
			"nReturned" : 1563,
			"executionTimeMillisEstimate" : 7,
			"works" : 1000001,
			"advanced" : 1563,
			"needTime" : 998437,
			"needYield" : 0,
			"saveState" : 7812,
			"restoreState" : 7812,
			"isEOF" : 1,
			"docsExamined" : 1563,
			"alreadyHasObj" : 0,
			"inputStage" : {
				"stage" : "IXSCAN",
				"filter" : {
					"ProductName" : {
						"$regex" : "^contoso battery charger",
						"$options" : "i"
					}
				},
				"nReturned" : 1563,
				"executionTimeMillisEstimate" : 6,
				"works" : 1000001,
				"advanced" : 1563,
				"needTime" : 998437,
				"needYield" : 0,
				"saveState" : 7812,
				"restoreState" : 7812,
				"isEOF" : 1,
				"keyPattern" : {
					"ProductName" : 1
				},
				"indexName" : "ProductName_1",
				"isMultiKey" : false,
				"multiKeyPaths" : {
					"ProductName" : [ ]
				},
				"isUnique" : false,
				"isSparse" : false,
				"isPartial" : false,
				"indexVersion" : 2,
				"direction" : "forward",
				"indexBounds" : {
					"ProductName" : [
						"[\"\", {})",
						"[/^contoso battery charger/i, /^contoso battery charger/i]"
					]
				},
				"keysExamined" : 1000000,
				"seeks" : 1,
				"dupsTested" : 0,
				"dupsDropped" : 0
			}
		}
	},
	"serverInfo" : {
		"host" : "b1c70a1e87a3",
		"port" : 27017,
		"version" : "4.2.0",
		"gitVersion" : "a4b751dcf51dd249c5865812b390cfd1c0129c30"
	},
	"ok" : 1
}

Even without the '^', the index is still being used. This is unexpected - I need to check this further.

Ah yes... it's a bit strange. Even with the $regex condition, the execution plan mentions the index. It appears to use it for the query as well. However, when the index is used with a specific field query, I see these stats:
	"executionStats" : {
		"executionSuccess" : true,
		"nReturned" : 492,
		"executionTimeMillis" : 4,
		"totalKeysExamined" : 492,

When using the $regex query, the stats show this:
	"executionStats" : {
		"executionSuccess" : true,
		"nReturned" : 1563,
		"executionTimeMillis" : 703,
		"totalKeysExamined" : 1000000,
		"totalDocsExamined" : 1563,

The regex case "examines" a million (all) keys, not just a few - so the index doesn't actually do anything to speed up the regex case. In fact, in several runs it was never faster than the non-index search I documented above (543 millis).
It is not surprising to me that the index doesn't improve performance for the $regex query. It is only surprising that the chosen plan uses that index, especially since it appears to slow down performance. I would have expected MongoDB to realize that the index won't improve things for the $regex query, and to decide against using it.

Now testing $indexOfCP:
> db.data.explain("executionStats").aggregate([{"$match": { $expr: {$gt:[ {$indexOfCP: ["$ProductName", "Contoso Battery charger"] } , -1 ] } } }])
{
	"queryPlanner" : {
		"plannerVersion" : 1,
		"namespace" : "demo.data",
		"indexFilterSet" : false,
		"parsedQuery" : {
			"$expr" : {
				"$gt" : [
					{
						"$indexOfCP" : [
							"$ProductName",
							{
								"$const" : "Contoso Battery charger"
							}
						]
					},
					{
						"$const" : -1
					}
				]
			}
		},
		"queryHash" : "0C022C46",
		"planCacheKey" : "0C022C46",
		"optimizedPipeline" : true,
		"winningPlan" : {
			"stage" : "COLLSCAN",
			"filter" : {
				"$expr" : {
					"$gt" : [
						{
							"$indexOfCP" : [
								"$ProductName",
								{
									"$const" : "Contoso Battery charger"
								}
							]
						},
						{
							"$const" : -1
						}
					]
				}
			},
			"direction" : "forward"
		},
		"rejectedPlans" : [ ]
	},
	"executionStats" : {
		"executionSuccess" : true,
		"nReturned" : 1563,
		"executionTimeMillis" : 2038,
		"totalKeysExamined" : 0,
		"totalDocsExamined" : 1000000,
		"executionStages" : {
			"stage" : "COLLSCAN",
			"filter" : {
				"$expr" : {
					"$gt" : [
						{
							"$indexOfCP" : [
								"$ProductName",
								{
									"$const" : "Contoso Battery charger"
								}
							]
						},
						{
							"$const" : -1
						}
					]
				}
			},
			"nReturned" : 1563,
			"executionTimeMillisEstimate" : 140,
			"works" : 1000002,
			"advanced" : 1563,
			"needTime" : 998438,
			"needYield" : 0,
			"saveState" : 7812,
			"restoreState" : 7812,
			"isEOF" : 1,
			"direction" : "forward",
			"docsExamined" : 1000000
		}
	},
	"serverInfo" : {
		"host" : "b1c70a1e87a3",
		"port" : 27017,
		"version" : "4.2.0",
		"gitVersion" : "a4b751dcf51dd249c5865812b390cfd1c0129c30"
	},
	"ok" : 1
}

It does not appear that this query uses the existing index on the field. I guess it could, so not sure why this doesn't happen.

I think the reason why $indexOfCP doesn't use the index is that its first parameter is an arbitrary string expression. MongoDB does not recognize the fact that I'm "just" pulling in a value from an indexed field - it could, since the Expression syntax $FieldName is a specific thing according to docs, but clearly it does not treat this as a special case where an index could be used.

Additional idea: MongoDB's built-in text search. This is based on a special index with a special search construct.
> db.data.createIndex({ProductName: "text"})
{
	"createdCollectionAutomatically" : false,
	"numIndexesBefore" : 2,
	"numIndexesAfter" : 3,
	"ok" : 1
}
> db.data.explain("executionStats").aggregate([{"$match": { $text: { $search: "\"contoso battery charger\"", $caseSensitive: false }  } }])
{
	"stages" : [
		{
			"$cursor" : {
				"query" : {
					"$text" : {
						"$search" : "\"contoso battery charger\"",
						"$caseSensitive" : false
					}
				},
				"fields" : {
					"$textScore" : {
						"$meta" : "textScore"
					}
				},
				"queryPlanner" : {
					"plannerVersion" : 1,
					"namespace" : "demo.data",
					"indexFilterSet" : false,
					"parsedQuery" : {
						"$text" : {
							"$search" : "\"contoso battery charger\"",
							"$language" : "english",
							"$caseSensitive" : false,
							"$diacriticSensitive" : false
						}
					},
					"queryHash" : "83098EE1",
					"planCacheKey" : "7E2D582B",
					"winningPlan" : {
						"stage" : "PROJECTION_DEFAULT",
						"transformBy" : {
							"$textScore" : {
								"$meta" : "textScore"
							}
						},
						"inputStage" : {
							"stage" : "TEXT",
							"indexPrefix" : {

							},
							"indexName" : "ProductName_text",
							"parsedTextQuery" : {
								"terms" : [
									"batteri",
									"charger",
									"contoso"
								],
								"negatedTerms" : [ ],
								"phrases" : [
									"contoso battery charger"
								],
								"negatedPhrases" : [ ]
							},
							"textIndexVersion" : 3,
							"inputStage" : {
								"stage" : "TEXT_MATCH",
								"inputStage" : {
									"stage" : "TEXT_OR",
									"inputStages" : [
										{
											"stage" : "IXSCAN",
											"keyPattern" : {
												"_fts" : "text",
												"_ftsx" : 1
											},
											"indexName" : "ProductName_text",
											"isMultiKey" : true,
											"isUnique" : false,
											"isSparse" : false,
											"isPartial" : false,
											"indexVersion" : 2,
											"direction" : "backward",
											"indexBounds" : {

											}
										},
										{
											"stage" : "IXSCAN",
											"keyPattern" : {
												"_fts" : "text",
												"_ftsx" : 1
											},
											"indexName" : "ProductName_text",
											"isMultiKey" : true,
											"isUnique" : false,
											"isSparse" : false,
											"isPartial" : false,
											"indexVersion" : 2,
											"direction" : "backward",
											"indexBounds" : {

											}
										},
										{
											"stage" : "IXSCAN",
											"keyPattern" : {
												"_fts" : "text",
												"_ftsx" : 1
											},
											"indexName" : "ProductName_text",
											"isMultiKey" : true,
											"isUnique" : false,
											"isSparse" : false,
											"isPartial" : false,
											"indexVersion" : 2,
											"direction" : "backward",
											"indexBounds" : {

											}
										}
									]
								}
							}
						}
					},
					"rejectedPlans" : [ ]
				},
				"executionStats" : {
					"executionSuccess" : true,
					"nReturned" : 1563,
					"executionTimeMillis" : 1413,
					"totalKeysExamined" : 301234,
					"totalDocsExamined" : 279712,
					"executionStages" : {
						"stage" : "PROJECTION_DEFAULT",
						"nReturned" : 1563,
						"executionTimeMillisEstimate" : 121,
						"works" : 580951,
						"advanced" : 1563,
						"needTime" : 579387,
						"needYield" : 0,
						"saveState" : 4540,
						"restoreState" : 4540,
						"isEOF" : 1,
						"transformBy" : {
							"$textScore" : {
								"$meta" : "textScore"
							}
						},
						"inputStage" : {
							"stage" : "TEXT",
							"nReturned" : 1563,
							"executionTimeMillisEstimate" : 121,
							"works" : 580951,
							"advanced" : 1563,
							"needTime" : 579387,
							"needYield" : 0,
							"saveState" : 4540,
							"restoreState" : 4540,
							"isEOF" : 1,
							"indexPrefix" : {

							},
							"indexName" : "ProductName_text",
							"parsedTextQuery" : {
								"terms" : [
									"batteri",
									"charger",
									"contoso"
								],
								"negatedTerms" : [ ],
								"phrases" : [
									"contoso battery charger"
								],
								"negatedPhrases" : [ ]
							},
							"textIndexVersion" : 3,
							"inputStage" : {
								"stage" : "TEXT_MATCH",
								"nReturned" : 1563,
								"executionTimeMillisEstimate" : 120,
								"works" : 580951,
								"advanced" : 1563,
								"needTime" : 579387,
								"needYield" : 0,
								"saveState" : 4540,
								"restoreState" : 4540,
								"isEOF" : 1,
								"docsRejected" : 278149,
								"inputStage" : {
									"stage" : "TEXT_OR",
									"nReturned" : 279712,
									"executionTimeMillisEstimate" : 88,
									"works" : 580951,
									"advanced" : 279712,
									"needTime" : 301238,
									"needYield" : 0,
									"saveState" : 4540,
									"restoreState" : 4540,
									"isEOF" : 1,
									"docsExamined" : 279712,
									"inputStages" : [
										{
											"stage" : "IXSCAN",
											"nReturned" : 15519,
											"executionTimeMillisEstimate" : 1,
											"works" : 15520,
											"advanced" : 15519,
											"needTime" : 0,
											"needYield" : 0,
											"saveState" : 4540,
											"restoreState" : 4540,
											"isEOF" : 1,
											"keyPattern" : {
												"_fts" : "text",
												"_ftsx" : 1
											},
											"indexName" : "ProductName_text",
											"isMultiKey" : true,
											"isUnique" : false,
											"isSparse" : false,
											"isPartial" : false,
											"indexVersion" : 2,
											"direction" : "backward",
											"indexBounds" : {

											},
											"keysExamined" : 15519,
											"seeks" : 1,
											"dupsTested" : 15519,
											"dupsDropped" : 0
										},
										{
											"stage" : "IXSCAN",
											"nReturned" : 6003,
											"executionTimeMillisEstimate" : 0,
											"works" : 6004,
											"advanced" : 6003,
											"needTime" : 0,
											"needYield" : 0,
											"saveState" : 4540,
											"restoreState" : 4540,
											"isEOF" : 1,
											"keyPattern" : {
												"_fts" : "text",
												"_ftsx" : 1
											},
											"indexName" : "ProductName_text",
											"isMultiKey" : true,
											"isUnique" : false,
											"isSparse" : false,
											"isPartial" : false,
											"indexVersion" : 2,
											"direction" : "backward",
											"indexBounds" : {

											},
											"keysExamined" : 6003,
											"seeks" : 1,
											"dupsTested" : 6003,
											"dupsDropped" : 0
										},
										{
											"stage" : "IXSCAN",
											"nReturned" : 279712,
											"executionTimeMillisEstimate" : 22,
											"works" : 279713,
											"advanced" : 279712,
											"needTime" : 0,
											"needYield" : 0,
											"saveState" : 4540,
											"restoreState" : 4540,
											"isEOF" : 1,
											"keyPattern" : {
												"_fts" : "text",
												"_ftsx" : 1
											},
											"indexName" : "ProductName_text",
											"isMultiKey" : true,
											"isUnique" : false,
											"isSparse" : false,
											"isPartial" : false,
											"indexVersion" : 2,
											"direction" : "backward",
											"indexBounds" : {

											},
											"keysExamined" : 279712,
											"seeks" : 1,
											"dupsTested" : 279712,
											"dupsDropped" : 0
										}
									]
								}
							}
						}
					}
				}
			}
		}
	],
	"ok" : 1
}

Relevant part is this:
				"executionStats" : {
					"executionSuccess" : true,
					"nReturned" : 1563,
					"executionTimeMillis" : 1413,
					"totalKeysExamined" : 301234,
					"totalDocsExamined" : 279712,

Short: this method takes at least twice as long as the regex search. It enables complex search functionality, but it's certainly not very performant in the test case.

At this point I'm at a loss. It appears that MongoDB does not support any substring search mechanism that actually uses an index efficiently.  The suggested method $indexOfCP is much slower than regular expressions and so is the specialized text search index. I wonder if the balance would shift with larger document counts, perhaps I'll take the time to test this. Meanwhile I think a collection of 1 million documents is a good test case - just large enough for significant time to be spent, but small enough to still be representative of a "typical" application use case.
I have not found a search method that could be used as a better alternative for the current regex-based implementations of the contains, startsWith and endsWith operators.


Error-handling post-processing snippet

The user wants a callback when the API call errors. We had this functionality, but it was removed at the design phase.
A simple way of emulating the same feature is having an error-handling post-processing step. Normally, when the API response is non-2XX, the Airnode always calls back fail(). We can have an errorHandlingPostProcessing snippet that only gets invoked when the API returns a non-2XX code and post-processes the API response to call back fulfill() with that.

@bbenligiray who is the user and what is the urgency please? Should it be ASAP or can be moved after 1.0 milestone? Thanks

This was a user on discord. I would say low priority because the user themselves admitted / recognized "my use-case is very niche" and there are creative workarounds that @bbenligiray mentioned, like a proxy or FailedRequest watcher, that could suffice if the feature is badly needed.

I think we should have this or something analogous at some point, but it shouldn't be included in our foreseeable plans. So I created the issue to document it for the future.


how to load args_parser and model?

Hi MARS team,
Thanks for such great package. As a beginner, when I try to run your notebook:
from args_parser import get_parser
from model.mars import MARS
from model.experiment_dataset import ExperimentDataset
from data.benchmarks import BenchmarkData
import warnings

I got an issue that No module named 'args_parser', 'get_parser', 'model'.
I am wondering whether you can help me to load these packages correctly?
Thanks,
Yale

You can solve this by moving notebook file to the directory where args_parser file is located.

@mbrbic  I did it, but still doesn't work.


better documentation for upgrading ibc-go

currently we have migration docs which are good but we have received feedback from @ValarDragon @assafmo that having something like this diff PR  which shows ie: necessary imports would be really helpful for devs looking to upgrade to newer versions of ibc-go
Summary of Bug

Version

Steps to Reproduce

For Admin Use

[ ] Not duplicate issue
[ ] Appropriate labels applied
[ ] Appropriate contributors tagged/assigned

I think that a demo repo with diffs to upgrade from v1 to v2, v2 to v3, etc. will go a long way here. It'll also show how to properly write upgrade handlers and how to evolve them over time.

Thanks for the feedback y'all. Upgrading versions of ibc-go should be as painless as possible. I originally added the migration docs because of how difficult I felt upgrading SDK versions was and I did not want that to be true for ibc-go. I'd really like to see the migration docs we have improved as it is really just what I scraped together for the v1 release of ibc-go and it hasn't changed much since then.
Could y'all help me understand in more detail the exact issues that are being faced. Are the migration docs difficult to navigate? Do they have insufficient information? Are they too vague? Or do they provide too much information? Are bugs getting through after upgrading versions of ibc-go? Are there sections that require upgrades that we do not provide any information for? Is it difficult to review ibc-go bump prs (due to too many diffs)? Are there new additions to the codebase that weren't indicated in the docs?
I'm guessing if a diff pr is the suggested solution here, then one difficulty is that the migration docs aren't explicit enough. That the desirable behaviour is for them to explain how the before looked/behaved and how the after should look/behave. I'm somewhat uncertain a diff pr will always be the optimal solution for both us (maintainers) and y'all (chain developers) as it adds a bit of maintenance burden to us and the diffs could be quite massive. It doesn't seem useful to have 30 files of import/name changes in my opinion as it just adds noise. I think a single diff showing the change would likely be easier to comprehend/follow
I don't want to jump too far ahead to solutions until I understand the problem better, but before I forget, this is one idea I thought of. We could maintain an upgrading check list which has a box for every change a chain developer needs to do. It would start from our latest supported version (v2) and have check boxes all the way up to the latest released version (v4 atm). Each check box would have a short description of the change to be made and link to more detailed information down below in the document. There should also be a check box for each potential change a chain might need to make. Then chains could copy/paste this doc and check off each box as pr's come into a feature branch which would bump the ibc-go version. It might look something like this (information used may be incorrect, just using as an example):

v5

[ ] bump import path to v5
[ ] asdf
[ ] asdf

v4

[ ] bump import path to v4
[ ] aadf
[ ] asdf

v3
This version introduces ICS 27 - Interchain accounts

[ ] bump import path to v3
[ ] Add ics27 to SDK AppModule list (if integrating the ics27 host module)
[ ] adsf

v2

[ ] bump import path to v2
[ ] asdf

Adding ICS 27 host to SDK AppModule
The ICS 27 IBC application module must be added to the SDK AppModule list within app.go.
[]module.AppModule{
        ...
        ibc.NewAppModule(app.IBCKeeper),
+	ica.NewAppModule(nil, app.ICAHostKeeper),
	params.NewAppModule(*app.ParamsKeeper),
        ...
}

I'm guessing if a diff pr is the suggested solution here, then one difficulty is that the migration docs aren't explicit enough. That the desirable behaviour is for them to explain how the before looked/behaved and how the after should look/behave. I'm somewhat uncertain a diff pr will always be the optimal solution for both us (maintainers) and y'all (chain developers) as it adds a bit of maintenance burden to us and the diffs could be quite massive. It doesn't seem useful to have 30 files of import/name changes in my opinion as it just adds noise. I think a single diff showing the change would likely be easier to comprehend/follow

Err, I think theres a discrepancy in viewed diff, not sure how we hit 30+ files changed. Presumably IBC-go maintains a simapp
The suggested diff is a PR like (https://github.com/osmosis-labs/osmosis/pull/1564/files):

Adding an upgrade handler with relevant logic
Adding relevant new keepers / module basic / ibc router update
param setting, etc.
I do agree that a lot of this becomes no longer necessary, with just needing to show:
New keeper updates
Param updates
given more upgrading dev UX wins. (e.g. v3.2.0 using a migration handler, made it super easy)

e.g. the following lines are present in your app.go and not really something you'd expect. (Though ideally this can become complexity that isn't by default exposed to app-devs, and only gets exposed if someone wants to override stuff)
	icaHostIBCModule := icahost.NewIBCModule(*appKeepers.ICAHostKeeper)
	// Create static IBC router, add transfer route, then set and seal it
	ibcRouter := porttypes.NewRouter()
	ibcRouter.AddRoute(icahosttypes.SubModuleName, icaHostIBCModule).
		AddRoute(ibctransfertypes.ModuleName, transferIBCModule)

Bumping the import path is a non-issue.

I think the main issue with a diff pr for our simapp is that it runs in the same go.mod so changes are made across many pr's, though we do have plans to give the testing package its own go.mod, which means then we could always bump the go.mod in a separate pr and link that pr

Err, I think theres a discrepancy in viewed diff, not sure how we hit 30+ files changed. Presumably IBC-go maintains a simapp

Yea sorry, I was thinking more of app/light client dependencies. I realize chains will only have a few files

Adding an upgrade handler with relevant logic
Adding relevant new keepers / module basic / ibc router update
param setting, etc.
I do agree that a lot of this becomes no longer necessary, with just needing to show:
New keeper updates
Param updates
given more upgrading dev UX wins. (e.g. v3.2.0 using a migration handler, made it super easy)

Makes sense. I guess we can link the simapp.go pr changes (several prs) for now until we add the testing package as its own go module.

Also I'd suggest a test we can import from IBC, and use in some module, to ensure that its configured correctly. (E.g. ibc.TestHostICAIntegration(ctx, baseapp, keepers), and if that passes we know were good / nothing to worry about for the feature)

Already on the todo list #2027

I discussed with @colin-axner and we think our upgrading docs have improved in the last few releases, so we will close the issue, but feel free to open a new one if there's anything specific that we could do improve them further.


Testing

I know It might be a bit too much but could you also add some testing framework to any of your scaffoldings?

Something like https://github.com/nwjs/nw.js/wiki/Testing#how-do-we-test-with-mocha ?

Yes that would be a great... adding mocha/intern with karma as test runner or maybe something else you might know of... I am still new to the node/node-webkit world so isn't sure of the frameworks are battle tested or not


graphene non-graph segmentation layers should be handled like precomputed

the proxy auth implementation works great for images, but not for segmentation volumes.  When the info file indicates that the segmentation layer you are accessing is not a PCG backend, it should work as a normal precomputed info file.
In particular, flat segmentations hosted on cloudian can't be made accessible via the proxy because of this.

I might not understand the issue you encounter: If the segmentation layer is not of type segmentation_with_graph, i.e. not related to the graph server, you should to load it using precomputed://link/to/layer and avoid all the ChunkedGraph/graphene related modifications and hacks.

But precomputed knows nothing about auth, so if we we want to use the proxy server to serve precomputed segmentations (like the nuclei) off of cloudian we can’t use the precomputed protocol in neuroglancer because it doesn’t know about auth.
Maybe the precomputed protocol should know about auth.. that would be acceptable I suppose
Sent from my mobile device.

On Jun 2, 2020, at 7:52 PM, Nico Kemnitz<EMAIL_ADDRESS>wrote:

I might not understand the issue you encounter: If the segmentation layer is not of type segmentation_with_graph, i.e. not related to the graph server, you should to load it using precomputed://link/to/layer and avoid all the ChunkedGraph/graphene related modifications and hacks.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.


leaflet event indicating markers are loaded

Let's say we have hundreds of markers rendering on the screen. Is there any ui-leaflet event fired which indicates all markers has been rendered/loaded or we can say, the map is ready?

ui-leaflet doesn't really make it's own events, afaik. It just relays some of the ones that are sent by leaflet itself. With that being the case, the only event ui-leaflet could tie into is the individual marker-add event. You could keep tabs on the markers that should be loaded with an array and remove them from the array as they fire the marker-add event. When your array is empty, you'll know they're all added.


INF-173: action vue build to github actions on pull request

Adds checks to bi-web on pull request. A bunch of our tests are broken, so the tests aren't added to this yet. But, those should be easy to add once they are fixed.
Test runs a vue-cli-service build and a npm audit check. The vue-cli-service runs a linter and type check on the project and fails if there are any errors there. You will see some fixes to the types in existing files because I couldn't get the build to run without fixing those. But, these changes are a good example of the things that will be caught in the future!
Also includes changes to the package-lock.json file to remove the bulk of the vulnerabilities.
I have add a merge condition that the tests must pass for bi-web in order to be merged.
I have add a commit to the PR for both failure scenarios:

Vulnerable libraries
Vue build error

You can see the results of both tests in the actions:
https://github.com/Breeding-Insight/bi-web/actions
Or, if you look down below in the list of commits it will shows X's and checkmarks for successes and failures on each commit to the existing PR.
Design
I made a custom npm audit check because npm audit returns an exit code of 1 if there are any vulnerabilities. There are some promising looking flags like --only=prod that don't work for npm audit, only npm audit fix. There is also --audit-level=<serverity> that works, but treats dev and prod errors the same. This seemed like it would cause unneeded burden on our dev libraries since the security on those shouldn't need to meet the same standard as our prod libraries.
The custom error checking accepts parameters for the maximum acceptable severity error level for dev and for prod separately. It's nothing too complicated, it just parses the return and checks the errors. I currently have dev set to high because our dev libraries shouldn't be in our production build, and prod set to moderate, because the meaning of moderate severity is: Address as time allows. I'm open to discussion on this.

 INF-173 - Configure GitHub Actions to automatically build bi-web

Not a reviewer on this but was looking at the github actions to see something and noticed there are a bunch of changed code files in here. Was that intended/needed for the build and will stuff have to be changed since some of that code is old?

@nickpalladino Only commenting because Chris is off today, and we just happened to talk about this yesterday.  The code changes were needed to get the linter checks to pass.

The build.js script looks good for setting risk thresholds for prod and dev. We are scheduled next Friday to discuss how to handle these audit results, and right now I am not convinced there is a need to filter audit results based on level for either dev or production.
Any audit flags for dev can be safely ignored because the vulnerable packages won't be bundled into the production build, and you don't want any known vulnerabilities in production regardless of severity level, so there is no need to filter by severity level for production packages.
The response from the npm team on a github issue requesting the --audit-level option was that the audit levels are meant only to suggest priority:
from npm dev in repo issue: npm/npm#20596
"Issues of any severity should really be fixed, the severity of a vulnerability assessment should only serve to prioritize the fixes."
If we want a passing test to be no security issues in prod, ignoring dev, then the following npm command can help:
bi-web_$npm audit fix --dry-run --json --only=prod
{ "added": [], "removed": [], "updated": [], "moved": [], "failed": [], "warnings": [ "The package bulma is included as both a dev and production dependency." ], "funding": "53 packages are looking for funding", "elapsed": 5306 }
A passing test results from parsing the json for empty arrays in the first six keys. This also checks for warnings that we may want to address. In this case bulma should be removed as a prod dependency.
In other cases we may want to ignore the warning, so having an option to ignore warnings seems good to have.

Thanks for checking it out! I would propose that we create a card to explore the option you list above. I added a none option for the vulnerability check in the case that we decide that there should be no vulnerabilities for prod. The severity check as it is works for what we need it for, and I don't want to get stuck on this card too much longer. Are you good with that?
It is outside the scope of this card, but I think bulma should be removed as a dev dependency, not a prod dependency. I believe we are going to need that for production.

Sounds good.


Update demo-django to Django 1.11

I've updated the demo-django example to use Django 1.11. This mostly involved rewriting some settings and updating deprecated functions.

Coverage remained the same at 95.85% when pulling e88ad9e33d87cfd23923143886e26b056ad1f368 on palbee:master into 192a3fd2318317bed71f46c8f9756eb51e1d19b9 on onelogin:master.

Hi, thanks for contributing.
I will need to review if makes sense to just require now django1.11 instead the old django1.6.5
but since django 1.11 is kinda new, let me some days to test and merge.

I appreciate the need for review. Even if you choose to not go to Django 1.11 (the current release), it would be helpful to have the example use a support version of Django.

Yes I'd appreciate this, since 1.6.5 is out of support.


Static error testing doesn't handle error in imported libraries

The test language/nonfunction_type_aliases/mixed/aliased_cyclic_superclass_error_test assumes errors reported in an imported library. The static error tests framework doesn't seem to handle that.

cc @eernstg @munificent

True, this has been reported as #44990 (so I think this one is a duplicate). I also changed said test in https://dart-review.googlesource.com/c/sdk/+/191365, such that we can see the actual behavior of tools as intended.


Data source

Is there anyway to make this work with a free data source (either yfinance but not ideal because of obsolesce or alphavantage)? I started making a parser that dumps into CSV format to sideload free data. If I understand correctly, I only need the data: Symbol,Name,Last,Change,%Chg,High,Low,Volume,Time ?

Thanks for these instructions. After finding out the Quandl API key isn't working, I'm looking to utilize my own data source (likely just csv files). Just one question:  I noticed that data_set.py appears to also handle the CSV enum. Is it possible that I might be able to reuse these CSV-related functions? In lieu of Quandl not working, might you have available the CSV data for the exact time-series used in the paper? I'd like to reproduce the results.

Line 336 in model.py:
if 'Open' in self.data_source.columns and 'Close' in self.data_source.columns:
I get an error that the CsvTimeSeriesDataset doesn't have columns as an attribute. I assume these are meant to be loaded when that object is instantiated from config? So config should have a field 'columns' with a list of the columns? Or am I missing something else there?
Thanks!

i get a quandl error. would have loved to check it out, but i am struggeling.


Website list filtering example doesn't work in IE Emulation Mode?

Clearly, this is not at all important, but wondering if anyone knows why https://jsfiddle.net/julmot/bs69vcqL/ returns 0 results as of second letter typed (one letter works - 'o' works but not 'oh') ONLY in IE using Emulation (any Document mode choice, even 'Edge').

Hi @terraelise,
Thanks for submitting.
I can confirm this behavior even when using IE11 without any emulation. I've replaced
$context.unmark().show();
```

with

````javascript
$context.show().unmark();
```

and it seems to work. I can't exactly tell you the cause of this problem. IE may disallow normalizing a text node when its parent is hidden. I'll have to investigate this a bit tomorrow.

I've also updated the fiddle. In the meantime please try again.

Still not working for me, maybe depends on IE version. insert flaming IE remark here But thanks for looking into it!!

Seems to be an IE bug not related to mark.js. Anyhow, I've created a question on SO. I'll let you know once there is a solution.

I've just added a bounty to that question – hopefully having an answer soon.

As it turned out that this is a IE11 bug, I've created a bug report.

The bug was marked as "fixed" without any additional comment. Hopefully it'll be deployed in the next days.

In the meantime you could use the option acrossElements to workaround this behavior of IE. This will search for matches even when text nodes are splitted – which is the bug of IE.
Just replace:
$context.mark(term, {
	done: function() {
	  $context.not(":has(mark)").hide();
	}
});

with:
$context.mark(term, {
	acrossElements: true,
	done: function() {
	  $context.not(":has(mark)").hide();
	}
});

Note the acrossElements option.


Stale read meet Data not ready for a long time

Bug Report

What version of TiKV are you using?
6.1.0
What did happened?
There only 17 regions, and 9 TiKVs, them distributed in three regions: Tokyo, USA, and Singapore.
Phenomenon
Login Tokyo node <IP_ADDRESS> and run MySQL client, querying the data using stale read with closest-replicas. on the Tokyo node, read xjpnew1 data of the table will trigger dataNotReady:

Placement Settings:

Regions of Table

Replicas' Distrubution
mysql> select peer_id, a.store_id, b.ADDRESS, b.label from information_schema.tikv_region_peers a left join information_schema.tikv_store_status b on a.store_id=b.store_id  where region_id=1020;
+---------+----------+---------------------+----------------------------------------------------------------------+
| peer_id | store_id | ADDRESS             | label                                                                |
+---------+----------+---------------------+----------------------------------------------------------------------+
|    1022 |       19 | <IP_ADDRESS>:20162   | [{"key": "host", "value": "tikv2"}, {"key": "zone", "value": "xjp"}] |
|    1025 |       21 | <IP_ADDRESS>:20163  | [{"key": "host", "value": "tikv3"}, {"key": "zone", "value": "xjp"}] |
|    1028 |       20 | <IP_ADDRESS>:20161  | [{"key": "host", "value": "tikv1"}, {"key": "zone", "value": "xjp"}] |
|    1048 |        3 | <IP_ADDRESS>:20162 | [{"key": "host", "value": "tikv2"}, {"key": "zone", "value": "mg"}]  |
|    2727 |        2 | <IP_ADDRESS>:20163 | [{"key": "host", "value": "tikv3"}, {"key": "zone", "value": "mg"}]  |
|    2938 |     2808 | <IP_ADDRESS>:20163   | [{"key": "host", "value": "tikv3"}, {"key": "zone", "value": "dj"}]  |
|    2939 |        6 | <IP_ADDRESS>:20161  | [{"key": "host", "value": "tikv1"}, {"key": "zone", "value": "dj"}]  |
+---------+----------+---------------------+----------------------------------------------------------------------+
7 rows in set (0.50 sec)

Other Information
TiKV Debug Logs
$ tail -f tikv.log|grep reject
[2022/07/19 04:16:30.346 +00:00] [DEBUG] [read.rs:332] ["reject stale read by safe ts"] ["read ts"=434688276561920000] ["safe ts"=0] [tag="[region 1020] 2938"]
[2022/07/19 04:16:35.714 +00:00] [DEBUG] [read.rs:332] ["reject stale read by safe ts"] ["read ts"=434688277872640000] ["safe ts"=0] [tag="[region 1020] 2938"]

$ tail -f tikv.log |grep reject
[2022/07/19 04:33:00.354 +00:00] [DEBUG] [read.rs:332] ["reject stale read by safe ts"] ["read ts"=434688536084480000] ["safe ts"=0] [tag="[region 1020] 2938"]
[2022/07/19 04:33:03.570 +00:00] [DEBUG] [read.rs:332] ["reject stale read by safe ts"] ["read ts"=434688536870912000] ["safe ts"=0] [tag="[region 1020] 2938"]

we can see the safe ts is always 0.

/assign @nolouch


Uncaught Error: Iframe has not been created yet.

On a installation of CRA 2.0, with this installed, hot loading fails.
It loads the file correctly, but when I save the file without any changes, it crashes and prompts me the following error:
See video for better understanding - https://www.loom.com/share/673390a71fdf411b8885dda1a88a9198
I've been trying to install Hot Reloader to fix the issue, but the problem consists.
Do you guys have any idea what's giving me the problem?
VM6563 0.chunk.js:29568 Uncaught Error: Iframe has not been created yet.
    at me (VM6563 0.chunk.js:29568)
    at Object.window.__REACT_ERROR_OVERLAY_GLOBAL_HOOK__.iframeReady (VM6563 0.chunk.js:29578)
    at Module.<anonymous> (<anonymous>:1:280713)
    at n (<anonymous>:1:110)
    at <anonymous>:1:904
    at <anonymous>:1:915
    at HTMLIFrameElement.e.onload (VM6563 0.chunk.js:29560)

package.json
{
  "name": "project_title",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "start": "react-app-rewired start",
    "build": "react-app-rewired build",
    "test": "react-app-rewired test --modulePaths=src",
  },
  "dependencies": {
    "react": "^16.8.6",
    "react-dom": "^16.8.6",
    "react-scripts": "^2.1.8"
  },
  "devDependencies": {
    "@adaptagency/fe_tools": "^1.6.4",
    "@svgr/webpack": "^4.2.0",
    "autoprefixer": "^9.5.1",
    "colors": "^1.3.3",
    "css-loader": "^2.1.1",
    "eslint": "^5.16.0",
    "eslint-config-airbnb": "^17.1.0",
    "eslint-plugin-import": "^2.14.0",
    "eslint-plugin-jsx-a11y": "^6.1.1",
    "eslint-plugin-react": "^7.11.0",
    "extract-text-webpack-plugin": "^3.0.2",
    "html-webpack-plugin": "^3.2.0",
    "mini-css-extract-plugin": "^0.6.0",
    "node-sass": "^4.9.4",
    "optimize-css-assets-webpack-plugin": "^5.0.1",
    "postcss-calc": "^7.0.1",
    "postcss-flexbugs-fixes": "^4.1.0",
    "postcss-inline-svg": "^3.1.1",
    "postcss-loader": "^3.0.0",
    "prompt": "1.0.0",
    "prop-types": "^15.6.2",
    "react-app-rewire-eslint": "^0.2.3",
    "react-app-rewired": "^2.1.1",
    "release-it": "^10.4.1",
    "sass-loader": "^7.1.0",
    "sass-resources-loader": "^2.0.0",
    "style-loader": "^0.23.1",
    "uglifyjs-webpack-plugin": "^2.1.2",
    "webpack": "^4.29.6",
    "webpack-bundle-analyzer": "^3.3.1",
    "webpack-cli": "^3.3.0",
    "webpack-delete-after-emit": "^1.0.5"
  },
  "eslintConfig": {
    "extends": "react-app"
  },
  "browserslist": [
    ">0.2%",
    "not dead",
    "not ie <= 11",
    "not op_mini all"
  ]
}

Closing this - as I can't support troubleshooting config issues.  If you believe this is a bug please try to create a demo repo that narrows things down.

I have the same problem. How did you solve it . thx

I ended up rewriting the whole config, and the error dissapeared.
You can look at my repo for inspiration:
https://github.com/adaptdk/react_boilerplate

Hi @mthines,
I am still facing the issue. Could you give me the root cause and how to resolve it?
Thanks!

@tommyspot
I ended up switching to nextjs, so couldn't tell, sorry ☺


Initial GET request date time formatting

Hello @mikhail-vl,
I am using an initial GET request to populate the data, all the string fields get populated correctly but date-time field is not properly populated kindly see the below images of actual and expected results. Also unable to open the calendar option to select date-time after the initial GET request is finished.
Actual

Error while selecting the calendar option

Expected

Another concern is, if date-time field is populated in the correct format, is it required to click on Apply button in the calendar (if the calendar issue is fixed)?
Can you please help me with the issues?

@Castlesai, I am sorry for the delayed response.
I reproduced the issue and will work on a fix.
Apply should not be required if the DateTime is parsed correctly.

Thanks, @mikhail-vl  for responding

Hello @mikhail-vl, can you please give me suggestions to fix this issue?

@Castlesai Issue was resolved and will be a part of today's release.
Please let me know if there is anything else.


[SPARK-7612][MLLIB] update NB training to use mllib's BLAS

This is similar to the changes to k-means, which gives us better control on the performance. @dbtsai

Merged build triggered.

Merged build started.

Test build #32639 has started for   PR 6128 at commit a90e3ec.

LGTM. We may refactor it in next round using the aggregator pattern in LIR/LOR to further avoid creating tuple.

Test build #32639 has finished for   PR 6128 at commit a90e3ec.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32639/
Test PASSed.

Merged build finished. Test PASSed.

Merged build triggered.

Merged build started.

Test build #32652 has started for   PR 6128 at commit b5c24c5.

Test build #32652 has finished for   PR 6128 at commit b5c24c5.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class TreeClassifierParams(object):
class GBTParams(object):
class DecisionTreeClassifier(JavaEstimator, HasFeaturesCol, HasLabelCol, HasPredictionCol,
class DecisionTreeClassificationModel(JavaModel):
class RandomForestClassifier(JavaEstimator, HasFeaturesCol, HasLabelCol, HasPredictionCol, HasSeed,
class RandomForestClassificationModel(JavaModel):
class GBTClassifier(JavaEstimator, HasFeaturesCol, HasLabelCol, HasPredictionCol, HasMaxIter,
class GBTClassificationModel(JavaModel):
        ("probabilityCol", "Column name for predicted class conditional probabilities. " +
class HasProbabilityCol(Params):
    probabilityCol = Param(Params._dummy(), "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities.")
        self.probabilityCol = Param(self, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities.")

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32652/
Test PASSed.

Merged into master and branch-1.4.


Index and LA

I have a replicated cluster (3 nodes: 54 cpu, 256gb each). All nodes are replica. Inserts mostly goes on one node. Selects on another two.
Every night in ~00-00 something generate huge load (LA) on one of the node which are for selecting.
I am thinking this is indexing from carbon-clickhouse, but not sure.
Can you provide me some info about how cache-ttl and index working? There is no info about this in readme. How this works, for what it is?
Also can you provide me some tips to pump my config:
[data]
path = "/var/spool/carbon-tagged/"
chunk-interval = "10s"
chunk-auto-interval = ""
compression = "lz4"
compression-level = 0

[upload.graphite]
type = "points"
table = "data.data"
threads = 5
url = "http://localhost:8124/"
timeout = "1m0s"
zero-timestamp = true
compress-data = true

[upload.tags]
type = "tagged"
table = "data.tags"
threads = 6
url = "http://localhost:8124/"
timeout = "2m0s"
cache-ttl = "48h0m0s"
compress-data = true
disable-daily-index = true

[upload.graphite_index]
type = "index"
table = "data.graph_index"
threads = 3
url = "http://localhost:8124/"
timeout = "1m0s"
cache-ttl = "48h0m0s"
compress-data = true
disable-daily-index = true

Regarding to this https://github.com/go-graphite/carbon-clickhouse/pull/91

I suggest enabling https://clickhouse.com/docs/en/operations/system-tables/query_log
cache-ttl AFAIR is an internal thing to not update tags and index tables.
I rather suspect that some of your users requests a huge number of metrics from graphite than it's somehow related to carbon.

I've been tried to stop graphite-clickhouse (and keep running carbon-clickhouse) in 23-50 and still getting the same problem.
I have no cron jobs on clickhouse. Graphite rollup have bigger period than data exist (rollup is 3 days, data TTL is 2 days) so there will be no retentions active.
So, next experiment will be with stopping carbon-clickhouse.
For what actually index table exist in this soft? Can I disable it if not using grafana in my scheme?

Now in new day carbon-clickhouse generate insert. Not really needed. Also expire cache TTL - also produce new insert.
So, no-daily index only save  disk space, not insert rate. Also produce high cpu usage, before parts are remerged (it's high cost on huge index/tags tables).

Also Clickhouse try to use all avaliable cores for  background processes (like merges). So restrict them for smaller value (than 54)  may be a solution.

I stopped 5 mins before 00-00 carbon-clickhouse and there was zero load spikes. So, this is not graphite-clickhouse for sure.
Will try to check out click background processes and carbon-clickhouse daily index enabled. thx @msaf1980

At 00:00 it generates heavy inserts - like larger x4-5 than normal size
─query_duration_ms─┬─written_bytes─┬─http_user_agent────┬────query_start_time─┬─query─────────────────────────────────────────────────────────────────────────────┐
│             58927 │<PHONE_NUMBER> │ Go-http-client/1.1 │ 2024-04-30 00:04:59 │ INSERT INTO graphite.graphite_index (Date, Level, Path, Version) FORMAT RowBinary
│
│             59455 │<PHONE_NUMBER> │ Go-http-client/1.1 │ 2024-04-30 00:05:01 │ INSERT INTO graphite.graphite_index (Date, Level, Path, Version) FORMAT RowBinary
│
│             59805 │<PHONE_NUMBER> │ Go-http-client/1.1 │ 2024-04-30 00:05:03 │ INSERT INTO graphite.graphite_index (Date, Level, Path, Version) FORMAT RowBinary
│
│             63204 │<PHONE_NUMBER> │ Go-http-client/1.1 │ 2024-04-30 00:05:00 │ INSERT INTO graphite.graphite_index (Date, Level, Path, Version) FORMAT RowBinary
│
│             60733 │<PHONE_NUMBER> │ Go-http-client/1.1 │ 2024-04-30 00:05:06 │ INSERT INTO graphite.graphite_index (Date, Level, Path, Version) FORMAT RowBinary
│
│             59462 │<PHONE_NUMBER> │ Go-http-client/1.1 │ 2024-04-30 00:05:59 │ INSERT INTO graphite.graphite_index (Date, Level, Path, Version) FORMAT RowBinary
│
│             58941 │<PHONE_NUMBER> │ Go-http-client/1.1 │ 2024-04-30 00:06:01 │ INSERT INTO graphite.graphite_index (Date, Level, Path, Version) FORMAT RowBinary
│
│             60223 │<PHONE_NUMBER> │ Go-http-client/1.1 │ 2024-04-30 00:06:02 │ INSERT INTO graphite.graphite_index (Date, Level, Path, Version) FORMAT RowBinary
│
│             60893 │<PHONE_NUMBER> │ Go-http-client/1.1 │ 2024-04-30 00:06:03 │ INSERT INTO graphite.graphite_index (Date, Level, Path, Version) FORMAT RowBinary

I found out why LA spikes, caused by rollup aggregation rules for too many metrics.


Minimize/Maximize/Close Buttons Glitching

I am on Windows, and I am using the repackaged version through the exe file. For some reason, the buttons in the top right corner are glitching for me. They are usually all lined up next to each other. They're not supposed to be underneath one another like shown in the photo...

Also, like 90% of the time, if I hover my mouse over the Close button, they disappear.
Not sure what to do. I'm not familiar with coding, so I have no idea where to go to fix it.

On the latest dev version of windows 11 (22449) they be like this.

#291

@Rereyuop see notion-enhancer/notion-enhancer#291, these are different bugs.

thanks, this app is brilliant!

@starryyskies see https://github.com/notion-enhancer/notion-enhancer/issues/572


Cross compile scala 2.10/2.11 and provide separate artifacts

Similar to
https://github.com/bigdatagenomics/adam/blob/master/scripts/move_to_scala_2.10.sh
https://github.com/bigdatagenomics/adam/blob/master/scripts/move_to_scala_2.11.sh
https://github.com/bigdatagenomics/adam/blob/master/scripts/release/release.sh

@heuermh If you want to tackle this, that'd be great! However, I would urge you to wait until Monday ;) as I am hoping to have two fixes for #174 and #175 pushed today, and I think there might be conflicts between the two.

Just wanted to note this as something that needed to be done, after #174 certainly.

Resolved by https://github.com/bigdatagenomics/avocado/commit/35c17805518bd6a80327ff8e93f8447be1acf3ff.


Update Laravel 10 dependencies

Update surgiie/console v4
Update illuminate/http v10

@surgiie here the pull request about make compatible with laravel 10

@jals-es Thanks. New release has been published https://github.com/surgiie/blade-cli/releases/tag/v4.0.0
Breaking change of dropping php 8.0 support is included, please upgrade to php 8.1 or above if not already.

Thanks @surgiie. Yes, I have php 8.1


Add docs generation

This PR borrows docs from #14 and I did my first implementation here: https://github.com/ulrikstrid/bankid-reason.

The only thing missing is generating docs with Reason syntax. By default odoc generates OCaml. I submitted a PR to fix this.
https://github.com/esy-ocaml/hello-reason/pull/22