Weird game resolutions not cropped by GSM

GSM still cant upscale these games which results in a supper letter boxed picture

The Silent Hill from 2 to 3 Use an abnormal vertical resolution... I will post here when I have them!

I get this in SSX Tricky

This is a GSM problem, and has nothing to do with OPL.
288px * 2 = 576px, which is greater than the height of the EDTV 480P resolution. Have you tried the EDTV 576P mode yet?

I Understand you...
Yes I just tried 576p and it is... fine o_o
But I use PAL/NTSC to force higher resoltion as my TV dont support it through component, also the game has interlaced FMVs... With IGR debug help its easier to show to you...

Silent Hill 2/3 are actually 512x448 nothing weird, but there is VRAM corruption??? at botton of screen...

Now those two games doesnt seem to be affected by V and H screen settings of GSM...

I don't understand. You cannot increase the resolution by changing the video mode, particularly between NTSC and PAL since all games will use either by default. Internally, we use the hardware magnification functions to magnify a low-resolution frame buffer to fit the display resolution. The frame buffer's resolution is independent of the display resolution, although there is usually some relationship (usually, it is a factor of the display resolution), but there is nothing that will stop developers from wasting memory.
OPL has no V and H controls, from what I remember. If you were using the standalone GSM, its 576P mode does not work properly, as it sets the wrong display resolution, causing graphical errors in some titles.
Interlacing is a video-rendering technique for doubling the vertical height. It is usually not related to the video files in the game itself, but how the game renders. How is that relevant to what you are trying here? NTSC and PAL are interlaced video modes.

@sp193 I am playing 480p games on a 480p LCD over Component. Is this just not supported by GSM for some games?

If your game uses 480P, your TV accepts 480P and you choose to use 480P, then it will work and you do not even need GSM.
In fact, if you can go without GSM, that would be best because it would result in less modifications to the content of EE memory.

The game does not have native 480p, which is why I use GSM to get it. But is appears letterboxed

Ah, I see. It is technically not possible to have a perfect transition to 480P. The resolutions for NTSC and EDTV 480P are quite different, which is why letterboxing occurs.
NTSC: 640 x 448
EDTV 480P: 720 x 480
The acceptable frame buffer sizes are quite a few in number for NTSC and PAL, due to the actual width of the display area in VCKs (256, 512 and 640 are all factors of 2560). The vertical height can be higher than 448px, but not all TVs can display all 480 lines and the undisplayed lines are at the expense of video memory.
It gets worse when the original width is 512px, as the unused screen width becomes 28% (720-512)/720=28%.

I don't understand. You cannot increase the resolution by changing the video mode, particularly between NTSC and PAL since all games will use either by default. Internally, we use the hardware magnification functions to magnify a low-resolution frame buffer to fit the display resolution. The frame buffer's resolution is independent of the display resolution, although there is usually some relationship (usually, it is a factor of the display resolution), but there is nothing that will stop developers from wasting memory.
OPL has no V and H controls, from what I remember. If you were using the standalone GSM, its 576P mode does not work properly, as it sets the wrong display resolution, causing graphical errors in some titles.
Interlacing is a video-rendering technique for doubling the vertical height. It is usually not related to the video files in the game itself, but how the game renders. How is that relevant to what you are trying here? NTSC and PAL are interlaced video modes.

I force NTSC/PAL because if dont my TV will display Unsupported format while playing this game, ICO, POPS emulator (In POPS I use the $HDTVFIX command). So yes, by Forcing PAL/NTSC with GSM I can get the picture to show in my TV, so I assume GSM is forcing a higher resolution than 640x288.
The V and H commands I said is the Y and X pos adjustment (Below VMODE and ABOVE Emulate FIELD flipping in GSM setting of OPL)
If I use 576p/480p the FMVs will stutter/audio out of sync with emulate FIELD flipping, without emulate flipping they not even play and the game hangs. Thats why i preffer NTSC/PAL over 576/480p with games like that.
---//----
Sometimes GSM Is better than the game built-in progressive scan... some games downgrade to 16bit color or use some ugly video filter and make it blurrier like SOCOM 2/3 and a Star War game that i dont remember the name!

Here's what SSX Tricky looks like when running at EDTV 480p.

Here's what SSX Tricky looks like when running at EDTV 480p.
I'd expect to see bars above and below, but instead here they are left and right?

That's because the console is sending a 512 or 640 wide image in a 720 wide frame. It's completely normal for a digital television to react like that.

Here is a better picture of I wanted to show about Silent Hill 2.

Damn in silent hill 2 you cant adjust X and Y screen position

That's because the console is sending a 512 or 640 wide image in a 720 wide frame. It's completely normal for a digital television to react like that.

I'm not sure I follow?
I am using a 640x480p TV. The game displays fine in 480i, it's just when i force 480p using GSM that it is displayed squished in the middle

Your TV is capable of displaying video in various modes. Each dot does not have the same size across all display modes.
Otherwise, how does a TV display video in NTSC, PAL, 480P, 720P etc?
Our 480P mode is wider than NTSC, as it does not share the same aspect ratio. The smaller the video width (e.g. 512px), the greater the number of columns that will not be used.
The NTSC resolution is something like 2560x448, which would allow for a lot of factors (256, 512, 640 are possible).
I read that it's because NTSC is analog and a CRT TV renders in lines, so any arbitrary number of pixels can be encoded within a line.

Sure, I understand how a panel of one native resolution can display numerous other resolutions.
I've only encountered SSX Tricky behaving oddly in the way this issue is describing.
Most game I force into 480p display just fine across the whole 640x480.
Why is there a difference?

Like multiple people tried to explain to you already...
Because the horizontal resolution is different and because the 'magnification' in integers (full numbers) only allow for doubling/tripling/quadrupling and so on...
If you would use a magnification which is one higher than the current value, it would stretch the left and right border to the extent, that they are not within the shown frame...
You have to differentiate between the 'rendered resolution' and the 'output-resolution' and regardless if your TVs Panel has a resolution of 640480 or 2134234, that has NOTHING to do with either of both resolutions (rendered res. and output res.)!
Tl;dr
@zappepappe and @SP193 explained it as easy as it gets...
'480p' is 720480...
'NTSC' is technically the same resolution, but since a console (and so on), usually renders in SQUARE PIXELS, the expected resolution is like 640480 in that case...
You want to stretch it more, via using an/one integer-MAGH higher? Fine... But that will chop off, left and right of the picture...
Most importantly to know... We can only MAGNIFY the RENDERED resolution, along integers (full numbers)... We can NOT (with the current code) manually 'stretch' the RENDERED picture to any shape, form, or size... Understood?!
For example:
A 512*240 RENDERED resolution...
512=X; 240=Y
Let's say we want to use 1080p as OUTPUT-RESOLUTION...
5124=2048 or 5123=1536
2404=960 or 2405=1200
You see?

As I mentioned above, I understand.
...but I don't think you're seeing my question?
When forcing 480p in GSM why does the output vary (squished/not) between games? Because they are rendering at different resolutions?

Yes. Developers were not restricted to use any one resolution. The magic of math allows a frame buffer of 512x448 to equally fill out a NTSC screen as much as a 640x448 frame buffer. But the latter will waste more screen space on the 480P display area, compared to the former.
(720-512)/720=28% is nearly a third of the width getting wasted, which is why it is so noticeable. On the other hand, (720-640)/720=11% is much more marginal.

Cant GSM magnify PAL 640x228 to 640x576 and even NTSC 640x576?
NTSC I know we will get cut off screen at botton and top but would be better than nothing...

First of all the PS2 outputs analog video. Analog video doesn't have the concept of pixels, it works with rows and timing. Every line starts and ends with a blanking area called horizontal blank to give the display time to move to the next line. The signal will hold for a certain amount of time on each line to represent a pixel on screen.
This does not translate well to a digital signal where the image is represented by distinct pixels.
To do this conversion most analog-to-digital converters will assume the most common resolutions for the specific number of lines in the signal on the specific input the signal comes from.
The PS2 usually communicates with CVBS (composit), S-Video, SCART or YPbPr (component), so the converter will assume it's a TV signal.
And it uses 525i, 625i or 525p lines which all most commonly have a line timing equal to 858.
Of these 525 and 625 lines the centre 480 and 576 lines are uses for picture and the rest is vertical blanking area and the centre 720 of the 858 lines is used for picture, the rest is horizontal blanking.
The issue you are seeing is that the PS2 doesn't use this common line samplerate.
While 640x448 and 640x512 does have a samplerate of 858, it doesn't use 720 but 640 of the pixels.
This means that the image will be slightly squished but the blank sides will likely be hidden by the TVs overscan.
With 512 wide games in interlaced however, the line samplerate should be something like 686.4. This will cause all the converted pixels to be blurry but when it takes those 858 samples and extracts the 720 centre pixels you will at least get a relatively close to correct aspect ratio.
So, finally to the issue you are seeing.
When you tell the PS2 to output a 512 wide game in progressive mode, it will put those 512 pixels and put it in a 720 wide frame adding blank space on the sides. This will then be sent as a 858 signal and correctly sampled by the converter (sharp pixels, yay!). But it will display the extra blanking area as part of the actual picture (boo!) and squees the picture even more than with a 640 game.
Home that at least clears some of it up.

More than satisfied — thanks all!

@gingerbeardman: Well, you did not understand it completely (before).
I see, understand and honestly tried to answer your questions.
Regarding your last 2 sentences: YES! EXACTLY!
Just for example:
GT4 uses a(n) '(internal) render-resolution' of 640(H/X) * 540(V/Y), when 1080i is chosen as 'output-resolution'... thus '640' is 'magnifyed' to 1920 via a Magnification-Integer (for the horizontal resolution [referred to as 'MAGH']) of '3' (3 * 640 = 1920) and either the vertical resolution is magnifyed by an integer value [referred to as 'MAGV'] of '2' (2 * 540 = 1080), or the interlace-framebuffer-switching-trick (or how to call it), is used to render the alternating half-frames and switch every half-frame (which would result in a true 1080px vertically resolution, contrary to the 'magnification-method').

The last answer @zappepappe wrote is also correct and PART OF the problem, but is rather related to what is referred to as 'dx' and 'dy' (or was it something with h and v?)!
You can still 'magnify' your RENDERED 'resolution' or 'picture' to fill the screen (output-resolution or 'frame' where the rendered picture is in) completely, but it will magnify it to the extent that some parts of the rendered picture will be 'outside' of the frame/output-resolution/'what's shown on screen'...

To put it short...
We could also set the output-resolution to i.e. 1080i for every game and let the render-resolution + game's usual/normal magnification on default and not recalculate it and you would get a mini-picture (black borders all around) in a big 'frame'.
The MAG-values are 'integers' (a Data-type to store a 'full number) and thus, using these we can only MULTIPLY the given render-resolution... i.e. 512 * 240 becomes 1024 * 720 (MAGV=2, MAGH=3)!
If we could use' floats' (floating point numbers/values) [those with a point or decimal], we could multiply by i.e. '1.36'...
BUT... I think we do theoretically can do that screen-stretching in 'smaller steps', although that probably would need manual tweaking of other values than MAGH&MAGV.

Tl;dr
To understand the problem, you need to understand these things.
-Render-Resolution
-output-resolution/frame (which GSM 'sets')
-magnification (multiplying of the render-resolution)
-'frame-composition' (which is the thing SP193 and zappepappe explained [the 2560-value and the other numerical values or 'Divisions' of that value])
Btw.: Me requesting if you understood it, wasn't meant mean. I removed it anyway, because I see it might 'sound' weird or harsh.

Not at all, I took it as intended 👍

Whuat? :-3
What exactly are you referring to?

Btw.: Me requesting if you understood it, wasn't meant mean.

I did not see it as being mean — don't worry

@ElPatas1 can this be closed too since an answer was provided?

Closed.
Best regards.


can I use unity weld on IL2CPP ?

hi
can I use unity weld for IL2CPP build ?

Yes if you set stripping level to a minimum.


Fix MapSnapshotter

Hopefully fixed #2869.
Frankly I'm not an Android developer, but a proper developer from our company says this line fixes things. He says, "NPE was because before the icon was png, but now it's svg, and you load those differently".

Okay I think we need more MS tests...

I agree...


Upgrade sense library to 0.4.2

Description:
This library update contains better connection management and error responses
Checklist:

[X] The code change is tested and works locally.
[X] Local tests pass with tox. Your PR cannot be merged unless tests pass

Please post a link to the change log next time


[BE] support for SendGrid unsubscribe groups

https://github.com/sklarsa/django-sendgrid-v5/blob/master/sendgrid_backend/mail.py#L208

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@1e2638a). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master      #39   +/-   ##
=========================================
  Coverage          ?   82.18%
=========================================
  Files             ?       31
  Lines             ?     1061
  Branches          ?        0
=========================================
  Hits              ?      872
  Misses            ?      189
  Partials          ?        0

Impacted Files
Coverage Δ

universal_notifications/notifications.py
95.86% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1e2638a...e28ad87. Read the comment docs.


Cypress: Make an API request to delete all reports in progress to webapp

What I mean here is the following. We can add an API call to the ladybug backend to remove all reports in progress. This call is not meant for the frontend itself, but it can be called directly by Cypress tests. This can make it easier to code these tests.


Matching IP to primary IP in Netbox

Changed matched device primary ip.
In Netbox 3.7.4 it was not working correctly.

Thank you, seems to me the data model has changed again. Will have a look into it.

Hi, I had a look at it. But I can't see the issue which this PR is trying to fix. Do you have an example?


[RFC]: Add/Update package of strided namespace for BooleanArray support (tracking issue)

Description
This RFC proposes add or update all packages of strided namespace that needs to be changed in order to add BooleanArray support to Stdlib. The purpose of this issue is to serve as a tracking issue for update or add packages.
Packages need to added:

[x] @stdlib/strided/base/reinterpret-boolean

Prerequisites of ndarray namespace:

[x] @stdlib/ndarray/base/dtype-char
[x] @stdlib/ndarray/base/dtype-desc
[x] @stdlib/ndarray/base/dtype2c
[x] @stdlib/ndarray/base/char2dtype
[x] @stdlib/ndarray/safe-casts
[x] @stdlib/ndarray/promotion-rules

Packages need to updated:

[x] @stdlib/strided/dtypes
[x] @stdlib/strided/base/nullary
[x] @stdlib/strided/base/nullary-addon-dispatch
[x] @stdlib/strided/base/unary
[x] @stdlib/strided/base/unary-addon-dispatch
[x] @stdlib/strided/base/mskunary
[x] @stdlib/strided/base/mskunary-addon-dispatch
[x] @stdlib/strided/base/binary
[x] @stdlib/strided/base/binary-addon-dispatch

Related Issues
None
Questions
No.
Other
No.
Checklist

[X] I have read and understood the Code of Conduct.
[X] Searched for existing issues and pull requests.
[X] The issue name begins with RFC:.

All tasks have now been completed. Closing...


Fixing tooltip JS error

At the moment if you use the fullscreen button, it renders a JS error. This stops that error from happening, as outlined here: https://github.com/summernote/summernote/issues/542

@cjeffries
Summernote's tooltip will be hide by bootstrap's tooltip component(http://getbootstrap.com/javascript/#tooltips). I thought you get error by using another jquery component, like jquery-ui's tooltip component.
Please refer to
twbs/bootstrap#6303

@san say this issue fixed by #30


List of Image show glitch when user scroll it.

Steps to Reproduce

Copy this code to main.dart
import 'dart:ui';

import 'package:flutter/material.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: Scaffold(
        body: ScrollConfiguration(
          behavior: ScrollConfiguration.of(context).copyWith(
            dragDevices: {
              PointerDeviceKind.touch,
              PointerDeviceKind.mouse,
            }
          ),
          child: ListView.builder(
            itemCount: 100,
            itemBuilder: (BuildContext context, int index) {
              return Image.network(
                'https://dummyimage.com/600x399/000000/0000ff.png&text=index$index',
                fit: BoxFit.fitWidth,
              );
            },
          ),
        ),
      ),
    );
  }
}

Expected results:
No Glitch.
Actual results:
Show Glitch.
Tested on macOS (glitched), android (glitched), web (no glitch).
In  macOS, resizing the window size sometime eliminate the glitch.
In android, resizing the image size sometime eliminate the glitch.

flutter doctor -v
[✓] Flutter (Channel dev, 2.5.0-1.0.pre, on macOS 11.4 20F71 darwin-x64, locale
    en-TL)
    • Flutter version 2.5.0-1.0.pre at /Users/anta/Mine/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 184e5871be (3 weeks ago), 2021-07-26 04:38:02 -0400
    • Engine revision a877f6fed6
    • Dart version 2.14.0 (build 2.14.0-350.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version
    31.0.0-rc4)
    • Android SDK at /Users/anta/Library/Android/sdk
    • Platform android-30, build-tools 31.0.0-rc4
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.5, Build version 12E262
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • CHROME_EXECUTABLE = /Applications/Google
      Chrome.app/Contents/MacOS/google-chrome-unsafe

[✓] Android Studio (version 4.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)

[✓] IntelliJ IDEA Ultimate Edition (version 2019.1.2)
    • IntelliJ at /Applications/IntelliJ IDEA.app
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin version 191.8593

[✓] Connected device (3 available)
    • H3223 (mobile)  • CQ3001CLTG • android-arm64  • Android 9 (API 28)
    • macOS (desktop) • macos      • darwin-x64     • macOS 11.4 20F71
      darwin-x64
    • Chrome (web)    • chrome     • web-javascript • Google Chrome

Video Record on macOS (Glitched)
https://user-images.githubusercontent.com/86121106/130063760-e361297e-67c6-4386-a72d-7f9cf4016195.mp4

Video Record on Android Device (Glitched)
https://user-images.githubusercontent.com/86121106/130064189-d56eda33-f919-4532-85c8-968108b6afd3.mp4

Video Record on Web (No Glitch)
https://user-images.githubusercontent.com/86121106/130064485-7c661ae9-21fc-4275-ad57-958dc80c10aa.mp4

Real world application troubled by this minor glitch is Comic-Like application, like WebToon.

Hi @hndrbrm, I am able to reproduce the issue on the latest stable and the master channel  looks like this is specific to this image asset and does not reproduce with other image assets whether local or network

Output
MacOS
https://user-images.githubusercontent.com/31410839/130100812-b18a4c76-b917-4c47-b81f-79ee0f74a37c.mov
Web
https://user-images.githubusercontent.com/31410839/130100854-b0479eab-0c33-421f-801c-b89e154780b7.mov
Android
https://user-images.githubusercontent.com/31410839/130100999-1aa55ff5-f929-46ba-8fb2-5772ea6b0161.mov
IOS
https://user-images.githubusercontent.com/31410839/130100975-c23409a6-cae7-499b-8298-835afe8bb03a.mov

Android
IOS
MacOs
Web

❌
❌
❌
✅

✅ : No Issue ❌: Issue reproduced

flutter doctor -v
[✓] Flutter (Channel stable, 2.2.3, on macOS 11.5.2 20G95 darwin-arm, locale en-GB)
    • Flutter version 2.2.3 at /Users/mahesh/Documents/flutter
    • Framework revision f4abaa0735 (6 weeks ago), 2021-07-01 12:46:11 -0700
    • Engine revision 241c87ad80
    • Dart version 2.13.4

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/mahesh/Library/Android/sdk
    • Platform android-31, build-tools 31.0.0
    • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.5.1, Build version 12E507
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)

[✓] VS Code (version 1.58.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.25.0

[✓] Connected device (4 available)
    • iPhone 12 Pro Max (mobile) • 5430F628-A903-402C-A635-D55D7E6EEFCA • ios            •
      com.apple.CoreSimulator.SimRuntime.iOS-14-5 (simulator)
    • macOS (desktop)            • macos                                • darwin-arm64   • macOS 11.5.2
      20G95 darwin-arm
    • Chrome (web)               • chrome                               • web-javascript • Google Chrome
      92.0.4515.131

• No issues found!

[✓] Flutter (Channel master, 2.5.0-7.0.pre.8, on macOS 11.5.2 20G95 darwin-arm, locale en-GB)
    • Flutter version 2.5.0-7.0.pre.8 at /Users/mahesh/Documents/flutter_master
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 40e8620a27 (7 days ago), 2021-08-06 09:14:50 +0430
    • Engine revision 431ac604da
    • Dart version 2.14.0 (build 2.14.0-383.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/mahesh/Library/Android/sdk
    • Platform android-31, build-tools 31.0.0
    • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.5.1, Build version 12E507
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)

[✓] VS Code (version 1.58.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.25.0

[✓] Connected device (4 available)
    • iPhone 12 Pro Max (mobile) • 5430F628-A903-402C-A635-D55D7E6EEFCA • ios            •
      com.apple.CoreSimulator.SimRuntime.iOS-14-5 (simulator)
    • macOS (desktop)            • macos                                • darwin-arm64   • macOS 11.5.2
      20G95 darwin-arm
    • Chrome (web)               • chrome                               • web-javascript • Google Chrome
      92.0.4515.131

• No issues found!

Looks like a duplicate of https://github.com/flutter/flutter/issues/37578

This looks like anti-aliasing at work.
For more details, see #14288


Automata

Enumerated data types in Lustre
type t = enum { A, B, C };
node n (x : enum { C1, C2 }, ...) ...

Enumerated datatypes are translated as is in the SMT solvers that support them directly (all of them for now).
N-way merge
Possibility to do a merge on a clock of an enumerated datatype.
merge c
 (A -> x when A(c))
 (B -> w + 1 when B(c));

Arguments of merge have to be sampled with the correct clock. Clock expressions for merge can be just a clock identifier or its negation or A(c) which is a stream that is true whenever c = A.
Merging on a Boolean clock can be done with two equivalent syntaxes:
merge(c; a when c; b when not c);

merge c
  (true -> a when c)
  (false -> b when not c);

Node restart / reset
Node can be reseted (or restarted, that's just a terminology naming) with a construct similar to activate. This follows the Scade 6 syntax.
(restart n every c)(a1, ..., an)

is a call to the node n that is restarted whenever the clock c is true. In practice, when this is the case, the internal state of the node is reset to its initial value.
Restart and activate can also be combined in the following way:
(activate (restart n every r) every c)(a1, ..., an)
(activate n every c restart every r)(a1, ..., an)

These two calls are the same (the second one is just syntactic sugar). The (instance of the) node n is restarted whenever r is true and the resulting call is activated when the clock c is true. Notice that the restart clock r is also sampled by c in this call.
Automata in Lustre
The major contribution of this pull request is the support for Lustre automata. The syntax is very close
to the one of Scade 6.
node n (i1, ..., in : ...) returns (o1, ..., on : ...);
let

   automaton automaton_name

     initial state S1:
       unless if c restart Si elsif c' resume Sj else restart Sk end;
       var v : ...;
       let
          v = ...;
          o1 = i1 -> last o2 + 1;
          o2 = 99;
       tel
       until c restart S2;

     state S2:
       let
          ...;
       tel
     ...
   returns o1, o2;

   o3 = something () ...;
tel

An automaton is declared inside a node (there can be several) and can be anonymous. This is followed by a list of states and a returns statement that specifies which variables (locals or output) are modified by the automaton.
Each state has a name and one of them can be declared initial (if no initial state is specified, the first one is considered initial). They can have local variables (to the state).
The body of the state contains Lustre equations (or assertions) and can use the operator last. In contrast to pre x which is the value of the stream x the last time the state was in the declared state, last x is the previous value of the stream x on the base clock. This construct is useful for communicating information between states.
States can have a strong transition (declared with unless) placed before the body and a weak transition placed after the body. The unless transition is taken when entering the state, whereas the until transition is evaluated after execution of the body of the state. If none are applicable then the automaton remains in the same state. These transitions express conditions to change states following a branching pattern. Following are examples of legal branching patterns (c* are Lustre Boolean expressions):
c restart S

if c1 restart S1
elsif c2 restart S2
elsif c3 restart S3
end;

if c1
  if c2 restart S2
  else if c3 resume S1
  end
elsif c3 resume S3
else restart S0
end;

Targets are of the form restart State_name or resume State_name.  When transiting to a state with restart, the internal state of the state is rested to its initial value. On the contrary when transiting with resume, execution in the state resumes to where it was when the state was last executed.
In counter-examples, we show the value of additional internal state information for each automaton:
automaton_name.state is a stream that denotes the state in which the automaton is and automaton_name.restart indicates if the state in which the automaton is was restarted in the current instant.
Other bug fixes

Functional constraints that are added in the solver are now scoped.
A bug was fixed for abstracting of pre which was not done correctly in some cases.

Mmmh, we have a performance issue with the non-guarded pre. Too many intermediate state variables for some benchmarks. This is due to the fact that we want pre x = pre x not to be true if non-guarded.

I'm not saying yet again that we should ban unguarded pre's because it makes no sense in a model checker on so many levels.
Nice try.


Can external tables connect to ADLS private endpoint?

I am able to create a linked service to a storage account using managed private endpoint and have Synapse pipeline use that linked service. However, can an external table from either serverless or dedicated sql pool connect to a storage account using private endpoint? If so, how can this be done? Right now, my external table will only work if the storage account has option "Allow trusted microsoft services" on, which indicates that it is not using the private endpoint.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f873d0f6-d930-d354-5da1-8fa698f77366
Version Independent ID: 3feff33e-55c1-90a3-ba2f-c27a71a1316e
Content: Create and use external tables in serverless SQL pool - Azure Synapse Analytics
Content Source: articles/synapse-analytics/sql/create-use-external-tables.md
Service: synapse-analytics
Sub-service: sql
GitHub Login: @vvasic-msft
Microsoft Alias: vvasic

@dckfnuance  Since this issue is not related directly to the doc, I would recommend you to create a thread on the forums - Microsoft Q&A or Stack Overflow
Once you post your issue on forums, it will have visibility across the community which is a better suited audience for such types of issues.


dynamic deserialisation

https://github.com/andrewheumann/jSwan
I was using this tool in the past and I am placing here the repository so we all know about the tool.
It create a grasshopper component following the json output. It might be pretty cool to "borrow it" and place it in FEM-Design

I've used it too and it is a really good way of deserializing and deconstructing objects in Grasshopper.
Also there is the Explode component in BHoM that does the same thing for any object, not just JSON

Duplicate of #92


More SpecialFunctions Tests

Resolves the cause of the error in #326 .
I need to go through the SpecialFunctions docs and check for any more public functions that we're not doing some testing on before merging this.
We definitely cannot handle most of the two-arg bessel functions still, because we do not have the ability to handle only partly-implement rules.

Additional tests have been added for various bits of functionality in SpecialFunctions, and an issue opened for a few things which aren't currently working, but really should be.
I will merge this PR and tag a release when CI finishes.


Basic cleanup tasks

This PR includes:

Add ability to add API to top-level TOC.
Fix templates to nest functions and use short names

Fixes #98
Fixes rtfd/pydoc.io#17
Fixes rtfd/pydoc.io#18

This should be ready for re-review.


[FIX] working in polymer2 and angular4, issue with px-app-nav-subgroup this.$.groupcontainer is undefined for when used in angular4

Pull Request
Hi,
Thanks for helping us improve the Predix UI platform by submitting a Pull Request.
To help us merge your Pull Request as fast as possible, please fill out the following sections:

A description of the changes proposed in the pull request:

animationConfig defined in ready() callback instead of properties as px-app-nav-subgroup this.$.groupcontainer returns undefined in properties when used in angular4

A reference to a related issue (if applicable):

@mentions of the person or team responsible for reviewing proposed changes:

working tests:

The tests need to be functional and/or unit tests, written for the wct framework, and placed in the test folder, following our testing guidelines.
tests are provided in test folder

Thanks for the contribution. We added a modified version of this fix in 26d93f230cbd0eea1fc9595c8f933aa8dceaec56. Please update your dependency on px-app-nav to v2.1.1 or later to get this fix.


Connect Dialog fails to list databases if selected DB doesn't exist

Connect dialog will fail to list databases if the selected database doesn't exist

Included in 2.24.0 🚀


Add controls to popup to customize the configuration

Yeah.

Good enough for now. Every time the configuration changes, it is saved, but new config is only queried on page-load.


fix(infra): Resolve capacity issues during rollouts

I've managed to finally reserve enough e2 instances for our needs and also used e2 for gateways to workaround the quota issues. The web app still used n2 because quota doesn't allow additional n4's. Rollouts also fixed to not go over the reservations/quotas.

Deployed, merging.


Upcoming updates May 2021

[x] builderbook: self-hosted fonts: using font from public folder for SSR page, using font from Cloudfront for CSR page (update Chapter 9)
[x] builderbook icons from @material-ui/icons (update Chapter 9)
[x] builderbook: update theme; styles for body, a, Login page, Button, Header
[x] builderbook: next v10, environmental variables: universal and server-only https://github.com/async-labs/builderbook/issues/357 (update Chapter 9)

[x] update book/9-end codebase
[x] update book's content

@tima101 I tested the main app in builderbook/builderbook folder. Features work as expected.


OnBeforeUnload handling not properly implemented

<EMAIL_ADDRESS>commented:

Which version of PhantomJS are you using?
1.8.2
What steps will reproduce the problem?

Using phantomJS open any webpage with onbeforeunload set.
Goto another page.

What is the expected output? What do you see instead?
I expect to be able to control whether or not to leave the page. The event do get caught, but it's sent as a onConfirm. If I choose to return true in the onConfirm it goes to the other page, if I select false the entire scripts stops running.
Which operating system are you using?
Windows 7 x64
Did you use binary PhantomJS or did you compile it from source?
Binary
Please provide any additional information below.
I use phantomJS to evaluate pages, some which include user supplied content, which in some cases contains redirects/reloads and I wish to be able to force phantomJS to stay on the page - I dont want to use the navigationLocked property.

Disclaimer:
This issue was migrated on 2013-03-15 from the project's former issue tracker on Google Code, Issue #1130.
:star2:   1 people had starred this issue at the time of migration.

Hello,
Is this issue still in the pipeline to being fixed? We are experiencing problems with it here, where Phantom is not willing to go to a different page because of our usage of beforeunload events.

@gdscei Are you experiencing this problem with PhantomJS 2.0?  If you're still using an older version, please test 2.0.

We are experiencing it in 1.9.8, we can't test in 2.0.0 as we rely on composer to import PhantomJS into our CI projects, and no 2.0.0 binary is available for Linux.

Looks like this issue can be closed as it's working with 2.0.
This test script returns:
$ ./phantomjs beforeunload.js
confirm: onbeforeunload

Indicating the onbeforeunload callback was called and triggered the page.onConfirm callback.

Looks like this issue can be closed as it's working with 2.0.
This test script returns:
$ ./phantomjs beforeunload.js
confirm: onbeforeunload

Indicating the onbeforeunload callback was called and triggered the page.onConfirm callback.

I just noticed it does have a major difference from all the major browsers. In webkit only iframes with the same origin as the main page can use the onbeforeunload callback. See FrameLoader.cpp#L2792 This is not the case in Chrome, Firefox or Opera (also Opera Next). I didn't have IE to test with.

I just noticed it does have a major difference from all the major browsers. In webkit only iframes with the same origin as the main page can use the onbeforeunload callback. See FrameLoader.cpp#L2792 This is not the case in Chrome, Firefox or Opera (also Opera Next). I didn't have IE to test with.

@erikdubbelboer, I'm not sure your test script tests the case where the window.location.href may be programmatically changed.  I believe your test is only testing changing the webpage externally.

@cgatesman How the location is changed doesn't seem to matter. Using this test script has the exact same result.

looks like in phantomjs 2.1.1 the problem still exists.   all subsequent webpage will not be opened while the onbeforeunload confirmation is not "confirmed"

Due to our very limited maintenance capacity (see #14541 for more details), we need to prioritize our development focus on other tasks. Therefore, this issue will be closed. In the future, if we see the need to attend to this issue again, then it will be reopened.
Thank you for your contribution!


typo of the configuration readme

Hi there... 2 minor issues to mention for future readers:
-- The following code in the readme should be using "vcap:services"
var instaneId = config["vcap:application:p-mysql:0:credentials:uri"]
-- To make it work , I have to change the string and use "user-provided". i.e. "vcap:services:user-provided:0:credentials:uri"
Lastly, I want to use poco object for the config in vcap service, and found CloudFoundryServicesOptions class is a collection and hard to parse. I ended up to write the following code. Welcome your suggestion/feedback.
public void ConfigureServices(IServiceCollection services){....
var configSection = GetVCAPService("my_config");
services.Configure(configSection);
...}
private IConfiguration GetVCAPService(string vcapServiceName)
{
var sections = Configuration.GetSection("vcap:services:user-provided");
        if (sections == null) return Configuration;

        foreach (var item in sections.GetChildren())
        {
            string name= (string)item.GetValue(typeof(string), "name");
            if (vcapServiceName == name)
                return item.GetSection("credentials");
         }
        return Configuration;
    }

Thanks,
Walter

Hi Walter,
We have fixed your simple to address concerns:

Fixing vcap:application: -> vcap:services: (where appropriate)
Fixing the instanceId typo
Replacing p-mysql with [service-name] to make its nature clear

Regarding the UX for reading values from the Configuration, we are not sure we have understood your concerns or what the best action to take is. For instance were you trying to do something similar to the following Linq ?
config.GetSection("vcap:services:user-provided").GetChildren()
                .Where(x => x["name"] == "jim")
                .Select(x => x["credentials:uri"])
                .First()

Dave and Keaty

I guess I am still old-fashion.I can use linq and inject into a poco class.


Material action button on the left of the screen

Subject

[ ] Bug
[x] Question

Question
Is there any way to add another action button on the left of the screen in material theme? We need two actions buttons: one on the right, another on the left.

Just add some custom style:

Thanks, I really thought it could have a hidden super css class. Thanks.


prevent duplicates issues to show up in sonar

for some reasons we got duplicated issues showing up in sonar, solved by ensuring only a unique issue is created

Thanks @jmecosta for this pull request.
Do you mean that inspectcode itself reports the same issue multiple times? Which build of it are you using, and on which rule do you see duplicates?

i dont know anymore, it was a couple of months ago so likely a outdated version of resharper also. I forgot about this, now was updating the c# plugin and notice this changes and decided to make the pr
the vs bootstrapper was off, and resharper was run only once for all the solution so am inclined to say resharper was to be blame
ive applied this and we havent seen this again, we also upgraded the cli also so perhaps there is no issues now, but best safe than sorry anyway.

I'll close this as I've never seen this in a R# myself: inspectcode.exe was in a beta stange (and perhaps still is?) and is very rapidly evolving.
They should definitely fix this on their side.

Yep its OK to close


fix: Hanging protocol state fixes

Context
After the state sync/hanging protocol merge, there were some issues with the hanging protocol state that need resolving.
Changes & Results

Testing

Checklist
PR

[] My Pull Request title is descriptive, accurate and follows the
semantic-release format and guidelines.

Code

[] My code has been well-documented (function documentation, inline comments,
etc.)

Public Documentation Updates

[] The documentation page has been updated as necessary for any public API
additions or removals.

Tested Environment

[] "OS:
[] "Node version:
[] "Browser:

Not sure if it is related, but after applying the recent state sync change some slices in volume appear to have their VOI too bright. It seems isolated to this commit: https://github.com/OHIF/Viewers/commit/803f6384017b61d8029ba553faa584dab6d92834. Rolling back to the previous removes the error. I'll try to screenshot error in action.

@wayfarer3130 please add more detail to the PR in terms of the specific changes made and scenarios to be tested. Thank you kindly.


How to cross-compile programs from amd64 to arm64

I'm trying to cross-compile some of the programs built on top of libbpf-bootstrap, but I'm lacking expertise with C/C++ ecosystem. I see that support for cross-compilation was added in https://github.com/libbpf/libbpf-bootstrap/commit/7b60b4fd6c6da6af046dc6f618c771f763b6a472
I would appreciate help with:

what are the packages I have to have installed on Ubuntu 22.04 (amd64) to cross-compile programs to arm64?
do I have to cross-compile libelf and zlib first? If so, any hints how can I do it?

My current results:
~ CROSS_COMPILE=aarch64-linux-gnu-g make all
  BINARY   bind
/usr/lib/gcc-cross/aarch64-linux-gnu/11/../../../../aarch64-linux-gnu/bin/ld: cannot find -lelf: No such file or directory
/usr/lib/gcc-cross/aarch64-linux-gnu/11/../../../../aarch64-linux-gnu/bin/ld: cannot find -lz: No such file or directory

@wangjie could you please help? I'm not very familiar with cross-compilation, unfortunately.

I'm trying to cross-compile some of the programs built on top of libbpf-bootstrap, but I'm lacking expertise with C/C++ ecosystem. I see that support for cross-compilation was added in 7b60b4f I would appreciate help with:

what are the packages I have to have installed on Ubuntu 22.04 (amd64) to cross-compile programs to arm64?
do I have to cross-compile libelf and zlib first? If so, any hints how can I do it?

My current results:
~ CROSS_COMPILE=aarch64-linux-gnu-g make all
  BINARY   bind
/usr/lib/gcc-cross/aarch64-linux-gnu/11/../../../../aarch64-linux-gnu/bin/ld: cannot find -lelf: No such file or directory
/usr/lib/gcc-cross/aarch64-linux-gnu/11/../../../../aarch64-linux-gnu/bin/ld: cannot find -lz: No such file or directory

Hi  bartsmykla,
Yes, libbpf depends on libelf and libz, so you need declare where the libraries and header files can be found.
such as, make EXTRA_CFLAGS="-IXXX" EXTRA_LDFLAGS="-LXXX" ARCH=arm64.
You can get a copy of these libraries from here for quickly verification if you don't want to cross-compile them yourself.

the same here，thanks

libbpf-bootstrap-android
I embbed deps and add externel path to involve BTF file for android kernel，wish this can hell you

May I merge this support to mainline?I'll remove some useless files in my repo.

can you submit a clear PR with only necessary changes?

can you submit a clear PR with only necessary changes?

Ok，I will do it recently.


Using Unity 2017.2.0b2 with HoloToolkit causes Unity to not build project

Once I add the HTK to my project, even if I am not using anything in it, and I try to build for the HoloLens from Unity, it fails with the following errors, however, if I pull the HTK out of my assets folder, it builds just fine.  I just tried this with the very latest merged code as of noon on 7/13 and am still seeing the same behavior.

KeyNotFoundException: The given key was not present in the dictionary.
System.ThrowHelper.ThrowKeyNotFoundException () (at :0)
System.Collections.Generic.Dictionary`2[TKey,TValue].get_Item (TKey key) (at :0)
MetroCSharpVisualStudioSolutionCreator.WriteSolutionFile (System.String solutionFileName, UnityEditor.Scripting.ScriptCompilation.ScriptAssembly[] csharpAssemblies) (at C:/buildslave/unity/build/PlatformDependent/MetroPlayer/Extensions/Managed/MetroCSharpVisualStudioSolutionCreator.cs:166)
MetroCSharpVisualStudioSolutionCreator.CreateSolutionFileFrom () (at C:/buildslave/unity/build/PlatformDependent/MetroPlayer/Extensions/Managed/MetroCSharpVisualStudioSolutionCreator.cs:716)
PostProcessWSA.CreateVisualStudioSolution () (at C:/buildslave/unity/build/PlatformDependent/MetroPlayer/Extensions/Managed/PostProcessWSA.cs:320)
PostProcessWinRT.Process () (at C:/buildslave/unity/build/PlatformDependent/WinRT/SharedSources/CSharp/PostProcessWinRT.cs:221)
UnityEditor.WSA.BuildPostprocessor.DoPostProcess (UnityEditor.Modules.BuildPostProcessArgs args) (at C:/buildslave/unity/build/PlatformDependent/MetroPlayer/Extensions/Managed/ExtensionModule.cs:142)
UnityEditor.WSA.BuildPostprocessor.PostProcess (UnityEditor.Modules.BuildPostProcessArgs args) (at C:/buildslave/unity/build/PlatformDependent/MetroPlayer/Extensions/Managed/ExtensionModule.cs:149)
UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr)

Second Error

Build completed with a result of 'Failed'
UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr)

Final Error

UnityEditor.BuildPlayerWindow+BuildMethodException: 2 errors
at UnityEditor.BuildPlayerWindow+DefaultBuildMethods.BuildPlayer (UnityEditor.BuildPlayerOptions options) [0x00207] in C:\buildslave\unity\build\Editor\Mono\BuildPlayerWindowBuildMethods.cs:172
at UnityEditor.BuildPlayerWindow.CallBuildMethods (System.Boolean askForBuildLocation, UnityEditor.BuildOptions defaultBuildOptions) [0x00050] in C:\buildslave\unity\build\Editor\Mono\BuildPlayerWindowBuildMethods.cs:83
UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr)

Confirming that I also have this issue with 2017.2.0b2 and latest HTK

This appears to be a bug in Unity.  If the option Unity C# Projects is selected then the build fails with the above errors, however leaving this unchecked allows the build to complete successfully.  Closing this issue and opening bug with Unity.

@jessemcculloch  Link to the Unity bug ticket?

Possibly this one?
https://issuetracker.unity3d.com/issues/wsa-error-messages-appear-while-slash-after-building-a-project-with-unity-c-number-projects-and-if-the-editor-folder-contains-a-script?_ga=2.42455462.28322216.1511996104-1442634586.1474396936
Claims fixed in 2017.3, but I haven't been hitting it in the 2017.2 MRTP builds (I haven't tried regular 2017.2 builds).

Thanks! We hit something with a similar error but turns out it was something else unrelated to the toolkit.

This issue is still coming in 2017.2.1.p4 with Unity c# projects option disabled. What is the solution for this ?

UnityEditor.BuildPlayerWindow+BuildMethodException: 4 errors
at UnityEditor.BuildPlayerWindow+DefaultBuildMethods.BuildPlayer (BuildPlayerOptions options) [0x00207] in /Users/builduser/buildslave/unity/build/Editor/Mono/BuildPlayerWindowBuildMethods.cs:172
at UnityEditor.BuildPlayerWindow.CallBuildMethods (Boolean askForBuildLocation, BuildOptions defaultBuildOptions) [0x00050] in /Users/builduser/buildslave/unity/build/Editor/Mono/BuildPlayerWindowBuildMethods.cs:83
UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr)
bug bug bug

This was a known issue in Unity 2017.2.0b2 when C# Projects was checked during the UWP build (and is unrelated to the Toolkit itself). Should be fixed on non-beta releases. Are you seeing it elsewhere?


Error t.get is not a function when used in the latest Mendix Pro 8 modeler

Hi,
We are facing the following error when upgraded our project to Mendix Pro 8. The Charts widget is throwing an error like t.get is not a function.

For data source MF is an known issue this will be fixed in Beta 2, which will be released soon. Please let us know when this does not work for you. Cheers.

Mendix 8 Beta 2 is available and solves this issue.


dart2js regression when using as with types w/ non-default type-arguments

void main(List<String> arguments) {
  final emptyMap = <String, dynamic>{};
  // this is fine!
  print((emptyMap['not here'] as List)?.length);
  // this throws on `as`
  print((emptyMap['not here'] as List<int>)?.length);
}

Works fine w/ DDC
Fails with dart2js
Dart SDK version: 2.11.0-176.0.dev

Error TypeError: null: type 'JSNull' is not a subtype of type 'List<int>'

@kevmoo I'm having trouble reproducing this on master.
In weak mode (whether the experiment is explicitly enabled or not), we print out null twice.
In strong mode, the line that you've marked as fine actually throws TypeError: null: type 'JSNull' is not a subtype of type 'List<dynamic>'
Are you running with a particular set of flags?

I think this was broken here: https://dart-review.googlesource.com/c/sdk/+/157562
and fixed here: https://dart-review.googlesource.com/c/sdk/+/164941
(This was the same issue that broke a bunch of benchmarks, although we have no tests in the SDK that detect this.)

Going to close this issue. I'm not sure if there's a good way to write a regression test for this with our current infrastructure, but the issue was detected in multiple other places.
cc @joshualitt in case he has thoughts on how we can write dart-lang/null_safety_sample_package as an SDK test.


[SPARK-17072] [SQL] support table-level statistics generation and storing into/loading from metastore

What changes were proposed in this pull request?

Support generation table-level statistics for

hive tables in HiveExternalCatalog
data source tables in HiveExternalCatalog
data source tables in InMemoryCatalog.

Add a property "catalogStats" in CatalogTable to hold statistics in Spark side.
Put logics of statistics transformation between Spark and Hive in HiveClientImpl.
Extend Statistics class by adding rowCount (will add estimatedSize when we have column stats).

How was this patch tested?
add unit tests

Can one of the admins verify this patch?

/cc @cloud-fan @rxin

Test build #3227 has started for PR 14712 at commit 4375e76.

Test build #3227 has finished for PR 14712 at commit 4375e76.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class Statistics(
case class AnalyzeTableCommand(tableName: String, noscan: Boolean = true) extends RunnableCommand

So far, the test coverage is weak. Could we add more test cases to cover all the corner cases? Thanks!

If the goal also includes the interoperability with Hive, the test cases should also verify whether the table property COLUMN_STATS_ACCURATE is true or not. This should be implicitly updated by Hive.

a high-level question: Looks like the current design depends on some features of hive metastore, e.g. the STATS_GENERATED_VIA_STATS_TASK flag. Is it possible that we just treat hive metastore as a persistent level? So that the statistics can still work if Spark SQL has its own metastore in the future.

@cloud-fan @gatorsmile Actually, we desperately need spark sql to have its own metastore, because we need to persist statistics like histograms which AFAIK hive metastore doesn't support.

@wzhfy I am kind of worried about the dependency of Hive metastore.

Using the same table property names like Hive: Hive metastore could change them in an unexpected way (for example, by a Hive metastore bug). The behaviors might be different in different metastore versions. However, the statistics can be shared by Hive execution and Spark execution. Thus, it could benefit the Spark users who are using Hive and Spark together on the same set of tables.

Using the different names, like what we did for data source table schema: Spark fully controls them and we just treat the Hive metastore as a persistent storage of these statistics.

Provided a configuration parameter so that users can control which option they preferred?

This is a design decision we need to make. @rxin @yhuai @hvanhovell @liancheng @cloud-fan @clockfly

My proposal is: Like data source table metadata, we store the table statistics using different names from hive, if the statistics is hive compatible, like row count, we also store the corresponding hive entries. In this way, we won't be affected by possible hive metastore bugs, and hive can also recognize table statistics generated by spark.
When we read in a hive table, if its statistics already exists but in hive format, we can generate the corresponding spark sql entries. Then spark sql can also recognize table statistics generated by hive.

I like @cloud-fan 's proposal. : )
When the values of these two copies are different, which one is preferred? a) Hive: if we prefer Hive's version, we might be affected by Hive. b) Spark: if we choose Spark's copy, we might lose the benefit of Hive-generated statistics. Adding a new configuration parameter? The default is Option a? (I also do not like adding any extra parameter)

If it is a hive table, I think we should respect hive's statistics.

I suggest in the current stage, we still follow hive's convention. When spark sql has its own metastore, we can bridge between these two metastores by a mapping between two different sets of names/data structures, and then provide a config for users to declare their preference.

Spark SQL already has its own metastore: InMemoryCatalog. And we do have an abstraction for metasotre: ExternalCatalog. We have 2 targets here:

add table statistics in Spark SQL
Spark SQL and Hive should recognize table statistics from each other.

I think target 1 is more important, and we do need an implementation that not depend on hive features.

Actually, we desperately need spark sql to have its own metastore, because we need to persist statistics like histograms which AFAIK hive metastore doesn't support.

We store table statistics in table properties, why would hive metastore not support it? Do you mean Hive can't recognize it? But I think it's ok, we should not limit our table statistics by what Hive supports.

@cloud-fan what i was saying is out of the range of this pr.
Table property is a string-string map, which means we need to transform every statistic into/from a string. This is ok for simple table statistics like "numRows", but not a good choice for complicated column statistics like histograms.
InMemoryCatalog is "in memory", we can have our own properties or data structures, but currently we still need Hive metastore api to “persist” these statistics. Hive has APIs for storing/loading table properties and ColumnStats, but no api for histograms.
What I'm trying to say is, we can use hive as a persistent level, but what we can store/load is still limited by its api. Of course we can put everything into properties, but it's not elegant.

Ok let's focus on this PR. You use the same table property names of Hive for the new statistics, then what if they are changed by Hive unexpectedly? Are they supported by all versions of Hive?
I think it's good to not mix the target 1 and 2 mentioned in my last comment together, and what do you think about my proposal in https://github.com/apache/spark/pull/14712#issuecomment-241207069? We can hide the hive-compatible logic in HiveExternalCatalog, so that InMemoryCatalog can support it too.
BTW in the view of API, InMemoryCatalog is also a persistent level, table statistics should also work in InMemoryCatalog.

Of course we can put everything into properties, but it's not elegant.
But sometimes we have to, like data source table metadata. I think it's ok if we can hide the logic in HiveExternalCatalog.

@cloud-fan I agree with you. I'll update this pr tomorrow.

What is our CBO plan for data source tables? Is it implemented in the prototype? It is not mentioned in the design doc. If we want to support InMemoryCatalog, we need to provide the support now.

@gatorsmile yes, we will support it in this pr

Great! Will review the new changes when it is ready. Thanks!

@cloud-fan Can you please launch test for this pr? thanks!

retest this please

Test build #64459 has started for PR 14712 at commit aee4139.

Test build #64459 has finished for PR 14712 at commit aee4139.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64459/
Test FAILed.

@wzhfy do you know how hive deal with the STATS_GENERATED_VIA_STATS_TASK flag and COLUMN_STATS_ACCURATE flag?

@cloud-fan did you see the comment by @gatorsmile ?

@gatorsmial, do you mean COLUMN_STATS_ACCURATE exists in any table's properties?

I wanna make sure we are in the same page:
top priority: users analyze the table in Spark and query it in Spark. (this must work)
second priority: the table is already analyzed by Hive, users query it in Spark.(this should work)
low priority: users analyze the table in Spark and query it in Hive.(it will be good if this works, but also fine if this doesn't)
I'm a little hesitant about dealing with the STATS_GENERATED_VIA_STATS_TASK and COLUMN_STATS_ACCURATE flags, are they only needed for the third(low priority) target? If they are, I'd like to ignore them to simplify the logic.

I think the problem lies in the dependence on hive metastore. As long as we use hive metastore to persist/retrieve statistics, we need to deal with these flags.

If we analyze the table in spark and want to persist stats in Hive metastore (and retrieve them when we do queries), we must set STATS_GENERATED_VIA_STATS_TASK, otherwise the stats won't be stored. - this is in top priority
If users alter table's properties (without setting STATS_GENERATED_VIA_STATS_TASK) in Spark or Hive, then COLUMN_STATS_ACCURATE will be false and stats be set to invalid, so it's unnecessary to read the stats in Spark. - this is in second priority

we must set STATS_GENERATED_VIA_STATS_TASK, otherwise the stats won't be stored

What if we use different property names to store statistics?

What if we use different property names to store statistics?
this may cause some consistency issues when both names exist in the metastore, we can't tell which one is the latest.

@cloud-fan I think the current implementation is ok and can support three level priorities. The logic is encapsulated in two methods for storing/retrieving stats. When the interaction behavior with hive is changed in the future, its influence range is controlled and we just need to modify these two methods.

@cloud-fan let me do more investigation and will reply your question later.

Very recently, Hive 2.0.0 fixed a serious bug. See the JIRA: https://issues.apache.org/jira/browse/HIVE-12661
We can get a wrong result when the statistics are out of dated (i.e., wrong). I can easily reproduce it.
Hive made a few changes in COLUMN_STATS_ACCURATE. Before Hive 2.0.0, it is like
COLUMN_STATS_ACCURATE true

After the fix, this becomes different.
COLUMN_STATS_ACCURATE {"COLUMN_STATS":{"key":"true","value":"true"},"BASIC_STATS":"true"}

I start worrying about the statistics value we populate might affect the query results through Hive interface, especially when we set STATS_GENERATED_VIA_STATS_TASK.
We are unable to implement a concurrency control between Hive and Spark. If we use the same names, statistics is like a shared memory space. Both Hive and Spark can modify it without notice. I have not found the bug, but it sounds risky.
Conceptually, setting STATS_GENERATED_VIA_STATS_TASK is wrong. As @wzhfy pointed out, numRows is always -1 if we do not set it. That also indicates numRows is not allowed external users to set it, right?
I am not very confident that we can implement a very stable solution for sharing Spark-generated statistics with Hive, since Hive is out of our control. However, Spark should be able to leverage Hive-generated statistics and the statistics we generated in this CBO work.

Found another related JIRA: https://issues.apache.org/jira/browse/HIVE-12730. This is only available in Hive 2.1. They are trying to resolve the issues we hit:

We would like to provide a way for developers/users to modify the numRows and dataSize for a table/partition. Right now although they are part of the table properties, they will be set to -1 when the task is not coming from a statsTask.

Now, users can change the statistics by the DDL statement. For example,
alter table s update statistics set('numRows'='1212', 'rawDataSize'='500500');

More important, after this fix, what we did in this PR does not work. STATS_GENERATED_VIA_STATS_TASK is not available. Instead, they changed it to STATS_GENERATED. Two options are availabel: TASK and USER.
Thus, I believe our solution should not be based on how Hive behaves, if possible.

One minor comment about the current PR and the future PRs in statistics. When we adding new statistics-specific properties, we should exclude them in the DDL command, like SHOW CREATE TABLE.
BTW, now, we have a bug in this part. Hive-generated statistics should not be shown too. I will fix it in a separate PR.

How about we use our own property names for now, and provide a Hive translation layer in a different PR. IMO it is fine to break a little bit of the behavior in master as long as we fix it (or make a well founded choice not to) before 2.1.

@gatorsmile lets also create another ticket for the explicit statistics updates. I do like the alter table s update statistics set ... option.

I like @hvanhovell 's proposal, since providing a perfect Hive translation layer is not trivial based on @gatorsmile 's investigation - we need to deal with different versions of Hive. It is better to be decoupled from this pr. What do you think? @gatorsmile @cloud-fan

@wzhfy Yeah, it sounds good to me to split the whole problem into multiple PRs.
@hvanhovell Sure, let me create the JIRA and I can work on this when the other dependent JIRAs are resolved.

yeah I think we should focus in the top priority target in this pr. And the hive related translation can be addressed in later prs.

@wzhfy sounds good to me, let's focus on the top priority in this PR :)

add to whitelist

retest this please

Looks like Jenkins doesn't work for a while.

Test build #64567 has started for PR 14712 at commit 3407c7f.

Test build #64567 has finished for PR 14712 at commit 3407c7f.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64567/
Test FAILed.

@cloud-fan @hvanhovell Could you launch a test for this pr? Thank you!

Test build #64625 has started for PR 14712 at commit 9c27071.

Test build #64625 has finished for PR 14712 at commit 9c27071.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64625/
Test FAILed.

Merged build finished. Test FAILed.

@cloud-fan @hvanhovell Oh, sorry, it's already been lauched. There's latency for about 5 mins.

Since you are added to the white list, you can trigger the test by yourself. Below is the command you can use:

"ok to test" to accept this pull request for testing
"test this please" for a one time test run
If the build fails for other various reasons you can rebuild. "retest this please" to start a new build

Test build #64626 has started for PR 14712 at commit 7e39a86.

@gatorsmile Thank you for the information!

Test build #64626 has finished for PR 14712 at commit 7e39a86.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64626/
Test FAILed.

Merged build finished. Test FAILed.

Test build #64641 has started for PR 14712 at commit aef78d4.

Test build #64641 has finished for PR 14712 at commit aef78d4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64641/
Test PASSed.

this pr is updated and passed all tests, please review @cloud-fan @hvanhovell @gatorsmile @viirya

Test build #64709 has started for PR 14712 at commit c7cc55f.

Test build #64709 has finished for PR 14712 at commit c7cc55f.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64709/
Test FAILed.

Test build #64713 has started for PR 14712 at commit 56ec68e.

Test build #64713 has finished for PR 14712 at commit 56ec68e.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64713/
Test FAILed.

Merged build finished. Test FAILed.

Test build #64726 has started for PR 14712 at commit 9715770.

Test build #64726 has finished for PR 14712 at commit 9715770.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64726/
Test PASSed.

Test build #64750 has started for PR 14712 at commit aa438c4.

Looks much better now.

Test build #64750 has finished for PR 14712 at commit aa438c4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64750/
Test PASSed.

Maybe I found another bug in the master branch?
When calculating statistics for data source tables, we do not exclude the staging directory. However, we exclude them when AnalyzeTableCommand calculating the size. Since we convert Hive serde tables to data source tables, it sounds like we should also exclude Hive staging directory, right?

@gatorsmile Yes, we should exclude the staging dir.

Test build #64789 has started for PR 14712 at commit b6c655a.

Test build #64789 has finished for PR 14712 at commit b6c655a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64789/
Test PASSed.

@cloud-fan @hvanhovell @gatorsmile Please review again, thanks!

looks pretty good now! I left some comments about some small issues, thanks for working on it!

Test build #64848 has started for PR 14712 at commit b946df0.

Test build #64848 has finished for PR 14712 at commit b946df0.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64848/
Test FAILed.

Merged build finished. Test FAILed.

Test build #3244 has started for PR 14712 at commit b946df0.

retest this please

Test build #64850 has started for PR 14712 at commit b946df0.

Test build #3244 has finished for PR 14712 at commit b946df0.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test build #64850 has finished for PR 14712 at commit b946df0.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64850/
Test PASSed.

Merged build finished. Test PASSed.

LGTM, @hvanhovell can you take another look?

Below is a test case for a table with empty column. Could you also add it here?
  test("statistics collection of a table with zero column") {
    val table_no_cols = "table_no_cols"
    withTable(table_no_cols) {
      val rddNoCols = sparkContext.parallelize(1 to 10).map(_ => Row.empty)
      val dfNoCols = spark.createDataFrame(rddNoCols, StructType(Seq.empty))
      dfNoCols.write.format("json").saveAsTable(table_no_cols)
      sql(s"ANALYZE TABLE $table_no_cols COMPUTE STATISTICS")
      checkLogicalRelationStats(table_no_cols, expectedStats =
        Some(Statistics(sizeInBytes = 30, rowCount = Some(10))))
    }
  }

In the future, we will do column-level statistics collection. This might help you when you implement collection of column-level statistics.

LGTM except two minor comments about test cases.

@gatorsmile Thank you for the good test cases!

Test build #64886 has started for PR 14712 at commit 5d6e559.

Test build #64886 has finished for PR 14712 at commit 5d6e559.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64886/
Test PASSed.

Merged build finished. Test PASSed.

LGTM. Merging to master. Thanks!

Can you take a look at the test at https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64956/testReport/junit/org.apache.spark.sql.hive/StatisticsSuite/test_statistics_of_LogicalRelation_converted_from_MetastoreRelation/? It is flaky.

I have created https://issues.apache.org/jira/browse/SPARK-17408. @wzhfy Can you take a look?

These tests block multiple PRs. It is midnight in China. : ) Let me do a quick fix based on the comments of @cloud-fan and @hvanhovell

@yhuai @hvanhovell @cloud-fan Sorry for the late response, I'm out of office for two days.
@gatorsmile Thanks to fix it!


 As visible on the screenshots, all 55 rows are displayed at once, the > and < buttons have no effect, and the page content display (1-50 of 55 or 51-55 of 55) is misleading.

Expected behavior
Either:

Remove the pagination UI elements entirely, or
Make it work. Since the query is typed by the user and since the query splitting is done in the driver, it doesn't seem possible to inject a limit clause. The pagination could be "offline", and simply display n elements at a time of the returned results.

Screenshots

Desktop (please complete the following information):

SQLTools Version 0.23.0
VSCode Version: 1.48.2
OS: Mac
Driver:

[x] PostgreSQL/Redshift
[ ] MySQL/MariaDB
[ ] MSSQL/Azure
[x] SQLite
[ ] Other? Which...

Database version: [e.g. MySQL v5.6, PostgreSQL vX...]

I also noticed this with MariaDB. Most of the time the pagination works for me, but sometimes it doesn't.

After further research, my understanding is that, first of all, at the very least, pageSize must be set in the NSDatabase.IResult object. Otherwise it's assumed to be 50.
Then, pagination can be removed entirely by specifying that the page size is the result size. In most case that appears to be the easiest way.
To actually implement pagination, queryType is required, as well as queryParams. However, only showRecords and describeTable are supported, and using showRecords results in the showRecords method, which is not convenient.
My recommendation would be to fix the included drivers to set pageSize == total in the result.

This results table is basically useless if more than 50 results are returned, because the sorting and filtering in the header do not work. The only work-around is to add query params to limit the number of results.


Neo4j version and missing index

I have loaded the data using SciGraph and moved the database folder to a Neo4j server 3.xx.
(migrating to 4.0 wasn't successful)
I am able to work with the graph but I am missing the index created.
I can see data stored in index store
:sysinfo
but nothing when I do
CALL db.indexes
Any suggestions?

for running scigraph output in neo4j you need to move the data out of the graph folder into a graph.db directory, for example
mv graph/* /some/path/databases/graph.db/
then in your neo4j.conf
dbms.directories.data=/some/path

I didn't have any problem loading the database folder. The issue is missing index.
I did some pizza tests, loading the database and upgrading it from 3.4.7 (current scigraph version) to 3.5.x data and index looks okay there.
In pizza example, when I CALL db.indexes, I can see "INDEX ON :food(label)"
provider: {  "version": "2.0", "key": "lucene+native"}
In my graph, I can't see that index, but I see data usage in Index Store.


Create a sample that shows Pixel Manipulation using JS Libraries

decode image
modify data
display modified image in UXP UI
(optionally) write back to the host app

@pklaschka @ErinFinnegan Since <canvas> api is not available in UXP environment there is no simple solution for this. I think i can use WASM to make it. I can work on it. Please let me know :)

@pklaschka @ErinFinnegan Since <canvas> api is not available in UXP environment there is no simple solution for this. I think i can use WASM to make it. I can work on it. Please let me know :)

@ThisIsMrAli That sounds awesome. I'll ask the team if anyone's currently working on this (to avoid two people doing the same thing). Still, generally speaking, we're always thankful for contributions (and expanding on the idea of using WASM in plugins definitely sounds cool), so I'll let you know if anyone else is working on this as soon as I'm able to confirm it.
Just as a heads up: If you haven't done so already, you'll need to sign a Contributor's License Agreement once you open a Pull Request so that we can accept contributions to the repository (just mentioning this so that you don't begin to work on this and in the end, it goes to waste because of the CLA).

@pklaschka Thanks for your reply. I am familiar with uxp and we released some plugins for Adobe Xd earlier in uxplugins as adobe's partner. Actually as Cep developer with several years experiences, I think UXP is really a great way to develop native plugins. So I really love to expand it:)
I was wondering how i can make pull requests and contribute to this repository, Please let me know where can i find the Contributor's License Agreement and how is the workflow.
Actually there are some more ideas that i like to work on them on UXP platform too such as:

Using Nodejs in UXP: Since Nodejs env is not available in uxp, we can use PKG to make executable file of nodejs, so its possible to communicate between nodejs and UXP.
Infinite Scroll Demo in UXP
Using Web Components in UXP
If we could use Nodejs in UXP, Maybe we can develop a tool that converts CEP based plugins to UXP, by seperating the nodejs and js part and connecting the nodejs part to nodejs Executer(explained in the first part) and for the UI part by fixing unsupported tags and attributes.

Please let me know you idea:)

@ThisIsMrAli Awesome! I'm not sure if there is currently any user-friendly way of using PKG since UXP plugins can't run arbitrary applications, which could make it hard to create a good UX for running a helper app, but this could be interesting to look into.
Also, Infinite scrolling and web components sound interesting for samples.
If you'd like to work on this issue: Noone else seems to be working on it right now => if you want to give it a shot, feel free to do so ☺.
Regarding the CLA: That's pretty easy. The only thing you'll need to do at some point in time before we merge your Pull Request is to head over to https://opensource.adobe.com/cla.html and fill out / e-sign the form there. Then, a green checkbox appears in your PRs (once you open them), and we can merge your contributions 😉 . Apart from that, you can just fork this repo, make your changes, and open Pull Requests. External contributions are welcome, and after signing the CLA, there's nothing in the way of your PRs getting merged 🙂 .

@pklaschka
Awesome Thanks!
I just signed the CLA. Should i start with this issue that i reported recently? :)


Canvas removes body.

Installing canvas causes the first person player model to stop rendering.
https://www.curseforge.com/minecraft/mc-mods/canvas-renderer

For now no intentions to explicitly make Canvas compatible(Focus only being on Sodium and Optifabric). Once all 3 support the Fabric Post-Render-Entities Event I'll switch to it and then Canvas should also work.


⚠ Processors has degraded performance

In 0227ce3, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 114 ms

Resolved: Processors performance has improved in 114aaf3.


rconn: Avoid assertion failure if peer fails to send hello.

If an rconn peer fails to send a hello message, the version number doesn't
get set.  Later, if the peer delays long enough, the rconn attempts to send
an echo request but assert-fails instead because it doesn't know what
version to use.  This fixes the problem.
To reproduce this problem:
make sandbox
ovs-vsctl add-br br0
ovs-vsctl set-controller br0 ptcp:12345
nc <IP_ADDRESS> 12345

and wait 10 seconds for ovs-vswitchd to die.  (Then exit the sandbox.)
Reported-by: 张东亚<EMAIL_ADDRESS>Signed-off-by: Ben Pfaff<EMAIL_ADDRESS>

This was committed as commit a203ce69c96e8a38c71113973caf64a22003616f.


Cleanup t.Logf & t.Fail

Tests have a lot of:
		t.Logf("error not ErrTruncated from header-only unpack: %v", err)
		t.Fail()

This should just be compacted into t.Fatalf(...)

[ Quoting<EMAIL_ADDRESS>in "Re: [dns] Cleanup t.Logf & t.Fail (..."

Yeah, I know :) But its two lines, we can get away by just trimming this to one
line.
/Miek
--
Miek Gieben

Closed with #367

Just note btw (might be useful for future)  t.Logf only prints when test ran with verbosity. Fatalf prints always. Also in places where t.Fail replaced with t.Errorf it'll be "print and continue" vs previous "print and return"


OCPBUGS-36296: fix: sort CredentialsRequest manifests after namespace

CredentialsRequests go into the openshift-cloud-credential-operator namespace, so they can come before or after the openshift-cluster-api namespace. But because they ask for Secrets in the openshift-cluster-api namespace, there would be less race and drama if the CredentialsRequest manifests.

/retest

/assign @wking @JoelSpeed

/approve

/assign @wking

/jira refresh

/label acknowledge-critical-fixes-only

/cherry-pick release-4.16 release-4.15 release-4.14


Fixed length with arrays?

I have a situation where I'd like to parse a flat file without linebreaks that also contains "sub" arrays.  Is there a way to specify that as a parser definition?  (I may not have the right syntax for your PZMAP) For example, if I had a definition like this:
<record name="owner">
    <column  length="5"/>
    <record name="pet" length="5" repeat="3" />
</record>

And a dataset like this (represents 2 records of 20 characters, pipes added for visibility):
|AliceTiny Fluff     Bob  Zap  Frog Wooly|

Would create:
<owner name="Alice">
    <pet>Tiny</pet>
    <pet>Fluff</pet>
</owner>
<owner name="Bob">
    <pet>Zap</pet>
    <pet>Frog</pet>
    <pet>Wooly</pet>
</owner>

Hi
I do not know if it is possible but I'd assume not.
A workaround would be to define pet1,pet2 and pet3 in the XML definition and then call those.
Also you could define a Pets subsection with a discriminator.  A common type of record could be embedded / different, eg if a record starts with 'X' then the format should be a record X; if it starts with 'Y' then follow the Y record.
I hope this helps
Feel free to submit a patch or pull request if you want to develop this 'repeat' mechanism but I'm not too sure how the getString('xxx') would work and it has to work with CSV and other delimited formats too.
Benoit

Important Notice
This communication contains information that is considered
confidential and may also be privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) please note that any form of distribution, copying or use of this communication or the
information in it is strictly prohibited and may be unlawful. If you have received this communication in error please return it to the sender and
delete the original

On 8 Aug 2016, at 17:15, Oblio<EMAIL_ADDRESS>wrote:
I have a situation where I'd like to parse a flat file without linebreaks that also contains "sub" arrays. Is there a way to specify that as a parser definition? (I may not have the right syntax for your PZMAP) For example, if I had a definition like this:

And a dataset like this (represents 2 records of 20 characters, pipes added for visibility):
|AliceTiny Fluff     Bob  Zap  Frog Wooly|
Would create:

Tiny
Fluff

Zap
Frog
Wooly

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.


fix(challenge): Stricter tests for "OOP: Iterate Over All Properties"

Pre-Submission Checklist

[x] Your pull request targets the staging branch of freeCodeCamp.
[x] Branch starts with either fix/, feature/, or translate/ (e.g. fix/signin-issue)
[x] You have only one commit (if not, squash them into one commit).
[x] All new and existing tests pass the command npm test. Use git commit --amend to amend any fixes.

Type of Change

[x] Small bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds new functionality)
[ ] Breaking change (fix or feature that would change existing functionality)
[ ] Add new translation (feature adding new translations)

Checklist:

[x] Tested changes locally.
[ ] Closes currently open issue (replace XXXX with an issue no): Progress towards #12966

Description

This is almost identical to #13131, read that description for more details.

LGTM!

LGTM!
pulled changes down and tested locally


Section URL category lookup incorrect with recent changes to the M365 security portal

With a recent change to the M365 Security Portal with searching moving from a blade under Endpoint to the title bar you can no longer pick what you want to search for (such as URL) and the results don't include a URL section. How do you accomplish this now?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: fbaaa90e-6975-ee29-335a-545cb6a55986
Version Independent ID: 75c93c17-4952-a13f-ed60-d2baefd0337e
Content: Web content filtering
Content Source: microsoft-365/security/defender-endpoint/web-content-filtering.md
Product: m365-security
Technology: mde
GitHub Login: @Dansimp
Microsoft Alias: dansimp

@philtapp Thank you for your feedback. I tested in multiple tenants and the feature to search URL  is still available

Thanks
Sri

Thanks. My portal experience has reverted back to the previous behaviour and matches what you are showing in the screenshot above. When I was having the issue the search blade was no longer present in the endpoint section and the search bar was up in the header bar with no dropdown. Let's close this and I'll open a new issue with some screenshots if it reoccurs.

The UI on my tenant is not showing "Endpoint > Search"
But there's now a search box at the top of the window. Enter the URL in question & select "Search as URL". Then on the right-side panel; select "Open url address page".
The category will then display on the page that loads.


side bar content chooser when activity bar is hidden

Can you replace side bar title with side bar content chooser when activity bar is hidden to leave possibility to switch, for example, from Explorer to SCM without hotkeys or menu.
thank you.

I understand the need for this, however it is not on our immediate plan to tackle this thus closing.
I personally use this setup and hotkeys do the job for me.
I do not like adding a mini activity bar if you have hidden the activity bar


Write docs on how to use caching and snapshots

I'm thinking about moving to readthedocs as it seems to be more flexible.

Docs available at https://php-v8.readthedocs.io/en/latest/getting-started/performance-tricks.html


Simplified wrappers for common use cases?

Any plans for creation of simplified wrappers for common transitions (e.g. adding items to list)?
Would be cool to have something similar to the <SimpleTransition /> I'd implemented in this demo, but better :smile:

Actually, before we get to wrappers, I might be able to release a simplified API for those things. Stay tuned tonight/tomorrow

@chenglou oh, sounds pretty awesome! can't wait to see it :+1:

Alright! @souporserious is experimenting with some simple wrappers here. I personally haven't used them yet but they look neat.
As for the simple wrappers, https://github.com/chenglou/react-motion/commit/372446fc8679dcdbf4de2e983d53709ab40ce88c exposes some commonly used spring constants. With these, I think something like:
<Spring endValue={{val: 10, config: wobbly}}>...

is short enough not to warrant actual wrapper components in core lib. Might change in the future if I spot some more complex use-cases (e.g. advanced TransitionSpring). But for now I'll close the issue =).

@chenglou looks great, thank you!


MPV not work Windows

Hello,
I have a problem with the MPV reader.
I added the path, but nothing happened.
I still have the same error
Thank's for your help

There were already several issues with advices on setting mpv in widnows, please search, maybe something will help.

Use /64bit/mpv-x86_64-20240317-git-3afcaeb.7z
v3 not working for me /64bit-v3/mpv-x86_64-v3-20240317-git-3afcaeb.7z


Switch to go mod

This commmit also update the kubernetes version.
Signed-off-by: Sebastian Sch<EMAIL_ADDRESS>

Hi @SchSeba thanks for this PR.,
I have following two questions, maybe we need input from other contributors.

Are we keeping the vendor/ and glide files for older version of Go? If so, then we may need some documentation for developer(in separate PR) when updating dependencies, we need to update both(for now, until we stop supporting vendoring altogether)

What are the implication of setting device plugin API to "k8s.io/kubelet v0.18.1"? I'm concerned of compatibility with older version of K8s(v1.13+)

Hi @ahalim-intel thanks for the comment here is my replay.

Are we keeping the vendor/ and glide files for older version of Go? If so, then we may need some documentation for developer(in separate PR) when updating dependencies, we need to update both(for now, until we stop supporting vendoring altogether)

I remove the glide files but keep the vendor folder.
I also add a new entry in the makefile deps-update that will run go mod tidy and also go mod vendor to update the vendor folder.
Not sure if this required any documentation change I search for glide in the documentation and wasn't able to find anything.

What are the implication of setting device plugin API to "k8s.io/kubelet v0.18.1"? I'm concerned of compatibility with older version of K8s(v1.13+)

This will continue to support the same versions as before because we are still using the v1beta1 API version.

@ahalim-intel can you please take another look on this PR?

/lgtm

/lgtm

@SchSeba when generating go.mod, did you manually remove those indirect dependencies or it is not generated at all?  In my local run, there are three indrect dependencies and I was using go1.12.17 with GO111MODULE set to auto.

@SchSeba when generating go.mod, did you manually remove those indirect dependencies or it is not generated at all? In my local run, there are three indrect dependencies and I was using go1.12.17 with GO111MODULE set to auto.

@zshi-redhat I did not remove anything from the go.mod file manually.
I am using 1.13 with GO111MODULE  set to on


Use col_max for format_commands

fixes #1842

Checklist:

[x] Add tests that demonstrate the correct behavior of the change. Tests should fail without the change.

[x] Add or update relevant docs, in the docs folder and in code.
I don't think all parameters for Context are documented (such as terminal_width, and this is more niche than that).

[x] Add an entry in CHANGES.rst summarizing the change and linking to the issue.

[x] Add .. versionchanged:: entries in any relevant code docs.

[x] Run pre-commit hooks and fix any issues.

[x] Run pytest and tox, no tests failed.

There are probably some other functions, not just format_commands, that should pipe this along too.

Thanks for working on this, however it's better served by solving #561. For now with #938, you can write a custom formatter subclass and set Context.formatter_class to use it.


Looking for a maintainer?

Package seems to be used widely but can't see a lot of commits in the past :pencil2:.
Would be happy to jump in, if you wish. :smirk:
Cheers,
Stefan

@cowboy  Hey, mate I can help. Feel free to contact me. 🦄


Postgres Bulk Update Array

SET
ADD
REMOVE

All extended Query to allow chaining.

This PR handles bulk update on array values of ticket - https://github.com/hypertrace/document-store/issues/82


changed vue syntax in readme, fixed broken ref to summernote

this.control.code() was returning "undefined is not a function". After looking at the summernote documentation this appears to be the solution.
There also seemed to be something wrong with the binding syntax, corrected in accordance with the vue documentation.

I'll update the libraries and get the tests to pass before resubmitting this. the Bugs I was having came as a result of using vue 1.0.16 and summernote 0.7.4


Reformat NetCDF output.

Restarting work on defining output NetCDF files.
Helene mentioned that there is a risk of data loss when using NCOs on NetCDF files larger than 12GB (she's pretty certain of that number)

Notes from working with Helene on this
Need output files from each stage. Not PR.
Dimension order. NetCDF examples tend to have time first (necessary, since it's unlimited), followed by other dimensions (in our case would be layer, pft, pftpart), and then y and x. There seems to be no specific NetCDF standard requirement for this, though, and since all variables have time,y,x, it's more logical to list those first.
Fill_Value is incredibly important! Pick something not at all feasible.
There are sometimes issues if you specify fill value and missing value and the values are not the same. Default to always specifying fill value
For some variables (e.g. soil carbon in unfrozen layers), compute on the run based on actual layer properties. Keep in mind that ALD would mess this up, as top layers may be frozen with thawed layers underneath (in the fall especially)

re: dimension order, I agree that there is not NetCDF specific standard for
order, but the CF conventions call for (time, z, y, x), and recommend that
if there are more dimensions, they should be added to the left:
http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#dimensions

Ah, so they do (of course the COARDS standards has them completely opposite...). It looks like that's because they expect longitude to be the fastest changing. No reason not to stick with that, I think. I'll get those switched around.

Thanks Tobey for checking this. Having the time, y,x as first dimensions
and the others like PFT,layer and compartment following those makes lots
of sense.
Helene

Hélène Genet, PhD
Institute of Arctic Biology
University of Alaska Fairbanks
902 Koyukuk Drive
Fairbanks, AK 99775 - USA
Phone: +1<PHONE_NUMBER>
Email<EMAIL_ADDRESS>Skype: helenegenet
On 1/8/17 7:32 AM, tobey wrote:

re: dimension order, I agree that there is not NetCDF specific
standard for
order, but the CF conventions call for (time, z, y, x), and recommend that
if there are more dimensions, they should be added to the left:
http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#dimensions
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/ua-snap/dvm-dos-tem/issues/44#issuecomment-271161784,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABtUl_qtT-0cFyNBLEYL4HMCOeC35ko0ks5rQQ-LgaJpZM4B0hQu.

Wait, just to make sure we are on the same page here: I am seeing both the
COARDS and CF standards calling for (time, z, y, x). The CF standards are
an extension of the COARDS standards, so I would be surprised if they
disagreed here.
CF:
http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#dimensions
COARDS: http://ferret.wrc.noaa.gov/noaa_coop/coop_cdf_profile.html
Also both standards indicate that any further dimensions should be to the
left of the spatiotemporal dimensions, so something like this for PFT
specific GPP:
float gpp(pft, time, y, x);
And this might be a definition for a per-pft, per compartment variable:
float VegC(compartment, pft, time, y, x);
I guess for a soil variable, we should use the depth dimension, so
something like this:
float temperature(time, depth, y, x);
Does this match what you both have been thinking, or am I reading those
documents incorrectly?

Tobey

Those examples show what I understood from the first link (cfconventions dimensions). I followed the link from there to
http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#coards-relationship
which indicates an ordering of lon, lat, vertical, and time. It does seem odd that CF and COARDS wouldn't agree, so perhaps I'm not reading it right?

Huh, that is strange. I agree, in the link you just sent describing the relationship between COARDS and CF, it says that the old COARDS standard was "lon, lat, vertical, time", but if you go to the COARDS webpage, they specify "time, height, lat, lon"...

Going back to one of your first comments, re: time necessarily being first due to it being unlimited, I am not really sure how to handle that in cases where we have more than 4 dimensions?? Any idea?

I didn't had time to read through the documentation yet, but one thing
we might have to consider as well is the unlimited nature of some of
those dimensions. Unlimited dimensions need to be listed first. But the
thing is that these unlimited dimensions might change depending on if we
run the time loop or the space loop first...
Helene

Hélène Genet, PhD
Institute of Arctic Biology
University of Alaska Fairbanks
902 Koyukuk Drive
Fairbanks, AK 99775 - USA
Phone: +1<PHONE_NUMBER>
Email<EMAIL_ADDRESS>Skype: helenegenet
On 1/9/17 12:10 PM, tobey wrote:

Huh, that is strange. I agree, in the link you just sent describing
the relationship between COARDS and CF, it says that the old COARDS
standard was "lon, lat, vertical, time", but if you go to the COARDS
webpage, they specify "time, height, lat, lon"...
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/ua-snap/dvm-dos-tem/issues/44#issuecomment-271408767,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABtUl84wfvEhSxT_N89aqE1tAar0TVIXks5rQqJdgaJpZM4B0hQu.

Most of the points in this thread are addressed by PR #257 . The rest have been moved to Issue #258


BLOCKING: Data is not saved when VC has speech functions in dataVC

QUERY FAILS.

Referenced for #11