Add support for font ligatures

Fira Code and others are monospaced fonts that use ligatures to combine multi-character symbols, e.g. -> will result in an arrow symbol →. This is quite nice and atom supports this font feature since 1.2.0, so it should work with vscode, too.

:+1:

+1

:+1: I would really like this feature.

I'm looking into this with the team and will keep you guys updated :).  Thanks for the idea!

+1

:+1:

Good News
I got ligatures to work by changing my user settings in VS Code with "editor.fontFamily": "Fira Code" and then adding the simple CSS rule below
div.monaco-editor.vs-dark {
text-rendering: optimizeLegibility;
}

Temporary Bad News
There's a side-effect we need to work through.

The problem I’m getting is that when I use this rule I get a glitch when clicking at the end of a line.  Clicking at the end of a line past the last character normally would put the cursor after the last cursor on the line, but as you can see, it’s putting my cursor at the beginning of the line.  However, when I enter text, it is appearing at the end of the line as it should (except the cursor is not in the right spot).
Atom had a similar problem in September for a while, when they enabled ligatures.  Apparently ligatures can cause problems with character spacing, even though they are still multiple ASCII characters, they render differently and this can affect spacing.  Perhaps this anchor on the page is relevant. @alexandrudima could you help with this?
By the way, this is only a problem if you click past the last character of any line (as if it is making the cursor position undefined???).  Anywhere else it’s fine.

How can I add this CSS rule ?

With that CSS style, the editing cursor renders in an incorrect position when moved to the end of a line. This happens regardless of whether it was placed there with the mouse or the keyboard.

I put up a hacky pull request with a workaround, but if you're like me, and want those sweet sweet ligatures right now, you can cram the fix into existing versions of VS Code.
For example, on OSX:

Close VS Code.

Enable ligature rendering by opening /Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.css and adding the following to the end:

.monaco-editor {
        text-rendering: optimizeLegibility;
}

Now you have ligatures and a broken editing cursor.

Apply the workaround to the broken cursor position by opening Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js and changing:

u.setStart(h,n),u.setEnd(p,r);var f=u.getClientRects(),m=null;return f.length>0&&(m=this._createRawVisibleRangesFromClientRects(f,o,s,a)),m

to
u.setStart(h,n),u.setEnd(p,r);var f=u.getClientRects(),m=null;return f.length>0&&(m=this._createRawVisibleRangesFromClientRects(f,o,s,a)),(h === p && n === r && r === p.textContent.length) ? (m[0].left = Math.max(0, p.parentElement.getBoundingClientRect().right - a)) : 0,m

Of course, the exact names used in this change depend entirely on the minification, so you may have to adjust accordingly if you have a different version (this example is from the latest version of VS Code for OSX, as of the time of writing). You can refer to the pull request above to get a better idea of what the (minified) change is doing.
If all went well, you now have some lovely ligatures, with a functioning editing cursor.

u.setStart(h,n),u.setEnd(p,r);var f=u.getClientRects(),m=null;return f.length>0&&(m=this._createRawVisibleRangesFromClientRects(f,o,s,a)),m

Does anybody have an idea where this line (above) is in the unminified source code?
All I can say is, that it's not in /src/vs/workbench/workbench.main.js.

The line you pasted is the minified version of the code here.
As mentioned in my earlier comment, the workaround is implemented in #1510. You can refer to #1510 to see the change in its un-minified form.

Will #1510 be compatible with the chromium version, which has fixed this ?
I mean, if Code will eventually update to the fixed chromium.....

What a great holiday present !!

Patch works as advertised here (Mac OSX, VSC 0.10.5)

BTW... I really don't want to bash Atom, it is a great editor.
But...
Opening the minified .js from this patch inside 'Visual Studio Code.app' and searching for that replacement + replacing it worked without a flaw in VSCode.
That same operation crashed Atom (V 1.3.2) unfortunately.
(had other crashes in Atom recently too, so it's not just a problem of minified js I guess)

The way I did it without having to fix the editor itself was to edit workbench.main.css and add the text-rendering: optimizeLegibility; to a new selector: token.keyword.operator - if you use Code's Developer Tools, you can see that these are defined per-language/token type (for instance, the span surrounding != in my Javascript file has the class token comparison keyword operator js).
This works, and avoids the broken line-selection in the editor, but there's a small tokenizing bug in JS: !== is not a single span, but rather 2:
!==
and this prevents the ligature from matching.

@clojj Glad you're enjoying it :smile:. I just discovered these ligature fonts recently, and also had to have them, heh. I believe #1510 would continue to work with the newer Chromium as it does with the current version. It doesn't add an offset to correct the bad coordinate; it outright replaces it with a new value calculated via other means (the element client box). The code would be wasteful (and probably confusing, heh) with newer Chromium, but shouldn't behave differently.
@jstclair That's an interesting approach as well, trying to select a subset of elements that need the style but won't appear in locations that break the cursor. Unfortunately, only applying it to operators does still produce some odd behavior; if operators appear at the end of a line, the cursor still breaks. e.g. in the following code, moving the cursor to the right side of the && makes it render in incorrect locations.
result = something &&
         somethingElse &&
         lastThing;

Additionally, applying it only to the operator class won't render ligatures in anything but operators; ligatures won't be displayed in other JS elements (comments, hexadecimal prefixes, etc.), nor will they appear in other languages (HTML, C#, etc.)

Just wanted to update everyone.  This feature will be supported soon in the main branch.  We are planning a temporary fix (based on the one above) while we wait for the fixed Chromium v48 to trickle down through an Electron Shell update soon.

IMHO the challenge here is that the implementation of text-rendering: optimizeLegibility in Chromium is buggy. Here's some random excerpt I've found at MDN -- besides the client rects being wrong -- Chromium 487028:

[1] The implementation has known bugs on Windows and Linux, which can break font substitition, small-caps, letter-spacing or cause text to overlap.

So while PR #1510 includes a workaround for Chromium 487028 that works for LTR text, I cannot deliver this change in its current form -- making text-rendering: optimizeLegibility the default for everybody. The unresolved Chromium issues above look like they would cause a lot of issues to CJK text, RTL text, and bi-di.
I am looking into enabling text-rendering: optimizeLegibility via a setting which will be off by default.

Kudos to @mattblagden for his PR #1510 that I used as a great inspiration source. :heart:

@alexandrudima: You can use the font-variant-ligatures: common-ligatures; CSS property instead. It works for me in Chrome with the Hasklig font. See: http://thenewcode.com/547/Make-Your-Web-Pages-More-Legible-With-Ligatures-and-Kerning-Pairs

@sanmai-NL We ended up with:
.monaco-editor.enable-ligatures {
	-webkit-font-feature-settings: "liga" on, "calt" on;
	font-feature-settings: "liga" on, "calt" on;
}

Let's revisit these and use font-variant-ligatures: common-ligatures if any issues pop up.

An alternative to ligatures is to use prettify-symbols-mode, which I've implemented as a plugin for the insider's build of vscode (but note that it is of beta quality right now).

I will just leave this here - enable font ligatures in settings with:
"editor.fontLigatures": true

(and also using appropriate font)

@PunchyRascal is correct.  Here are all the settings together.
make sure your system has FiraCode font and add change the following settings
{
    "editor.fontFamily": "FiraCode-Retina",
    "editor.fontLigatures": true
}

FiraCode is an example.. there are other fonts that support Ligatures now

It's diferent on the current version.
{
    "editor.fontFamily": "Fira Code",
    "editor.fontSize": 18,
    "editor.fontLigatures": true
}


a task with idea in the name will not be created

If a new task (type: RunIdeTask) has idea in the name it's not created.
For example, next task creation will be silently ignored:
task runIdea_192_5728(type: RunIdeTask){...}

works fine for me

Hmmm, very strange. I still not able to create any *Idea* task (but no problems creating *Iddea*):

Maybe it's because I create new tasks inside subprojects{}?...
Could you please try to create *Idea* task at https://github.com/ArtsiomCh/CMake/blob/master/build.gradle ?

I'm pretty sure the task is created and you can run it.
Regarding its representation in IDEA Gradle tool window, please file a bug report at https://youtrack.jetbrains.com.


chore: bump version of nbconvert

Bump version of Elyra dependency "nbconvert"
How Has This Been Tested?
Run the terminal on the image built for this PR. Check the version of nbconvert.
Merge criteria:

[x] The commits are squashed in a cohesive manner and have meaningful messages.
[x] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[ ] The developer has manually tested the changes and verified that the changes work

Can we refresh the pipfile.lock files for the notebooks utilizing pipenv by executing pipenv lock?

Thank you, Rama, for addressing the Pipfile.lock as well.
/lgtm

Hi @rkpattnaik780 could you take a look and rebase this PR, please?

/cherrypick 2023b


Animation wrong direction when stepping back in DotStepper.activeStep controlled Widget

When stepping backwards (i.e. activeStep--) the animation goes in the wrong direction.
See screen recording:

Hi Elliot,
Sorry for the inconvenience caused. Unfortunately, I am not able to replicate the issue from my end. Please see a screenshot of my pubspec.yml file below. It would be helpful if you could provide more information for me to replicate the issue.

Also, here is a recording, when I tried to replicate the issue:-

Please find the code below or clone this repo.
import 'package:flutter/material.dart';

import 'stepper.dart';

void main() {
  runApp(IconStepperDemo());
}

class IconStepperDemo extends StatefulWidget {
  @override
  _IconStepperDemo createState() => _IconStepperDemo();
}

class _IconStepperDemo extends State<IconStepperDemo> {
  // THE FOLLOWING TWO VARIABLES ARE REQUIRED TO CONTROL THE STEPPER.
  // Controls the currently active step. Can be set to any valid value i.e., a value that ranges from 0 to upperBound.
  int activeStep = 0; // Initial step set to 5.

  int lowerBound = 0;
  int upperBound = 9;

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      debugShowCheckedModeBanner: false,
      home: Scaffold(
        appBar: AppBar(
          title: Text('IconStepper Example'),
        ),
        body: Padding(
          padding: const EdgeInsets.all(8.0),
          child: Column(
            children: [
              ProgressStepper(
                stepReached: activeStep,
              ),
              Row(
                mainAxisAlignment: MainAxisAlignment.spaceBetween,
                children: [
                  previousButton(),
                  nextButton(),
                ],
              )
            ],
          ),
        ),
      ),
    );
  }

  /// Returns the next button.
  Widget nextButton() {
    return ElevatedButton(
      onPressed: () {
        if (activeStep < upperBound) {
          setState(() {
            activeStep++;
          });
        } else {
          print('No more NEXT steps.');
        }
      },
      child: Text('Next'),
    );
  }

  /// Returns the previous button.
  Widget previousButton() {
    return ElevatedButton(
      onPressed: () {
        if (activeStep > lowerBound) {
          setState(() {
            activeStep--;
          });
        } else {
          print('No more PREVIOUS steps');
        }
      },
      child: Text('Prev'),
    );
  }
}

Hi Elliot,
Sorry for the inconvenience caused. Unfortunately, I am not able to replicate the issue from my end. Please see a screenshot of my pubspec.yml file below.
It would be helpful if you could provide more information for me to replicate the issue.

Also, here is a recording, when I tried to replicate the issue:-

Please find the code below or clone this repo.
import 'package:flutter/material.dart';

/// Don't forget to import the stepper.dart class.
import 'package:im_stepper/stepper.dart';

void main() {
  runApp(DotStepperDemo());
}

class DotStepperDemo extends StatefulWidget {
  @override
  _DotStepperDemo createState() => _DotStepperDemo();
}

class _DotStepperDemo extends State<DotStepperDemo> {
  // MUST DEFINE THE FOLLOWING THREE VARIABLES.
  // activeStep must be Set to 1 and NOT to 0
  int activeStep = 1;

  int lowerBound = 0;
  int upperBound = 0;

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: Scaffold(
        appBar: AppBar(
          title: Text('DotStepper Example'),
        ),
        body: Padding(
          padding: const EdgeInsets.all(8.0),
          child: Column(
            children: [
              Row(
                children: [
                  previousButton(),
                  Expanded(
                    child: DotStepper(
                      dotCount: 5,
                      activeStep: activeStep,
                      lowerBound: (bound) => lowerBound = bound,
                      upperBound: (bound) => upperBound = bound,
                      indicatorEffect: IndicatorEffect.jump,
                    ),
                  ),
                  nextButton(),
                ],
              ),
              Expanded(
                child: FittedBox(
                  child: Text(
                    '$activeStep',
                  ),
                ),
              ),
            ],
          ),
        ),
      ),
    );
  }

  Widget nextButton() {
    return ElevatedButton(
      child: Text('Next'),
      onPressed: () {
        // Constraint is a must.
        if (activeStep < upperBound) {
          setState(() {
            activeStep++;
          });
        }
      },
    );
  }

  Widget previousButton() {
    return ElevatedButton(
      child: Text('Previous'),
      onPressed: () {
        // Constraint is a must.
        if (activeStep > lowerBound) {
          setState(() {
            activeStep--;
          });
        }
      },
    );
  }
}

Warm Regards,
iMujtaba Nazki

Hi iMujtaba,
I have just tested that code on Android and iOS. It works perfectly on Android but check it out on iOS:

What do you think?
Elliot

Hi Elliot,
Just a second, please. Let me check it on iOS. Between, is it possible you could send me your WhatsApp no. on my email, just so we get to discuss things a little better, however, only if you're comfortable. I will delete this message shortly.
Thanks,
iMujtaba Nazki

Sent

Got it. Thanks. Now deleting these comments.

Fixed the unpredictable scrolling behavior on iOS devices.

Fixed the unpredictable scrolling behavior on iOS devices.


Unnecessary boxing of IndexOptions enum

Found another easy improvement:
There are plenty of calls to IndexOptions.CompareTo(IndexOptions.DOCS_AND_FREQS_AND_POSITIONS) in the code, such as this one:
if (outerInstance.fieldInfo.IndexOptions.CompareTo(IndexOptions.DOCS_AND_FREQS_AND_POSITIONS) < 0)
{
    // Positions were not indexed:
    return null;
}

The problem is that CompareTo receives an object, which leads to boxing the enum:

From a quick glance, this seems to be only happening for the IndexOptions enum:


Lastest Cleave.js breaks when component is server-side-rendered

this
target.document = opts.document || document;

from here causes
ReferenceError: document is not defined
    at Object.assign (./node_modules/cleave.js/dist/cleave-react.js:2618:45)
    at Object.getInitialState (./node_modules/cleave.js/dist/cleave-react.js:133:47)
    at new <anonymous> (./node_modules/cleave.js/dist/cleave-react.js:1316:55)
    at ./node_modules/react-dom/lib/ReactCompositeComponent.js:292:18
    at measureLifeCyclePerf (./node_modules/react-dom/lib/ReactCompositeComponent.js:73:12)
    at ReactCompositeComponentWrapper._constructComponentWithoutOwner (./node_modules/react-dom/lib/ReactCompositeComponent.js:291:16)
    at ReactCompositeComponentWrapper._constructComponent (./node_modules/react-dom/lib/ReactCompositeComponent.js:282:19)
    at ReactCompositeComponentWrapper.mountComponent (./node_modules/react-dom/lib/ReactCompositeComponent.js:185:21)
    at Object.mountComponent (./node_modules/react-dom/lib/ReactReconciler.js:43:35)
    at ReactDOMComponent.mountChildren (./node_modules/react-dom/lib/ReactMultiChild.js:234:44)
    at ReactDOMComponent._createContentMarkup (./node_modules/react-dom/lib/ReactDOMComponent.js:657:32)
    at ReactDOMComponent.mountComponent (./node_modules/react-dom/lib/ReactDOMComponent.js:524:29)
    at Object.mountComponent (./node_modules/react-dom/lib/ReactReconciler.js:43:35)
    at ReactDOMComponent.mountChildren (./node_modules/react-dom/lib/ReactMultiChild.js:234:44)
    at ReactDOMComponent._createContentMarkup (./node_modules/react-dom/lib/ReactDOMComponent.js:657:32)
    at ReactDOMComponent.mountComponent (./node_modules/react-dom/lib/ReactDOMComponent.js:524:29)

in my SSR node webserver process

PR released: https://github.com/nosir/cleave.js/releases/tag/v1.3.8
@anfedorov @Lavoaster @nika-begiashvili
Sorry guys, please let me know if it's fixed

NPM published too

Hi, I add cleave by yarn. But i don't know how to include cleave.min.js file into project. I add //= require cleave/dist/cleave.min.js in application.js file but it raise this error. Can you help me? Thank you so much!


SessionsList comp: Update to display link, attendees, end_date (duration??)

Got duration to display instead of end date, added other fields to each card


Can't build the container image during pnpm build

Summary
When building from the Dockerfile, the build ends with following errors:
[3/4] STEP 12/12: RUN pnpm build

><EMAIL_ADDRESS>prebuild /app
> pnpm run generate

><EMAIL_ADDRESS>generate /app
> graphql-codegen --config .graphqlrc.ts

[STARTED] Parse Configuration
[SUCCESS] Parse Configuration
[STARTED] Generate outputs
[STARTED] Generate to src/gql/
[STARTED] Load GraphQL schemas
error: unknown option '--config'
 ELIFECYCLE  Command failed with exit code 1.
 ELIFECYCLE  Command failed with exit code 1.

I have no idea about what could possibly cause this, I am not too familiar with web development, and what puzzled me is that I couldn't even find any reference to pnpm build.
Steps to reproduce
Checkout the storefront project, main branch, 57dfc2c52a76f010ff150060f2a73bbb32eb3d46 on 10th of Dec 2023.
Add context: . under build in the docker-compose.yml
Build the container image using docker-compose with correct build args.
A link to a reproduction repository
No response
Output from envinfo
No response

@mmiszy Hello, it looks like that this is your level of expertise.

I am having this same issue. I have followed the directions: cloned the repo, cp .env.example to .env, I have run pnpm i. When I create a docker image of this repo it fails with the same image above. I saw also that the .env file was being ignored by the .gitignore file and even changing this does not fix it. It has been 3 years since I worked with a saleor store and I am not sure what I could be doing wrong since the directions seem simple enough to follow.

@matejak Were you ever able to resolve this?

I have even specified these in the build variables section of the hub.docker system and the build still fails. I don't understand why the other 2 images will build correctly but this one does not.

Thank you for the pointer, I have set the build args incorrectly, and I got no complaints, however the build has failed in that weird way. With correct build arguments, it works :-)
I wouldn't say that the issue is fixed or invalid, as a small typo can lead one completely astray.
@jdmueller I guess that what you are after are build arguments, and not env variables. Env vars are overriden with build args in the Dockerfile. I am not familiar with the build system that you are using though.

@matejak I am using the hub.docker.com system that automatically generates an image file from my GitHub responistory whenever there is a push to the master. This builds a new image automatically and I can just restart the service and it automatically runs the new build. I have found this to work great for everything. And it works for the other 2 repos from saleor, but for some reason not this one. It even builds the image correctly without changing the .env file from .env.example. I just don't have enough experience with docker to understand what is going on or why it wouldn't even build the image. I haven't even tried to run it yet.

So I have created a new issue that specifies my problem since it sounds like the issue should be closed as you were able to figure out your problem. My issue is more specific to hub.docker.com.

Well, I close this as my issue is indeed resolved.
Anybody looking for the solution - pay attention to contents of your build arguments, make sure that they point to where the README says they should, and that those URLs are available from your build environment.


Validate against extra fields being supplied

Hi there,
I've just come across this module and it seems quite useful, a lot simpler to use than other similar modules. I have a schema that looks like this:
register_user_schema = {
    'type': 'object',
    'properties': {
        'userId': {'type': 'string'},
        'name': {'type': 'string'}
    },
    'required': ['userId']
}

Is it possible to throw a validation when extra parameters are being supplied in the body? For example:
POST {
  "userId": "1234",
  "somerandomfield": "blah"
}

This doesn't throw a validation for me even though, somerandomfield isn't in the schema. Is there a way to make it throw an ValidationError?

Hi rtnolan, this library is based on JSON Schema. More specifically it is based on the python implementation of the spec: jsonschema.
You can find the spec here and the documentation for the python package here.
As for extra properties, you can use the flag additionalProperties. Set it to False to disallow them.
In your schema:
register_user_schema = {
    'type': 'object',
    'properties': {
        'userId': {'type': 'string'},
        'name': {'type': 'string'}
    },
    'required': ['userId'],
    'additionalProperties': False
}

@Fischerfredl thanks for the great reply and fantastic software :)


Database opening slow with large DB

A large keepass DB, about 14 MB, now takes almost two minutes to open in keepassc, even on a fast computer (i7-4790K). The same DB opens instantly with KeePass (Linux and Windows). When checking with top it is obvious that one core is loaded a lot doing some python stuff during the opening of the DB.

I will later look into that but an educated guess would be that PyCrypto (the used crypto library) is the bottleneck.

PyCrypto seems to be dead https://github.com/dlitz/pycrypto/issues/173
And is dangerously buggy https://github.com/dlitz/pycrypto/issues/176
Best would be to replace with something else. Maybe https://github.com/Legrandin/pycryptodome? Claims to be fork and replacement for PyCrypto.

Thanks for the suggestion, I changed it to PyCryptodome

It looks like it is faster with pycryptodomex, about half a minute to open the large 14 MB DB (I haven't timed it precisely). But I made the DB smaller (about 2 MB) by removing old stuff, so now it is not a big problem anymore (opens in a second or two).


Feature/query multiple tables

Dear Reviewer,
Please review the changes made and the queries implemented. Your feedback is greatly appreciated.
Project Requirements

Created owners table with columns id, full_name, and age.
Created species table with columns id and name.
Modified animals table to include species_id and owner_id columns.
Inserted provided data into owners and species tables.
Updated animals with appropriate species_id and owner_id.
Implemented all changes in schema.sql and data.sql files.

Screenshots of Queries
What animals belong to Melody Pond?

List of all animals that are pokemon (their type is Pokemon).

List all owners and their animals, including those without any animal.

How many animals are there per species?

List all Digimon owned by Jennifer Orwell.

List all animals owned by Dean Winchester that haven't tried to escape.

Who owns the most animals?

_Notes

Implemented the specified database schema and data modifications.
Ensured that queries produce accurate results according to requirements._

Thank you.
@Zafron047

@Munsa1 Fixed the broken link which had duplicate https in twitter url.

@tufoinnkuo10 Thanks for the review and approval.


Tracking Issue: Missing Redis 6.2 Commands

As part of extending support for redis in predis, this tracking issue will list unimplemented (as of yet) redis 6.2 commands. Working with this tracking issue is an experiment, to see if as a community we can leverage the same issue in from redis-py here, and if this works for us.
It is important to note, that these issues are in no way prioritized, they're merely a single point for tracking unimplemented commands associated with a specific version.

[x] #819
[x] #818
[x] #817
[x] #816
[x] #815
[x] #814
[x] #813
[x] #836
[x] #837
[x] #838
[x] #839
[x] #840
[x] #841
[x] #842
[x] #843
[x] #844
[x] #845
[x] #846
[x] #847
[x] #848
[x] #849
[x] #850
[x] #851
[x] #852
[x] #853
[x] #854
[x] #855
[x] #856
[x] #975
[x] #976
[ ] #979
[ ] #980
[x] #981
[x] #982
[x] #983
[x] #984
[x] #985
[x] #986
[x] #987
[x] #988
[x] #989
[x] #990
[x] #991
[ ] #992
[ ] #993
[x] #994
[x] #995
[x] #996
[x] #997
[x] #998
[x] #999
[x] #1000

For 2.2.1 -> 3.x


Fix using arrow keys to move object when move tool is selected causes changed of focus

Fixes #70789
What a goose chase this was. Turns out inside the _gui_input handling, we call _update_cursor(), which in turn pushes a mouse motion event to the viewport. This RESETS the gui.key_event_accepted to false, when the move tool already accepted the event and set it to true!
So the fix here is actually to ensure viewport->set_default_cursor_shape(c); (at the bottom of _update_cursor()) never gets called because the new cursor is same as current.
This might need to be looked into as this was quite a hard bug to find due to that interaction. Simplfied, this was happening:

_gui_input
handle move tool
move calls accept_event()
viewport sets gui.key_event_accepted to true
move calls update_cursor()
update_cursor() dispatches an event to the viewport via viewport->set_default_cursor_shape(c);
viewport sets gui.key_event_accepted to false
handling move tool finished
viewport checks gui.key_event_accepted - it is now false! When it should be true since move tool handled the event

I believe that a better way would be to get rid of the set_default_cursor_shape call.
Please see #59081

Yes you are probably right


An empirical study of just-in-time defect prediction using cross-project models

paper link:http://dl.acm.org/ft_gateway.cfm?id=2597075&ftid=1467103&dwn=1&CFID=626469285&CFTOKEN=55193119
dataset: please go to see this paper A large-scale empirical study of just-in-time quality assurance

Too big.
Mozilla and Bugzilla: http://msr.uwaterloo.ca/msr2007/challenge/index.html - see Data section.
Eclipse JDT and Eclipse Platform data: - http://msr.uwaterloo.ca/msr2008/challenge/index.html - see Data section.
Columba: https://hal.archives-ouvertes.fr/hal-00769121/file/cvs-vintage-light-v20120509.tar.bz2
PostgreSQL - not sure.

This data is important enough for us to collect.  So I'm culling the "too big" tag and going back to "collect"

As twinitialize said, Mozilla and Bugzilla can be obtained from 2007 MSR:
http://msr.uwaterloo.ca/msr2007/challenge/index.html
Eclipse JDT and Eclipse Platform data comes from 2008 MSR:
http://msr.uwaterloo.ca/msr2008/challenge/index.html
Columbia can be obtained at
https://hal.archives-ouvertes.fr/hal-00769121/file/cvs-vintage-light-v20120509.tar.bz2
or at
rsync://columba.cvs.sourceforge.net/cvsroot/columba/
PostgreSQL is supposedly available through rsync, though I couldn't get a response:
rsync://anoncvs.postgresql.org/cvsroot/pgsql/
The paper also says that the authors collected data from Gimp, Maven-2, Perl, Ruby on Rails, and Rhino. These are open-source projects, so data could be re-collected, but the authors did not provide datasets.

1. Data source
An Empirical Study of Just-in-Time Defect Prediction using Cross-Project Models
2. Link to the material associated with the dataset (if available)
Examples:
http://posl.ait.kyushu-u.ac.jp/~kamei/publications/Fukushima_MSR2014.pdf
3. Attribution list for the material
Takafumi Fukushima, Yasutaka Kamei, Shane McIntosh, Kazuhiro Yamashita, and Naoyasu Ubayashi
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>4. BibTeX reference
@inproceedings{fukushima2014empirical,
title={An empirical study of just-in-time defect prediction using cross-project models},
author={Fukushima, Takafumi and Kamei, Yasutaka and McIntosh, Shane and Yamashita, Kazuhiro and Ubayashi, Naoyasu},
booktitle={Proceedings of the 11th Working Conference on Mining Software Repositories},
pages={172--181},
year={2014},
organization={ACM}
}
5. Link to the datasets
Eclipse CVS repository:
JDT: eclipse-cvs-jdt.tgz (1.10G, retrieved on 2006-11-25)
SWT: eclipse-cvs-swt.tgz (488M, retrieved on 2006-11-25)
Rest: eclipse-cvs-rest.tgz (1.82G, retrieved on 2006-11-25)
Eclipse Bugzilla export (in XML):
Bugs: 1-162656: eclipse-bugs.zip (125M, retrieved on 2006-10-30)
Firefox/Mozilla CVS repository: mozilla-cvs.tgz (609M, retrieved on 2006-12-17)
Firefox/Mozilla Bugzilla export (in XML):
Bugs 35-100000: mozilla-bugs-000001-100000.zip (886M, retrieved on 2007-01-06)
Bugs<PHONE_NUMBER>00: mozilla-bugs-100001-200000.zip (1.10G, retrieved on 2007-01-06)
Bugs<PHONE_NUMBER>00: mozilla-bugs-200001-300000.zip (1.00G, retrieved on 2007-01-06)
Bugs<PHONE_NUMBER>12: mozilla-bugs-300001-367500.zip (724M, retrieved on 2007-01-06)
Eclipse CVS repository in TARE format:
eclipse-tare.tgz (247M, retrieved by Tom Zimmermann on 2006-11-25)
Eclipse Bug Data (more details):
eclipse-bug-data-1.1a.zip (920K, retrieved by Adrian Schröter)
Eclipse CVS repository:
JDT: eclipse-cvs-jdt.tgz (1.1G, retrieved on 2007-12-19 by Thomas Zimmermann)
SWT: eclipse-cvs-swt.tgz (686M, retrieved on 2007-12-19 by Thomas Zimmermann)
Platform: eclipse-cvs-plarform.tgz (686M, retrieved on 2007-12-19 by Thomas Zimmermann)
Rest: eclipse-cvs-rest.tgz (1.7G, retrieved on 2007-12-19 by Thomas Zimmermann)
Eclipse Bugzilla export (in XML):
Bugs: 1-213000: eclipse-bugs--000001-213000.zip (3.2G, retrieved on 2007-12-19 by Thomas Zimmermann)
6. PROMISE repo category (effort, requirements, model, defect, etc.)
Others. It is a generic data set which can be categorized as bug analysis, change analysis, architecture and design, process analysis, team structure etc.
7. General overview of the data

8. Attribute info
Too big for me to check
9. Paper abstract (if appropriate)
Prior research suggests that predicting defect-inducing changes,
i.e., Just-In-Time (JIT) defect prediction is a more practical
alternative to traditional defect prediction techniques, providing
immediate feedback while design decisions are still
fresh in the minds of developers. Unfortunately, similar to
traditional defect prediction models, JIT models require a
large amount of training data, which is not available when
projects are in initial development phases. To address this
flaw in traditional defect prediction, prior work has proposed
cross-project models, i.e., models learned from older
projects with su!cient history. However, cross-project models
have not yet been explored in the context of JIT prediction.
Therefore, in this study, we empirically evaluate the
performance of JIT cross-project models. Through a case
study on 11 open source projects, we find that in a JIT
cross-project context: (1) high performance within-project
models rarely perform well; (2) models trained on projects
that have similar correlations between predictor and dependent
variables often perform well; and (3) ensemble learning
techniques that leverage historical data from several other
projects (e.g., voting experts) often perform well. Our findings
empirically confirm that JIT cross-project models learned
using other projects are a viable solution for projects with
little historical data. However, JIT cross-project models perform
best when the data used to learn them is carefully selected.
10. Is this dataset part of a larger series or collection?
No

https://www.dropbox.com/s/8npl5jlgfuwuryr/Screenshot 2015-04-03 15.48.11.png?dl=0

Added as jit. Data not uploaded.

Is this data the same as msr11? Most of the filenames are the same.

Added as jit.


[BUG] Admin permissions list display

A clear and concise description of what the issue is.
Selecting next refreshes back to the first page of the permission list.
How to reproduce the issue

Go to any App/Model/Database settings page
Click on next page of permission list

Expected results
View next page in list
Actual results
List refreshes to show only first page regardless of number of rows displayed
Screenshots

Environment
https://workshop.cfg.deloitte.com/cfg-ai-demo/SemossWeb/packages/client/dist/#/

#296 I tried to fix this same issue on that PR. Please review & let us know...


can't install

can't open JAR executable on windows 8.

describe the problem in more detail. how should you be helped?


Add Option to Disable Copying Code to Clipboard

Reported by gfactor on 16 Feb 2015 00:28 UTC
Can you provide an option to disable the automatic copying of OTP codes to the clipboard? I realize that the autocopying of OTP codes can be a great help for most users, but I usually don't use my OTP codes on my Android device. This makes the autocopying of codes a potential security hazard for me without any benefit.

Any update on this?


FoundationXML: convert CFXMLInterface to ImplementationOnly

The CoreFoundation interfaces are not expected to be exposed.  Convert
the import of CFXMLInterface to @_implementationOnly.

@swift-ci please test

@swift-ci please test


Backtracking some python 2.5 incompatible changes

In this branch I have withdrawn two changes I did which were incompatible with python 2.5:

2.5 doesn't support the decorator @xxx.setter on properties
2.5 doesn't support collections.MutableSequence

Perhaps it's time to create a new branch for the 2.5 compatible version. I have called this branch py25-compatible, but if you want another name, that's ok with me.

Thanks Peter!


Maybe it is related: It used to refer to URL using the local IP: https://192.168.1.XX/thumbs/etc.jpg but that dissapeared after I made a forced domain mapping (not sure if is valid, but log file now mentions the domain instead of the local IP)
ModPagespeedMapOriginDomain https://www.example.com https://www.example.com

Can it be an url rewrite issue ? The htaccess looks like this:
RewriteRule ^thumbs/(?:(?:.*)-)?([0-9]+)-([0-9]+).jpg$ photo_thumbs/$1_$2.jpg [QSA]

The apache log errros:
[Wed Nov 27 23:43:29.818887 2019] [pagespeed:warn] [pid 28114] [mod_pagespeed <IP_ADDRESS>-0 @28114] RateController: drop deferred fetch of https://www.example.com/thumbs/alfa-romeo-2015-1.jpg?1373456351 on shutdown
[Wed Nov 27 23:43:29.818938 2019] [pagespeed:warn] [pid 28114] [mod_pagespeed <IP_ADDRESS>-0 @28114] RateController: drop deferred fetch of https://www.example.com/thumbs/alfa-romeo-1-6-t-spark-6672-1.jpg?1376999878 on shutdown
[Wed Nov 27 23:43:29.818997 2019] [pagespeed:warn] [pid 28114] [mod_pagespeed <IP_ADDRESS>-0 @28114] RateController: drop deferred fetch of https://www.example.com/thumbs/alfa-romeo-147-4483-1.jpg?1374435125 on shutdown
[Wed Nov 27 23:43:29.819039 2019] [pagespeed:warn] [pid 28114] [mod_pagespeed <IP_ADDRESS>-0 @28114] RateController: drop deferred fetch of https://www.example.com/thumbs/alfa-romeo-gt-distinctive-1-9-jtd-2004-1913-1.jpg?1373391691 on shutdown
[Wed Nov 27 23:43:29.819082 2019] [pagespeed:warn] [pid 28114] [mod_pagespeed <IP_ADDRESS>-0 @28114] RateController: drop deferred fetch of https://www.example.com/thumbs/alfa-romeo-156-1861-1.jpg?1373379761 on shutdown
[Wed Nov 27 23:43:29.819126 2019] [pagespeed:warn] [pid 28114] [mod_pagespeed <IP_ADDRESS>-0 @28114] RateController: drop deferred fetch of https://www.example.com/thumbs/vand-urgent-alfa-romeo-147-6824-1.jpg?1376389267 on shutdown
[Wed Nov 27 23:43:29.819181 2019] [pagespeed:warn] [pid 28114] [mod_pagespeed <IP_ADDRESS>-0 @28114] RateController: drop deferred fetch of https://www.example.com/thumbs/alfa-romeo-156-7415-1.jpg?1375606615 on shutdown
[Wed Nov 27 23:43:29.819232 2019] [pagespeed:warn] [pid 28114] [mod_pagespeed <IP_ADDRESS>-0 @28114] RateController: drop deferred fetch of https://www.example.com/thumbs/alfa-romeo-5837-1.jpg?1375002874 on shutdown

It's not an issue but a feature I guess; the module throttles the amount of parallel resource fetches to avoid overwhelming the server. See https://www.modpagespeed.com/doc/system#rate_limit_background_fetches


Knowledge map for average user does not work

It shows knowledge map for the given user.

Fixed via https://github.com/adaptive-learning/geography/commit/f57e69b91b8fb04129ee00b0ac810f12296e3e88


Turn on alerting in load test for pod latency and scheduling throughput

…ing throughput below 100

What type of PR is this?

/kind feature
What this PR does / why we need it:
We are in a process of merging density test into load test. Based on data gathered in perfdash for 100 and 5k nodes runs, we are comfortable with turning on alerting for SchedulingThroughput and PodStartupLatency.
Metrics:
100 Nodes:
Pod startup latency:
http://perf-dash.k8s.io/#/?jobname=gce-100Nodes-master&metriccategoryname=E2E&metricname=LoadPodStartup&Metric=pod_startup
Scheduling throughput:
http://perf-dash.k8s.io/#/?jobname=gce-100Nodes-master&metriccategoryname=Scheduler&metricname=SchedulingThroughput&TestName=load
5k Nodes:
Pod startup latency:
http://perf-dash.k8s.io/#/?jobname=gce-5000Nodes&metriccategoryname=E2E&metricname=LoadPodStartup&Metric=pod_startup
Scheduling throughput:
http://perf-dash.k8s.io/#/?jobname=gce-5000Nodes&metriccategoryname=Scheduler&metricname=SchedulingThroughput&TestName=load
The difference between scheduling throughput in density test and load test comes from interval (1s vs 5s) and burstQPS.
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

/assign @mborsz

/lgtm
/approve

/ok-to-test


Run jazzy without bundler

Goal
Fix the Update Docs action that started failing with release 6.18.0 (apparently due to a change in Gemfile.lock)
Changeset
Runs jazzy without Bundler.
Removes jazzy from Gemfile (it's pre-installed on GitHub's macOS images.)
Testing
See successful dry run: https://github.com/bugsnag/bugsnag-cocoa/runs/7095074190?check_suite_focus=true

I'm not opposed to removing bundler (generally the opposite), though it would make sense to still cache the gem to avoid extra work. This is still possible by caching the result of calling gem install (inside $GEM_HOME), but slightly more effort to grab the right paths than using bundler.
There's also the potential of future breakages between the ruby version used and the latest version of jazzy, which is harder to track without a Gemfile.lock pinning versions.
In this particular case it seems like preventing arbitrary updates to the bundler version is ideal.

Now uses the pre-installed version of jazzy provided by Github


How to add new certificates into the existing keyvault extension ,Uninstalling of keyvault extension is not deleting the certificates installed on the VMs

Hi Team,
Can you tell me how to add new certificates under existing keyvault extension in VMSS.
Uninstall of the keyvault extension is not deleting the certificates in the Vm.
pollingIntervalInS =3600 mean extension will check for change in keyvault certificate for every hour?
When there is a change in the keyvault certificate it is not overriding the existingcertificiate ,it is installing as new certificate.
Thanks.

@Bangarraju1005
Thanks for the Feedback.
Could you provide the URL of the specific documentation that you are following?

I am following the below document
https://docs.microsoft.com/en-us/azure/virtual-machines/extensions/key-vault-windows

@Bangarraju1005
Azure Key Vault will not replace the old cert that is not a feature, as a work around you can use custom script and periodically delete old certs.
Yes, pollingIntervalInS =3600 is for checking the changes for every 1 hour.
Also, as confirmed by Azure support it is currently not possible to add new certificate into existing key vault through VMSS extension. So you need to uninstall and re-install with all the old and new certificates again.
We will now close this issue. If there are further questions regarding this, please update the support case that is opened for this issue.


Fixing documentation in Glossary and Links in Examples and Addressing Issue #66

Hi, was reading the glossary and found a small typo. Fixed it.

Some links and description pages in Krotov.jl  have to be modified as well.

https://juliaquantumcontrol.github.io/QuantumControlExamples.jl/stable/tutorials/krotov_pulse_**parametrization**/#Pulse-Parametrization-for-Krotov's-Method

https://juliaquantumcontrol.github.io/QuantumControlExamples.jl/stable/tutorials/krotov_pulse_**parametrization**/

I will change the spelling in the link to parameterization in Example Page of QuantumControl.jl for Pulse Parameterization example for Krotov Method. Later, after this PR, we can modify thekrotov.jlso that everything is consistent in all places.
https://juliaquantumcontrol.github.io/QuantumControlExamples.jl/stable/tutorials/krotov_pulse_**parameterization**/)

Sorry for another workflow run! Now, I made the spelling in the links leading to krotov.jl examples consistent as well.
The pulse_parameterization and other tests except CI doctype all passed in first workflow run though.

The CI doctype test failing because of extra line. Fixed it.
Also, improved the documentation in QuantumControlExamples.jl as well: https://github.com/JuliaQuantumControl/QuantumControlExamples.jl/pull/1

Cool, thanks! Since this is a breaking change, I'll want to let it sit for a bit to accumulate some other breaking changes. In particular, I'm organizing a workshop in two weeks that will feature the QuantumControl package, and I'm not going to merge anything breaking before that.
I'll have a look at these PRs afterward.

Hi Michael,
Is this codestyle error simply occuring because we normalized the spelling of one word? In the main, src/ file, the evaluate function used which is same as shown via +, but the diff assumes it's the - minus one.
https://github.com/JuliaQuantumControl/QuantumControl.jl/blob/1f575a22de150dd8e4b8528998013630cbb73a46/src/pulse_parametrizations.jl#L406-L411
diff --git a/src/pulse_parameterizations.jl b/src/pulse_parameterizations.jl
│ index 09f5f1f..ebacbdf 100644
│ --- a/src/pulse_parameterizations.jl
│ +++ b/src/pulse_parameterizations.jl
│ @@ -400 +400,6 @@ end
│ -function evaluate(deriv::ShapedParameterizationPulseDerivative, tlist, n; vals_dict=IdDict())
│ +function evaluate(
│ +    deriv::ShapedParameterizationPulseDerivative,
│ +    tlist,
│ +    n;
│ +    vals_dict=IdDict()
│ +)
└ @ Main ~/work/_temp/[24](https://github.com/JuliaQuantumControl/QuantumControl.jl/actions/runs/9052722822/job/24928610529?pr=68#step:6:25)a1a795-44c6-4977-b0c9-9b0c8d967774:5
Error: Process completed with exit code 1.

Is this codestyle error simply occuring because we normalized the spelling of one word?

Sure, there's a maximum line length, so if you add a letter to word that can push it over the limit, and the line has to be broken into multiple lines

if you add a letter to word that can push it over the limit, and the line has to be broken into multiple lines

I see, we can change the maximum line length by a couple letters so that the doc-style test passes?
I checked for any potential whitespaces, even used the same src file present in main, just changed ..ri.. to ..eri.. to be sure that there was not any potential whitespaces.

we can change the maximum line length

No, definitely not. Use make codestyle (or the equivalent command in the Julia REPL) to reformat the code to adhere to the required style.

Thanks! I'll explore make codestyle or alternative commands.

Applied make codestyle! Sorry, I didn't knew about the make stuff before!
I will now usetest()to see to make sure that all the tests pass!
julia --project=test -e 'using JuliaFormatter; format(".", verbose=true)'
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/devrepl.jl
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/docs/generate_api.jl
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/docs/make.jl
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/ext/QuantumControlFiniteDifferencesExt.jl
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/ext/QuantumControlZygoteExt.jl
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/src/QuantumControl.jl
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/src/deprecate.jl
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/src/functionals.jl
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/src/print_versions.jl
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/src/pulse_parameterizations.jl
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/src/reexport.jl
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/src/set_default_ad_framework.jl
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/src/workflows.jl
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/test/clean.jl
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/test/init.jl
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/test/runtests.jl
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/test/test_functionals.jl
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/test/test_optimize_or_load.jl
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/test/test_pulse_parameterizations.jl
Formatting /home/Desktop/New/qc/4/JuliaQuantumControl/QuantumControl.jl/test/test_run_or_load.jl

Test Results
Test Summary:           | Pass  Total  Time
QuantumControl versions |    3      3  1.9s

* Functionals (test_functionals.jl):
189.185000 seconds (73.64 M allocations: 4.808 GiB, 1.38% gc time, 46.03% compilation time: <1% of which was recompilation)

* Run-or-load (test_run_or_load.jl):
134.203390 seconds (21.57 M allocations: 1.472 GiB, 0.71% gc time, 38.25% compilation time: 41% of which was recompilation)

* Optimize-or-load (test_optimize_or_load.jl):

 17.979704 seconds (23.57 M allocations: 1.657 GiB, 5.28% gc time, 99.17% compilation time: 9% of which was recompilation)

* Pulse Parameterizations (test_pulse_parameterizations.jl):
 26.718649 seconds (23.18 M allocations: 1.617 GiB, 3.75% gc time, 84.87% compilation time: 14% of which was recompilation)

Test Summary:             | Pass  Total     Time
QuantumControl            |   96     96  6m08.3s
  Functionals             |   19     19  3m09.2s
  Run-or-load             |    5      5  2m14.2s
  Optimize-or-load        |   25     25    18.0s
  Pulse Parameterizations |   47     47    26.7s
368.754314 seconds (142.45 M allocations: 9.587 GiB, 1.49% gc time, 48.70% compilation time: 15% of which was recompilation)
┌───────────────────────────────────────────┬────────┬────────┬────────┬──────────┐
│ File name                                 │  Total │    Hit │ Missed │ Coverage │
├───────────────────────────────────────────┼────────┼────────┼────────┼──────────┤
│ ext/QuantumControlFiniteDifferencesExt.jl │     13 │     13 │      0 │     100% │
│ ext/QuantumControlZygoteExt.jl            │     12 │     12 │      0 │     100% │
│ src/QuantumControl.jl                     │      6 │      6 │      0 │     100% │
│ src/deprecate.jl                          │      0 │      0 │      0 │        - │
│ src/functionals.jl                        │     99 │     90 │      9 │      91% │
│ src/print_versions.jl                     │     15 │     14 │      1 │      93% │
│ src/pulse_parameterizations.jl            │    129 │     88 │     41 │      68% │
│ src/reexport.jl                           │      7 │      7 │      0 │     100% │
│ src/set_default_ad_framework.jl           │      7 │      6 │      1 │      86% │
│ src/workflows.jl                          │    151 │    135 │     16 │      89% │
├───────────────────────────────────────────┼────────┼────────┼────────┼──────────┤
│ TOTAL                                     │    439 │    371 │     68 │      85% │
└───────────────────────────────────────────┴────────┴────────┴────────┴──────────┘
nothing


RKE2 clusters - Snapshots are not available on the UI for a cluster with cloud provider enabled

Rancher Server Setup

Rancher version: 44013dc
Installation option: Docker install

Information about the Cluster

Kubernetes version: v1.22.7+rke2r2
Cluster Type (Downstream):

If downstream, what type of cluster? (Custom/Imported or specify provider for Hosted/Infrastructure Provider): Custom

User Information

What is the role of the user logged in? Admin/Cluster

Describe the bug

Deploy an RKE2 cluster - AWS ec2 node driver with Cloud provider enabled on the cluster
take snapshots on the cluster
Navigate to Cluster Management --> cluster details --> Snapshots tab
No snapshots are listed
Snapshots are available when I ssh into the etcd node:

/var/lib/rancher/rke2/server/db/snapshots# ls -ltr
total 38268
-rw------- 1 root root 13058080 Mar 23 05:00 etcd-snapshot--1648011600
-rw------- 1 root root 13058080 Mar 23 05:06 on-demand-sowmya-cp-pool1-59523e22-blrvg-1648011980
-rw------- 1 root root 13058080 Mar 23 05:32 on-demand-sowmya-cp-pool1-59523e22-blrvg-1648013523
Note:
On a DO Node driver cluster, the snapshots are available on the UI.
On an EC2 node driver with no Cloud provider enabled, snapshots are available on the UI.

To Reproduce

Deploy an RKE2 cluster - AWS ec2 node driver with Cloud provider enabled on the cluster
take snapshots on the cluster
Navigate to Cluster Management --> cluster details --> Snapshots tab

Result

No snapshots are listed

Expected Result

On an EC2 node driver with Cloud provider enabled, snapshots are listed

Screenshots

RKE2 Amazon EC2 With Cloud Provider

RKE2 Amazon EC2 No Cloud Provider

Additional context

On the frontend, we get a list of all snapshots via /v1/rke.cattle.io.etcdsnapshots, this only returns snapshots for the No Cloud Provider cluster.
rancher/dashboard#5476

Filed https://github.com/k3s-io/k3s/issues/5418 as this is causing our problems

This is currently blocked by https://github.com/k3s-io/k3s/issues/5418

Summary of discovered issue:
K3s/RKE2 create an %s-etcd-snapshots configmap in the kube-system namespace that is used to store the metadata of snapshots taken by the supervisor processes on the node. Within this configmap, there is a field on each snapshot that is nodeName.
Rancher has a controller called the snapshotbackpopulate controller that is responsible for watching the %s-etcd-snapshots configmap on the downstream cluster, and will proceed to create corresponding etcdsnapshots.rke.cattle.io objects that are what the UI display/what Rancher refers to for taking snapshots.
The snapshotbackpopulate controller sets an owner reference on the snapshot objects it creates, depending on the type of snapshot (S3 or local). If the snapshot is an S3 snapshot, the cluster object owns the snapshot, so if the cluster is deleted, the snapshot is properly cleaned up. The second type of owner reference is the machine itself and is used for local snapshots -- the snapshot is owned by the machine. If the machine is deleted, the local snapshots are also deleted.
We use the nodeName heavily for local snapshot reconciliation. For example, we use it for the owner reference, but also for the purposes of individual object clean up. The snapshotbackpopulate controller will NOT delete etcdsnapshot objects, but rather it will update them to set a missing field, that the planner will use when it is reconciling the cluster, as the planner delivers a periodic instruction to the various nodes to run an etcd-snapshot list. If the snapshot is missing from both the configmap and the actual node, the etcdsnapshot object will be deleted from Rancher/the local cluster.
The issue we are running into is that the cloud provider enabled (aws in this case) is changing the node name, and RKE2/K3s are remaining "sticky" with the node name they are using to take snapshots. Once the node name is changed in K8s, we no longer have a reference to the node name that was originally set for the node, and cannot create the etcdsnapshot objects as we cannot cleanly provide an owner reference for them.
I'm thinking the best way to work around this issue without K3s/RKE2 changes would be to use the machine-id for correlation...

Okay I've been giving this some hard thought.
I think no matter what, we need to change from using nodeName for correlation of snapshots to machine-id. This should be relatively simple, as we are already retrieving machine based on nodeName.
What I will do is change behavior of the snapshotbackpopulate controller to be able to also simply backfill metadata, and change the plansecret controller to create snapshots based on the output of rke2/k3s etcd-snapshot ls. plansecret will know specifically what node is being worked on, and will only create etcdsnapshot objects in the event that they don't already exist for the machine in question.

Test Environment:
Rancher version: v2.6-head a478def
Rancher cluster type: HA
Docker version: 20.10
Downstream cluster type: RKE2 EC2 node driver

Testing:
Tested this issue with the following steps:

Deploy an RKE2 cluster - AWS ec2 node driver with Cloud provider enabled on the cluster
Take snapshots on the cluster
Navigate to Cluster Management --> cluster details --> Snapshots tab

Result - Pass
Snapshots are now available on the UI for a cluster with cloud provider enabled


Fixes developed from mypy_django script

A few discoveries from running mypy with Django-awareness. Nothing critical, but it caught a few odds and ends.

Lovely, thanks!


meta(engine): switch etcd business metastore to sql metastore

What problem does this PR solve?

Issue Number: close #6496
What is changed and how it works?

Switch etcd metastore to mysql metastore
Require Schema for sql metastore
Move metastore config to toml file
Replace 'frame-metastore-xx' witth 'framework-metastore-xxx'

Check List
Tests
Questions
Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?
Release note
`None`.

/area engine
/hold

Codecov Report

Merging #6435 (65f3dc1) into master (f14088c) will increase coverage by 0.1911%.
The diff coverage is 32.8571%.

Flag
Coverage Δ

cdc
66.0803% <69.7777%> (+0.1869%)
:arrow_up:

dm
52.0027% <15.3684%> (-0.0105%)
:arrow_down:

engine
64.1048% <ø> (+0.6417%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.
@@               Coverage Diff                @@
##             master      #6435        +/-   ##
================================================
+ Coverage   59.2403%   59.4314%   +0.1910%
================================================
  Files           754        762         +8
  Lines         86755      87351       +596
================================================
+ Hits          51394      51914       +520
- Misses        30799      30828        +29
- Partials       4562       4609        +47

/unhold
/status ptal
/cc @amyangfei  @CharlesCheung96 @gozssky

Wait for remove embeded etcd pt to be merged to master.

/hold

/unhold

/run-dm-compatibility-test
/run-dm-integration-test
/run-kafka-integration-test
/run-integration-test

/merge

/run-dm-integration-test

/run-dm-integration-test

/run-dm-integration-test


Submit R model deployer package to CRAN

The R package depends on a Python package (clipper_admin), so we will be unable to get it past the CRAN review process. For now, the recommended way to install the R package is from github.


Make it possible to use restic check --with-cache in a way that everything will be checked (eventually)

Output of restic version
restic 0.11.0 compiled with go1.15.3 on linux/amd64
What should restic do differently? Which functionality do you think we should add?
As the title said: Make it possible to use restic check --with-cache in a way that everything will be checked (eventually).
What are you trying to do? What problem would this solve?
I would like to run restic check often (like every backup), but for each run only a small and random subset of the repository should be checked. The idea is that errors are detected eventually but the amount of data that is downloaded at once is limited.
When only a small subset of the repository should be checked, downloading the index can take longer and use more data than reading the subset that should be checked. There is the option --with-cache but when using this option some files don't get checked ever.
I suggest that the files that are selected by the --read-data, --read-data-subset and especially the new --read-data-subset variant from #3038 are always read from the repository and not from the cache. In addition, these options should include index files and other files that they currently do not include.
That would mean that restic check --with-cache --read-data would be just as good as restic check --read-data. (For --read-data this isn't very useful of course but for --read-data-subset it can be very useful.)
Did restic help you today? Did it make you happy in any way?
Restic is great, it helped me to revolutionize my backup strategy.

Related to https://github.com/restic/restic/pull/3747


issue about letsencrypt with apache 2.2 in ubuntu 14

and my dns is:

how solve that?

Please try again...


Null Safety Support

Hi there,
Any chance of getting this package updated for null safety support and published as beta like most other packages are doing?

use this
responsive_grid:
git:
url: git://github.com/mohamed-selim-a/ResponsiveGrid_Flutter.git
ref: 22ee88eedca76cd70c0b9bacf7f506fd69ee20a6

use this
responsive_grid:     git:       url: git://github.com/mohamed-selim-a/ResponsiveGrid_Flutter.git       ref: 22ee88eedca76cd70c0b9bacf7f506fd69ee20a6

Excuse me for the late response
I can't figure out whether the migration of the pkg requires the referencing app to also migrate to null safety or not

Excuse me for the late response
I can't figure out whether the migration of the pkg requires the referencing app to also migrate to null safety or not

Official guide https://dart.dev/null-safety/migration-guide
Not sure what you mean with referencing but I have already made a PR https://github.com/mohamed-selim-a/ResponsiveGrid_Flutter/pull/14 with all the changes.

I mean the app that is using the pkg
should it be migrated also to null safety to be able to use this version

Yes all packages need to be migrated to migrate the package itself to null safety too. But I'm checking pubspec.yaml and don't see any package dependency?

@mohamed-selim-a I've resolved conflicts from the 1.3.0 version. If there's anything needed to be addressed please let me know

Thanks for your PR
I'll merge it and close the issue


Avoid reverse engineering

Hi, i use this library to encrypt string resource. With smali after decompiling app it's very easy to see original value. How can avoid reverse engineering ?

Hi @strike76 😊
Are you talking about hardcoded strings on the code?

Hi @efraespada i mean a string into strings.xml file. For example :
www.example.com

@strike76 then, you say:
1º APK generated using StringCare
2º APK decompiled
3º strings.xml file shows original values when they should be encrypted:
<string name="test" hidden="true" translatable="false">www.example.com</string>

Am I right?

@efraespada Of course no !
But

decompile app
create smali (do you know ?)
add LOG instruction to smali code
recreate APK
install APK
launch log app from playstore (For example LogCat)
decrypt strings are visible

Okay, finally I understood you.
StringCare doesn't take care about what happens from the decrypted string output.

With smali after decompiling app it's very easy to see original value

Of course, it is effortless without ProGuard obfuscation or obfuscating and knowing where SC.getString(...) is called on the code 😅
Proguard can obfuscate the Java classes, but not strings (StringCare purpose).

How can avoid reverse engineering?

No idea, If I knew how to do it this project wouldn't exist.

ok thanks my friend! @efraespada

Thanks to you for using the plugin/lib. Hope Proguard + StringCare could make tough the code inspection.
Alternatively, you can try DexGuard (ProGuard creators), but it's so expensive. 😕

Dexguard it's very expensive! Thanks


Model configuration: Ability to override IS NOT NULL clauses of an index

This is for SQL Server and I am using EF Core 1.0.1.

Assume that I have the below entity:
public class Person
{
    public string Name { get; set; }
    public string Lastname { get; set; }
    public DateTime? DeletedOn { get; set; }
}

Inside OnModelCreating, I also have this to create a multi-column unique index:
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    modelBuilder.Entity<Person>().HasIndex(x => new { x.Name, x.Lastname, x.DeletedOn }).IsUnique();
}

This causes EF Core to generate something similar to below:
CREATE UNIQUE NONCLUSTERED INDEX [IX_People_Name_Lastname_DeletedOn] ON [dbo].[People]
(
    [Name] ASC,
    [Lastname] ASC,
    [DeletedOn] ASC
)
WHERE ([Name] IS NOT NULL AND [Lastname] IS NOT NULL AND [DeletedOn] IS NOT NULL)

However, I want not to allow DeletedOn column to be null for this index. So, I am try to make EF Core to generate the below index creation script for me:
CREATE UNIQUE NONCLUSTERED INDEX [IX_People_Name_Lastname_DeletedOn] ON [dbo].[People]
(
    [Name] ASC,
    [Lastname] ASC,
    [DeletedOn] ASC
)
WHERE ([Name] IS NOT NULL AND [Lastname] IS NOT NULL)

QUESTION: Is this possible to do this with EF Core fluent API?
I dug a bit into the fluent API and was able to find some things related to HasAnnotation API but was not able to understand enough to achieve this.

This is not possible. Assigning to 1.2 to look at providing this functionality.

Fixed in 428a328

#1 Is it possible to have some example how to use it ?
#2 Is it possible that this can be back port to EF Core 1.0.5 ? Because i have that exact same issue, but i can't use EF Core 2.0, because i need to migrate my project to dotnet Core 2.0, and for the current moment is impossible. We have so much breaking change with the new mechanism of the Identity implementation.

@Davilink I filed https://github.com/aspnet/EntityFramework.Docs/issues/610 for the docs. We will not be porting this fix to the 1.x versions of EF Core--the bar is very high to get patch changes into those releases, and this issue does not meet that bar.

@ajcvickers Thx for the quick response.


vagrant up Command error

Hello, when I run vagrant up I get this error:
Vagrant failed to initialize at a very early stage:
There is a syntax error in the following Vagrantfile. The syntax error
message is reproduced below for convenience:
F:/MyOS/Vagrantfile:21: syntax error, unexpected tIDENTIFIER, expecting end
accessing "localhost:8080" will access port 80 on...
Please help, thanks DolphinFlopFish

I fixed it


关于CEE

请问知识库是否已开源？

We have already open-sourced the KB (Seeker/pipeline/cee.json), but considering some recent problems encountered in the discussion, we will further improve the experiments and optimize (submit will updates to ACL) repository and paper, so please stay tuned.

谢谢！


Add AccountingGroup to job classAds

@Farrukh-Aftab tells me it's needed for proper job accounting in global pool (between CERN and FNAL agents).
I still need to test it, @ticoann don't merge yet please.

Looks good from my side and Farrukh also gave the green light.
Feel free to merge it.

Thanks Seangchan, I added back that line.
Will fix those things in another PR (probably there are some outdated stuff)

Patch updated! Thanks


Feature request: Skip assets

tl;dr: It would be nice to be able to add e.g. -a to the decode command to skip assets altogether.
Information

Apktool Version (apktool -version) - last checked 2.2.2
Operating System (Mac, Linux, Windows) - Windows
APK From? (Playstore, ROM, Other) - Any

Questions to ask before submission

Have you tried apktool d, apktool b without changing anything? Yes
If you are trying to install a modified apk, did you resign it? N/A
Are you using the latest apktool version? No, but changelog doesn't indicate feature has been added.

Short and simple -- would like to be able to skip extracting assets. As far as I've seen personally these can usually be extracted with a simple unzip tool, but regardless, extracting the assets is (at least in situations where I've been using apktool) often unnecessary and in some cases flat out undesirable (for instance, I have an APK here (which I don't think I can share) that's 260MB compressed, 1.6GB uncompressed, and all we're interested in in this particular case is some of the metadata).

Currently, we have 2 options that either skip sources or resources.
As you probably know, if we skip resources than we just pass the resources.arsc file onward to the decode. It then is copied back during compile.
Likewise for the sources, if enabled we just pass the classes#.dex(s) onward instead of disassembling them.
So enter assets. Right now we move them to the filesystem while remembering their compression. If we skipped assets they would be stuck in the original application and to the best of my knowledge short of making an "asset" zip in root, there is no way around keeping the compressed files outside the zip.
I'm not sure for the best course of action yet for this feature request.

(I responded to this earlier but either I can't reply-to-comment like Github seems to imply, something went wrong, or it went to your personal email instead or something.)
What I would expect in this case is that they are simply not extracted. Yes, this would mean that you can't rebuild the app as-is with the directory where everything else was extracted, but I'm not sure how else the feature would be interpreted. If it's a question of terminology, "don't extract" could be used instead of "skip". I doubt that in every usage of apktool the intent is to later rebuild the app.
If there were really intent to rebuild, I dunno, I'd say there'd need to be a separate feature for that that would request the original APK or something. At surface value I'd say rebuilding & skipping assets are simply mutually exclusive.

We have -m / --match-original which is used for analysis. It prevents any of the "fixes" apktool does to allow rebuild. However, analysis sometimes is required for the unknown/asset files so it doesn't make sense to add this feature to that option (Just something I was thinking)
I don't have a problem adding an additional parameter for this. It may be an advanced command (thus only long version like --skip-assets or something). I don't want to pollute the shorthand command parameters anymore unless crucial and since this command won't act the same as the skip resources/sources it makes sense to separate it.
Let me get https://github.com/iBotPeaches/Apktool/pull/1602 merged first as it touches the CLI/parameters and I can swing back to this.

Well, it's possible for there to be incompatible arguments; If you want to prevent any confusion then -m / --match-original could block/warn usage of --skip-assets (or whatever the name ends up being) with some error/warning message explaining the conflict.
At any rate, that all sounds good to me and --skip-assets or --no-assets would make sense to me. I appreciate the quick responses on this, too.

So I can get an idea of if I should look for alternatives in the interim, when do you think you would be able to work on this? (I'm not trying to add any pressure.)

I have Thursday set aside this week at work to work on Apktool. I will fix up the pending PRs and see if I can add this real quick. That would seal it into the 2.3 release coming hopefully early next week.

Any luck on this?

Done. Commit above. It uses --no-assets and prevents copying of the assets folder.

Thanks! I noticed you have a CI but I don't see any way to download the artefact / build from it. Is it possible?

Unfortunately not, there is no publishing attached to our CI. I am planning a release for Wed/Thur of this week.


Upgrade q_compress -> pco

Hi, I'm the main author of q_compress, and I think metricsql would benefit from upgrading to my new library, pco. Across various real-world datasets, it has

2-100% higher compression ratio
20-50% faster compression
300-400% faster decompression

Plus, pco will continue to improve. q_compress will still work, and I'll still maintain it for bug fixes, but limitations in its format prevent it from achieving extremely fast decompression.
Q: If the qco format had limitations that forced you to release a new format after 2 years, how do we know the new format will last?
A: I don't profess to know the future, but I expect pco's format to be state of the art for many years. I consulted an expert in performant compression to help vet the format. I've iterated based on assembly code to make sure pco can use SIMD where possible. And I've carefully made the format extensible to more types of data for future adaptation.
Q: What does the upgrade look like?
A: The auto_[de]compress are still there, under pco::standalone::, so those ones are trivial to port over. I see you're using parts of the low-level API, and pco is slightly different, but will probably simplify your code somewhat. The entry points are in pco::wrapped::{FileCompressor, FileDecompressor}. docs.rs is the best place for details. And of course, if you have existing .qco data, you'll need to keep using q_compress for decompression until it's ported over. If necessary, you can distinguish standalone .qco data from .pco data by the magic 4 bytes in the header.

Hey @mwlon , thanks for reaching out !
Fortunately, I've been tracking development on pco and fully intend to switch. The code is not yet used in production so transition costs will amount to a few hours of refactoring and testing.
And thanks for the library - great work !


Can I use synced-cron to time a single server event?

I'm building an auction system where the bidding on each item ends at a specific time. When an auction ends, certain operations should be executed. This is a one-time event, of course.
SyncedCron seems like the most robust solution for timed server events – is there an easy way of creating a job that runs once at a specific time in the future?
Thanks!

I've reearched a bit deeper.
It seems that synced-cron expects there to always be a next event on the Later.js object – however, in some cases, Later.js doesn't have any more events. It may be that a schedule is only be active between two points in time, or – as in my case – simply is a point in time:
SyncedCron.add({
  name: cron_name,
  schedule: function (parser) {
    // ending_at is a Date object set to some future date
    return parser.recur().on(ending_at).fullDate();
  },
  job: function () {
    console.log("hep!");
    // remove job so it doesn't recur
    SyncedCron.remove(cron_name);
  }
});

The above code results in the following exception:
Exception in setTimeout callback: TypeError: Cannot call method 'getTime' of undefined
     at scheduleTimeout (packages/percolate:synced-cron/synced-cron-server.js:240:1)

... since in the synced-cron code, the call to getTime fails when there are no more events. Is there a specific point in my code I could call SyncedCron.remove before the failing call is made?
https://github.com/percolatestudio/meteor-synced-cron/blob/125ae97a05fbc27831762e239ecce6cf5fd97555/synced-cron-server.js#L240
Other ways to make this work? Would be easy to patch synced-cron to stop when there are no next events, but I think this is an API decision.

So, I created a patch... What it does is simply removing the job if the schedule doesn't have any more events. Also, it makes sure the next event isn't triggered if the entry is removed inside the job function body.

I ran into a similar problem on one of my projects, I opted to go with the CRON parser (parser.cron) which accepts standard CRON syntax. From there I was able to formulate a single event using an expression such as; "10 18 26 4 ? 2015"

Thanks for the input @tylerstraub . My plan is still to take a look at this ticket as soon as I have some free time.

Don't know how much the code has developed, but my pull request solving this problem is quite simple – you could just use that :) I'm using it in production, anyways.

@dalgard - The reason we ask that PR authors write tests is because we maintain quite a few o/s projects and don't necessarily have the bandwidth to go through each PR in detail and support it into the future. It's actually not that we're in doubt whether the code is production ready (we give you guys the benefit of doubt). If the PR comes with tests however, we're comfortable merging it with less scrutiny knowing that the author has described exactly what the functionality does and in fact we won't break it in future versions.
This way we can support functionality that we don't actually use ourselves as the tests become the vehicle that ensures we don't break stuff that others rely on.
In short, if we don't use something ourselves and there are no tests for it there is a pretty good chance that functionality may break in between releases - this is something we obviously want to avoid.
Hence with PR's like yours that don't come with tests, we need to schedule in the time and come up to speed with the functionality + write tests ourselves. Unfortunately, during busy times this falls down our priority queue.

I totally understand and earnestly consider it a healthy priority – had it been my project, I hope I would be equally methodical. All I would do, is encourage you to review my PR and see how limited the adjustments would be for resolving this issue. Throwing together a patch on the basis of the lines I've submitted is a convenient path at times where professional obligations and other things take rightful precedence :)

Any update on this.  Just noticed this issue on my project.  It actually causes my server to crash and restart.  In the meantime I'll see about adopting the solution that @dalgard came up with.

FYI there's an orphaned package which uses dalgard's solution.  I've been using it... so far so good.
https://atmospherejs.com/orphan/synced-cron

+1 for merge.

+1 for merge

+1 for tests to be added + merge

+1 for merge.

+1 as well!

Released a fix to point in time schedules in version 1.3.0. Let me know if there are any issues with it. Thanks for your patience.

I am using version 1.3.2, and want to fire a once-off job, but still facing the same issue:
    SyncedCron.add({
      name: 'reminder_email_' + uid,
      schedule: function(parser) {
        return parser.recur().on(new Date() + 60000).fullDate();
      },
      job: function() {
        // functions here
      }
    });

Console prints:
I20160519-17:56:27.539(-4)? Exception while invoking method 'addTask' TypeError: Cannot call method 'getTime' of undefined
I20160519-17:56:27.539(-4)?     at scheduleTimeout (packages/percolate_synced-cron/synced-cron-server.js:309:1)
I20160519-17:56:27.540(-4)?     at Object.SyncedCron._laterSetTimeout (packages/percolate_synced-cron/synced-cron-server.js:289:1)
I20160519-17:56:27.540(-4)?     at Object.SyncedCron._laterSetInterval (packages/percolate_synced-cron/synced-cron-server.js:257:1)
I20160519-17:56:27.540(-4)?     at scheduleEntry (packages/percolate_synced-cron/synced-cron-server.js:104:1)
I20160519-17:56:27.540(-4)?     at Object.SyncedCron.add (packages/percolate_synced-cron/synced-cron-server.js:127:1)
I20160519-17:56:27.541(-4)?     at [object Object].addTask (server/methods.js:107:20)
I20160519-17:56:27.541(-4)?     at maybeAuditArgumentChecks (packages/ddp-server/livedata_server.js:1704:12)
I20160519-17:56:27.541(-4)?     at packages/ddp-server/livedata_server.js:711:19
I20160519-17:56:27.541(-4)?     at [object Object]._.extend.withValue (packages/meteor/dynamics_nodejs.js:56:1)
I20160519-17:56:27.541(-4)?     at packages/ddp-server/livedata_server.js:709:40

@yyl The cron is working as it should, the on method just requires a date not a timestamp.
The correct way to initialise the cron, if you just wanted to add 6s to the current date-time then you could just do something like:
    SyncedCron.add({         name:"Test single time cron",         schedule(parser){             let time = new Date(Date.now() + 6000);             return parser.recur().on(time).fullDate();         },         job(){             console.log("IT IS WORKING")         }     });

@adrianbadarau great that works, thanks!


New-AzDiskConfig expects size in MiB multiples

This code from the referenced page
$vhdSizeBytes = (Get-Item "<fullFilePathHere>").length

$diskconfig = New-AzDiskConfig -SkuName 'Standard_LRS' -OsType 'Windows' -UploadSizeInBytes $vhdSizeBytes -Location '<yourregion>' -CreateOption 'Upload'

Can produce an error like this
New-AzDisk: The upload size in bytes<PHONE_NUMBER>6 - 512 bytes for the VHD footer<PHONE_NUMBER>4 in this case) must be a multiple of MiB.
ErrorCode: BadRequest
ErrorMessage: The upload size in bytes<PHONE_NUMBER>6 - 512 bytes for the VHD footer<PHONE_NUMBER>4 in this case) must be a multiple of MiB.
ErrorTarget:
StatusCode: 400
ReasonPhrase: Bad Request
OperationID : 2f0528cd-63f7-4ff2-8560-cc128e42d34f

I had to calculate a disk size in multiples of MiB
([Math]::Ceiling(($vhdSizeBytes - 512) / (2 -shl 19)))*(2 -shl 19)

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 912457e0-26de-cb94-fa50-075f986ce595
Version Independent ID: 0d50c9b3-7d08-79ba-3a8f-b6319b19b435
Content: Upload a VHD to Azure or copy a disk across regions - Azure PowerShell - Azure Windows Virtual Machines
Content Source: articles/virtual-machines/windows/disks-upload-vhd-to-managed-disk-powershell.md
Service: virtual-machines
Sub-service: disks
GitHub Login: @roygara
Microsoft Alias: rogarana

@sensomatic - Thanks for brining this to our attention. We will investigate it further and update you shortly.

Looks like a note was added in regards to this already.
#please-close

([Math]::Ceiling(($vhdSizeBytes - 512) / (2 -shl 19)))*(2 -shl 19)

I tried this and still got the same error:
New-AzDisk: The upload size in bytes<PHONE_NUMBER> - 512 bytes for the VHD footer<PHONE_NUMBER> in this case) must be a multiple of MiB.
ErrorCode: BadRequest
ErrorMessage: The upload size in bytes<PHONE_NUMBER> - 512 bytes for the VHD footer<PHONE_NUMBER> in this case) must be a multiple of MiB.
ErrorTarget:
StatusCode: 400
ReasonPhrase: Bad Request
the original file size is:<PHONE_NUMBER>

I experience the same error. Even I tried this recalculation with ([Math]::Ceiling(($vhdSizeBytes - 512) / (2 -shl 19)))*(2 -shl 19).
What should I do ?
Is there any workaround ?


Stepfunctions-Textract-StartDocumentTextDetection does not accept SNSTopicArn in NotificationChannel

I think there must be some validation error on the backend of step function schema validation when validating NotificationChannel parameter on async Textract operations such as StartDocumentTextDetection.
According to the documentation https://docs.aws.amazon.com/textract/latest/dg/API_StartDocumentTextDetection.html the operation takes parameter SNSTopicArn (absolutely crucial param) but this parameter can't be configured when we try to integrate it as a step in a step function.
On save the step function gets validated and validation error shows up:
The field "SNSTopicArn" is not supported by Step Functions /States/StartDocumentTextDetection/Parameters
For more information, see Amazon States Language

This basically disqualifies Textract with its async invocated operations from use in step functions. Workaround is probably to launch it from lambda function step.
Reproduction Steps
Create step function via aws console with this content:
{
  "StartAt": "StartDocumentTextDetection",
  "States": {
    "": {
      "Type": "Task",
      "Parameters": {
        "DocumentLocation": {},
        "NotificationChannel": {
          "RoleArn": "string",
          "SNSTopicArn": "string"
        }
      },
      "Resource": "arn:aws:states:::aws-sdk:textract:startDocumentTextDetection",
      "End": true
    }
  }
}

What did you expect to happen?
step function gets successfuly saved

What actually happened?
On save the step function gets validated and validation error shows up:
The field "SNSTopicArn" is not supported by Step Functions /States/StartDocumentTextDetection/Parameters
For more information, see Amazon States Language

Environment

AWS Step Functions Data Science Python SDK version  :
Python Version: <!-- Version of Python (run the command python3 --version) -->

Other

This is :bug: Bug Report

Ok, it's not a bug, the SNSTopicArn must be spelled out in pascal case - SnsTopicArn


updates branding text and copyright date

LGTM - leaving it for @joeconwaystk to merge when he's good with it.


Upgrade to babel@7

Codecov Report

Merging #381 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master   #381   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           1      2    +1
  Lines          15     21    +6
=====================================
+ Hits           15     21    +6

Impacted Files
Coverage Δ

.babelrc.js
100% <100%> (ø)

src/index.js
100% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8c55f18...8d3f7c5. Read the comment docs.

Seems like this still is relevant.  @Andarist , any chance you could update the PR to merge cleanly?

@markerikson I've rebased and confirmed that it still compiles but please re-check the current setup.

Oh, I haven't spotted in a rush that this repo is not using Travis any longer 😅

that's because I only merged the other PR like 10 minutes ago :)

Well, this seems to build cleanly.  Let's get it in and move things along.  Thanks, @andarist !


Who to contact for security issues

Hey there!
I belong to an open source security research community, and a member (@michaellrowley) has found an issue, but doesn’t know the best way to disclose it.
If not a hassle, might you kindly add a SECURITY.md file with an email, or another contact method? GitHub recommends this best practice to ensure security issues are responsibly disclosed, and it would serve as a simple instruction for security researchers in the future.
Thank you for your consideration, and I look forward to hearing from you!
(cc @huntr-helper)

Hey @JamieSlome, thank you for your advice and I will create a SECURITY.MD later！
Thanks to @michaellrowley as well


fix: Use std::sync::OnceLock instead of once_cell::sync::OnceCell

Refactor to use std::sync::OnceLock instead of once_cell::sync::OnceCell

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Files
Coverage Δ

src/config.rs
100.00% <ø> (ø)

src/context.rs
98.79% <ø> (ø)


saving dataframe generated via from_delayed in parallel to hdf5 fails

I tried saving a dataframe generated via from_delayed to hdf. I tried all schedulers. Additionally, I tried saving to several files (asterisk in path) or to several nodes (asterisk in node). In none of the cases, the result was as expected. I tested this on the master branch. To make it short:

I was unable to save in parallel using the multiprocessing scheduler.
The distributed scheduler throws rather uninformative errors. However, if lockis set to Falseand the asterix is provided in the path, everything works apparently fine. Is it safe to disable the lock?

Now the details.
create example dataframe
import tempfile
import shutil
import os
import random
import string
import numpy as np
import pandas as pd
import dask
import dask.dataframe as dd
import distributed
from multiprocessing import Lock

#generate test dataframe, containing a variety of data
rowsPerPartition = 5000
npartitions = 500

def pdfGenerator(nrows):
    '''generates pandas dataframe containing dtypes float64, int64 and object (random text)'''
    randomTextGenerator = lambda length: "".join( [random.choice(string.letters + string.digits) for i in xrange(length)] )
    textColumn1 = map(randomTextGenerator, [50]*nrows)
    floatColumn1 = np.random.uniform(-100, 100, nrows)
    intColumn1 = np.random.randint(-100, 100, nrows)
    return pd.DataFrame({'t': textColumn1, 'f': floatColumn1, 'i': intColumn1})

d = dask.delayed(pdfGenerator(rowsPerPartition))
ddf = dd.from_delayed([d]*npartitions)

threaded and multiprocessing scheduler use only one core
#save to hdf5, many files
print('multiprocessing scheduler, save to many hdf5 files:')
path = tempfile.mkdtemp()
%time ddf.to_hdf(os.path.join(path, 'a*.h5'), '/test', get = dask.multiprocessing.get, complib = 'blosc')
shutil.rmtree(path)
print('')

print('multithreading scheduler, save to many hdf5 files:')
path = tempfile.mkdtemp()
%time ddf.to_hdf(os.path.join(path, 'a*.h5'), '/test', get = dask.threaded.get, complib = 'blosc')
shutil.rmtree(path)
print('')

#save to hdf5, many nodes
print('multiprocessing, from delayed, many hdf5 nodes:')
path = tempfile.mkdtemp()
%time ddf.to_hdf(os.path.join(path, 'a.h5'), '/test*', get = dask.multiprocessing.get, complib = 'blosc')
shutil.rmtree(path)
print('')

print('threaded, from delayed, many hdf5 nodes:')
path = tempfile.mkdtemp()
%time ddf.to_hdf(os.path.join(path, 'a.h5'), '/test*', get = dask.threaded.get, complib = 'blosc')
shutil.rmtree(path)
print('')

multiprocessing scheduler, save to many hdf5 files:
CPU times: user 22.3 s, sys: 5.52 s, total: 27.8 s
Wall time: 25.2 s
multithreading scheduler, save to many hdf5 files:
CPU times: user 16.3 s, sys: 2.32 s, total: 18.6 s
Wall time: 17.3 s
multiprocessing, from delayed, many hdf5 nodes:
CPU times: user 44.6 s, sys: 10.3 s, total: 55 s
Wall time: 51.5 s
threaded, from delayed, many hdf5 nodes:
CPU times: user 17.9 s, sys: 2.99 s, total: 20.9 s
Wall time: 19.5 s

At least, I would expected the multiprocessing scheduler to work in parallel.
The distributed scheduler requires a lot of trail and error
Using the distributed scheduler, it is possible to save in parallel, but only if:

lock = False (is this risky?)
asterix in path and not in node

print('distributed, from delayed, save to many hdf5 files:')
path = tempfile.mkdtemp()
#needs lock = False, otherwise: can't pickle thread.lock objects.
#lock = True or lock = multiprocessing.Lock() does not work.
%time ddf.to_hdf(os.path.join(path, 'a*.h5'), '/test', get = c.get, complib = 'blosc', lock = False)
shutil.rmtree(path)
print('')

distributed, from delayed, save to many hdf5 files:
CPU times: user 1.62 s, sys: 152 ms, total: 1.77 s
Wall time: 2.33 s

In the case that the asterix is provided in the node, I got unlcear / confusing error messages:
#Causes error, no matter what:
#lock = False --> HDF5-Error
#lock = True --> TypeError: can't pickle thread.lock objects
#lock = None --> TypeError: can't pickle thread.lock objects
#lock = multiprocessing.Lock() --> EOFError
print('distributed, from delayed, many hdf5 nodes:')
path = tempfile.mkdtemp()
%time ddf.to_hdf(os.path.join(path, 'a.h5'), '/test*', get = c.get, complib = 'blosc', lock = None)
shutil.rmtree(path)
print('')

Misc

Writing in parallel to csv works out of the box, if the distributed scheduler is set as default.
Writing in parallel to hdf using the multiprocessing scheduler works, if the dataframe is generated via from_csv.
If repartition is used on a dataframe generated via from_csv, it cannot be saved in parallel any more using the multiprocessing scheduler (this was my original problem).

As far as locking is concerned, probably worth trekking over to issue ( https://github.com/dask/distributed/issues/780 ).

Coming back to this after one year, the problems I mentioned here are tracked in other issues in a more concise way.


Displaying many tracks leads to the timeline controls not being visible

Displaying many tracks (depending on screen size) leads to the timeline controls no longer being visible. Seems related to #197
Steps to reproduce:

Add more tracks then can be displayed in the current size of the timeline
The timeline controls at the bottom are no longer visible

Time controls get still hidden

Is fixed. Thx!


Feature/application monitoring

I've added Prometheus for FastAPI. There are two new Docker Container: Prometheus and Grafana.
Grafana is the Frontend tool to show Prometheus monitoring data.

@philippabele
Solved this issue. FYI: I changed pytest.yaml to build the Docker containers now. Pytest is now executed in the microservice container.
If this is alright, I'm going to implement this in the other PR as well.


Is this project actively maintained?

I see lots of pull requests and issues like keyboard navigation, setting initial value etc. Will there be any progress on integrating these contributions?

It looks like the answer is no for now. I'm interested in picking this back up and getting a few of the requests merged in. I've reached out to the author to see if he's interested in passing it along or if I should go ahead and fork it. Hopefully we can get this project moving again, there's a number of decent looking PRs that could be merged in a provide good value.

I was never able to reach the author, so forked this repo and am actively maintaining it here:
https://github.com/mattzollinhofer/vue-typeahead-bootstrap
I pushed an updated version to support accessibility. Specifically it  handles arrow keys moving through the list. It adds a few more commonly requested issues. Working through the backlog slowly, feel free to ping on an older issue to let me know which ones are still causing people difficulty!
Grab the new version from npm here: https://www.npm.com/vue-typeahead-bootstrap


Lists

Allows user to edit/add/delete the lists.

is this version published anywhere for testing?

For some reason the /next/ does not work, but here you can see it here: https://kedyn.github.io/fusliez-notes/next/index.html

on an unrelated note... a lot of people seems to want a default not active/played section for colors that aren't in the current game.
should we add one? or let the users do it themselves now we enabled the option to add/remove sections?

I think just letting them change it should be fine, besides we are reverting back to saving session data. Therefore they would only have to do it once. This would also keep the same size for stream.


Added note about behaviors & inheritance

I ran into this problem today. I had a parent class with behaviors on it, and it resulted in errors (in Astronomy, not my code). Once I moved the behaviors to the child classes, it worked ok.

It's not true. Create reproduction. If it's a bug then it should be fixed.

Ahh gotcha, I assumed it was a limitation. I'll create a repro when I get a chance, thanks

Ok thanks


Bug: Invalid arguments

Following to my PR to transformers repo, I've tried tested multi-GPU and multi-machine setting and got these two errors:
accelerate <command> [<args>] launch: error: argument --main_process_ip: invalid typing.Union[str, NoneType] value: {some_ip_address}

and
accelerate <command> [<args>] launch: error: argument --main_process_port: invalid typing.Union[int, NoneType] value: '{some_port_value}'

It seems to me this error can be fixed with changing type=Optional[str] -> type=str for --main_process_ip arg and type=Optional[int] -> type=int for --main_process_port arg.
@sgugger

Fixed in this commit. Thanks for flagging!


 Nobody takes them into account when doing anything.
In my opinion infrastructure built ad-hoc should not be as good as already-existing infrastructure as well as that players should have the opportunity to break structures encroaching on their land within a reasonable time period. Current maturation times are so low that there's no realistic scenario in which someone would try to break unmatured blocks to take advantage of their weakened state. There's simply not enough time to act.
To start with I propose going with a maturation time of one day for diamond, to match bastion maturation time:

Stone: 30min
Iron: 3h 30min
Diamond: 1d

Note that I think these are too low still, but it'd at least be a step in the right direction.

This is a solid point and good recommendation, I'm just wondering if anyone else has any input?

For reference, note that the damage done to maturing reinforcements is multiplied by the inverse of the maturation progress. Meaning a maturation which has completed  1/x of its maturation time has its damage taken multiplied by x.

Resolved by #87


feat(header) add simple navigation header

fixes #76
Made a minimal navigation component, just to get it going, very minimal styling.
https://user-images.githubusercontent.com/56167866/107590483-12015400-6bbd-11eb-93b9-a523998427cc.mov

Woah, that's janky on the raid stats screen - nice! :rofl:

Woah, that's janky on the raid stats screen - nice! :rofl:

Lol i tried to make it cover at least all the title when you scroll so it's not too bad 😅


Add Godzilla by Slim Sh4dy (eminem)

Song Name
Godzilla
Artist Name
Slim Sh4dy (Eminem)
Source
Youtube
Youtube Link
ol-6J3AmrB2X2w6S
Direct File Link
No response
Song ID
4
Start Offset [ms]
0
Extra
Created from Auto Nong.
Created for level: 59847342

reject
invalid yt link