Output profiling from C# that might prove useful to JSCAD development.

Below is an image of a profile of my translation of JSCAD "union" performance tests to C#.
In particular what it shows is that unions (and it will be true for all booleans) are dominated by the seemingly simple
routines poly3.measureBoundingBox, poly3.measureBoundingSphere and Math.hypot.
I renamed a few things in my CS version, so here is a translation:
JSCAD poly3.measureBoundingBox is CSCAD Poly3.BoundingBox
JSCAD poly3.measureBoundingSphrere is CSCAD Poly3.BoundingSphere
JSCAD (really Nodejs) Math.hypot is Vec3.Hypot.
The column you want to look at is Self CPU... this is the % of CPU the given routine is using inside itself (not in things it calls).
The profile is from running the "union" performance tests and as you can see, BoundingBox/BoundingSphere/Hypot need to be as optimized as possible. With some hard work, one might be able to speed up the boolean operations by 25% or more.
My implementation of hypot that matches the new IEEE spec.
    /// Returns the hypotenuse of the three points, avoiding unnecessary underflow/overflow.
    public static double Hypot(double _x, double _y, double _z)
    {
        var x = _x < 0.0 ? -_x : _x;
        var y = _y < 0.0 ? -_y : _y;
        var z = _z < 0.0 ? -_z : _z;
        var m = x > y ? x : y;
        m = m > z ? m : z;
        if (m == 0.0)
        {
            return 0.0;
        }
        x /= m;
        y /= m;
        z /= m;
        return m * Math.Sqrt(x * x + y * y + z * z);
    }

In accordance with the issue I raise in #1095 I will be changing this to:
    /// Returns the hypotenuse of the three points.
    public static double Hypot(double x, double y, double z)
    {
        return Math.Sqrt(x * x + y * y + z * z);
    }

@briansturgill
i've been trying to understand the reasoning behind the IEEE-like implementations. it seems that the scaling (by largest value) makes an assumption that ALL values are of the same 'scale'. i that isn't true then there's not much value in scaling to eliminate loss of precision errors.
did i understand correctly?
and yeah. i think JSCAD doesn't benefit from the use of Math.hypot()

It isn't entirely unreasonable to believe they are likely to be of the same scale... How many triangles do you have that have two sides a light-year long and and the third a nano-meter long? The problem is that hypot if you deal with very large or very small numbers, hypot done with Sqrt fails. Fortunately that doesn't apply to CAD. I'm assuming that the IEEE people made the assumption that people that didn't need the scaling would just do it the traditional way. I don't really know, I derived the code above by reverse engineering what G++ did with inlined code. It does what IEEE intended, but under C# runs about 5x slower.

@briansturgill thanks for the nice performance hints.
The change from hypot() to sqrt() has been released. As well as, the improvements to measureBoundingSphere(). Performance of the BSP trees has been boosted by 40 to 50%.
I think this issue could be closed now. Please let us know if you find any more areas for improvement.

Yeah, it's looking good, I'll try to run a full performance test on it tonight.
I reran profiling on CSharpCAD after making similar changes and believe what's left is just indicative of what it takes to do the job.
I also looked at many other places and it looks like most things are fine.
One exception, extrudes... but I'm just now looking into why. I'll post more if I decide there's potential significant gains.
If you're curious, for ExtrudeLinear and ExtrudeRotate, 48%/40% are spent in Extrude Walls, plays an 11% role in Extrude Rectangular.
Also big in ELinear,ERotate: Plane.FromPoints (17%/26%) and Slice.Transform(8%, 9%).
The percentages reflect the amount of CPU time running the named functions called, but without the time of functions that function calls.
The total CPU time is the time (running individually) the Performance Test for the 3 Extrudes.

OK, generally nothing wrong in extrudes. I found a few problems in my translation, but I think it unlikely any of it applies to JSCAD.
It might be worth someone's time to take a look at extrudeWalls as it is very critcal path and moves a lot of data around.

the extrude alg probably needs revisiting. Maybe splitting into few variants. I am sure there is subset of use-cases (that are actually likely very common) that can be superfast.

the extrude alg probably needs revisiting. Maybe splitting into few variants. I am sure there is subset of use-cases (that are actually likely very common) that can be superfast.

Interesting... as you know I'm review the primitives to finalize CSCAD API. One thing I've been doing is rewriting (actually massively refactoring) the primitives (for example circle to circle (full) and semicircle).
One thing I've found is that there is quite a bit of unnecessary work done. For example "center" always gets added, even though that wouldn't be the usual case. (I mean, does one really want to be adding zero to EVERY point, EVERY time)?
Anyway, the other thing that became clear to me is that all of the 3D's that are essentially extruded Geom2's are doing their own linearExtrude... and that linearExtrude really is taking much too much CPU in its current form.
I'm working on a rewrite of that now.

An optimization I've been meaning to implement on extrudeLinear is a check for convexity. Right now it runs the ear cut algorithm, cutting off triangles "ears" recursively, until everything is triangles. But there is no need to do that if the polygon is convex.
First of all, if the poly is convex, then triangulation is trivial, no need for complicated earcut algorithm. But also, JSCAD allows faces to be any convex polygon, not just triangles. So we can save time and memory be returning early from earcut. Furthermore, extrudeLinear is already tracking the convexity of each corner, so it should be easy to determine when a polygon is convex.

The extrude functions were created using the same functionality to make the process exactly the same. There are cases where the number of points are not the same, and therefore triangles are required to create the walls.
I'm sure optimizations are possible but don't forget that the extrude functionality is very powerful.

I am in favor of having a different extrude for cases that can benefit in performance. I would welcome even going to a next level general aproach of adding more performance hints similar to HasOnlyOnePath .

Oh, I'd appreciate if those of you that understand sides (likely better than I do) check that the HasOnlyOnePath property checks  what it says it checks.

@z3dev

The extrude functions were created using the same functionality to make the process exactly the same. There are cases where the number of points are not the same, and therefore triangles are required to create the walls.
Could you elaborate on the cases where there number of points are not the same?
At least in extrudeLinear, I expect it is always the same.


Fix #672 - handle invalid UTF input more gently

I'm not so sure about this. It will create a lot of overhead :/

Could an option be to handle it lazily? I.e. wrap in try/catch and do the decoding only if there's an exception? Or have an error?
Or writefln it? (since it seems like writeln deals with this?)

I.e. wrap in try/catch and do the decoding only if there's an exception?

Yeah that does sound like the reasonable thing to do. :+1:

core.exception.AssertError@source/rest/apiv1.d(255): unittest failure
----------------
??:? _d_unittestp [0xdb1f6d]

Urgh that passes on my local machine (added debug output)


Postgres

Pull Request Test Coverage Report for Build 54afcc635ae5f9221edaa9a7fe26a6320921a649-PR-249

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.1%) to 83.523%

Totals

Change from base Build df09dc48615803e075c704416c0803e44a234cb8:
0.1%

Covered Lines:
4106

Relevant Lines:
4916

💛  - Coveralls

runs most test vs PostgreSQL


How do I use this properly?

Sorry to bother anybody but, i ran the  toypadDemo.js, and it gave me an error saying there was a syntax error on line 8
character 23
code 800A03EA
source Microsoft JScript compilation error.

It has to be run using node.js on the commandline, doubleclicking the js file just tries running the windows JS environment.
Node.js version 4.1 or higher is recommended https://nodejs.org/
from a command prompt in the node-ld folder:
npm install (This installs any dependencies needed)
node samples\toypadDemo.js (Runs the demo)

thank you

I am sorry but when i type "node samples\toyboxDemo.js"  I get the error of
Error: Cannot find module 'node-hid'
at Function.Module.resolveFilename (module.js:399:15)
at Function.Module.load (module.js:290:25)
at Module.require (module.js:367:17)
at require (internal/module.js:16:19
at Object. (C:\Users\UsernameHere\Desktop\node-ld-master\transports\hid.js:6:11)
at Module._compile (module.js:413:34)
at Object.Module._extensions..js (module.js:422:10)
at Module.load (module.js:357:32)
at Function.Module._load (module.js:314:12)
at Module.require (module.js:367:17)

the npm install command should install that

when i do it gives me errors and is only able to do some of it

what does node -v return?

5.5.0

Try with node 4.2, it doesn't look like node-hid has prebuild binaries for 5.5

You will need to uninstall 5.5 before installing 4.2

Ok

same error

forgot to do npm install

Please can you report what npm -v shows?

it says for some reason 2.14.12 when i installed 4.2.6

now i am confused

npm is the package manager that comes with node, its supposed to be a different version

delete the node_modules folder (To ensure a clean run) and run npm install again, as long as it doesnt throw ERROR, you should be able to run toypadDemo

the node_modules in the node-ld folder?

yes

I deleted the folder, ran it, got this error, reinstalled the node-ld-master folder, ran it and still this error

Thats not an error, its just a warning. node samples/toypadDemo.js should work now

oh ok

gave me an error saying cannot write to the HID device

Try it again, I get that error sometimes and it usually works after a couple tries

cool.

so now how do i edit the toy tags and stuff?

also how do i terminate the program without closing the command prompt
sorry i am really a huge noob

Mostly you would need to write your own code for editing, this is mostly a library that makes it easier to work with the raw data.
Ctrl-C should terminate the program

thank you

sorry to bother again but, on the newVehicle program what is "var out1 = new Buffer ([0x8E, 0x04, 0x00, 0x00])" and how do i change it to what i want with the data given in the token map

out1.writeUInt32LE(1234) after that var line would update that buffer with token id 1234

k thanks

it gave me an unexpected token var error

Has anybody even found out how to edit characters?

Vehicles are easy but characters seem like there is no way to break through them

You cannot edit character tags; they are permanently locked.

oh

but people have been searching for a pwd gen to get through havent they?

PWD only works if the lock bits haven't been set, on characters they are set

No password will help. Character tags are permanently locked at a hardware level; they cannot be rewritten. Vehicle tags are also permanently locked at a hardware level to be a vehicle tag.

so, there will be no way to ever write to the character tags?

No. They are permanently locked on the tag itself.

oh well. Thanks

I was spending the  past hour figuring that out lol


JS: Handle surrogate pairs correctly

The following PR makes the JS algorithm aware of surrogate pairs and will avoid splitting a pair into two different Diff objects.
I think there might be more places which needs to be surrogate-pair-aware, but I haven't been able to find test cases to confirm this.
Fixes #59, #68, #10

Hey @judofyr, thanks for your work on this! I was playing around with it and found a case that still throws a URI error:
oldText = `h
  🅰
\t🅱\n`;
newText = `hello
  🅰
\t🅱\n`;
dmp.patch_toText(dmp.patch_make(oldText, newText));

https://jsfiddle.net/g7jsa4me/

Oooo, very nice! Thanks for the reduced test case. I haven't had time to look more into this issue, and will probably not have time in the next few weeks. I'll report here if I end up picking it up again.

:+1: eagerly waiting!

Hi, this bug is a tad worrying.
So I thought I'd see if I could see were the problem is.
For @josephrocca issue the following function is the cause ->
Function diff_match_patch.prototype.patch_addContext_ = function(patch, text) {
// Add the suffix.
var suffix = text.substring(patch.start2 + patch.length1,
      patch.start2 + patch.length1 + padding);

Above the suffix is getting splitting on a surrogate.
Using @judofyr fix and applying here seems to fix it..
eg.
// Add the suffix.
var pointermid = patch.start2 + patch.length1 + padding;
  if (insideSurrogate(text, pointermid)) {
    pointermid--;
}
var suffix = text.substring(patch.start2 + patch.length1,  pointermid);

Not sure, but I would also expect it might be possible for the prefix to also suffer with this issue, so applying the above code for prefix might make sense.
But doing the above seems to makes @josephrocca code work.
Hope this helps..

@KpjComp Can you put your fix into a pull request so I can run some tests on it? I would be great if this issue were finally fixed! Might be able to attract the attention of a maintainer and actually get this merged. I'm very ignorant when it comes to Unicode stuff, so at this point I can't verify any of your code by reading it, but I can put it through some significant testing, and hopefully @judofyr will be able to have a quick read over it at some point to provide feedback.

@josephrocca ,  I was kind of hoping @judofyr might be able to apply the patch when he see's this message.  Looks like he just needs to reply with I signed it! or something.
What you could do, is do what I have done as a temp measure,.. Just open the index.js file in the diff-match-patch in your node_modules and edit this file directly and apply mine and @judofyr fix.  I could maybe attach my modified version here if you want.
Also, just in case there are any more issues with unicode surrogate pairs and patching what I have done in my application is when I create a diff, I also then apply the diff to the original and verify it it's the same, so if there any any more problems I will know about it.. :)

I've replaced this:
  // Add the suffix.
  var suffix = text.substring(patch.start2 + patch.length1,
                              patch.start2 + patch.length1 + padding);

with this:
  // Add the suffix.
  var pointermid = patch.start2 + patch.length1 + padding;
  if (insideSurrogate(text, pointermid)) {
    pointermid--;
  }
  var suffix = text.substring(patch.start2 + patch.length1,  pointermid);

And it did fix the original error I reported! But unfortunately when I ran this test on it:
// random emoji + text copied from wikipedia:
const originalText = `U+1F17x	🅰	🅱		🅾	🅿 safhawifhkw
U+1F18x															🆎
0	1	2	3	4	5	6	7	8	9	A	B	C	D	E	F
U+1F19x		🆑	🆒	🆓	🆔	🆕	🆖	🆗	🆘	🆙	🆚
U+1F20x		🈁	🈂							sfss.,_||saavvvbbds
U+1F21x	🈚
U+1F22x			🈯
U+1F23x			🈲	🈳	🈴	🈵	🈶	🈷	🈸	🈹	🈺
U+1F25x	🉐	🉑
U+1F30x	🌀	🌁	🌂	🌃	🌄	🌅	🌆	🌇	🌈	🌉	🌊	🌋	🌌	🌍	🌎	🌏
U+1F31x	🌐	🌑	🌒	🌓	🌔	🌕	🌖	🌗	🌘	🌙	🌚	🌛	🌜	🌝	🌞	`;

// applies some random edits to string and returns new, edited string
function applyRandomTextEdit(text) {
  let textArr = [...text];
  let r = Math.random();
  if(r < 1/3) { // swap
	let swapCount = Math.floor(Math.random()*5);
    for(let i = 0; i < swapCount; i++) {
	  let swapPos1 = Math.floor(Math.random()*textArr.length);
      let swapPos2 = Math.floor(Math.random()*textArr.length);
      let char1 = textArr[swapPos1];
      let char2 = textArr[swapPos2];
      textArr[swapPos1] = char2;
      textArr[swapPos2] = char1;
    }
  } else if(r < 2/3) { // remove
    let removeCount = Math.floor(Math.random()*5);
    for(let i = 0; i < removeCount; i++) {
      let removePos = Math.floor(Math.random()*textArr.length);
      textArr[removePos] = "";
    }
  } else { // add
    let addCount = Math.floor(Math.random()*5);
    for(let i = 0; i < addCount; i++) {
      let addPos = Math.floor(Math.random()*textArr.length);
      let addFromPos = Math.floor(Math.random()*textArr.length);
      textArr[addPos] = textArr[addPos] + textArr[addFromPos];
    }
  }
  return textArr.join("");
}

let dmp = new diff_match_patch();

for(let i = 0; i < 1000; i++) {
  newText = applyRandomTextEdit(originalText);
  dmp.patch_toText(dmp.patch_make(originalText, newText));
}

It produced the "URI malformed" error again: https://jsfiddle.net/78fw5cjv/1/  :(
I didn't know how to apply your fix to the prefix stuff so I wasn't able to test whether that might be required, as you suggested.

Wow, that's a very neat way of finding issues! And great work with finding another fix.
The status for me is the same though: I don't have time to look at this at the moment. Feel free to bring my changes into another branch/fork and continue there.

@josephrocca   Ok, digging a bit deeper, I've found a much simpler fix..
Basically the diffing algorithm actually has no problems with surrogate pairs,  but what does have a problem is the encodeURI that's used for the patch_toText function ,  if the prefix or suffix has half a surrogate pair it will error.
So another fix is to create a simple encodeURI that removes part surrogate pairs.
Basically its only ever going to be the first & last character that will ever have a part surrogate pair, so by checking the first character and if it's an ending surrogate pair, remove it, and then check the last character and if its a starting surrogate pair remove that.
Here is a function I've created that does this ->
function encodeURISafe(s) {
  var st = 0, en = s.length;
  var stcode = s.charCodeAt(0);
  var encode = s.charCodeAt(en - 1);
  if (stcode >= 0xDC00 && stcode <= 0xDFFF) {
    st += 1;
  }
  if (encode >= 0xd800 && encode <= 0xdbff) {
    en -= 1;
  }
  return encodeURI(s.substring(st, en));
}

Now the only place we need to use this encodeURISafe function is in the diff_match_patch.patch_obj.prototype.toString function.
Basically replace the encodeURI with encodeURISafe, and this will work.  And because prefix & suffix has a default padding of 4, removing an invalid character I don't think should cause any issues.
Also you will want to now remove @judofyr fix, as it actually does cause other issues with stack overflows using your demo.

So another fix is to create a simple encodeURI that removes part surrogate pairs.

Have you tried applying the patches to the original text? It seems to me that if you remove the erroneous surrogate character you will end up with a patch that doesn't include that character. This returns false in my console with your fix:
p = dmp.patch_make('\u{1F30D}', '\u{1F308}');
strp = dmp.patch_toText(p);
p2 = dmp.patch_fromText(strp);
console.log(dmp.patch_apply(p2, '\u{1F30D}')[0] == '\u{1F308}')

(Your fix also causes 5 test failures in this branch.)

However, the approach your taking might actually work. Just instead of removing that character, we want to add the missing part of the surrogate pair.

Another realization I just had: the coords/numbers in the patches refers to UTF-16 units, not Unicode code point units. This is probably not what the other implementations do. It seems to not cause a problem in practice because DMP is good a recovering context, but it's certainly something to have in mind if you're using DMP across languages.

Have you tried applying the patches to the original text? It seems to me that if you remove the erroneous surrogate character you will end up with a patch that doesn't include that character. This returns false in my console with your fix:

Unfortunately it was late when I got this working, and never really had chance to give a good test in this regards.  I was kind of hoping this would only be applying to prefix & suffix context, and I don't believe lengths matter here, but thinking about it there is nothing stopping a change breaking on a surrogate either.. :(

Another realization I just had: the coords/numbers in the patches refers to UTF-16 units, not Unicode code point units.

Yes, I did wonder about that.
Also does Diff patches work out the box with utf-8 anyway.  If so, there would be no reason even to encodeURI Unicode strings.  And does a standard Diff even understand URI encoding for diffs?
Also there is unicode's ZERO WIDTH JOINER: to maybe think about.  Not sure this would be an issue, as encodeURI has no issues with a lone joiner, and inside strings will just be invisible.  But would make human readable Diffs impractical.
Taking all this into consideration, it would appear the Javascript version here has many issues with unicode.  Apart from the ZERO WIDTH JOINER issue most issues could be fixed by re-writing the code to use Array.from / [...string] manipulation, but this would be a bit task. :(
So this is what I'm going to do, and in my case should fix any problems.
Basically encodeURI the whole input text.
And decodeURI any results, and also remember to encodeURI any source when applying patches before decodeURI.
IOW: keep all Diff changes in encoded format, this will fix any surrogate & offset problems.

Thanks for the work on this. We discovered this problem today while debugging a problem where certain edits crash Simplenote. The Objective-C library at least also demonstrates the same root problem whereby commonPrefix splits at code units but then toDelta tries to construct URL-encoded substrings.
Another thought on post-processing…
If instead of creating a URIEncodeSafe function could we analyze the previous diff string and look for surrogate breaks, then rewrite the diffs to secure them? It seems like those groups are safe to rewrite without having to maintain indices or other constraints.
diff_match_patch.prototype.diff_toDelta = function(diffs) {
  var text = [];
  var lastEnd = null;
  for (var x = 0; x < diffs.length; x++) {

    var thisDiff = diffs[x];
    var thisTop = thisDiff[1][0];
    var thisEnd = thisDiff[1][ thisDiff[1].length - 1 ];

    // don't end mid-surrogate-pair
    if ( isHighSurrogate( thisEnd ) ) {
      thisDiff[1] = thisDiff[1].slice(0, -1);
    }

    // don't start mid-surrogate-pair
    if ( isHighSurrogate( lastEnd ) && isLowSurrogate( thisStart ) ) {
      thisDiff[1] = lastEnd + thisDiff[1];
    }

    lastEnd = thisEnd;

    switch (diffs[x][0]) {
      case DIFF_INSERT:
        text[x] = '+' + encodeURI(diffs[x][1]);
        break;
      case DIFF_DELETE:
        text[x] = '-' + diffs[x][1].length;
        break;
      case DIFF_EQUAL:
        text[x] = '=' + diffs[x][1].length;
        break;
    }
  }
  return text.join('\t').replace(/%20/g, ' ');
};

Obviously some things here would need bounds checking but that was the idea I had. I'll try and test it out to see if I can recreate/resolve the tests cases in this thread.

The above link to https://github.com/google/diff-match-patch/pull/80 is my attempt to demonstrate working code with tests. My patch only changes the diffing where it becomes a problem - when attempting to build a valid UTF-8 string in URIEncode().
The method appears to work and I think all it needs for totality is to handle cases where our diff groups are too short to chop off the front or back of them.

@judofyr today I came back to your patch and I think the problem with the extended tests stems from the way the patch conflates high surrogates and low surrogates.
in general, we can end a group on a low surrogate and we can start a group on a high surrogate. the reverse is problematic: ending on a high surrogate and starting on a low surrogate.
that is, instead of using insideSurrogate(), if you use isHighSurrogate() in diff_commonPrefix() and isLowSurrogate() in diff_commonSuffix() then the patch should work again.
diff --git a/javascript/diff_match_patch_uncompressed.js b/javascript/diff_match_patch_uncompressed.js
index 88a702c..c731b8c 100644
--- a/javascript/diff_match_patch_uncompressed.js
+++ b/javascript/diff_match_patch_uncompressed.js
@@ -88,6 +88,15 @@ diff_match_patch.Diff.prototype.toString = function() {
   return this[0] + ',' + this[1];
 };

+diff_match_patch.prototype.is_high_surrogate = function(c) {
+  var v = c.charCodeAt(0);
+  return v >= 0xD800 && v <= 0xDBFF;
+};
+
+diff_match_patch.prototype.is_low_surrogate = function(c) {
+  var v = c.charCodeAt(0);
+  return v >= 0xDC00 && v <= 0xDFFF;
+};

 /**
  * Find the differences between two texts.  Simplifies the problem by stripping
@@ -439,6 +448,12 @@ diff_match_patch.prototype.diff_bisect_ = function(text1, text2, deadline) {
  */
 diff_match_patch.prototype.diff_bisectSplit_ = function(text1, text2, x, y,
     deadline) {
+  if (this.is_low_surrogate(text1, x)) {
+    x--;
+  }
+  if (this.is_low_surrogate(text2, y)) {
+    y--;
+  }
   var text1a = text1.substring(0, x);
   var text2a = text2.substring(0, y);
   var text1b = text1.substring(x);
@@ -569,6 +584,14 @@ diff_match_patch.prototype.diff_commonPrefix = function(text1, text2) {
     }
     pointermid = Math.floor((pointermax - pointermin) / 2 + pointermin);
   }
+
+  if (
+    this.is_high_surrogate(text1[pointermid - 1]) ||
+    this.is_high_surrogate(text2[pointermid - 1])
+  ) {
+    return pointermid - 1;
+  }
+
   return pointermid;
 };

@@ -601,6 +624,14 @@ diff_match_patch.prototype.diff_commonSuffix = function(text1, text2) {
     }
     pointermid = Math.floor((pointermax - pointermin) / 2 + pointermin);
   }
+
+  if (
+    this.is_low_surrogate(text1[text1.length - pointermid]) ||
+    this.is_low_surrogate(text2[text2.length - pointermid])
+  ) {
+    return pointermid - 1;
+  }
+
   return pointermid;
 };

diff --git a/javascript/tests/diff_match_patch_test.js b/javascript/tests/diff_match_patch_test.js
index 109e56a..ede410c 100644
--- a/javascript/tests/diff_match_patch_test.js
+++ b/javascript/tests/diff_match_patch_test.js
@@ -89,6 +89,17 @@ function testDiffCommonPrefix() {

   // Whole case.
   assertEquals(4, dmp.diff_commonPrefix('1234', '1234xyz'));
+
+  // surrogate pairs
+  assertEquals(0, dmp.diff_commonPrefix(
+    '\ud83c\udd70', // 🅰
+    '\ud83c\udd71', // 🅱
+  ));
+
+  assertEquals(2, dmp.diff_commonPrefix(
+    '\ud83c\udd70\ud83c\udd70', // 🅰🅰
+    '\ud83c\udd70\ud83c\udd71', // 🅰🅱
+  ));
 }

 function testDiffCommonSuffix() {
@@ -101,6 +112,17 @@ function testDiffCommonSuffix() {

   // Whole case.
   assertEquals(4, dmp.diff_commonSuffix('1234', 'xyz1234'));
+
+  // surrogate pairs
+  assertEquals(0, dmp.diff_commonSuffix(
+    '\ud83c\udd70', // 🅰🅰
+    '\ud83d\udd70', // 🅰🕰
+  ));
+
+  assertEquals(2, dmp.diff_commonSuffix(
+    '\ud83c\udd70\ud83c\udd70', // 🅰🅰🅰
+    '\ud83d\udd70\ud83c\udd70', // 🅰🕰🅰
+  ));
 }

 function testDiffCommonOverlap() {

All in all I think the approach in #80 is stronger because the assumption of working
on UTF-16 code units goes deep in here and some of the cleanup and merging and
bisecting code is at least hard for me to reason about when thinking of adjusting
indices.
In another test case we came up with one of the semantic cleanup functions is returning
empty patch groups when using the optimized diff method too but I couldn't find out
why. It seems like it must be related to the same Unicode encoding issue.
dmp.diff_main(
  '😃🖖🏻🖖🏻🖖🏿\n🥶🥶🥶👈🏼🖖🏻😉🖖🏽🖖🏿😉🥶🥶🥶👈🏼🖖🏻😉🖖🏽🖖🏿😉🥶🥶🥶👈🏼🖖🏻😉🖖🏽🖖🏿😉\n\n.👉🏽👉🏽👉🏽👉🏽👉🏽👉🏽👉🏽👉🏽👉🏽s👈🏿👈🏿👈🏿👈🏿👈🏿👈🏿👈🏿👈🏿👈🏿.\n🥶🥶🥶👈🏼🖖🏻😉🖖🏽🖖🏿😉🥶🥶🥶👈🏼🖖🏻😉🖖🏽🖖🏿😉🥶🥶🥶👈🏼🖖🏻😉🖖🏽🖖🏿😉\n.👉🏽👈🏿👈🏿👈🏿🥶🥶🥶🥶🥶👈🏼🖖🏻👈🏼s🖖🏻👈🏼😋\n\ndasdas\ndasvafksdldasfadsxc vzx',
  '😃🖖🏻🖖🏻🖖🏿\n🥶🥶🥶👈🏼🖖🏻😉🖖🏽🖖🏿😉🥶🥶🥶👈🏼🖖🏻😉🖖🏽🖖🏿😉🥶🥶🥶👈🏼🖖🏻😉🖖🏽🖖🏿😉\nfds\n.👉🏽👉🏽👉🏽👉🏽👉🏽👉🏽👉🏽👉🏽👉🏽s👈🏿👈🏿👈🏿👈🏿👈🏿👈🏿👈🏿👈🏿👈🏿.\n🥶🥶🥶👈🏼🖖🏻😉🖖🏽🖖🏿😉fdsvadsfewdsfdsafsdafsd'
) // produces an empty `[ DELETE, "" ]` group in the middle of the string

in general, we can end a group on a low surrogate and we can start a group on a high surrogate. the reverse is problematic: ending on a high surrogate and starting on a low surrogate.

Aah. Maybe. I always confuse what "low" and "high" means.

All in all I think the approach in #80 is stronger because the assumption of working
on UTF-16 code units goes deep in here and some of the cleanup and merging and
bisecting code is at least hard for me to reason about when thinking of adjusting
indices.

I feel like the approach in this PR is probably the "correct" way of doing it, but it requires detailed knowledge about how the algorithm works and I don't feel that comfortable with it. From a practical point of view #80 is much simpler and likelier to handle all cases. I like it! They are also not mutually exclusive. We can introduce #80 and then also try to make the core algorithm work properly on UTF16.

In another test case we came up with one of the semantic cleanup functions is returning
empty patch groups when using the optimized diff method too but I couldn't find out
why. It seems like it must be related to the same Unicode encoding issue.

Ouch. That's good to know.
The biggest problem here is the maintenance of the project. There doesn't seem to be any progress here. We at @sanity-io are considering forking the JS version and incorporating the fixes in here and/or #80, but currently it's not been a priority.

Thanks @judofyr for your quick response!

They are also not mutually exclusive.

That's been our finding too. It was that empty DIFF_DELETE group that tossed a thorn into our approach in #80 (though the fix for that should be quite simple and I hope to post an update today).
My reasoning for #80 is purely because nothing else in the code depends on or assumes valid Unicode data, so I frame this PR as "we don't have to do this but we are because it prevents a number of confusing situations" - I am quite fond of the fact that we have the liberty to make these semantically-equivalent diff changes.
Where I ran into trouble here were the places you mentioned - I don't know the algorithm well enough given the code and in some strange sense I suppose it might be easier to rewrite the entire file with a tighter Unicode assumption/constraint. _bisect and _cleanupMerge are where I had the most difficulty.
Well, my point is that I think that the approach in this PR is incomplete until we make the same surrogate checks everywhere, or at least it's hard to reason about the fix until that has been done  or we find more comprehensive test cases.

We at @sanity-io are considering forking the JS version and incorporating the fixes

I've been thinking of this too. @NeilFraser what can we do to prevent forking? How can we help you with this? It's amazing that the repository has been moved to GitHub and that you've updated so much of it. We're happy to help steward what we can since Simplenote/Simperium depends so fundamentally on diff-match-patch.

@judofyr are you using this patch at @sanity-io? I finally got around to digesting the Myers algorithm and the things that @NeilFraser has written about this library over the years (super helpful, thanks Neil!) and I'm more doubtful of this approach now.
Tonight I attempted to alter the library so that it would properly handle surrogate pairs, from the ground up if you will, and that broke down much harder than I anticipated.
This library relies heavily on array indexing which gets destroyed if the characters are variable-length, which indeed they are. The original Myers algorithm too depends on it, and possibly even more so. When I got to y1 = x1 - k1 I hit a particularly conundrum. The concept is that the index into text2 can be implicitly stored just with the knowledge of the index into text1, but we have to traverse the string differently to find the previous or successive code points and so everything gets hairy. I think we can do it if we double all the variables and track x1 and x1_u (the code-unit index into the string + the code-point index into the string) but that leads me to the next problem…
This library relies on being able to primitively scan forward and backward in a string by adding or subtracting from an index. Each time we do this and are aware of the Unicode rules we have to iterate - meaning that we turn constant operations into ones that have an unspecified number of iterations, and we do that every time we perform indexing.

My main concerns revolve around finding the middle snake and then the shortest edit script. Maybe we can get away with some pin-pointed index adjustments but I feel like it was getting out of hand as I actually started to do it.

That makes #80 more convenient and robust than I first realized. I found a Rust implementation of this library that performs Unicode boundary cleanup in the diffing process alongside the semantic and lossless cleanups. Apart from that and the fact that they are using string slices the fix was almost identical.

Independently I'd like to try splitting the text when it enters the library and then re-joining it when it leaves. That would be a manual version of [...str] for compatibility. Moving from strings to arrays carries big implications as well though, but at least our diff "alphabet" remains fixed-width.
I'm afraid that no matter what we do to fix it the performance will suffer. In other words, it's really fast because it gives the wrong answer for cases where it would otherwise be slow 🙃

So this may not be the best place to hold this discussion but I've had a hard time coming to…terms…with how the algorithm handles variable-width letters/alphabets. What I'm nervous about with in-place adjustments like pointermid-- is that we might run into situations where we shift the surrogate-splitting into different parts of the edit graph and come to an invalid sub-diff in the process. I have given a name to this method as "Unicode rewind" since it involves backing up one character when we detect an invalid Unicode boundary.
Creating the iterator was itself a bad idea because it created the two-worlds divide between JS strings and Unicode Scalar Values. It turned out to be much easier to reason about the code and "backup" when necessary (or advance), as I have been doing in https://github.com/google/diff-match-patch/commit/00260dd3a0d9451258e97cef35445b8baaf2d444. So far it looks better than I expected it to be with the backup/advance logic. Put in different words I don't consider it my work so much as my attempt at reviewing this PR, which I had to do by starting over from principle and seeing how close our patches came. I haven't reviewed the bisect or cleanup functions yet but so far most of my tests are passing which seems like a good sign.
Where did we differ?
I found that we need to perform surrogate checks in compute_ as well because otherwise we might cut out surrogates from the middle of a string. In review I think your code is right and those aren't needed because we'd have to start with an invalid string to find a whole match that ends on half a surrogate (in cases where the shorter string is entirely inside the longer string).
However, I think it may still be necessary inside halfMatch because there I'm pretty sure this case demonstrates where it might otherwise find a substring at least as long as half the length of the longer text which splits surrogate halves. In the example below h stands for a high surrogate and 1, 2, 3, etc… stand for different low surrogates.
ah0h1h2h3b
xh0h1h2h4h5x

Here the substring h0h1h2h from the shorter string exists entirely within the longer string but we can't take that whole length. When added as a test this case does indeed fail, but it appears to fail inside bisect so I'll have to review that usage on another day when I can get my focus into it.
I'm worried again though that we're going to have to start re-scanning the string inside bisect if we're going to do this right and that will become quadratic or worse, quickly.
Another idea I have been pondering is that we could scan the input string at the start of the program and for each character relate the index into the string with the index into the equivalent Unicode Scalar Value array
// how many USVs are we in?
'abc\ud83c\udd70\ud83c\udd71xyz'
[0123344567]

// convert up-front inside bisect
t1 = [ ...text1 ]
t2 = [ ...text2 ]

…

diff_bisectSplit_(t1.join(''), t2.join(''), utf16length(t1.slice(0, x1), utf16length(t2,slice(0, y1), deadline);

Right now what I see as the big problems are:

delta has to be Unicode Scalar Values, not UTF-16 code units otherwise we might stop at the wrong point when finding the "middle snake"
x1 = v1[k1_offset + 1]; and x1 = v1[k1_offset - 1] + 1; do not move over or down, they might move into the middle of a surrogate pair
var y1 = x1 - k1 may give us a y-value in the middle of a surrogate pair
text1.charAt(x1) == text2.charAt(y1)) only accounts for code units, not code points
x1++ and y1++ have obvious problems

Maybe if I can chart out every counter and label which coordinates they are using this will become clearer. In any case, to find the middle snake I think we have to perform at minimum one translation into USVs if for no other reason than to figure out delta.
That's it for me for tonight 😴

@dmsnell: Very impressive work! I'll have to admit that the cases where I added Unicode rewinding was mainly determined "experimentally": I added various test cases and tried to figure out where the patch was created. What you've done is far more extensive.
At Sanity.io we're still using the current version which is buggy. We were hoping that the work in #80 would be accepted and merged in.

I've tried all 3 solutions (https://github.com/google/diff-match-patch/pull/13, https://github.com/google/diff-match-patch/pull/69, https://github.com/google/diff-match-patch/pull/80) to this problem and https://github.com/google/diff-match-patch/pull/13 is the only one that solved the encodingURI exception for me. I created a minimal repo which clearly reproduces the issue: https://github.com/gamedevsam/diff-match-patch-emoji-issue
Hope it's helpful to make this solution more robust.


fix: addressing CVE-2022-41723, CVE-2023-25173, and CVE-2023-25153

What this PR does / why we need it:

Addresses CVE-2022-41723 which is due to indirect dependency on an older version of golang.org/x/net (mitigated in v0.7.0)
Addresses CVE-2023-25173 and CVE-2023-25153 which is due to indirect dependency on an older version of github.com/containerd/containerd (mitigated in v1.6.18)

This is to address issue #11850, #11912
Special notes for your reviewer:
N/A
If applicable:

[ ] this PR contains documentation
[ ] this PR contains unit tests
[ ] this PR has been tested for backwards compatibility

Closing as another set of PRs updated these. This is out now in v3.11.3


Change metric type to counter

Kubernetes apiserver already logs request latency metrics. Here we are more
interested in request counts per client. It makes easier to see rate() per
operator for example.

Going with a ping


Please can anybody help?

Hi I have Emulationstation on lubuntu linux. I run this program at start up and edited /etc/profile with this and put it to the bottom and save [ -n "${SSH_CONNECTION}" ] || emulationstation
I have startx as a menu item and i press that and when i do the screen turns black with an X as a cursor. It comes back into emulationstation a few seconds. When i exit emulationstation the desktop shows. How can i fix the problem of the menu item? I want when i press the item which has startx as a batch script to start the desktop. I was told to have a file called .xinitrc in my home directory i put that file in with exec lxde and i still got a black screen with cursor as an X.

Here is the log error
Attempting to launch game...
lvl2: 		bash /home/student/Desktop/desktop.sh
lvl1: 	...launch terminated with nonzero exit code 256!


Remove deploy/crd.yaml

What type of PR is this?
/kind cleanup
What this PR does / why we need it:
We now automatically ship the CRD which means we do not have to provide
it as separate deployment file any more.
Which issue(s) this PR fixes:

None
Does this PR have test?
None

Special notes for your reviewer:
None
Does this PR introduce a user-facing change?

None

/lgtm


forcing data frame cast in WildcatConnectionCSV

This will also bring us to version 1.1.0

All DataFrame values, with the exception of SubmissionID will be automatically be cast into str. This allows us to generalize to candidates whose names are solely a series of numbers. While this will never happen (outside of a major cultural upheaval), Slivka's VPs and ACs may now test candidates that are simply numbers.


additional data for Solax X3 Hybrid G3, when using a second inverter and a second meter

hello,
i'm using a second inverter with a second DTSU-666-meter.
The solax-adapter in cloud-mode report the power of this inverter as feedinpowerM2 (see log)
solaxRequest: {"success":true,"exception":"Query success!","result":{"inverterSN":"xXxXxXxXxXxX","sn":"xXxXxXxXxXxX","yieldtoday":0.7,"yieldtotal":5160.7,"feedinpower":-564,"feedinenergy":5579.32,"consumeenergy":1942.26,"feedinpowerM2":-8,"soc":10,"inverterType":"5","inverterStatus":"109","uploadTime":"2023-12-23 21:27:52","batStatus":"0"},"code":0}
In local-mode, this data is at position 100 (value: -8, see log). The unit is "Watt".
(The value is negative, because this is the standby-power of the inverter at night.)
{"type":"X3-Hybiyd-G3","SN":"xXxXxXxXxXxX","ver":"2.033.20","Data":[0,0,0,0,0,0,0,0,0.7,5160.7,-571,0,0,0,0,0,23,0.9,0,1385,0,10,216,174,25,25,17,16,3.1,3,165,1,0,0,0,0,0,0,0,0,0,5579.32,1942.18,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,9,1,1,1,900,100,10,20,25,0,0,0,0,443.4,1818.2,5568.3,0,0,2,0,0,0,0,0,0,0,0,0,1,1,1,-8,1185.49,17.29,1,273,190.6,0,0],"Information":[6,5,"X3-Hybiyd-G3","xXxXxXxXxXxX",1,4.71,0,4.53,1.07],"battery":{"brand":"1","masterVer":"5.06","slaveNum":"4","slaveVer":[0,0,0,0,0,0,0,0],"slaveType":[0,0,0,0,0,0,0,0]}}
Is it possible, to integrate this value in a future version of the adapter?
Thank you
Frank

Fix in the next Version


[bug] Disconnecting wallet from wagmi hook doesn't trigger SIWE on reconnect

Link to minimal reproducible example
https://codesandbox.io/p/devbox/vite-react-forked-5hydhc
Summary
Hello everyone,
I’m encountering a bug since migrating to the latest version of @reown/appkit. When I disconnect my wallet from the AppKit interface, everything works fine, and the SIWE authentication is properly triggered. However, if I disconnect the wallet using the wagmi hook useDisconnect, as indicated in the documentation, the SIWE trigger does not occur when I try to reconnect.
I created a CodeSandbox to reproduce the bug in question.
Thanks for your help.
List of related npm package versions
"@reown/appkit": "^1.0.6",
"@reown/appkit-adapter-wagmi": "^1.0.6",
"@reown/appkit-siwe": "1.0.6",
"siwe": "2.3.2",
"viem": "^2.21.15",
"wagmi": "^2.12.14"

https://github.com/reown-com/appkit/pull/3121/files @chaffy999 @2facedsquid
this is the first disconnect improvement we're pushing in the next release. We'll make sure your issue is fixed, too as part of this or shortly thereafter. thank you!

the fix was released in in v.1.1.7 - customers reported the fix works - closing


Update https://www.scala-sbt.org/download.html about 1.4.6

The latest version mentioned at https://www.scala-sbt.org/download.html
is still 1.4.5.
But 1.4.6 has been released:
https://github.com/sbt/sbt/releases

This was fixed in https://github.com/sbt/website/pull/979

This was fixed in https://github.com/sbt/website/pull/979


Automated Package Upgrade

:tada: This PR is included in version 5.0.557 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Are there Assembly informations for the .Net Framework?

Hello, I have a question to the library as a whole:
When using a standard console Application, the library works just fine! However, I am unable to install it on a winforms application using the .NET Framework 4.6.2 and upwards. The console is throwing an error, stating that there are no assembly informations for the project. Am I doing something wrong or is there a simple way to fix that?

Is there a reason why they can't use .NET 5/6?

I have push a new package with .netstandard 2.0 support with this package you have support for net 4.6.1
https://docs.microsoft.com/en-us/dotnet/standard/net-standard

The new package is available can you test with it again?

I will do that right away

Works! Thanks!

We would love a star for our project ⭐


Lua Hooks: lakefs.delete_branch operation

Created at
2023-11-09T12:57:25.000Z
Priority
p3
PRD
https://github.com/treeverse/lakeFS/issues/6884
Feature Definition
T-shirt size
S
Related feature requests
https://github.com/treeverse/product/issues/86

@talSofer this looks like a duplicate of https://github.com/treeverse/lakeFS/issues/6884
Can we close it?

@N-o-Z yes, closing it
deplicates https://github.com/treeverse/lakeFS/issues/6884


Remove requirement for sources to have a holding institution if source is a reconstruction

As is being discussed in #1630, the holding institution relationship on a source is ill-suited for source reconstructions. While source reconstructions may, at some point, require additional attention in terms of how they are modeled, at present they can be accommodated by removing the requirement for a source to have a holding institution.

[ ] Make "Reconstruction" an option in the full_source field. (Change to ChoiceField).
[ ] Remove the requirement that a source have a holding institution relationship

Right now full_source is a Boolean field in the database, but a choice field in the form. This seems really weird.

Hmm... that is weird. Yeah, this would be a good chance to clean that up. Revised steps above.

Decided with Debra, Anna, Lucas that no source should have to have a holding institution.


Improve parse for module sources

This changes the parsing behavior of DeclarationReference.parse when parsing module sources and components:

@ is not allowed in an un-quoted symbol component (i.e. @a is not valid, but "@a" is).
@microsoft/rush-stack-compiler-3.5 will throw, as @ and 5 are not valid identifiers and there is no trailing ! token.

NOTE: While we could detect the leading @ and treat it as a path, I believe it is better to be unambiguous. An @ is only allowed if we scan ahead and find a ! token.

@microsoft/rush-stack-compiler-3.5! will parse with a module source of @microsoft/rush-stack-compiler-3.5 as we will scan ahead and find the ! token.
If for some reason a module source includes a ! token in its text, the module source must be quoted (i.e. a!b!c is an error, but "a!b"!c is valid, with "a!b" as the module source).

Also there are a few additional changes to make the API easier to use:

DeclarationReference.isWellFormedModuleSourceString(text) - Returns whether the text is a valid module source.
DeclarationReference.escapeModuleSourceString(text) - Surrounds a module source in quotemarks if it is not well-formed.
DeclarationReference.unescapeModuleSourceString(text) - Unquotes a module source if necessary.
ModuleSource#escapedPath - Preserves the module source as written (including quotemarks). This was previously the value for ModuleSource#path
ModuleSource#path - The unescaped module source.
ModuleSource#scopeName - The scope name for the module, if it has one. Includes the leading @.
ModuleSource#unscopedPackageName - The package name excluding the scope name.
ModuleSource.fromScopedPacage(scopeName, unscopedPackageName, importPath) - Create a module source from a scope.

I've also added several suites of tests for various behaviors.
Fixes #174

@octogonz do you have some time to take a look?

I looked over the code this weekend, but I wanted to test it with my PR branch before we merge it. I should have time in the next few days. I've been super busy with another project this past week.


Cannot install Python extension due to version conflict installing ms-toolsai/jupyter extension

Environment data

VS Code version: 1.60.0
Extension version (available under the Extensions sidebar): v2021.8.1159798656
OS and version: macOS 11.5.2
Python version (& distribution if applicable, e.g. Anaconda): Python 3.9.7 from home-brew
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A
Relevant/affected Python packages and their versions: N/A
Relevant/affected Python-related VS Code extensions and their versions: ms-toolsai/jupyter, I don't know its version because the error doesn't report that.
Value of the python.languageServer setting: N/A (I can't install the extension!)

[NOTE: If you suspect that your issue is related to the Microsoft Python Language Server (python.languageServer: 'Microsoft'), please download our new language server Pylance from the VS Code marketplace to see if that fixes your issue]
Expected behaviour
Clicking "install" in the extensions marketplace for the ms-python extension, in the current version of VS Code, installs the Python extension.
Actual behaviour
It does not install. Instead, the following alert is presented:

Can't install 'ms-toolsai.jupyter' extension because it is not compatible with the current version of VS Code (version 1.60.0).

BTW this problem also manifested in version 1.59.1 of VS Code.
Steps to reproduce:

navigate to extensions marketplace
search for "Python"
Try to install the Python extension

Logs
I see no relevant logs in the Extensions or Logs (Extension Host) output.

@iamleeg just to be sure, you're clicking install here?

And then clicking install when this comes up?

This all works for me with the latest VS code stable.
@karthiknadig Not sure how this is a jupyter extension problem as we're not involved at all here.


Fix some typos in docs

Thanks for great docs about RabbitMQ, upon reading I've found some typos, this PR fixes these typos.😇

@charltonliv Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

I will apply this manually, thank you.


📝 docs/deploy.md

#43  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @RTLS and the rest of your teammates on  Graphite


PowerBI and Jira Search connectors

First pull request ever - be gentle :-)

When submitting a connector, please check the following conditions for your PR.

[x] I attest that the connector works and I verified by deploying and testing all the operations.
[x] I validated the swagger file, apiDefinition.swagger.json, by running paconn validate command.
[x] I confirm that apiProperties.json has a valid brand color and doesn't use an invalid brand color, #007ee5 or #ffffff.

Well done @paulculmsee! Congrats on the first PR! 🥳

Will resubmit as 2 separate connectors


changes to data loading process

This PR include a few changes to the data utils file to allow for loading further information from the datasets (additional information about items and users). It also adds the lastfm-360K dataset.

LGTM :)


e2e: disable GatewayInfraResourceTest

this cause too much flaky, let's disable it first.
cc @ardikabs

/retest


Alternate linestyles

Hey, so I've had another issue, namely with using linestyle for grouping datasets.
I've got 2 datasets which need to separated by linestyle, but the 2 first default linestyles solid line ('-') and dashed line ('--'), which, under the default legend size, look pretty much the same:

this is firstly a bug that needs to be taken care of, but more importantly it highlights the fact that the user does not have the freedom to specify the linestyles used at any point, so as to avoid this kind of mix-up.

Ugly dashed lines is almost a matlab bug, but I see what you mean. Until I add an option to set the line style order (I'm away these weeks), you can modify the line:
line_styles={'-' '--' ':' '-.'};
In draw.m
Cheers

Awesome, works like a charm!
Thanks for the super fast reply rate BTW!

Now available with set_line_options() along with size preferences.


Show the current image in the top bar

As mentioned in https://github.com/jtpio/jupyterlab-topbar/issues/35, we could make use of the jupyterlab-topbar-text extension to show to the users the image they are currently using.
This means adding the jupyterlab-topbar-text to the dependencies here: https://github.com/plasmabio/template-python/blob/master/postBuild
And define a pre_spawn_hook in the plugin to update the lab settings.
Something that would look like the following:

That is definitely a good idea to remind user in which environment/container he's working in.
:+1:

@jtpio do you have any idea how jupyterlab-topbar-text could be aware of the environment name?
Could you do a demo with template-python? I'll try to propagate it with other kernels.

We'll probably want to place this in the entrypoint, to be able to update the name of the image dynamically on server startup.
Something similar to the approach mentioned in https://github.com/jtpio/jupyterlab-topbar/issues/35#issuecomment-580713551, by creating the settings file in the .jupyter/lab/user-settings/jupyterlab-topbar-text of the environment folder.


add readme for safe paste plugin

I've add a readme for safe-paste plugin with a description of this plugin
Refer #7175

Thanks


Some modules resolve as undefined if old .cache is present

If the old .cache/[config hash]/ folder is present - some modules resolve to undefined so build breaks. I assume that something is off with restoring modules from the cache files.
Environment:

Mac OSX El Capitan
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
Plugin's config:
// ...
new HardSourcePlugin({
    cacheDirectory: path.normalize(`/.cache/[confighash]/`),
    recordsPath: path.normalize(`${dirname}.cache/[confighash]/records.json`),
    configHash: function (webpackConfig) {
        const hash = require('node-object-hash')().hash(webpackConfig);
        return hash;
    },
    environmentHash: {
        root: process.cwd(),
        directories: ['node_modules'],
        files: ['package.json']
    }
})
// ...

I understand that this is probably not enough info to track down this issue, so will be happy to provide more upon your request.

What other plugins/loaders are you using? And can you share their configuration? Most issues with hard-source so far are are plugins I haven't yet tested against yet that need a minor change in hard-source to support. 0.3.9 for example was a minor change to support webpack.IgnorePlugin.

Closing since this never got more info on the issue to nail down the problem.


Add convenience methods to Sort.java

These methods are meant to make it easier to initialize sort stats that are often repetitive. For example, the values passed to setSortListName, setRunAllSortsName, and setRunSortName are often very similar.

Ima actually close this, because I have something even better in mind.


[Snyk] Upgrade eslint-config-next from 13.4.12 to 13.4.18

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to upgrade eslint-config-next from 13.4.12 to 13.4.18.
:information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project.

The recommended version is 36 versions ahead of your current version.
The recommended version was released 22 days ago, on 2023-08-18.

Release notes

Package name: eslint-config-next

13.4.18 - 2023-08-18Core Changes

refactor: remove edge condition for module proxy path: #54167
Remove unused variables: #54149
chore: improve ts types for position in dev-build-watcher: #54124
Turbopack: Strip relative path prefix when generating PageLoaderAsset: #54040
Add size property to ReadonlySearchParams: #53144
Assign default not-found boundary if custom not-found is not present for root layer only: #54185
Allow range version for eslint config: #53751
Automatically modularizeImports for the popular @ headlessui/react library: #54188
fix bfcache restoration behavior: #54198

Misc Changes

Update rust toolchain: #54130

Credits
Huge thanks to @ huozhi, @ shuding, @ styfle, @ jridgewell, @ bencmbrook, @ cramforce, and @ ztanner for helping!

13.4.18-canary.0 - 2023-08-17Core Changes

refactor: remove edge condition for module proxy path: #54167
Remove unused variables: #54149
chore: improve ts types for position in dev-build-watcher: #54124
Turbopack: Strip relative path prefix when generating PageLoaderAsset: #54040
Add size property to ReadonlySearchParams: #53144
Assign default not-found boundary if custom not-found is not present for root layer only: #54185
Allow range version for eslint config: #53751
Automatically modularizeImports for the popular @ headlessui/react library: #54188
fix bfcache restoration behavior: #54198

Misc Changes

Update rust toolchain: #54130

Credits
Huge thanks to @ huozhi, @ shuding, @ styfle, @ jridgewell, @ bencmbrook, @ cramforce, and @ ztanner for helping!

13.4.17 - 2023-08-17 Read more

13.4.17-canary.2 - 2023-08-17Core Changes

Revert "fix(47299): allow testing pages with metadata in jsdom test environment": #54160

Credits
Huge thanks to @ huozhi for helping!

13.4.17-canary.1 - 2023-08-17Core Changes

node-web-streams: remove tee shim, use ReadableStream.tee: #54079
Debug tracing: Include navigation through hydration timing: #54078
fix: cookies().has() breaks in app-route: #54112

Credits
Huge thanks to @ MaxLeiter, @ wbinnssmith, and @ devjiwonchoi for helping!

13.4.17-canary.0 - 2023-08-16 Read more

13.4.16 - 2023-08-15Core Changes

Concept: test mode for Playwright and similar integration tools: #52520
Turbopack: fix hiding node_modules warnings in error overlay.: #54022
ci(next-swc): print glibc version when build: #54026
Adjust internal action proxy export: #54004

Documentation Changes

Update 05-client-side-rendering.mdx with latest tanstack query version: #54009
Open Graph Image font declaration moved to correct place: #53998
Update opengraph-image.mdx: Fix typo: #54020

Misc Changes

Remove extra label from runner: #54002
add standalone testcase for ipv6 hostnames: #53999
release: add release log generation script: #54006
test(ci): refine test suite name unique: #54013
Leverage previous swc build images: #54027
chore: mark build folder indexable: #54029
Move turbo outside of build for docker swc builds: #54035

Credits
Huge thanks to @ ijjk, @ ztanner, @ huozhi, @ lacymorrow, @ dvoytenko, @ kylemcd, @ kwonoj, @ tibi1220, @ wbinnssmith, and @ shuding for helping!

13.4.16-canary.1 - 2023-08-15 Read more

13.4.16-canary.0 - 2023-08-14 Read more

13.4.15 - 2023-08-14 Read more

13.4.15-canary.0 - 2023-08-14

13.4.14-canary.5 - 2023-08-14

13.4.14-canary.4 - 2023-08-14

13.4.14-canary.2 - 2023-08-11

13.4.14-canary.1 - 2023-08-08

13.4.14-canary.0 - 2023-08-07

13.4.13 - 2023-08-07

13.4.13-canary.18 - 2023-08-07

13.4.13-canary.17 - 2023-08-06

13.4.13-canary.16 - 2023-08-05

13.4.13-canary.15 - 2023-08-04

13.4.13-canary.14 - 2023-08-04

13.4.13-canary.13 - 2023-08-03

13.4.13-canary.12 - 2023-08-02

13.4.13-canary.11 - 2023-08-02

13.4.13-canary.10 - 2023-08-02

13.4.13-canary.9 - 2023-08-01

13.4.13-canary.8 - 2023-07-31

13.4.13-canary.7 - 2023-07-31

13.4.13-canary.6 - 2023-07-28

13.4.13-canary.5 - 2023-07-28

13.4.13-canary.4 - 2023-07-27

13.4.13-canary.3 - 2023-07-26

13.4.13-canary.2 - 2023-07-25

13.4.13-canary.1 - 2023-07-25

13.4.13-canary.0 - 2023-07-24

13.4.12 - 2023-07-21

    from eslint-config-next GitHub release notes

Commit messages

Package name: eslint-config-next

87b66f6 v13.4.18
89ed4dd v13.4.18-canary.0
c676f93 fix bfcache restoration behavior (#54198)
631018a Automatically modularizeImports for the popular @ headlessui/react library (#54188)
7114929 Allow range version for eslint config (#53751)
e20c8c8 Assign default not-found boundary if custom not-found is not present for root layer only (#54185)
847c14e Add `size` property to `ReadonlySearchParams` (#53144)
11dfdf8 Turbopack: Strip relative path prefix when generating PageLoaderAsset (#54040)
8def0c8 Update rust toolchain (#54130)
3fadba5 chore: improve ts types for `position` in `dev-build-watcher` (#54124)
df1de76 Remove unused variables (#54149)
bc87328 refactor: remove edge condition for module proxy path (#54167)
2360753 v13.4.17
645ba00 v13.4.17-canary.2
02c258d Revert "fix(47299): allow testing pages with metadata in jsdom test environment" (#54160)
fc3bf37 v13.4.17-canary.1
0431147 fix: `cookies().has()` breaks in app-route (#54112)
bd5b715 Debug tracing: Include navigation through hydration timing (#54078)
6767454 node-web-streams: remove tee shim, use ReadableStream.tee (#54079)
93259ca v13.4.17-canary.0
8589a6d Next build: use exported `handle_issues` from turbopack (#52972)
4f7fc05 cleanup: remove unnecessary effect dep (#54134)
bdd0f54 OpenTel: remove the internal (ipc) fetched from traces in a non-verbose mode (#54083)
af97755 fix(edge): override init when cloning with `NextRequest` (#54108)

Compare

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs.
For more information:
🧐 View latest project report
🛠 Adjust upgrade PR settings
🔕 Ignore this dependency or unsubscribe from future upgrade PRs

Pull request merged

This is an automated message from Fork, Commit, Merge - web [BOT].

Thank you for your contribution! Your pull request has been merged and your name has been added to the List of Contributors in README.md.
Please leave a star if you enjoyed working on this project.
What's next?
If you're looking for more ways to contribute, I invite you to check out my other projects here.


As a Qiskit Developer, I need the UI shell component available via a custom CDN so that I can integrate it into my HTML project

Background:
The unified top menu has been released on all the pages within Qiskit.org other than the documentation, the below will address the final blocker we have preventing this from getting released to the documentation pages.
Acceptance Criteria:

Need to set up a CDN in IBM cloud -> link to tutorial
Need to make UI shell component available via the CDN
Set up automatic deployment when to the CDN when changes are made in the web components (so that users of the component via the CDN can get the latest versions)
Document to users how to use the component via the CDN

Related open issues:

https://github.com/carbon-design-system/carbon-for-ibm-dotcom/issues/6361

Additional context
We are unable to use the existing third party CDN because it is incompatible with certain dependencies in the carbon component, this is a blocker for this open issue: https://github.com/Qiskit/qiskit_sphinx_theme/issues/64
In order to have the unified navbar on the documentation page, this blocker needs to be addressed. Additionally, if we want to be able to use web components within the docs in the future, we will need components available via the CDN.
cc @javabster @abdonrd

As mentioned in the last sprint planning, creating a CDN (and maintaining!) takes time and effort that the development team cannot address in the short term.
We have tried to use a public CDN like: https://jspm.org
And it should work fine, but there is an issue:
We have carbon-web-components as a dependency. And they use Lit v1 (lit-html@1 & lit-element@2).
And we use Lit v2 (lit-html@2 & lit-element@3).
And the CDN doesn't seem to be able to resolve dependencies well, because it only use Lit v2. And then, the components from carbon-web-components don't work.
But the plan to update carbon-web-components to Lit v2 is for this quarter! 🎉

https://github.com/carbon-design-system/carbon-web-components/issues/1042
https://github.com/carbon-design-system/carbon-for-ibm-dotcom/issues/6361#issuecomment-1277538116

Meantime, I think the easy idea is bundle the UI Shell on our side and add it manually to the docs site. We have this branch that creates the bundle:
https://github.com/Qiskit/web-components/compare/ui-shell-bundle
So the docs site can import ui-shell-bundle/index.js and should work.

Hi @abdonrd, there are a few reasons I think we should consider the CDN route more seriously:

The carbon-web-components haven't given us a concrete date for the release other than 'some point in Q4'. If we want to make real progress with documentation before the end of the year I don't think we can wait around for their release (also it's possible the issue could get bumped to a later release or the release date delayed, who knows 🤷♀ )

the way the sphinx theme works is that we make changes to the theme, release the package to pypi, then every qiskit repo needs to pip install it to start using the latest version of the theme. So if we use the bundled file for the ui shell component then every time the ui shell component gets updated here we will have to update the bundle file in the sphinx theme repo, release it again and every qiskit team will need to bump their version of the theme package. This is a lot of overhead for many teams and will just aggravate the problem of there being a lack of consistent UI between qiskit.org and the other qiskit.org/documentation pages.  It would be much much better if we didn't have to do this process and could just automatically update the component in qiskit.org/documentation when it gets updated here

(maybe can resolve this with some help) I have been trying for a while now to get the bundled file to work in the sphinx theme repo but it's just not working

Attached the PR following https://github.com/Qiskit/web-components/issues/172#issuecomment-1290930826:

https://github.com/Qiskit/qiskit_sphinx_theme/pull/116

the way the sphinx theme works is that we make changes to the theme, release the package to pypi, then every qiskit repo needs to pip install it to start using the latest version of the theme. So if we use the bundled file for the ui shell component then every time the ui shell component gets updated here we will have to update the bundle file in the sphinx theme repo, release it again and every qiskit team will need to bump their version of the theme package. This is a lot of overhead for many teams and will just aggravate the problem of there being a lack of consistent UI between qiskit.org and the other qiskit.org/documentation pages.  It would be much much better if we didn't have to do this process and could just automatically update the component in qiskit.org/documentation when it gets updated here

I think will be the same case... Because you still need to manage the versions.
Link example: https://generator.jspm.io/#U2NgYGBkLM3ULc5IzclhcCjMLM7OLNEvT03STc7PLcjPS80rKXYw0LPUM9KHqQIAgTBUfDQA
With this you are using<EMAIL_ADDRESS>When we have<EMAIL_ADDRESS>you also need to update the script.
<script type="importmap">
{
  "imports": {
    "@qiskit/web-components/ui-shell": "https://ga.jspm.io/npm<EMAIL_ADDRESS>  },
  "scopes": {
    "https://ga.jspm.io/": {
      "@babel/runtime/helpers/esm/decorate": "https://ga.jspm.io/npm<EMAIL_ADDRESS>      "@babel/runtime/helpers/esm/defineProperty": "https://ga.jspm.io/npm<EMAIL_ADDRESS>      "@babel/runtime/helpers/esm/get": "https://ga.jspm.io/npm<EMAIL_ADDRESS>      "@babel/runtime/helpers/esm/getPrototypeOf": "https://ga.jspm.io/npm<EMAIL_ADDRESS>      "@babel/runtime/helpers/esm/objectSpread2": "https://ga.jspm.io/npm<EMAIL_ADDRESS>      "@lit/reactive-element": "https://ga.jspm.io/npm<EMAIL_ADDRESS>      "@lit/reactive-element/decorators/": "https://ga.jspm.io/npm<EMAIL_ADDRESS>      "carbon-components/es/globals/js/misc/on"<EMAIL_ADDRESS>      "carbon-components/es/globals/js/settings"<EMAIL_ADDRESS>      "carbon-web-components/es/components/ui-shell/"<EMAIL_ADDRESS>      "lit"<EMAIL_ADDRESS>      "lit-element"<EMAIL_ADDRESS>      "lit-element/lit-element.js"<EMAIL_ADDRESS>      "lit-html"<EMAIL_ADDRESS>      "lit-html/directives/class-map"<EMAIL_ADDRESS>      "lit-html/directives/if-defined"<EMAIL_ADDRESS>      "lit-html/"<EMAIL_ADDRESS>      "lit/"<EMAIL_ADDRESS>      "tslib"<EMAIL_ADDRESS>    },
<EMAIL_ADDRESS>{
      "lit-html"<EMAIL_ADDRESS>    },
<EMAIL_ADDRESS>{
      "lit-html"<EMAIL_ADDRESS>    }
  }
}
</script>

<!-- ES Module Shims: Import maps polyfill for modules browsers without import maps support (all except Chrome 89+) -->
<script async<EMAIL_ADDRESS>crossorigin="anonymous"></script>

<script type="module">
  import '@qiskit/web-components/ui-shell';
</script>

Maybe @Qiskit/qiskit-digital-devs want to give their opinions?

Meanwhile we also bundled the ui-shell before publishing to npm:

https://github.com/Qiskit/web-components/pull/177

So we can replace the local import with a CDN: https://github.com/Qiskit/qiskit_sphinx_theme/pull/116/commits/37de7268cca510a306648c21f99466441a567833


Issue: NoSuchEntityException when calling ListUserPolicies (AWS IAM)

Title: NoSuchEntityException when calling ListUserPolicies (AWS IAM)
Description:

What issue is being seen?  Cartography should not crash on NoSuchEntityException

To Reproduce:

Boto3 returns NoSuchEntityException when the ListUserPolicies function is provided a user that doesn't exist.
Specifically this happens in our situation with service users created by vault. Perhaps a timing issue.

Logs:
Traceback (most recent call last):
  File "/usr/local/bin/cartography", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/cartography/cli.py", line 308, in main
    return CLI(default_sync, prog='cartography').main(argv)
  File "/usr/local/lib/python3.7/site-packages/cartography/cli.py", line 288, in main
    return cartography.sync.run_with_config(self.sync, config)
  File "/usr/local/lib/python3.7/site-packages/cartography/sync.py", line 145, in run_with_config
    return sync.run(neo4j_driver, config)
  File "/usr/local/lib/python3.7/site-packages/cartography/sync.py", line 71, in run
    stage_func(neo4j_session, config)
  File "/usr/local/lib/python3.7/site-packages/cartography/util.py", line 58, in timed
    return method(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/__init__.py", line 144, in start_aws_ingestion
    _sync_multiple_accounts(neo4j_session, aws_accounts, config.update_tag, common_job_parameters)
  File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/__init__.py", line 92, in _sync_multiple_accounts
    _sync_one_account(neo4j_session, boto3_session, account_id, sync_tag, common_job_parameters)
  File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/__init__.py", line 25, in _sync_one_account
    iam.sync(neo4j_session, boto3_session, account_id, sync_tag, common_job_parameters)
  File "/usr/local/lib/python3.7/site-packages/cartography/util.py", line 58, in timed
    return method(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/iam.py", line 623, in sync
    sync_users(neo4j_session, boto3_session, account_id, update_tag, common_job_parameters)
  File "/usr/local/lib/python3.7/site-packages/cartography/util.py", line 58, in timed
    return method(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/iam.py", line 515, in sync_users
    sync_user_inline_policies(boto3_session, data, neo4j_session, aws_update_tag)
  File "/usr/local/lib/python3.7/site-packages/cartography/util.py", line 58, in timed
    return method(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/iam.py", line 531, in sync_user_inline_policies
    policy_data = get_user_policy_data(boto3_session, data['Users'])
  File "/usr/local/lib/python3.7/site-packages/cartography/util.py", line 58, in timed
    return method(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/iam.py", line 83, in get_user_policy_data
    policies[arn] = {p.name: p.policy_document["Statement"] for p in resource_user.policies.all()}
  File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/iam.py", line 83, in <dictcomp>
    policies[arn] = {p.name: p.policy_document["Statement"] for p in resource_user.policies.all()}
  File "/usr/local/lib/python3.7/site-packages/boto3/resources/collection.py", line 83, in __iter__
    for page in self.pages():
  File "/usr/local/lib/python3.7/site-packages/boto3/resources/collection.py", line 166, in pages
    for page in pages:
  File "/usr/local/lib/python3.7/site-packages/botocore/paginate.py", line 255, in __iter__
    response = self._make_request(current_kwargs)
  File "/usr/local/lib/python3.7/site-packages/botocore/paginate.py", line 332, in _make_request
    return self._method(**current_kwargs)
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 635, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.NoSuchEntityException: An error occurred (NoSuchEntity) when calling the ListUserPolicies operation: The user with name <SANITIZED_USERNAME> cannot be found.

Please complete the following information::

Cartography release version: 0.22
Python version: 3.7.5
OS: Tested on OSX and Ubuntu. Not relevant in any case.

Suggested Solution:
https://github.com/lyft/cartography/blob/master/cartography/intel/aws/iam.py#L83 is the failure point.
I'd like to help contribute to this project and fix this issue.
Want to get your thoughts on below fix to make sure it's in line with your error handling methods.
Possible solution:
def get_user_policy_data(boto3_session, user_list):
    resource_client = boto3_session.resource('iam')
    policies = {}
    for user in user_list:
        name = user["UserName"]
        arn = user["Arn"]
        resource_user = resource_client.User(name)
        try:
            # if we'd like default empty policies the commented line below could be used.
            # policies[arn] = {}
            policies[arn] = {p.name: p.policy_document["Statement"] for p in resource_user.policies.all()}
        except botocore.errorfactory.NoSuchEntityException as e:
            logger.debug("Error occurred getting user policies due to boto3 NoSuchEntity", exc_info=True)
            logger.error(
                (
                    "Unable to get AWS account number, an error occurred: '%s'. Make sure your AWS credentials are "
                    "configured correctly, your AWS config file is valid, and your credentials have the SecurityAudit "
                    "policy attached."
                ),
                e,
            )

    return policies

Thanks for bringing this up. This looks like essentially the same problem as #277 and #285 but for user policies; we can't get the policies of a user if they don't exist anymore haha :(.
As for a solution, your idea is right. I'd recommend copying https://github.com/lyft/cartography/pull/278.
Very much appreciate your help!


Vale Linter edits

Various edits based on what the Vale Linter is picking up.

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/9b0998824c41a59b1b94a2a3c9ddd07115cf4d43/


ocamlfind dependency?

While wanting to try out olly on a PR against trunk I spotted a dependency on ocamlfind in the opam file.
I'm unsure if it is a true dependency or legacy/copy-paste line? :thinking:
(The dune files do not seem to require ocamlfind)

Fixed in #36.


Reject nodes with overlapping IP addresses

This is a larger change than expected, but most of it is in test code. We should
refactor and consolidate some of the test setup code into *Tester classes, as
there's currently a lot of repetition.
Notable changes:

Everything related to IP addresses is now contained in the IP class.

New class, LockedNodeList, for code that expects a consistent state of the
entire node repository. This is just a wrapper around a NodeList and
Mutex.

Test failure:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project athenz-identity-provider-service: Compilation failure: Compilation failure:
[ERROR] /source/athenz-identity-provider-service/src/main/java/com/yahoo/vespa/hosted/athenz/instanceproviderservice/instanceconfirmation/InstanceValidator.java:[151,49] cannot find symbol
[ERROR]   symbol:   method ipAddresses()
[ERROR]   location: variable node of type com.yahoo.vespa.hosted.provision.Node
[ERROR] /source/athenz-identity-provider-service/src/main/java/com/yahoo/vespa/hosted/athenz/instanceproviderservice/identitydocument/IdentityDocumentGenerator.java:[63,49] cannot find symbol
[ERROR]   symbol:   method ipAddresses()
[ERROR]   location: variable node of type com.yahoo.vespa.hosted.provision.Node

Thanks, will fix (plus some others).

Noticed that the implementations of HostProvisioner update IP addresses, but do not have access to NodeRepository. Will try a different (and hopefully less invasive) approach.


Entire chapters truncated

Hi,
I am experiencing entire chapters being truncated. However, no error appears when running the tool.
Unsure if this relates to #103 .

From what I can see the truncated chapter appears as if one isn't logged in, i.e., is just the chapter preview.

Seems exactly like that, yes.

I'm having exact same issues.  Some chapters download perfectly, others are obviously trial mode where they will trail off with a ..., others are entirely blank.
But still most of the chapters come thru perfectly!  And I am using valid credentials.

Yep, I had the same issue.
I investigated and found that by avoiding updating the 'sessionid' in the cookie attribute fix the problem.
So replacing the method/function update_cookies by
    def update_cookies(self, jar):
        for cookie in jar:
            if cookie.name != 'sessionid':
                self.cookies.update({
                    cookie.name: cookie.value
                })

fixed the issue.
Having the sessionid updating after each request prevents the retrieval of other pages/chapters/section to be done correctly.

Yep, I had the same issue.
I investigated and found that by avoiding updating the 'sessionid' in the cookie attribute fixed the problem.
So replacing the method/function update_cookies by
    def update_cookies(self, jar):
        for cookie in jar:
            if cookie.name != 'sessionid':
                self.cookies.update({
                    cookie.name: cookie.value
                })

fixed the issue.
Having the sessionid updating after each request prevents the retrieval of other pages/chapters/section to be done correctly.

Works nicely, you should do a PR :)

yep, I'm  planning to to it (just testing it a little bit more) ;)


OpenStreetMap import will break if links aren't updated before May 7

http://planet.openstreetmap.org is available over https, and will start redirecting to https on 2018-05-07.
Your project seems to be using curl, which will not follow redirects by default.
Would it please be possible to update all files from http://planet.openstreetmap.org to httpS://planet.openstreetmap.org before then?
Ideally, would it be possible to make all requests to openstreetmap.org over https?

fixed! thanks for the heads up


Contrast Error in Issue #439

I fixed the contrast error in issue #439 after  changing the color and background color, I ran a wave accessibility test and it came out with zero contrast error however, I couldn't save the changes to the _site folder as gitpod doesn't track it.

This was fixed in #500. Thanks


Number of motions computation

Hello, I have a question regarding the method used to compute the number of motion. While investigating the Motiondataset and dataloader, it appears that the motion numbers during training are counted as the total number of motion snippets across all training subset, divided by the batch size. I can understand the reasoning behind this approach, but it leads to a significant reduction on the real data size. Specifically, the number of motions in the training set was initially 23,384. After removing motions under window-size 64, it decreased to 20,942, and further reduced to 14,435 training motions using the aforementioned method.
In your paper the numberof motion mentioned and total number of description seems different and less than those of the training
Your clarification on this behavior would be greatly appreciated. Thank you.

OK, the number is instead referred to # iterations per epoch


Admin Dashboard for Deactivated

Add admin dashboard page for deactivated users. Work on services, events, applications related to it.

Deactivated page is prepared.


Publish 02/02/2022, 10:30 AM

Docs Build status updates of commit 748198f:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/New-DlpComplianceRule.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/Set-DlpComplianceRule.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Provide a way to lock ENTRYPOINT

Unless I missed something, it is not possible to create an image and lock down ENTRYPOINT. With the following command, a user can bypass any instruction that may be required by the image to be performed in order to properly execute:
    $ docker run --entrypoint some-other-existing-script.sh someimage

It must be possible for an image author to define and lock an entrypoint to make sure a given image is executed by only calling that entrypoint definition and whatever operations that call allows.
It is also important to ensure that the ENTRYPOINT of an image is always called, either when creating a new container out of this image, or of an extended image. To verify this use case, I created this Test Scenario. Run the script test.sh.
Here's the output:
bruno@lenovo:~/test-lock-entrypoint$ sh test.sh
Untagged: entrypoint:latest
Untagged: overridden:latest
Deleted: ccbeaae06c6d9236727cfa0293e26acaa912df691f7dfa9014b31d02148de66f
Deleted: 590ab72673b0e94d815d0370b53c358dce108b9c26366d32ddc70640a0909e6f
Deleted: 1427d84e7af9c86d173e852157389c7bf650d9b4e1b953e8516a15f6111cd933
Deleted: 05efde4dcccc7f9c8fdb167467424543453dbd2ade7575b316ceb3678a821932
Deleted: 2887b23be62d48d530b077e9530af2795938d9d4c528f633a3b2242d82ac3c37
Deleted: 1e625e037281571d3fd2ea08d677a56d05ae2adcbe8c566d9379281254a8d4e6
Deleted: 1c0d1742c5c7cd50be97309f3808b356bdfa1047d90a340833fe3818d8532d11
Deleted: 02c6919bae7562f30ab264c9ad68583d13d507f93b41fdc750f3f2336a38c032
Sending build context to Docker daemon 6.656 kB
Step 1 : FROM alpine
 ---> d6ead20d5571
Step 2 : COPY verify.sh ./
 ---> 883e7c374571
Removing intermediate container 83006da82f39
Step 3 : RUN chmod +x verify.sh
 ---> Running in 3ba6771d6b7b
 ---> b3fb87b1bd67
Removing intermediate container 3ba6771d6b7b
Step 4 : ENTRYPOINT ./verify.sh
 ---> Running in 3aaa08502726
 ---> e7aeda0d084d
Removing intermediate container 3aaa08502726
Step 5 : CMD sh
 ---> Running in 5acf82a75b52
 ---> 652c119975c7
Removing intermediate container 5acf82a75b52
Successfully built 652c119975c7
Sending build context to Docker daemon 6.656 kB
Step 1 : FROM entrypoint
 ---> 652c119975c7
Step 2 : COPY overridden.sh ./
 ---> a8ece7704616
Removing intermediate container a6e3bc1666d5
Step 3 : RUN chmod +x overridden.sh
 ---> Running in 2506fca8cb7d
 ---> d29bff80ded7
Removing intermediate container 2506fca8cb7d
Step 4 : ENTRYPOINT ./overridden.sh
 ---> Running in 2bb4f06c0d6e
 ---> 014226640ad5
Removing intermediate container 2bb4f06c0d6e
Step 5 : CMD sh
 ---> Running in bddb37662abb
 ---> 6e7eb9b1d4c6
Removing intermediate container bddb37662abb
Successfully built 6e7eb9b1d4c6

Docker run 'entrypoint' without VAR=ok must fail:
[entrypoint/verify.sh] Verifying variable VAR...
[entrypoint/verify.sh] Missing $VAR=ok
Test passed

Docker run 'entrypoint' with VAR=ok must pass:
[entrypoint/verify.sh] Verifying variable VAR...
[entrypoint/verify.sh] variable exists and it is 'ok'
Test passed

Docker run 'overridden' without VAR=ok must fail:
[overridden/overridden.sh] Overridden entrypoint. Should be OK to display this, but not without '-e VAR=ok'
Test failed.

Docker run 'overridden' with VAR=ok must pass:
[overridden/overridden.sh] Overridden entrypoint. Should be OK to display this, but not without '-e VAR=ok'
Test passed
bruno@lenovo:~/test-lock-entrypoint$

Hi!
Please read this important information about creating issues.
If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.
This is an automated, informational response.
Thank you.
For more information about reporting issues, see https://github.com/docker/docker/blob/master/CONTRIBUTING.md#reporting-other-issues

BUG REPORT INFORMATION
Use the commands below to provide key information from your environment:
docker version:
docker info:
uname -a:
Provide additional environment details (AWS, VirtualBox, physical, etc.):
List the steps to reproduce the issue:
1.
2.
3.
Describe the results you received:
Describe the results you expected:
Provide additional info you think is important:
----------END REPORT ---------
#ENEEDMOREINFO

I'm -1 on this; users of an image should be able to use that image in any way they want. Overriding an entrypoint is not something that's easily done "by accident" (you'd have to explicitly provide the --entrypoint flag).
Also, this wouldn't prevent users from using the image as FROM in a Dockerfile, and override the entrypoint.
However, I can see organizations where you don't want users to override entrypoints. With the "Authz" plugins feature, that will be part of 1.10, it would be possible to deny users to override entrypoints, depending on their role; https://github.com/docker/docker/pull/15365

@thaJeztah are you -1 as a user, or as image author? As a user, I understand, but this is a feature for image authors.
Image authors already decide what goes inside the image they are building. How is it a negative thing to empower them with the right to decide how their images must be executed? Besides, it is a request for an optional parameter image authors may or may not use. If a power user wants to complain for some reason, the user contacts the image author.
Now, in my first comment, if you had read it through, I do mention that such feature must guarantee that inherited images may define their own ENTRYPOINTS, but the entry point of the FROM image must be performed. This raises question tough on how to also lock some files/folders in a way that inherited images can't modify these with some RUN command. Which brings us to issue #18812.
This is an important feature for image authors.

Well, I'm looking for a use case; point is that IMO this should be a documentation issue ("how to use this image"). If the thought here is to implement a "copy protection" system, I'm just not sure this would work. Users can compile their own version of docker that ignores the "don't override" part, or simply export the image, modify the layers, and import the image again.
Just my 0.02c, others may think differently.

As an image author, I use sensible defaults in my environment variables so that it works for them out-of-the-box without the user needing to decide on how their image must be executed.
When you look at the documentation for official images, it is all centered around using ENV variables at deploy-time to allow images to be customised in a supported way and to provide the contract between the image author and the user.
The use case isn't clear to me but I could also be missing something. Is it possible to provide an example scenario where having this feature could add value for the user?
If it's just about the image author then I would put a -1 on this as well. That's only because in my own experience I've never had a problem with someone being upset with my image because they stepped off the beaten track and tried to override the ENTRYPOINT and it broke things. I'm not saying that doesn't happen but it seems to be an extreme edge case, at least in my experience.

I'm -1 on this, same as @thaJeztah said, as a user and as image author. I'm also looking for a use case, and I feel that "Authz" will cover most of this. In the end, the image is just a rootfs, it feels like it would be pretty easy to bypass this by exporting a container from this image and re-importing it (just a tiny bit more complicated than --entrypoint) — thus it feels like an overly complicated thing to implement (which could lead to bugs, …) for no gain in the end.

Image authors already decide what goes inside the image they are building. How is it a negative thing to empower them with the right to decide how their images must be executed?

For me it sounds like asking Linux distribution builder (debian/redhad/…) "that already decide what goes inside the distribution they are building", that they should be able to decide how their distribution must be executed (locking down the user) — but well, if you are root on the machine you can pretty much messing it up and doing whatever you want. That's the beauty of it so I really don't see the point.
I feel the same about that it is a documentation issue on "how to use this image" ("This image is intended to be used like that […]."). You can provides default sane behavior and a good documentation on how to run the image and how to configure/custom the runtime (i.e. container), but locking down user, it should be separate from the image (by engine/cluster) — hence the "Authz" stuff. Docker is almost all about separation of concern after all.

It looks like there is an agreement among maintainers. I'm sorry but we're not going to implement this.
Thanks for taking your time to write it. It's always good to hear from the community.


Remove non-inclusive language

Frankly, the fact you'd use a word like this is incredibly ableist, and I feel the need to fix this issue, in order to make a more inclusive bot.

happy?


xgboost in sklearn pipeline

How can the additional parameters e.g. early_stopping_rounds be passed to a sklearn pipeline in the fit step?
As pipeline.fit(X_train, y_train, early_stopping_rounds=20, verbose=True) will result in an error of the pipeline:
-- fitting xgboost pipeline
in fit_model
    pipeline.fit(X_train, y_train, early_stopping_rounds=20, verbose=True)
  File "/usr/local/lib/python3.5/site-packages/sklearn/pipeline.py", line 268, in fit
    Xt, fit_params = self._fit(X, y, **fit_params)
  File "/usr/local/lib/python3.5/site-packages/sklearn/pipeline.py", line 227, in _fit
    step, param = pname.split('__', 1)
ValueError: not enough values to unpack (expected 2, got 1)

Whereas these options work just fine if they are just passed directly to the xgboost estimato (without a pipeline)

Did you find any workaround for that?

try prefixing your fit_params with the name of the xgboost estimator step in the pipeline, followed by two underscores. see: https://stackoverflow.com/questions/40329576/sklearn-pass-fit-parameters-to-xgboost-in-pipeline


Implement Sequential Monte Carlo

This issue tracks progress towards implementing Sequential Monte Carlo aka Particle Filtering.
For general control flow this appears to be difficult due to Python's lack of forkable coroutines.
For static computation graphs we may be able to implement tensorized forking via sparse tensors.

may be useful but extremely outdated: @OptimusLime 's implementation using OS forking iirc


Add Chaos Engineering: The Art of Breaking Things Purposefully

Adding Chaos Engineering: The Art of Breaking Things Purposefully - An article by Adrian Hornsby with the story of the GameDays, the prerequisites, the phases and the benefits of doing Chaos Engineering.

Great article! Thanks for your contribution! :+1:


Godot v4.3 Editor - Windows 11 - Crash when editing any project

Tested versions
I only tried Godot v4.3.
System information
Windows 11 Pro 23H2 - NVIDIA RTX 3070 Ti - Meta Quest 2
Issue description
I am experiencing a problem where the GODOT Editor crashes when I try to edit a project.
I open the Godot XR Tools Demo and after about 30 seconds, the editor just crashes. I can run it but cannot open the project in the editor.
I updated my NVIDIA drivers to the latest which is currently v566.14. I have an NVIDIA RTX 3070 Ti and a Meta Quest 2 headset.
Windows events viewer shows the following error:
Faulting application name: Godot_v4.3-stable_win64.exe, version: <IP_ADDRESS>, time stamp: 0x66bd392d
Faulting module name: D3D12Core.dll, version: 10.0.22621.4391, time stamp: 0x3ffb1bfa
Exception code: 0xc0000005
Fault offset: 0x00000000000b59a2
Faulting process id: 0x0x60D4
Faulting application start time: 0x0x1DB405BF676B4B3
Faulting application path: G:\GODOT Game Engine\Godot_v4.3-stable_win64.exe
Faulting module path: C:\Windows\system32\D3D12Core.dll
Report Id: 224a335a-1182-4c7a-8135-cf4b02771d6b
Faulting package full name:
Faulting package-relative application ID:
Steps to reproduce
Download the Godot XR Tools Demo  or Godot XR Template in the Asset Library and edit it. The editor will crash after a few seconds.
Minimal reproduction project (MRP)
See above

I just tried Godot v4.2.2 and it is also crashing while editing the template called FPS template by shidoengie.


release optimized docker image to Dockerhub

Release Docker image with the fix #7 to Dockerhub, since Docker image size is dramatically reduced.

released ks6088ts/azure-openai-playground:0.0.4


Define and parse letter document meta data format

The letter meta data format with file extension .lmd is intended to contain information about the letter document or project. The file syntax is TOML.
Example:
title = "My document title"
authors = ["The authors name"]
version = 0.0.1
language = "de-de" # Language of the document - used for example for hyphenation when typesetting

[variables]
release-date = "2022-04-30" # Variable that can be used in letter script files using the variable extension: [var]release-date[/var]

Other than a letter style file .lst, having a .lmd and .lsc (letter script) is required for a letter document project. When running the document build process using letter build in the current folder, those two files will be checked.

This is needed in the walking skeleton of Letter as we need the language for hyphenation (currently only english is supported)

See also https://github.com/bennyboer/letter/issues/28

We now use the .toml extension instead of .lmd. It is easier if IDEs directly see the intended format.
In this first iteration we simply read the meta data file with the name meta_data.toml in the same directory as the letter script file.


Standalone Tunnel establishment should disable border gateway configuration at the platform level.

Fix for Issue #172
This was being achieved by the routingRestricted flag sent in the
StatusReport in response to the TunnelOpen. Instead, the
WeaveTunnelAgent should consult its configured role(Standalone) to
disable platform-level border gateway function.

Codecov Report

Merging #174 into master will increase coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #174      +/-   ##
==========================================
+ Coverage   53.87%   53.88%   +<.01%
==========================================
  Files         326      326
  Lines       55626    55626
==========================================
+ Hits        29968    29973       +5
+ Misses      25658    25653       -5

Impacted Files
Coverage Δ

src/lib/profiles/security/WeaveCert.cpp
83.76% <0%> (-0.65%)
:arrow_down:

src/lwip/standalone/sys_arch.c
69.48% <0%> (+0.46%)
:arrow_up:

third_party/lwip/repo/lwip/src/core/pbuf.c
36.79% <0%> (+1.59%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a8008cb...7bcc304. Read the comment docs.


Gopi : student's bio

[ ] I wrote my markdown locally on a new branch
[ ] My bio file is named exactly the same as my GitHub username
[ ] My bio link in the class README opens to my student bio file
[ ] I pulled and merged main into my branch before pushing
[ ] I pushed my branch to the class repository and opened a PR
[ ] I went through the PR checklist
[ ] All the CI check pass
[ ] I assigned a coach to review my PR
[ ] I
linked my PR to this issue

@Gopiben  Are you a student in class 21 or 20 ?

Im in class 20
On Thu, 23 Mar 2023, 22:27 Samir M., @.***> wrote:

@Gopiben https://github.com/Gopiben Are you a student in class 21 or 20
?
—
Reply to this email directly, view it on GitHub
https://github.com/HYF-Class21/home/issues/34#issuecomment-1481936888,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AZXDZGKM7CJTFDHHJBRR7PTW5S555ANCNFSM6AAAAAAWFBSVZU
.
You are receiving this because you were mentioned.Message ID:
@.***>


meta: support of es module

This should be marked as a BREAKING CHANGE, since supporting esm is currently infectious, everything depends on it would have to move to esm after this.
Things to wait for:

jest with esm support
ts-jest

Things to change:

worker-pool.spec.ts (__dirname)
package-lock-resolved.js (__dirname)
delay-calc-worker.js (require)
copy-src.js (__dirname)
tsconfig.json
package.json

Things breaking by this:

p-queue 7.0.0 (https://github.com/website-local/website-scrap-engine/pull/222)

Maybe we can use rollup to package p-queue to commonjs module again


Default to api-key auth plugin

Since it's required now to provide an auth plugin, we might as well default to api-key so people can still use the syntax:
var flickr = new Flickr({ api_key: 'abcd1234' });

No longer need this since https://github.com/flickr/flickr-sdk/tree/next#new_Flickr_new