How to exclude generated sources from ktlint --relative?

./ktlint --version
0.48.2

After run the command from pre-commit:
git diff --name-only -z --cached --relative -- '*.kt' '*.kts' | ./ktlint --relative --patterns-from-stdin=''

Output is :
...
app/build/generated/source/kapt/debug/....kt:46:1: Unexpected indentation (4) (should be 8) (indent)

Summary error count (descending) by rule:
  indent: 464
  parameter-list-wrapping: 194
  wrapping: 194
  curly-spacing: 27
  no-consecutive-blank-lines: 1

ktlint also include files from folder: app/build/generated/source/kapt/

Duplicate of #1793.
This problem is known and has already been fixed, though the fix is not yet released.
In the meanwhile, you can refer to the linked issue for a workaround.

@Kolyall You can also use the ktlint snapshot (https://pinterest.github.io/ktlint/install/snapshot-build/) to verify that your problem is solved.

Having similar issue with generated code from Kotlin build scripts (under buildSrc folder). Can confirm 0.49-snapshot fixes that. But I run ktlint without githooks, any workaround to exclude paths (buildSrc folder in this case)?

Thanks! found the docs https://pinterest.github.io/ktlint/install/cli/#globs
Only excluding didn't work, we also need to include smth.
So this worked for me:
    "**/*.{kt,kts}",
    "!**/build/**",

So this will make sure ktlint is running on both kt and kts files (event in buildSrc), but not in the generated build folder

Only excluding didn't work, we also need to include smth.

Yes that is correct. The default (include) patters will only be applied whenever no patterns are specified. But that does not make sense when only an exclude pattern is given.


ci self pear

also introduces
pear sidecar shutdown
and modifies pear info --key to just print key with no additional visual-framing chars

I think we can remove workflow cache now since seems like its not used anymore https://github.com/holepunchto/pear/blob/main/.github/workflows/ci.yml#L34-L38


address some minor typos and formatting errors

Just fixed typo and applied some formatting best practices.

nvm. doing some upgrading that will include these.


mouse events not delivered using conhost mode and vt output

Environment
PS C:\Users\garre\Work\tcell_demos> [Environment]::OSVersion
Platform ServicePack Version      VersionString

Win32NT             10.0.19041.0 Microsoft Windows NT 10.0.19041.0
PS C:\Users\garre\Work\tcell_demos>
Windows build number: [run `[Environment]::OSVersion` for powershell, or `ver` for cmd]
Windows Terminal version (if applicable):

Any other software?

Steps to reproduce
Easiest reproduced with my open source tcell demonstration application:
github.com/gdamore/tcell
go run _demos/mouse.go
On a traditional cmd.exe this delivers mouse events and the mouse position is tracked, and using any of the three mouse buttons starts an application layer event (we create a numbered box with the number matching the mouse button).
This is using the following modes on CONIN$:
ENABLE_MOUSE_INPUT | ENABLE_WINDOW_INPUT | ENABLE_EXTENDED_FLAGS   -- basically we want to receive all the mouse positions, as well as window resize events.
This works beautifully on cmd.exe.  But the mouse doesn't work at all on Terminal -- instead it seems like the mouse is treated as a region select/copy.
There is also a branch (win256color) that changes the code to use VT output mode, but it behaves the same with respect to the mouse selection.
(I'd like to avoid using VT style inputs if possible, as those are both less rich (some keys are not mapped), and I think this would cause me to lose resize events which may be important.)

Expected behavior
Mouse events are passed to my application using the normal Console API.
Actual behavior
Mouse events don't make it through at all.

Thanks! The generation of windows mouse events (instead of VT mouse events) is being tracked using /dup #376.
Extended flags doesn’t impact your use of mouse and window events, so unless you are using it as documented to manipulate the edit mode you do not need to include it.
(Also, thanks for tcell! Since it looks like it’s cross-platform, how do you intend to offer the otherwise-unmappable keys off Windows? Using Windows input events is usually a significant burden to cross-platform libraries, not something that makes them better. We’ve been trying to focus on improving our VT story for that reason!)

@DHowett the reason I use legacy events for several reasons:

I wrote tcell to work with Windows console applications (as well as VT applications) in the days before Windows 10 had VT support.

VT events are necessarily limited.  So yes, we can only offer certain key combinations on Windows, whereas for XTerm and similar we wind up not being able to report key presses that are not reported by the system (i.e. those that don't have escape codes associated with them.)  I do generally discourage application authors to avoid reliance on those for that reason, but some application developers prefer to support them where they can, and accept that they can't on other systems.

Window size reporting.  On *nix, we get a SIGWINCH to tell us the window has changed size.  I cannot imagine how we will get that in a Windows app -- do we get a Window resize signal as well?

Yes, this does impose a portability burden.  The tcell application tries to provide for the necessary plumbing so that application authors don't have to worry about this, and it will Just Work regardless of using a Windows (old) console (think Windows 8 or earlier), a new Windows 10 console, Windows Terminal, iTerm (or any of the various alternatives) on macOS, a Linux or FreeBSD console, or an of the various X11 based terminal emulators.


Enhancement(Add option to create folders to json reports)

Addresses issue https://github.com/angular/protractor/issues/4107.
TO DO: write tests for it, may be in next PR!

I have seen the same CI errors in other PR's as well, not sure what is causing this!


raftpicker: do not use picker

Just recreate connection on leader change.
ping @aaronlehmann

Current coverage is 54.99% (diff: 25.00%)

Merging #1313 into master will decrease coverage by 0.17%

@@             master      #1313   diff @@
==========================================
  Files            80         80
  Lines         12541      12559    +18
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
- Hits           6919       6907    -12
- Misses         4678       4697    +19
- Partials        944        955    +11

Powered by Codecov. Last update a210cdd...b9cc543

This is one more fix for problem with wrong rescheduling on leader losing.

@aaronlehmann fixed. Thanks!

LGTM if this works well in practice.

I'm fine with the change but on leader switch now I get:
user@node01:~# docker node ls
Error response from daemon: rpc error: code = 2 desc = grpc: the client connection is closing

And I have to input the command a second time to have the result. Is this the behavior we want?

@abronan I think it's lesser evil. ping @aaronlehmann

Yeah.

LGTM


Uncaught TypeError: undefined is not a function

[Enter steps to reproduce below:]

Add a project folder with at least one file with "todo" extension
Add a task
Press cmd + d keyboard shortcut to mark the task finished

Atom Version: 0.202.0
System: Mac OS X 10.10.3
Thrown From: tasks package, v2.0.1
Stack Trace
Uncaught TypeError: undefined is not a function
At /Applications/Atom.app/Contents/Resources/app.asar/node_modules/text-buffer/lib/text-buffer.js:755

TypeError: undefined is not a function
  at Object.module.exports.setMarker (/Users/igorstojanovic/.atom/packages/tasks/lib/tasksUtilities.coffee:219:28)
  at /Users/igorstojanovic/.atom/packages/tasks/lib/tasks.coffee:182:17
  at Array.map (native)
  at /Users/igorstojanovic/.atom/packages/tasks/lib/tasks.coffee:160:44
  at TextBuffer.module.exports.TextBuffer.transact (/Applications/Atom.app/Contents/Resources/app.asar/node_modules/text-buffer/lib/text-buffer.js:750:18)
  at TextEditor.module.exports.TextEditor.transact (/Applications/Atom.app/Contents/Resources/app.asar/src/text-editor.js:1136:26)
  at Object.module.exports.completeTask (/Users/igorstojanovic/.atom/packages/tasks/lib/tasks.coffee:159:12)
  at atom-text-editor.atom.commands.add.tasks:complete (/Users/igorstojanovic/.atom/packages/tasks/lib/tasks.coffee:57:29)
  at CommandRegistry.module.exports.CommandRegistry.handleCommandEvent (/Applications/Atom.app/Contents/Resources/app.asar/src/command-registry.js:238:29)
  at /Applications/Atom.app/Contents/Resources/app.asar/src/command-registry.js:3:61
  at KeymapManager.module.exports.KeymapManager.dispatchCommandEvent (/Applications/Atom.app/Contents/Resources/app.asar/node_modules/atom-keymap/lib/keymap-manager.js:521:16)
  at KeymapManager.module.exports.KeymapManager.handleKeyboardEvent (/Applications/Atom.app/Contents/Resources/app.asar/node_modules/atom-keymap/lib/keymap-manager.js:356:22)
  at HTMLDocument.module.exports.WindowEventHandler.onKeydown (/Applications/Atom.app/Contents/Resources/app.asar/src/window-event-handler.js:178:20)

Commands
     -0:11.1.0 editor:select-to-first-character-of-line (atom-text-editor.editor.is-focused)
     -0:06.5.0 core:move-left (atom-text-editor.editor.is-focused)
     -0:06.3.0 editor:select-to-end-of-line (atom-text-editor.editor.is-focused)
     -0:05.1.0 core:select-down (atom-text-editor.editor.is-focused)
     -0:04.5.0 core:copy (atom-text-editor.editor.is-focused)
  3x -0:04.2.0 core:move-down (atom-text-editor.editor.is-focused)
     -0:03.4.0 core:paste (atom-text-editor.editor.is-focused)
  5x -0:02.8.0 core:move-up (atom-text-editor.editor.is-focused)
     -0:01.0 tasks:complete (atom-text-editor.editor.is-focused)

Config
{
  "core": {
    "ignoredNames": [
      ".git",
      ".svn",
      ".DS_Store"
    ],
    "themes": [
      "atom-dark-ui",
      "monokai"
    ]
  }
}

Installed Packages
# User
color-picker, v2.0.2
command-logger, v0.23.0
editor-stats, v0.17.0
linter, v0.12.6
linter-write-good, v0.4.3
monokai, v0.14.0
save-session, v0.15.3
tasks, v2.0.1
visual-bell, v1.0.0

# Dev
No dev packages

Strange! Can you send the todo content that you have that caused the issue? Internally the tests pass fine, so it might be something weird with the content or command order.
You might also try re-installing the plugin to see if it persists!

The content doesn't matter. I tried to reinstall the plugin, but the bug still persist.
Here's a small video which shows all steps I did.
https://dl.dropboxusercontent.com/u/3437823/tasksProblem.mov

Thanks for the video! I'll take a look and see if I can reproduce the problem.

So it looks like a method that gets a screen column for a given token was removed in a recent commit to the atom core: https://github.com/atom/atom/commit/2379b3803fbc9e09dce4192dc4c01b93d1c23249#diff-dc35bc3db558965c0b4211dd52bc19cbL294
Now looking for an alternative!

@irrationalistic Works now. Thanks.


Suggestion: Adding helper function to replace matches with components

Hi @dabakovich, amazing work on this library. It is so clearly written and really smart with all the helpers and managing the displayed text vs. state text.
I have been using this library and found myself wanting to replace the @mentions with clickable text component. Both for styling purposes but also so that users can click on a @mention in the app and be navigated to that mentioned user's profile.
I wrote a helper that you can consider if you want to include in the library. I am not so proficient with TypeScript, so can't help you directly myself with a PR.
This is the helper:
  const replaceMentionsTextWithComponents = (text) => {
    // Check how many @ mentions there are in the text, if none just return text
    const numMatches = text.match(mentionRegEx)?.length;
    if (!numMatches) {
      return text;
    }

    // Store all the matches and their values in an array to access later
    let matchedArray = [];
    let match;
    do {
      match = mentionRegEx.exec(text);
      if (match) {
        // Only interested in the named groups
        matchedArray.push(match.groups);
      }
    } while (match);

    // Split the text into its separate parts, so that matches can be replaced by components
    const parts = text.split(/(.\[[^[]*]\([\d\w-]*\))/gi);

    // Loop through each part and replace matches with the chosen component
    // Accessing the previously stored array to insert these values into the component
    let counter = -1;
    const formattedText = parts.map((part, index) => {
      if (part.match(mentionRegEx)) {
        counter += 1;
        const id = matchedArray[counter].id;
        const name = matchedArray[counter].name;
        return (
          <ScaledText
            key={index}
            style={{ fontWeight: "bold", color: "blue" }}
            onPress={() => {
              navigation.navigate("Profile", {
                userId: id,
              });
            }}
          >
            @{name}
          </ScaledText>
        );
      }
      return part;
    });

    // Return the text with the matches replaced by components
    return formattedText;
  };

Just an idea and suggestion, keep up the good work!!

Thanks a lot for your feedback!
That's a great idea and it should be a useful helper.
FYI. You can use the existing helper parseValue and your own part renderer to resolve the issue. Here is an example:
/**
 * Part renderer
 *
 * @param part
 * @param index
 */
const renderPart = (
  part: Part,
  index: number,
) => {
  // Just plain text
  if (!part.partType) {
    return part.text;
  }

  // Mention type part
  if (isMentionPartType(part.partType)) {
    return (
      <Text
        key={`${index}-${part.data?.trigger}`}
        style={part.partType.textStyle}
        onPress={() => console.log('Pressed', part.data)}
      >
        {part.text}
      </Text>
    );
  }

  // Other styled part types
  return (
    <Text
      key={`${index}-pattern`}
      style={part.partType.textStyle}
    >
      {part.text}
    </Text>
  );
};

/**
 * Value renderer. Parsing value to parts array and then mapping the array using `renderPart`
 *
 * @param value - value from MentionInput
 * @param partTypes - the part types array that you providing to MentionInput
 */
const renderValue: FC = (
  value: string,
  partTypes: PartType[],
) => {
  const {parts} = parseValue(value, partTypes);

  return <Text>{parts.map(renderPart)}</Text>;
};

Thanks a lot for your feedback!
That's a great idea and it should be a useful helper.
FYI. You can use the existing helper parseValue and your own part renderer to resolve the issue. Here is an example:
/**
 * Part renderer
 *
 * @param part
 * @param index
 */
const renderPart = (
  part: Part,
  index: number,
) => {
  // Just plain text
  if (!part.partType) {
    return part.text;
  }

  // Mention type part
  if (isMentionPartType(part.partType)) {
    return (
      <Text
        key={`${index}-${part.data?.trigger}`}
        style={part.partType.textStyle}
        onPress={() => console.log('Pressed', part.data)}
      >
        {part.text}
      </Text>
    );
  }

  // Other styled part types
  return (
    <Text
      key={`${index}-pattern`}
      style={part.partType.textStyle}
    >
      {part.text}
    </Text>
  );
};

/**
 * Value renderer. Parsing value to parts array and then mapping the array using `renderPart`
 *
 * @param value - value from MentionInput
 * @param partTypes - the part types array that you providing to MentionInput
 */
const renderValue: FC = (
  value: string,
  partTypes: PartType[],
) => {
  const {parts} = parseValue(value, partTypes);

  return <Text>{parts.map(renderPart)}</Text>;
};

Added that example to the readme.
Do you have any other questions here?

Added that example to the readme.
Do you have any other questions here?


Read only volume mount support

Change Overview
Support of read-only mount required by Ceph-CSI shallow volumes feature.
PR series:

https://github.com/kanisterio/kanister/pull/2376
https://github.com/kanisterio/kanister/pull/2366
https://github.com/kanisterio/kanister/pull/2378
https://github.com/kanisterio/kanister/pull/2377
https://github.com/kanisterio/kanister/pull/2379
https://github.com/kanisterio/kanister/pull/2383
https://github.com/kanisterio/kanister/pull/2386
=> https://github.com/kanisterio/kanister/pull/2365

Pull request type
Please check the type of change your PR introduces:

[ ] :construction: Work in Progress
[ ] :rainbow: Refactoring (no functional changes, no api changes)
[ ] :hamster: Trivial/Minor
[ ] :bug: Bugfix
[x] :sunflower: Feature
[ ] :world_map: Documentation
[ ] :robot: Test

Test Plan

[ ] :muscle: Manual
[ ] :zap: Unit test
[ ] :green_heart: E2E

I agree to the DCO for all the commits in this PR.

These changes needed for K10-18331 Support CephFS snapshot-backed (shallow) volumes
TL;DR; To support Ceph-CSI shallow volumes feature, we need to mount snapshot (temp PVC, which has snapshot as source) in ReadOnlyMany mode, then using of Ceph-CSI shallow volumes will be possible.

@viveksinghggits please continue review


toml.load(path) no directory found

I have tried to use toml.load(path) where the path is a mounting directory in synapse (synfs:/.../../example.toml)
However, it returns FileNotFoundError(2, 'No such file or directory')
May I know whether synfs:/ path will be supported?

It won't be supported, unless it is also supported by Python's open.
The load function expects a regular readable stream, so it might be possible to use an intermediary package to do the I/O with synfs -- that's research that you'll have to do yourself.


Incorrect parsing error 'Cannot use subscript operator on a base of type "null"'

Tested versions

Reproducible in: v4.3.stable.mono.official [77dcf97d8]

System information
Windows 10.0.19045 - GLES3 (Compatibility) - AMD Radeon RX 7800 XT (Advanced Micro Devices, Inc.; 31.0.24019.1006) - AMD Ryzen 7 1700 Eight-Core Processor (16 Threads)
Issue description
Godot is in some circumstances incorrectly determining that a value is "null" at parsing time.
It seems to be related to using higher-order functions in conditions - Godot seems to be assuming that the function returned by the high-order function always returns false, though that doesn't entirely explain it.
When this occurs, the project refuses to run, ('Cannot use subscript operator on a base of type "null"') though the code is correct.
My full source code is included in "Steps to reproduce"
As an aside, I'm using the GDscript language server, and you can also see the error within VS code

Steps to reproduce
Created a new project, created a single Node2D, added a script to it.
Script is below:
extends Node2D

func returns_callable():
    var f = func(): return true
    return f

func uses_returns_callable():
    if returns_callable().call(): # Always true
        return [1, 2, 3]
    return null

func _ready():
    var value = uses_returns_callable()
    print(value) # prints [1, 2, 3]

    # Though the following line should work, if uncommented it results in an error during parsing
    # 'Cannot use subscript operator on a base of type "null".gdscript(-1)'
    # if value: print(value[0])

    # Curiously, if I actually replace 'var value = uses_returns_callable()` with `var value = null`
    # no parsing error is produced

Minimal reproduction project (MRP)
minimal-parser-bug-example.zip

When adding -> Variant as return type to uses_returns_callable no error is produced, so for some reason the analyzer tries to infer a return type of the function instead of using Variant by default


Related Entity Not Inserted

In EF Core 3.0, related entities to the aggregate are not inserted. So, when inserting lists of Parent objects with one or more Child objects, only the Parents are inserted into the database. The related entities have foreign keys defined in the model. The Children have a foreign key pointing to the Parent, and both Parent and Children have navigation properties. The navigation relationships are wired up in code both ways, i.e. child.Parent = parent and paent.Children.add(child). Is this by design?

Reading into the docs I determined how to accomplish this. I suggest making this more clear in the docs, but other than that I really like this package.

@jeffgbradley2 Can you point to your solution?


[Merge] MONDO:0012461	bulimia nervosa, susceptibility to, 2

Mondo term (ID and Label)
MONDO:0012461	bulimia nervosa, susceptibility to, 2
Reason for deprecation
OMIM merged these
Term to be merged with
MONDO:0011727	ANON1

I am not sure these should be merged. ANON1 is anorexia nervosa, susceptibility to, 1 (I think we should add this as the primary label).
It doesn't make sense to me to merge susceptibility to bulimia with susceptibility to anorexia, as they are two different disorders, and in Mondo, that are noted to have dysfunction of two different genes.
The OMIM record says this was moved to:
OMIM:607499 (which is MONDO_0011846 'bulimia nervosa, susceptibility to, 1')
(I think I had the wrong Mondo class in #1704 )
and OMIM:606788 (which is MONDO:0011727 ANON1)
I think the correct course of action is:

[ ] merge MONDO:0012461 bulimia nervosa, susceptibility to, 2 with MONDO:0011846 'bulimia nervosa, susceptibility to, 1'
[ ] relabel ANON1 to anorexia nervosa, susceptibility to, 1

Here is a [spreadsheet(https://docs.google.com/spreadsheets/d/1pASj2MnBI285ogOeKVqEVe2TsKCvOWDFFNq_ekN2juw/edit#gid=0) with terms tagged for obsoletion, we should start here


[VS Code] Response status code does not indicate success: 400 (Bad Request).

log.zip

Error:
Response status code does not indicate success: 400 (Bad Request).
Steps to Reproduce:
1.click sign-in
2.click login with MS account
3. browser navigate back
4. click login with gayhub account
5. click have problem
6. copy the code,then paste in vscode
7. got the toast message,click it, then here.

Version Data

extensionName
VSLS

extensionVersion
0.3.264

protocolVersion
2.2

applicationName
VSCode

applicationVersion
1.23.1

platformName
Linux

platformVersion
4.14.0-deepin2-amd64

[2018-06-07 14:11:13.889 Agent E] HTTP exception in auth.login request handler: System.Net.Http.HttpRequestException: Response status code does not indicate success: 400 (Bad Request).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at Microsoft.Cascade.Agent.AuthenticationService.d__26.MoveNext() in E:\A_work\4\s\src\Agent\AuthenticationService.cs:line 153
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Cascade.Rpc.RpcDispatcher1.<>c__DisplayClass23_2.<<BuildMethodMap>b__2>d.MoveNext() in E:\A\_work\4\s\src\Rpc\RpcDispatcher.cs:line 218 --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Microsoft.Cascade.Rpc.RpcDispatcher1.d__16.MoveNext() in E:\A_work\4\s\src\Rpc\RpcDispatcher.cs:line 99
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Cascade.Rpc.RpcServiceUtil.d__3.MoveNext() in E:\A_work\4\s\src\Rpc\RpcServiceUtil.cs:line 56
[2018-06-07 14:11:13.890 Agent.Rpc.Auth V] < #1 [7] Error: Response status code does not indicate success: 400 (Bad Request).

Thanks for reporting this issue.  I've created a bug on our side and we'll investigate.

@songlairui do you still have this issue on the latest version of the extension?

Closing due to inactivity. Please reopen if you can provide us with more info.


Loading of Kohana config needs to be updated to work

in database.php, change like 100 to:
$config = Kohana::$config->load('mongo')->$name;

This was changed Oct 25


Can’t close WinUI Win32 apps with alt+F4

Describe the bug
Alt+F4 commonly closes windows on Windows. I assume that this shortcut not working is not deliberate as I have seen it before with XAML islands somehow consuming this shortcut.
Steps to reproduce the bug

Steps to reproduce the behavior:

In a WinUI Win32 app, press alt+F4
Observe the app remain opened

Expected behavior
The app should close on key down of alt+F4.
Version Info

NuGet package version:
Microsoft.WinUI 3.0.0-preview1.200515.3

Windows 10 version
Saw the problem?

Insider Build (xxxxx)

November 2019 Update (18363)
Yes

May 2019 Update (18362)

October 2018 Update (17763)

April 2018 Update (17134)

Fall Creators Update (16299)

Creators Update (15063)

Device form factor
Saw the problem?

Desktop
Yes

Mobile

Xbox

Surface Hub

IoT

This has been fixed for some time now.


Spanner: Implement CMEK support

Internal tracking: b/157706906

Note: this work is currently blocked on us being able to get access to the protos.

The CMEK protos are available in the spanner_cmek branch here: https://github.com/googleapis/googleapis/blob/spanner_cmek/google/spanner/admin/database/v1/common.proto

This is done aside from implementing a code sample, which I'll take care of:
Show how to create a database with an encryption key. In the same sample, call GetDatabase (or the equivalent) to verify that the encryption key is set as expected.
Region tag: spanner_create_database_with_encryption_key

geez, rebasing made a mess out of this issue's history.
this is done except for backup/restore, I will complete that and close this issue.

geez, rebasing made a mess out of this issue's history.
this is done except for backup/restore, I will complete that and close this issue.


Thoughts on instance variables wrt nil-able variables

I have a class where an instance-variable is basically defined as @locationKey: String | Nil (with a getter also defined), and then in a method I happened to write:
    lsize = locationKey.nil?  ? 0 : locationKey.size

That generates an error message of undefined method 'size' for Nil (compile-time type is (String | Nil)).  I looked at that error for a few seconds, and then remembered that crystal complains about the second reference to locationKey because it is an instance-variable.  I realize that the goals for crystal include future support for excellent parallel processing, and when that happens then the value of @locationKey could turn into a nil between the first and second reference.  And I also remembered that for this specific example, there's a standard-crystal way to handle that:
   lsize = location.try &.size || 0

Okay, so I understand the reasons for that.  But then I looked at all the other code I have in the same method.  I have a lot of code which does something like:
   if location == otherEntry.location
      #   Do a lot of processing using the values
      #   of "location" and "otherEntry.location".
   end

But then I got to thinking:  If the complier believes that location could be changed into a nil between an if and the code inside that if, then it must be true that location could also be changed to some new string value.  That new value will be a valid string, so there won't be any runtime errors due to it being nil.  However, if the value did change then it may not be true that location == otherEntry.location.  Both location and otherEntry.location could change to new values, and the processing inside the if might run into run-time problems because that code assumes the two variables have the same value.
And if that's true, then doesn't it mean that instance-variables are going to be very awkward to work with in a totally safe manner?  Will we always have to copy instance-variables into method-local variables if the values are referenced multiple times inside the method?

Will we always have to copy instance-variables into method-local variables if the values are referenced multiple times inside the method?

In multi-thread environments, you always need to obtain a local copy of the value you want to work with as the shared reference might be changed by another thread.
While Crystal is not yet multi-thread, it has no GIL (Global Interpreter Lock) like Ruby or other languages that warrant you that the value obtained in the if condition will be the same used or passed along in the then block.
You can find more about this reasoning in the documentation:
https://crystal-lang.org/docs/syntax_and_semantics/if_var.html
On more system level languages (like C) you need to adopt local variable + atomic-write/read functionality to avoid parallel threads from affecting the same value.
I guess could be possible for the compiler to detect a if @a and hold a local value to it when referenced inside the block, however it might get tricky if in your code you're also assigning a new value to it.
Example:
if @a
  puts @a
end

Might be able to be converted by the compiler:
if __temp1 = @a
  puts __temp1
end

But it might be tricky on assign:
if @a
  @a = new_value
end

As another answer to the question: yes, it can happen that between reading location for the first time and reading it for the second time it could be a different string value. Or even something like this:
@location = "Hello"
# Here we maybe switch to another thread,
# which sets @location to "Bye"
puts @location # => "Bye"

So yes, you can get wrong behaviour if you don't synchronize well your code. But in the previous snippet at most we got a different value than what we expected. However, if we don't do this for types, this might happen:
@location = "Hello"
# Now another thread sets @location to nil
# Let's assume that the compiler still thinks that
# @location is a string
puts @location # ...???

If the compiler assumes that @location holds a string, which is basically represented as a pointer, it will say "Great! This isn't nil, I can directly dereference this". Well, if it's nil, the pointer will be the null pointer, and dereferencing it will give you a segmentation fault. And we can't really let a program crash just like that. This is what types can't change under the hood for the compiler. But it's OK if values change across the same type.
If you assign an instance variable to a local variable, no other thread can change it, and the compiler can assume and know that the type won't change, and do optimizations like not having to check for null to dereference a pointer.

Heh.  In re-reading this today, I realize that I merely stated the obvious, and stated it very emphatically.  In my defense, when I was about halfway through writing that, I started thinking about how I should be checking the election results...

I'd argue that if the program operates on a different value than the code "believes" that it has, then you can still get run-time failures.  For instance, in the above I was getting the length of that string because I expected to operate on subsets of the string.  And those operations were going to key off of "lsize".  So what happens if the code references subsets of what-it-thinks is a 1000-character string, but that is now a 10-character string?
Index out of bounds (IndexError)
[4401291906] *CallStack::unwind:Array(Pointer(Void)) +82
[4401291809] *CallStack#initialize:Array(Pointer(Void)) +17

Note that I agree (almost) completely with how the compiler treats composite types.  My claim is that it isn't doing enough.  It's saving the programmer from this one case, but there's an infinite number of other cases where problems will come up for the exact same problem.
So... What I'm wondering is whether the compiler could do something to point out the general problem, instead of the one special case of nil.  I didn't write that up last night, because I realized that I did not have a good idea of how the compiler could do that.
In the method in question, I'm comparing two objects which were created by reading in some JSON strings, and I want to compare the individual values of each object.  So I'm doing a lot of code similar to if location == otherEntry.location.  The first 'location' is an instance variable in the current object, and the otherEntry.location is a method call into the other JSON object.
What I realized last night is that the only really safe way to do that is to copy all of those instance variables and method-calls into local variables, and then do the actual processing off of those copies.  When I look at the crystal code that I'm writing, it seems to kinda ugly due to all those copies.
So I guess what I'm asking is if there is any way the compiler could handle that for me.  While I have some vague ideas, I have come up with nothing which I really like.

Vague idea, assume a 'localize' function which was supported by the compiler itself.
  def someMeth
      localize @location, otherEntry.location

      # lots and lots of code.  Somewhere in that code, I
      # reference both variables:
  if location == otherEntry.location
      #   Do a lot of processing using the values
      #   of "location" and "otherEntry.location".
   end

   # and maybe later on I have more code which
   # references those same variables.

   end

My vague idea is that once I tag a variable as "localized", then the first time that I reference that variable, the compiler makes a local copy of the value.  And all subsequent references in that same method will see that local copy, without me having to create dummy variables to do that work.
I can think of some possible problems with this, but that's the kind of thing I'm thinking about.  I don't want the compiler to stop doing the checking that it already does!

Sorry for the length of these comments, but I tend to be a wordy person when I'm focused in on some topic.
Note that it may be that the compiler would also have to add an invisible variable to keep track of when the copy had been made.  Such as:
   if someThing == true
      first8 = location[0...8]
   else
      first8 = otherEntry.location
   end
   last8 = location[-8..-1]
   puts first8, location

The thing I'm thinking about here is not the values of first8 or last8.  It's the issue where the copy of location might need to be done when first8 is created, but if the copy isn't done there then the copy would need to be made when last8 is created.
I realize that this "solution" may create as many problems as it solves, but I wanted to suggest some way that the crystal compiler could be even more helpful than it currently is.

Maybe this is the kind of problem that you would like to solve at a higher level with strong standard library support for concurrency services (like Actors in Erlang/OTP). Crystal will let you blow your leg if you use concurrency primitives and provide much saner guarantees if you stick to higher level constructs. Just a thought!

Nothing actionable here, so I'm closing this.

Is it only me who find this "multi threading context" insane? I thought this is why fibers exists in crystal, not to deal with this. Two different threads should NOT be allowed to share memory by default.

@jsaak the samples exposed in this issue can be extrapolated to fibers also. If a fiber performs I/O between reads of the same ivar it might be paused and anither fiber might change the ivar.

Yes, it is true. But you have much more control when your current fiber yields. There is no way that after a simple if statement the data will change.
I hope you know what you are doing. I feel, it is a recipe for disaster.

@jsaak it is not always obvious if method call will trigger a pause of the current fiber. Unless crystal flags methods that might do that. In that case the compilation time will increase because then that information in sensible to the actual method. If the method is in an object that might be of different types then you might even find that code that one compiles then it doesn't.
So, it is safer this way.

Yes, it is true. But you have much more control when your current fiber yields. There is no way that after a simple if statement the data will change.

It's possible.  Consider:
@in_var : String | Nil
def meth1
   if @in_var == "Some String" && other_condition?
      printf "%s\n", @in_var[5..-1]
   fi
end

def other_condition?
   #  Think of all the things which could happen
   #  in this method, and which the compiler won't
   #  know about when it is compiling meth1.  This
   #  method can do anything it wants with the value
   #  of @in_var, and in a world of fibers it could
   #  do anything it wanted with switching fibers.
end

I consider the compiler's concern about "multi threading context" to be a good idea.  My only complaint is that it should notice more than just when the type of an instance variable could change.  And I think that it might be possible to add features to the compiler which would make it easier for the programmer to address these issues.
Perhaps I've just written more crazy code in Ruby than many people, but in thinking over some of the code that I've written, I do understand and appreciate what crystal is trying to do here.  Let me add that all of my code is perfectly valid Ruby code if run in a single thread, but some things I've written would be extremely fragile (i.e., loaded with intermittent and obscure bugs) if it ran in a multi-threaded environment!


Handle 'thin' callees in extractStringConcatOperands

radar rdar://problem/37916552
StringConcatenationOptimizer assumes that the callees got method calling convention. This is incorrect - function signature ops can change that. (if we modified self we have thin instead of method representation)

@swift-ci Please smoke test and merge


Fix the assignment to wrong variable

We should be assigning value to minFreeMetadata instead of minFreeData. This
is copy/paste error.
Signed-off-by: Vivek Goyal<EMAIL_ADDRESS>

LGTM

LGTM


Support Phi-2

Description

Add support for Phi-2.
Corresponding documentation PR

TBD
Types of change

New feature.
Checklist

[x] I confirm that I have the right to submit this contribution under the project's MIT license.
[ ] I ran all tests in tests and usage_examples/tests, and all new and existing tests passed. This includes

all external tests (i. e. pytest ran with --external)
all tests requiring a GPU (i. e. pytest ran with --gpu)

[ ] My changes don't require a change to the documentation, or if they do, I've added all required information.

Removed Mixtral. Will update this and add a doc PR after https://github.com/explosion/spaCy/pull/13288#issuecomment-1929477779 has been merged.


[SPARK-12507][Streaming][Document]Expose closeFileAfterWrite and  allowBatching configurations for Streaming

/cc @tdas @brkyvz

Test build #48251 has started for PR 10453 at commit 4295137.

Test build #48251 has finished for PR 10453 at commit 4295137.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48251/
Test PASSed.

Merged build finished. Test PASSed.

Let's improve the title of items like this. "Update x" is never descriptive

I have a few comments on phrasing but otherwise it lgtm

@BenFradet Addressed. Thanks for your reviewing.

Test build #48436 has started for PR 10453 at commit bce7a29.

Test build #48436 has finished for PR 10453 at commit bce7a29.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48436/
Test PASSed.

LGTM

Maybe we can also include that allowBatching is not just helpful when closeFileAfterWrite is enabled, but is also very helpful to scale to a large number of receivers 50+ for example.

Maybe we can also include that allowBatching is not just helpful when closeFileAfterWrite is enabled, but is also very helpful to scale to a large number of receivers 50+ for example.

I guess enabled should be disabled?

Test build #48516 has started for PR 10453 at commit 7d9b038.

Test build #48516 has finished for PR 10453 at commit 7d9b038.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48516/
Test PASSed.

Test build #48971 has started for PR 10453 at commit 4d55b03.

Test build #48971 has finished for PR 10453 at commit 4d55b03.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48971/
Test PASSed.

just one more comment. then LGTM.

Test build #48980 has started for PR 10453 at commit 28a750d.

Test build #48980 has finished for PR 10453 at commit 28a750d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48980/
Test PASSed.

Merged build finished. Test PASSed.

LGTM. Merging this to master and 1.6. Thanks!


Focus inputbox when its label is clicked

Mouse pointer changes pointer face on <label>, but nothing occured when it was clicked. I added for attributes to <label>, id to <input> or <select> so they get focus when its label is clicked.

LGTM

thanks X)


Refactor utils modules

Let's create back branch cpp-utils.
Let's refactor then all utils modules by inheriting from OpenCV classes.

Completed.


Custom Highlight: CursorLineNr has no effect

Hi, I've tried setting my CursorLineNr to a brighter colour, but doing this in init.lua with the following has no effect:
require('onedark').setup {
    colors = {
        cursor_line_nr_white = "#ffffff", -- define a new color
    },
    highlights = {
        ["CursorLineNr"] = { fg = '$cursor_line_nr_white' },
    }
}
require('onedark').load()

If I try to set LineNr to this colour it works. I tried editing the values directly in ~/.local/share/nvim/lazy/onedark.nvim/lua/onedark/highlights.lua and I have the same issue: CursorLineNr does not change (but changing LineNr colour does).
I'm probably missing something obvious but for the life of me can't figure out what.
Neovim version 0.8.3
Lazy plugin manager
based off kickstart.nvim

After a bit more tinkering I figured out that vim.opt.cursorline = true needs to be set to make this work.


How do I write an enum as a raw string?

Let's say that I have an enum type, for which i've "enabled stringification".
enum class Color
{
    Red, Green, Blue
};

...
static constexpr auto value = enumerate(Red, Green, Blue);
...

I will also use this as a type for a member in a struct.
struct ColorWrapper
{
     Color color { Color::Green };

     struct glaze
     {
          using T = ColorWrapper;
          static constexpr auto value = glz::object("color", glz::raw<&T::color>);
     };
};

What I expect is that the value of the color will appear as a raw string in the json output, however the actual value will still have quotes around it. Is there a way in which I can solve this problem?
Desired output:
{
    "color": Green
}

Thanks for brining this up, enums were simply overlooked, but support has now been added in #1071


Fix tag bug

Fixes #2751. Cleans up metadata editor tag code to fix tag saving bug.
Developer's Certificate of Origin 1.1
   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

Thanks @kiersten-stokes and @karlaspuldaro for testing it out! @karlaspuldaro I added a fix for the empty spaces bug you found cause it was a quick fix.


feat(fastify): Support {hookName: 'onRequest'}

Type of change

[ ] 🐛 Bug fix
[x] 🌟 New feature
[ ] 🔨 Breaking change
[ ] 📖 Refactoring / dependency upgrade / documentation
[ ] other:

Packages affected

[ ] @clerk/clerk-js
[ ] @clerk/clerk-react
[ ] @clerk/nextjs
[ ] @clerk/remix
[ ] @clerk/types
[ ] @clerk/themes
[ ] @clerk/localizations
[ ] @clerk/clerk-expo
[ ] @clerk/backend
[ ] @clerk/clerk-sdk-node
[ ] @clerk/shared
[x] @clerk/fastify
[ ] @clerk/chrome-extension
[ ] gatsby-plugin-clerk
[ ] build/tooling/chore

Description

[x] npm test runs as expected.
[x] npm run build runs as expected.

Adds support for option {hookName: 'onRequest'}, which binds the callback to the specified hook.
By default, or when specified as such, the callback will continue to be bound to preHandler.
This update prevents binding to any other hook to minimize compatibility requirements going forward.

Fixes #1350

That's exactly what causes the error. The type of opts.hookName is already 'onRequest' | 'preHandler'. I've encountered this issue in TS before. Discrimination doesn't work as expected with overloaded functions. So you either have to split up the union or use @ts-expect-error.
Here's the error:
No overload matches this call.
  The last overload gave the following error.
    Argument of type '"onRequest" | "preHandler"' is not assignable to parameter of type '"onClose"'.
      Type '"onRequest"' is not assignable to type '"onClose"'.ts(2769)
instance.d.ts(467, 3): The last overload is declared here.

And here's the type that causes the issue:
type Hooks = Parameters<FastifyInstance['addHook']>[0]; // === 'onClose' SERIOUSLY!

Anyway, I updated the code to read more simply. I hope that works!

Merging so we can get this to staging.
Thank you @mikestopcontinues

This PR has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.


where is attrs.pkl and trajs.pkl

Excuse me, where are the attrs.pkl and trajs.pkl files? They don't seem to be in the project.Looking forward to your reply

I also want to ask the question, what is the format and meaning of trajectory data and 'head'?


Issue 146 - encrypting passwords

USERS_PASS_ENCRYPTED & usersPassEncrypted tells proxy that passwords are encrypted.
usersPath parameter is a mean to tell proxy to use users from this file for given service.
It contains changes from my PR https://github.com/vfarcic/docker-flow-proxy/pull/145. My bad because I've messed up merging. Anyway this PR is just to have some place to discuss changes. Hope you  don't mind.

A note for later... Once we agree on the design and implementation is done, we need to update the docs. Otherwise, no one will know about it :)

Thx for code review, as soon as I'll get in touch with my pc I'll fix all that.
If you don't mind my english I'm willing to update the docs of course.
Changing the param to usersSecret and looking for file in /run/secrets/df_{usersSecret} would be encouraging to actually use secrets from docker and will be consistent with how USERS env variable works. Reducing it to one file will not solve my use case which is the reason for this PR, so I vote no:)
The idea regarding sending just user names and using central store is quite interesting, but i would make it work together with usersSecret parameter as described above. So the service could decide which data store to use and optionally which users from it. It's not too hard to do, and straightforward to test so I'm in. If usersSecret is absent and only user names are in users then usersSecret defaults to USERS (the env var or secret, global anyway).
BTW I didn't know how to integration test usersPath parameter because from test I have no access to host as they are run in docker which spawns new containers through mapped socket on host so I quit. It would require some remodeling of test procedure (additional environment variables or dind containers or some hack like running container that just to figure out where the project is located on host). I just didn't want to make to much revolution in it as you put a lot of effort to put it all together and lokks like really thought through stuff, so I decided to consult it with you firs: how to mount some file in proxy container?
About all your concerns on when secret file will appear, my use case, I feel I should have started from that.
I just want to split traffic coming through proxy to some categories:

some services would have authentication on proxy, mainly management services grafana, kibana, prometheus etc
some services would also have authentication but for other set of users (consul, portainer etc - stuff that can break other stuff)
application traffic would go pass through or just terminate ssl - no authentication on proxy

How i see it: I would create secrets /run/secrets/df_monitoring, /run/secrets/df_admins even before I start proxy. So passwords will be there when services start. Then I would just label my services with com.df.usersSecret=monitoring or com.df.usersSecret=admins. The only problem I see is how will i tell proxy to reload users information when i change it. Hmm... I'll update proxy service? No idea actually. It would be nice if proxy was monitoring changes in user files. But I thing the whole approach of basic authentication in proxy is a poor man solution so changes in these files should not be frequent. As I look at stability of swarm mode I'm afraid that docker will take care of reloading my proxy from time to time, and will keep secrets in sync.
To reacp:

fix code
change usersPath to usersSecret
add Ability to send users without password (again if no users found I'd generate some random one)
fix the documentation

What do you think?

One more thing regarding problem with my merge. This PR consists of 2 commits (currently) one of them is already in your master, so after merge it should look almost ok. Because any time I try to fix sth in git history I tend to do a lot of damage I'd suggest I close this PR ad file it again, because this additional commit in this PR is already in master It would look exactly as it should then.

Sorry for not responding earlier. I was on a trip until now.

If you don't mind my english I'm willing to update the docs of course.

👍

Changing the param to usersSecret and looking for file in /run/secrets/df_{usersSecret} would be encouraging to actually use secrets from docker and will be consistent with how USERS env variable works. Reducing it to one file will not solve my use case which is the reason for this PR, so I vote no:)

Let's go with multiple files :)

The idea regarding sending just user names and using central store is quite interesting, but i would make it work together with usersSecret parameter as described above. So the service could decide which data store to use and optionally which users from it. It's not too hard to do, and straightforward to test so I'm in. If usersSecret is absent and only user names are in users then usersSecret defaults to USERS (the env var or secret, global anyway).

👍

How i see it: I would create secrets /run/secrets/df_monitoring, /run/secrets/df_admins even before I start proxy. So passwords will be there when services start. Then I would just label my services with com.df.usersSecret=monitoring or com.df.usersSecret=admins.

Maybe it would be a good idea to use some prefix that would distinguish those files from other types of secrets. For example, it could be user_monitoring. Having some kind of a convention would help with one of the next questions (read on).

The only problem I see is how will i tell proxy to reload users information when i change it.

If I understood correctly what you're trying to do, you should be able to reload proxy config when users change through the reload request. A few days ago I added the recreate param that should do the trick.

It would be nice if proxy was monitoring changes in user files.

That would indeed by a nice feature. We can spawn a separate go routine that could monitor files and act when they change. If file naming uses some convention (e.g user_*, cert_), that should be fairly easy.

But I thing the whole approach of basic authentication in proxy is a poor man solution so changes in these files should not be frequent.

That's my main concern with file monitoring. Is it worth having it when changes are very infrequent?

As I look at stability of swarm mode I'm afraid that docker will take care of reloading my proxy from time to time, and will keep secrets in sync.

True. On top of that, proxy is also reloaded on many other actions (create/remove a service, add a cert, and so on).

fix code
change usersPath to usersSecret
add Ability to send users without password (again if no users found I'd generate some random one)
fix the documentation
What do you think?

Sounds great :)

I'd suggest I close this PR ad file it again.

Agreed. Closing this PR.


Crash with using FlutterIsolate

I need to use Flutter Isolate (https://pub.dev/packages/flutter_isolate). I dont call api of your package in created isolate. but after kill isolate I got exception on your package.
Version of flutter_inapp_purchase
2.1.1
Platforms you faced the error (IOS or Android or both?)
iOS
Tested environment (Emulator? Real Device?)
both
Steps to reproduce the behavior
make spawnIsolate and after kill.
App will crash and you will see in console something like this:

*** First throw call stack:
(
0   CoreFoundation                      0x00007fff23c7127e __exceptionPreprocess + 350
1   libobjc.A.dylib                     0x00007fff513fbb20 objc_exception_throw + 48
2   CoreFoundation                      0x00007fff23c70ff8 +[NSException raise:format:arguments:] + 88
3   Foundation                          0x00007fff256e9b51 -[NSAssertionHandler handleFailureInMethod:object:file:lineNumber:description:] + 191
4   Flutter                             0x0000000101da3b16 -[FlutterEngine setMessageHandlerOnChannel:binaryMessageHandler:] + 271
5   flutter_inapp_purchase              0x0000000103e8917b -[FlutterInappPurchasePlugin dealloc] + 171
6   libobjc.A.dylib                     0x00007fff514110d6 _ZN11objc_object17sidetable_releaseEb + 174
7   libsystem_blocks.dylib              0x00007fff522d09c2 _Blo<…>

Same here.
I don't use flutter_isolate directly, but I use the audio_service plugin, which internally uses flutter_isolate. The flutter_inapp_purchase cause crash when audio_service is destroying.
Any help?

@siqwin Thanks for sharing. So this doesn't look it's on our plugin so I'll close it for now.


 This causes Thanos sidecar to upload new ones (without removal of old ones) which causes compactor to stop because of overlap.
Race conditions (rewrite during Thanos shipper upload)

It is worth to note that it does not make sense much to do delete_series on local Prometheus with Thanos backup, because most likely all blocks are already uploaded at the moment when Prometheus replaces these with rewritten blocks. So deleted series will be seen only locally. Plus every older block will have still that series
Solution:

We don't want to replace the blocks in sidecar. We don't want keep whole process coordination free, so only compactor is allowed to remove blocks on regular basis (compaction process)
We could instead of specifying which ULIDs where deployed in Thanos meta, specify also time ranges, and ignore or error out the rewritten blocks with new ULID and same time range.

Thanks [slack]@jean-louis for spotting this!

Great we found the root cause :)
Even better would be (in a dream scenario), that you can do delete_series in Thanos. And that Thanos deletes series in the bucket also.
But it might be a good start to let the compactor just delete the new/adjusted blocks on each run.

But it might be a good start to let the compactor just delete the new/adjusted blocks on each run.

What do you mean? I think it is way more clear to block this behaviour and be explicit about it. (not sure yet how). Basically if we do this ("let the compactor just delete the new/adjusted blocks on each run.") we have few problems:

Maybe these new blocks are not because of delete_series but some other configuration errors? (e.g same external labels)
User is confident that actually she/he deleted these series everywhere, but actually, fresh series are only deleted.
What if user deleted series before Thanos being able to upload some of the blocks? Really big place of inconsistencies.

All of this makes me think to do either:
a. Block or explicitly do not allow this logic
b. Block or explicitly do not allow this logic, but implement some batch job code that will delete series from the whole history (from bucket) - actually user can implement that code on its own even now.
c. Integrate delete series with the whole stack (sidecar + compactor) - but I think that is way to complex for the actual value.
Can we first enumerate what are the use cases for delete series?
I thought the main one is to reduce the amount of unnecessary metric after they are stored in Prometheus, but with Thanos, space is not that much of a problem.
Second, maybe some "right to be forgotten" rule, but never seen that needed in metrics.
Any ideas?

I guess, it's a rare case. TSDB buckets are supposed to be immutable, so deleting the series from the storage means literally recalculate whole data, which could be unwarrantedly expensive.
Bear in mind this is seldom case, to solve the right to be forgotten problem, it could be a query/store side decision: somehow preserving/applying rules (could be limiting by time ranges to apply to requests when series had existed?) to the series you want to be deleted, just skipping them in a query time.

@xjewer: its rare indeed. And doesn't happen by default.
But at least it should be documented/handled in Thanos when it does happen :) Or there should be a warning in the docs.

I have an issue where the delete series from S3/GCS is really needed.
When running compactor I get:
level=error ts=2018-08-20T21:55:21.849911888Z caller=main.go:160 msg="running command failed" err="compaction: gather index issues for block /var/lib/thanos-compact/compact/0@{monitor=\"master\",replica=\"1\"}/01CN46CYFR2CQCH567H5Q33007: out-of-order label set {ArchPath=\"MC.BBM.whatever1.cluster.whatever2\",__name__=\"test_metric\",az=\"us-east-1b\",cluster=\"whatever\",env=\"stage\",instance=\"xxx\",job=\"node\",key1=\"0\",key1=\"1\",service=\"xxxx\"} for series 346654"
as someone created some funky metrics which breaks the compactor process. The only reasonable way I can continue using Thanos now is by deleting these metrics from S3. I cannot afford to delete all the blocks

Small update: the "out-of-order label set" error in the compactor was caused by the honor_labels: true option for that specific job. I guess Prometheus label overwrite messes up the way Thanos handles the metrics when compacting.

Cool, have you put an issue on Prometheus github?
Let's figure out how to unblock you. I think that all of those issues, like out of order label set etc, is nice to know and super important to solve, but fixing this is generally not necessary for overall thanos compaction. It seems like the solution here would be to just alert on this (produce metric, log) but continue the compaction as it's critical for system healthiness.  Will try to find some time to produce a fix for this.
Not sure, if delete series helps here as you would have something produced broken series continously (: so you would endup deleting this series every 3h (:

Can we first enumerate what are the use cases for delete series?
I thought the main one is to reduce the amount of unnecessary metric after they are stored in Prometheus, but with Thanos, space is not that much of a problem.

This is generally true, but there is another dimension to consider: memory. We have been experiencing recently problems with the thanos store processes being OOMkilled. The underlying cause is the number of series we have in the object bucket, which is significant as a result of a long period of time in which we (inadvertently) have been uploading metrics with high cardinality dimensions. Now that we have identified the issue, we'd like to be able to remove those metrics from the storage in order to reduce the memory footprint of the store processes, but alas, I haven't found a way to do so yet that does not involve removing most of the blocks.

Ok, so:
Use cases
- Reduce cardinality once happen and fixed root cause.
- Potentially, "right to be forgotten" rule?

We have a couple of options:

Tombstones. This is what Prometheus uses for blocks. Tombstones are used when querying to exclude certain series, but also used on compaction. Compactor understand those and removes the series on certain time ranges from there. This is faster than rewriting whole block on the request, but what if there are more compactions for the block? (it's the biggest one). It also does not help immdiately with block size, you need to wait for compaction to sort it out...
Rewrite of whole block. This is time and resource consuming process, but doable. The question is where. Compactor? CLI tool? if tool then how to sync this with compactor?

Can we first enumerate what are the use cases for delete series?
I thought the main one is to reduce the amount of unnecessary metric after they are stored in Prometheus, but with Thanos, space is not that much of a problem.

This is generally true, but there is another dimension to consider: memory. We have been experiencing recently problems with the thanos store processes being OOMkilled. The underlying cause is the number of series we have in the object bucket, which is significant as a result of a long period of time in which we (inadvertently) have been uploading metrics with high cardinality dimensions. Now that we have identified the issue, we'd like to be able to remove those metrics from the storage in order to reduce the memory footprint of the store processes, but alas, I haven't found a way to do so yet that does not involve removing most of the blocks.

I have the same scenario, I found metrics with high cardinality and need to remove them because of.

Store is going down due to the amount of data it fetches from S3.
Query memory goes pretty high due to the amount of data store retrieves.

AFAIK the only solution would be either scaling up thanos resources or to delete data from S3?

Related to https://github.com/improbable-eng/thanos/issues/903

is it possible to just delete entire tsdb blocks?
for me the first blocks had a lot of metrics with high cardinality, and, if deleting those metrics is not possible, I would be happy to be able to delete entire blocks instead.

@caarlos0 you can do that just fine - but better turn off compaction temporarily for that. The concern is that compactor might be compacting that very block to bigger block and thus deletion would not help.


Bot autonomously votes on-chain on behalf of the group via Governance Proxy which is assigned to the group’s Pure Proxy

Proxy voting implemented


[IPMI] ipmi poweron and off test cases

[x]  ipmi poweron
[x] ipmi poweroff

Instead of making a separate testcase as IPMI power on & off .... i would like to add it as a keyword in utility. which can be reused.. @prkatti1 also has this keyword requirement.


Slightly saner {Flowable,Observable}Operator shutdown semantics

An operator is between an upstream Flowable and a downstream
subscriber/subscription.  The stream of signals it receives from one is not
synchronized with the other.  e.g. We can't assume that we still have a
downstream subscriber when FlowableOperator::onComplete() is called from
the upstream as the subscriber might have sent a cancel just before.
We consider an operator "terminated" if it has lost its references to its
upstream and downstream.  A terminated operator can still have methods called
upon it, so we need to make sure we don't send any signals from one side to the
other.
Enables the Observable.SimpleTake test, which previously hit ASAN errors because
of this bad behavior.

Good update. I would ask if we should favor simplicity over code duplication. It looks like all the terminate/terminateErr/cancel/onComplete/onError functions are almost the same.

I'll see if I can merge them a bit.


Be able to debug WASM modules

We currently can't debug our WASM modules (that we run not in a browser), this will become a bigger and bigger issue going forward so we should investigate what are the paths for getting some type of debugging support either with wasmer or wasmtime.
We would want it to be super easy to debug the code for a small (or big) WASM module directly in VS Code as well as in other editors.
Related:

[ ] wasmer: Add support for step debugging and Virtual Machine introspection

A probably related link, screencast of debugging with wasmtime.

I'd be interested in hearing more about what you need to fully address this. As shown in the link @pepyakin posted, we support LLDB debugging on Linux and macOS in Wasmtime, which hopefully should go a long way to addressing this. Windows is trickier, but apparently feasible.
Beyond just supporting more platforms, are there specific features or pieces of functionality you'd need for this to be usable?

@tschneidereit sorry I missed your comment here, so very late reply ☹
the wasmtime work on debugging looks promising! though assume it is primarily working for loose WASM modules loaded & run through the wasmtime executable host?
The use case we after is debugging WASM modules loaded by our own executable host that embeds a WASM runtime engine (like wasmtime), and ideally debugging using VS Code or other standalone IDE debuggers. Though I a CLI-based debugger interface like GDB/LLDB could also be a start.
Have there been any more developments with wasmtime around debugging and would the debugging support that it has be able to be used in the context of wasmtime embedded into our own executable host (that also implements multiple of our own WASM FFI APIs).

@repi no worries at all! 🙂

The use case we after is debugging WASM modules loaded by our own executable host that embeds a WASM runtime engine (like wasmtime), and ideally debugging using VS Code or other standalone IDE debuggers. Though I a CLI-based debugger interface like GDB/LLDB could also be a start.

If I understand the use case correctly, that should work perfectly well: Wasmtime debugging is not in any way tied to the wasmtime CLI. You should be able to connect a debugger to the host application and step through it as you normally would, and things should continue working when stepping into a Wasm call.
By the same token, debugging in VS Code should work just the same as it would for debugging the host application.

Have there been any more developments with wasmtime around debugging and would the debugging support that it has be able to be used in the context of wasmtime embedded into our own executable host (that also implements multiple of our own WASM FFI APIs).

How the Wasm/host context switch happens shouldn't make much of a difference, so the fact that you have your own FFI APIs shouldn't affect things much. (Also, I'd be amiss if I didn't mention the possibility of defining host APIs using witx, as described in this very nice post by @radu-matei.)

As for platforms, we work pretty evenly split between Windows, Linux and Mac OS. Though having an initial solution that works on Linux & Mac that would work for most of us

Windows should actually work by now. We haven't tested it too much, but debugging in VS Code using LLDB seems to work just fine.
If you end up looking into this more, don't hesitate to reach out and let us know if you run into issues — or just generally how things are going, and what could be improved!

@tschneidereit oh that is good news! both that debugging with LLDB of WASM modules should just work inside VSCode and for Windows. we'll try it out!

Also, I'd be amiss if I didn't mention the possibility of defining host APIs using witx, as described in this very nice post by @radu-matei.)

Interface types and WITX is one of the things we've been eyeing and are interested in trying out with wasmtime, haven't seen that specific article though so that is a good resource, thanks!
We have a pretty extensive set of versioned private APIs right now that we define the FFI and Rust interfaces for using Rust + some proc macros that @Jake-Shadle created. A bit like wasm-bindgen macros with higher-level types in but where we generated both the FFI and the safe host and module Rust APIs for, it is a but custom for our use though but has been a good help.

If you end up looking into this more, don't hesitate to reach out and let us know if you run into issues — or just generally how things are going, and what could be improved!

Thanks, we are starting up looking into wasmtime and experimenting with it in more detail now again and was planning to reach out to you all soon when we have some more concrete topics and questions to discuss

Our in-house wasmtime embedder now has the ability to enable debug information generation, which is a huge improvement over printf debugging 🎉
Right now, this enables us to run our app in a recent version of LLDB, and then allows us to:

set breakpoints in wasm code generated from rust,
use control flow debugging commands (next, step, etc.) to observe control flow,
look at the backtrace (on platforms where unwinding is fully supported, that is, not aarch64-darwin right now)
in some cases, even inspect the content of variables (local, parameters, etc.), after running the __vmctx->set() lldb command. Yet it is a bit fragile: in some cases we're getting the escape hatch warning ("variable may have been optimized out"), which is fine, but in other cases we're seeing trash values in place of what should be correct. I'll try to make smaller repros, so as to report them to wasmtime's main repository. During latest Cranelift's meeting we even talked about what could be one cause of these bugs.

Overall it's pretty nice that enabling debug info support was just a configuration toggle to enable, and people in the wasmtime's zulip have been pretty helpful explaining how to get variable inspection to work at all!
We can consider this particular issue closed: while debugging support is not complete yet, we now have some debugging support, that is a very nice start and something we can build upon.


pytest worker crash seen on newly introduced unittest test_profiler_gpu.py::test_aggregate_duplication

Description
tests/python/gpu/test_profiler_gpu.py has recently started importing unittests/test_profiler.py, and so now runs those tests for the first time with a gpu default context.  With that change, I have seen on centos-gpu:
worker 'gw3' crashed while running 'tests/python/gpu/test_profiler_gpu.py::test_aggregate_duplication'

Occurrences
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fcentos-gpu/detail/PR-18694/11/pipeline
@leezu You might want to look into this or watch for more occurences.
What have you tried to solve it?

I've bypassed this test in my current PR with a del test_aggregate_duplication after the newly introduced import.

@DickJC123 I'm not familiar with the feature, but @Zha0q1 added the feature and may help


Read a product

Reading a single product by ID:
As a developer, I need to create an API endpoint that allows users to view detailed information about a specific product using its unique ID. The product is identified by a unique ID in the database, such as a MongoDB ObjectId.. Therefore the user can search for a specific product by ID.

Paginating the list of products:
As a developer, I need to implement pagination in the API to handle large datasets efficiently. The database may contain a large scale of products, so loading them all at once could cause performance issues. When a GET request is made to the products endpoint, the response should support pagination using query parameters like page and limit.

Filtering products by price range:
As a developer, I need to allow users to request products within a specified minimum and maximum price range. This allows a GET request to be made with minPrice and maxPrice query parameters, the API should return products within that price range.

Searching products by name:
As a developer, I need to implement a search function that allows users to search for products by name. Users may search using partial or full name matches. Therefore when a GET request is made with a search term, the API should return products whose names contain the search term in a case-insensitive manner.

Sorting products by price:
As a developer, I need to implement sorting so that users can view products in ascending or descending order by price. So that when a GET request is made with sort and order query parameters, the API should return products sorted by price accordingly, either ascending or descending.


runtime: inappropriate deadlock detection on windows with syscall.NewCallback

What version of Go are you using (go version)?

# go version
go version go1.20.2 windows/amd64

Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (go env)?
go env Output
# go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\jack\AppData\Local\go-build
set GOENV=C:\Users\jack\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\jack\go\pkg\mod
set GONOPROXY=*
set GONOSUMDB=*
set GOOS=windows
set GOPATH=C:\Users\jack\go
set GOPRIVATE=*
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Users\jack\scoop\apps\go\current
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Users\jack\scoop\apps\go\current\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.20.2
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=0
set GOMOD=C:\Users\jack\Src\go-toast\go.mod
set GOWORK=
set CGO_CFLAGS=-O2 -g
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-O2 -g
set CGO_FFLAGS=-O2 -g
set CGO_LDFLAGS=-O2 -g
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=C:\Users\jack\AppData\Local\Temp\go-build1330827647=/tmp/go-build -gno-record-gcc-switches

What did you do?
When interacting with Windows COM, Go detects a "deadlock" while waiting for the Windows Runtime
to call the program back. In particular I have authored git.sr.ht/~jackmordaunt/go-toast which
implements Windows Toast Notifications. We create a "callback" COM object using a Vtable populated
with syscall.NewCallback function pointers.
The program issues the toast and waits for the Windows Runtime to invoke our syscall.NewCallback
wrapped Go function.
The Go runtime believes this is a deadlock, when in reality the Windows Runtime might call us back.
The following reproducer demonstrates the deadlock behavior. The non-deadlock function uses a noop goroutine to trick the Go runtime into not panicking. The reproduces invokes the git.sr.ht/~jackmordaunt/go-toast package for real because I don't know how to reproduce this without engaging the Windows Runtime.

Reproducer

package main

import (
	"flag"
	"fmt"
	"time"

	"git.sr.ht/~jackmordaunt/go-toast"
)

func main() {
	var (
		runDeadlock bool
	)

	flag.BoolVar(&runDeadlock, "deadlock", false, "run the deadlocking function")
	flag.Parse()

	if runDeadlock {
		deadlock()
	} else {
		noDeadlock()
	}
}

func noDeadlock() {
	done := make(chan struct{})

	toast.SetActivationCallback(func(args string, data []toast.UserData) {
		fmt.Printf("args %q, data %v\n", args, data)
		done <- struct{}{}
	})

	n := toast.Notification{
		AppID:          "deadlock test",
		Title:          "testing deadlock",
		Body:           "deadlock occurs when waiting on Windows Runtime",
		ActivationType: toast.Foreground,
	}

	n.Push()

        // Useless goroutine tricks the Go runtime to avoid deadlock panic.
	go func() {
		for range time.NewTicker(time.Second).C {

		}
	}()

	<-done
}

func deadlock() {
	done := make(chan struct{})

	toast.SetActivationCallback(func(args string, data []toast.UserData) {
		fmt.Printf("args %q, data %v\n", args, data)
		done <- struct{}{}
	})

	n := toast.Notification{
		AppID:          "deadlock test",
		Title:          "testing deadlock",
		Body:           "deadlock occurs when waiting on Windows Runtime",
		ActivationType: toast.Foreground,
	}

	n.Push()

	<-done
}

What did you expect to see?
The program should wait until it is called back by via the syscall.NewCallback mechanism, rather than panicking.
What did you see instead?
fatal error: all goroutines are asleep - deadlock!

goroutine 1 [chan receive]:
main.deadlock()
        C:/Users/jack/Src/go-toast/examples/deadlock-bug/main.go:69 +0xb1
main.main()
        C:/Users/jack/Src/go-toast/examples/deadlock-bug/main.go:20 +0xbb
exit status 2

I believe this is a duplicate of https://github.com/golang/go/issues/55015


Add JVM Parser

I created a first implementation of a TAML parser in Java.
Java library is based on .NET project.
This pul requested is linked to #28

Looks good...  as a follow up, I've added #86 to help ensure your code tests properly and is ready to be published

Looks good...  as a follow up, I've added #86 to help ensure your code tests properly and is ready to be published


[QUINTEROS]  Lock the release branch to test with ruby 3.0 / Fixup CI 'if' ruby-version check for 3.0

[QUINTEROS]
Lock the release branch to test with ruby 3.0
Fixup CI 'if' ruby-version check for 3.0

Replaces #22675 22675


Processing exits from in-flight transaction fails when the transaction has ETH and ERC20 token outputs

Issue Type

[x ] bug report
[ ] feature request

Current Behavior
Processing exits from in-flight transaction fails when the transaction has ETH and ERC20 token outputs
Expected Behavior
Exits are successfully processed.
Steps to Reproduce

Start in-flight exit from a transaction with 2 outputs: ETH and ERC20.
Piggyback both.
Call processExits twice. The second call fails.

Suggested Fix
Change how exit game controller handles exit processor for a given exit in enqueue and processExits functions.

oh.....awww, this happens since priority is not unique 🤦♂
Nice catch. Is this found on the way of migrating python integration tests?

@boolafish yes, I enabled python tests and found out that failing tests have something in common.

ref, the change of removing the nonce into priority brings this issue: https://github.com/omisego/plasma-contracts/issues/283
Might consider to revert the change to solve the root cause if we see any non-solvable issue by the PR #537 .


Namespaced send example in Readme

From the readme:

Outside the model they're called by send('todos:add') and state.todos.items. Inside the namespaced model they're called by send('todos:add') and state.items

I suspect it should be:
Outside the model they're called by send('todos:add') and state.todos.items. Inside the namespaced model they're called by send('add') and state.items
Or else I don't understand because it is called the same way?

It is called the same way. There used to be a difference (exactly as you suspected it was), but people were running into problems with namespaces being too restrictive - so we both loosened up the send() call namespaces and provided a way for hierarchical send() calls in v3.
We might restore the namespaced behavior for send() as I think the cleanliness makes a lot of sense; but before we do that it's probably best to lay out a good story for async / provide sufficient examples and documentation.
Does this answer your question? Cheers!

Definitely answered my question, thanks!
A quick follow up question in the same vein. Is there a way for a namespaced model to read data from state from outside it's own namespace? I have not used choo yet just looking into it, but in my current app we have a part of state that holds immutable product data that all parts of the app are allowed to read. Trying to figure out how this would be done with choo.

Just to add to the above, it's mainly in the effects I see the need to access "global" state for namespaced modules. Another way to do it if many namespaced modules need the exact same data is that they share a reducer and project the same state within their namespaces which can then be read in the effects. Not sure if this is the recommended way in choo?

@jonaskello no, namespaced models cannot read data from outside; feel free to chime in on #252 as state management is definitely an issue we're trying to resolve correctly; input of any kind would be very helpful. Thanks!


doesn't work with mohist 1.16.5 last update 392

i can't use worldedit and fast async, your version or official...

Now, Worldedit is directly fix in mohist...


EVEREST-1247 | Re-design of operator upgrades API

Adds new APIs for operator upgrades such that all operators in a namespace are upgraded at once. This is done due to an OLM limitation that doesn't actually allow us to individual upgrade an operator in the namespace.

Add GET /namespaces/{namespace}/database-engines/upgrade-plan

Returns a list of pending upgrades in the given namespace
Returns a list of pending action items in order to perform the upgrades
If there are no upgrades, the action items contains post-upgrade tasks

Add POST /namespaces/{namespace}/database-engines/upgrade-plan/approval

Starts the operators upgrade process

Earlier, the actual upgrade itself was handled by everest-operator for each individual dbengine. Now that this is no longer possible, we directly approve InstallPlans from the Everest API. (A corresponding PR has been created in everest-operator to remove this functionality) - https://github.com/percona/everest-operator/pull/452
Older APIs are still kept for backward compatibility for now. They should be removed in future release. We must also refactor some code when we remove these deprecated APIs.

@mayankshah1607 can we merge this?


readme: Update install instructions

Coverage remained the same at 64.539% when pulling 40e13de7a8a333e6c4771cee772d2eaf95ca5e55 on EdDev:install-instructions into 33d6e1baea5dacf3f24f9565a0e18d7725c069ca on nmstate:master.


fix view part for new version Nova + dark theme

Thanks for this!
I merged it and will create a release.

Great. thank you!


Don't Merge! PR real time test for target branch full build

Docs Build status updates of commit b6c5ef0:
:clock10: Full build: the average full build time is 6 min(s) 5 sec(s), this is based on the last 333 build(s) for this repository.
:warning: This is a FULL build due to the incremental build being disabled in configuration file (.openpublishing.publish.config.json in 'live' branch). For more detailed info, please click here.

Docs Build status updates of commit b6c5ef0:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details
Dependent Files

docs-ref-conceptual/testing.md
:white_check_mark:Succeeded
View (azure-cli-latest) View (azure-cli-2017-03-09-profile) View (azure-cli-2018-03-01-hybrid)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Add ETL example notebooks from Crystal's branch

Rebased the examples from simple_femr_and_switch onto main and consolidated some of the files
This leaves us with two tutorials:

"Simple" Format: A tutorial encouraging ETL into the 4-column structure and running the "simple" ETL to create a PatientDatabase
"Custom" Format: A tutorial on the more specific details of FEMR, since at this point if they want to do something custom they'll need to write a Python-based ETL implementation and should have the fundamentals

Moved the other scripts to a reference folder since those workflows aren't intended for most end-users

FYI: in the diff view,  a lot more files are edited since I rebased onto main, and the commits on main might not have run through the formatter that is part of pre-commit (black).
The main files to focus on are the ipynb notebooks

Can you rerun both notebooks? I'm pretty sure they don't work now since there have been some API changes

database.get_code_dictionary()[3]

This line in particular will not run since that method has been deleted. print(event.code) should work as is without any dictionary mapping.

@Lalaland can we look to merge this and fix other issues later (i.e. iterate again on the tutorial).
I really want to make sure the branch doesn't go stale again with all of the changes in main that I may not be aware of. The next PR can be "cleaning-up" the tutorials which is going to be my focus rest of week


generate-wpt-report: fix WPT_REPORT path

Sorry, broke by https://github.com/GoogleChromeLabs/chromium-bidi/commit/179942ca5e7c6328cc267b6bd59101747e6353a2


Enable authentication for ElasticSearch/OpenSearch

Description
List of itesm to be added

Investigate auth plugin - (should also support Opensearch)
Investigate omnibus changes for omnibus build
Investigate es_gateway to add basic authentication header
Investigate habitat changes for habitat build

Issues Resolved
#2233
Check List

[ ] New functionality includes tests
[ ] All buildkite tests pass
[ ] Full omnibus build and tests in buildkite pass
[ ] All commits have been signed-off for the Developer Certificate of Origin. See https://github.com/chef/chef/blob/main/CONTRIBUTING.md#developer-certification-of-origin-dco
[ ] PR title is a worthy inclusion in the CHANGELOG

Pipeline Information -
Adhoc - https://buildkite.com/chef/chef-chef-server-main-omnibus-adhoc/builds/3315
Umbrella -


TestFlight build fails at startup with Termination Description: DYLD, dependent dylib<EMAIL_ADDRESS>not found for....

I'm getting an immediate startup crash when opening my App Store Connect / TestFlight build which requires the Swift v4.1.2 libs:
Exception Type:  EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note:  EXC_CORPSE_NOTIFY
Termination Description: DYLD, dependent dylib<EMAIL_ADDRESS>not found for
'/private/var/containers/Bundle/Application/6AE3A428-810B-4803-B5D5-7125C52E51C7/MyApp.app/Frameworks/Charts.framework/Charts',
tried but didn't find:
'/private/var/containers/Bundle/Application/6AE3A428-810B-4803-B5D5-7125C52E51C7/MyApp.app/../Frameworks/libswiftCore.dylib'
'/private/var/containers/Bundle/Application/6AE3A428-810B-4803-B5D5-7125C52E51C7/MyApp.app/Frameworks/Charts.framework/Frameworks/libswiftCore.dylib'
'/private/var/containers/Bundle/Application/6AE3A428-810B-4803-B5D5-7125C52E51C7/MyApp.app/Frameworks/libswiftCore.dylib'
<EMAIL_ADDRESS>'/usr/local/lib/libswiftCore.dylib' '/usr/lib/libswiftCore.dylib'
Highlighted by Thread:  0

I'm trying to use UltimateXF in a Xamarin Forms (v4.2.x) project which wraps iOSCharts v<IP_ADDRESS> (from @Flash3001).
My XF project uses PackageReferences and the Xamarin.Swift4 packages (as per the requirements of UltimateXF):
<PackageReference Include="Xamarin.Swift4">
    <Version><IP_ADDRESS></Version>
</PackageReference>

Xamarin.Swift4.Core etc v4.1.2

Everything works fine in all builds on Simulator and devices until I try to open a build archived, signed, uploaded and installed from App Store Connect.
I'm following the steps detailed here to create my ipa after archiving and signing, ensuring that the SwiftSupport folder is created and populated using the ipa-packager.sh script from https://github.com/bq/ipa-packager.
The UltimateXF library has wrapped iOSCharts at version <IP_ADDRESS> and requires using the Xamarin.Swift4 support packages at version 4.1.2, which as far as I can tell means I need to use the Swift libraries from Xcode 9.4.
I'm using Xcode 11 and so I've lightly modified the ipa-packager.sh script to use the xcode-select command to switch to a downloaded install of Xcode 9.4 to ensure the Swift libs are picked up from /Applications/Xcode9.4/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/iphoneos/ and copied in to the SwiftSupport folder which should be the correct 4.1.2 versions to match iOSCharts at version <IP_ADDRESS>.
After running the script and unpacking my modified ipa I can see that the SwiftSupport folder has been created and the Swift 4.1.2 dylib files are present:
ls -al SwiftSupport

-rwxr-xr-x   1 gaz  staff  66662432  7 Feb 14:06 libswiftCore.dylib
-rwxr-xr-x   1 gaz  staff    659808  7 Feb 14:06 libswiftCoreAudio.dylib
-rwxr-xr-x   1 gaz  staff    740160  7 Feb 14:06 libswiftCoreData.dylib
-rwxr-xr-x   1 gaz  staff    257616  7 Feb 14:06 libswiftCoreFoundation.dylib
-rwxr-xr-x   1 gaz  staff   1628432  7 Feb 14:06 libswiftCoreGraphics.dylib
-rwxr-xr-x   1 gaz  staff    307520  7 Feb 14:06 libswiftCoreImage.dylib
-rwxr-xr-x   1 gaz  staff    483504  7 Feb 14:06 libswiftCoreMedia.dylib
-rwxr-xr-x   1 gaz  staff    838608  7 Feb 14:06 libswiftDarwin.dylib
-rwxr-xr-x   1 gaz  staff   2722176  7 Feb 14:06 libswiftDispatch.dylib
-rwxr-xr-x   1 gaz  staff  30165120  7 Feb 14:06 libswiftFoundation.dylib
-rwxr-xr-x   1 gaz  staff    448768  7 Feb 14:06 libswiftMetal.dylib
-rwxr-xr-x   1 gaz  staff    553984  7 Feb 14:06 libswiftObjectiveC.dylib
-rwxr-xr-x   1 gaz  staff    310496  7 Feb 14:06 libswiftQuartzCore.dylib
-rwxr-xr-x   1 gaz  staff   1042912  7 Feb 14:06 libswiftUIKit.dylib
-rwxr-xr-x   1 gaz  staff    497040  7 Feb 14:06 libswiftos.dylib

They appear to closely match those found in app Framework folder although the file sizes are marginally different:
ls -al Payload/MyApp.app/Frameworks

-rw-r--r--    1 gaz  staff  66704320  7 Feb 14:01 libswiftCore.dylib
-rw-r--r--    1 gaz  staff    668944  7 Feb 14:01 libswiftCoreAudio.dylib
-rw-r--r--    1 gaz  staff    749280  7 Feb 14:01 libswiftCoreData.dylib
-rw-r--r--    1 gaz  staff    299520  7 Feb 14:01 libswiftCoreFoundation.dylib
-rw-r--r--    1 gaz  staff   1637568  7 Feb 14:01 libswiftCoreGraphics.dylib
-rw-r--r--    1 gaz  staff    349408  7 Feb 14:01 libswiftCoreImage.dylib
-rw-r--r--    1 gaz  staff    525408  7 Feb 14:01 libswiftCoreMedia.dylib
-rw-r--r--    1 gaz  staff    847744  7 Feb 14:01 libswiftDarwin.dylib
-rw-r--r--    1 gaz  staff   2764080  7 Feb 14:01 libswiftDispatch.dylib
-rw-r--r--    1 gaz  staff  30207024  7 Feb 14:01 libswiftFoundation.dylib
-rw-r--r--    1 gaz  staff    457904  7 Feb 14:01 libswiftMetal.dylib
-rw-r--r--    1 gaz  staff    595872  7 Feb 14:01 libswiftObjectiveC.dylib
-rw-r--r--    1 gaz  staff    352384  7 Feb 14:01 libswiftQuartzCore.dylib
-rw-r--r--    1 gaz  staff   1084800  7 Feb 14:01 libswiftUIKit.dylib
-rw-r--r--    1 gaz  staff    506160  7 Feb 14:01 libswiftos.dylib

Running otool confirms that the file versions do match even with the slight size differences and Finder > Get Info indicates the files in both folders are Version 4.1.2:
otool -l Payload/MyApp.app/Frameworks/libswiftCore.dylib  | grep "current version" | head -1
      current version 902.0.54

otool -l SwiftSupport/libswiftCore.dylib  | grep "current version" | head -1
      current version 902.0.54

I then use the Transporter mac app to Verify the modified ipa (all ok) and upload it to TestFlight (passes processing) but when I install and run I'm at a loss as to why I'm seeing the dependent dylib<EMAIL_ADDRESS>not found error when it seems I've added the required Swift libs into a SwiftSupport folder and they should be present in at least one of the paths mentioned in the error message - any help or insight on how to resolve this would be much appreciated
Also, I've seen posts mention using a SwiftSupport folder and also mention using a SwiftSupport/iphoneos folder (I've tried both) - it would be good to understand which I should be using
Configuration
Apple Developer Tools:
Xcode 11.3 (15712)
Build 11C29
Xcode 9.4 (in a named Application subfolder)
Visual Studio Community 2019 for Mac:
Version 8.4.3 (build 12)
Xamarin.iOS:
Version: <IP_ADDRESS> (Visual Studio Community)

Further to this it appears the App Store Connect build will run on an old iPhone 6 (iOS 11.2.2) but none of my iOS 13 devices - is there anything different in how legacy Swift libs are handled in later iOS versions?

Shame not to get any responses on this as I'm curious to know if I've been trying to package legacy Swift 4 which simply isn't possible now since iOS 12.2 and Swift 5. If it helps anyone else I was unable to package my app so that it would run on both 11.2.2 and iOS 13 devices and ultimately ended up ditching the UltimateXF package as it is tied to Swift 4.1.2.
I switched instead to using the latest Flash3001/iOSCharts.Xamarin & Flash3001/MPAndroidChart.Xamarin binding packages directly with my own custom renderers to surface the chart views in my Xamarin Forms app, which works well and means the latest versions of the chart packages can be used instead of being tied to the older bound versions used in UltimateXF.
Using the latest Flash3001/iOSCharts.Xamarin binding meant I could remove all the Xamarin.Swift4.xxxx packages and just use the latest Xamarin.Swift v1.0.4 package as the binding is now built for Swift 5.x, which greatly simplifies App Store release packaging.
In closing this issue I'd like to say many thanks to the repo owner for creating and maintaining such a useful library.


Magically find exactly the requested non-key frame when seeking

This is the sort of thing you need to do when you want a precise (non-key) frame:
fh = av.open(args.movie)

video = fh.streams.video[0]

def iter_frames():
    for packet in fh.demux(video):
        for frame in packet.decode():
            yield frame

for i in xrange(100):

    pts = fh.duration * i / 100
    time_ = pts / float(av.time_base)

    fh.seek(pts)

    for j, frame in enumerate(iter_frames()):
        if j > 100:
            print 'Did not find target within 100 frames of seek.'
            break
        if frame.pts * frame.time_base >= time_:
            print 'Found target in', j, 'frames.'
            break

   # Do something with frame.

This should be done by default, and disabled by bare_bones=True (or whatever #209 decides).

I have encountered this issue too implementing a PyAV wrapper in PIMS:
https://github.com/soft-matter/pims/blob/master/pims/pyav_reader.py#L70
Including this seeking functionality within PyAV would be a great way to go. However, to me one of the advantages of using PyAV is exactly this "bare_bones" access into ffmpeg, which makes it really fast and also understandable from a ffmpeg point of view.
Personally, I would add all the "costly" magic (like this one) in a different class that is derived from the bare-bones ffmpeg interface. In that way, the distinction between "bare bones" and "magic" would be more clear, and also the flexibility in the API would be greater.

Hi there -- is this still the preferred way to find a precise non-key-frame in pyav? Acc to the docs, it looks like the use_magic type options never got implemented. Thanks!


Error: the option Z is only accepted on the nightly compiler

Hi everyone 👋
Today I'm have an issue
I installed everything according to the instructions at anchor-lang.com/docs/installation
I tried to build it, but I ran into such an error
BPF SDK: /Users/alexey/.local/share/solana/install/releases/1.10.31/solana-release/bin/sdk/bpf
cargo-build-bpf child: rustup toolchain list -v
cargo-build-bpf child: rustup toolchain link bpf /Users/alexey/.local/share/solana/install/releases/1.10.31/solana-release/bin/sdk/bpf/dependencies/bpf-tools/rust
cargo-build-bpf child: cargo +bpf build --target bpfel-unknown-unknown --release
error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `/Users/alexey/.rustup/toolchains/stable-aarch64-apple-darwin/bin/rustc - --crate-name ___ --print=file-names -Zremap-cwd-prefix= --target bpfel-unknown-unknown --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=cfg` (exit status: 1)
  --- stderr
  error: the option `Z` is only accepted on the nightly compiler

About my enviroment
OS Macbook m1pro Monterey
> rustc -Vv
rustc 1.62.0 (a8314ef7d 2022-06-27)
binary: rustc
commit-hash: a8314ef7d0ec7b75c336af2c9857bfaf43002bfc
commit-date: 2022-06-27
host: aarch64-apple-darwin
release: 1.62.0
LLVM version: 14.0.5

> rustup toolchain list
stable-aarch64-apple-darwin (default)
bpf

yarn  --version && solana --version && anchor --version
3.2.1
solana-cli 1.10.31 (src:77a40cd8; feat:4192065167)
anchor-cli 0.25.0

Thanks!

Lets check this issue https://github.com/solana-labs/solana/issues/26583


Msal.js has Integration testing for all important flows

@sameerag  to detail plan / goals / tests to be written

E2E tests now exist, closing.


Fix v2 SPI bugs and improve bsp_pins macro

SPI
The DATA register is only 16-bits for SAMD11 & SAMD21 chips, but it is
represented as a 32-bit register by the SVD. Change the SPI API to
accept u16 instead of u32.
Also correct a bug in SpiFuture, where the SS pin would never be
properly deasserted.
Thanks to @jbeaurivage for finding both of these.
bsp_pins
Allow commas after each pin block, and improve the docs to point out
where to find a list of PinModes.
Thanks to @TDHolmes for trying out the bsp_pins macro in #396.

The DATA register is only 16-bits for SAMD11 & SAMD21 chips, but it is
represented as a 32-bit register by the SVD.

Does this mean we should patch the SVD?

@Sympatron, the DATA field is correctly 16 bits, but the register itself is given as 32. The datasheet says it's only 16, but they probably reserve the upper bytes of the address space for code compatibility with SAMx5x chips. At least that's the conclusion @jbeaurivage and I came to. I don't really care either way. I think their SVD choice does probably make sense here.