6.4.0 certain playlist causes innertube.getPlaylist to report This playlist does not exist

Steps to reproduce

Save following script into a file e.g. script.js
Install youtube.js (I did this in FreeTube project folder)
node script.js

const { Innertube } = require('youtubei.js')

async function createInnertube(options = { withPlayer: false, location: undefined, safetyMode: false, clientType: undefined, generateSessionLocally: true }) {
  return await Innertube.create({
    enable_safety_mode: false,
    generate_session_locally: !!options.generateSessionLocally
  })
}

(async () => {
  const innertube = await createInnertube()
  let playlist = await innertube.getPlaylist('PLT1F2nOxLHOcmi_qi1BbY6axf5xLFEcit')

  const videos = playlist.items

  while (playlist.has_continuation) {
    playlist = await playlist.getContinuation()

    videos.push(...playlist.items)
  }

  console.info('Done!', videos.length)
})()

Failure Logs
// Playlist ID: PLT1F2nOxLHOcmi_qi1BbY6axf5xLFEcit

/Users/pikachuexe/projects/electron/FreeTube/node_modules/youtubei.js/bundle/node.cjs:11568
      throw new InnertubeError("This playlist does not exist");
            ^

InnertubeError: This playlist does not exist
    at new Playlist (/Users/pikachuexe/projects/electron/FreeTube/node_modules/youtubei.js/bundle/node.cjs:11568:13)
    at Playlist2.<anonymous> (/Users/pikachuexe/projects/electron/FreeTube/node_modules/youtubei.js/bundle/node.cjs:11593:14)
    at Generator.next (<anonymous>)
    at fulfilled (/Users/pikachuexe/projects/electron/FreeTube/node_modules/tslib/tslib.js:166:62)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  date: 2023-10-02T01:07:41.952Z,
  version: '6.4.0'
}

Expected behavior
No error
Current behavior
Error reported as seen in log
Version
Default
Anything else?

https://www.youtube.com/playlist?list=PLT1F2nOxLHOcmi_qi1BbY6axf5xLFEcit
https://github.com/FreeTubeApp/FreeTube/issues/4086

There is another error type reported for another playlist https://youtube.com/playlist?list=PLcyItF5YjT0dYq2E7Oti1pgBsRNq3A3jW
Not sure if should be reported here too
Error message is There are too many continuations, you'll need to find the correct one yourself in this.page
Checklist

[X] I am running the latest version.
[X] I checked the documentation and found no answer.
[X] I have searched the existing issues and made sure this is not a duplicate.
[X] I have provided sufficient information.

Fixed in https://github.com/LuanRT/YouTube.js/commit/8e372d5c67f148be288bb0485f2c70ec43fbecd0 & https://github.com/LuanRT/YouTube.js/commit/987f50604a0163f9a07091ce787995c6f6fddb75.

There is a new errors thrown (Same script)
playlist PLT1F2nOxLHOcmi_qi1BbY6axf5xLFEcit
The script should be getting just the 1st page here
https://www.youtube.com/playlist?list=PLT1F2nOxLHOcmi_qi1BbY6axf5xLFEcit
/Users/pikachuexe/projects/electron/FreeTube/node_modules/youtubei.js/bundle/node.cjs:11637
      throw new InnertubeError("Got empty continuation response. This is likely the end of the playlist.");
            ^

InnertubeError: Got empty continuation response. This is likely the end of the playlist.
    at new Playlist (/Users/pikachuexe/projects/electron/FreeTube/node_modules/youtubei.js/bundle/node.cjs:11637:13)
    at Playlist2.<anonymous> (/Users/pikachuexe/projects/electron/FreeTube/node_modules/youtubei.js/bundle/node.cjs:11662:14)
    at Generator.next (<anonymous>)
    at fulfilled (/Users/pikachuexe/projects/electron/FreeTube/node_modules/tslib/tslib.js:166:62)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  date: 2023-10-04T00:43:09.708Z,
  version: '6.4.1'
}

playlist PLcyItF5YjT0dYq2E7Oti1pgBsRNq3A3jW
https://youtube.com/playlist?list=PLcyItF5YjT0dYq2E7Oti1pgBsRNq3A3jW
ParsingError: Type mismatch, got FeedFilterChipBar expected CommentsHeader | ItemSectionHeader | ItemSectionTabbedHeader.
    at ERROR_HANDLER (/Users/pikachuexe/projects/electron/FreeTube/node_modules/youtubei.js/bundle/node.cjs:10399:20)
    at Object.parseItem (/Users/pikachuexe/projects/electron/FreeTube/node_modules/youtubei.js/bundle/node.cjs:10680:13)
    at new ItemSection (/Users/pikachuexe/projects/electron/FreeTube/node_modules/youtubei.js/bundle/node.cjs:6219:34)
    at parseItem (/Users/pikachuexe/projects/electron/FreeTube/node_modules/youtubei.js/bundle/node.cjs:10698:22)
    at Object.parseArray (/Users/pikachuexe/projects/electron/FreeTube/node_modules/youtubei.js/bundle/node.cjs:10718:22)
    at new SectionList (/Users/pikachuexe/projects/electron/FreeTube/node_modules/youtubei.js/bundle/node.cjs:1127:36)
    at Object.parseItem (/Users/pikachuexe/projects/electron/FreeTube/node_modules/youtubei.js/bundle/node.cjs:10698:22)
    at new Tab (/Users/pikachuexe/projects/electron/FreeTube/node_modules/youtubei.js/bundle/node.cjs:8997:35)
    at parseItem (/Users/pikachuexe/projects/electron/FreeTube/node_modules/youtubei.js/bundle/node.cjs:10698:22)
    at Object.parse (/Users/pikachuexe/projects/electron/FreeTube/node_modules/youtubei.js/bundle/node.cjs:10736:22) {
  info: {
    contents: [ [Object], [Object], [Object] ],
    trackingParams: 'CCcQ4M4DIhMIo56-65PbgQMVkldgCh3_lwCu',
    styleType: 'FEED_FILTER_CHIP_BAR_STYLE_TYPE_HASHTAG_LANDING_PAGE'
  },
  date: 2023-10-04T00:39:29.159Z,
  version: '6.4.1'
}

@PikachuEXE Yes the first one is intentional, I told you so on the FreeTube repo and also how to deal with it.
You can ignore the second one though, it's printed with console.warn.

For the 2nd one is a warning I get it
For the 1st one I am simply confused by has_continuation() returning true in which the code checks for (this.#memo.get('ContinuationItem') || []).length > 0
While getContinuationData() checks for this.#continuation.length === 0
Should has_continuation() use this.#continuation.length instead? (i.e. Can library users rely on has_continuation solely or have to catch error from getContinuation?)

YouTube includes the continuation token in their first page response, the issue is not with YouTube.js's checks, it's with YouTube returning a useless continuation token. But we don't know it's useless/broken until we send that continuation token to YouTube and it returns a broken response.

If you open the playlist on the YouTube website and scroll all the way down, you'll notice in the network tools that it requests the continuation as well.


[IDEA] Is it possible to target Firebase Cloud Function?

Motivation
Cloudflare worker/Rust is always my first choice but sometime it's too small or too limit to fit some lib/unsupported features.
So I've an idea to continue writing in Rust and deploy as WASM to Cloudflare (if its fit) or Firebase (if it's not).
POC

[x] Try run wasm_bindgen on Firebase Cloud Function to proof wasm32-unknown-unknown working fine.

TODO

[ ] Explore existing stacks (which is miniflare), find intersection then modify/fork from it if possible. 👈 I'm here.
[ ] Or create my own template, use firebase-tools (nothing to do with miniflare in this case).
[ ] Support ENV from Firebase.
[ ] Support Firestore instead of KV.
[ ] Support JsValue.
[ ] Support fetch.
[ ] Not sure.

That's a lot of me-unknown-unknown here, so the and answer can be.

No, you're go in a wrong way don't do it, create some template from Firebase side, use firebase-tools then share crate would be easier.
Yes, you can fork and modify some cargo, webpack, define cfg target this and that and then magic happen.

Any idea is welcome, Thanks!

Consider it done (partially) without miniflare https://github.com/katopz/hello-firebase-wasm


SCRIPT ERROR: [SERVER] Entity is null (soul.soul.lastOwner = player)

I've seen this happen more than 5 times in the last hour on my dedicated turbo_ttdm server. I'm not sure what the context is since I haven't been in-game for any of these occurrences.

northstar1_1  | te[01:06:29] [info] [SERVER SCRIPT] AttackerShouldTriggerReplay(): Not doing a replay by default..
northstar1_1  | _v[01:06:29] [info] [SERVER SCRIPT] AttackerShouldTriggerReplay(): Not doing a replay by default..
northstar1_1  | al[01:06:29] [info] [SERVER SCRIPT] AttackerShouldTriggerReplay(): Not doing a replay by default..
northstar1_1  | u[01:06:29] [info] [SERVER SCRIPT] AttackerShouldTriggerReplay(): Not doing a replay by default..
northstar1_1  | es?[01:06:29] [info] [SERVER SCRIPT] AttackerShouldTriggerReplay(): Not doing a replay by default..
northstar1_1  | id[01:06:29] [info] [SERVER SCRIPT] AttackerShouldTriggerReplay(): Not doing a replay by default..
northstar1_1  | =8f1[01:06:29] [info] [SERVER SCRIPT] Post impact,anim is done.
northstar1_1  | cb5657[01:06:29] [info] [SERVER SCRIPT] pilot primary skin/camo 1 151.
northstar1_1  | b[01:06:29] [info] [SERVER SCRIPT] pilot secondary skin/camo 1 151.
northstar1_1  | 4[01:06:29] [info] [SERVER SCRIPT] pilot weapon3 skin/camo 1 150.
northstar1_1  | 1d37096d3[01:06:29] [info] [SERVER SCRIPT] SCRIPT ERROR: [SERVER] Entity is null.
northstar1_1  | [01:06:29] [info] [SERVER SCRIPT]  -> soul.soul.lastOwner = player.
northstar1_1  | 5[01:06:29] [info] [SERVER SCRIPT].[K.
northstar1_1  | CALLSTACK.
northstar1_1  | *FUNCTION [PilotBecomesTitan()] mp/_titan_transfer.nut line [524].
northstar1_1  | *FUNCTION [RespawnAsTitan()] mp/_base_gametype_mp.gnut line [481].
northstar1_1  |
northstar1_1  | 3[01:06:29] [info] [SERVER SCRIPT] LOCALS.
northstar1_1  | [soul] NULL.
northstar1_1  | [fullCopy] true.
northstar1_1  | [titan] ENTITY (npc_titan [109] (NPC "npc_titan_auto_atlas_stickybomb" at <8416.26 -6640.95 4055.51>)).

(sorry about the screwed up log formatting)

Getting more and more of these too, I believe the first time was yesterday.
[14:13:09] [info] [SERVER SPEW_MESSAGE] Setting player "player --snip [4]" predicted first person proxy model to "models/weapons/arms/pov_pilot_medium_geist_f.mdl" (class "pilot_grapple_female")
[14:13:09] [info] [SERVER SCRIPT] SCRIPT ERROR: [SERVER] Entity is null
[14:13:09] [info] [SERVER SCRIPT]  -> soul.soul.lastOwner = player
[14:13:09] [info] [SERVER SCRIPT]
CALLSTACK
*FUNCTION [PilotBecomesTitan()] mp/_titan_transfer.nut line [515]
*FUNCTION [RespawnAsTitan()] mp/_base_gametype_mp.gnut line [462]

[14:13:09] [info] [SERVER SCRIPT] LOCALS
[soul] NULL
[fullCopy] true
[titan] ENTITY (npc_titan [242] (NPC "npc_titan_auto_atlas_stickybomb" at <-5066.76 1989.51 56.0325>))
[player] ENTITY (player --snip [4] (player "--snip" at <-5066.76 1989.51 56.0325>))
[this] TABLE
[xyOffset] <44, 0, 520>
[camera] ENTITY (point_viewcontrol [273] at <-5064.92 1989.51 190.361>)
[titan] ENTITY (npc_titan [242] (NPC "npc_titan_auto_atlas_stickybomb" at <-5066.76 1989.51 56.0325>))
[warpAttach] unknown struct
[model] $"models/titans/medium/titan_medium_ajax.mdl"
[titanLoadout] unknown struct
[spawnpoint] ENTITY (info_spawnpoint_titan info_spawnpoint_titan_3 [0] at <-5066.78 1989.51 48.0313>)
[manualPosition] false
[player] ENTITY (player --snip [4] (player "--snip" at <-5066.76 1989.51 56.0325>))
[this] TABLE

DIAGPRINTS

[14:13:09] [info] [SERVER SPEW_WARNING] point_viewcontrol point_viewcontrol [273] called Disable when it wasn't the players active view entity.
[14:13:09] [error] Northstar has crashed! a minidump has been written and exception info is available below:
[14:13:09] [error] Cause: Access Violation
Attempted to read from: 0x00000000
[14:13:09] [error] At: engine.dll + 0x247d3a
[14:13:09] [error]     Northstar.dll + 0x128d9e60
[14:13:09] [error]     ntdll.dll + 0x17002567d
[14:13:09] [error]     ntdll.dll + 0x1700548c5
[14:13:09] [error]     ntdll.dll + 0x17005245e
[14:13:09] [error]     engine.dll + 0x19607d3a
[14:13:09] [error]     engine.dll + 0x197d7466
[14:13:09] [error]     engine.dll + 0x194e0c2f
[14:13:09] [error]     engine.dll + 0x194e0542
[14:13:09] [error]     engine.dll + 0x1952db55
[14:13:09] [error]     engine.dll + 0x195888c0
[14:13:09] [error]     Northstar.dll + 0x128c18bc
[14:13:09] [error]     engine.dll + 0x1958652f
[14:13:09] [error]     engine.dll + 0x19750e2d
[14:13:09] [error]     engine.dll + 0x195879d9
[14:13:09] [error]     engine.dll + 0x1958793c
[14:13:09] [error]     launcher.dll + 0x12af5afd
[14:13:09] [error]     launcher.dll + 0x12af5afd
[14:13:09] [error]     launcher.dll + 0x12aed386
[14:13:09] [error]     NorthstarLauncher.exe + 0x140003bbd
[14:13:09] [error]     NorthstarLauncher.exe + 0x140006d40
[14:13:09] [error]     kernel32.dll + 0x7b62c599
[14:13:09] [error]     ntdll.dll + 0x170059d53
[14:13:09] [error]     NorthstarLauncher.exe + 0x0
[14:13:09] [error]     NorthstarLauncher.exe + 0x140006db0
[14:13:09] [error]     NorthstarLauncher.exe + 0x67ff0000
[14:13:09] [error]     NorthstarLauncher.exe + 0x0
[14:13:09] [error] RAX: 0x0
[14:13:09] [error] RBX: 0x19ba56d0
[14:13:09] [error] RCX: 0x0
[14:13:09] [error] RDX: 0x0
[14:13:09] [error] RSI: 0x2c49bb78
[14:13:09] [error] RDI: 0x2c49bb78
[14:13:09] [error] RBP: 0x2c49bd88
[14:13:09] [error] RSP: 0x7cf4a0
[14:13:09] [error] R8: 0x2c49bb78
[14:13:09] [error] R9: 0x65
[14:13:09] [error] R10: 0x2c49bd94
[14:13:09] [error] R11: 0x199c78ec
[14:13:09] [error] R12: 0x0
[14:13:09] [error] R13: 0x0
[14:13:09] [error] R14: 0xffffffff
[14:13:09] [error] R15: 0x1
0024:fixme:dbghelp_dwarf:dwarf2_read_range no entry found
0024:fixme:dbghelp_dwarf:dwarf2_read_range no entry found
0024:fixme:dbghelp:MiniDumpWriteDump NIY MiniDumpScanMemory
Error: Failed to run server: server exited: exit status 100.

Skirmish only server running ghcr.io/pg9182/northstar-dedicated-dev:dev.20220126.git5714a27.

@pg9182 what exactly causes this issue? I don't know under what circumstances a pilot would be null when entering a titan.


separate startgies as own extra requirement

What does this PR do?
moving Horovod and the additional strategy requirement to own file
updating and cleaning all dockers
update CI workflows
Fixes: failing HPU testing
Does your PR introduce any breaking changes? If yes, please list them.
Horovod is not installed with .[extra] but with .[strategies]
Before submitting

[ ] Was this discussed/approved via a GitHub issue? (not for typos and docs)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
[ ] Did you make sure to update the documentation with your changes? (if necessary)
[x] Did you write any new necessary tests? (not for typos and docs)
[ ] Did you verify new and existing tests pass locally with your changes?
[x] Did you list all the breaking changes introduced by this pull request?
[ ] Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review
Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

[x] Is this pull request ready for review? (if not, please submit in draft mode)
[x] Check that all items from Before submitting are resolved
[x] Make sure the title is self-explanatory and the description concisely explains the PR
[x] Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?
Make sure you had fun coding 🙃
cc @carmocca @akihironitta @borda @tchaton @rohitgr7

have an n00b question, would be possible to unify the package name and Cuda-version have in a version like PT, for example, torch==1.11.0+cu113  instead of bagua-cuda113==0.9.0? this makes it impossible to include in PL requirements directly...
cc: @wangraying @kaushikb11

I like the idea of not putting everything inside extra.txt!
Can be done in a separate PR, but I'm also considering if we should split the requirements into smaller pieces so that people can install dependencies of their interest and nothing more. Do you think it's reasonable to have the following installation options for example?
pip install pytorch-lightning[fsdp]
pip install pytorch-lightning[bagua]
pip install pytorch-lightning[horovod]
...

pip install pytorch-lightning[strategies]  # to install all of the above

Can be done in a separate PR, but I'm also considering if we should split the requirements into smaller pieces so that people can install dependencies of their interest and nothing more. Do you think it's reasonable to have the following installation options for example?

that would be considering as next step :)

/usr/local/lib/python3.7/dist-packages/_pytest/runner.py:340: in from_call
    result: Optional[TResult] = func()
/usr/local/lib/python3.7/dist-packages/_pytest/runner.py:371: in <lambda>
    call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
/usr/local/lib/python3.7/dist-packages/_pytest/doctest.py:545: in collect
    module = import_path(self.path, root=self.config.rootpath)
/usr/local/lib/python3.7/dist-packages/_pytest/pathlib.py:553: in import_path
    raise ImportPathMismatchError(module_name, module_file, path)

not sure about the source :/

There was an error with GPU CI installing deepspeed:
Collecting deepspeed>=0.6.0
  Downloading deepspeed-0.6.2.tar.gz (542 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 542.4/542.4 KB 21.5 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      Traceback (most recent call last):
        File "<string>", line 36, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-3rgf_22_/deepspeed_cc308bd058f44eaa93222e60c059a30f/setup.py", line 35, in <module>
          from op_builder import ALL_OPS, get_default_compute_capabilities, OpBuilder
      ImportError: cannot import name 'OpBuilder' from 'op_builder' (/usr/local/lib/python3.7/dist-packages/op_builder/__init__.py)
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

deepspeed==0.6.2 has some trouble with its installation: https://github.com/microsoft/DeepSpeed/issues/1918

@williamFalcon Could you have a look as codeowner?

@Borda To merge this PR, we need @williamFalcon's approval or admin power, right?

@Borda To merge this PR, we need @williamFalcon's approval or admin power, right?

or @lexierule


Add a Gitter chat badge to README.md

PEZ/SketchDistributor now has a Chat Room on Gitter
@PEZ has just created a chat room. You can visit it here: https://gitter.im/PEZ/SketchDistributor.
This pull-request adds this badge to your README.md:

If my aim is a little off, please let me know.
Happy chatting.
PS: Click here if you would prefer not to receive automatic pull-requests from Gitter in future.

I merged it even though the badge was a bit strangely placed; in the beginning of the file and just in front of the first paragraph. I think it would have been better if you placed it last.
Also it feels really odd to see that I created this pull request myself! You should use your own user, fork and issue the pull request. IMO.


how to install netopeer2 in cent os 7

Hi,
generally, just follow the instructions. If you encountered any error, please provide details.
Regards,
Michal

I am very new to Netopeer. Can you share any docker file or can I get step
by step instructions to install?
On Mon 25 Feb, 2019, 7:06 PM Michal Vasko<EMAIL_ADDRESS>wrote:

Hi,
generally, just follow the instructions. If you encountered any error,
please provide details.
Regards,
Michal
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/CESNET/Netopeer2/issues/399#issuecomment-467013516,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AtbOkG9QsUQU1phQZvVYioU0Kl-tJGqsks5vQ-b5gaJpZM4bPo4e
.

Is there any tutorial to understand about the working of netopeer??
On Mon 25 Feb, 2019, 7:08 PM arun ashok<EMAIL_ADDRESS>wrote:

I am very new to Netopeer. Can you share any docker file or can I get step
by step instructions to install?
On Mon 25 Feb, 2019, 7:06 PM Michal Vasko<EMAIL_ADDRESS>wrote:

Hi,
generally, just follow the instructions. If you encountered any error,
please provide details.
Regards,
Michal
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/CESNET/Netopeer2/issues/399#issuecomment-467013516,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AtbOkG9QsUQU1phQZvVYioU0Kl-tJGqsks5vQ-b5gaJpZM4bPo4e
.

Hi,
before looking at the links below, you should have at least basic understanding of YANG and NETCONF, we have no tutorial on these topics alone.
Then, main documentation is in the sysrepo project as it is used as the back-end (datastore), netopeer2 just provides the NETCONF layer. Some basic sysrepo tutorial is in the full documentation. You can also find docker files there with all the software running including netopeer2.
Regards,
Michal

Ok Thank You.
I will go through those.
Regards,
Arun Ashok
On Mon 25 Feb, 2019, 7:21 PM Michal Vasko<EMAIL_ADDRESS>wrote:

Hi,
before looking at the links below, you should have at least basic
understanding of YANG and NETCONF, we have no tutorial on these topics
alone.
Then, main documentation is in the sysrepo
https://github.com/sysrepo/sysrepo project as it is used as the
back-end (datastore), netopeer2 just provides the NETCONF layer. Some
basic sysrepo tutorial is in the full documentation
http://www.sysrepo.org/static/doc/html/start_page.html. You can also
find docker files
https://github.com/sysrepo/sysrepo/tree/master/deploy/docker there with
all the software running including netopeer2.
Regards,
Michal
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/CESNET/Netopeer2/issues/399#issuecomment-467018172,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AtbOkEjmLIxGrS86bY0bwOQFCd3qQbA1ks5vQ-pigaJpZM4bPo4e
.


Get Started with the SAP BTP Command Line Interface (btp CLI)

Tutorials: https://developers.sap.com/tutorials/cp-sapcp-getstarted.html
About step: "Step 11: Enable command autocompletion", there is a typo about this command line.
From BTP Help (CLI: btp --help), the correct one would be:
btp enable autocomplete SHELL

done. sorry for the delay, my process of forwarding issues to a substitute during my long absence did not work.
Sincere apologies for that!


Darktable: 4.2.0 -> 4.2.1

Description of changes
Darktable 4.2.0 -> 4.2.1, described as a bug fix release
See https://github.com/darktable-org/darktable/releases/tag/release-4.2.1
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 229142 run on x86_64-linux 1

2 packages built:

darktable
photoprism

@ofborg build darktable


ABI Upload feature for UI

What would you like to be added?
A new feature in the UI that allows users to upload ABIs.
Why is this needed?
The uploaded ABIs could then be used to enrich information on screen.

ABI Upload and lookup
The new feature would make use of the ptx_storeABI RPC method in order to submit an ABI to the Paladin node.
The UI could also feature a lookup option where, provided a hash, the corresponding ABI would be shown (if present) using the ptx_getStoredABI RPC method.
Enriching information
The following is an example of the details of a transaction shown in the UI:

Here is where the abiReference could be used to lookup the corresponding ABI.
Key thing to define: how exactly is the information going to be enriched on screen?

Changes in https://github.com/LF-Decentralized-Trust-labs/paladin/pull/471


NavigationListItem is not showing the anchor element properly

OpenUI5 version: latest
Browser/version (+device/version): Chrome (latest)
Any other tested browsers/devices(OK/FAIL): Same behaviour in Firefox and Edge.
URL (minimal example if possible): https://ui5.sap.com/#/entity/sap.tnt.SideNavigation/sample/sap.tnt.sample.SideNavigation/code
Steps to reproduce the problem:

Using NavigationListItem, add a href attribute to it, for example "https://ui5.sap.com/".
Hover over the navigation item or try opening a context menu on the item.

What is the expected result?
Normally, I'd expect seeing the URL in the browser and also open it in the new tab for example.
What happens instead?
The URL is not shown and it's not possible to open it in the new tab.

I am not sure whether this is the expected design but I was able to reproduce it and created an internal incident<PHONE_NUMBER>. Colleagues will update... Thank you!

Thanks! Specifically, this issue is happening when I use the navigation items inside the sap.tnt.SideNavigation control.

Hi @whydrae,
This issue is fixed in the main, 1.120 and 1.108 branches.
Best,
Teo


Allow custom templates.

Documentation for config_content doesn't match description.
Adding additional variable and keeping existing to not break already existing setups.

This change is

@EmilienM could you review?

Cool, thanks!

@EmilienM do you plan release?

@KostyaSha I don't work on puppetlabs releases, sorry.

How release process works? I though some release merges...

@bmjen @hunner maybe you can help with release? There are a lot of merges after latest release.


OIDC support for signatures other than RS256

Checklist

[X] I agree to the terms within the OpenFGA Code of Conduct.

Describe the problem you'd like to have solved
Could OpenFGA support signing algorithms for OIDC authentication other than RS256? ES256 would better suit our needs.
Describe the ideal solution
OpenFGA OIDC authn would support ES256 signatures.
Alternatives and current workarounds
No response
Additional context
I'm working with a system that currently issues ES256 tokens and was trying to get OpenFGA authentication running against that. It would be a pain to update this system to issue tokens with RS256.

@miparnisari @rhamzeh what would we need to change to support this? Can we flag it as 'hacktoberfest'?


Laravel Linter

This pull request includes changes and recommendations for crafting your application "The Laravel Way". Feel free to commit any additional changes to the shift-56612 branch.
Before merging, you need to:

Checkout the shift-56612 branch
Review all pull request comments for additional changes
Thoroughly test your application

If you need help with your upgrade, check out the Human Shifts. You may also join the Shifty Coders Slack workspace to level-up your Laravel skills.

:warning: The following files reference Laravel facades through the global namespace. For example, you're referencing \DB or importing use DB. Instead import Illuminate\Support\Facades\DB and reference DB.
While global references are allowed through aliases, you should import the facade explicitly. This can improve clarity not only for developers, but static analysis used by your IDE.

[ ] app/Providers/AppServiceProvider.php
[ ] tests/Feature/AttendanceTest.php
[ ] tests/Feature/CoreTest.php
[ ] tests/Feature/CoursesTest.php
[ ] tests/Feature/EnrollmentViewAuthTest.php
[ ] tests/Feature/Http/Controllers/Admin/BookCrudControllerTest.php
[ ] tests/Feature/Http/Controllers/Admin/CourseCrudControllerTest.php
[ ] tests/Feature/Http/Controllers/Admin/EvaluationTypeCrudControllerTest.php
[ ] tests/Feature/Http/Controllers/AttendanceControllerTest.php
[ ] tests/Feature/Http/Controllers/GradeControllerTest.php
[ ] tests/TestCase.php

:warning: The following classes do not extend the standard Laravel Model and Controller class. This may add complexity which makes your application harder to upgrade. Often a trait can be used instead of inheritance. You should review the following classes:

[ ] app/Http/Controllers/Admin/BookCrudController.php
[ ] app/Http/Controllers/Admin/CampusCrudController.php
[ ] app/Http/Controllers/Admin/CommentCrudController.php
[ ] app/Http/Controllers/Admin/ConfigCrudController.php
[ ] app/Http/Controllers/Admin/CouponCrudController.php
[ ] app/Http/Controllers/Admin/CourseCrudController.php
[ ] app/Http/Controllers/Admin/DiscountCrudController.php
[ ] app/Http/Controllers/Admin/EnrollmentCrudController.php
[ ] app/Http/Controllers/Admin/EvaluationTypeCrudController.php
[ ] app/Http/Controllers/Admin/EventCrudController.php
[ ] app/Http/Controllers/Admin/ExternalCourseCrudController.php
[ ] app/Http/Controllers/Admin/FeeCrudController.php
[ ] app/Http/Controllers/Admin/GradeTypeCategoryCrudController.php
[ ] app/Http/Controllers/Admin/GradeTypeCrudController.php
[ ] app/Http/Controllers/Admin/InstitutionCrudController.php
[ ] app/Http/Controllers/Admin/InvoiceCrudController.php
[ ] app/Http/Controllers/Admin/LeadTypeCrudController.php
[ ] app/Http/Controllers/Admin/LeaveCrudController.php
[ ] app/Http/Controllers/Admin/LevelCrudController.php
[ ] app/Http/Controllers/Admin/MemberCrudController.php
[ ] app/Http/Controllers/Admin/PartnerCrudController.php
[ ] app/Http/Controllers/Admin/PaymentCrudController.php
[ ] app/Http/Controllers/Admin/PaymentmethodCrudController.php
[ ] app/Http/Controllers/Admin/PeriodCrudController.php
[ ] app/Http/Controllers/Admin/ProfessionCrudController.php
[ ] app/Http/Controllers/Admin/RemoteEventCrudController.php
[ ] app/Http/Controllers/Admin/ResultCrudController.php
[ ] app/Http/Controllers/Admin/ResultTypeCrudController.php
[ ] app/Http/Controllers/Admin/RhythmCrudController.php
[ ] app/Http/Controllers/Admin/RoomCrudController.php
[ ] app/Http/Controllers/Admin/SchedulePresetCrudController.php
[ ] app/Http/Controllers/Admin/ScheduledPaymentCrudController.php
[ ] app/Http/Controllers/Admin/ScholarshipCrudController.php
[ ] app/Http/Controllers/Admin/SkillCrudController.php
[ ] app/Http/Controllers/Admin/SkillScaleCrudController.php
[ ] app/Http/Controllers/Admin/SkillTypeCrudController.php
[ ] app/Http/Controllers/Admin/StudentCrudController.php
[ ] app/Http/Controllers/Admin/TaxCrudController.php
[ ] app/Http/Controllers/Admin/TeacherCrudController.php
[ ] app/Http/Controllers/Admin/UserCrudController.php
[ ] app/Http/Controllers/Admin/YearCrudController.php
[ ] app/Http/Controllers/Auth/RegisterController.php

:warning: The following controllers contain actions outside of the 7 resource actions (index, create, store, show, edit, update, destroy). For more details, review the docs or watch Cruddy by Design to see if you can rework these into resource controllers.

[ ] app/Http/Controllers/Admin/BookCrudController.php
[ ] app/Http/Controllers/Admin/CampusCrudController.php
[ ] app/Http/Controllers/Admin/CommentCrudController.php
[ ] app/Http/Controllers/Admin/ConfigCrudController.php
[ ] app/Http/Controllers/Admin/CouponCrudController.php
[ ] app/Http/Controllers/Admin/CourseCrudController.php
[ ] app/Http/Controllers/Admin/DiscountCrudController.php
[ ] app/Http/Controllers/Admin/EnrollmentCrudController.php
[ ] app/Http/Controllers/Admin/EvaluationTypeCrudController.php
[ ] app/Http/Controllers/Admin/EventCrudController.php
[ ] app/Http/Controllers/Admin/ExternalCourseCrudController.php
[ ] app/Http/Controllers/Admin/FeeCrudController.php
[ ] app/Http/Controllers/Admin/GradeTypeCategoryCrudController.php
[ ] app/Http/Controllers/Admin/GradeTypeCrudController.php
[ ] app/Http/Controllers/Admin/InstitutionCrudController.php
[ ] app/Http/Controllers/Admin/InvoiceCrudController.php
[ ] app/Http/Controllers/Admin/LeadTypeCrudController.php
[ ] app/Http/Controllers/Admin/LeaveCrudController.php
[ ] app/Http/Controllers/Admin/LevelCrudController.php
[ ] app/Http/Controllers/Admin/MemberCrudController.php
[ ] app/Http/Controllers/Admin/Operations/ShowStudentListOperation.php
[ ] app/Http/Controllers/Admin/Operations/ShowStudentPhotoRosterOperation.php
[ ] app/Http/Controllers/Admin/PartnerCrudController.php
[ ] app/Http/Controllers/Admin/PaymentCrudController.php
[ ] app/Http/Controllers/Admin/PaymentmethodCrudController.php
[ ] app/Http/Controllers/Admin/PeriodCrudController.php
[ ] app/Http/Controllers/Admin/ProfessionCrudController.php
[ ] app/Http/Controllers/Admin/RemoteEventCrudController.php
[ ] app/Http/Controllers/Admin/ResultCrudController.php
[ ] app/Http/Controllers/Admin/ResultTypeCrudController.php
[ ] app/Http/Controllers/Admin/RhythmCrudController.php
[ ] app/Http/Controllers/Admin/RoomCrudController.php
[ ] app/Http/Controllers/Admin/SchedulePresetCrudController.php
[ ] app/Http/Controllers/Admin/ScheduledPaymentCrudController.php
[ ] app/Http/Controllers/Admin/ScholarshipCrudController.php
[ ] app/Http/Controllers/Admin/SkillCrudController.php
[ ] app/Http/Controllers/Admin/SkillScaleCrudController.php
[ ] app/Http/Controllers/Admin/SkillTypeCrudController.php
[ ] app/Http/Controllers/Admin/StudentCrudController.php
[ ] app/Http/Controllers/Admin/TaxCrudController.php
[ ] app/Http/Controllers/Admin/TeacherCrudController.php
[ ] app/Http/Controllers/Admin/UserCrudController.php
[ ] app/Http/Controllers/Admin/YearCrudController.php
[ ] app/Http/Controllers/AttendanceController.php
[ ] app/Http/Controllers/Auth/MyAccountController.php
[ ] app/Http/Controllers/BookController.php
[ ] app/Http/Controllers/ConfigController.php
[ ] app/Http/Controllers/ContactPhoneNumberController.php
[ ] app/Http/Controllers/CourseController.php
[ ] app/Http/Controllers/CourseSkillController.php
[ ] app/Http/Controllers/EnrollmentController.php
[ ] app/Http/Controllers/EventController.php
[ ] app/Http/Controllers/GradeController.php
[ ] app/Http/Controllers/HRController.php
[ ] app/Http/Controllers/HomeController.php
[ ] app/Http/Controllers/InvoiceController.php
[ ] app/Http/Controllers/Language.php
[ ] app/Http/Controllers/LevelController.php
[ ] app/Http/Controllers/ReportController.php
[ ] app/Http/Controllers/ResultController.php
[ ] app/Http/Controllers/RhythmController.php
[ ] app/Http/Controllers/ScheduledPaymentController.php
[ ] app/Http/Controllers/StudentController.php
[ ] app/Http/Controllers/StudentPhoneNumberController.php
[ ] app/Http/Controllers/TeacherController.php
[ ] app/Http/Controllers/TeacherLeaveController.php

:warning: Shift detected Eloquent calls in the following views. You should keep views from interacting directly with your models by passing the necessary data from the controller.

[ ] resources/views/courses/child-course-alert.blade.php
[ ] resources/views/vendor/backpack/base/auth/register.blade.php
[ ] resources/views/vendor/backpack/crud/buttons/enroll-student-in-course.blade.php

:warning: Shift found opportunities to use the built-in Blade directives like @csrf, @method in the following views:

[ ] resources/views/student/account/change_password.blade.php
[ ] resources/views/student/account/update_info.blade.php
[ ] resources/views/student/account/update_phone.blade.php
[ ] resources/views/student/account/update_profession.blade.php
[ ] resources/views/student/account/update_student_info.blade.php
[ ] resources/views/students/edit-contact.blade.php
[ ] resources/views/vendor/backpack/base/auth/demo-login.blade.php

:warning: Shift detected the following HTTP components using the Request or Input facade to access the request. Within these components you should leverage the request object which is automatically injected to any controller action or Middleware.

[ ] app/Http/Controllers/Admin/BookCrudController.php
[ ] app/Http/Controllers/Admin/CampusCrudController.php
[ ] app/Http/Controllers/Admin/CommentCrudController.php
[ ] app/Http/Controllers/Admin/ConfigCrudController.php
[ ] app/Http/Controllers/Admin/CouponCrudController.php
[ ] app/Http/Controllers/Admin/CourseCrudController.php
[ ] app/Http/Controllers/Admin/DiscountCrudController.php
[ ] app/Http/Controllers/Admin/EvaluationTypeCrudController.php
[ ] app/Http/Controllers/Admin/EventCrudController.php
[ ] app/Http/Controllers/Admin/ExternalCourseCrudController.php
[ ] app/Http/Controllers/Admin/FeeCrudController.php
[ ] app/Http/Controllers/Admin/GradeTypeCategoryCrudController.php
[ ] app/Http/Controllers/Admin/GradeTypeCrudController.php
[ ] app/Http/Controllers/Admin/InstitutionCrudController.php
[ ] app/Http/Controllers/Admin/InvoiceCrudController.php
[ ] app/Http/Controllers/Admin/LeadTypeCrudController.php
[ ] app/Http/Controllers/Admin/LeaveCrudController.php
[ ] app/Http/Controllers/Admin/LevelCrudController.php
[ ] app/Http/Controllers/Admin/MemberCrudController.php
[ ] app/Http/Controllers/Admin/PartnerCrudController.php
[ ] app/Http/Controllers/Admin/PaymentmethodCrudController.php
[ ] app/Http/Controllers/Admin/ProfessionCrudController.php
[ ] app/Http/Controllers/Admin/RemoteEventCrudController.php
[ ] app/Http/Controllers/Admin/ResultTypeCrudController.php
[ ] app/Http/Controllers/Admin/RhythmCrudController.php
[ ] app/Http/Controllers/Admin/RoomCrudController.php
[ ] app/Http/Controllers/Admin/SchedulePresetCrudController.php
[ ] app/Http/Controllers/Admin/ScheduledPaymentCrudController.php
[ ] app/Http/Controllers/Admin/ScholarshipCrudController.php
[ ] app/Http/Controllers/Admin/SkillCrudController.php
[ ] app/Http/Controllers/Admin/SkillScaleCrudController.php
[ ] app/Http/Controllers/Admin/SkillTypeCrudController.php
[ ] app/Http/Controllers/Admin/StudentCrudController.php
[ ] app/Http/Controllers/Admin/TaxCrudController.php
[ ] app/Http/Controllers/Admin/TeacherCrudController.php
[ ] app/Http/Controllers/Admin/UserCrudController.php
[ ] app/Http/Controllers/Admin/YearCrudController.php
[ ] app/Http/Middleware/ForceUpdate.php

:warning: Previous versions of Laravel converted the second argument passed to the query builder orderBy method to desc when not set to asc. Starting with Laravel 5.8, an InvalidArgumentException is throw.
If you were attempting order by multiple columns, you should instead chain multiple calls to orderBy.
Shift detected the following uses of the orderBy() passing a second argument:

[ ] app/Http/Controllers/Admin/EnrollmentCrudController.php
[ ] app/Http/Controllers/Admin/StudentCrudController.php
[ ] app/Http/Controllers/Admin/TeacherCrudController.php

:information_source: Shift detected controller namespaces being set in your RouteServiceProvider. Laravel 8 began registering controllers using static class references instead of namespace prefixes and controller names.
You may automate this conversion using the Laravel Fixer or tasks within the Shift Workbench.

:warning: Shift found the following config files differ from the defaults. While you are welcome to customize your configuration, you should leverage ENV variables rather than hardcode values. If you find you're adding a lot of configuration options, consider creating a domain specific config file, such as core.php. Both will make app upgrades and deployments easier.

[ ] config/app.php
[ ] config/logging.php
[ ] config/services.php

:warning: Laravel 8 began PSR-4 autoloading for factories and seeders. Shift detected the following files are missing a namespace:

[ ] database/factories/AttendanceFactory.php
[ ] database/factories/AttendanceTypeFactory.php
[ ] database/factories/BookFactory.php
[ ] database/factories/CampusFactory.php
[ ] database/factories/CommentFactory.php
[ ] database/factories/ConfigFactory.php
[ ] database/factories/ContactFactory.php
[ ] database/factories/ContactRelationshipFactory.php
[ ] database/factories/CouponFactory.php
[ ] database/factories/CourseFactory.php
[ ] database/factories/CourseTimeFactory.php
[ ] database/factories/DiscountFactory.php
[ ] database/factories/EnrollmentFactory.php
[ ] database/factories/EnrollmentStatusTypeFactory.php
[ ] database/factories/EvaluationTypeFactory.php
[ ] database/factories/EventFactory.php
[ ] database/factories/ExternalCourseFactory.php
[ ] database/factories/FeeFactory.php
[ ] database/factories/GradeFactory.php
[ ] database/factories/GradeTypeCategoryFactory.php
[ ] database/factories/GradeTypeFactory.php
[ ] database/factories/InstitutionFactory.php
[ ] database/factories/InvoiceDetailFactory.php
[ ] database/factories/InvoiceFactory.php
[ ] database/factories/LeadTypeFactory.php
[ ] database/factories/LeaveFactory.php
[ ] database/factories/LeaveTypeFactory.php
[ ] database/factories/LevelFactory.php
[ ] database/factories/PaymentFactory.php
[ ] database/factories/PaymentmethodFactory.php
[ ] database/factories/PeriodFactory.php
[ ] database/factories/PhoneNumberFactory.php
[ ] database/factories/ProfessionFactory.php
[ ] database/factories/RemoteEventFactory.php
[ ] database/factories/ResultFactory.php
[ ] database/factories/ResultTypeFactory.php
[ ] database/factories/RhythmFactory.php
[ ] database/factories/RoomFactory.php
[ ] database/factories/SkillEvaluationFactory.php
[ ] database/factories/SkillFactory.php
[ ] database/factories/SkillScaleFactory.php
[ ] database/factories/SkillTypeFactory.php
[ ] database/factories/StudentFactory.php
[ ] database/factories/TeacherFactory.php
[ ] database/factories/UserFactory.php
[ ] database/factories/YearFactory.php


fix: handle document author values for CBOS cases

Describe your changes
APPLICS 722:  Documents page not showing the From details on C-BOS cases

Added author and author_welsh keys to the object passed into the view.
Updated the conditional logic in the template to include these variables.

Type of change 🧩

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation update
[ ] Other (please explain in the description section above)

Checklist before requesting a review

[x] I have performed a self-review of my own code
[x] I have double checked this work does not include any hardcoded secrets or passwords
[ ] I have made corresponding changes to the documentation
[ ] I have provided details on how I have tested my code
[x] I have referenced the ticket number above
[x] I have provided a description of the ticket
[ ] I have included unit tests to cover any testable code changes

:tada: This PR is included in version 2.28.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 2.34.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.14.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.2.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


ACK subscription hitting consumer too slow in pub-req

Hello,
This is more of an FYI and follow up question, but I expect others will run into this as well.
The docs show an example like..
// Requests
var response string
err := nc.Request("help", "help me", &response, 10*time.Millisecond)

// Replying
c.Subscribe("help", func(subj, reply string, msg string) {
    c.Publish(reply, "I can help!")
})

the request and reply use the same connection, which generally works fine. But in my stress tests the consumer wasn't firing fast enough, and the publish couldn't send anymore messages since "Slow Consumer Detected".
The solution was to have two separate connections to the nats server, one for the producers and one for the consumers. This solved my problem, but it makes me think if the nats client should be connection pooling?

You should not ever receive a slow consumer with request patterns since they wait for the response. Do you have a working example that you could send in a gist so that I can take a look?

Hey @derekcollison - yea.. here it is: https://gist.github.com/pkieltyka/4de2d53d707e13014e4f
let me know if you have any suggestions. I am sending the PubReq asynchronously..
in this example I end up getting 2015/08/18 21:32:51 Err! write tcp [::1]:64236->[::1]:4222: write: broken pipe  other times I was getting the slow consumer error

Yes if you are simply flooding publishes as fast as you can them you may
get a slow consumer. Best to possibly model by using a channel and a group
of workers who do straight nc.Request() to throttle properly.
On Tue, Aug 18, 2015 at 6:34 PM, Peter Kieltyka<EMAIL_ADDRESS>wrote:

Hey @derekcollison https://github.com/derekcollison - yea.. here it is:
https://gist.github.com/pkieltyka/4de2d53d707e13014e4f
let me know if you have any suggestions. I am sending the PubReq
asynchronously..
in this example I end up getting 2015/08/18 21:32:51 Err! write tcp
[::1]:64236->[::1]:4222: write: broken pipe other times I was getting the
slow consumer error
—
Reply to this email directly or view it on GitHub
https://github.com/nats-io/nats/issues/61#issuecomment-132411280.

I see.. right now it's not a problem for us, even at our scale. I've just made a separate connection for now. It would be nice to buffer and somehow prevent the client from exploding though.
I'm just getting into nats, so I'm still learning the best way to crank out performance and build systems with it. So far I love it!

In a fire and forget system like NATS, unless you do ACKs / Requests, you
really don't know if you are over running the subscribers. You could
periodically do a Flush() call after N async requests.
On Tue, Aug 18, 2015 at 7:04 PM, Peter Kieltyka<EMAIL_ADDRESS>wrote:

I see.. right now it's not a problem for us, even at our scale. I've just
made a separate connection for now. It would be nice to buffer and somehow
prevent the client from exploding though.
I'm just getting into nats, so I'm still learning the best way to crank
out performance and build systems with it. So far I love it!
—
Reply to this email directly or view it on GitHub
https://github.com/nats-io/nats/issues/61#issuecomment-132415991.

Hmm, how can we make this better though? I don't plan to use blocking Requests via Request(), and isn't a PublishRequest() is the same as a fire+forget packet but it includes a reply inbox payload for the consumer to pick up on and send something back? but for NATS to be what it has the potential to be, then IMO the producers / consumers need to be resilient to load out-of-the-box.
I've been writing a library that wraps the nats go client to provide my services semi-guaranteed delivery in a pub-req+queue setup where each queue group is the name of the service to act on an event. The consumers on those services will need to reply to the inbox, which looks like fmt.Sprintf("%s.%s", ackInboxForAll, requestID) and the client has a simple ack inbox which parses out the requestID from the reply topic by listening on #{ackInboxForAll}.>. Etc.. I also want to add a middleware chain like with http handlers so I can let other apps inject across the request/response chain.

Peter,
If your program needs to know that something was received and processed,
then it should wait for the ACK. Also, request allows timeouts so that you
do not wait forever. Moreover, in Go, Goroutines are cheap and you can have
lots of them, so the blocking nature should not be of too much concern. In
your model, how do you know when one failed? What do you do when it does
fail?
In a real life scenario, you would scale the receivers up to a level to
support the publish load, usually as a queue group that can easily be
extended. I still would do a Request in a language like Go.
Also, as a last piece of information, messaging systems were publishing and
consuming are disconnected, you are always challenged with understanding
the deltas between the publishing rate and the consuming rate. I have
designed systems that tried to normalize that system wide, such that they
would slow publishers down if the consumer(s) could not keep up. NATS
specifically did not do this, but these flow patterns can be easily built
on top.  Obviously you don't know this if the publisher and
receiver of the answer are not in the same process, but hopefully this
gives some insights into how to balance the deltas.
On Wed, Aug 19, 2015 at 6:06 AM, Peter Kieltyka<EMAIL_ADDRESS>wrote:

Hmm, how can we make this better though? I don't plan to use blocking
Requests via Request(), and isn't a PublishRequest() is the same as a
fire+forget packet but it includes a reply inbox payload for the consumer
to pick up on and send something back? but for NATS to be what it has the
potential to be, then IMO the producers / consumers need to be resilient to
load out-of-the-box.
I've been writing a library that wraps the nats go client to provide my
services semi-guaranteed delivery in a pub-req+queue setup where each queue
group is the name of the service to act on an event. The consumers on those
services will need to reply to the inbox, which looks like fmt.Sprintf("%s.%s",
ackInboxForAll, requestID) and the client has a simple ack inbox which
parses out the requestID from the reply topic by listening on
#{ackInboxForAll}.>. Etc.. I also want to add a middleware chain like
with http handlers so I can let other apps inject across the
request/response chain.
—
Reply to this email directly or view it on GitHub
https://github.com/nats-io/nats/issues/61#issuecomment-132584383.

Hi Derek,
Thanks for the insights, it definitely helps and makes me realize some flaws in my design.
My goal for a class of events is to ensure a message was consumed/processed successfully, and if it hadn't, then the publishers will persist that message and try again later. The flow is sending a request to a queue group and waiting for an ack.
I've written a very basic version of that for my stack and it works. But there are things I've missed like flow rate throttling and message persistence for failed attempts.
Coordinating the distributed NATS servers will be tricky.. ie. where to persist failed messages for future retry? I would need a central or distributed data store for those messages to be retried, or just let the sender have a local store (but less fault-tolerant). Btw, I'm not suggesting NATS come with a distributed data store, but would be nice if it helped with the discovery/addressing part, in effect NATS could be the thing used to build a distributed data store (/end-tangent).
An example where guaranteed delivery helps is say a "user A follows user B". The event is fired by a frontend application (external stimuli), and it fires an event into the neuron network, different groups of the network will subscribe to the event "user.follows" and act on it. The goal is for each group to process the message, and if any group fails to do so, the message will be published to the failed groups again and again.
I'm thinking service design in the form of CSP over a network.. but is that over thinking it? .. I do think it would change how we design systems if it worked reliably and fast.

This is great feedback, and we have been thinking about how to enhance the
NATS ecosystem with this functionality. Stay tuned. NATS was designed to be
a fire and forget (nervous system like) messaging platform for always on
components. Asynch messaging and persistence for failures or disconnected
receivers is not part of core, but we are evaluating this as we extend
beyond the original design.
On Thu, Aug 20, 2015 at 8:15 AM, Peter Kieltyka<EMAIL_ADDRESS>wrote:

Hi Derek,
Thanks for the insights, it definitely helps and makes me realize some
flaws in my design.
My goal for a class of events is to ensure a message was
consumed/processed successfully, and if it hadn't, then the publishers will
persist that message and try again later. The flow is sending a request to
a queue group and waiting for an ack.
I've written a very basic version of that for my stack and it works. But
there are things I've missed like flow rate throttling and message
persistence for failed attempts.
Coordinating the distributed NATS servers will be tricky.. ie. where to
persist failed messages for future retry? I would need a central or
distributed data store for those messages to be retried, or just let the
sender have a local store (but less fault-tolerant). Btw, I'm not
suggesting NATS come with a distributed data store, but would be nice if it
helped with the discovery/addressing part, in effect NATS could be the
thing used to build a distributed data store (/end-tangent).
An example where guaranteed delivery helps is say a "user A follows user
B". The event is fired by a frontend application (external stimuli), and it
fires an event into the neuron network, different groups of the network
will subscribe to the event "user.follows" and act on it. The goal is for
each group to process the message, and if any group fails to do so, the
message will be published to the failed groups again and again.
I'm thinking service design in the form of CSP over a network.. but is
that over thinking it? .. I do think it would change how we design systems
if it worked reliably and fast.
—
Reply to this email directly or view it on GitHub
https://github.com/nats-io/nats/issues/61#issuecomment-133045284.

awesome, I'm looking forward to hearing more and trying it out eventually!


unresolved external symbol TfLiteGpuDelegateV2Create linker error with Visual Studio

Issue type
Bug
Have you reproduced the bug with TensorFlow Nightly?
Yes
Source
source
TensorFlow version
TfLite nightly, 2.16, 2.10
Custom code
No
OS platform and distribution
Windows 10 Pro & Home
Mobile device
No response
Python version
Irrelevant, C++ API is used
Bazel version
No response
GCC/compiler version
Microsoft Visual Studio 2022 C++ compiler
CUDA/cuDNN version
No response
GPU model and memory
Geforce RTX 2080 (8 GB), Geforce GT 1030 (2 GB GDDR5), Intel HD Graphics 520
Current behavior?
The function TfLiteGpuDelegateV2Create in the minimal-working example below yields the linker error unresolved external symbol __imp_TfLiteGpuDelegateV2Create with Visual Studio.
Why?
Short answer:
Function TfLiteGpuDelegateV2Create is prefixed with preprocessor macro TFL_CAPI_EXPORT as TFL_CAPI_EXPORT TfLiteDelegate* TfLiteGpuDelegateV2Create(const TfLiteGpuDelegateOptionsV2* options); (see tensorflow/lite/delegates/gpu/delegate.h) When building TfLite, TFL_CAPI_EXPORT is empty. When using delegate.h in my application, TFL_CAPI_EXPORT contains __declspec(dllimport). This __declspec(dllimport) causes the linker error.
Long answer:
Function TfLiteGpuDelegateV2Create is prefixed with preprocessor macro TFL_CAPI_EXPORT. TFL_CAPI_EXPORT is defined in tensorflow\lite\core\c\c_api_types.h as
// Define TFL_CAPI_EXPORT macro to export a function properly with a shared
// library.
#ifdef SWIG
    #define TFL_CAPI_EXPORT
#elif defined(TFL_STATIC_LIBRARY_BUILD)
    #define TFL_CAPI_EXPORT
#else  // not definded TFL_STATIC_LIBRARY_BUILD
    #if defined(_WIN32)
        #ifdef TFL_COMPILE_LIBRARY
            #define TFL_CAPI_EXPORT __declspec(dllexport)
        #else
            #define TFL_CAPI_EXPORT __declspec(dllimport)
        #endif  // TFL_COMPILE_LIBRARY
    #else
        #define TFL_CAPI_EXPORT __attribute__((visibility("default")))
    #endif  // _WIN32
#endif  // SWIG

TFL_CAPI_EXPORT is empty when building TfLite with the defaults via CMake. TFL_CAPI_EXPORT is empty because TFL_STATIC_LIBRARY_BUILD is defined in TfLite's CMakeLists.txt. TFL_CAPI_EXPORT is not empty when tensorflow\lite\core\c\c_api_types.h is being included in my application, e.g. indirectly in the minimum-working example by including tensorflow/lite/delegates/gpu/delegate.h. TFL_CAPI_EXPORT then expands to __declspec(dllimport) because TFL_STATIC_LIBRARY_BUILD is not defined. This leads to the linker error unresolved external symbol __imp_TfLiteGpuDelegateV2Create.
Work-around:
Define TFL_STATIC_LIBRARY_BUILD before including the TfLite headers in my application. Then TFL_CAPI_EXPORT is empty when tensorflow\lite\core\c\c_api_types.h is included in my application. But TFL_STATIC_LIBRARY_BUILD is an internal macro, the user should not be aware of this macro.
Proposed solution:
Remove __declspec(dllimport) from TFL_CAPI_EXPORT. It is optional for functions, it is merely a small optimization when calling functions and not of significant impact in compute-intensive applications as DNN processing.
Alternative: remove prefix TFL_CAPI_EXPORT from function declarations that are part not of the C API. Because I believe the TfLite C API by default builds to a .dll and the regular TfLite by default builds to a static lib.
Alternative2: Rethink this design with TFL_CAPI_EXPORT.
N.B. I build the regular TfLite, not the TfLite C API.
(Possibly) Related:
https://github.com/tensorflow/tensorflow/issues/61442
https://github.com/tensorflow/flutter-tflite/issues/83
https://github.com/tensorflow/flutter-tflite/issues/82
https://github.com/am15h/tflite_flutter_plugin/issues/60
https://groups.google.com/g/angleproject/c/xKmUgKZFpgY/m/65FHXZJjBwAJ
Standalone code to reproduce the issue
Minimal working example:
Build commands (in Windows Command Prompt):
git clone --single-branch --branch nightly https://github.com/tensorflow/tensorflow tensorflow_src
mkdir tflite_x64_release
cd tflite_x64_release
cmake -G "Visual Studio 17 2022" -A x64 -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded -DBUILD_SHARED_LIBS=OFF -DCMAKE_BUILD_TYPE=release -DTFLITE_ENABLE_GPU=ON ..\tensorflow_src\tensorflow\lite
cmake --build . -j 8 --config release

//#define TFL_STATIC_LIBRARY_BUILD  //Define this macro to avoid unresolved external symbol of TfLiteGpuDelegateV2Create
#include "tensorflow/lite/logger.h"
#include "tensorflow/lite/model.h"
#include "tensorflow/lite/kernels/register.h"
#include "tensorflow/lite/interpreter.h"
#include "tensorflow/lite/delegates/gpu/delegate.h"
#include <iostream>

int main() {
    tflite::LoggerOptions::SetMinimumLogSeverity(tflite::TFLITE_LOG_VERBOSE);

    std::unique_ptr<tflite::FlatBufferModel> model = tflite::FlatBufferModel::BuildFromFile("./lite-model_deeplabv3_1_metadata_2.tflite");  //Model from https://storage.googleapis.com/download.tensorflow.org/models/tflite/task_library/image_segmentation/rpi/lite-model_deeplabv3_1_metadata_2.tflite
    tflite::ops::builtin::BuiltinOpResolver resolver;
    tflite::InterpreterBuilder interpreter_builder(*model, resolver);
    interpreter_builder.SetNumThreads(1);
    TfLiteDelegate* gpu_delegate = TfLiteGpuDelegateV2Create(nullptr);
    interpreter_builder.AddDelegate(gpu_delegate);

    std::cout << "Done\n";
    return EXIT_SUCCESS;
}

Relevant log output
No response

Hi @misterBart, I noticed you are using GeForce GPU's ... CUDA isn't supported for Windows past TF 2.11, please review this note: https://www.tensorflow.org/install/source_windows#gpu to ensure your setup is supported. That being said I see you are trying in nightly, 2.16, and 2.10 ... which version do you wish to try? (If you want to try with WSL2 or tensorflow-cpu with TensorFlow-DirectML-Plugin)

I'm not using CUDA. This issue is completely unrelated to CUDA and Nvidia GPUs. And aside from testing with Geforce GPUs I also tested with an Intel HD 520 GPU, as I wrote down under "GPU model and memory" in my opening post. This issue is about a linker error under Windows, and I already posted a PR to solve this issue (see above).
This issue is present in Tf 2.10, 2.16 and in Nightly.


MARS on and pooling on throws quite a few exceptions in Debian 9

Describe the bug
Running with MultipleActiveResultSets=True it crashes the process in Debian 9
I have included some code below that reproduces the problem.
Once you run that code provided, after about 20 seconds it starts throwing:
System.Data.SqlClient.SqlException (0x80131904): Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.ComponentModel.Win32Exception (258): Unknown error 258
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
   at System.Data.SqlClient.SqlDataReader.get_MetaData()
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, SqlDataReader ds)
   at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior)
   at System.Data.SqlClient.SqlCommand.ExecuteReader()
   at Program.DoSelect(SqlConnection connection) in /home/john/test/Program.cs:line 96
   at Program.<>c__DisplayClass1_0.<Main>b__2() in /home/john/test/Program.cs:line 65
ClientConnectionId:75adcf5a-46df-4d92-96ec-475eb9761cbc
Error Number:-2,State:0,Class:11
ClientConnectionId before routing:39fc66d8-f418-471f-b691-a2af8c6ab694
Routing Destination:f4c9c9048e30.tr1.australiacentral1-a.worker.database.windows.net,11072

And then after a minute or so it crashed the process with:
Unhandled Exception: System.AggregateException: One or more errors occurred. (Timeout expired.  The timeout period elapsed prior to obtaining a connection from the pool.  This may have occurred because all pooled connections were in use and max pool size was reached.) (Timeout expired.  The timeout period elapsed prior to obtaining a connection from the pool.  This may have occurred because all pooled connections were in use and max pool size was reached.) ---> System.InvalidOperationException: Timeout expired.  The timeout period elapsed prior to obtaining a connection from the pool.  This may have occurred because all pooled connections were in use and max pool size was reached.
   at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
   at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)
   at System.Data.SqlClient.SqlConnection.Open()
   at Program.<>c__DisplayClass1_0.<Main>b__2() in /home/john/test/Program.cs:line 61
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.WaitAllCore(Task[] tasks, Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at Program.Main(String[] args) in /home/john/test/Program.cs:line 48
Aborted

To reproduce
Run the following snippet of code in a Debian 9 Azure VM.
You also need an Azure SQL database.
The connection string for an azure database should look something like:
Server=tcp:xxxx.database.windows.net,1433;Initial Catalog=test;Persist Security Info=False;User ID={your_username};Password={your_password};MultipleActiveResultSets=True;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;
Just be aware I modified the connection string to have MultipleActiveResultSets=True.
Snippet:
using System;
using System.Collections.Generic;
using System.Data.SqlClient;
using System.Threading;
using System.Threading.Tasks;

class Program
{
    static volatile int connectionOpen = 0;

    static void Main(string[] args)
    {
        var cb = new SqlConnectionStringBuilder(args[0]);

        try
        {
            using (var connection = new SqlConnection(cb.ConnectionString))
            {
                connection.Open();

                CreateTable(connection);
            }
        }
        catch (SqlException e)
        {
            Console.WriteLine(e.ToString());
        }

        var stop = false;
        Console.CancelKeyPress += (sender, eventArgs) =>
        {
            Console.Out.WriteLine("Stopping....");
            eventArgs.Cancel = true;
            stop = true;
        };

        var rnd = new Random();

        var task = Task.Run(() =>
        {
            while (!stop)
            {
                Console.Out.WriteLine($"Connections open {connectionOpen}");
                Thread.Sleep(1000);
            }
        });

        while (!stop)
        {
            var tasks = new List<Task>();
            for (var i = 0; i < 300; i++)
            {
                tasks.Add(Task.Run(() =>
                {
                    var opened = false;
                    try
                    {
                        using (var connection = new SqlConnection(cb.ConnectionString))
                        {
                            connection.Open();
                            Interlocked.Increment(ref connectionOpen);
                            opened = true;

                            DoSelect(connection);
                            Thread.Sleep(rnd.Next(10, 500));
                        }
                    }
                    catch (SqlException e)
                    {
                        Console.WriteLine(e.ToString());
                    }
                    finally
                    {
                        if (opened)
                        {
                            Interlocked.Decrement(ref connectionOpen);
                        }
                    }
                }));
            }

            Task.WaitAll(tasks.ToArray());
        }

        task.Wait();

        Console.Out.WriteLine("Done");
    }

    static void DoSelect(SqlConnection connection)
    {
        var tsql = @"SELECT * FROM test;";

        using (var command = new SqlCommand(tsql, connection))
        {
            using (SqlDataReader reader = command.ExecuteReader())
            {
                while (reader.Read())
                {

                }
            }
        }
    }

    static void CreateTable(SqlConnection connection)
    {
        var tsql = @"
                DROP TABLE IF EXISTS test;
                CREATE TABLE test
                (
                    DepartmentCode  nchar(4)          not null    PRIMARY KEY,
                    DepartmentName  nvarchar(128)     not null
                );";

        using (var command = new SqlCommand(tsql, connection))
        {

            command.ExecuteNonQuery();
        }
    }
}

Expected behavior
I expect the process not to crash if the max pool connections are reached, the same behaviour as in Windows.
Further technical details
I tried the following nuget packages:
System.Data.SqlClient version: 4.6.1
System.Data.SqlClient version: 4.7.0-preview6.19303.8
Microsoft.Data.SqlClient version: 1.0.19189.1-Preview
.NET target: Core 2.2
SQL Server version: SQL Azure, using Elastic Standard tier
Operating system:

Additional context
Also tried it on a docker container, same issue.

So I modified my repro code to use async/await code, and it seems to work fine!
using System;
using System.Collections.Generic;
using System.Data.SqlClient;
using System.Threading;
using System.Threading.Tasks;

class Program
{
    static volatile int connectionOpen;

    static async Task Main(string[] args)
    {
        var cb = new SqlConnectionStringBuilder(args[0]);

        try
        {
            using (var connection = new SqlConnection(cb.ConnectionString))
            {
                await connection.OpenAsync();

                await CreateTable(connection);
            }
        }
        catch (SqlException e)
        {
            Console.WriteLine(e.ToString());
        }

        var stop = false;
        Console.CancelKeyPress += (sender, eventArgs) =>
        {
            Console.Out.WriteLine("Stopping....");
            eventArgs.Cancel = true;
            stop = true;
        };

        var rnd = new Random();

        var task = Task.Run(() =>
        {
            while (!stop)
            {
                Console.Out.WriteLine($"Connections open {connectionOpen}");
                Thread.Sleep(1000);
            }
        });

        while (!stop)
        {
            var tasks = new List<Task>();
            for (var i = 0; i < 300; i++)
            {
                tasks.Add(Task.Run(async () =>
                {
                    var opened = false;
                    try
                    {
                        using (var connection = new SqlConnection(cb.ConnectionString))
                        {
                            await connection.OpenAsync();
                            Interlocked.Increment(ref connectionOpen);
                            opened = true;

                            await DoSelect(connection);
                            await Task.Delay(rnd.Next(10, 500));
                        }
                    }
                    catch (SqlException e)
                    {
                        Console.WriteLine(e.ToString());
                    }
                    finally
                    {
                        if (opened)
                        {
                            Interlocked.Decrement(ref connectionOpen);
                        }
                    }
                }));
            }

            await Task.WhenAll(tasks);
        }

        await task;

        Console.Out.WriteLine("Done");
    }

    static async Task DoSelect(SqlConnection connection)
    {
        var tsql = @"SELECT * FROM test;";

        using (var command = new SqlCommand(tsql, connection))
        {
            using (SqlDataReader reader = await command.ExecuteReaderAsync())
            {
                while (await reader.ReadAsync())
                {

                }
            }
        }
    }

    static async Task CreateTable(SqlConnection connection)
    {
        var tsql = @"
                DROP TABLE IF EXISTS test;
                CREATE TABLE test
                (
                    DepartmentCode  nchar(4)          not null    PRIMARY KEY,
                    DepartmentName  nvarchar(128)     not null
                );";

        using (var command = new SqlCommand(tsql, connection))
        {

            await command.ExecuteNonQueryAsync();
        }
    }
}

So the process crash was entirely my bad coding, I was only catching SqlException and when the pool overflows it throws a different exception, so I modified the catch (SqlException e) to catch (Exception e) and it now does not crash.
But I am still seeing lots of System.Data.SqlClient.SqlException (0x80131904): Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.ComponentModel.Win32Exception (258): Unknown error 258

Hi @johnsimons
It looks similar to #129 to me. Although that one doesn’t mention Mars.
If you can confirm it’s same, we can track this issue with #129 and close this one.

@cheenamalhotra it may be similar, but the issue I am seeing is when using System.Data.SqlClient not Microsoft.Data.SqlClient, and https://github.com/dotnet/SqlClient/issues/129#issuecomment-506802350 seems to be saying that it works in  System.Data.SqlClient

So this issue does occur with System.Data.SqlClient v4.6.1 as well?
I’m just trying to connect the dots as both issues have timeout expiration when pool reaches max limit.

Also you did mention “Microsoft.Data.SqlClient latest public” in the issue description.. Can you correct that so we can look into the exact version/range of versions?

@cheenamalhotra I updated those details, let me know if you need anything else?

@johnsimons
Would request you to test the scenario once again with latest preview version of the driver. We have made some related fixes, it may fix this issue. If not, we'll keep it open for more investigations into MARS.

@cheenamalhotra Have just tried the repro with the latest preview ( 1.0.19221.1-Preview) and still get the same errors

@cheenamalhotra as far as I can see only the new SqlClient was patched.
Taking into account that:
a) The new client is still at the pre-release stage
b) All bug reports use the old client
It would be great if the old client was patched too.

We are closing this issue due to inactivity. Feel free to open a new ticket if you still have the issue with the fix mentioned above.


fix(python): less breaking changes

🧭 What and Why
🎟 JIRA Ticket: -
Changes included:
A bit of DX improvements by removing some unnecessary breaking changes, which will hopefully help user migrate

will keep it flaky until https://github.com/algolia/api-clients-automation/pull/2441 is merged


LP-446, new encryption scheme implemented

C L.:  Only thing is to make sure to be careful about padding when you use this utility. nacl.box doesn’t do padding for you so you can trivially recover the length of the cleartext message from the ciphertext.


Add simple MySQL CDC integration test

The sample subscriptions include a MySQL CDC -> Kafka pipeline. For a simple integration test, we can try querying the table and the resulting Kafka topic.
We can do the same thing in the readiness probe to ensure that the integration tests don't run until after the pipelines are running. Eventually, the integration tests should check for correctness etc, while the readiness probes can remain trivial.
Testing
Integration tests show the CDC pipeline working:
--->%---
9/10         -- MySQL CDC -> Kafka
10/10        SELECT * FROM RAWKAFKA."products" LIMIT 1;
--->%---
+---------+-----+
| PAYLOAD | KEY |
+---------+-----+
| 3       | 101 |
+---------+-----+

I tripped over a couple bugs in the integration test workflow. ftr:

The K8s cluster was running out of CPU quota. I reduced Flink's CPU requests to 0.1, which seems to be the lowest I can go.
The readiness probe cannot include subscription tests -- subscriptions aren't deployed until after readiness.


Fastly Redirect Support

This PR will add a redirect to the redirects table of the provided FASTLY_SERVICE_ID.

The first entry in the example has been made manually, the second by running npm test.

I re-deployed the action with the new code and the corresponding required env variables (FASTLY_SERVICE_ID and FASTLY_TOKEN). Once new posts will start appearing on theblog.adobe.com, you should see entries being created (between 0 and 3 end of day).

As long as it's fewer than 1000 per hour, we should be good. Otherwise we might hit the rate limit.


Stabilization of builders suffixed by OrNull and OrThrow for NotBlankString

Description
Like discussed in #104, we would like to stabilize the experimental builders suffixed by OrThrow and OrNull for the NotBlankString type.

At this stage, we should remove the experimental annotations from these builders and, if needed, improve their tests, their performance and their documentation.

Dependencies
This issue is blocked by the following ones:

[x] #174

Checklist

[ ] Stabilize the builder suffixed by OrNull.
[ ] Stabilize the builder suffixed by OrThrow.
[ ] Test.
[ ] Refactor.
[ ] Check the generated documentation.
[ ] Check and update the public API binaries.
[ ] Update the Unreleased section in changelog.

These changes will be implemented by #262.


Incomplete and unclear

I would like to see the following included on the poster:

Namespace planning
Certificate requirements
Protocol flow
Similar to what the SfB team developed in its protocol workloads poster at https://www.microsoft.com/en-us/download/details.aspx?id=46448.

Also, some clear guidance on namespace planning would be in order to avoid those annoying certificate warnings. RDS has been around for quite some time, one would think it is about time to develop a solution that would work with a single certificate for the publicly facing namespace, without getting those annoying certificate warnings for NETBIOSNAME.domain.local. Citrix got it sorted quite nicely, hiding internal namespaces from the remote client. Why can't Microsoft do it? Why does Microsoft still think that loading the internal CA's cert on random public machines is the way to go?
According to https://blogs.technet.microsoft.com/askpfeplat/2017/12/18/remote-desktop-connection-rdp-certificate-warnings/, "The warning messages / pop-ups that end users see connecting via RDP are a GOOD THING."
WRONG. It is a BAD thing. The GOOD thing would be an RDS infrastructure that is consistent, secure, and it adheres to best practice (a.k.a. doesn't expose internal namespaces), to the point where there is NO NEED for these warnings.
Please fix it for once.
Thanks.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5daa735c-67fc-4461-682c-def85c8732d6
Version Independent ID: 831c1ab7-2fda-55a9-ffb2-98d59c915989
Content: Remote Desktop Services - planning poster
Content Source: WindowsServerDocs/remote/remote-desktop-services/rds-poster.md
Product: windows-server-threshold
GitHub Login: @lizap
Microsoft Alias: elizapo

Thanks for your dedication to our documentation. If we have been unable to review your issue in a timely manner, we sincerely apologize for the delayed response. We have captured the informtion you have provided for review. We are closing this issue for now, but if you feel that it's still a concern, please respond and let us know. If you determine another possible update to our documentation, please don't hesitate to reach out again. #please-close


Adds multiple pin option to bitcoinSwitch in LNURLDevices

Codecov Report

Merging #1080 (1bfa28d) into main (09871bb) will decrease coverage by 2.79%.
The diff coverage is 95.34%.

@@            Coverage Diff             @@
##             main    #1080      +/-   ##
==========================================
- Coverage   51.52%   48.72%   -2.80%
==========================================
  Files         238      238
  Lines       13396    13436      +40
==========================================
- Hits         6902     6547     -355
- Misses       6494     6889     +395

Impacted Files
Coverage Δ

lnbits/extensions/lnurldevice/crud.py
33.33% <ø> (ø)

lnbits/extensions/lnurldevice/lnurl.py
16.93% <ø> (ø)

lnbits/extensions/lnurldevice/tasks.py
64.28% <0.00%> (ø)

lnbits/extensions/lnurldevice/views.py
49.25% <50.00%> (ø)

lnbits/extensions/lnurldevice/migrations.py
89.65% <100.00%> (+9.65%)
:arrow_up:

lnbits/extensions/lnurldevice/models.py
92.95% <100.00%> (+4.06%)
:arrow_up:

lnbits/extensions/boltz/mempool.py
32.75% <0.00%> (-50.00%)
:arrow_down:

lnbits/extensions/boltz/crud.py
29.23% <0.00%> (-49.24%)
:arrow_down:

lnbits/extensions/boltz/utils.py
23.52% <0.00%> (-41.18%)
:arrow_down:

lnbits/extensions/boltz/boltz.py
17.91% <0.00%> (-39.31%)
:arrow_down:

... and 11 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


insert ca, cert and key to route if has byo

https://github.com/open-cluster-management/backlog/issues/10827

/hold

/unhold

Testing result append to https://github.com/open-cluster-management/backlog/issues/10828#issuecomment-809184373

/assign @clyang82

/lgtm


sessionStarted is not triggered sometimes on Android platform

Describe the bug
Create a speechRecognizer, call startContinuousRecognitionAsync, speak something, wait for the recognized result, then call stopContinuousRecognitionAsync.
Repeat the start --> stop process.
After N times; call start, sessionStarted is not triggered, so do other listeners.
After that, speech service is no longer available any more.
To Reproduce
Steps to reproduce the behavior:

speechRecognizer = new SpeechRecognizer()
Add listeners: speechRecognizer.sessionStarted.addEventListener()  ...
speechRecognizer.startContinuousRecognitionAsync()
wait for recognized result
speechRecognizer.stopContinuousRecognitionAsync()
repeat step 3 - step 5

Expected behavior
Repeat step 3 - step 5, speech service always works, no bug occurs.
Version of the Cognitive Services Speech SDK
'com.microsoft.cognitiveservices.speech:client-sdk:1.13.0'
Platform, Operating System, and Programming Language

OS: [e.g. Windows, Linux, Android, iOS, ...] - Android 9
Hardware - ARM
Programming language:  Java

my first question is:
When the sessionStarted is not triggered after call startContinuousRecognitionAsync().

Hi, thanks for logging this issue.
To clarify, are you waiting for the return from speechRecognizer.stopContinuousRecognitionAsync.get()?
I did not notice a ".get()" part in step 5 above.
Also, are you waiting to receive a canceled event before repeating steps 3-5?

@glharper
Thanks for your help.
I don't wait for the return from speechRecognizer.stopContinuousRecognitionAsync.get().
Since .get() will block the current thread, right?
I do listen canceled event,  speechRecognizer.canceled.addEventListener(),
but only used when no recognized result is returned.

@jzhw0130 sessionStopped (not canceled) is the event to wait for before restarting recognition. Also, you should make sure the stopContinuousRecognitionAsync call has completed (in whatever manner of your choosing for awaiting an asynchronous method) before restarting, not doing so can put the recognizer in an undefined state. Are you able to reproduce after taking these measures?

@glharper
Thanks for your support.
I will try your suggestion.
I still want to known in which case sessionStarted  is not triggered after all startContinuousRecognitionAsync()?

@glharper
BTW, is there a way to get the recognizer status directly except listening the events?

@glharper
As you suggested, I encountered 2 weird issues:

It cost about 1 minute to get sessionStopped after call stopContinuousRecognitionAsync() !!!
Get cached recognized result from the past round

Refer to the logs:
// ROUND 1
2020-08-02 13:53:19.622 16972-16972/com.mosaix.sdkdemo I/MosaixSDK: MicrophoneStream startRecording()
2020-08-02 13:53:19.638 16972-16972/com.mosaix.sdkdemo I/MosaixSDK: startContinuousRecognitionAsync()
2020-08-02 13:53:19.652 16972-17046/com.mosaix.sdkdemo V/MosaixSDK: ---> callback sessionStarted triggered
2020-08-02 13:53:19.815 16972-17058/com.mosaix.sdkdemo V/MosaixSDK: azure audio level: 19.194327739455797
...
2020-08-02 13:53:49.618 16972-17058/com.mosaix.sdkdemo V/MosaixSDK: azure audio level: 24.44538076510554
2020-08-02 13:53:49.664 16972-17067/com.mosaix.sdkdemo V/MosaixSDK: MicrophoneStream stopRecording()
2020-08-02 13:53:49.664 16972-17067/com.mosaix.sdkdemo I/MosaixSDK: stopContinuousRecognitionAsync()
2020-08-02 13:54:29.196 16972-17046/com.mosaix.sdkdemo W/MosaixSDK: ---> callback recognizing triggered. !!!!!
2020-08-02 13:54:37.299 16972-17046/com.mosaix.sdkdemo W/MosaixSDK: ---> callback recognizing triggered. !!!!!
2020-08-02 13:54:37.819 16972-17046/com.mosaix.sdkdemo W/MosaixSDK: ---> callback recognized triggered. !!!!!
2020-08-02 13:54:48.133 16972-17046/com.mosaix.sdkdemo V/MosaixSDK: ---> SessionStoppedHandler. !!!!!
// ROUND 2
2020-08-02 13:56:17.248 16972-16972/com.mosaix.sdkdemo I/MosaixSDK: MicrophoneStream startRecording()
2020-08-02 13:56:17.274 16972-16972/com.mosaix.sdkdemo I/MosaixSDK: startContinuousRecognitionAsync()
2020-08-02 13:56:17.290 16972-17046/com.mosaix.sdkdemo V/MosaixSDK: ---> callback sessionStarted triggered
2020-08-02 13:56:17.459 16972-17669/com.mosaix.sdkdemo V/MosaixSDK: azure audio level: 25.26270201624833             // Speak how come
…
2020-08-02 13:56:19.459 16972-17669/com.mosaix.sdkdemo V/MosaixSDK: azure audio level: 34.33886589857751
2020-08-02 13:56:19.538 16972-17046/com.mosaix.sdkdemo W/MosaixSDK: ---> callback ResultReason.NoMatch triggered
2020-08-02 13:56:19.539 16972-17046/com.mosaix.sdkdemo I/MosaixSDK: MicrophoneStream stopRecording()
2020-08-02 13:56:19.589 16972-17046/com.mosaix.sdkdemo V/MosaixSDK: stopContinuousRecognitionAsync().
2020-08-02 13:56:20.905 16972-17046/com.mosaix.sdkdemo V/MosaixSDK: ---> SessionStoppedHandler.
// ROUND 3
2020-08-02 13:56:28.279 16972-16972/com.mosaix.sdkdemo I/MosaixSDK: MicrophoneStream startRecording()
2020-08-02 13:56:28.302 16972-16972/com.mosaix.sdkdemo I/MosaixSDK: startContinuousRecognitionAsync()
2020-08-02 13:56:28.314 16972-17046/com.mosaix.sdkdemo V/MosaixSDK: ---> callback sessionStarted triggered
2020-08-02 13:56:28.481 16972-18064/com.mosaix.sdkdemo V/MosaixSDK: azure audio level: 27.82586154758063.         // Never speak
…
2020-08-02 13:56:29.785 16972-18064/com.mosaix.sdkdemo V/MosaixSDK: azure audio level: 34.708136415663276
2020-08-02 13:56:29.838 16972-17046/com.mosaix.sdkdemo I/MosaixSDK: callback recognizing triggered: how.                // But received recognized result from the past round
2020-08-02 13:56:29.979 16972-17046/com.mosaix.sdkdemo I/MosaixSDK: callback recognizing triggered: how come
2020-08-02 13:56:29.982 16972-17046/com.mosaix.sdkdemo I/MosaixSDK: callback recognized triggered: How come
2020-08-02 13:56:30.015 16972-17046/com.mosaix.sdkdemo V/MosaixSDK: stopContinuousRecognitionAsync()
2020-08-02 13:56:30.015 16972-17046/com.mosaix.sdkdemo I/MosaixSDK: MicrophoneStream stopRecording()
2020-08-02 13:56:30.971 16972-17046/com.mosaix.sdkdemo V/MosaixSDK: ---> SessionStoppedHandler

@jzhw0130 Thanks for your work reproducing this. Have you been able to reproduce the original issue (sessionStarted not called) in your test?
Answers to your questions/issues:

There is not a good way to get an actual current state of the recognizer, other than listening to callbacks and debug messages.
The one minute delay in closing the recognizer sounds frustrating, but is dependent on the underlying network connection
The past recognizer events triggered for a new session sounds like a bug, I will put it on our bug list. Thanks for finding it, we'll ping you when it's fixed.

@glharper
Thanks for your quick response.
Since there's a long delay between stopContinuousRecognitionAsync() and SessionStoppedHandler triggered.
If I call startContinuousRecognitionAsync()  before SessionStoppedHandler triggered, it will reproduce the original issue (sessionStarted not called)

@jhakulin I've added a bug with a quickly started session triggering recognized events from a previous session. Is there also a perfomance issue here with sessionStopped not being sent for a minute after stopContinuousRecognition has been called?

So internally what's happening is that StopContinuousRecognitionAsync is that the SDK will tell the Speech Service that its done sending audio, and it will wait for the Speech Service to return the results for all the audio that's been sent, or 10 seconds to happen.
Normally this delay is small as the Speech Service isn't more than a few hundred ms behind the SDK in audio recognition, especially for a microphone source where audio is always available.
Sometimes that's not the case, and things get less predictable.
From the log, @jzhw0130 do you happen to be sending audio in through a Push or a Pull Stream? The rules around preserving unsent audio between start / stop calls are different for files and streams vs microphones, I see your log has entries like:
"MicrophoneStream startRecording"
And I'm wondering if the app is handling the microphone directly and pushing the audio into the SDK?
I've got a few theories that I think at least explain the behaviors you're seeing, but they depend on how the audio is being delivered to the SDK.

@rhurey
Thanks for your help.
Yep, I create the AudioRecord instance, and set it as the stream input of speechRecognizer.
Refer to code:
public class MicrophoneStream extends PullAudioInputStreamCallback  {
...
public MicrophoneStream() {
this.recorder = new AudioRecord.Builder()
.setAudioSource(MediaRecorder.AudioSource.VOICE_RECOGNITION)
.setAudioFormat(af)
.build();
}
@Override
public int read(byte[] bytes) {
    long ret = this.recorder.read(bytes, 0, bytes.length);
    return (int)ret;
}
....

}
microphoneStream = new MicrophoneStream();
speechRecognizer = new SpeechRecognizer(config, preferredLanguage, AudioConfig.fromStreamInput(microphoneStream));

@rhurey
BTW, I replicated the issue and caught the azure audio log, help check, thx
I tested 2 times, it works well for 1st time, but did not work for 2nd time.
2020-08-10 09_57_36 -azure-speech-audio.txt

Log was useful...
So in the log we have 2 phrases and a lot of silence.
The first phrase everything looks ok, the duration of transmitted audio was ~5 seconds, and when stop was called the service had recognized ~3.1s of speech and acknowledged the full 5s of audio. (Mostly, about 640b (20ms) remained in the transmit buffer.)
When the 2nd session started up, that remaining 640b was sent and audio was pulled in.
There's about 1.5s of silence and 2.5s of speech.
At this point latency with the service is running ~1s, while on the high side not horrible.
About 20s later the service reports that it's been receiving silence for 20s.
Latency is running ~650ms behind real time.
~25s later we get a message that the service has seen 5s of silence.
At this point things are running ~22s behind real time.
The 5s silence notifications go on until the 2 minute mark when the SDK reconnects.
Over the next 30 seconds recognition catches back up and the service winds up ~330ms behind.
What's happening is that the recognition in the speech service is (for some reason) running behind occasionally. When this occurs the client SDK will endeavor to make sure all speech is recognized by replaying buffered speech, and has some rules around how that works.
When a call is made to stop recognition, the SDK will stop sending audio and wait for the Speech Service to catch up and report its done with recognition.
There's three criteria (roughly) for the Speech Service to report being done with recognition.

The Speech Service hits a marker in the audio stream placed by the SDK to indicate quitting. This marker is placed when there's no more audio (because an end of file was reached) or a call to stop recognizing was made.
The maximum silence before or after speech has been reached. (Typically defaulting in the 20s range)
The recognition requested was for only a single phrase.

In the event #2 happens and continuous speech recognition has been requested, the SDK will signal the Speech Service to continue recognition anyways.
With that information as a foundation...

// ROUND 1
2020-08-02 13:53:19.622 16972-16972/com.mosaix.sdkdemo I/MosaixSDK: MicrophoneStream startRecording()
2020-08-02 13:53:19.638 16972-16972/com.mosaix.sdkdemo I/MosaixSDK: startContinuousRecognitionAsync()
2020-08-02 13:53:19.652 16972-17046/com.mosaix.sdkdemo V/MosaixSDK: ---> callback sessionStarted triggered
2020-08-02 13:53:19.815 16972-17058/com.mosaix.sdkdemo V/MosaixSDK: azure audio level: 19.194327739455797
...
2020-08-02 13:53:49.618 16972-17058/com.mosaix.sdkdemo V/MosaixSDK: azure audio level: 24.44538076510554
2020-08-02 13:53:49.664 16972-17067/com.mosaix.sdkdemo V/MosaixSDK: MicrophoneStream stopRecording()
2020-08-02 13:53:49.664 16972-17067/com.mosaix.sdkdemo I/MosaixSDK: stopContinuousRecognitionAsync()
2020-08-02 13:54:29.196 16972-17046/com.mosaix.sdkdemo W/MosaixSDK: ---> callback recognizing triggered. !!!!!
2020-08-02 13:54:37.299 16972-17046/com.mosaix.sdkdemo W/MosaixSDK: ---> callback recognizing triggered. !!!!!
2020-08-02 13:54:37.819 16972-17046/com.mosaix.sdkdemo W/MosaixSDK: ---> callback recognized triggered. !!!!!
2020-08-02 13:54:48.133 16972-17046/com.mosaix.sdkdemo V/MosaixSDK: ---> SessionStoppedHandler. !!!!!

Here the odds are the Speech Service got behind in the first recognition, and there's ~30 seconds of audio it's recorded, but it's not clear how much it took to match the recognition.

// ROUND 2
2020-08-02 13:56:17.248 16972-16972/com.mosaix.sdkdemo I/MosaixSDK: MicrophoneStream startRecording()
2020-08-02 13:56:17.274 16972-16972/com.mosaix.sdkdemo I/MosaixSDK: startContinuousRecognitionAsync()
2020-08-02 13:56:17.290 16972-17046/com.mosaix.sdkdemo V/MosaixSDK: ---> callback sessionStarted triggered
2020-08-02 13:56:17.459 16972-17669/com.mosaix.sdkdemo V/MosaixSDK: azure audio level: 25.26270201624833 // Speak how come
…
2020-08-02 13:56:19.459 16972-17669/com.mosaix.sdkdemo V/MosaixSDK: azure audio level: 34.33886589857751
2020-08-02 13:56:19.538 16972-17046/com.mosaix.sdkdemo W/MosaixSDK: ---> callback ResultReason.NoMatch triggered
2020-08-02 13:56:19.539 16972-17046/com.mosaix.sdkdemo I/MosaixSDK: MicrophoneStream stopRecording()
2020-08-02 13:56:19.589 16972-17046/com.mosaix.sdkdemo V/MosaixSDK: stopContinuousRecognitionAsync().
2020-08-02 13:56:20.905 16972-17046/com.mosaix.sdkdemo V/MosaixSDK: ---> SessionStoppedHandler.

Here, I suspect with the InitialSilenceTimeout value of 5s, the NoMatch is in regards to cached audio from the previous recognition being recognized.

// ROUND 3
2020-08-02 13:56:28.279 16972-16972/com.mosaix.sdkdemo I/MosaixSDK: MicrophoneStream startRecording()
2020-08-02 13:56:28.302 16972-16972/com.mosaix.sdkdemo I/MosaixSDK: startContinuousRecognitionAsync()
2020-08-02 13:56:28.314 16972-17046/com.mosaix.sdkdemo V/MosaixSDK: ---> callback sessionStarted triggered
2020-08-02 13:56:28.481 16972-18064/com.mosaix.sdkdemo V/MosaixSDK: azure audio level: 27.82586154758063. // Never speak
…
2020-08-02 13:56:29.785 16972-18064/com.mosaix.sdkdemo V/MosaixSDK: azure audio level: 34.708136415663276
2020-08-02 13:56:29.838 16972-17046/com.mosaix.sdkdemo I/MosaixSDK: callback recognizing triggered: how. // But received recognized result from the past round
2020-08-02 13:56:29.979 16972-17046/com.mosaix.sdkdemo I/MosaixSDK: callback recognizing triggered: how come
2020-08-02 13:56:29.982 16972-17046/com.mosaix.sdkdemo I/MosaixSDK: callback recognized triggered: How come
2020-08-02 13:56:30.015 16972-17046/com.mosaix.sdkdemo V/MosaixSDK: stopContinuousRecognitionAsync()
2020-08-02 13:56:30.015 16972-17046/com.mosaix.sdkdemo I/MosaixSDK: MicrophoneStream stopRecording()
2020-08-02 13:56:30.971 16972-17046/com.mosaix.sdkdemo V/MosaixSDK: ---> SessionStoppedHandler

And now we see the last of the unrecognized audio get recognized.
I'll speak with some folks from our service team and see if we can determine what happened to the session from the log you sent on the service side. The session ID is the important part from the client to trace things through the service.

@rhurey
Thank you for your detailed explanation.
Let's wait for the investigation from your service team.

Got some information to add.
There's a ~23 second gap in audio arrival at the service that isn't accounted for in the logs where the platform under the SDK was given audio packets at a consistent rate. This gap accounts for the large fall behind real-time transcription.
Is there any connectivity tracing available from the device? I noticed you have another issue with problems connecting...

@rhurey
They are the same issue on different level.
The current issue is on APPLICATION level, the other is on SDK level.
When the issue occured, I tried to play youtube videos, it worked, seems the network is ok

They all feel slightly related based on SDK behaviors.
Looking at things, the log / session ID we have in this issue is likely caused by two factors:

The service got behind due to a 20 second audio arrival gap.
The SDK when stopping waits 10 seconds for the service and then returns, but can still receive events.

Looking at issue #696, calling Close() will internally call Stop() to put an internal state machine into a known state before destroying resources, which can be subject to the same delay.
#700 is likely also some fallout from the same behaviors.
So.... What to do....
@glecaros picked up a task to improve the stop algorithm to both be faster with the service isn't responding quickly and do a better job of terminating the underlying connection to prevent zombie events.
I opened another task to look at a way to get the volume information out of the SDK so you wouldn't need to use a stream to hand the audio in, I suspect that may help with the audio caching.
As for the gaps in audio arrival. A bit stuck here. We'd need to look at more sessions with problems to see if a pattern can be found.

@rhurey
Thanks for your help.
BTW, We still need to keep stream audio feature.
Since our TV application support two audio input, one from remote control (mic), another one from mobile application which is installed on the phone (audio input stream).

Closing this as we now have a work item for this issue on the backlog to be prioritized and planned. We'll update with information about the release schedule when known. Currently there is a tentative target for some improvement on this and related issues in the Speech SDK 1.16 release next year.