trackBitrate and bufferedPosition are always 0 for RTMP

Describe the bug
When reproducing an RTMP stream both trackBitrate and bufferedPosition are always 0.
It works as expected when reproducing HLS for instance.
To get these values i'm accessing them like this:
let bitrate = player.getCurrentTrack(CicadaTrackType(rawValue: 0))?.trackBitrate
let buffer = player.bufferedPosition

To Reproduce
Steps to reproduce the behavior:

Just open any RTMP stream

Expected behavior
We should get the current values for bitrate and buffer as we do for HTTP streams.
Also I have one more question, is there any way to obtain bandwidth information for the current stream? Couldn't find anything in the public API.
Thanks!

Thanks for the feedback，
I think there is no bitrate provided by CicadaPlayer except hls master playlist which have multiple bitrate.
We haven't updated the buffer position for live stream, you can modify the logic like below if you want it:
void SuperMediaPlayer::PostBufferPositionMsg()
{
    if (mPlayStatus == PLAYER_PAUSED || mPlayStatus == PLAYER_PLAYING) {
        int64_t duration;

        if (isSeeking()) {
            duration = 0;
        } else {
            duration = getPlayerBufferDuration(false, false);
        }

        if (duration >= 0) {
            mBufferPosition = getCurrentPosition() + duration;

            if (mEof) {
                mBufferPosition = mDuration;
            }

            mPNotifier->NotifyBufferPosition((mBufferPosition <= mDuration ? mBufferPosition : mDuration) / 1000);
        }
    }
}

But what your usecase of this ?

I need to report stats of each stream and it should be consistent for all protocols, so that we can report bitrate, bandwidth and buffer position for all kind of streams.

Bitrate: You say then there is no way to obtain it, correct?
BufferPosition: Could i send a Pull Request with this change or is this something you don't want included in the project?
Bandwidth: Is there any way to obtain the current bandwidth used by the player?

Is buffer duration will be better for your usecase to report, buffer position is used to display the seek bar usually, which is the reason of our logic for now.
If so, you can add a api to get the buffer duration, I think it's very easy.
The bitrate is not a fix value except a cbr video, for now, there is a value calculated by ffmpeg when call avformat_find_stream_info(), is this value satisfied for you？
If not, we need to calculate it along with the playing progress, and then, how to define the current bandwidth? The average value
for a period time? Or the total average value?

Yes, i'm looking to report buffer duration, so i guess that bufferPosition - currentPosition should do the trick.
But looking into the PostBufferPositionMsg method, if it's not being reported because of the condition you mention
if (duration >= 0) { // delete this condition

does that mean that getPlayerBufferDuration(false, false); is always 0 for RTMP? Or am I missing something?

No, getPlayerBufferDuration get the real duration in mBufferController,  I think you can get the buffer duration for all tracks, and report them, not use getPlayerBufferDuration


Loading orders when chain out of date goes over 100%

See screenshot ^

No worries, this has been fixed but currently it's not deployed. Thank you for the report though!


Use the built-in fileURLToPath that works cross-platform

Currently hq does not work properly on Windows because HQ_ROOT is not set correctly.
console.log(import.meta.url);
// file:///v:/hq/server.mjs
console.log(import.meta.url.slice('file://'.length));
// /v:/hq/server.mjs <== illegal for windows
console.log(path.dirname(url.fileURLToPath(import.meta.url)));
// v:\hq <== correct one

This pull request replaces .slice  with the built-in url.fileURLToPath that ensures a cross-platform valid absolute path string.

Great job @liqi0816! Thanks for the help. Did you check if everything else work as expected? Unfortunately I don't have windows to test it.

Great job @liqi0816! Thanks for the help. Did you check if everything else work as expected? Unfortunately I don't have windows to test it.

I have tested an html/sass/typescript setting and it worked as expected. I guess(?) most other compilers will also work. Do we have a test case or a list of features that should be tested?

Great job @liqi0816! Thanks for the help. Did you check if everything else work as expected? Unfortunately I don't have windows to test it.

I have tested an html/sass/typescript setting and it worked as expected. I guess(?) most other compilers will also work. Do we have a test case or a list of features that should be tested?

Thanks, I guess it will work then. At the moment there are no proper tests I have about 20 projects that I'm trying before the release.

published along with other improvements as<EMAIL_ADDRESS>


Replace instances of PHP with Node.js

Fixed mentions of PHP as this is not PHP tutorial

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. We will now review your pull request.
Thanks,
Microsoft Pull Request Bot

@thehungrycoder : Thanks for your contribution to the Azure documentation! The author, @cephalin, has been notified to review your proposed change.

@thehungrycoder Thanks for the correction! Sorry I didn't see this one for a long time. #sign-off


ARROW-17453: [Go][C++][Parquet] Inconsistent Data with Repetition Levels

Both the C++ and Go parquet implementations assumed that if the max repetition level was 0, that there were no bytes to be skipped when initializing decoders for DataPageV2 but the Parquet files produced by Athena in this case had repetition bytes to be skipped before getting the definition level bytes. Since the byte wasn't skipped, the wrong values were decoded for Definition levels.
In the case of the Go implementation, it made additional assumptions that proved to be incorrect on top of the same bug.
This fixes both of them to properly respect the repetition level byte length reported in the DataPageV2 header.

Added comment with extra clarification and changed std::array to std::vector.
Also, this is the same issue in https://issues.apache.org/jira/browse/ARROW-17452 so I resolved it as a duplicate.

R failure is unrelated to this change.
@thisisnic @dragosmg could one of you or someone else look into it when you get a chance? I think it's a flaky workflow and spurious failure... but i could be wrong.

R failure is unrelated to this change.
@thisisnic @dragosmg could one of you or someone else look into it when you get a chance? I think it's a flaky workflow and spurious failure... but i could be wrong.

The error appeared after #13665 was merged, so definitely unrelated to this change (but also not spurious). It's present in some of my recent PRs too. See #13960.

@dragosmg thanks for the confirmation. Once the last couple checks pass i'll merge this.


If number of threads are changes, which is a live change, we must han…

…dle that seamlessly with metrics too.
@geirst and @vekterli PR

There will be a separate PR to safeguard the API.


1.15.2 Target Selectors Don't Work

The selectors @td, @t, @tn, and @tuuid do not resolve to anything on 1.15.2 Paper. Everything else like @uuid, @tz, and so on works perfectly fine. There are no errors in console or in chat.
 PVP protection was interfering with the target selectors!


Warning when updating scoop

Every time I run scoop update I get the following warning at the Updating Scoop... step:
WARN  Overwriting shim to scoop.ps1 installed from

Is that intended behavior or is there something wrong with my installation?

We are on it: https://github.com/lukesampson/scoop/pull/2564

Ah, that's nice. I only looked for the problem in the issues.


reduce verbosity of rule definitions

Might be even more compact than your initial syntax, the goal is to make the flow/sequence of rules more obvious.
The reason I did this small refactor is that I want to try to support single/double quoted strings in places where the syntax is known like GM_addStyle and /* css */ etc. but now I wonder maybe there was a reason you didn't add it in the first place?
@orionlee, ping!

@tophf I was hoping to get to it this weekend. Sorry for not getting back to it for so long.

Before addressing the PR, I'd to address supporting the single/double quoted strings in places where the syntax is known like GM_addStyle and /* css */ etc.
I shied away from supporting it initially mainly to reduce complexity or providing less than half-baked support. I'll use the current (limited) support for single/double quoted strings for HTML as motivation.

Typical use cases single/quoted strings are when such strings are concatenated together.
Therefore, when the tokenizer passes individual string, it'll only see a fragment of incomplete html, leading to incorrect parsing. E.g., in the above codes, the strings '">
', 'plain text
' are not recognized at all.

So if we support single/double quoted string for CSS using similar approach,

we'll also have some incomplete / incorrect parsing
the rule to support it is not exactly trivial. For HTML, we use a very simple heuristics that recognize strings that being with a tag by a simple regexp match locally. The codes don't really need to consider the context.
There is no similar local heuristics for CSS. Consider the folllowing snippets:

  GM_addStyle(".class1 { font-size: " + getZoomRatio() + "; } .class2 { border: "  +
     getBorderWidth() " solid red; }");

The rules would need to recognize:

string beyond the first one, e.g.,  "; } .class2 { border: ", are still part of GM_addStyle call, and should be in CSS mode.
it needs to know when to end it too. In this case, recognize the token ")" at the end as ending the GM_addStyle call.
Such bookkeeping might have been done in the javascript mode parser. I don't know the internals of it enough to access it.
It also has the similar (to HTML) incomplete parsing problem. E.g., it probably can't recognize " solid red; }" as part of the border: .

All in all, I feel we might end up requiring a lot of complexity to handle different cases, and the support might still be incomplete.
I feel syntax highlighting single / doubted quoted strings is less useful in practice as well (other than aesthetically being more pleasing and uniform). E.g., the highlighter would be less helpful in recognizing syntax issues, e.g., missing ending } bracket, as the css text is typically in multiple string fragments.

For the PR, I'll address the compact syntax in this comment, and other minor ones in another one to make things clearer.
With the more compact syntax, I feel the rules are easier to read, but changing rules might be more error prone, as it is less clear on what is allowed and what is not allowed. E.g., in the css rule,
      // GM_addStyle(`css-string`);
      [
...
        ['quasi', { // if it's a string template
          next: 'css-in',
          mode: cssMode,
          onMatch: prepReparseStringTemplateInLocalMode,
        }],
        [isEndBacktick, {
          id: 'css-in',
...

In the 'quasi' check rule, I thought "next: 'css-in'," is optional, as it seems that next is defaulted to the next rule in the list. But it turns out it is required after all: taking it out leads to some exception.
@tophf Given there is some distance in time (from the PR you submitted initially a few months back), what do you feel about the syntax now? Do you feel it is as maintainable as it was back then, when everything was fresh in your mind? If so, let's proceed to merge the PR.

Only 1 other comment for the PR:

There is some small enhancement, of supporting lang=html as a hint (and similar ones for CSS), it'd be great if they are the new syntax is added to the test (in test/javascript-mixed/code-source.txt), which serves as a documentation of sort on what the mode supports and its limitation.

AFAIK even IDE don't support fragmented nested syntax... but even if they do it sounds so complicated - and the result may still be fragile - that I have a feeling we shouldn't implement it.

Indeed, I negated my own idea and now the syntax is less obvious so I'll think about it more.

We can support the trivial case: just the first single/double-quoted string.

That'd work too. It's limited, but in a way that developers could easily understand.
Bear in mind that html is largely supported in practice due to the support of automatic highlighting single/double quoted string as html as long as it starts with a tag. The main missing piece is css
E.g., the following code snippets are highlighted as:
// automatic highlight for HTML single/double-quoted string, as long as the string begin with a tag
let someMsg = 'hello';

// no highlight for HTML single/double-quoted string when the string does not begin with a tag
// a html hint will not help either.
let someMsg2 = 'Foobar hello';
let someMsg3 = /* html*/ 'Foobar hello';

// no highlight for CSS single/double-quoted string
let someCSS = /* css */ ".msg { border: 1px; }";
GM_addStyle(".msg { border: 1px; }");

Current result:

@orionlee, I've removed the fuzziness from definitions so only Rule is accepted with one convenience: match can be string/RegExp, which is produces the match function automatically in makeRules.


Switch block_time to unix timestamp

TL;DR
Changed timestamp representation from time.Time to uint64 across multiple structs and functions.
What changed?

Modified Block, Log, Trace, and Transaction structs to use uint64 for timestamp fields instead of time.Time.
Updated ClickHouseConnector methods to handle uint64 timestamps directly, removing time.Unix() conversions.
Adjusted serializeBlock and serializeTransaction functions to work with uint64 timestamps.
Removed hexSecondsTimestampToTime function as it's no longer needed.
Removed time import from affected files.

How to test?

Run unit tests for affected structs and functions.
Perform integration tests with ClickHouse to ensure proper data insertion and retrieval.
Verify that timestamp-related operations in the application still work correctly with the new uint64 representation.

Why make this change?
Using uint64 for timestamps instead of time.Time offers several benefits:

Improved performance by reducing type conversions.
Simplified data handling and storage in ClickHouse.
Consistent representation of timestamps across the application.
Reduced memory usage by using a more compact data type.

This change aligns the timestamp representation with common blockchain standards and optimizes data processing throughout the indexer.

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#71  👈
#70
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @AmineAfia and the rest of your teammates on  Graphite


refactor: convert panel actions to enums

may as well while we're here.
I think all the switches are already switch expressions?

I think all the switches are already switch expressions?

Not quite. The method you just fixed is like this:
    method {
      int quantity = 0;
      switch (quantityType)
      ... case statements to calculate/set quantity
      }
      return quantity;
    }

I've converted more switch expressions, but I've left that one. It is a switch expression (case -> instead of case:), just the expression isn't used. It's a nontrivial conversion as quantity changes in the middle of some expressions, and there aren't tests around there.

I've converted more switch expressions, but I've left that one. It is a switch expression (case -> instead of case:), just the expression isn't used. It's a nontrivial conversion as quantity changes in the middle of some expressions, and there aren't tests around there.

Fair enough.


Creating IsPrime function

A function to check whether a number is prime

دي اول function ليك في ال challenge ؟؟؟
On Fri, 27 Dec 2024, 9:00 pm Moealfadil, @.***> wrote:

Assigned #16
https://github.com/MIT-Emerging-Talent/ET6-foundations-group-18/pull/16
to @TagwaHashim https://github.com/TagwaHashim.
—
Reply to this email directly, view it on GitHub
https://github.com/MIT-Emerging-Talent/ET6-foundations-group-18/pull/16#event-15774423192,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BN5KGKVDG3J2KW2IW2CFYM32HWPTJAVCNFSM6AAAAABUI7MOXGVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJVG43TINBSGMYTSMQ
.
You are receiving this because you were assigned.Message ID:
<MIT-Emerging-Talent/ET6-foundations-group-18/pull/16/issue_event/15774423192
@github.com>

ااي
On Fri, 27 Dec 2024 at 9:34 PM TagwaHashim @.***> wrote:

دي اول function ليك في ال challenge ؟؟؟
On Fri, 27 Dec 2024, 9:00 pm Moealfadil, @.***> wrote:

Assigned #16
https://github.com/MIT-Emerging-Talent/ET6-foundations-group-18/pull/16

to @TagwaHashim https://github.com/TagwaHashim.
—
Reply to this email directly, view it on GitHub
<
https://github.com/MIT-Emerging-Talent/ET6-foundations-group-18/pull/16#event-15774423192>,

or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/BN5KGKVDG3J2KW2IW2CFYM32HWPTJAVCNFSM6AAAAABUI7MOXGVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJVG43TINBSGMYTSMQ>

.
You are receiving this because you were assigned.Message ID:

<MIT-Emerging-Talent/ET6-foundations-group-18/pull/16/issue_event/15774423192

@github.com>

—
Reply to this email directly, view it on GitHub
https://github.com/MIT-Emerging-Talent/ET6-foundations-group-18/pull/16#issuecomment-2563976321,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BB3SKKRVU56U55UTDO7CRG32HWTTZAVCNFSM6AAAAABUI7MOXGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRTHE3TMMZSGE
.
You are receiving this because you authored the thread.Message ID:
@.***
com>


Would like a way to create an outgoing TLS VConn

I'd like an API function that looks like this:
tsapi TSAction TSSslNetConnect(TSCont, const struct sockaddr *, TSSslContext *);

This would initiate an outgoing connection and negotiate TLS with the remote server, then return a TLS VConn that I can use for I/O.

The recommendation from the ATS Community is to use the OpenSSL API directly.


Hello, was wondering if you could bring back item shadowing?

I noticed that item shadowing is once again broken. Was wondering if this was on the list to fix or if this is just being left behind. Was just wondering as I really wanted it. Thank you anyways.

It's being worked on

I noticed that item shadowing is once again broken. Was wondering if this was on the list to fix or if this is just being left behind. Was just wondering as I really wanted it. Thank you anyways.

I just learned about this project...
If you need item shadowing right now, try my project, I remade it
In 1.20.4, you may need other features in my project to use item shadowing
https://github.com/Crystal0404/CrystalCarpetAddition


Remove useless attributes from implementation files

Remove reduntant / useless attributes
[[maybe_used]] on unconditionally always used variables
[[nodiscard]] on function definitions (they are only needed in declarations)

SFMLBuildMaster: Build this please.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>1
Details

3 of 3 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.009%) to 55.817%

Totals

Change from base Build<PHONE_NUMBER>8:
0.009%

Covered Lines:
11671

Relevant Lines:
19688

💛  - Coveralls


[SPARK-27925][ML] Better control numBins of curves in BinaryClassificationMetrics

What changes were proposed in this pull request?
bring the grouping op forward into the sort op, and regard one partition as one bin.
How was this patch tested?
existing tests and modified tests (to test the case with > 1 partitions)

Test build #106103 has started for PR 24775 at commit 1ab2dd1.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/11354/
Test PASSed.

Test build #106103 has finished for PR 24775 at commit 1ab2dd1.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106103/
Test PASSed.


Fixes to automatically update documentation on opae.github.io

Add a directory 'latest' that stores the latest documentation that is generated with every Travis build on a commit that is not tagged for release
Generate and update a landing page to link to the respective document versions: Currently just a basic page with links and will go through improvements in visualization after the release.

There seems to be an issue with the document generation, the documentation is generated and uploaded to opae.github.io correctly but the webpages don't seem to be loading with the proper theme.
@luebbers @abelardojarab @r-rojo can you think of a reason why there could be a problem?

@nakulkorde it seems it is not finding the .css style file

I fixed it, just had to add the .nojekyll file to the root directory so that github pages would not try to use it's default parsing.


Blocks placed in water turn to ice

When you use Glacen to turn water to ice, any block placed about 3-5 blocks away from the ice also turns to ice, be it wood, stone, dirt, whatever

Fixed it!


[CHORE]: Readme typo's and corrected link.

[ ] Tested on local & docker
[x] Documentation Updated
[ ] Attached required context / screenshot

Thank you @s-r-aman , Looks good to me.


Error handling on data models

I didn't find any examples of error handling on data models. Basically when ever some error happens in data model I would like to notice actual widget about that. eg. show some specified error on widget when error occurs.
Any pointers for this?

Hi,
This might be related to something I was also trying to do; that is, to add some special widget content under special circumstances. The idea is that I would have the ability to show/hide that div content on the fly. See :
  https://github.com/DataTorrent/malhar-angular-dashboard/issues/112

In the meantime, I would just add a  within the main dashboard.html template - but then you would need a way to show/hide that div when needed (i.e. Angular's ng-if or ng-show).
ex/
 {{myError}}
I hope this helps a bit...
Bob

Hi @tarlepp,
I'm not sure which data models you're referring to.
Assuming you are polling using an API, then that service should handle any network/server errors and update the data models according.  Then the binding between the data model and the html should do it's magic to notify you of errors.
If you have some code whether in a branch or plunker/jsfiddle, I'd be happy to review it with you.
Regards,
Willet

@tarlepp I apologize for responding for late.  I'm going through all the open issues (a few every month).  At the current rate, we should either respond or resolve all open issues in a couple of months.
I'm glad to hear you found a solution for your situation.  This is an open source project, so feel free to contribute if you feel someone else will benefit from this feature.
Regards,
Willet


Component demo copy code button provides no SR feedback when actioned

Summary
When interacting with component demos using a SR(NVDA), no confirmation is announced when the button is clicked. When it is clicked a tooltip appears on screen saying 'Code copied to clipboard'. This content should be announced by a screen reader.
💬 Description
Make tooltip content annouceable when a copy code button is clicked so that SR users receive some feedback confirmation on their action.
💰 Use value
Providing feedback after a user action is important as it lets the user know if their action was performed correctly or not. This principle should be the same for SRs too.
📝 Acceptance Criteria
If relevant, describe in full detail the different interactions and edge cases that the component or patterns needs to fulfil.
Given that a user is navigating through a code guidance page using a SR
When  focussing and triggering a copy code button
Then the SR should announce the contents on the tooltip that appears visually reading 'Code copied to clipboard.'
✏ Designs
If there's a Figma design file (or other mock-up), include it here.
## 🧾 Guidance
If there's written guidance or documentation, include a link to it here.
Additional info
Tell us anything else useful to help us understand your suggestion.

@DBD324  we agreed in refinement that we should not be using a tooltip to confirm the action of a button.
Would you agree that a Toast would be more appropriate? (as it would provide built in screenreader confirmation)


Change  to

Directions in the comment of the snippet indicate before the closing head tag.

Hi @SvenAelterman, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no contribution license agreement is required at this point. Real humans will now evaluate your PR.

TTYL, AZPRBOT;

@SvenAelterman : Thanks for your contribution to the Azure documentation! The author, @alancameronwills, has been notified to review your proposed change.

Thanks Sven!
#sign-off


Provides a JupyterDisplay protocol package?

Seems we are the only two people use Swift with notebooks. Anyway, I think we need to collaborate on a new header-like library called JupyterDisplay. I have a prototype: https://github.com/liuliu/swift-jupyter/blob/main/display/JupyterDisplay.swift
It is really simple. The idea is that any other libraries can depend on this library and provide notebook rich display support. If you looked at Python packages, many of them "battery-included" for notebook displays either inside the main package or in the support package.
So far, the EnableJupyterDisplay.swift implementation took hands into itself and provides backends for matplotlib or Python or SwiftPlot. That is really limited.
On the technical side, this package simply exposes a JupyterDisplay instance that EnableJupyterDisplay.swift can switch to a different implementation, thus, enabling the said behavior.
Let me know what do you think. I mainly want to propose moving this into a "neutral" org and package it lightly (like package to SwiftPM and Bazel) and everyone can use.

This sounds like a good idea. I can look into it when I get around to the next major update to Swift-Colab. But probably for the next few months, I’ll be occupied with the Metal/GPU side of my project. We will probably need to wait until changes can be made to Swift-Colab, because Jupyter display stuff is deeply integrated into the kernel. I removed the EnableJupyterDisplay file a long time ago and only EnableIPythonDisplay is maintained. That demonstrates the crux of the issue here, lack of local Jupyter notebook support.

I saw that your fork supports local Jupyter notebooks, a feature I would like to integrate into Swift-Colab via JupyterLab (Swift-CoJupyterLab). For example, I would like to use my upcoming Metal backend for S4TF in a familiar Colab-like environment. I made some highly optimized SwiftPM caching stuff for Colab, which would fit perfectly into JupyterLab like a Swift alternative to Python pip.
Could you provide a quick explanation of what’s been going on with the swift-jupyter fork lately? What bug fixes, changes to the README, etc. have you added? I’d like to help your contributions reach a larger audience, and I can leverage the large traffic Swift-Colab gets (48 vs 9 stars). We can start off with a link in my README directed to your fork, but eventually I might merge your work in entirely into Swift-Colab 3.0.

Instead of a Swift package dependency, this library could be a system module imported like Apple’s Foundation library. You don’t compile that with SwiftPM; it just comes with the toolchain. Same thing with this. In v2.3, I’m planning a way to link modules into each other like they’re system modules, enabling the compilation of https://github.com/s4tf/models. This idea can be repurposed for the package you propose, and I think it will be easier to maintain that way.

Are you using Bazel for internal development too? Curious why it is slated to v2.3.

I have only ever used Bazel to compile tensorflow/tensorflow, but not to compile Swift code. I am not very familiar with it and do not use it internally. I wanted to put it v2.3 because that's less difficult than adding local notebook support. I was also going to change the inline SwiftPM package spec parser, supporting multiple types of syntax:
'.package(url: "https://...", .branch("main")' // 4.2 style, currently only supported
'.package(url: "https://...", branch: "main")' // 5.0 style, coming in v2.3
'.bazel(url: "https://...", some other arguments)' // coming in v2.3

One thing I liked yours is the independency LLDB integration without Python.

That was one of the big reasons why I rewrote Swift-Colab to create v2.0. Development toolchains kept crashing, and I didn't know why. I assumed that only release toolchains could be compatible with Swift-Colab. Apparently, they were compatible but I had to rewrite the LLDB dependency from scratch.  That was a good choice, because I think Swift 5.6 dropped the Python bindings.

There are definitely quite a bit more weirdness with Bazel than SwiftPM

From my experience, SwiftPM is quite buggy. I reported two bugs on the swift-package-manager repo, along with an issue in swift-colab.

https://github.com/apple/swift-package-manager/issues/5505
https://github.com/apple/swift-package-manager/issues/5565
https://github.com/philipturner/swift-colab/issues/14

Working with buggy build systems is nothing new to me, but I am concerned about immediate download time. Google Colab has an incredibly fast network bandwidth, so downloading Swift toolchains is fine. I found that downloading via apt-get can take a few seconds, as can compiling a Swift package like swift-crypto. Any optional delay is unacceptable in Colab, so I will lazily install Bazel via apt the moment it's requested.

Default to optimized version of the final .so library for importing, this helps quite a bit for Swift dependencies that is computationally-heavy;

I want to adapt the mindset of "zero tolerance for latency" to local notebook support, which should create the best possible user experience. For example, users should not need to compile LLDB Python bindings themselves; that takes minutes. The SwiftPM engine is really powerful in this respect, because it compiles in debug mode by default. Bazel should not compile in fully-optimized mode by default, because that increases compile time. However, we should provide a well-documented mechanism to enable release-mode compilation with Bazel.

Is there any chance we can translate SwiftPM flags into Bazel build options? For example, -c release could translate into fully-optimized .so/.dylib/.dll binaries. I would like to reuse %install-swiftpm-flags for both SwiftPM and Bazel. The alternative is adding a %install-bazel-flags command, but:

The user might not know Bazel well.
The user might add -c release to the SwiftPM flags, but forget to modify how Bazel compiles.
The user every SwiftPM builds being in release mode, while every Bazel build is in debug mode.

When I designed Swift-Colab, I thought from the end-user's experience more than anything else. They are often impatient and don't know as much about SwiftPM/command-line stuff as we do. But I also can't change the semantic purpose of certain features just for the sake of ergonomics. This is why I did not include the Python runtime-restart code in template notebooks, while still mentioning it on the README.

This differs form your fork's new %bazel magic command. I avoid adding magic commands unless absolutely necessary, because it breaks backward compatibility if you change your mind and want to rename it. But that also goes for how you process arguments to magic commands, hence I must retain support for Swift 4.2-style manifests.

I can go more detail into how I use Bazel, and in general how it works if you want. Basically, it is unlike SwiftPM which has a vibrant GitHub presence that you can simply include a URL and pulling in dependencies. However, if it great if you have a projects that has mixed-language dependencies, such as mine, which mixed C libraries, Python libraries and tied together into a Swift binary (or notebook).

That was a good choice, because I think Swift 5.6 dropped the Python bindings.

TBH, this is probably an oversight. Last time this happened for 5.4, I pinged in #swift channel and the CI were fixed, so we have it in 5.5 :)

From my experience, SwiftPM is quite buggy. I reported two bugs on the swift-package-manager repo, along with an issue in swift-colab.

Ha, nothing is easy then. I suspect REPL mode is used by very few especially the library support. But hopefully Swift Playground on various platforms can help this.

I want to adapt the mindset of "zero tolerance for latency" to local notebook support, which should create the best possible user experience.

Yeah, at that point, probably have a common cache for people to mount :) TBH, the speed improvements when linking to optimized version v.s. debug version is light-and-day.

Is there any chance we can translate SwiftPM flags into Bazel build options? For example, -c release could translate into fully-optimized .so/.dylib/.dll binaries. I would like to reuse %install-swiftpm-flags for both SwiftPM and Bazel. The alternative is adding a %install-bazel-flags command, but:

Yeah, Bazel have 3 build modes for all its supported languages by default (that includes Rust, Go, C, C++ and Swift): debug, fastbuild and opt, you can switch between them with --compilation_mode=opt for example. The only difference between debug and fastbuild is that latter strips debug symbols so it is leaner.

Balancing theoretical correctness with ergonomics led to some other wierd decisions previously.

Agreed. Hence I asked why Bazel. It is not exactly a popular choice. However, it does give you some advantages, such as setup a consistent environment.
Now, if you want a quick starter on Bazel as a build system for Swift code:
Your repo needs to have a WORKSPACE file at the root, it contains references to other repos, including repos that provides Bazel rules (more on this later).
You specify the code to build with BUILD file in directories, and use Python-like language (it is imperative, but looks declarative) to compile, for example:
swift_library(
  name = "SomePackageName",
  srcs = ["some.swift", "package.swift"],
  deps = ["OtherPackages"]
)

You can build, or run (if it is a executable target) with bazel command-line:
bazel run //Directory1/Directory2:TargetName

Here is the thing that probably doesn't map well mentally between Bazel / SwiftPM: SwiftPM has only one Package.swift file, Bazel can have as many BUILD files as you can in one repo. That also means it needs to be efficient to reference dependencies. Thus, all dependencies from other repos (or places), needs to be first defined in WORKSPACE and reference through the syntax like @RepoName//Directory:TargetName. The RepoName is what you defined in WORKSPACE. That explains the @PythonKit//:PythonKit I referenced so much about in the notebook.
Minor but often important, you can put some bazel build parameters in .bazelrc file to avoid typing it every time when bazel run or bazel build.
Here is what my setup looks like when using Bazel + swift-jupyter:

A new Git repo, such as https://github.com/liuliu/sim2real;
Create WORKSPACE file (https://github.com/liuliu/sim2real/blob/main/WORKSPACE), reference to rules_swift, rules_python, and other dependencies (for example, in this repo, I have SwiftAtomics, SwiftNIO, swift-mujoco, PythonKit, s4nnc to name a few);
Because many of these dependencies doesn't have Bazel BUILD files (you can notice that by the prefix new_git_repository, rather than git_repository), I have to provide these in-tree under ./external/ directory;
Only after above, I can put BUILD file for launch jupyterlab in place https://github.com/liuliu/sim2real/blob/main/lab/BUILD#L3 and call bazel run //lab:bootup -- --notebook-dir $PWD;

Once setup, this is great for me. However, as you can see, it differs from mental model when operating with SwiftPM. Most notably: with SwiftPM, the notebook can operate almost independently, it doesn't need a repo with WORKSPACE file.
Although you can, in theory, just create a temporary directory with WORKSPACE file and host the notebook there. It is actually not ideal for me, because I need to reference to libraries from within the repo to use in the notebook. If I made a modifications to the library, I just need to restart the kernel. If I use SwiftPM setup (or go with separate WORKSPACE route), I have to setup these libraries in a separate repo and push a new commit to use.
All these, not to mention that not many Swift projects have no Bazel BUILD files, and the support to directly use SwiftPM Package.swift through https://github.com/cgrindel/rules_spm is not well-tested for non-Apple platforms.

The only difference between debug and fastbuild is that latter strips debug symbols so it is leaner.

That's basically -c release -Xswiftc -Onone, which is debug mode without debug symbols. That shortens S4TF's compile time from 3 to 2 minutes, and I tried really hard to get that build option supported in a Colab notebook.

I linked your repository in the README - would you mind reading my additions (+ drop-down) and telling me if there's anything that can be written better? The new text appears between the Table of Contents and Getting Started.

Thank you! It reads great!

I'd prefer if we interface with libraries using virtual function calls. For example, a top-level function could take an any JupyterDisplayProtocol as a parameter. The library would implement the header that defines the protocol, but the notebook would instantiate the conforming instance. This is similar to how I'm implementing Metal and OpenCL backends for S4TF, using any PluggableDevice as the main unit of computation.
For example, we might extend the JupyterDisplay protocol to something that can submit streamed images for video processing.
// Encoding happens behind the scenes; the calling library submits uncompressed frames.
// Always let the user decide the codec.
enum JupyterVideoCodec {
  case motionJPEG
  case automatic // backend chooses the fastest codec
  // case mpeg4 - might be added in the future
}

struct JupyterVideoConfig {
  var width: Int
  var height: Int
  var codec: JupyterVideoCodec
  // Best practice: rendering library intercepts this variable and sets the timestamp.
  var timestamp: Double?
}

protocol JupyterDisplayProtocol {
  ...
  /*static*/ func submitVideoFrame(bytes: Data, config: JupyterVideoConfig)

  // Perhaps also a feature for accepting keyboard/mouse input?
  /*static*/ func addEventListener(_ body: @escaping (???) -> Void)
  /*static*/ func setEventListener(for event: ???, _ body: @escaping (???) -> Void)
}

A library function for rendering might do this:
import JupyterDisplayModule // pre-installed into the system

func renderGLVideo(display: any JupyterDisplayProtocol, config: JupyterVideoConfig) {
  var receivedEvent = false
  display.addEventListender { receivedEvent = true }

  // Loop ends when the user manually interrupts cell execution, so the program should save game simulation progress accordingly.
  while true {
    usleep(16_000) // Interrupt might stop the program here.
    if receivedEvent { /*Do something*/ }

    // Base its size on the video config's `width` and `height`.
    let rawFrameBuffer = Data(...)

    var currentFrameConfig = config
    currentFrameConfig.timestamp = ... // Optional; just enhances encoder's ability to synchronize frames.
    display.submitVideoFrame(bytes: rawFrameBuffer, config: currentFrameConfig)
  }
}

Usage in a Jupyter notebook:
import SomeLibrary
setupSimulation()
// Allow interop between IPythonDisplay <-> JupyterDisplay either by:
// (a) Making them typealiases of each other.
// (b) Using conditional compilation for Colab vs. local notebook.
// (c) Retaining `IPythonDisplay` for backward-compatibility, but introducing a new
//     `JupyterDisplay` type to Swift-Colab.

let config = JupyterDisplayConfig(
  width: 720,
  height: 1280,
  codec: .motionJPEG)
// Several possibilities for how to input the local display variable:
renderGLVideo(display: IPythonDisplay.current, config: config)
renderGLVideo(display: JupyterDisplay.current, config: config)
renderGLVideo(display: jupyterDisplay, config: config)
renderGLVideo(display: JupyterDisplay.self, config: config)

// Alternatively, the user can render frames with custom code:
while true {
  usleep(...)
  JupyterDisplay.current.submitVideoFrame(...)
}

Yes. The inspiration is mostly from Python side, where things like matplotlib or tdqm can identify Jupyter environment automagically and act accordingly.
Having explicit pass in from library point view is great. But I maintain that also provides a singleton access would enable that "automagic" feeling for library users (they don't need to pass in anything to get the benefit).
Thinking about the tdqm use cases, if we have a Swift equivalent:
for i in tdqm(someCollection) {
}

that can render progress bar in Jupyter correctly, it is automagic. If we require a pass in:
for i in tdqm(someCollection, display: IPythonDisplay.current) {
}

it is less so magical, but OK. However, if we wrap it one layer deeper:
final class SomeTrainer {
  public func train() {
    for i in tdqm(data) {
    }
  }
}

let trainer = SomeTrainer()
trainer.fit()

comparing with:
final class SomeTrainer {
  public func train(display: any JupyterDisplayProtocol) {
    for i in tdqm(data, display: display) {
    }
  }
}

let trainer = SomeTrainer()
trainer.fit(display: IPythonDisplay.current)

Much more needle to threading through now.
I think for libraries that require such rigor, having dependency passed in is great. But for libraries that don't, have an escape hatch (a global singleton access) would be good too.

In that case, I could just create a system library with a concrete type called JupyterDisplayManager or JupyterContext. Either a package or the notebook itself can import the dependency, and everyone accesses the same singleton variable. I should probably pipe the framebuffer through a file in case the package and notebook open two different .so instances, or to offload serialization/networking to a background process. I'll have to find some good way to set settings like resolution, if that's even needed.
We may not need a protocol to encapsulate the functionality because there's one concrete type. Would you be fine if your current rendering notebooks are not backward-compatible, and have to be written after the final iteration of this design?
Also, I should just typealias JupyterDisplay == IPythonDisplay in Colab and IPythonDisplay == JupyterDisplay in local notebooks so that notebooks run everywhere, despite the fragmented history. That necessitates creating something new for the functionality we'll add, hence alternative names like JupyterContext.

I think you focused too much on streaming aspect of it :) The proposal for protocolize it is to encapsulate notebook differences, if there are any, when displaying images or text or htmls. For example, the concrete implementation in mine EnableJupyterDisplay.swift involves format the message and pipe it through stdout, it will then be retrieved by the Swift kernel and send to the notebook. It may not be the same for your implementation or IPythonDisplay.

Then we can just add static methods to the JupyterDisplay type object from the old EnableJupyterDisplay.swift, then do typealias IPythonDisplay == JupyterDisplay. That solves API differences between Colab and local notebooks, and it's even source-compatible with your existing Jupyter notebooks for simulation.
We already have:
https://github.com/philipturner/swift-colab/blob/405e27197757f545457fb0bd8e049666ffd6b40c/Sources/include/EnableIPythonDisplay.swift#L81-L93
How about we change your JupyterDisplay.swift to the following, then seeing whether your test notebooks still work? You may have to fill in empty methods with the actual implementation logic.

Click to expand
/// A display manager for Jupyter notebook.
///
/// This provides necessary methods for other Swift libraries to introspect whether you can
/// display from within a Jupyter notebook, and methods to send HTML, images or texts over
/// to the notebook.
public enum JupyterDisplay {
  /// flush display messages to the notebook immediately. By default, these messages are
  /// processed at the end of cell execution.
  public func flush() {}
  /// Display a base64 encoded image in the notebook.
  public func display(base64EncodedPNG: String, metadata: String?) {}
  /// Display a snippet of HTML in the notebook.
  public func display(html: String, metadata: String?) {}
  /// Display plain text in the notebook.
  public func display(text: String, metadata: String?) {}
  /// Check whether the notebook is active and accepting display messages.
  public private(set) var isEnabled: Bool = false
  /// Get the cell number.
  public var executionCount: Int { 0 /*TODO: Return actual cell count*/ }

  public func enable() {
    // TODO: Some other logic
    JupyterDisplay.isEnabled = true
  }
}

extension JupyterDisplay {
  /// Display a base64 encoded image in the notebook.
  public func display(base64EncodedPNG: String) {
    display(base64EncodedPNG: base64EncodedPNG, metadata: nil)
  }
  /// Display a snippet of HTML in the notebook.
  public func display(html: String) {
    display(html: html, metadata: nil)
  }
  /// Display plain text in the notebook.
  public func display(text: String) {
    display(text: text, metadata: nil)
  }
}

public typealias IPythonDisplay = JupyterDisplay

Then %include JupyterDisplay.swift becomes:
import JupyterDisplay

extension JupyterDisplay {
  static func foo() {} /* methods for communicating with the kernel */
}

JupyterDisplay.enable()

Also, would you mind reimplementing your logic for HTTP streaming using Python libraries, with PythonKit installed as a system library for all conformant packages? Then source code would work across both Colab and local notebooks, swapping the two different implementations at runtime. You could try different build configurations if this doesn't work out verbatim.
#if canImport(PythonKit)
let jpegDependency1 = try? Python.attemptImport("jpegDependency1")
#elseif canImport(SwiftNIO)
...
#endif

Also, unify IPythonDisplay.swift and JupyterDisplay.swift by parsing the filename before executing the %include command. If it matches IPythonDisplay.swift, redirect it to JupyterDisplay.swift. I can easily change Swift-Colab to use a new file for IPythonDisplay.

We may talk about slightly different things. I am not interested in how the file we use for %include "EnableJupyterDisplay.swift" was written. I am interested in having a "protocol" library that other libraries can depend on to provide display support. For example, now, the MuJoCoViewer.swift inside my Gym package https://github.com/liuliu/s4nnc/blob/main/gym/renders/MuJoCoViewer.swift#L32 can detect whether Jupyter notebook is on or not and choose to display differently.
Thus, when you use it in the notebook, it looks like this:
import Gym

%include "EnableJupyterDisplay.swift"

let viewer = MuJoCoViewer()

viewer.render()

If it is not in notebook, this will create a window and render a image. If it is in notebook, this will pipe an image through the notebook.
To implement this kind of support, JupyterDisplay token has to be a global instance, rather than a enum. Like you said, it is possible still to not provide the global instance, and just pass the instance in:
viewer.render(with: IPythonDisplay.current)

In this way, we still provides a protocol, but library doesn't need to talk to the global instance, and feels "better engineered". As I elaborated earlier, this may not feel as magical as Python, so it is a balanced act.

So what you're saying is:
That package has a mutable variable called JupyterDisplay. By default, it's an instance that doesn't do anything. At runtime, the file EnableJupyterDisplay.swift will mutate the module's variable, changing it to something that is enabled. For example:
// EnableJupyterDisplay.swift
import JupyterDisplay

// Renamed the existing `JupyterDisplay` type to `_JupyterDisplay`
enum _JupyterDisplay {
  ... // already existing code
}
extension _JupyterDisplay: JupyterDisplayProtocol {}
JupyterDisplay.JupyterDisplay = // something that delegates to `_JupyterDisplay`

// EnableIPythonDisplay.swift
import JupyterDisplay

enum IPythonDisplay {
  ... // already existing code
}
extension IPythonDisplay: JupyterDisplayProtocol {}
JupyterDisplay.JupyterDisplay = // something that delegates to `IPythonDisplay`

That would be very flexible, although I'm not sure we should expose a global variable like that. Perhaps we at least make it immutable at face-value, but provide an API to mutate it. For example:
extension JupyterDisplayProtocol {
  public func _setJupyterDisplayInstance(_ instance: any JupyterDisplayProtocol) {
    // Optionally do some cleanup on the existing instance.
    // Optionally do some validation on the input instance.
    JupyterDisplay = instance
  }
}

I think I'm getting it now, but I'd like to consider alternative approaches as well. For example, we could substitute a protocol with an API that just swaps out function pointers, which a static JupyterDisplay type object calls into. Capitalizing something and then making it a variable goes against traditional naming conventions (it should start with lowercase). If we can accomplish the same library interface while sticking more to the semantic meaning, I would like that.
For example, we could do the following:
protocol JupyterDisplayProtocol {
  // non-static function
  func display(base64EncodedPNG: ...)
}

enum JupyterDisplay {
  private var delegate: JupyterDisplayProtocol = EmptyJupyterDisplay()
  public func _setCurrentInstance(_ instance: JupyterDisplayProtocol) {
    Self.delegate = instance
  }

  // static function
  static func display(base64EncodedPNG: ...) {
    delegate.display(base64EncodedPNG: base64EncodedPNG)
  }

  static var isEnabled: Bool {
    delegate.isEnabled
  }
}

You stated above that we could make a neutral organization for hosting the Swift package. I'm not yet sure that's the best path forward. If it's specific to the Jupyter kernel and maintained by the Jupyter kernel, perhaps it should be embedded inside each Jupyter kernel's repository. Also it's an irreversible decision; if we change our mind later, people now have Swift package dependencies to a stale repository. Would you mind trying out alternative options first?
In previous comments, I had the ideas of embedding JupyterDisplay as a system library. That means the JuypterDisplay module doesn't exist in a non-Jupyter environment. You need to doubly check that the JupyterDisplay exists, and the current JupyterDisplay is enabled.
#if canImport(JupyterDisplay)
import JupyterDisplay
#endif

extension MuJoCoViewer: Renderable {
  public func render() {
    #if canImport(JupyterDisplay)
    if JupyterDisplay.isEnabled {
      // Check to see if we launched the render server yet.
      if httpChannel == nil {
        httpChannel = try? renderServer.bind(host: "<IP_ADDRESS>", port: .random(in: 10_000..<20_000))
          .wait()
      }
      if JupyterDisplay.executionCount != renderCell {
        JupyterDisplay.display(html: renderServer.html)
        JupyterDisplay.flush()
        renderCell = JupyterDisplay.executionCount
      }
    }
    #endif
    ...
  }
}

Another concern it how to detect a module's dependency to the JupyterDisplay repository. The SwiftPM engine doesn't yet have introspection capabilities. Also, if two separate libraries depend on the same JupyterDisplay repo (possibly with different package versions), which one do I import into LLDB? Or what if someone installs a Swift package after including "EnableJupyterDisplay.swift"? The file can only run once in Swift-Colab, so it blew its chance to mutate the JupyterDisplay.JupyterDisplay variable. This problem would be solved by making JupyterDisplay an internal rather than external module.

Also, why can't we use IPythonDisplay in local Jupyter notebooks? Your README said it crashes after a few dozen cell executions, but I couldn't find any more information on the crash. EnableIPythonDisplay.swift is simpler to implement than EnableJupyterDisplay.swift, which requires cryptography and compiling large external dependencies. Perhaps we could replicate the crash in Colab, and I could determine what's causing it.

I think I'm getting it now, but I'd like to consider alternative approaches as well. For example, we could substitute a protocol with an API that just swaps out function pointers, which a static JupyterDisplay type object calls into. Capitalizing something and then making it a variable goes against traditional naming conventions (it should start with lowercase). If we can accomplish the same library interface while sticking more to the semantic meaning, I would like that.

Yes, there are several ways to implement this. It is a kind of dependency injection pattern. The important thing is to expose a protocol-only interface module to other libraries and the heavy-weight dependency can be injected from notebook end. Your enum example would be an alternative way to implement the same thing.

Another concern is how to detect a module's dependency to the JupyterDisplay repository. My SwiftPM engine doesn't yet have introspection capabilities. Also, if two separate libraries depend on the same JupyterDisplay repo (possibly with different package versions), which one do I import into LLDB? Or what if someone installs a Swift package after including "EnableJupyterDisplay.swift"? The file can only run once in Swift-Colab, and it ran when the JupyterDisplay module couldn't be imported. It lost its chance to mutate the JupyterDisplay.JupyterDisplay variable. This problem would be solved by making JupyterDisplay an internal rather than external module.

That's correct, SwiftPM doesn't support system module detection, due to it is a very cross-compilation friendly nature (iOS always cross-compiled on a macOS system). That's why have a protocol-only library as the root dependency is ideal actually. The order of importing and include "EnableJupyterDisplay.swift" is not important because the injected dependency only become relevant upon use. Luckily, in Swift, there is no eager initialization for library level code, unlike Python or C++ (where you can run code when import __init__.py, or at startup for your static initializers (C++)). In Swift, static initializers are all lazy.

Also, why can't we use IPythonDisplay in local Jupyter notebooks? Your README said it crashes after a few dozen cell executions, but I couldn't find any more information on the crash. EnableIPythonDisplay.swift is simpler to implement than EnableJupyterDisplay.swift, which requires cryptography and compiling large external dependencies. Perhaps we could replicate the crash in Colab, and I could determine what's causing it.

TBH, I am a bit burned by all weirdness of Swift REPL. The last shenanigan I found is that by linking to liblinear, for loop will crash with Swift 5.6.x REPL. The EnableIPythonDisplay.swift problem with Swift 5.5.x (I haven't tried with 5.6.x) is not exactly crash. After a few cell executions, the Swift kernel will die if EnableIPythonDisplay.swift is called prior. The kernel is dead with the lldb exited with code 0. Thus, if you attach a debugger, it doesn't really show why. At that time, I guess it may due to the fact that it doesn't like a IPython kernel (launched by the EnableIPythonDisplay.swift code) connected along with a Swift kernel, but because it doesn't really "crash", I have limited way to debug.

I'm deciding to take a break from my Metal backend for S4TF, and complete Swift-Colab v2.3 now. For starters, I resolved #17. I'm also going through the documentation and slowly completing it, which I've been too lazy to do for a while. I plan to add two new magic commands:

%install-swiftpm-environment  - motivation: compiling S4TF on release toolchains when Swift 5.7 releases
%install-swiftpm-import - motivation: injecting TensorFlow into s4tf/models as if you were using a custom S4TF toolchain

The current documentation of %install-swiftpm-import is in MagicCommands.md. Behind the scenes, I'm going to prepend a call to this command for JupyterDisplay (with a special execution path), before compiling every Swift package. The user will not have to (and cannot) declare the import in the Jupyter notebook. This makes logical sense because you never exported JupyterDisplay with an %include command. Are you fine with this being how we inject a dependency to JupyterDisplay, once liuliu/swift-jupyter and philipturner/swift-colab converge on the matter?

The current documentation of %install-swiftpm-import is in MagicCommands.md. Behind the scenes, I'm going to prepend a call to this command for JupyterDisplay (with a special execution path), before compiling every Swift package. The user will not have to (and cannot) declare the import in the Jupyter notebook. This makes logical sense because you never exported JupyterDisplay with an %include command. Are you fine with this being how we inject a dependency to JupyterDisplay, once liuliu/swift-jupyter and philipturner/swift-colab converge on the matter?

I need to learn more about differences between what you called "environment" and "import". import JupyterDisplay automagically sounds good (from my understanding, this will make it simpler later when we %include EnableXXXDisplay.swift?). This change doesn't change the dependency tree for other SwiftPM packages right? (If they don't use JupyterDisplay, it is not one of their dependencies, and if they use, it doesn't matter if we import JupyterDisplay for they automagically or not?).

It definitely have tensions with how traditionally dependency-injection was done

If possible, I encourage the user to create a concrete dependency in Package.swift. This is a solution to a problem that cannot be solved any other way. When compiling S4TF, it takes 3 minutes. When compiling both s4tf/s4tf and s4tf/models, you would compile S4TF two times, because each repository makes their own copy of the TensorFlow module.
This feature augments the "clever idea" to bypass S4TF's long compile time, where you rewrite a notebook's contents without recompiling S4TF. A user would have just looked through several tutorials with their one S4TF instance compiled. Now they encounter the tutorial that runs s4tf/models, and have to wait 3 extra minutes because that GitHub repo has its own copy of s4tf/s4tf. With %install-swiftpm-import, someone could reuse the existing TensorFlow module inside s4tf/models and skip the 3 minutes of wait time.
Other than this specific use case, and JupyterDisplay, I can't see a reason someone would use the feature.

 You test that the code works by firing up Swift-Colab and running the notebook cells. This setup does not permit running swift test on any dependencies downloaded by a notebook. Instead, you copy the function bodies of each XCTestCase member into its own Jupyter cell, encapsulated in a do {} statement for safe measure. I'm oversimplifying the process, but this is the general idea.
However, I would like to add a dedicated SwiftPM testing feature. I often set certain environment variables before running SwiftPM tests, which %install-swiftpm-environment would help with.

This is a solution to a problem that cannot be solved any other way. When compiling S4TF, it takes 3 minutes. When compiling both s4tf/s4tf and s4tf/models, you would compile S4TF two times, because each repository makes their own copy of the TensorFlow module.

You probably already thought through this many times, but have you tried to just support a generic cache directive? This is often supported in CI environments to make CI jobs faster. SwiftPM have everything checked out and compiled under .build directory. Since Colab has a pretty consistent environment, you can provide a GDrive directory with compiled .o / .a files for S4TF that people can do %checkout-cache SomeGDriveLink to populate .build directory.

I recently though of using Google Drive to cache build products, but it doesn't solve the problem that necessitates %install-swiftpm-import. My SwiftPM engine allows for loading each package dynamically because they all have separate .build directories. It would cache the S4TF builds, but the user still has to wait 6 minutes on their very first time ever using Swift-Colab, instead of 3.
This is unlike the swift-jupyter or Swift-Colab pre-v2.0 mechanism, which I assume your fork is still using. With that mechanism, all packages compile into one mega-package, so no dependencies duplicate. There is one .build directory for everything combined.
Google Drive caching seems like a good analogue to local filesystem caching, because it stores data over long periods of time. I am revamping the experience of mounting Google Drives, and just bypassed a major restriction around calling google.colab.drive.mount (still only a POC). Perhaps that feature can become part of Swift-Colab v2.3, and help me plan for local notebook support.

The restriction on Google Drive was that I couldn't mount the drive from Swift code. I had to switch over to the Python Jupyter kernel, mount the drive, then switch back. I am bypassing this mechanism by forwarding stdin requests from the LLDB process to the SwiftKernel process and back - through a custom piping mechanism. This removed the need to cache and asynchronously execute send_multipart's in KernelCommunicator, thus I may delete the kernel communicator soon. Would your fork be able to remove the kernel communicator and directly copy image data into a Jupyter display?
For reference, here is the serialization mechanism. Encode 8 bytes, which are an Int64 containing how large the payload is. Then, encode the payload. The recipient deserializes the data and moves the file's cursor upward. At the end of each Jupyter cell, delete and recreate the files that facilitate inter-process communication. Otherwise, they may grow infinitely large.

I recently though of using Google Drive to cache build products, but it doesn't solve the problem that necessitates %install-swiftpm-import. My SwiftPM engine allows for loading each package dynamically because they all have separate .build directories. It would cache the S4TF builds, but the user still has to wait 6 minutes on their very first time ever using Swift-Colab, instead of 3.

I was actually talking about provide a public accessible cache, so whether it is compiled separately or compiled into a megapackage will be the same. And for first-time user, it will be as fast because you pull that from public cache everyone uses. Whether it is from GDrive or a S3 bucket is an implementation detail.

I am bypassing this mechanism by forwarding stdin requests from the LLDB process to the SwiftKernel process and back - through a custom piping mechanism. This removed the need to cache and asynchronously execute send_multipart's in KernelCommunicator, thus I may delete the kernel communicator soon. Would your fork be able to remove the kernel communicator and directly copy image data into a Jupyter display?

Similar, I haven't removed the KernelCommunicator method yet, as it seems more efficient (avoid formatting and copying). But I do provided an alternative flush mechanism to enable send display messages to notebook in the middle of execution. The implementation is similar, I wrapped the JSON payload with ---SOMEBOUNDARYSTRINGS and the stdout reader (from Swift kernel, reads LLDB process) will parse this particular stdout and treat it as a notebook message instead.

I was actually talking about provide a public accessible cache,

I don't think that's a very good idea. You would have to cache exponential variations of different exact Swift toolchain + different build flags + different version of the external Swift package. I'm not even sure you can distinguish each unique environment. If one factor is different, the .build folder could have different contents. That's why I'm so hesitant to make a public cache as described in this thread.
However, making a cache just for the environment combos found in tutorial notebooks could artificially boost performance. It would make tutorials go faster for beginners, but it doesn't augment power users' workflows. In other words, it's optimizing for benchmark performance.

Similar, I haven't removed the KernelCommunicator method yet, as it seems more efficient (avoid formatting and copying).

KernelCommunicator pipes the data through the file system, which has the same overhead as the new mechanism I propose (which also pipes). I think it also serializes the data before copying, which is what my new mechanism does. KernelCommunicator got much more painful when I had to marshal the data through LLDB's C++ API, rather than its Python API. I had to make a recursive serialization protocol just for one Swift <-> C++ bridging function, which used exponentially expanding buffers while encoding.

I don't think that's a very good idea. You would have to cache exponential variations of different exact Swift toolchain + different build flags + different version of the external Swift package. I'm not even sure you can distinguish each unique environment. If one factor is different, the .build folder could have different contents. That's why I'm so hesitant to make a public cache as described in this thread.

Depends on how SwiftPM's .build directory structured, I didn't read too much into their code to know how they keep these directories up-to-date. Sharing cache for Bazel is safe and efficient (due to content hashing). If the cache is on object / library level, it will benefit both beginners and power users because the later can still reuse big chunk of compiled code. Anyway, SwiftPM is out of my expertise, just want to clarify some of these claims might be build system specific, not an inherited limitation of doing public read-only cache itself.

KernelCommunicator pipes the data through the file system, which has the same overhead as the new mechanism I propose (which also pipes). I think it also serializes the data before copying, which is what my new mechanism does. KernelCommunicator got painful when I had to marshal the data through LLDB's C++ API, rather than its Python API. I had to make a recursive serialization protocol just for one Swift <-> C++ bridging function, which used exponentially expanding buffers while encoding - written in straight C.

Hmm, that doesn't match what I saw in KernelCommunicator.swift inherited from google/swift-jupyter (https://github.com/google/swift-jupyter/blob/main/KernelCommunicator.swift#L60)? For that file, it serializes into JSON and pass the pointer of that serialized JSON back to LLDB. In LLDB, it can use read memory method to read it back. By saving one memcpy, I meant that. But I guess it may not worth it though.

In LLDB, it can use read memory method to read it back

I was thinking that the LLDB process is a separate system-level process, which you spawn and communicate with over pipes.  That means you could access stuff stored in raw memory. However, the lifetime overhead of each data segment is probably dominated by disk writing/piping in kernel.iopub_socket.send_multipart. The worst I could do is double the overhead (roughly), but that's outweighed by having much greater flexibility.

I was thinking that the LLDB process is a separate system-level process, which you spawn and communicate with over pipes.  That means you could access stuff stored in raw memory. However, the lifetime overhead of each data segment is probably dominated by disk writing/piping in kernel.iopub_socket.send_multipart. The worst I could do is double the overhead (roughly), but that's outweighed by having much greater flexibility.

Yeah, once it is to iopub_socket.send_multipart, I am oblivious what kind of communication mechanism they use (I think they use ZeroMQ encoded named pipes, but whatever). I am more interested in how we can get from notebook code (in Swift) to the kernel (in my case, Python, in your case, Swift with PythonKit). Currently I am happy with my ad-hoc stdout mechanism, but definitely can learn if there are better ways.

I plan to add a %test directive in this round of changes to Swift-Colab. The documentation link is here. It allows running SwiftPM tests, but you can't specify any [PRODUCT ...] to export. The test build products will be cached between executions in a containerized folder. Let me know what you think!
I should have explained this, but I like to hop around between components of the S4TF project. It's so massive in scope, that I can get bored of one part and move on to another. When I work on Swift-Colab, I usually spend a week or two making several disparate changes, culminating in a minor release. The changes I'm discussing in this thread are achievable within that time frame, but local notebook support is not. Thus, local notebook support will come in the round after this, circa early 2023.
I'm also simultaneously building SwiftOpenCL (where I reused the dynamic loading mechanism from PythonKit), patching up s4tf/s4tf, and creating the Metal backend for S4TF. More info here, which is linked on my GitHub profile.


Add support for "inline queries"

Blocked on:

https://github.com/whosonfirst/go-whosonfirst-index/issues/12
https://github.com/whosonfirst/go-whosonfirst-sqlite-features-index/issues/6

Available as of v0.3.3

Available as of v0.3.3


Add fix, enhancements and pytests for CLI

~Fixes | Closes | Resolves #~ Related to https://github.com/Quansight/qhub/issues/1478

Please remove anything marked as optional that you don't need to fill in.
Choose one of the keywords preceding to refer to the issue this PR solves, followed by the issue number (e.g Fixes # 666).
If there is no issue, remove the line. Remove this note after reading.

Changes introduced in this PR:

Types of changes
What types of changes does your PR introduce?
Put an x in the boxes that apply

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds a feature)
[ ] Breaking change (fix or feature that would cause existing features to not work as expected)
[ ] Documentation Update
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no API changes)
[ ] Build related changes
[ ] Other (please describe):

Testing
Requires testing

[x] Yes
[ ] No

In case you checked yes, did you write tests?

[x] Yes
[ ] No

Documentation
Does your contribution include breaking changes or deprecations?
If so have you updated the documentation?

[x] Yes, docstrings
[ ] Yes, main documentation
[ ] Yes, deprecation notices

Further comments (optional)
If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered and more.

Blocked by #1501

@iameskild is this ready for review?

@trallard @viniciusdc @aktech this PR is ready for review.
We currently still have the old CLI in the codebase, it think we can remove it in another PR, what are your thoughts?

silly request here, but can we modify the name of this test to PyTest & Formatting? It took me a while to remember where pytest was running...
https://github.com/nebari-dev/nebari/actions/runs/3336080683/workflow#L30

@viniciusdc I ended up just removing the black and flake8 steps since both of these are being called in the pre-commit action. How does that work for you?


fixed sample XML formatting

@alfredmyers,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by Microsoft. We will now review your pull request.
Thanks,
Microsoft Pull Request Bot

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/code-quality/ca0001.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Thanks!


Use async flag in isos post

My understanding:

It is task for other tools to review outcome of testing.
Release Managers shouldn't be required to look to sync status in addition.

So, even if the gru failed to start some tests - the sync is complete from ObsRsync point of view and release tools should review that all required tests are started and have passed properly.

My understanding:
* It is task for other tools to review outcome of testing.

* Release Managers shouldn't be required to look to sync status in addition.

So, even if the gru failed to start some tests - the sync is complete from ObsRsync point of view and release tools should review that all required tests are started and have passed properly. Showing failure in both places will not help much and will only create unneeded noise. I see it as kind of single responsibility here - the sync does only what is defined in xml; other components should keep responsibility for verifying outcome and completeness of tests started.

Sure. I would only be concerned with anything actually falls in the scope of openqa-trigger-from-obs then. For example let's say the API of openQA changes and openqa-trigger-from-obs fails. It would be good if the tooling would know about that.

Currently if any command during sync fails - the gru job (obs_rsync_run) will be in failed state, plus column Last failure in the table at /admin/obs_rsync references to it and the details should be picked up in the logs.
So if API changed - I expect isos post?async=1 will return an error and it will be reflected.
Not sure what kind of additional indication will be helpful.

So if API changed - I expect isos post?async=1 will return an error and it will be reflected as described above.

I'd expect the same. It was already bad if we'd break backwards compatibility but breaking it in a non-detectable way would be even worse (so I suppose it won't happen).


Not able to create service with scale  more than 10 .

Currently UI allows for creating a service with scale set to a max of 10. We should allow for creating services with scale > 10.

If I create a service through rancher-compose with a scale greater than 11, In the edit UI it shows up as scale : 11.

Verified with UI v0.59.0


For Each activity that references the output of the previously defined Power Query for its Items property

You can also use the output of your Power Query activity as inputs to other activities. Here is an example of a For Each activity that references the output of the previously defined Power Query for its Items property. Its Items support dynamic content, where you can reference any outputs from the Power Query used as its input.
[Enter feedback here]  -- There is no output of items like Lookup activity to Iterate the power query data.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: d5b463b1-36a5-baa8-1085-d72fea129f9c
Version Independent ID: 144bf021-a61e-ee70-4f10-3a9da430b069
Content: Power Query activity in Azure Data Factory - Azure Data Factory
Content Source: articles/data-factory/control-flow-power-query-activity.md
Service: data-factory
Sub-service: data-flows
GitHub Login: @kromerm
Microsoft Alias: makromer

I am unable to attach file or screenshot to the issues

@snukalagit
Thanks for your feedback! We will investigate and update as appropriate.

@snukalagit
The error message suggests that the input to the length function is not an array or a string, but an object. This could happen if the output of your Power Query activity is not in the expected format.To resolve this issue, you can check the output of your Power Query activity and make sure it is in the expected format. You can also check the input to the length function and make sure it is an array or a string.
If you are still having trouble, I'd recommend you create a support ticket since the support team will be able to respond much more quickly and have a conversation with you to figure out what could be going on.


Feature - Allow system environment fallback in get() #742

I've changed the proposed name in the original issue from fallback_to_environ to sysenv_fallback to stress that those are system environment variables, but it is totally fine to switch it back or to another name.
Also, I had some doubts on dealing with mixed casing, because windows seems to uppercase all environment variables and linux don't. To be safe, I chose to implement an uppercased-lookup of system variables, even though linux would support matching mixed-case.

I'll see if I can do it

Looks like we are already using the proper version of the action https://github.com/dynaconf/dynaconf/blob/4dcfc79a4f19ea9b36b78f76dc9668bae5d7124e/.github/workflows/main.yml#L92-L98

we just need to remove codecov from requirements
ERROR: No matching distribution found for codecov

https://github.com/dynaconf/dynaconf/pull/918

Codecov Report

Merging #916 (e77225f) into master (c322950) will increase coverage by 0.00%.
The diff coverage is 100.00%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@           Coverage Diff           @@
##           master     #916   +/-   ##
=======================================
  Coverage   99.72%   99.72%
=======================================
  Files          21       21
  Lines        1838     1844    +6
=======================================
+ Hits         1833     1839    +6
  Misses          5        5

Impacted Files
Coverage Δ

dynaconf/base.py
100.00% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Some OSTs are not being discovered during installation of IML

Description
Some of our OSTs are not being located during the IML installation process.
Each of our eight OSS servers have five OSTs, for a total of 40 OSTs.
We have two OSS servers where none of their OSTs are discovered.
Repro
We've tried:

Complete tear-down and re-installation of IML.

We originally had three OSS servers where none of their OSTs were discovered. During the re-installation, one of the originally missing OSS servers had it's OSTs discovered and added successfully.

Remove and re-add of the two problematic OSS servers.

This threw the same "AssertionError" error in the plugin_runner.log file.
Version
What version of IML?
iml-gui-6.4.0-827.1.el7.noarch
python2-iml-manager-cli-<IP_ADDRESS>-5131.30.g054a091.el7.noarch
iml-old-gui-3.0.0-40.el7.noarch
iml-update-handler-1.0.0-24.12.gb359059.el7.noarch
iml_sos_plugin-2.2.2-1.el7.noarch
python2-iml-manager-<IP_ADDRESS>-5131.30.g054a091.el7.noarch
python2-iml-manager-libs-<IP_ADDRESS>-5131.30.g054a091.el7.noarch
iml-view-server-8.0.2-142.el7.noarch
iml-realtime-7.0.0-85.el7.x86_64
iml-socket-worker-4.0.0-98.el7.noarch
iml-online-help-2.5.1-1.el7.noarch
python2-iml-common1.4-1.4.4-43.2.el7.noarch
iml-device-scanner-aggregator-2.2.0-198.el7.x86_64
iml-srcmap-reverse-3.0.7-1.el7.noarch
What Operating System?
What Operating System version?
$ cat /etc/redhat-release
CentOS Linux release 7.5.1804 (Core)
$ uname -a
Linux exaiml 3.10.0-862.el7.x86_64 #1 SMP Fri Apr 20 16:44:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
iml-diagnostics
Attached as sosreport-exaiml-20181017092030.tar.xz.gz

Figured we could add a little more background which may be helpful. Our site originally ran the IEEL <IP_ADDRESS> version of lustre. We used the version of IML bundled with this software to create a HA setup enabled between our MDS and MGS. Our OSS servers are ZFS-backed, so we left those out of the HA setup and created the OSTs by hand. Support for ZFS-backed OSSes in HA configuration was not supported in this earlier version of IML.
We've recently upgraded our Lustre file system to 2.10.5-1 on our MDS, MGS and OSSes. Instead of a nuke and re-pave, we were directed to try and upgrade the file system while saving the existing data in lustre. This we were able to do.
So now we are trying to get all of our lustre servers as "monitored storage servers" into this new version of IML. The MDS and MGS are no longer added as "managed storage servers".
I do notice the following errors in the plugin_runner.log on our new IML server when using "Add Server" on our two "problematic" OSS servers:
`[2018-10-17 16:07:34,518: ERROR/rpc] RunOneRpc: exception calling setup_host: Traceback (most recent call last):
File "/usr/share/chroma-manager/chroma_core/services/rpc.py", line 125, in run
'result': self.rpc._local_call(self.body['method'], *self.body['args'], **self.body['kwargs']),
File "/usr/share/chroma-manager/chroma_core/services/rpc.py", line 629, in _local_call
return fn(*args, **kwargs)
File "/usr/share/chroma-manager/chroma_core/services/plugin_runner/init.py", line 23, in setup_host
self.handlers[plugin_name].setup_host(host_id, data)
File "/usr/lib/python2.7/site-packages/django/db/transaction.py", line 224, in inner
return func(*args, **kwargs)
File "/usr/share/chroma-manager/chroma_core/services/plugin_runner/agent_daemon.py", line 89, in setup_host
assert(session is not None)
AssertionError
`

There are two other issues we have noticed, that perhaps are related?

IML dashboard is using IDs instead of OST names in the Read/Write Heat Map and OST Balance graphs

The older version was able to use the OST names. Is there method or conf file that we can use to have IML use the OST names instead?

All the OSTs are flagged as Offline

Each of our OSTs have errors associated with them; e.g.
Errror	TargetOfflineAlert	2018-10-16 16:35:45		Target lustre1-OST0000 offline Errror	TargetOfflineAlert	2018-10-16 16:35:45		Target lustre1-OST0001 offline ... Errror	TargetOfflineAlert	2018-10-16 16:35:45		Target lustre1-OST001d offline


UI state does not persist during reload

Haven't seen this in a while. Might've been a transient issue. Will reopen if it comes back.


Show pagination/infinite scroll in places page

Add pagination feature in /places page.

This might be later prioritized and implemented, after the places have been filtered to show only the verified and published places. See #33
Then we can finally have the pagination.
Because at the moment we still only have less than 10-20 places, which is still fine without pagination.


Restrict terraform version due to terraform BSL license change

This is a 🙋♂ feature or enhancement.

[ ] I've added tests (if it's a bug, feature or enhancement)
[ ] I've adjusted the documentation (if it's a feature or enhancement)
[ ] The test suite passes (run bundle exec rspec to verify this)

Summary
The new Terraform BSL license announcement on August 10, 2023 restricts usage of future terraform versions from being used with offerings Hashicorp deems competitive to their own solutions. Here's the Terraform license. Terraform 1.5.5 and below are unaffected.
To address this:
The Terraspace framework CLI checks and only allows Terraform 1.5.5 and below usage by default. You can bypass the check if your usage is acceptable with the BSL, but you must explicitly do so with TS_VERSION_CHECK=0
Note
You have full control over the Terraform installation since terraspace and terraform run entirely from your machine. It's just a matter of installing the Terraform version that Hashicorp considers acceptable for your use case.
There's an opentf manifesto that will likely result in a Terraform fork. We recommend using that fork once it's available.
Going forward, Terraspace will develop and test against Terraform 1.5.5 and below and future Terraform forks only. Also, even if you bypass the check, Terraspace Cloud will deny requests from newer versions of terraform regardless.
Context

Community Discussion https://community.boltops.com/t/future-of-terraspace-with-the-terraform-bsl-license-change/1080/6

How to Test
Running a terraform version that is newer than 1.5.5 and errors and exits.
Version Changes
Patch

Released in 2.2.9


[Issue #242] Web: Forcing Stanford NLP to load when starting the server

During the demo last week, we realized that any queries with the NLPExtractor operator tend to be slow the first time because Stanford NLP has to load.
This PR executes a simple query plan when the TextDB Web Server is launched, which forces Stanford NLP to load all its necessary resources.
This ensures that any requests made to the server can be handled in a timely manner without incurring the loss of performance due to Stanford NLP loading.

I left a minor clarification comment.  The rest looks good to me.  To be safe, @zuozhiw please also give a quick review.

@kishore-narendran Don't merge this PR yet, since I'll add the code to run SampleExtraction in the main() function.

@zuozhiw Would you like me to do that?

I will let @zuozhiw review and approve this PR.

LGTM

@chenlica @zuozhiw I am going to merge this now.


Allow single input for generation endpoint

According to spec inputs can be either a single string or a string array, with the result beeing either a single TextGenerationOutput or and array of them.

Hello, your PR changes the  output type but your PR title mentions the input type, is it intended?

The output type is inputtype dependent, therefore yes. If a single input is provided to tgi it will result in an single output, multiple inputs will result in multiple outputs. Therefore a change handling a single output, will allow a single input to work.

@Wauplin how does the python lib handle this? :pray:

https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/inference/_client.py#L1876

Thanks @28Smiles !
Will still wait a bit for other inputs before merging

While further investigating https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/inference/_client.py#L2197 also throws an exception if tgi returns a single object, which seems odd. Intriguingly, it was implemented like this from the get-go either its my older tgi version (v1.4.3). I am currently stuck on this tgi version and can't test newer versions.

Although TGI/InferenceAPI support batched requests, the Python client does not implement that. The inference specs also does not allow array as input. The main reason is not technical but to avoid inconsistencies in the API + potentially block us in the future if we want to add a feature that breaks the batched requests. In any case, TGI is already batching requests server-side so looping over your inputs client-side should not be causing much delay. You only lose the extra HTTP overhead but we consider it as neglictable compared to the time it takes to generate tokens.

Although TGI/InferenceAPI support batched requests, the Python client does not implement that. The inference specs also does not allow array as input. The main reason is not technical but to avoid inconsistencies in the API + potentially block us in the future if we want to add a feature that breaks the batched requests. In any case, TGI is already batching requests server-side so looping over your inputs client-side should not be causing much delay. You only lose the extra HTTP overhead but we consider it as neglictable compared to the time it takes to generate tokens.
I would therefore not proceed with this PR, at least short-term. cc @SBrandeis @osanseviero

This pr is not handling batched requests, its handling single return objects, cause tgi (at least in my version, can’t verify for others), is returning a single object instead of an array of length 1

I can also verify this for tgi v2.1.1 now.
curl -X 'POST' \
  'http://<IP_ADDRESS>:8000/generate' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "inputs": "My name is Olivier and I"
}'

Response:
{
  "generated_text": " am from Belgium. I've been playing classical guitar for almost 5 years and I am now gaining interest in Flamenco guitar. I want to take lessons in a traditional flamenco way for 3 to 6 months with a good master in Barcelona but I don't know how to get started. Could you have some recommendations about how to approach this? Maybe websites, contact details of good masters, schools, etc. I've tried contacting the 'Amics"
}

This is not an array.

Thanks for the PR and explanation @28Smiles! We double-checked internally and this change is indeed necessary to ensure consistent behavior.

Should I quickly add a pr to the python client for you aswell?

Thanks for opening it on the Python side as well!

Thanks for opening it on the Python side as well!

I am glad to be helpful


check/varfont_weight_instances: Does not explain why the check is needed

Observed behavior
The current rationale reads as follows:
The named instances on the weight axis of a variable font
must have coordinates that are multiples of 100 on the design space.

Expected behavior
We need a simple explanation of WHY this is required added to the rationale. Preferably with a link to a spec containing a further explanation.

Just curious, can you think of why we might want this?
I'm asking you this because I think there's another issue here.

Ah, yes. It's conflicting with this condition in /Lib/fontbakery/profiles/googlefonts_conditions.py:
@condition
def expected_os2_weight(style):
  """The weight name and the expected OS/2 usWeightClass value inferred from
  the style part of the font name
  The Google Font's API which serves the fonts can only serve
  the following weights values with the corresponding subfamily styles:
  250, Thin
  275, ExtraLight
  300, Light
  400, Regular
  500, Medium
  600, SemiBold
  700, Bold
  800, ExtraBold
  900, Black
  Thin is not set to 100 because of legacy Windows GDI issues:
  https://www.adobe.com/devnet/opentype/afdko/topic_font_wt_win.html
  """

I found some good older discussion on this from 2017: https://github.com/googlefonts/fontbakery/issues/1355

Also, the link from the condition is dead and should be updated:
https://www.adobe.com/devnet/opentype/afdko/topic_font_wt_win.html

I think I'll setup a policy that any URL mentioned in the documentation must be "cached" in the Internet Archive to avoid this kind of bit-rotting...
It is done by posting the URL at https://archive.org/web/ in the "Save Page Now" form:

In this specific case, we were lucky that the Internet Archive was already taking care of saving copied of those pages. So, the resulting URL would then be:
https://web.archive.org/web/20180120062603/http://www.adobe.com/devnet/opentype/afdko/topic_font_wt_win.html


botty 0.4.1 bug can not find left corner window mode

not sure why but its a bug for everyone

I see. You deleted the whole template of stuff to fill out.

Press f11 while you are on hero selection screen, not while you are outside of d2r.


v3.1-branch: some cases failed with build warnings after upgrading zephyr sdk to 0.15.0.

Describe the bug
On v3.1-branch, after upgrading to zephyr-sdk-0.15.0, some tests failed due to build warnings, like some compilation warnings:

"warning: 'elapsed_ms' may be used uninitialized [-Wmaybe-uninitialized]".
":error: comparison between two arrays -Werror=array-compare
if (_net_if_list_start == _net_if_list_end)"
warning: 'version' may be used uninitialized [-Wmaybe-uninitialized]
get_version_check(&version);

To Reproduce
Steps to reproduce the behavior:

west build -b qemu_x86 -p auto tests/kernel/sleep/
See error

Logs and console output
zephyrproject/zephyr/subsys/testsuite/ztest/include/ztest_assert.h:101:21: warning: 'elapsed_ms' may be used uninitialized [-Wmaybe-uninitialized]
bool _ret = z_zassert(cond, msg ? ("(" default_msg ")") : (default_msg),
zephyrproject/zephyr/tests/kernel/sleep/src/usleep.c:68:17: note: 'elapsed_ms' was declared here
int64_t elapsed_ms;
Environment (please complete the following information):

OS: Ubuntu 20.04
Toolchain: Zephyr-sdk-0.15.0
Commit ID: 6628b0631

As noted in the SDK release notes, Zephyr SDK 0.15.0 is "only compatible with the Zephyr development branch (main) and the Zephyr release v3.2 and above," and there currently is no plan to support SDK 0.15.0 in Zephyr 3.1.
Please use Zephyr SDK 0.14.x if you are using Zephyr 3.1 or an earlier release.


[5.3] Laravel View Loading for Router

Inspired by AdonisJS.
Sometimes we need to return only a view on route matches.
This method will just owerwrite standart situation with function.
Instead of writing
app('router')->get('page', function(){
   return view('view');
});

we will write
app('router')->view('view');

Sorry for my English and such short message.
This is my first pull request.

Sorry for reverting, Tryied to fix StyleCI styles.

Router::macro('view', function ($page) {
    return view($page);
});

app('router')->view('view');

@crynobone's way is fine, in your app.