[Bug]: Issue with Fetching Data from Chroma Database after data persisted in database

What happened?
Summary:
I am encountering an issue with fetching data from the Chroma database. Initially, I can successfully fetch data after storing embeddings using client.persist(). However, after a few successful fetch operations, I am consistently encountering a "ValueError: Collection xxx does not exist" error.
Steps to Reproduce:

Store embeddings in the Chroma database using client.persist() method.
Attempt to fetch data using the stored collection and data identifiers.
After a few successful fetch operations, subsequent fetch attempts result in the "ValueError: Collection xxx does not exist" error.

Expected Behavior:
The application should be able to consistently fetch data from the Chroma database using the stored collection and data identifiers without encountering errors.
Observed Behavior:
After several successful fetch operations, the application consistently encounters the "ValueError: Collection xxx does not exist" error when attempting to fetch data.
Additional Information:

The issue occurs both during local development and after deploying the application to Google Cloud Run.
The database connection is established using client.persist() to store embeddings.
The same collection name and data identifiers are used for both storing and fetching data.
The issue seems to persist across different environments, indicating a potential code or configuration problem.
I have confirmed that the collection exists in the Chroma database before attempting to fetch data.
The error message indicates that the collection does not exist, even though it was created and data was stored successfully.

Environment:

Development Environment: Local machine (OS: macOS)
Deployment Environment: Google Cloud Run
Database: Chroma

Versions
chromadb~=0.3.26
langchain==0.0.262
Python 3.9
Relevant log output
chromadb_client = chromadb.Client(Settings(
    chroma_db_impl='duckdb+parquet',
    persist_directory=Config.CHROMA_DB_PERSIST_DIR
))

add data
chromadb_client.get_or_create_collection(name=collection_name,embedding_function=langchain_embedding_function)
.add( embeddings=embeddings, metadatas=metadata, ids=ids)

chromadb_client.persist()

search
def similarity_search(self, collection_name: str,
                          query: str,
                          limit: int) -> List[SearchResult]:
        collection = chromadb_client.get_collection(collection_name, embedding_function=chromadb_embedding_function)
        results = collection.query(
            query_texts=query,
            n_results=limit,
            include=["metadatas"]
        )

        return [SearchResult(results['ids'][0][idx], results['metadatas'][0][idx]) for idx in
                range(0, len(results['ids'][0]))]

Are you able to upgrade to the latest version of chroma? Chroma < 0.4 has issues in the persistence logic.

@HammadB thank you.  I will try the latest one.


Macro: Numbers on top of letters are not centered

Steps to Reproduce

Switch to Macro mode
Add any presets on canvas, bond them
Switch to 'Sequence mode'

Actual behavior
Numbers on top of letters are not centered
Expected behavior
Numbers on top of letters are centered or aligned to the right edge of the number
Screenshots

Desktop (please complete the following information):

OS: Windows 11
Browser Chrome
Version 123.0.6312.106 (Official Build) (64-bit)

Ketcher version
[Version 2.21.0-rc.1]
Related ticket: https://github.com/epam/ketcher/issues/3876

Problem highlighted by @vanoprenko


[entropy_src/dv] Removed LFSR mode, changed efuse to otp interface

Signed-off-by: Steve Nelson<EMAIL_ADDRESS>

Hi Steve, it looks like this needs a manual rebase, but after that is this ready to be merged?

I rebased and pushed it up again. When it passes CI, it's ready to be merged.


Unify actor & class syntax

Supporting:

base classes
override
implicit self
implicit __init__
class vs instance attributes
protocols (to be tracked in separate issue when the time comes)

This syntax unification seems a great idea. So when this is in the air I would like to mention something that has always disturbed me.

A class is a template from which we can generate individual instances (objects), which carry their own state and so on... The word "class" gives at least some intuition in this direction.

An actor is also a template from which we can generate instances with state, but these individuals are more encapsulated, communicate via message passing etc. But the word "actor" for the template definition gives the wrong intuition, and leads thoughts to an individual.  Already when I read the first sentence in this bullet point "An actor is ..." I feel that something is wrong. The actors are the instances; when we create three instances of "actor A...", then we have three actors entering the game, but the sentence speaks about the template.

I have been thinking about  "async class" for actor templates.  From plain classes  we generate (passive) objects, from async classes we generate (active) actors. Another possibility would be "actor class".
Just wanted to raise the question...

On 4 Nov 2022, at 12:44, sydow @.***> wrote:
This syntax unification seems a great idea. So when this is in the air I would like to mention something that has always disturbed me.
• A class is a template from which we can generate individual instances (objects), which carry their own state and so on... The word "class" gives at least some intuition in this direction.
• An actor is also a template from which we can generate instances with state, but these individuals are more encapsulated, communicate via message passing etc. But the word "actor" for the template definition gives the wrong intuition, and leads thoughts to an individual. Already when I read the first sentence in this bullet point "An actor is ..." I feel that something is wrong. The actors are the instances; when we create three instances of "actor A...", then we have three actors entering the game, but the sentence speaks about the template.
I have been thinking about "async class" for actor templates. From plain classes we generate (passive) objects, from async classes we generate (active) actors. Another possibility would be "actor class".
Just wanted to raise the question...

I'm very much in favor of emphasizing the distinction between actor classes and their instances. I think we shouldn't hesitate to use such terminology when we talk or write about actors in Acton.
But I also think there's no obviously best solution to what syntax we should use for that purpose. Saying "async class" conveys too little, in my opinion, as if actors were just a minor asynchronous variant of regular classes. The "actor class" syntax is more accurate, but a bit clumsy as there are no "actor " phrases in the language grammar.
We do have candidates for " class" declarations, though, like "object class" and "value" class" instead of the current "class X (object)" and "class X (value)" forms. Perhaps even "protocol class" or "extension class" could be considered.
But these variants all have the drawback of using more keywords than necessary for stating an unambiguous intent. So maybe the phrases "object X", "value X", "protocol X", "extension X", and "actor X" should also be considered, they are at least consistent -- although with the big caveat that "object" and "actor" means something that creates instances of objects and actors. But even that isn't very controversial, there are other languages whose "module" or "data" declarations take parameters and thereby introduce names that should more accurately be called templates for generating module or data instances.
Be that as it may, however. In our case I think we're trapped between programmer convenience, terminological accuracy and tradition (mostly our Python heritage). The "class" concept is quite deeply rooted and I don't think replacing it with "object" and "value" will be very illuminating. The "protocol" and "extension" declarations carry less tradition, but here programmer convenience might win -- shorter declarations beats accuracy.
Perhaps this is also the case for actors. At least I don't have any problem looking at existing Acton code in this respect. The problems occur when one starts describing what an "actor" declaration really is, and there I think there's no way around the fact that actor declarations must be explained as a "templates", "patterns",  "classes" or "generators" of actors. And while it's straightforward to use the word "actor" for a particular actor at run-time, it doesn't hurt to also push the term "actor instance" in those cases.
In the end I don't know what syntax I prefer. While I fully subscribe to the idea of terminological accuracy, we also don't want the syntax to look overly pedantic. The " class" forms are at least systematic but maybe too verbose.
Let's discuss further at our next meeting!
-- Johan

I agree.

good to use actor classes and actor instances in documentation etc
actor is better than async class, we are an actor based language so it is natural to define actor classes rather than async classes IMHO
actor is better than actor class because it is shorter

it is technically less correct but meh

class is better than object class because it is shorter, on the other hand, I think the outcome from yesterdays meeting is that it will be class Foo() from object and then object Foo() is shorter


add test data as a submodule

add Submodule Sarek-data
enhance ascii-art

/!\ do not merge yet, need to try the test data procedure on bianca

I believe we should now be good with the new submodule and this update help


[BUG] Authentication does not work correctly

Environment
Self-Hosted (Docker)
System
Windows 10 21H2 , Firefox 109.0 x64
Version
2.1.1
Describe the problem
I have set up two users, malordin and admin, with hash and type : admin. But if I log in without cookies and then log in, I can't change the configuration under any user with admin rights. In addition, I have 2 sections hidden from Guests, but when I log in as any user with admin privileges, I still can not see them. What is the problem?

Additional info
Here is my beginning of conf.yml
appConfig:
  theme: crayola
  layout: auto
  iconSize: medium
  language: en
  startingView: default
  defaultOpeningMethod: newtab
  statusCheck: false
  statusCheckInterval: 1
  faviconApi: allesedv
  routingMode: history
  enableMultiTasking: false
  widgetsAlwaysUseProxy: false
  webSearch:
    disableWebSearch: false
    searchEngine: duckduckgo
    openingMethod: newtab
    searchBangs: {}
  enableFontAwesome: true
  enableMaterialDesignIcons: false
  hideComponents:
    hideHeading: false
    hideNav: false
    hideSearch: false
    hideSettings: false
    hideFooter: false
  disableConfigurationForNonAdmin: true
  auth:
    enableGuestAccess: true
    users:
      - user: admin
        hash: 12345
        type: admin
      - user: malordin
        hash: 1245
        type: admin
    enableKeycloak: false
  showSplashScreen: false

There is no information in the logs:
➜  dashy sudo docker-compose logs -f
Dashy  | yarn run v1.22.15
Dashy  | $ node server
Dashy  |
Dashy  | Checking config file against schema...
Dashy  | ✔ Config file is valid, no issues found
Dashy  |
Dashy  | SSL Not Enabled: Public key not present
Dashy  |
Dashy  |
Dashy  |  ██████╗  █████╗ ███████╗██╗  ██╗██╗   ██╗
Dashy  |  ██╔══██╗██╔══██╗██╔════╝██║  ██║╚██╗ ██╔╝
Dashy  |  ██║  ██║███████║███████╗███████║ ╚████╔╝
Dashy  |  ██║  ██║██╔══██║╚════██║██╔══██║  ╚██╔╝
Dashy  |  ██████╔╝██║  ██║███████║██║  ██║   ██║
Dashy  |  ╚═════╝ ╚═╝  ╚═╝╚══════╝╚═╝  ╚═╝   ╚═╝
Dashy  |
Dashy  | *******************************************************************************************
Dashy  | Welcome to Dashy! 🚀
Dashy  | Your new dashboard is now up and running with Docker
Dashy  | *******************************************************************************************
Dashy  |
Dashy  |
Dashy  | Using Dashy V-2.1.1. Update Check Complete
Dashy  | ✅ Dashy is Up-to-Date
Dashy  |

Please tick the boxes

[X] You have explained the issue clearly, and included all relevant info
[X] You are using a supported version of Dashy
[X] You've checked that this issue hasn't already been raised
[X] You've checked the docs and troubleshooting guide
[X] You agree to the code of conduct

I do have a very similar problem. Sometimes i can see my Dashboard (Auth Enabled, Guestmode Disabled) but still i am not logged in (can't do any changes).

I can add to this problem as well. I have installed fresh install yesterday in docker with auth enabled (guestmode disabled). And when I open the site up it shows dashboard firstly, not the login page. But when I create something and save it locally (I am only allowed to save it locally), after that I refresh the page and then the site forces you to the login page, but if I delete the cache, dashboard opens up first again before the login page. I tried with guestmode enabled, it does the same thing (opens dashboard firstly before the login page).

  If I enable keycloak with or without guest mode, I'm shown the dashboard and a log-out button without every being prompted to log in, and when I click the log-out button, this appears in the console:

I am also experiencing authentication issues: Even though enableGuestAccess is set to false, I can access my dashboard using a Chrome in incognito mode. It seems to indicate my dashboard is now publicly accessible.
Also, I can't find the login button anymore on the UI.

 If I enable keycloak with or without guest mode, I'm shown the dashboard and a log-out button without every being prompted to log in, and when I click the log-out button, this appears in the console:

I can also confirm when using the docker image for 2.1.1, a perfectly functioning conf.yml from 2.1.0 will not work for keycloak authentication - actually I did not even notice up until today. I sent the dashboard to someone else and they asked me why the do not need to log in to see elements (so it shows it as if you are already logged in and presents every link to the public!). I rolled back to 2.1.0 and it instantly worked again.
As a test I started cutting everything out of my conf.yml but I never succeeded in getting keycloak running with 2.1.1.


I/O abstraction layer

Create an I/O abstraction layer that support asynchronous and synchronous behaviour
Platforms: Linux, FreeBSD and OpenBSD

Rejected


KeyPointLabels tag color conflict with Label tag

Describe the bug
color conflict when using "KeyPointLabels" tag and "Label" tag
To Reproduce
Steps to reproduce the behavior:

Go to Settings --> Labeling Interface
Use the following labeling configuration as an example:

<View>
  <KeyPointLabels name="highest point" toName="img-1" strokeWidth="20" stokeColor="yellow" fillColor="black">
    <Label value="Face" />
  </KeyPointLabels>
  <KeyPoint name="highest point1" toName="img-1" strokeWidth="10" fillColor="black" strokeColor="yellow"/>
  <Image name="img-1" value="$img" />
</View>

"KeyPoint" tag show color with expected result but "KeyPointLabels" tag resulted in color not functioning
"Label" tag and "KeyPointLabels" is treated as independent

Expected behavior
Label tag color customizable for point labeling
Screenshots

KeyPoint and KeyPointLabels same parameter for color pointer but It doesn't work on KeyPointLabels

Thank you for pointing this out!
These parameters were not used, I removed them from docs. (also there was a typo)
*Labels tags use colors from respective Label and they always have a color either set by background parameter or automatically assigned. So in your case you can use <Label value="Face" background="yellow" />, but currently it allows you to set only one color for KeyPoint.
Also try to use just a separate Labels tag along with KeyPoint tag — that allows you to create generic key points and assign labels to them if needed. To fix empty hole inside them use opacity="0.9" in Labels tag.
Later we'll go through all the parameters and make some improvements and unifications.


why do we need to pull a $RUNNER_TOKEN

Hello,
excellent docker image, I'm reading through the code,  and I found this piece of code
        RUNNER_TOKEN="$(curl -XPOST -fsSL \
            -H "Authorization: token ${GITHUB_ACCESS_TOKEN}" \
            -H "Accept: application/vnd.github.v3+json" \
            "https://api.github.com/${SCOPE}/${_PATH}/actions/runners/registration-token" |
            jq -r '.token')"

the reason why I asked is that, if we need to go to GitHub anyways either to create a $GITHUB_ACCESS_TOKEN or getting $RUNNER_TOKEN, I think is easier to get $RUNNER_TOKEN

Hello,
GITHUB_ACCESS_TOKEN doesn't expire whereas RUNNER_TOKEN expires in 1 hour.
By calling the API to dynamically generate this token in the entrypoint script, we ensure that even if the container restarts, it won't fail because the token is expired.
Hope this clarifies.

yes, it does, thanks for the explanation


chore: bdd-spring-1590288367 to 0.0.1

chore: Promote bdd-spring-1590288367 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


compiler crashes with floating point exception

Occurs when compiling this code:
typedef struct {
  int a, b;
} S;

S convert(int n) {
  S s = { n, n};
  return s;
}

kernel void foo(global S* A, global uchar4* B, int n) {
 *B = (uchar4)((uchar)n,1,2,3);
 *A = convert(10);
}

Does not occur when S gains two extra int members, and the convert function is adjusted accordingly.

Another test case:
typedef struct {
  uint x;
  uint y;
} S;

kernel void foo(global S *A, int n) {
  S before = *A;
  before.y += 2;
  *A = before;
}

It's a divide by zero exception:
Program received signal SIGFPE, Arithmetic exception.
0x000000000050d072 in (anonymous namespace)::ReplacePointerBitcastPass::runOnModule (this=0x5fd0bb0, M=...) at /lib/ReplacePointerBitcastPass.cpp:761
761                 unsigned NumElement = DstTyBitWidth / SrcTyBitWidth;
DstTyBitWidth is 64, SrcTyBitWidth is 0

Running Instcombine results in
store i64 %4, i64 addrspace(1)* %3, align 4
So the source pointee type is a 64bit scalar.  But it looks like that part of the code generator is expecting a vector.

Working as of 00e7a58a2834ee938830f82b42a0692fc2232511 at least. I believe it was fixed by #370.


serialization optimizations

* add daos-gen tool to generate a DAOS_OF_ARRAY object
* add feature checking in serialization for DAOS_OF_ARRAY

Signed-off-by: Danielle Sikich<EMAIL_ADDRESS>

@daltonbohning closing this since the plan is not to do the DAOS_OF_ARRAY optimizations anymore, and I will submit a new PR for DAOS_OF_KV_FLAT.


chore: drop kitkat support

A lot of cleanup. All tests pass. Please test on iOS.

It looks like AES support is dropped. Can you confirm? If so, doesn't this affect the other platforms? Shouldn't the documentation be updated?
I think this change needs a new major version number due to both dropping old Android version and apparently dropping support for an encryption algorithm (AES). Or am I missing something here?

It looks like AES support is dropped. Can you confirm? If so, doesn't this affect the other platforms? Shouldn't the documentation be updated?

I haven't had the time to look into this properly but from a quick glance it seems to me that native AES is used for Android now. For iOS this is unnecessary as the native keychain is used. So AES is still used for Android but we do not rely anymore on sjcl for it.

I think this change needs a new major version number due to both dropping old Android version and apparently dropping support for an encryption algorithm (AES). Or am I missing something here?

SecureStorage will be bumped to a major version yes. The current version can be used by folks needing compatibility with older version of android.

No. AES is not dropped. Usage of sjcl AES  that supported KitKat is dropped.
Native AES is used for LOLLIPOP and up.
the docs will be updated to mention the drop of support for KitKat.
new major version will be assigned.

Thank you very much Demetris, this looks good. I should be able to review and test on Android as well as iOS next week.
Could you please update the docs.
Is there someone who can verify tests pass on Windows?

@brodybits Thanks for your comments.

Thakns @demetris-manikas for the updates and clarification.
I would be happy to give it a try on Windows, probably sometime next week. Please ping me here in case I forget.

I am good with iOS and the code review. If @brodybits can verify all ok under windows I will merge and release.

@brodybits Hi. Can you please perform the tests on windows so that we can go on with this?
Thanks in advance.

I will need a few more days, apologies.


Connect the FE to the BE

[x] Correct the url for Login and Sign-Up of User
[x] Add Login identifier (so on User Profile current logged in user can write Tweets else for other account - sees Follow / Unfollow button)
[ ] Option for Log out?
[x] Add Redux methods for Create Tweet
[ ] Add Infinite Scroll Logic -- https://www.npmjs.com/package/react-infinite-scroll-component
[x] Logic (UI and Redux) for Follow / Unfollow

[x]  Correct the url for Login and Sign-Up of User
[x]  Add Login identifier (so on User Profile current logged in user can write Tweets else for other account - sees Follow / Unfollow button)
[ ]  Option for Log out? - not required
[x]  Add Redux methods for Create Tweet
[ ]  Add Infinite Scroll Logic -- https://www.npmjs.com/package/react-infinite-scroll-component - not required for now
[x]  Logic (UI and Redux) for Follow / Unfollow


Make anchorableBehaviours actually respect anchorRotation property

Herp.
That's all I have to say really

[x] Verify that an anchorableBehaviour only rotates to match its anchor if anchorRotation is checked

Eyy


Implement ES2015 module system

This patch is the first part of the ES2015 module system implementation.
The language specification can be found here
The implemented feature is documented here
JerryScript-DCO-1.0-Signed-off-by: Daniel Vince<EMAIL_ADDRESS>

Other notes:

jerry-core/module/module.c should be renamed to module_import.c (or something like this). An other module.c already exist in the jerry-ext folder.
The Makefile of the NuttX target doesn't like if a filename is used multiple times (see nuttx-stm32f4/Makefile#L47 - All the archive files are copied from JerryScript to NuttX. The copied archives are extracted to get the object files. If there are two module.o files (in different archive files) the (jerry-core)module.o will be overwritten by the (jerry-ext)module.o file). This causes linker errors.

The targets/nuttx-stm32f4/jerry-main.c implements jerry-port-log. This functions is also implemented in jerry-port/default/default-io.c. Since both files are compiled, there is a linker error that Travis shows. (I've removed jerry-port-log from jerry-main.c to solve the linker error.)

Test files contain paths (tests/jerry/...) that should be opened. These paths can be different on boards - e.g. js-remote-test mounts tests/jerry/ folder as /test/.

Other notes:

jerry-core/module/module.c should be renamed to module_import.c (or something like this). An other module.c already exist in the jerry-ext folder.
The Makefile of the NuttX target doesn't like if a filename is used multiple times (see nuttx-stm32f4/Makefile#L47 - All the archive files are copied from JerryScript to NuttX. The copied archives are extracted to get the object files. If there are two module.o files (in different archive files) the (jerry-core)module.o will be overwritten by the (jerry-ext)module.o file). This causes linker errors.
The targets/nuttx-stm32f4/jerry-main.c implements jerry-port-log. This functions is also implemented in jerry-port/default/default-io.c. Since both files are compiled, there is a linker error that Travis shows. (I've removed jerry-port-log from jerry-main.c to solve the linker error.)
Test files contain paths (tests/jerry/...) that should be opened. These paths can be different on boards - e.g. js-remote-test mounts tests/jerry/ folder as /test/.

Thank you @rtakacs for your review. I've moved and renamed the module.{h, c} as you suggested.
The paths in the test files are adjusted to the ./tools/runners/run-test-suite.sh, which runs on Travis too. I think we should figure out how to adjust these paths when the the js-remote-test environment is running.


CameraX support on external cameras using USB

Hello,
Can anyone tell if USB cameras are already supported by CameraX, using the latest version?
Thanks

The issue tracker is being used for problems with the unmodified samples. If you have questions about usage, please use tag android-camerax in StackOverflow or start a thread in the developer forums. CameraX currently does not support USB cameras.


Update direct-routing-sbc-multiple-tenants.md

@NMuravlyannikov cc @CarolynRowe
Added information about requirement to create DNS record for the derived trunk name. Might be obvious but seems it is something that could get missed.

@nenonix @JohanFreelancer9 Copy editing is needed for this PR. Thanks.

@CarolynRowe PR has been copyedited and is ready for final review, could you please check and merge? Thanks!


Update deps

@TabDigital/ping-api updated all dependencies
(detective@4 uses acorn instead of esprima so needed some tweaks to the error reporting)

looks good :+1:


Fixed issue with paging in /repos/{owner}/{repo}/git/trees/{sha} api

Fixed issue with paging in /repos/{owner}/{repo}/git/trees/{sha} api. Basically, any page requested other than page 1 (where items should be returned) would fail with an index out of range error (user gets http 500) because the wrong var was being used to build the return list.
Ref issue: #9453

Codecov Report

Merging #9459 into master will decrease coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #9459      +/-   ##
==========================================
- Coverage   41.52%   41.52%   -0.01%
==========================================
  Files         570      570
  Lines       74642    74642
==========================================
- Hits        30996    30992       -4
- Misses      39806    39809       +3
- Partials     3840     3841       +1

Impacted Files
Coverage Δ

modules/repofiles/tree.go
81.01% <100%> (ø)
:arrow_up:

modules/indexer/code/indexer.go
44.73% <0%> (-10.53%)
:arrow_down:

modules/log/event.go
64.61% <0%> (-1.03%)
:arrow_down:

services/pull/check.go
53.52% <0%> (-0.71%)
:arrow_down:

models/webhook.go
70.46% <0%> (+1.06%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8149ed7...525ada3. Read the comment docs.

Please send backport to 1.10


Style Guide

Summary
Create a style guide
Why?
Consistency.  Save time on review of current materials
Details
It doesn't have to be extensive, but it would cover things like:

Spelling (American in the app and documentation)
Capitalization rules (I suggest Title Case for all headings, labels, and buttons.)
Rules for sorting (lower case alphabetical)
colours , or some equivalence -- like what the css classes are for warnings and the like

Ultimately, we might like to have this "live" -- in that we can show actual examples -- that is, it would be an erb.
Linked to from Contributing.md
Thoughts?

It's probably out of date but the wiki has one from (2018) [https://github.com/rubyforgood/human-essentials/wiki/STYLEGUIDE]

Thank you -- that'll be a good start.

Limiting this to a style guide for the user documentation.   The style guide from the wiki above deals with the app itself.


SSD LIte backbone mobilenet in_channels=4

https://github.com/open-mmlab/mmdetection/blob/main/configs/ssd/ssdlite_mobilenetv2-scratch_8xb24-600e_coco.py
I want the in_channel of mobilenet for that model to be 4 because the data I will be inputting is rgbd. How should I modify that config?

You can refer this https://github.com/open-mmlab/mmyolo/blob/main/docs/zh_cn/common_usage/single_multi_channel_applications.md#单通道和多通道应用案例


Render vertices as boxes

As discussed in our meeting today, it would be nice to have an option to render vertices as boxes instead of points.

You mean vertices, not 0-cells.

Thanks, that clears it up a little. I've been working on making the vertices individually restylable (ie. each vertex can be a box or a circle or some other shape). I think this will be relatively straightforward to finish but it may conflict slightly with the global setting.
For now I'll keep working on individual styles. Let me know if I should refocus on the global setting instead.

Perfect, that's even better.

Nice work @thud! Maybe split out a GeneratorStyle struct that can be shared rather than sharing the whole GeneratorInfo?
Let me know if you need a hand making your state changes backwards compatible with the serialiser!

Maybe split out a GeneratorStyle struct that can be shared rather than sharing the whole GeneratorInfo?

Oh yeah, that's much better actually. I'll work that through today and see what I can get running.

We might also need the name if say we decided to label the vertices, so it might be good to share the whole GeneratorInfo just for future convenience.

We definitely need tooltips showing the generator name when we hover the
mouse cursor over the diagram.
On Fri, 1 Jul 2022, 11:29 Calin Tataru, @.***> wrote:

We might also need the name if say we decided to label the vertices, so
it might be good to share the whole GeneratorInfo just for future
convenience.
—
Reply to this email directly, view it on GitHub
https://github.com/homotopy-io/homotopy-rs/issues/460#issuecomment-1172140146,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACQ4OHQQBTZ65XLWBTTBHO3VR227BANCNFSM5UL5EVJA
.
You are receiving this because you commented.Message ID:
@.***>

We already have that right? Also it's not related to this bug I think.

This is a GitHub bug. I made that comment months ago by email and it only just got processed.

Haha interesting.


Questions about DIce coefficient

Unet is the basic network, and the others are improved versions of unet, but why is the dice coefficient of segmenting liver tumors lower than that of unet?

Sorry, I am confused about it too.


Unable to create a plugin

Hi there,
I love the idea for this library, thank you for creating it.
I was following the basic outline of the example, but as I was attempting to write tests for my plugin module, I noticed that the internal definition for plugins: https://github.com/SamVerschueren/clinton/blob/master/lib/utils/rule.js#L7 defaults to an array, and the following line uses .indexOf to test for plugin matching. Relative to the example providing:
{
		"plugins": {
			"file-exists": ["error", "index.js"]
		}
}

How is one supposed to supply plugins to the config? If it's as an object, then I am concerned that the existing test (plugins.indexOf(ruleId)) will never match anything. If it's as an array, then I don't know how to pass configuration to it, and the example code is mismatched.
Could you provide a simple standalone example of how to create a plugin with tests?
If needed I can publish my example code.

You are right, I believe the docs are wrong. I'll look into it and get back to you.

Any update on this?

Sorry for the late reply and thanks for pinging me back, lost this one out of sight.
The package.json example should be like this.
{
	"name": "Unicorn",
	"description": "My unicorn package",
	"version": "1.0.0",
	"scripts": {
		"test": "clinton"
	},
	"devDependencies": {
		"clinton": "*",
		"clinton-plugin-file-exists": "1.0.0"
	},
	"clinton": {
		"plugins": [
			"file-exists"
		],
		"rules": {
			"file-exists": ["error", "index.js"]
		}
	}
}

So yes, the plugins list is an array and just defines the additional plugins. And then you have to enable them in the rules object and you can pass in some parameters.


Added support of query parameters for RestDocs for WireMock stubs

@leebake Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

@leebake Thank you for signing the Contributor License Agreement!

Cześć @leebake ! :) Would you mind adding a small test for this case?

@marcingrzejszczak as you wish ;)

Thanks @leebake for your contribution!

@marcingrzejszczak is there any chance to merge and release it in 2.1.x or should I create new PR?

Cherry-picked to 2.1.x :)

Thank you so much!


Disable FCM-only messaging

If a user has iOS notifications disabled it is still possible for FCM-based messages to be received when the app is in the foreground.
What is the recommended approach for disabling/suppressing FCM-based messaging when APNs-based notifications are disabled?

Not sure we have a specific recommendation there - you can choose not to respond to messages in applicationReceivedRemoteMessage. Are you seeing some unexpected behavior?

We thought we were but haven't seen this for a while.


UI Proposal: Alignment

Alignment is a little off on a few pages, but most notably the Treasury page. The 'Show Other Fund Sources' button is also a little cramped.

@esot321c I can fix this


Antipoison timer never showing

Describe the bug
The antiposion only shows on login and then never more.
To Reproduce
A)

Login to the game
Anti-poison timer starts ticking 90s without drinking anything.

B)

Drink antiposion.
No timer shows. (If the timer was still active since the login, it continues ticking down the 90sec from login, does not reset).

Environment (please complete the following information):

OS: Windows 10 - 64bit
RuneLite version: 1.6.30
Launcher version: 2.1.3

Additional context
I have tried turning off/on multiple settings in "Timers" and "Boosts Information", none of them seem to do anything.
There were no extra plugins installed at the time of the error, was a clean instal.

So long as your Timers plugin is enabled and you have the "Antipoison/Venom timers" option enabled, this should work as expected. I cannot reproduce this issue ingame.

Hello, thank you for your response. I am attaching a video, excuse the quality, but it should show clearly what I was thinking about. The antipoison timer always shows after login, even though I did not even use it the session before. And after drinking a whole potion, not justone swig. The timer never resets. And if I would wait another minute and drink again, new timer does not show up. Is there any info/data I may supply?
Timers plugin is turned on along with all of its options selected.

Hello, thank you for your response. I am attaching a video, excuse the quality, but it should show clearly what I was thinking about. The antipoison timer always shows after login, even though I did not even use it the session before. And after drinking a whole potion, not justone swig. The timer never resets. And if I would wait another minute and drink again, new timer does not show up. Is there any info/data I may supply?
Timers plugin is turned on along with all of its options selected.
Video: https://we.tl/t-xfjunhLo6c (mp4)

This isn't a support forum.
Please join the discord for help. https://runelite.net/discord

This isn't a support issue, this is a bug that we've heard about before--it apparently only happens on free-to-play worlds.

Thank you Nightfirecat, I was beginning to feel really dumb. Anyways I tried turning off every single feature of RuneLite except for "Timers" to see if maybe something is colliding there, but to no difference. Tried turning off all settings inside Timers too except for the antiposion/venom one. Same thing. There is only one condition under which the timer shows - login or switch worlds. Cant test wheather its F2P only, just yet.
Cant think of anything else to try or test, but if you could use a help with any testing or data, do let me know.


Remove VOID from procedure signatures

The PR

changes procedure signatures in TCK scenarios that have return type VOID to return type () and
removes duplicate scenarios that result from this change.

Return type VOID to return type () do not have a semantics difference in openCypher at the moment. The Call CIP does not allow VOID as output signature. Hence, there is no point in keeping such scenarios.

@thobe I have made the minimal changes to the CIPs to remove VOID, I hope. (... and fixed two typos on the way)
If you look a little closer, you will see that I also changed the nullable vs. mandatory logic, i.e. T? vs T!, first. The CIP is inconsistent with what we came up for CREATE INDEX. However, the CIP is consistent with what APOC does around procedure signatures. A quick investigation with
CALL apoc.help('') YIELD name, signature
WITH name, (signature CONTAINS "?") as opt
WHERE opt
RETURN COUNT(DISTINCT name) as c
UNION ALL
CALL apoc.help('') YIELD name, signature
RETURN COUNT(DISTINCT name) as c

revealed, though, that all signatures contain at least one nullable type, which suggests that nullable as default would be the better choice here too.
Anyway, I decide that this is a topic in and of itself and reverted these changes here in this PR.

@thobe did you had a change to look at the changes I made up on your review? What are the next steps?


FirstSteps

I have some first interactions and game events.
Now:

[x] How do I switch between different views?
[x] How do I properly handle all the resources and workers, including updating the text stats?
[ ] How to handle story, e.g., if there are no more trees, you loose; or if there is no more food people die?
[ ] How to store state?
[x] Dark mode :D

Data handling resolved by switching to Jetpack Compose and MVVM structure


Make --init auto evaluate to raw code

Instead of evaluating to a "switch" statement, make --init auto evaluate
to the designated code for the current $SHELL. This improoves the
cache using
fasd_cache="$HOME/.fasd/fasd.$(basename $SHELL)"
if [ "$(command -v fasd)" -nt "$fasd_cache" -o ! -s "$fasd_cache" ]; then
  fasd --init auto poisx-alias >| "$fasd_cache"
fi
source $fasd_cache
unset fasd_cache

Also removes some redundant (?) messages about completion in $_FASD_SINK

This does not work: fasd script can be run by a different shell than the interactive shell. If the shell running the script and the shell executing the init script are different, then this will not work. For instance, if I have $_FASD_SHELL set to dash and use zsh as my interactive shell, then the init script will be for POSIX but not for zsh.

Oh didn't think about that, never mind then.


Inference timestamp is saved as created_at in database

Describe the bug
Inference timestamp is saved as created_at in database. Should be the actual timestamp of client's data.
To Reproduce
Just create any report based on report functions (performance metrics, drift etc.)
Expected behaviour
Timestamp should be the data's actual timestamp. This ticket is highly related to #126 . Actually is blocked by this.

@NickNtamp please elaborate based on the above template

We should be using the timestamp of the inference instead of now() method


[Feature Request]: The testimonials link in navbar isn't working when viewed on small screens.

Is there an existing issue for this?

[X] I have searched the existing issues

Feature Description
The testimonials link in navbar isn't responsive. I am a contributor in GSSoC' 24. I can solve this issue as I have worked upon similar issues earlier.
Please assign this task to me under GSSoC'24.
Use Case
Testimonials link should be redirected to home page testimonials section but the feature is actually not working.
Benefits
No response
Add ScreenShots
https://github.com/user-attachments/assets/0c7f122c-5179-4a90-a9f4-756d5b406025
Priority
High
Record

[X] I have read the Contributing Guidelines
[X] I'm a GSSOC'24 contributor
[X] I have starred the repository

Thank you for creating this issue! 🎉 We'll look into it as soon as possible. In the meantime, please make sure to provide all the necessary details and context. If you have any questions or additional information, feel free to add them here. Your contributions are highly appreciated! 😊
You can also check our CONTRIBUTING.md for guidelines on contributing to this project.

Go ahead @AlaynaShaheen

Thanks for assigning the issue @ashutosh-rath02. I'll resolve the issue and raise a PR soon.

Actually after analyzing the code I found that there is no testimonials page seperately and when we click on testimonials tab it is getting redirected to the testimonials section of the same home page. Sorry for the issue created. @ashutosh-rath02

Hello @AlaynaShaheen! Your issue #421 has been closed. Thank you for your contribution!


run permission callback within lock resolution closure

I had/have a use-case where I need to lookup the authenticated user's attributes within the permission callback to use them within the ArrayDriver. I came across an issue where by the time the permission callback is called, the authenticated user is not yet available Moving the callback into the provider resolution closure fixed my issue.

Should be fixed in the latest release.


D is missing

Unfortunately D is missing.  Could you add support for the D Programming Language?

Would you consider contributing D support to quicktype?  We'd love to help.

Except my mobile phone I will have hardly access to a computer in upcoming weeks.
I can point you to the the right resources:
http://ddili.org/ders/d.en/index.html
Also there is a quite good video tutorial
https://www.youtube.com/watch?v=rwZFTnf9bDU
You could also  announce quicktype
On forum.dlang.org. maybe someone also already has some experience with TS.

What about the weeks after the upcoming weeks?

I will re-check this topic when I have time again.
In general I avoid writing code in script languages. My personal experience is, for a quite high time investment, the outcome is very low. And even if the outcome is high, the quality of the outcome is low. The best mix of time investment, outcome and quality I found in the D programming Language  .

Closing this per our policy of not leaving language requests open as issues.


Make app/react/package.json private to not leak dependencies

Issue: Previously the dependencies of the react app were leaking into my project
What I did
set the app/react/package.json to private
How to test
in an app that uses storybook run yarn install and no longer see the warning below
@kadira/storybook > babel-preset-react-app ><EMAIL_ADDRESS>💥  preset-latest accomplishes the same task as babel-preset-env. 🙏  Please install it with 'npm install babel-preset-env --save-dev'. '{ "presets": ["latest"] }' to '{ "presets": ["env"] }'. For more info, please check the docs: http://babeljs.io/docs/plugins/preset-env 👌. And let us know how you're liking Babel at @babeljs on 🐦

Hey can you direct me to the location where this behavior is described as intended?
AFAIK this property is intended to mark a package as non-public.

@ndelangen when you do yarn in a project that uses storybook you get an error as I've pasted above.

@ndelangen this is also causing compilation errors in create-react-app 1.0 due to the leaked dependencies

@ndelangen I figured out why storybook is breaking create react app, the reason has to do with the dependencies. all babel dependencies should be marked as dev dependencies.

@ndelangen I've updated what I can, but if you know how to test your dependencies within your app I think you should take it further as I'm unsure how to test much future due to me not being familiar with the project structure.

@shilman these are in master, I'm just moving them around so they don't get polluted into my app.

@shilman @ndelangen okay, I found the issue. It lies within the addons package, I believe you've released the addons package while not releasing the v3 of the main package. https://github.com/lifeiscontent/storybook-cra-example you can checkout the package.json

@shilman @ndelangen Okay, I figured out it was because I was using the old version of getstorybook. Sorry for all the noise.

No problem at all! I'm glad you found the solution 👍


driver: i2c: esp32: fix I2C statistics

I2C driver overwrites i2c_msgs buffers, which
causes statistics to fail counting.
This creates new variables to hold msg buffer information.
Fixes #57532

Fixes I2C_DUMP_MESSAGES and read bytes.  But write bytes are still wrong!  But it's no longer this code that is breaking them.  There is also a bug in the i2c stats code itself.  They don't check the message flags in the correct way.

Fixes I2C_DUMP_MESSAGES and i2c stats for read bytes. But i2c stats written bytes are still wrong! But it's no longer this code that is breaking them. There is also a bug in the i2c stats code itself. They don't check the message flags in the correct way.

Yes, I2C statistics only take into account when flags are either READ or WRITE. Feel free to submit a PR to update that.


Added Euro Converter USD

Description
Currency Converter is a simple and convenient Chrome browser extension that allows you to easily convert currency from Euro to USD.
Features:

Quick Conversion: Enter the amount in Euro and click the "Convert" button to instantly convert it to USD.
Fixed Exchange Rate: Calculates with a fixed exchange rate of 1 Euro = 1.08 USD.

How to Use:

Click on the Currency Converter icon on the Chrome toolbar.
Enter the amount in Euro that you want to convert.
Click the "Convert" button to see the equivalent amount in USD.

Say goodbye to complex currency conversions with Currency Converter!
Fixes:  #1322
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Checklist:

[x] I have made this from my own
[ ] I have taken help from some online resources
[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings

ATTACH SCREEN-SHOTS / DEPLOYMENT LINK

@Sulagna-Dutta-Roy  i have reviewed all the changes and created this new pull request and deleted the old ,now this has no conflicts and errors please have a look.


[modern] Device Direction

Hi
I have a problem, in the modern interface there is no longer the arrow that moves according to the movement of the devices, I want to implement that but I have problems because I can not directly modify the icons because it is only passed as a parameter the colors.

Hello Anton! Is it possible to add to the vehicle icon the direction that it is moving, the same as it was in the old interface?
This information is excellent for when sending a screenshot to the user, he knows in which direction the vehicle was moving on a highway for example.

The old solution doesn't work very well and it also doesn't look very nice. We're thinking of some other option.

OK. Thanks for the feedback


Integrate github-api structs generation into server and cargo build

We need to integrate generation of Open API structs from github-api dir, since we depend on them in our implementation. I think we can do something similar to scripts in npm.
Possibly, we also need to make github-api a member of workspace (let's rename it to github-mirror).

@l3r8yJ WDYT?

@l3r8yJ I found this: https://github.com/JoshMcguigan/cargo-run-script

@l3r8yJ another alternative: https://crates.io/crates/just

Let's stick to the just, it has 18k+ stars, and looks nice


WIP feat: process json primitives

What
The ability to process json primitives (values not contained in maps)
Why
Fuctionality does not seem to exist
How
Funcitonality + unit tests added
Test
cargo test
Checklist

[x ] I have self-reviewed this PR
[ x] I have added tests that prove the feature works or the fix is effective

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


[WIP] Add mTLS CA Option on tornjak server

Addresses https://github.com/spiffe/tornjak/issues/2, by providing a —mtls-ca flag to validate mTLS certificates for validation of client certs.
Signed-off-by: Brandon Lum<EMAIL_ADDRESS>

Checke with @lumjjb if ready to merge

Testing and documentation (explain the server params, key management)


MinMaxScaler from training data is not used to normalize the test data

https://github.com/yiwen26/WindChaser/blob/master/WindChaserModules/RecurrentNeuralNetwork/RNN_5Input/RNN_5input_XXJ.ipynb

Actually this can be regarded as Minmax normalization: https://www.quora.com/What-is-the-meaning-of-min-max-normalization


header

tell me how to put in the library header?
header ('Access-Control-Allow-Origin: *');
It does not work

This is the socket server library. It asynchronously serve raw data over streams. You're trying to set an HTTP header. See reactphp/http

made as in the example, the result is still not fulfilled the title
$loop = React\EventLoop\Factory::create();
$socket = new React\Socket\Server($loop);

$http = new React\Http\Server($socket);
$http->on('request', function ($request, $response) {
    $response->writeHead([200, 'Access-Control-Allow-Origin' => '*']);
    $response->end("");
});
$socket->on('connection', function ($conn) {
    $conn->on('news', function ($data) use ($conn) {
        $conn->write("ok");
    });
});
$socket->listen(1337, 'localhost');
$loop->run();


How to create a new ontology from scratch with data or metadata

Tutorial Request:
I am interested in learning how ontologies are created from data, ,as I see many tutorial starting off with one or more ontologies at the start of ontology creation.  I am faced with such problem as having just a list of controlled vocabularies and terminologies, how do / should I create an ontology from such data and metadata if there are no existing ontologies in the domain?

Thank you @S2Ola
We just changed our system for requesting tutorials, can you copy your request here https://github.com/OBOAcademy/obook/discussions and close this issue please?

Thank you @matentzn  closed!


737 rich texts preview is not refreshed after changing its content

it should be v8.30.1 , but im too tired and mistakenly versioned it v8.31.0


AutoCompleteTextFieldState not disposing

Navigation is Popped while autocomplete is in focus with a list of items being displayed and no item is selected.
When the ensuing Navigation screen is popped, the following exception is thrown.

Both Widgets use which create back button at the top:
return new Scaffold(
appBar: AppBar(
title: Text("Title"),
),
body: ....
════════ Exception caught by gesture ═══════════════════════════════════════════════════════════════
The following assertion was thrown while handling a gesture:
setState() called after dispose(): AutoCompleteTextFieldState#0a15a(lifecycle state: defunct, not mounted)
This error happens if you call setState() on a State object for a widget that no longer appears in the widget tree (e.g., whose parent widget no longer includes the widget in its build). This error can occur when code calls setState() from a timer or an animation callback.
The preferred solution is to cancel the timer or stop listening to the animation in the dispose() callback. Another solution is to check the "mounted" property of this object before calling setState() to ensure the object is still in the tree.
This error might indicate a memory leak if setState() is being called because another object is retaining a reference to this State object after it has been removed from the tree. To avoid memory leaks, consider breaking the reference to this object during dispose().
When the exception was thrown, this was the stack:
#0      State.setState. (package:flutter/src/widgets/framework.dart:1105:9)
#1      State.setState (package:flutter/src/widgets/framework.dart:1140:6)
#2      AutoCompleteTextFieldState.updateOverlay... (package:autocomplete_textfield/autocomplete_textfield.dart:322:37)
#3      _InkResponseState._handleTap (package:flutter/src/material/ink_well.dart:654:14)
#4      _InkResponseState.build. (package:flutter/src/material/ink_well.dart:729:32)
...

Same issue here!


neighbouring cells by provider cell

Hello ,
I'm trying to get neighbouring cells but unable to do so , I'm getting only cid that I'm connected to !
Do you have any configuration to help me to get these neigboring cells in order to geolocate the device using opencellID API with more accuracy .
thanks,
Omar Atia

Hi Omar, I don't have any suggestions, I haven't used that API in quite a while. Maybe try https://android.stackexchange.com/?

Closing.


Allow targeting of ListItem content

Currently ListItem renders a <li> and group of divs in it. For accessibility purposes, it needs to behave similar to MenuItem and have a wrapper and item slot.

this is now outdated as we will use Menu variant for the required purpose


Prevent Row Click from selecting row

Issue Type

[x] Question

Is there a way to prevent clicking on a row from selecting the row ?
I'd like to limit row selection to just to just click the tick box.
The reason for this is so I can have separate events triggering for specific cell clicks.
Equally I have buttons included in the rows that trigger actions and they cause the row to be selected which is a bit jarring. Sorry if i've missed something in the docs but there is nothing that I can find.
 I'll check this out! Thanks !


ABORTED: Shutdown in tf.raw_ops.PriorityQueueV2

Issue type
Bug
Have you reproduced the bug with TensorFlow Nightly?
Yes
Source
source
TensorFlow version
tf 2.16
Custom code
Yes
OS platform and distribution
Linux Ubuntu 22.04.3 LTS (x86_64)
Mobile device
No response
Python version
3.11
Bazel version
No response
GCC/compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current behavior?
When the component_types parameter of tf.raw_ops.PriorityQueueV2 contains some special characters, such as "-", "*", "!" etc., it will trigger "Local rendezvous is aborting with status: ABORTED: Shutdown" error.
Standalone code to reproduce the issue
import tensorflow as tf
shapes = [1, 1]
component_types = [tf.float32, tf.int32]
capacity = 10
container = "test_container"
shared_name = "testsharedname"
queue_handle = tf.raw_ops.PriorityQueueV2(
    shapes=shapes,
    component_types=component_types,
    capacity=capacity,
    container=container,
    shared_name=shared_name
)

Relevant log output
2024-05-15 14:25:39.269296: W tensorflow/core/framework/op_kernel.cc:1839] OP_REQUIRES failed at resource_op_kernel.h:76 : INVALID_ARGUMENT: container contains invalid characters: test!container
2024-05-15 14:25:39.269340: W tensorflow/core/framework/local_rendezvous.cc:404] Local rendezvous is aborting with status: INVALID_ARGUMENT: container contains invalid characters: test!container
Traceback (most recent call last):
  File "/home/tf/test.py", line 10, in <module>
    queue_handle = tf.raw_ops.PriorityQueueV2(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/envs/tf2.16/lib/python3.11/site-packages/tensorflow/python/util/tf_export.py", line 377, in wrapper
    return f(**kwargs)
           ^^^^^^^^^^^
  File "/root/anaconda3/envs/tf2.16/lib/python3.11/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 3240, in priority_queue_v2
    return priority_queue_v2_eager_fallback(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/envs/tf2.16/lib/python3.11/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 3312, in priority_queue_v2_eager_fallback
    _result = _execute.execute(b"PriorityQueueV2", 1, inputs=_inputs_flat,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/envs/tf2.16/lib/python3.11/site-packages/tensorflow/python/eager/execute.py", line 53, in quick_execute
    tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tensorflow.python.framework.errors_impl.InvalidArgumentError: {{function_node __wrapped__PriorityQueueV2_device_/job:localhost/replica:0/task:0/device:CPU:0}} container contains invalid characters: test!container [Op:PriorityQueueV2]
2024-05-15 14:25:39.422495: W tensorflow/core/framework/local_rendezvous.cc:404] Local rendezvous is aborting with status: ABORTED: Shutdown

@x0w3n tf.raw_ops.PriorityQueueV2  function creates a priority queue data structure. The container name "test_container" contains characters that are not allowed. Could we try to rename the container to use only alphanumeric characters (a-z, A-Z, 0-9) and underscores (_). Avoid special characters like "-", "*", "!". I was able to replicate the issue reported here. Thank you!

@x0w3n Could you please let us know if the issue got resolved ?


onConnect and onDisconnect hooks

Hi @mochi-co
I think it would be interesting to have onConnect and onDisconnect event hooks, at least to log connections.
Igor

I agree! I have added OnConnect and OnDisconnect hooks. They have been merged and released as v1.0.5.
OnConnect
server.Events.OnConnect is called when a client successfully connects to the broker. The method receives the connect packet and the id and connection type for the client who connected.
import "github.com/mochi-co/mqtt/server/events"

server.Events.OnMessage = func(cl events.Client, pk events.Packet) (pkx events.Packet, err error) {
    fmt.Printf("<< OnConnect client connected %s: %+v\n", cl.ID, pk)
}

OnDisconnect
server.Events.OnDisconnect is called when a client disconnects to the broker. If the client disconnected abnormally, the reason is indicated in the err error parameter.
server.Events.OnDisconnect = func(cl events.Client, err error) {
    fmt.Printf("<< OnDisconnect client dicconnected %s: %v\n", cl.ID, err)
}


Feature Request: audio rate sample-and-hold

A sample-and-hold locked to an audio rate signal (switching on 0/nonzero, but not necessarily locked to the transport) would be a very useful feature for LFOs.
I came up with this (which technically is double the rate of the LFO but easy enough to halve the signal coming in). There are two problems with my approach: there's still occasional noise at the switchover, and it's not technically legal, because Noise() abstracts the audio buffer playbackRate from a signal to a positive and its _source is private. I'm open to other suggestions or paradigms, thanks for the great library!
// gates that return opposite values (0, 1) on positive part of LFO
const one = new Tone.Signal(1);
const gate  = new Tone.GreaterThanZero();
const inverseGate = new Tone.Subtract();
one.connect(inverseGate);
gate.connect(inverseGate.subtrahend);

// connect LFO to gate
const osc = new Tone.Oscillator(1).start();
osc.connect(gate);

// Noise signal runs muted on positive and holds on negative, illegally connected
// to _source.playbackRate. Gain node because the incoming signal is in gain, not
// dB, so can't be connected to noisePos.volume.
const noisePos = new Tone.Noise().start();
const noisePosGain = new Tone.Gain();
noisePos.connect(noisePosGain);
gate.connect(noisePos._source.playbackRate);
inverseGate.connect(noisePosGain.gain);

// Opposite signal holds on positive and runs muted on negative, same connections.
const noiseNeg = new Tone.Noise().start();
const noiseNegGain = new Tone.Gain();
noiseNeg.connect(noiseNegGain);
inverseGate.connect(noiseNeg._source.playbackRate);
gate.connect(noiseNegGain.gain);

// Sum them
const output = new Tone.Add();
noisePosGain.connect(output);
noiseNegGain.connect(output.addend);

// Check it out
const meter = new Tone.DCMeter();
output.connect(meter);
setInterval(() => console.log(meter.getValue()), 10);

It just occurred to me that a single impulse in an oscillator tied to the playback rate is a lot simpler 😬. Well, sometimes the brain goes crazy.

Thanks for the reference implementation. I think a Sample-And-Hold component would be really cool. The big issue blocking your implementation for the time being is that it still doesn't seem like it'd work on Safari since in Safari you can't connect a signal to playbackRate to control that AudioParam. See this issue for more context.
I think given that, the best alternative would be to implement it with either a Tone.Clock which could schedule the playback rate as 0 or 1 each time it ticks or implement it within an AudioWorklet.

Thanks for the reference implementation. I think a Sample-And-Hold component would be really cool. The big issue blocking your implementation for the time being is that it still doesn't seem like it'd work on Safari since in Safari you can't connect a signal to playbackRate to control that AudioParam. See this issue for more context.
I think given that, the best alternative would be to implement it with either a Tone.Clock which could schedule the playback rate as 0 or 1 each time it ticks or implement it within an AudioWorklet.

Ah, thanks for the reply, good to know! Lately I've been learning C++ audio development, but if I come back to the web side of things I may pick this back up.

This is something I'm interested in too. The processing code for a latch is trivial. Here is a Max codebox example:
History x (0);

if (in2 != 0) {
	out1 = in1;
	x = in1;
} else {
	out1 = x;
}

Would it be a good candidate for making an AudioWorklet and plugging it in that way?


Please cast all buffer_put() return type

Like so:
uint32_t *p = (uint32_t * ) buffer_put(b, 4);
It would not harm C and enable a better CPP compilation.
Thank you!

C ++ should be able to use better C ++ libraries.


Add missing allocate, capacity, pop for DS\PriorityQueue

Hi @tfedor. Could you please address the changes requested here if you have spare time?

@DmitryTronin sorry for delay, should be fixed now


UI Text Input - Add icons and clear option

Description
Changes to ui-text-input:

Send message on leave/ onBlur event even if the value is an empty string, fixes #996
Add option to show the clear button and to control if it should send a message or not
Added option to add icon, setup position (right/left) and if (inside/outside)

Related Issue(s)

Checklist

[X] I have read the contribution guidelines
[ ] Suitable unit/system level tests have been added and they pass
[X] Documentation has been updated

[ ] Upgrade instructions
[X] Configuration details
[ ] Concepts

[ ] Changes flowforge.yml?

[ ] Issue/PR raised on FlowFuse/helm to update ConfigMap Template
[ ] Issue/PR raised on FlowFuse/CloudProject to update values for Staging/Production

Labels

[ ] Includes a DB migration? -> add the area:migration label

In the case of an empty string, does this send only if the content has changed (to empty)?

The events leave and the new clear have the same behaviour as pressing Enter, it always sends the current value, even if it is an empty string.
The clear button (cross) only shows if there is text.

I am surprised to see that the current code sends on focus leave even if the value has not changed, but logically I suppose that is correct.
Does it need a button for clearing? I would have thought that erasing it would be sufficient.
With the new code will it send on focus leave if the value is manually erased?

All sounds good to me :)

Minor typo fix - otherwise - excellent stuff :+1:


Summer24 Update

Add Exile, and update HNW covers.

👷 Deploy Preview for bongosian-press processing.

Name
Link

🔨 Latest commit
3272dc7b48cf07dd72dc68f16b5f93e6b81b77bb

🔍 Latest deploy log
https://app.netlify.com/sites/bongosian-press/deploys/66bcfb3608bf0b0008d43a32


File Manager Ideas

would be great to see the download speed in kb/s
is there any other way to show the diffrent drives? dropdown is not my favorite for this - you cant image if the drive is an HDD, CD or Network-Drive
i think you need to limit the maximum simultaneous file downloads - looks crazy when you select 20 or more files for download
cant remove downloaded items from the "transfer" window

Added all your ideas to #262.
Hehe, yea i tried this too with over 100 files just for fun (stability test). Gonna give this a maximum.


[Release helper] add assign policy

https://github.com/Azure/azure-sdk-for-python/issues/23537
test link: https://github.com/Azure/sdk-release-request/issues/2637

/check-enforcer override


Dev

Deploy preview processing.
Built with commit 3eec80f0dcb06afb4ade4ebc39ae8ea70127bca4
https://app.netlify.com/sites/fcccolumbus/deploys/59e7e9890b79b7122661aa98

Coverage remained the same at 100.0% when pulling 3eec80f0dcb06afb4ade4ebc39ae8ea70127bca4 on dev into 1fe3125e3836a4372f2aebd23170b1f4d6c0f0be on master.


Javalin attempts to resolve .vue components from src/main/resources/vue

Actual behavior (the bug)
I compiled a jar and created a docker container, which works perfectly. But when I access it from localhost, JavalinVue#setRootDirPathIfUnset sees that the request is coming from localhost, and tries to resolve the components from src/main/resources/, eventhough the server is running as a standalone jar.
Expected behavior
A check if it is running as a jar
To Reproduce
Create a standalone jar, run it and connect from localhost
Additional context
Add any other context about the bug here

You can set the path using JavalinVue#rootDirectory, that way Javalin won't try to guess the correct location. The reason it checks for localhost is to enable quick-reload of templates :)


Weird rendering of PPT

Take a look at https://osf.io/2gzhy/?pid=w7m2d, where the background as rendered on the Web site is ...quite strange and shows a lot of circular artifcats. thx @raynamharris for pointing it out!

This appears to be a result of converting the PPT to PDF for viewing in the browser. The OSF file renderer (MFR) uses LibreOffice to convert to PDF. I downloaded the PPT and converted to PDF in LibreOffice manually with the same result:

The problem is that the gradient used in the background does not convert to PDF well.
I found this related issue: https://gitlab.freedesktop.org/poppler/poppler/issues/626
The issue mentions PDFs with gradients rendering properly in Acrobat Reader, but I see the same result there:

Unfortunately, there's not much we can do until LibreOffice fixes this issue. It's possible that exporting to PDF with PowerPoint may yield better results. If so, you could upload that in place of the PPT file.

thanks for looking into it!


Create a team PR to [nus-cs2103-AY2223S2/tp] master branch.

Follow the format:
PR name: [Team ID] Product Name e.g., [CS2103-T09-2] InsureList. As you merge code to your team repo's master branch, this PR will auto-update to reflect how much your team's product has progressed.
PR subject: the name of your product e.g., [CS2103-T09-2] InsureList
Description: a 1-2 sentence overview (plain text only, no formatting or links) of your project indicating the target user and the value proposition e.g., InsureList helps insurance agents manage detail of their clients. It is optimized for CLI users so that frequent tasks can be done faster by typing in commands.

@AY2223S2-CS2103T-W12-1/developers anyone wants to take up this issue?

I have created a PR


Add metav1.List decoding on YAML Kubernetes serializer

Closes #88
This PR adds support for decoding metav1.List objects. It checks if the decoded objects are an Unstructured or UnstructuredList, in the case is the second one it will get all the Items and treat them as regular Unstructured resources.

Codecov Report

Merging #89 into master will decrease coverage by 0.02%.
The diff coverage is 77.77%.

@@            Coverage Diff             @@
##           master      #89      +/-   ##
==========================================
- Coverage   77.21%   77.19%   -0.03%
==========================================
  Files          20       20
  Lines         856      864       +8
==========================================
+ Hits          661      667       +6
- Misses        148      150       +2
  Partials       47       47

Impacted Files
Coverage Δ

internal/kubernetes/yaml.go
84.61% <77.77%> (-2.49%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 12adff7...b4d9380. Read the comment docs.


Do not send outdated MNs on sync

Makes sense but is there any reason to think this is causing problems ?

I noticed that node was burning cpu and spamming debug.log processing few hundreds of outdated nodes which all came "in one pack".  I guess it was also busy requesting invs at the very same time and as a result of this coincidence sync timed out too early. Even if that was not the main reason I guess it's no good to spam nodes with outdated MNs anyway (could probably add Misbehaving on receiving side too to mitigate this better but I wouldn't push that without protobump and without rolling out on testnet first).

UPDATE_REQUIRED notes only represent about 10% of the total do I don't see this having a large effect.
Still I don't see a downside so, utACK.


Asymmetric Crosslinking Reagents

Related to #1
We have developed crosslinkers with asymmetric cleavage sites that produce twin doublet peaks which we rely on to isolate CSMs that contain crosslinks. Are there any plans to incorporate these types of crosslinkers in SCOUT?

Yes, we have plans to adapt Scout to perform searches by using this kind of cross-linkers.
As soon as we release a new version with this feature, I'll let you know.


fix: bug in showing element in context

This commit includes:

Remove unused else if, as the logic for getting the prediction has changed.
Adding an additional condition for the prediction's falsy value to match the values that are coming from the backend API, which are string values in case of false or true.

Codecov Report

Merging #144 (79f7211) into main (354aed9) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #144   +/-   ##
=======================================
  Coverage   81.95%   81.95%
=======================================
  Files          56       56
  Lines        3392     3392
=======================================
  Hits         2780     2780
  Misses        612      612

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 354aed9...79f7211. Read the comment docs.


[automation][jenkins] Automate jar deployment process

Hi Franklin, I am currently unassigning you since there is no updates for a long time. If you want to contribute please comeback and self assign.


missing c_types.h and/or stddef.h?

I am using your esp-open-rtos in eclipse and really like it, thanks a lot for all the work :-)
What i find strange, is that i can't find the typedef for "size_t" inside my included headers.. and i included alot:
esp-open-rtos/include
esp-open-rtos/FreeRTOS/Source/include
esp-open-rtos/FreeRTOS/Source/portable/esp8266
esp-open-rtos/libc/xtensa-lx106-elf/include
esp-open-rtos/core/include
esp-open-rtos/open_esplibs/include
esp-open-rtos/lwip/include
esp-open-rtos/lwip/lwip/src/include/ipv4
esp-open-rtos/lwip/lwip/src/include/posix
esp-open-rtos/lwip/lwip/src/include/lwip
esp-open-rtos/lwip/lwip/src/include
there is a c_types.h from the original SDK at  esp-open-rtos/lwip/lwip/espressif/include, but thats somehow counter-intuitive to use that one.
What i also miss is a definition for va_list. Is it not supported yet?

size_t defined in <your esp-open-sdk path>/xtensa-lx106-elf/lib/gcc/xtensa-lx106-elf/4.8.5/include/stddef.h
You don't need to add this path manually, compiler does it. Just #include <stddef.h>

Duh, didn't think about the toolchain-includes since I didn't configure the toolchain in Eclipse!
Thanks a lot!


feat(STONEINTEG-314): add integrationTestScenario test in MVP-demo

Description
This is to add integrationTestScenario CR in the application and verify the related test PipelineRun is created/Excuted in mvp-demo test suite.
Issue ticket number and link
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
Checklist:

[ ] I have performed a self-review of my code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added meaningful description with JIRA/GitHub issue key(if applicable), for example HASSuiteDescribe("STONE-123456789 devfile source")
[ ] I have updated labels (if needed)

/retest

The recent error might be related to the previous one and there's an ongoing discussion around that in this slack thread.

/retest

/retest

/retest

/retest

/retest

/retest

/retest

/retest

/retest

Rebase on the latest main please

/retest

/retest

The CI complained [here]:
could not find Release CR based on associated Snapshot 'mvp-test-app-lvz42' in 'mvp-dev-ywzn-tenant' namespace

:thinking:

/retest

/retest-required

/retest-required

/test redhat-appstudio-e2e

/retest

/test redhat-appstudio-e2e


Webassembly backend (WIP)

This PR adds a Webassembly backend to the QCert compiler framework. The PR is WIP.
We have a discussion thread in the Accord project's slack: https://accord-project.slack.com/archives/C013AS35ZU3

@pkel Do you know if the OCaml wasm package require a special purpose opam repository?
File "compiler/dune", line 12, characters 32-36:
12 |  (libraries base64 uri calendar wasm)

I think just need to add wasm to the CircleCI script here:
https://github.com/querycert/qcert/blob/a3891edc8c9297ec2b126927ef68018fe2142434/.circleci/config.yml#L41

@pkel Do you know if the OCaml wasm package require a special purpose opam repository?
File "compiler/dune", line 12, characters 32-36:
12 |  (libraries base64 uri calendar wasm)

No, but you need a recent OCaml (>=4.08.0)
https://opam.ocaml.org/packages/wasm/


Typo on website [Magic]

🐛 Bug Report
Write a description of the bug here, give all the information you can about what it is, how it happened.
"...you will have a 3x boost when chanrging glories."

fixed


Merge refs/heads/refactoring/require-admin into alpha

on push to any branch that is not alpha, create a pull request to merge the branch into alpha

This PR has been merged into alpha and tagged as 1.5.47-alpha-202303042256. Thanks for your contribution!


Refactor Mutation API

Context
This PR refactors the API for mutation consuming resources so that the use of Snapshots is transparent to the end user.
Under the hood we continue to use Snapshots.
This means that we do not reference a component in the spec for the object. An explicit kind must always be declared for sourceRef and configRef  fields.
For example a Configuration  will now look as follows:
apiVersion: delivery.ocm.software/v1alpha1
kind: Configuration
metadata:
  name: deployment
spec:
  interval: 10m0s
  sourceRef:
    kind: Localization
    name: podinfo
  configRef:
    kind: ComponentVersion
    name: podinfo
    resourceRef:
      name: config

We declare a MutationObject interface with GetSpec() and GetSnapshotDigest() methods. This allows us to watch snapshots and
also define/handle the mutation spec in a consistent way.
Signed-off-by: Piaras Hoban<EMAIL_ADDRESS>

Nearly there...
Just need to address one or two bugs and the merge conflicts...
go test -v -count=1 ./e2e
2023/04/20 22:24:01 --- starting e2e test suite
=== RUN   TestOCMController
    main_test.go:33: running e2e ocm-controller tests
=== RUN   TestOCMController/Configure_Management_Repository
Handling connection for 3000
    env.go:422: successfully created repository at url http://localhost:3000/e2e-tester/ocm-controller-test.git
    env.go:422: Created token secret flux-system/ocm-controller-test
    env.go:422: Created git repository flux-system/ocm-controller-test
    env.go:422: Created kustomization flux-system/ocm-controller-test
=== RUN   TestOCMController/Create_Component_Version
    env.go:422: successfully added apps/component_version.yaml to repository ocm-controller-test
=== RUN   TestOCMController/Create_Component_Version/check_that_component_version_is_ready
    env.go:422: checking if componentversion podinfo is ready...
=== RUN   TestOCMController/Create_Resource
    env.go:422: successfully added apps/resource.yaml to repository ocm-controller-test
=== RUN   TestOCMController/Create_Resource/check_that_resource_is_ready
    env.go:422: checking if resource deployment is ready...
=== RUN   TestOCMController/Create_Localization
    env.go:422: successfully added apps/localization.yaml to repository ocm-controller-test
=== RUN   TestOCMController/Create_Localization/check_that_resource_is_ready
    env.go:422: checking if localization deployment is ready...
=== RUN   TestOCMController/Create_FluxDeployer
    env.go:422: successfully added apps/deployer.yaml to repository ocm-controller-test
=== RUN   TestOCMController/Create_FluxDeployer/check_that_flux_deployer_is_ready
    env.go:422: checking if fluxdeployer podinfo is ready...
    env.go:422: checking if oci repository podinfo is ready...
    env.go:422: checking if kustomization podinfo is ready...
--- PASS: TestOCMController (42.33s)
    --- PASS: TestOCMController/Configure_Management_Repository (0.53s)
    --- PASS: TestOCMController/Create_Component_Version (10.41s)
        --- PASS: TestOCMController/Create_Component_Version/check_that_component_version_is_ready (10.00s)
    --- PASS: TestOCMController/Create_Resource (5.45s)
        --- PASS: TestOCMController/Create_Resource/check_that_resource_is_ready (5.01s)
    --- PASS: TestOCMController/Create_Localization (10.46s)
        --- PASS: TestOCMController/Create_Localization/check_that_resource_is_ready (10.00s)
    --- PASS: TestOCMController/Create_FluxDeployer (15.48s)
        --- PASS: TestOCMController/Create_FluxDeployer/check_that_flux_deployer_is_ready (15.02s)
PASS
ok      github.com/open-component-model/ocm-controller/e2e      157.493s


A11y: On narrower screens the Filter button has no text & announces as just "button"

Describe the task
When using a screenreader to navigate through the property page buttons they should always announce what they are, "button, Filter" or "button, Download". On desktop screen widths we show the text within the button, and that text is announced. When the screen is narrower, the text disappears. The download button has an aria-label, so it announces. The Filter button needs an aria-label="Filter" as well.
Acceptance Criteria

[ ] Go to the Find Properties page
[ ] Resize the screen so that it text in the button disappears.
[ ] Turn on screenreader. Navigate to the Filter button.
[ ] It announces as "Filter button"

I am ready to take this issue, I can tackle this

@10234567Z Assigned!


Web UI permissions overview should allow for different group name prefixes

User story

As an admin, I want to inspect permissions for arbitrary RBAC groups.

Details
Currently the permission inspection UI in its "Group" mode restricts the user entry to group names starting with customer:. This is the name prefix assigned to all group names for users authenticated via OIDC via the Dex connector with the ID customer.

This restriction already affects us (as Giant Swarm staff) currently when testing and using a different Dex connector, in our case giantswarm.
Soon we will allow for additional Dex connectors to be configured.
Questions

How about not restricting entry at all? In that case we should provide guidance, as users would be likely to omit the group prefix, leading to wrong results (in most cases: no permissions displayed).

Such guidance could be given for example if we don't find a : (colon) in the group name, in the form of an error message.

Alternatively, can the web UI provide a list of all valid prefixes to choose from via e. g. a dropdown menu?

If the user has list access to RoleBinding and ClusterRoleBinding resources, we can collect prefixes from those. Might be a slow procedure. What about users who don't have access to these?

The description looks correct to me. I would vote for the first option (without a restriction and with an error in case a group name doesn't contain a :). Additionally, we can have the field initially pre-filled with customer:, the same way we do it right now, but allowing a user to modify it. I think it would remind about the group name format.


[BUG] Multi-part upload API takes as much memory as the size of parts being uploaded

Describe the bug
The attached test project is quite simple and can be built using maven.  It shows that more memory is needed in the JVM than the size of file parts being uploaded, despite using a "markable" input file stream to try to make the library not buffer everything.  It does anyway, apparently.
My observation is that the SDK will use memory equal to the total size of all parts being uploaded in parallel, excepting it seems to max out at 1GB per part.
Exception or Stack Trace
The OOM stack looks something like
Exception in thread "main" java.util.concurrent.ExecutionException: java.lang.IllegalStateException: io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 16777216 byte(s) of direct memory (used:<PHONE_NUMBER>, max:<PHONE_NUMBER>)
        at java.util.concurrent.FutureTask.report(Unknown Source)
        at java.util.concurrent.FutureTask.get(Unknown Source)
        at net.redpoint.test.Test2.main(Test2.java:100)
Caused by: java.lang.IllegalStateException: io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 16777216 byte(s) of direct memory (used:<PHONE_NUMBER>, max:<PHONE_NUMBER>)
        at io.netty.channel.AbstractCoalescingBufferQueue.releaseAndCompleteAll(AbstractCoalescingBufferQueue.java:361)
        at io.netty.channel.AbstractCoalescingBufferQueue.releaseAndFailAll(AbstractCoalescingBufferQueue.java:208)
        at io.netty.handler.ssl.SslHandler.releaseAndFailAll(SslHandler.java:1907)
        at io.netty.handler.ssl.SslHandler.setHandshakeFailure(SslHandler.java:1886)
        at io.netty.handler.ssl.SslHandler.setHandshakeFailure(SslHandler.java:1851)
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:780)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:750)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:742)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:728)
        at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.flush(CombinedChannelDuplexHandler.java:531)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:125)
        at io.netty.channel.CombinedChannelDuplexHandler.flush(CombinedChannelDuplexHandler.java:356)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:750)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:742)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:728)
        at reactor.netty.channel.MonoSendMany$SendManyInner.run(MonoSendMany.java:324)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Unknown Source)
        Suppressed: java.lang.Exception: #block terminated with an error
                at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:139)
                at reactor.core.publisher.Mono.block(Mono.java:1731)
                at com.azure.storage.common.implementation.StorageImplUtils.blockWithOptionalTimeout(StorageImplUtils.java:130)
                at com.azure.storage.file.datalake.DataLakeFileClient.appendWithResponse(DataLakeFileClient.java:480)
                at net.redpoint.test.Test2.lambda$main$0(Test2.java:93)
                at java.util.concurrent.FutureTask.run(Unknown Source)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
                ... 1 more
Caused by: io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 16777216 byte(s) of direct memory (used:<PHONE_NUMBER>, max:<PHONE_NUMBER>)
        at io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:802)
        at io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:731)
        at io.netty.buffer.PoolArena$DirectArena.allocateDirect(PoolArena.java:648)
        at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:623)
        at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:202)
        at io.netty.buffer.PoolArena.tcacheAllocateSmall(PoolArena.java:172)
        at io.netty.buffer.PoolArena.allocate(PoolArena.java:134)
        at io.netty.buffer.PoolArena.allocate(PoolArena.java:126)
        at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:395)
        at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:188)
        at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)
        at io.netty.handler.ssl.SslHandler$SslEngineType$1.allocateWrapBuffer(SslHandler.java:231)
        at io.netty.handler.ssl.SslHandler.allocateOutNetBuf(SslHandler.java:2256)
        at io.netty.handler.ssl.SslHandler.wrap(SslHandler.java:823)
        at io.netty.handler.ssl.SslHandler.wrapAndFlush(SslHandler.java:797)
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:778)
        ... 17 more

To Reproduce
Build the attached project:
mvn install
Run it and tell it to use 2GB of memory, uploading 3 x 1GB parts in parallel
java -Xmx2g -jar target/*shad*jar 3 1000
It will also fail with 12 x 100MB parts and 1GB of memory
java -Xmx1g -jar target/*shad*jar 12 100
Code Snippet
See attached sample
Expected behavior
Since the part upload data is on local disk, I expect the upload process to use a lot less memory, like maybe 8MB per part, since it can easily rewind and find the data on disk.
Screenshots
n/a
Setup (please complete the following information):

OS: Windows 10
IDE: n/a
Library/Libraries: See dependencies in attached project
Java version: 8
App Server/Environment: n/a
Frameworks: none

Additional context
Given that we are also facing https://github.com/Azure/azure-sdk-for-java/issues/27473, we are really in a bind with ADL2.  We seem to have to choose between an unstable upload and one that uses great gobs of memory.

multi_part_upload.zip
See attachment

@wheezil Thank you for opening this issue. The append method you're using in this sample does indeed do a lot of buffering, and that is by design. It is intended to be able to support non markable streams, and it doesn't have any awareness of the fact that the data is on the disk. It has to do a fair amount of buffering both to support retries in the general case and also to have a full payload to send.
I think focusing on one issue at a time will give us the best progress, and I think the better method for your scenario is uploadFromFile, so let's focus there for now.

@rickle-msft I would certainly love to have the uploadFromFile() bug fixed, as it seems like a better solution.
However, I have to disagree with your statement that "doesn't have any awareness of the fact that the data is on the disk".  The SDK code is keenly aware that stream is markable, and complains when it is not.  Yet, despite insisting that you give it a markable stream, it takes no advantage of that capability and buffers all of the data in memory anyway.  Frankly I can kind of see why... the "markable" stream concept is weak and barely useful, kind of retrograde from 1980.  It would be much better to find the underlying SeekableByteChannel and use it for random access.
Anyway, call this an ER, call it a bug, PO-TAY-TOH, PO-TAH-TOH.  Using this much memory is a surprising behavior that IMHO should be addressed at some point, or at least document this isn't suitable for large parts unless you have a lot of memory.
Cheers,
john

Oh, and speaking of documentation... I'm pretty sure at some point I read a suggestion (either a support response, or online doc) that you wrap your FileInputStream in a BufferedInputStream to make it markable, but this makes it use double the memory as the size of the data!  We discovered this surprise in production a while ago.

We don't use a BufferedInputStream anywhere in the Storage sdks, so we aren't doing any wrapping. We also do all our buffering using async logic, so we wouldn't use streams anyway, but yes, we do have to do some buffering. We do suggest in the docs that the customer use a BufferedInputStream to make their stream markable if the stream is not already markable and they don't have another means of making it markable, but they are also free to implement a more memory efficient wrapper to support marking. We do check that the stream is markable (hence the docs suggestion), but it doesn't have any awareness of the means of supporting marking; there is no knowledge that the data is on disk or simply in memory or another third option.
We do make use of marking and resetting to support retries as we reset to the mark before reading from the data. However, there isn't a way to get the memory straight from the input stream and put it on the network stream. It has to be read from the stream into a buffer first, hence why we are still instantiating our own buffers. Additionally, not all streams are backed by a SeekableByteChannel, so we can't rely on that.
The purpose of marking and resetting in this case is to enable the ability to retry uploads, not to necessarily to save on memory. Granted, we shouldn't use more memory than we need, but the insistence on a markable stream is not primarily a memory optimization issue.
There are multiple upload options, each written with a different scenario in mind:

uploadFromFile. The data is assumed to be coming from a file. We can take advantage of greater parallelization by reading from different parts of the disk pretty much simultaneously. We can also afford to do less buffering as we can read from disk and have more information about the data from the file's metadata that helps us make a number of decisions about our strategy.
parallel uploadFromStream, often called buffered upload: This can take any arbitrary stream as its source and will upload in parallel. Because it is so generic, we have very little room to assume anything about the data. Hence, we buffer the stream both to ensure retryability, but also to be able to break the stream up into chunks to support parallelized uploads. These will tend to be called upload and take a ParallelTransferOptions
Standard upload: These are the methods that are meant to match the REST API 1 to 1. The append method you call in the sample is of this nature. We don't do anything around parallelism or anything, just put the bytes on the wire. That said, there are a couple constraints. 1. The stream must be markable. Primarily for retry support as mentioned. It is a policy of the sdks to always ensure retries are possible given how frequently they are necessary in the wild. 2. All http clients are ultimately async http clients in this sdk. That means we have to convert the stream into an async object, which requires some buffering. While we don't eagerly read the whole stream into memory at once, depending on a number of configurations on the machine, that may be what ends up happening in effect. I believe azure-core is working on a fully sync stack to decouple the sync apis from a dependence on the async apis, which should hopefully remove some of the cost of this conversion.

@rickle-msft  thanks for the explanation!  I do find the SDK to be rather expansive and I'm sure there is more there than I am aware of.
Do you think that, in the fullness of time, we might be better off working with the core/sync API you mentioned?  I have often found that the closer we are to the metal, the more control we have over behavior, but it comes at the expense of needing to deal with more details (like HTTP response codes, exception classes, etc) that are often rather detailed and finicky.  Clearly at some level we could just use basic HTTP calls and read/write directly to the stream, but I doubt we have the fortitude to program at the assembly-language-of-REST.  But we might find benefit from working at a level of the API where there is less magic, if that seems like a viable thing.  We are OK getting our hands dirty!
Thanks
john

After an offline conversation, we've agreed that the current best option is to wait for a preview of the sync http stack and see what perf improvements that yields.

Another alternative that we discussed but is not immediately necessary is the idea of an appendFromFile method which takes in a range for reading from a file on disk and appending to an azure file. This way we can get some optimizations from being able to read from the disk but it's not an all or nothing file transfer.

Agreed, we have a go-to-production approach which is non-optimal in that there are some time/space tradeoffs we would rather not make, but it is workable.  The background story is that we are implementing an OutputStream that writes to ADL2 using multi-part uploads, so we don't know how big the data is until we are done.  Basically we are starting with a small part size and high parallelism to get performance.  But as the file grows larger we increase part sizes to avoid the 50k part limit, and at that point we may need to reduce parallelism to conserve memory, depending on the JVM settings.  Please do keep me apprised about developments in the sync API and we'll try it out.

@rickle-msft , is the sync http stack available yet?  I would like to try it.

Anyone, has there been any progress on a sync API SDK?


Safari 15.2

Following #6107 here are the changes B-)
https://developer.apple.com/documentation/safari-release-notes/safari-15_2-release-notes
"Added support for the File System Access API for creating and iterating files in an origin-specific storage space."
The situation appears to be confusing and frankly kind of a mess

https://twitter.com/tomayac/status/1471255239894392836 (the thread in general)
https://github.com/WICG/file-system-access/issues/342#issuecomment-995699770
https://github.com/WICG/file-handling/issues/70

I think we either could represent Safari as having no support or partial (not outside the sandbox/origin it appears), but since we had partial before I would think of partial here as hint for developers to check out whats possible…
Re-enabled SharedArrayBuffer for sites that use COOP and COEP securely to create an isolated environment.
✔
Added wide gamut support for canvas.
=> #6110
Added support for COOP and COEP HTTP headers.
Via MDN:

https://caniuse.com/mdn-http_headers_cross-origin-embedder-policy
https://caniuse.com/mdn-http_headers_cross-origin-opener-policy

There doesn't appear to be an issue/PR just yet, maybe I'll do it… Has been adjusted in https://github.com/mdn/browser-compat-data/pull/14006, and is in main (e.g. COEP), but not live yet it seems: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cross-Origin-Embedder-Policy#browser_compatibility

Looks good, thanks for the research!