[$500] Android - User allowed to paste text in BNP page for a second

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 1.4.24-0
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: Applause internal team
Slack conversation:
Action Performed:

Launch app
Tap fab----Request money/send money
In BNP, paste text in amount field

Expected Result:
User must not be allowed to paste text in BNP page for a second.
Actual Result:
User is allowed to paste text in BNP page for a second.
Workaround:
unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence
https://github.com/Expensify/App/assets/43996225/8909149c-13b2-4f08-9463-a51ad74acd0e

View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~014d093bb089ba85dd
Upwork Job ID:<PHONE_NUMBER>937509888
Last Price Increase: 2024-01-11

Proposal
from: @unicorndev-727
Please re-state the problem that we are trying to solve in this issue.
What is the root cause of that problem?
The root cause is that shouldUpdateSelection is initially true so that setSelection works at first loading in MoneyRequestAmountForm.
https://github.com/Expensify/App/blob/7ae1ec7e74b908e6340bccaf7e42b0345d766711/src/pages/iou/steps/MoneyRequestAmountForm.js#L90
What changes do you think we should make in order to solve the problem?
We need to set shouldUpdateSelection to false.
What alternative solutions did you explore?
N/A

Proposal
Please re-state the problem that we are trying to solve in this issue.

User allowed to paste text in BNP page for a second

What is the root cause of that problem?

When we paste the text, all the text will be put to the input field for a while, it is the default behavior of the native device.

What changes do you think we should make in order to solve the problem?

Once we long press on the text input, we should clear the Clipboard if it contain the invalid amount.
For example, update:
https://github.com/Expensify/App/blob/7ae1ec7e74b908e6340bccaf7e42b0345d766711/src/pages/iou/steps/MoneyRequestAmountForm.js#L279
to:

// Pseudo code
<TextInputWithCurrencySymbol onFocus={()=> {
if(!MoneyRequestUtils.validateAmount(Clipboard.getString()) Clipboard.setString('')
}}

What alternative solutions did you explore? (Optional)

NA

Proposal
Please re-state the problem that we are trying to solve in this issue.
User allowed to paste text in BNP page for a second
What is the root cause of that problem?
It is the behavior of react-native TextInput
What changes do you think we should make in order to solve the problem?

Add function shoulHideContexMenu

const shoulHideContexMenu = useCallback(async () => {
     const currentClipboard = await Clipboard.getString()
     return !MoneyRequestUtils.validateAmount(currentClipboard, decimals)
},[]}

Add prop to TextInputWithCurrencySymbol
contextMenuHidden={shoulHideContexMenu()}

What alternative solutions did you explore? (Optional)

@fedirjh My proposal does not "disable pasting for native", can you help check again or mention, what section in my proposal lead to your opinion bellow:

I don't think we should disable pasting for native

@fedirjh Let me know if you need my help to test the solution

@shahinyan11 Your proposal didn't worked for me , even when contextMenuHidden is set to true , I am still able to elect the text and trigger the context menu.

Are you done all 5 steps ?

@fedirjh After investigating, I think that my proposal is just workaround. So I think we need to hire the proposal that uses the native code, or we can consider closing this one because it is a minor issue

@fedirjh
https://github.com/Expensify/App/assets/46921547/9c111486-a08b-45e3-ac99-b967bc2b5f7f

@shahinyan11 Suppose that the contextMenuHidden works as you expected. But I think that, the context menu contains not only the "Paste" button. but also the "Copy", "Cut", ... So if we hide the context menu, we also can not interact with the rest buttons

@DylanDylann I totally agrees with you , any solution beside native code is just a workaround.
@CortneyOfstad Should we close ?

Sorry for the delay here — I think we can close in this scenario 👍


Space between symbol and value

I'm using the "pt-br" location, I'd like to add a space between the symbol and the value. When I add space it goes wrong.
Current example: R $1.000,1236 // space between R and $, but not value
*Correct example: R$ 1.000,1236 // current space between R$ and value
How can I adjust this?

The simillar case. I'm using "cs" locale, when I try to format value I get:

@kamihouse Seems to have fixed and made pull request but the same is not accepting, having accumulated several pull requests. Weird.

The library works fine for single letter currency symbols. Spacing issues for multi letter currency symbols has been fixed in this PR #461 . Simply replaces $ with current locale symbol. Depends on you were you put $ symbol

I don't know if it's the same problem, but I find another issue with space between value and symbol:
numeral("999999").format("0[.]0 a") -> 1 k
numeral("999999").format("0[.]0a") -> 1m

fixed in 2.0.5


sick_safevisionary_base: 1.0.1-1 in 'noetic/distribution.yaml' [bloom]

Increasing version of package(s) in repository sick_safevisionary_base to 1.0.1-1:

upstream repository: https://github.com/SICKAG/sick_safevisionary_base.git
release repository: https://github.com/SICKAG/sick_safevisionary_base-release.git
distro file: noetic/distribution.yaml
bloom version: 0.11.2
previous version for package: null

sick_safevisionary_base
* Update top-level readme with build instructions (#2 <https://github.com/SICKAG/sick_safevisionary_base/issues/2>)
  * Update top-level readme with build instructions
  * Mention the *send when* option in the safety designer
  * Extended tutorial on initial safety designer setup
  * Fix duplicate .png extension
  * Added structure and fixed minor errors
  ---------
  Co-authored-by: Marvin Große Besselmann<EMAIL_ADDRESS>* Deactivate the CI pipelines for testing
* Add CI pipelines for ROS2 Iron and Rolling
* Add build and license badges to top-level readme
* Add CI pipeline for Noetic and Humble
* Fixed License file and added maintainer in package xml
* Updated license
* Changed ci to master branch
* Added ci script
* clang formate/tidy
* Adjusted CMakeList file to support client library function
* Moved header into seperate include folder
* fixed package name
* Removed old copied files
* Initial commit
* Contributors: Jochen Lienhart, Marvin Große Besselmann, Stefan Scherzinger

Closed as duplicate of https://github.com/ros/rosdistro/pull/38776

@quarkytale Please reopen :)
This is not a duplicate of #38776. It's the driver's library that is also required to run the sensor. https://github.com/ros/rosdistro/pull/38776 will not work without this pull request here..

Ohh really sorry, my bad, read the package name wrong


Update Streams index page table, add tooltip for status badges

Related to #786, in that we want to use the same badge/tooltip approach as described in that ticket. Aside that that part, there are a handful of minor table arrangement updates.
Table updates

[x] A: Move the "Name" column to be the first two column of the table.
[x] B: Add a Status heading for the second column (it doesn't currently have a heading, and is the first column).
[x] C: Add an Actions column heading, centered, for the last column that displays the "Make default" and "Destroy" buttons (if the user has permissions to those).
[x] C: Swap order of the the "Make default" and "Destroy" buttons.
[ ] D: I'm not sure if this is represented in another ticket, but I believe we want to prevent any stream that was previously a default from being made the default again. So we should not display the "Make default" button for previously default streams.
[ ] E: Change the label of the "Destroy" button to "Delete".
[x] F: When a stream was previously the default stream, add a Previous default badge to its Status column. Styling is badge bg-warning text-dark (I forgot to update the badge label color in my mockup, but it should be dark).
[x] F: Update the current default stream badge with the text-dark class so its label is also dark text.
[x] F: For both badges, add an info-circle-fill icon to the badge.
[x] I'm not sure how the current table is sorted, but it probably should be changed to sort by the "Last updated" column, most recent first (my mockup doesn't show it that way, just because I was following the current implementation).

Add tooltip to badge to display default stream date range

This is the same as described in #786:

[ ] Trigger a Bootstrap tooltip by mouseover/keyboard selection of the info icon in the stream status badge.
[ ] If the stream is a previous default, display the start and end date/time values separated by "to.
[ ] If the stream is the current default, use the text Default since followed by the date/time it became the default stream.
[ ] A previous default stream might have more than one date range when it was previously the default. If so, display each date range on a separate line.

Note that the requirement to display multiple date ranges means we will likely need to use HTML elements inside the tooltip markup, such as wrapping each date range line in a <div> or <p>.  Also, to ensure the tooltips work for keyboard and screenreaders, we might have to wrap the badge tooltip in a link. See https://getbootstrap.com/docs/5.0/components/tooltips/#markup for more about this.

@ggeisler thanks for all these details, getting close! A couple things:

change Destroy to Delete -- last week @corylown  and I went through the app and aligned permissions for delete vs destroy, this button triggers the destroy on the backend, so I'm hesitant to change it again

For now we are assuming a stream can be the default more than once, so I'm leaving the button for now

@marlo-longley

change Destroy to Delete -- last week @corylown and I went through the app and aligned permissions for delete vs destroy, this button triggers the destroy on the backend, so I'm hesitant to change it again

Hmm, not sure we are talking about the same thing here. I'm suggesting we change the button label to "Delete" simply because "Destroy" is a default Rails term that is developer jargon; it makes more sense to developers than to end users who have no knowledge about the underlying system (i.e., most web applications enable their users to remove something by "deleting" it, not "destroying" it; end users don't normally think in terms of "destroying" a post or comment or whatever in a web app). I'm not suggesting the action triggered by the button is any different. Selecting the button labeled "Delete" would do the same destroy-the-stream action as it currently does.
I guess I don't know what this means:

aligned permissions for delete vs destroy

because I don't know how they are different?

For now we are assuming a stream can be the default more than once, so I'm leaving the button for now

Sure, as long as the UI reflects how we want to handle making streams the default, sounds fine.

Thanks @ggeisler
I guess the word Destroy is already common throughout buttons the app, and it also aligns with developer jargon/backend actions. For example see the /organizations page, where admins can Destroy an organization. I understand you are only talking about the label for the button on this page. I will change the label to Delete if that's your preference.

I plan to go through the app for consistency on things like button labels at some point, but I wanted to wait until we have more of the intended UI changes complete. So this isn't an intentional one-off label inconsistency, it's just one step in the process of improving the UI from its MVP state.
To be clear, from a UX point of view consistency of button labels with how Rails and developers refer to the corresponding actions is not as important as providing our end users with labels that are clear to them. For example, while there are probably isolated, overlooked exceptions, I don't think we use the term "Destroy" for buttons in any of the SUL applications our team works on.

Sounds good thanks for clarifying! @ggeisler


fix: added possible switch to how docker debian installs packages for stretch security updates, which have been acrhived

Related: https://stackoverflow.com/questions/76094428/debian-stretch-repositories-404-not-found
We might just want to switch off stretch itself and not do it this way
bullseye:

buster:

stretch:

Only items outstanding were to verify that the stretch updates repo is now gone:
https://lists.debian.org/debian-devel-announce/2023/03/msg00006.html also seems to indicate this (also from soverflow thread)
Another thread, and another
I think it also looks missing when looking at the ftp stores, so I think it's just gone as both the threads seem to indicate


Add providerPostfix configuration

Closes https://github.com/rrousselGit/riverpod/issues/1851
Add the configuration to be able to have another postfix than "Provider".

@rrousselGit  I'm creating a draft PR because I would have several questions before going further with this:

[ ] Do you prefer "postfix" or "suffix"?
[ ] How would you want me to write a test for that?
I could create a "golden" test like what is done in riverpod_graph: I create a sub-package / app with its own pubspec.yaml and build.yaml (so I can modify the build options).
I could for example copy the sync.dart file in it. But then do I also copy the sync_test and use all the renamed providers? This doesn't seem so right to me. Do you have any suggestions?

I didn't see there already was a PR for that ... 🤦


Implement ways to reset tracker without having to open Server window.

Either through tapping the tracker, controller button press and gesture, or any other potential idea.
This will be a huge QOL upgrade to people who moves a lot, having the need to reset tracker in multiple succession for experimenting tracker adjustments, or unlucky enough to just have a lot of drift happening to them, worth to be in the issue here as a constant reminder.
Update 1: Tap function for BNO is too inconsistent, this has be done with gesture/button press or both.

Is it fine if it's Fast Reset not full reset? So there is less chance to screw up alignment by body not being pefrectly straight when doing it?

Is it fine if it's Fast Reset not full reset? So there is less chance to screw up alignment by body not being pefrectly straight when doing it?

Preferably Fast Reset actually, it's been working okay for me for the past week, but it's probably good to always have a choice.

 Reset without SlimeVR window or OVRAS keybind.

I would love it to have integration with steam controller binding via feeder or overlay though...

I would love it to have integration with steam controller binding via feeder or overlay though... But maybe it's overkill at this point, we have a bunch of options now. It's technically implemented, yes.

This is implemented in the feeder now, closing this issue.


CPP: AV Rule 114 test cases

Test the __builtin_complex issues in the AV Rule 114.ql test (as they occurred in https://github.com/Semmle/ql/pull/2077).  This was fixed (and tested) by https://github.com/Semmle/ql/pull/2115 / https://git.semmle.com/Semmle/code/pull/34689, but I felt it's worth having query tests for the problem as we encountered it, in addition to the library tests.

The tests that are running now, did they start before or after #2077 was merged?

The tests that are running now, did they start before or after #2077 was merged?

I believe I made this PR a few minutes before merging #2077, so it should have started before.  I suppose it might be worth running them again afterwards (does the test runner test on the PR branch as it is, or the PR branch rebased/merged onto latest master?).

It tests on a merge of master and the PR head. I'll trigger it to run again just to be safe.

The history isn't clear on here.

It looks like you did.

Yes, and the tests passed.


Riot Alternative?

As Riot has suffered a bit of a slowdown in the last few years, I'm wondering if there's another package that you guys recommend that has the same high-level goals? (speed/expressive/simple).

I don't know where @gus stands on this, but rspec is not super kludgy anymore like it was at the time Riot came out. I haven't done any ruby work in about 6 months at this point, but the latest rspec stuff is fairly simple and fast nowadays.
I also am a fan of simplebdd if you want something totally declarative. There are plugins for vim to make using simplebdd easier.

"Minitest" sounds like it has some similar goals, as well.


Highlight correct in-page TOC item when clicking on link at bottom of page

Right now, clicking on any links that go to the bottom of the page will only highlight the closest section to the top instead of the section clicked in the TOC

When I click to go to the last nav item, if its content is short enough that the top of the window is "within" the section above the one I clicked to go to, that other section's name in the ToC is still shown as selected. But the URL hash shows the actual section I selected. So the actual section I clicked in the right sidebar should be shown in the selected style, not the section previous to it.

Can this be closed?  (Also, could someone add me as an admin on this repo?)


Module "request" is a dependency, but not installed by "npm install"

Adding this module causes MM2 not to start, complaining about missing module "request" even after running npm install in the module directory.  Presumably this is an undeclared dependency which needs to be added to package.json.

Found the answer in another thread. Ran "npm install request" in the module folder.
https://github.com/73cirdan/MMM-rainfc/issues/15


[format] Improve internationalization

Need to update format, particularly the formatNumberAsStyle to make sure it handles other currencies (ex. € vs $, and possibly other separators (like commas))
See discussion

Implemented via https://github.com/techniq/svelte-ux/pull/134


Potential improvements to I/O wrt /tmp/artifacts and /tmp/src

I've been playing a lot with incremental builds recently and trying to optimize some of the time spent in them, here's some of the things I've noticed and potentially some improvements that can be made there:

/tmp/artifacts and /tmp/src are added to the builder, nearly every builder must rm -rf /tmp/artifacts /tmp/src as part of assemble

this is pretty easy to forget and can cause images to balloon
not only that, the rm -rf ... call can be slow

When using --copy, the entire source directory is copied to /tmp/s2i#### and then copied into the container (2 copies)

this can be slow for very large source directories

When saving artifacts, the intermediate tar representation is untarred into /tmp/s2i####/artifacts

this can be slow if the tar contains many many small files

For 1. and 2., it seems this could be solved relatively easily by using read-only docker --volumes -- though this would be a (potentially difficult?) breaking change:

builders which currently rm -rf /tmp/artifacts /tmp/src would need to stop doing that
this avoids the overhead of copying those into the container, as well as the overhead from rm -rfs
builders which assume /tmp/src / /tmp/artifacts should be present in the resulting image would need to adjust (do any do this intentionally? I can't think of a valid usecase)

For 3. I don't have a great solution, I've been double-tarring (tar once, then tar the tar and stream that to stdout) which seems to improve the many-small-files case.
Thoughts on this?  I've tried sketching up some of my ideas in a quick proof of concept but wanted to get some feedback before trying to incorporate some of these ideas.

An idea for 3. would be to change the tar protocol to a docker volume:

create a docker volume
run save-artifacts-v2 in an old image, this script now has the responsibility of copying artifacts into /tmp/artifacts (the docker volume, mounted rw)
mount the docker volume to /tmp/artifacts (ro), assemble can copy things out of it

/tmp/artifacts and /tmp/src are added to the builder, nearly every builder must rm -rf /tmp/artifacts /tmp/src as part of assemble

they should do a mv to move the artifacts/source to the desired location, not a cp+rm -rf:
https://github.com/sclorg/s2i-python-container/blob/master/3.6/s2i/bin/assemble#L15

When using --copy, the entire source directory is copied to /tmp/s2i#### and then copied into the container (2 copies)

obviously the second copy is inevitable.  The first one could potentially be avoided but i believe today we use the /tmp/s2iXXX as a working dir and do some manipulation of the source there, that can't be done on the user's actual source directory since we wouldn't want to mutate that.

For 1. and 2., it seems this could be solved relatively easily by using read-only docker --volumes -- though this would be a (potentially difficult?) breaking change:

Unfortunately that's not a viable approach within openshift which is the primarily consumer of s2i.  If the performance is truly critical and problematic for you I suppose we'd consider an optional path which allows the source to be injected via a mount instead of via tar, but the bar is going to be pretty high for such a change since it will add a lot of new codepaths.
Doing it as an option would help address the compatibility problem as well.

builders which assume /tmp/src / /tmp/artifacts should be present in the resulting image would need to adjust (do any do this intentionally? I can't think of a valid usecase)

in some cases this is used to run unit/smoke tests on the application image after it is built.  See: openshift postcommit hook.  But certainly this requirement can be addressed by explicitly copying the source if it's desired.

they should do a mv to move the artifacts/source to the desired location, not a cp+rm -rf:
https://github.com/sclorg/s2i-python-container/blob/master/3.6/s2i/bin/assemble#L15

ah I guess I was assuming more than checking things -- I've been avoiding mv here as it pulls in a bunch of unnecessary things that would usually get filtered out by .dockerignore / etc.
I'll look into the other things but I don't really want to add to the tangle 😆

Seems the primary confusing is I usually like to assume source is immutable during builds whereas s2i allows (encourages?) mutation -- I'll think about this some more :)

fwiw, here's a prototype of how /tmp/artifacts as a volume (instead of tar) might work: https://github.com/asottile/minis2i/pull/1/files

Point 3 is really a stopper for us to use s2i. Incremental builds with large node_modules folder with many small files is very slow because of streaming those files to the host, and then to the build container.
I was about to post some proposals for improving that and found out this issue. The solution with volumes can really speed up incremental builds and also can bring some other benefits:

There is no need to stream artifacts with save-artifacts script, maybe needed files can be specified with labels or CLI flags.
S2I would check if there is any previous builds for this image, create the container of it with docker create command specifying volume mounts for all specified artifacts.
S2I would run a build container with --volumes-from option mounting artifacts from the previous build volumes into /tmp/artifacts and then copy artifacts to the WORKDIR (this way artifacts remain present in the new built image).
When build finished, S2I would remove the previous build container and it's volume mounts and newly created builds becomes the last build.

Disk IO should also be significantly less when doing incremental builds this way.
This change can be done in a backward compatible way so that both workflow could work at the same time: for example if save-artifacts scripts is present, run it the old way, otherwise use volumes approach.


UML-2671: Show metrics to accounts in stats page

Purpose
https://opgtransform.atlassian.net/browse/UML-2671
Fixes UML-2671
Approach

Introduced new handler for metrics page
Introduced new data handler to query metrics as a batch and handle the responses [to return a map of records for the last three months time period]
New UI page
Tests

Learning
Any tips and tricks, blog posts or tools which helped you. Plus anything notable you've discovered about the Use a Lasting Power of Attorney service
Checklist

[X] I have performed a self-review of my own code
[ ] I have added relevant logging with appropriate levels to my code
[ ] New event_codes have been documented on the wiki page
[ ] I have updated documentation (Confluence/GitHub wiki/tech debt doc) where relevant
[X] I have added tests to prove my work
[ ] I have added welsh translation tags and updated translation files
[ ] I have run an accessibility tool on any pages I have made changes to and fixed any issues found
[ ] I have notified the Interaction Designer of any content changes so that appropriate screenshots/flow diagram changes can be made
[ ] The product team have tested these changes

Report page design updated to now have aggregated blocks in individual tables [as per Sarah]


put(Collection<?>) method is added to conform old json library

Would appreciate if you can release library with this addition :)

Looks good to me.
LGTM

Thanks!
Do you have any roadmap for releases?

On Fri, Dec 2, 2016 at 7:09 PM, Maxim Solodovnik<EMAIL_ADDRESS>wrote:

Do you have any roadmap for releases?

Not really. This isn't really a project ... it is a temporary measure that
people can use as they move to something modern like Jackson.
That said, as people need updates to help stop using json.org stuff, I am
happy to help with that.
So, I guess that the nice way to say it is that this will be "community
driven".

Could you please release this version?

Just did although I haven't seen it appear on central yet.
Also added some other changes.
On Fri, Dec 2, 2016 at 7:28 PM, Maxim Solodovnik<EMAIL_ADDRESS>wrote:

Could you please release this version?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/tdunning/open-json/pull/4#issuecomment-264422953, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAPSepjerC367t8hgGp7zFpzYDIw4W2eks5rD_LcgaJpZM4LCN1Q
.

Thanks a lot!


Fix v1 issues

This PR is an attempt at fixing the issues mentioned in #147.

[x] Reverse the hash map so font.woff2 is matched before font.woff.
[x] Fix the way extensions are removed as this would have failed for a file called test.css.css (test case incoming)

I'll commit and merge in the .sort().reverse() change as that will fix an immediate issue. Please see my comment about cleaning up after the plugin and handling already busted files - the TL;DR being that it's not the responsibility of this plugin to do that

Of course there will still be an issue when you have two files

test.png
test.png.png

test.png.png will be transformed twice: test.[hash].png.[hash].png.

Hi Ben,
is there a reason, why the .sort().reverse() is not in the 1.0.0-rc?
This is the only issue i found with 1.0.0-rc so far..
Cheers,
Lietzi

No reason, it's just been missed off the list. I've created a new issue #178 to remind me to do this. Thanks


10635 delete layers on user deletion

Fixes #10635 and #10792 (by deleting visualizations before the user)
The trick is in UserTable.after_destroy. In this case, super triggers map destruction (via Sequel "FK"), and service.after_destroy triggers visualization destruction. If the map is destroyed first, the reference to the layers is lost.
Maybe layers should be destroyed from map (instead of from visualization), but I'm feeling cautious and I think this could be done as part of the move to AR model.

CR @gfiorav

ping!

This breaks visualization deletion, cannot believe the tests didn't catch it 😞

Ready for a second CR round @gfiorav. Please note that this is merged with #10837 so I can have proper tests, so a lot of the changes are being reviewed separately. If you prefer, we can wait until the other PR is in master so the changes of this PR are easier to read.

Merged with master


[enhancement]: 77 maximum tokens?

Is there an existing issue for this?

[X] I have searched the existing issues

Contact Details
No response
What should this feature add?
Hello, any specific reason why there is a hard cap of the maximum number of tokens?
merge_embeddings.py line 28
max_length=77,
One would think tokens would be discarded based on relevance but they are simply trimmed after the 77th token.
Alternatives
No response
Aditional Content
No response

CLIP does not support more than that.

the limit have been lifted here:
max_position_embeddings (int, optional, defaults to 77) — The maximum sequence length that this model might ever be used with. Typically set this to something large just in case (e.g., 512 or 1024 or 2048).
I read somewhere that this change have been reverted for unknown reason, might worth investigating.

This likely won't be ported due to political/PR reasons - The CLIP token limit removal is based on leaked NovelAI knowledge.

Yikes, I didn't know that. Well, cant erased the fact that it is possible. Having max tokens capped to 77 due to political/PR reasons seems to me like a crime against humanity.

Could have sworn that this was a toggle on A1111 from before the NovelAI situation... hmm

Could have sworn that this was a toggle on A1111 from before the NovelAI situation... hmm

99% sure that's not the case, it was added just days after the leak, if not on the same

99% sure that's not the case, it was added just days after the leak, if not on the same

Yeah, the dates on the commit match. My mistake

so... unfortunate circumstances for this particular case aside, could we skip to the part where this feature would be a great addition or there is no way other way to implement this without asking for trouble?

I had a look at the implementation here: https://github.com/huggingface/diffusers/blob/main/examples/community/lpw_stable_diffusion.py#L209
My initial take after reading this is: this is something that wouldn't be too difficult to add. With or without diffusers doesn't make much difference, everyone is using transformers.CLIPTextModel to turn text to CLIP embeddings, and all the weighting and blending InvokeAI does means it is already messing with that stuff anyway.
It is not quite as simple as pretending the token limit doesn't exist. You still can only send that many through CLIPTextModel at one time. but apparently if you feel like it, you can do that a couple times and concatenate them all together and send the whole mess to the diffusion model?
 In this case we're changing the dimensions of the embeddings instead of the latents, to something different than what it was trained on.
When preparing those embeddings in chunks, it might matter how you divide up those chunks. The most obvious case is if you have a multi-token word, I have a strong suspicion that it won't be represented very well if it gets chopped in half and the first syllable is at the end of one embedding and the rest of it is in the next.

Personal recommendation: this isn't something that should be done transparently by default.
First, focus on some UI features to make it more visible how InvokeAI is interpreting your prompt. e.g. what the positive and negative prompts are (conditioning and un-conditioned), whether it is splitting and blending multiple prompts, etc. This information is currently available in a very dense syntax-heavy format if you use log-tokenization in the TUI, but is invisible to the Web UI, and you only know about it after you start things generating.
Then we can build on that and show if your long prompt is split in to multiple embeddings, where it is split, etc.

Would love to hear ideas on how we move closer to this, and what the implications are on the prompt. We’re contemplating the prompt crafting UX and think this is going to have to play mightily into it

Novel has stolen from SD first, why do we even care. The cat is out of the bag anyway.

Novel has stolen from SD first, why do we even care. The cat is out of the bag anyway.

SD is Open Source, you can't steal from it.
NovelAI is not.

Is open-source allowed to be used for profit though? I recall many licenses prohibit that kind of thing.

Yes. For example, InvokeAI is MIT licensed and can (and is) used to build for profit commercial products

Given that NAI themselves describe their technique in a public blog post, it should be entirely possible to implement without referencing any "tainted" code.

@feffy380 they don’t seem to provide any code?
At this point, we’re sufficiently convinced it would be ok to implement so long as we have something TO reference in implementing (that isn’t autos code)

and all the weighting and blending InvokeAI does means it is already messing with that stuff anyway.

Not liking the sound of that. I was under the impression that given the same parameters, Automatic111 and Invoke AI (or any other SD fork for that matter) would spit the same results/identical image. Apparently the Frankenstein-ism has begun a long time ago.

the limit-removal code right on the AUTOMATIC webui repository has changed from what @Neosettler linked at the top and i'm pretty sure it's original now

the limit-removal code right on the AUTOMATIC webui repository has changed from what @Neosettler linked at the top and i'm pretty sure it's original now

Link to code or commit?

https://github.com/AUTOMATIC1111/stable-diffusion-webui/commit/706d5944a075a6523ea7f00165d630efc085ca22 @n00mkrad - I think

This limit is what keeps me from using InvokeAI more often. On my local system, InvokeAI runs much better with low VRAM, making it a great choice. The web UI of this project is also so beautifully modern. Where not being able to be more creative and transfering prompts from Automatic1111 web UI is a real bummer.
From own experience I can say, the increased tokens do a lot. It is notable that the tokens before the official limit do have a much heavier influence in Automatic1111 web UI. But later tokens are still considered, making it easier to be creative with more words. After like 3 or 4 * 77 prompt lengths though, there is no further effect in Automatic1111's implementation. So the limit cannot be endlessly stretched.

A cursory look suggests that the later actual implementation of unlimited tokens via concatenation was here: https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/2138
Not sure if it has changed since then.
Their wiki describes the unlimited token feature here: https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features#infinite-prompt-length

Typing past standard 75 tokens that Stable Diffusion usually accepts increases prompt size limit from 75 to 150. Typing past that increases prompt size further. This is done by breaking the prompt into chunks of 75 tokens, processing each independently using CLIP's Transformers neural network, and then concatenating the result before feeding into the next component of stable diffusion, the Unet.
For example, a prompt with 120 tokens would be separated into two chunks: first with 75 tokens, second with 45. Both would be padded to 75 tokens and extended with start/end tokens to 77. After passing those two chunks though CLIP, we'll have two tensors with shape of (1, 77, 768). Concatenating those results in (1, 154, 768) tensor that is then passed to Unet without issue.

It does sound similar to what NovelAI described in the blog post linked by feffy380 (https://blog.novelai.net/novelai-improvements-on-stable-diffusion-e10d38db82ac)

InvokeAI refuses to implement it because it's based on leaked NAI info.

After playing around with both InvokeAI and Automatic1111's web UI, I can say that the missing ability to handle longer prompts is the one major thing that prevents InvokeAI from being usable for me. Without that feature, I cannot properly get any work done in it. Too many of my prompts simply require more than 75 tokens, especially when using models that require comma separated tokens (as each comma itself is a token).
I tried manually specifying prompt blending myself, but the results I'm getting are simply not good. For whatever reason it seems like large portions of my prompt are simply being ignored. And while there may be something I'm doing wrong that I could correct, the cognitive load of having to manually break apart and maintain my prompt in different sections is simply not something I want to nor should I have to deal with. Automatic1111's web UI simply handles it for me and does a great job. It's so effective that I think such a feature should be considered standard and a must-have for a proper stable diffusion interface. And from what I can tell, there are implementations of it now that don't directly stem from the NovelAI leak so I see no reason not to include it.
It's a shame, too, because I vastly prefer the UI and workflow of InvokeAI. Its inpainting feature is killer. If it simply had the ability use longer than 75 token prompts then I would ditch the Automatic1111 UI in a heartbeat.

@damian0815 - Acknowledging that it’s effectively just a blend, what are your thoughts on alerting the user of truncated tokens and allowing them to run it with an “auto-blended” form of the prompt? I don’t want to become so opaque as to obscure what’s happening because I think that the blend approach is way less than ideal, but the “average” user may benefit
cc: @psychedelicious @blessedcoolant

@briankendall allowing more then 77 tokens seems to be critical a critical move to make for users.

InvokeAI refuses to implement it because it's based on leaked NAI info.

Sounds noble but does it have real weight in the big scheme of things? Find a way to make it legit and/or seek advise from a lawyer stand point if it helps you sleep better.

Agreeing with @briankendall allowing more then 77 tokens seems to be a critical move to make for users.

InvokeAI refuses to implement it because it's based on leaked NAI info.

Sounds noble but does it have real weight in the big scheme of things? Find a way to make it legit and/or seek advise from a lawyer stand point if it helps you sleep better.

It's not like invokeai doesn't want to support this, it's that the code was apparently lifted from a code base which does not have a license that allows us to do it. At least, the legal side of this is very vague and risky.
Also, we have been busy with a lot of other things and haven't given this the revisit it deserves.
After reading @briankendall 's feedback, it's clear that the token limit is a major hindrance to productivity.
Can anybody point us to an implementation which is from a properly licensed open source project?
Flagging @lstein for input on this.

Sounds noble but does it have real weight in the big scheme of things? Find a way to make it legit and/or seek advise from a lawyer stand point if it helps you sleep better.

To be clear, it sounds like you're asking the question "Do ethical decisions have a real weight in the grand scheme of things?" In case this is news - Yes, they do. I'm as pragmatic a person as you can find, but even I recognize the difference between dealing with practical realities and excusing immoral behavior. There's no reason to lift stolen code when we can come up with something that works as good, or better.

Can anybody point us to an implementation which is from a properly licensed open source project?

@psychedelicious - I think we can work to implement our own, using blends and pads. It may be an experiment/science project, but at the very least we can do "something" with a prompt that would otherwise be truncated.

Surely the idea of cutting something up, proccessing it and putting the
results back together cant be copyrightable…
On Fri, 10 Feb 2023 at 10:24 AM, Kent Keirsey @.***>
wrote:

Sounds noble but does it have real weight in the big scheme of things?
Find a way to make it legit and/or seek advise from a lawyer stand point if
it helps you sleep better.
To be clear, it sounds like you're asking the question "Do ethical
decisions have a real weight in the grand scheme of things?" In case this
is news - Yes, they do. I'm as pragmatic a person as you can find, but even
I recognize the difference between dealing with practical realities and
excusing immoral behavior. There's no reason to lift stolen code when we
can come up with something that works as good, or better.
Can anybody point us to an implementation which is from a properly
licensed open source project?
@psychedelicious https://github.com/psychedelicious - I think we can
work to implement our own, using blends and pads. It may be an
experiment/science project, but at the very least we can do "something"
with a prompt that would otherwise be truncated.
—
Reply to this email directly, view it on GitHub
https://github.com/invoke-ai/InvokeAI/issues/1541#issuecomment-1424859043,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIRC5P7DWWT4267RNDSQBFTWWVN77ANCNFSM6AAAAAASJOTBEQ
.
You are receiving this because you commented.Message ID:
@.***>

Surely the idea of cutting something up, proccessing it and putting the results back together cant be copyrightable…

No, it isn't. Get someone to write the feature who hasn't seen any NovelAI code (or the A1111 code, to be safe) and there isn't an issue.

@damian0815 has already started exploring how this could be enabled without directly referencing their code, and he's shared positive results. Once we're confident we can implement this well (and or better), we will!

Any updates on this? I still miss using InvokeAI!

there's PR #2896 . please test if you're used to using longer prompts and post your experience on that PR - if it isn't working as intended ,please post the prompt you're using and what you expect it to do vs what it actually does.

I have a broader question about whether bypassing the 77 token limit will actually improve results. I find that as prompts get longer there's a distinct sense of diminishing returns where previous well-tuned details start dropping out at random. Most of the textbook-length prompts that I see out there are voodoo concatenations of superlatives that are subject to the same stochastic variations in qualilty that short prompts have.
If anyone has actually conducted a systematic study of long vs short prompt (e.g. using a randomized double-blind trial to judge how well the prompt and the image line up), please point me to it and I'll happily eat my words.

That being said, I'll give @damian0815 's PR a spin and see.

To jump on @lstein comment, it's all about control and sometimes it's not double the tokens that is necessary but a few could make a difference. In all cases, I'd suggest to make this feature optional, --extended_tokens. Also, having less than 77 tokens should result in the same output image if --extended_tokens is true or false.

If anyone has actually conducted a systematic study of long vs short prompt (e.g. using a randomized double-blind trial to judge how well the prompt and the image line up), please point me to it and I'll happily eat my words.

@lstein - I would love to see this. Maybe we can get the discord community to be Guinea pigs 😜 - I tend to believe your intuition is right, fwiw

I my experience there is an optimal token count range. Too few and results are vague - too many and it becomes superstition.
Negative conditioning seems like a thing where having infinite tokens could really help, but maybe that's a mistaken intuition.

I my experience there is an optimal token count range. Too few and results are vague - too many and it becomes superstition.

the diffusion process has a fixed number of attention "heads" , almost 2x as many for SD2 as SD1, so that would make sense. on the other hand, there's only so much attention to go around - what is happening with longer prompts i think is that increasing amounts of the prompt are effectively ignored, and when people say "longer prompts do have an effect" the effect they're seeing is not that there is more detail rather that the attention heads are fixating on different parts of the prompt (and, because there is only a fixed number of them, ignoring different parts)

I'm glad there is an objective explanation for these subjective
experiences. Is there a way to infer the maximum number of tokens that will
be paid attention to given the finite number of attention "heads"? Knowing
this value would make it easier for people to tune their prompts by
budgeting their token usage intentionally, rather than letting the system
stochastically ignore tokens it can't pay attention to.

@lstein What about an instant feedback on tokens analyzer?
Same as the token log but on demand. Something like:
!tokenize [prompt] [negative]

Is there a way to infer the maximum number of tokens that will be paid attention to given the finite number of attention "heads"?

@lstein i'm a little fuzzy on the details tbh, but my understanding is that there's no rigid logic that's doing this, it's just NN magic convoluting a fat 77x768 tensor against some sort of 77-wide activation tensor. (which is part of why i can't wrap my head around why >77 tokens even works)

Is attention value limited on my by the precision of the numbers used or is there a Planck length for attention?

dunno

oh we can close this issue i think. implemented in #2896

merged right?

merged right?

It's merged into Main, but not "released" in a stable version yet.

It is nice to see progress on this. Having backward compatibility in mind. Is there a way to turn this feature on/off?
The reason I'm asking is that if previously generated images had tokens being discarded, using the same settings (from metadata) the result image would be completely different.

very well, thank you for clarifying. see new request:
https://github.com/invoke-ai/InvokeAI/issues/3263

I changed completely to stable diffusion webui, because of that limit in InvokeAI. It's nearly impossible to create the images I want with only 77 tokens.
On the other side, I miss the stability of InvokeAI !!!

I'm getting more and more familiar with stable diffusion webui and I'm no more missing InvokeAI now.
In contrary, I'm happy I did this step. :)

Closing this, as the change was made prompts are appended. Any issues with this should be considered bugs.

Are Token indices sequence length is longer than the specified maximum sequence length for this model (xx > 77). Running this sequence through the model will result in indexing errors lines still supposed to appear in the terminal?


[components]: Toolbar custom components

Notes for Reviewers
This PR adds:
Custom  components for toolbar
Related Issue: #118 #86 #125
I will be adding column component for tables
Signed commits

[ ] Yes, I signed my commits.

Can we merge this @nebula-aac

You got it, @sudhanshutech


Checking for all the values in a template files are defined fails when using helm lint --strict

It took a bit of digging but there are multiple tickets of users reporting [ERROR] templates/: render error in ... <.Values.KeyName>: map has no entry for key "KeyName" when using helm lint --strict. It is solved by setting a default value type for empty default values such as [], {} or "".
It seems that this check was supposed to be disabled based on https://github.com/helm/helm/issues/1463. I think this is confusing because map has no entry for key is being treated as a warning and displayed as an error when using --strict. I think it should either:

Be removed completely as desired in https://github.com/helm/helm/issues/1463
Be treated as an error and cause helm lint to fail
Be treated as a warning and display in helm lint with [WARNING] and failed in helm lint --strict

Helm version: v2.13.0-rc.2

Tested this and it seems to be working as appropriately. Tests as follows:
Created a scaffold chart using helm create.  Disabled the service map entries:
[.....]
service:
        #type: ClusterIP
        #port: 80
[.....]

Tested as follows:
$ helm lint tstchart/
==> Linting tstchart/
[INFO] Chart.yaml: icon is recommended
[ERROR] templates/: render error in "tstchart/templates/tests/test-connection.yaml": template: tstchart/templates/tests/test-connection.yaml:17:61: executing "tstchart/templates/tests/test-connection.yaml" at <.Values.service.port>: can't evaluate field port in type interface {}

Error: 1 chart(s) linted, 1 chart(s) failed
$ helm lint --strict tstchart/
==> Linting tstchart/
[INFO] Chart.yaml: icon is recommended
[ERROR] templates/: render error in "tstchart/templates/tests/test-connection.yaml": template: tstchart/templates/tests/test-connection.yaml:17:61: executing "tstchart/templates/tests/test-connection.yaml" at <.Values.service.port>: map has no entry for key "service"

Error: 1 chart(s) linted, 1 chart(s) failed
$ helm template tstchart/
Error: render error in "tstchart/templates/tests/test-connection.yaml": template: tstchart/templates/tests/test-connection.yaml:17:61: executing "tstchart/templates/tests/test-connection.yaml" at <.Values.service.port>: can't evaluate field port in type interface {}

Can you provide more details of as specific example with the issue?

@hickeyma That is a different error. You are looking for <.Values.KeyName>: map has no entry for key "KeyName". Reference a value as {{ .Values.KeyName }} and leave the value undefined in values.yaml. It will not display a warning when using helm lint, but will fail in helm --strict but use the [ERROR] tag.

@walkafwalka Thanks for clarifying. I am able to reproduce now.
Template:
$ cat mychart1/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ .Release.Name }}-configmap
data:
  myvalue: "Hello World"
  drink: {{ .Values.favoriteDrink }}

Values:
$ cat mychart1/values.yaml
#favoriteDrink: coffee

Tested as follows:
$ helm lint mychart1/
==> Linting mychart1/
[INFO] Chart.yaml: icon is recommended

1 chart(s) linted, no failures

$ helm lint --strict mychart1/
==> Linting mychart1/
[INFO] Chart.yaml: icon is recommended
[ERROR] templates/: render error in "mychart1/templates/configmap.yaml": template: mychart1/templates/configmap.yaml:7:19: executing "mychart1/templates/configmap.yaml" at <.Values.favoriteDrin...>: map has no entry for key "favoriteDrink"

Error: 1 chart(s) linted, 1 chart(s) failed

$ helm template mychart1/
---
# Source: mychart1/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: release-name-configmap
data:
  myvalue: "Hello World"
  drink:

$ helm install mychart1/
NAME:   excited-manatee
LAST DEPLOYED: Mon Mar 11 09:29:16 2019
NAMESPACE: default
STATUS: DEPLOYED

RESOURCES:
==> v1/ConfigMap
NAME                       DATA  AGE
excited-manatee-configmap  2     0s

I can see how this would be confusing. I wonder would it make more sense for issues raised by linter in strict mode to be "warnings" if they are not an error for rendering and install? @bacongobbler what do you think?

@hickeyma Exactly! In addition to being labeled as warnings in strict mode, I think they should also be displayed as warnings by the linter in non-strict mode without failing similarly to informational messages.

@walkafwalka would you be interested in doing a PR for this?

I actually tried to look for the why this was happening but I failed. I am not very familiar with Helm's code. I  found the this line in kubernetes: https://github.com/kubernetes/kubernetes/blob/master/test/cmd/get.sh#L173.
Though I cannot tell how this ties into Helm or why https://github.com/helm/helm/pull/1730 did not fix the issue. I thought that only the errors defined here were caught: https://github.com/helm/helm/blob/master/pkg/lint/rules/chartfile.go

The linter fails here The error seems to be generated when the linter calls the go template library to render the templatehere. When this is called in strict mode, it requires the value object to be set.
If this is the case then it is working as expected.

Oh, I see. I did not expect that there.
Maybe we can move the strict logic out of renderWithReferences and back into Templates as a warning.

Or maybe we should remove the logic from renderWithReferences?
Should missing keys even a be warning?

If yes, should we restore the old logic so that it warns on helm lint and fails on --strict?
If no, should we remove the error from the renderer?
Should we downgrade it to an informational message?

@walkafwalka Do you want to create a PR and give it a go?

Yes, I have something almost complete. Hopefully should have something ready for review by next week.

@hickeyma Sure. Do you want me to remove the error or do you want me to change it to a warning?

closing as inactive.


pagination not working in github page

The jekyll-paginate-v2  plugin work properly in the local server. but in live server and github-page the plugin doesn't work.

0


Support using Hyperfoil without endpoint override

Hyperfoil supports using multiple http endpoint definitions.
In such case it is impossible to identify which one should be replaced by the endpoint override logic.
So, adding support to avoid endpoint override and throwing an exception in case override is applied for benchmark with multiple endpoints defined.
Signed-off-by: Karel Suta<EMAIL_ADDRESS>

Hello @sutaakar, thanks for the fix.
Do you have an example, how would you use they hyperfoil system-x without the host override?

@Croway In our tests we plan to override the specific endpoint before storing it for TNB:

Specify hyperfoil scenario using cucumber with placeholder for hostname
Process the scenario with cucumber engine, replace the placeholder with real endpoint, store the scenario in a temporary location
Trigger the benchmark using TNB


Loading fails on Safari

The app has been working flawlessly in Safari until at last the 12th of July.
When opening the web app in Safari now, loading the app fails with the following message on the console:
SyntaxError: Invalid regular expression: invalid group specifier name

A quick search brought up an answer on stack overflow indicating that the issue is due to lacking support for look-behind in RegEx in safari.
Such a regex was recently added to StashView.

Ha, I didn't test on Safari. This is my highest priority bug at the moment, will look at it as soon as I have a little time. Sorry about the breakage, I hope you were able to use Chrome or Firefox in the meantime.

Fixed.

Works again, thanks!


[Insight] YAML files should not contain syntax error - in .travis.yml, line 31

in .travis.yml, line 31

Malformed inline YAML string ('%{repository}) at line 31 (near "- '%{repository}").

notifications:
  hipchat:
    rooms:
      secure: l4RUOBY44kVhSxx54NU3cmvn598rRMB7Y272Ct/W0fBm1tdGncp42A/rwg7JkiZH9EWXs3tKtoSjqw5xT6QeScUGQDdykS5QVO8lEIkDRGD4WwQ3VDJmRy1+04WtgqFEK8SyYTCcCl4ZJ0rtOcJZgDMvigkOJuLxwGXTDuvfu1k=
    template:
    - '%{repository}
      Build %{build_number} on branch %{branch} by %{author}: %{message}View on GitHub'
    format: html
    notify: true

Posted from SensioLabsInsight

We are not seeing this error on our Travis builds, see: https://travis-ci.org/sendgrid/sendgrid-php
Please advise if you believe we are missing something.
Thanks!


Day 443

Good morning! Here's your coding interview problem for today.
This problem was asked by Apple.
Implement a queue using two stacks. Recall that a queue is a FIFO (first-in, first-out) data structure with the following methods:

enqueue, which inserts an element into the queue
dequeue, which removes it.

Identical to Day 53: https://github.com/vaskoz/dailycodingproblem-go/issues/114


Greenkeeper integration

Hey, it looks like you don't have greenkeeper set up on this repo. Worth doing to make sure you support latest versions of express 😄

@hughrawlinson I hadn't tried Greenkeeper before. I have just integrated it, so I'll keep an eye on PRs with updates for the dependencies.

Fixed through https://github.com/JMPerez/passport-spotify/pull/13

Thanks @JMPerez! :)


IA uploader 403 error on bucket name disallowed causers full disk

Around 500 warcs in a single job, uploaded to a single bucket, we often see 403 errors uploading subsequent warcs.  Have a selection of alternate bucket names to solve this, probably.  It would be good to hear from archive.org if there is some limit.

Could be the item size limit of 1 TiB: https://github.com/jjjake/internetarchive/issues/290


fix link to object-flow options in general README

Noticed that this link didn't take me where I expected to go.
I went ahead and made it link to the heading that you were referring to, but you may have wanted to just link to the page itself.
Let me know.  :)

Good catch, thanks a ton


Improve error messages of array type check.

This is a proposal of improved error messages in FunctionNode.apply(). It also changes the type of exception.
The new message should be more understandable, for example, when a user forgot to use utils.force_array() in their own FunctionNode.
Original and new error messages look like these:
Original
ValueError: numpy and cupy arrays are mixed in the forward output (MyFunc).
<class 'cupy.core.core.ndarray'>, <class 'float'>

This PR
TypeError: forward outputs must be `ndarray`s of the same array module (numpy or cupy) (MyFunc).
Actual: <class 'cupy.core.core.ndarray'>, <class 'float'>

I think this fix is reasonable. How do you think @niboshi?
When merged we should fix the doc too. https://docs.chainer.org/en/latest/upgrade.html#prohibition-of-mixed-use-of-arrays-on-different-devices-in-function-arguments

I rebased and adopted the new message from #4346. It now looks like this:
TypeError: incompatible array types are mixed in the forward output (MyFunc).
Actual: <class 'cupy.core.core.ndarray'>, <class 'float'>

Does this still have a point?

I think message like Types of all arrays in the forward input must be the same is better, but would like to ask opinion of @niboshi san.
I'm also positive for changing the exception type.

Jenkins, test this please.

@kmaehashi
Sorry, I overlooked the mention.
I agree with change of exception type.

Ok, I'll merge this after test pass.

Jenkins, test this please.

@mizuno-gsinet Could you please fix unit tests?
https://github.com/chainer/chainer/blob/v4.0.0b4/tests/chainer_tests/links_tests/connection_tests/test_embed_id.py#L56-L67

I fixed them.

Jenkins, test this please.

Jenkins, test this please.

Sorry, some more tests for this case has been added since then (via #4044), and test is failing.
https://github.com/chainer/chainer/blob/e5fe298e7d69f9ba1c773b5b1ab01b74dfd32e08/tests/chainer_tests/test_function_node.py#L325-L357
Could you please fix them also?

Sure. I rebased and fixed them.

Thank you!
Jenkins, test this please.

LGTM!


Refactor custom toString implementation to use buildString and properly print ByteArrays

I struggled with preserving proper formatting with multiline strings so decided to switch to buildString, it's a lot easier to generate.


feat: support semver aware exclusions

:warning: DO NOT MERGE
Still a couple issues left:

define_translate macro broken
Instruction vs Operation use in wasmparser::for_each_operator!

   Compiling js-component-bindgen v1.2.4 (/home/user/code/work/cosmonic/forks/jco/crates/js-component-bindgen)
error: no rules expected the token `ordering`
   --> crates/js-component-bindgen/src/core.rs:690:57
    |
614 | macro_rules! define_translate {
    | ----------------------------- when calling this macro
...
688 |     (translate $self:ident $op:ident $($arg:ident)*) => {{
    |                                        ----------
689 |         $(
690 |             let $arg = define_translate!(map $self $arg $arg);
    |                                                         ^^^^ no rules expected this token in macro call
...
776 |     wasmparser::for_each_operator!(define_translate);
    |     ------------------------------------------------ in this macro invocation
    |
note: while trying to match `memarg`
   --> crates/js-component-bindgen/src/core.rs:732:33
    |
732 |     (map $self:ident $arg:ident memarg) => {$self.memarg($arg)};
    |                                 ^^^^^^
    = note: captured metavariables except for `:tt`, `:ident` and `:lifetime` cannot be compared to other tokens
    = note: see <https://doc.rust-lang.org/nightly/reference/macros-by-example.html#forwarding-a-matched-fragment> for more information
    = help: try using `:tt` instead in the macro definition
    = note: this error originates in the macro `define_translate` which comes from the expansion of the macro `wasmparser::for_each_operator` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
   --> crates/js-component-bindgen/src/core.rs:693:32
    |
693 |         $self.func.instruction(&insn);
    |                    ----------- ^^^^^ expected `&Instruction<'_>`, found `&Operator<'_>`
    |                    |
    |                    arguments to this method are incorrect
...
776 |     wasmparser::for_each_operator!(define_translate);
    |     ------------------------------------------------ in this macro invocation
    |
    = note: expected reference `&wasm_encoder::Instruction<'_>`
               found reference `&wasmtime_environ::wasmparser::Operator<'_>`
note: method defined here
   --> /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasm-encoder-0.202.0/src/core/code.rs:216:12
    |
216 |     pub fn instruction(&mut self, instruction: &Instruction) -> &mut Self {
    |            ^^^^^^^^^^^
    = note: this error originates in the macro `define_translate` which comes from the expansion of the macro `wasmparser::for_each_operator` (in Nightly builds, run with -Z macro-backtrace for more info)

For more information about this error, try `rustc --explain E0308`.
error: could not compile `js-component-bindgen` (lib) due to 18 previous errors

OK it looks like the project builds now with upstream updates! Going to make this PR against the real jco


ui-router abstract state child doesnt render

I'm having some difficulties to understand how ui-router abstract state work. Here's the thing : I have an index.html with only a <ui-view> tag inside its .
There is two main states : Landing and App ( /landing and /app ).
No problem with the landing state for now, because its only a static file ( no child view etc ).
But for my /app state, i need an abstract parent state that cover the whole application ( need to resolve User Profile for each child states ).
The thing is that childs from this abstract state also have sub-view. And I can't make those render.
My $stateProvider config (simplified) :
                //landing state, no problem here
                .state('landing', {
                    url: '/landing',
                    templateUrl: 'landing.html'
                })

                // my abstract parent state, with the resolve
                .state('root', {
                    url: '',
                    abstract: true,
                    template: "

",
                    resolve: {
                        current_user: function (UserFactory) {
                            return UserFactory.initCurrentUserProfile();
                        }
                    }
                })

                // the child which cannot render the view headerAndSearchbar
                .state('root.app', {
                    url: '/app',
                    templateUrl: 'app.html',
                    views: {
                        'headerAndSearchbar': {
                            templateUrl: './partials/header/headerAndSearchbar.html'
                        }
                    }
                })

app.html :

        This is app.html

Note that if i remove views declaration in the state root.app, i can see the text "This is app.html". headerAndSearchbar.html only contains simple html & css
Any ideas ? I'm bashing my head on this- What am I missing?

You cannot mix views and state-level default view.  Instead, use:
views: {
  'headerAndSearchbar': {
    templateUrl: './partials/header/headerAndSearchbar.html'
  },
  '': {
     templateUrl: 'app.html',
  }
}


Why different trace on the same device, the type of cpu is different and max frequency is also different?

This is fixed in https://r.android.com/3160741


Ubuntu 18.04 support added in 18.03.1-ee-1

Proposed changes

Unreleased project version (optional)

Related issues (optional)

Deploy preview for modest-edison-79d82e ready!
Built with commit 4dc15233cb87247173e8fa8fdef7627b52fa36f8
https://deploy-preview-7008--modest-edison-79d82e.netlify.com

Additional runtime note for 18.03.1.-ee-1.


hello，I have two questions

1、I downloaded the dataset from the paper home page, and extracted it as shown below.
But there doesn't seem to be a training .sh for this dataset in scripts folder.

2、The trainning doesn't seem to use GPU. The following information is displayed:

Is it a Jax version issue?

It looks like Jax isn't finding your GPU. This is probably an issue with how Jax or Cuda have been installed on your computer. I'd check that you can run Jax on your GPU and then circle back to this codebase.


How to build a pandas dataframe from Semi_ATE.STDF.utils.records_from_file

Hi,
Is there a suggested way I can easily read a STDF file (or transfrom Semi_ATE.STDF.utils.records_from_file) into a pandas dataframe?

In fact, that's pretty much also my question - nice coincidence  :-)

Hello Daniel
I made a tool stdf2ph5.py in the Semi-ATE's Metis project:
https://github.com/Semi-ATE/Metis/tree/main/Metis/tools
The tool will parse the STDF file and will write pandas dataframe in HDF5 file.
I do not had time to tested fully. There is some preliminary documentation in the documentation folder.
I will try my best to update the doc and to give example how to use the tools.
Let me know if there is an issues.
Also there is stdf to csv file convertor as well.
Regards,
Seimit

@seimit Thanks a lot for your prompt response! I only now had time to look into this. As a side note, I'm trying to evaluate options to convert STDF to CSV, and to figure out how much information loss would result from such a conversion. For this, I'm trying to use the attached file (which I found here). However, running your conversion tool results in the following output:
C:\Users\chris\git\Metis\venv\Scripts\python.exe C:/Users/chris/git/Metis/Metis/tools/stdf2csv.py -i C:\Users\chris\git\Metis\STDF.stdf -o C:\Users\chris\git\Metis\conversion_result -d

Tool for converting STDF binary files into CSV files.
Debug is enabled
ERROR : Can not decode FAR record. Make sure the file is not compressed : C:\Users\chris\git\Metis\STDF.stdf

The file (the one within the zip file ;-)) does not look like it's compressed to me (since opening it in an editor shows a lot of readable strings) - any idea what the problem is? Does the file contain constructs which are not supported by Semi_ATE?

@csoltenborn Thanks for reporting this! I will fix it in the next minutes.

Hi csoltenborn
I fixed the original problem, but there is exception after FAR record processing.
Because it is too late now for me, I will fix it next week, may be Thursday/Friday.

Hi
The issue with reading STDF file was fixed.
Could you try now?
Regards,
Seimit

Thanks a lot! :-)
I will try probably today and let you know...

Easiest (and fastest I think) to put stdf records into a dataframe  is to follow these steps:

create an empty list, and iterate over all records in the stdf file.
stdf_dictionary_list =[]
for REC in STDF.records_from_file(StdfFileName):

convert each record to a python dictionary.  Recommend a filter to only include specific RECORD types,  for example if REC.id == 'PTR':
stdf_dictionary_list.append(REC.to_dict())  #you can use a filter to only add specific record types

create your dataframe as shown here:
df_stdf = pd.DataFrame(stdf_dict_list)

works very well for me and quite fast.   I use the stdf dictionary for PTR, MPR, FTR, and PRR.  For HBR, SBR, and MRR, I just add them to dedicated lists for each record type, and parse them to make summaries.  This work from SEMI-ATE is very good and I am so happy to have found it.

Hi Mike,
Thanks for the tip!
I think is time to close this issue.
Thanks to all of you!
Best Regards,
Seimit

@seimit Sorry for being late to the party - I finally tried your change, and it works like a charm now. Thanks once more! :-)
@Mike-Purtell Very helpful - thanks a lot!


👽(backends) fix double quoting in ClickHouse backend server parameters

Purpose
In the current clickhouse-connect release (0.5.19) (see CHANGELOG), the quoting and escaping of array literals in server parameters is handled internally. Thus, we do not need to add quotes on our side.
Proposal

[x] remove quoting of ids in ClickHouse backend

Don't forget to fill the CHANGELOG!

Well spotted, thanks) updated the CHANGELOG)


Tag Tendermint image with latest in workflow

Proposed changes
The e2e tests were pulling from the docker hub:
Unable to find image 'valory/open-autonomy-tendermint:latest' locally
latest: Pulling from valory/open-autonomy-tendermint

This was happening because we were using DEFAULT_IMAGE_VERSION = "latest":
https://github.com/valory-xyz/open-autonomy/blob/704dfb5ec0b10f560e12d9f2806df2cae15c3ba7/plugins/aea-test-autonomy/aea_test_autonomy/configurations.py#L39-L42
and this was propagating to the e2e tests' image:
https://github.com/valory-xyz/open-autonomy/blob/704dfb5ec0b10f560e12d9f2806df2cae15c3ba7/plugins/aea-test-autonomy/aea_test_autonomy/docker/tendermint.py#L137-L140
However, we were not tagging with latest when building the images in the workflow file:
https://github.com/valory-xyz/open-autonomy/blob/704dfb5ec0b10f560e12d9f2806df2cae15c3ba7/.github/workflows/main_workflow.yml#L219-L220
Therefore, we now also tag latest.
Fixes
n/a
Types of changes
What types of changes does your code introduce? (A breaking change is a fix or feature that would cause existing functionality and APIs to not work as expected.)
Put an x in the box that applies

[x] Non-breaking fix (non-breaking change which fixes an issue)
[ ] Breaking fix (breaking change which fixes an issue)
[ ] Non-breaking feature (non-breaking change which adds functionality)
[ ] Breaking feature (breaking change which adds functionality)
[ ] Refactor (non-breaking change which changes implementation)
[ ] Messy (mixture of the above - requires explanation!)

Checklist
Put an x in the boxes that apply.

[x] I have read the CONTRIBUTING doc
[x] I am making a pull request against the main branch (left side). Also you should start your branch off our main.
[x] Lint and unit tests pass locally with my changes
[x] I have added tests that prove my fix is effective or that my feature works
[x] I have locally run services that could be impacted and they do not present failures derived from my changes

Further comments
We also need to update our image to the docker hub at some point.

The issue is still present on CI, I'll further investigate that.

~The issue is still present on CI, I'll further investigate that.~
I noticed that we do not build the images on CI for the e2e tests at all:
https://github.com/valory-xyz/open-autonomy/blob/704dfb5ec0b10f560e12d9f2806df2cae15c3ba7/.github/workflows/main_workflow.yml#L450-L452
This is now fixed on 6d39609. However, do we need to build the image out of the scope of the e2e tests at all @angrybayblade?
https://github.com/valory-xyz/open-autonomy/blob/6d3960988457547f6e038c3cdfaf84f43b07234c/.github/workflows/main_workflow.yml#L219-L221

I was not aware that we were not building the images in the e2e CI, but that brags a question, why was the e2e CI not failing?

~The issue is still present on CI, I'll further investigate that.~
I noticed that we do not build the images on CI for the e2e tests at all:
https://github.com/valory-xyz/open-autonomy/blob/704dfb5ec0b10f560e12d9f2806df2cae15c3ba7/.github/workflows/main_workflow.yml#L450-L452
This is now fixed on 6d39609. However, do we need to build the image out of the scope of the e2e tests at all @angrybayblade?
https://github.com/valory-xyz/open-autonomy/blob/6d3960988457547f6e038c3cdfaf84f43b07234c/.github/workflows/main_workflow.yml#L219-L221

I was not aware that we were not building the images in the e2e CI, but that brags a question, why was the e2e CI not failing? if we were using tendermint image in the CI without building it

Take a look at the description at the top :)
They were being pulled from the hub.
The question is; do you think that we need to also build them out of the scope of the e2e tests?

Take a look at the description at the top 😄 They were being pulled from the hub.
The question is; do you think that we need to also build them out of the scope of the e2e tests? I believe we can remove this part.

Makes sense. So yes, we need to build the tendermint images in the e2e context as per our discussion (we need latest images in the CI)


Add smallvec support

thanks! can you also update the documantation like this?

Done! Anything else?

Can you resolve the merge conflict and run cargo readme again?

One more thing, can you add sme tests for this feature in the ts-rs/tests directory?

This has been stale for some time, and #[ts(as = "Vec<T>")] is probably a drop in replacement for this so I'll close this for now. If there is still interest in this, feel free to reopen the PR and merge main into your branch to resolve the merge conflicts


The dataset

There has some error when i test your  code on the fr3-nostruc-notex dataset and fr3-longoffice dataset. The error seems to occur when the program reachs the detectOrthogonalLineRANSAC_ODEP, which makes the program unable to run normally. The other five datasets are normaly. If you have time, please help me  solve this problem.

There has some error when i test your code on the fr3-nostruc-notex dataset and fr3-longoffice dataset. The error seems to occur when the program reachs the detectOrthogonalLineRANSAC_ODEP, which makes the program unable to run normally. The other five datasets are normaly. If you have time, please help me solve this problem.

Could you describe the error in a bit more detail? LPIC sometimes cannot run correctly depending on the user-defined parameters such as the maximum line lengths, # of lines, etc.

I tried to change the parameters like you said  when i test LPIC on  fr3-nostruc-notex dataset,at first ,it  can run normaly ,but after running about 120 pictures,it kept to prompt me that "Fail 1-line RANSAC,Try again

I also have problem on fr3_longoffice,i don't know how to init the parameters correctly ，it stop running at the fourth pictures.

I don't know how to fix it.If it's convenient for you,please help me.

The fr3-nostruc-notex dataset is a very difficult dataset to run stably. I think that LPIC will fail from time to time depending on the user-defined parameters. The main reason why it fails might be the insufficient number of lines following the MW. You know, the LPIC can only work well in the strict MW, so if that assumption is broken, LPIC cannot work at all.

If you don't have any further questions, I will close this issue.


PEAR-1633: Modals - 508 compliance issues

Description
Checklist

[ ] Added proper unit tests
[ ] Left proper TODO messages for any remaining tasks
[x] Scanned for web accessibility with aXe, and mitigated or documented
flagged issues

Screenshots/Screen Recordings (if Appropriate)

All reported issues are fixed.
The summary modals have a focus indicator issue, which will be addressed in a separate ticket.


add two proxy methods and fix typo

fix two places of typo
add two proxy methods for ServerLoader

Coverage decreased (-0.5%) to 99.4% when pulling a386f7828fe04ab16eca14b03ff85aefd62f3944 on vincent178:master into 168ced92387da5e17e138137a4fcb0751791eaa9 on Romakita:master.

Thanks vincent :D . I built a new release v1.3.0-5 (beta) ;)


Flutter adds additional one frame touch latency compared to native

Bad
Expected

Repository: https://github.com/knopp/flutter_scroll_overlay
When touch event is received, instead of immediately beginning a frame Flutter waits for next vsync. But that skips an entire frame. By the time vsync comes, the frame should be already rendered and ready to go. But instead it has just began. On iOS touch events seem to be delivered within few milliseconds of the display link callback so there should be enough time.
You can pause the "bad" video to see that during scrolling Flutter lags behind native by one frame exactly.
The expected video is with quick and dirty engine modification that forces BeginFrame immediately on input event.
The way vsync is implemented in Flutter might need some work. It should make sure that repaints are synchronized with display updates, but it shouldn't be causing additional latency.

To clarify - in first video you'll find frames like this:

Where the flutter scroll view doesn't keep up with UITableView (on the right) because is always one frame behind. The video on the right stays aligned at all times.

cc @iskakaushik

https://www.youtube.com/watch?v=N66MdEDHh2Q
This is a slo-mo video recorded that can demonstrate this issue.
Up top is the Wonderous app, and down below is a swift app.

@knopp were you able to reproduce this on Android?

This should not be a P4 @chinmaygarde @knopp
Should be a P1

cc @cyanglaz

@knopp btw please don't hesitate to land your test app in https://github.com/flutter/platform_tests if you want to. Collecting test apps like this is what that repo was made for.

I'm not currently working on this so I'll un-assign myself. I might circle back to this eventually.
@knopp Do you mind to open a draft PR with your engine modification that results the right video?

@cyanglaz  did you see this as being reasonably in-reach to resolve?
There's comments about Flutter being in an "uncanny valley" - close to native, yet slightly different in an uncanny way. Outside of just saying "performance" or "jank" I couldn't identify why scrolling feels off in every flutter app I tried. So I find relief at finding issues like this since they identify why that's the case, and fixing these issues overtime make true native experience feel feasible! 🤞
If I may suggest, this feels like a higher priority issue as scrolling is a major interaction in any app.

Would like to throw in my 0.02c in favor of fixing things like this above other issues.  Getting the app to feel native is much more important then implementing new features.  These apps usually "do" what they are supposed to but just feel slightly "off" what is expected from a polished app

@knopp do you have any workaround regarding this or any PR or may any sample repo with your modification. I wish I could  help but due to limited knowledge, i can't. And this is first think i always notice when i run my app . As a perfectionist, you foolishly try to split flutter app and other app side by side and try to compare but there is always a feeling of something being "off" but can't figure out. i tried all best performance pratice but i think this is the issue if i am not wrong.

@yashwant1999 meanwhile the requested PR is not here, I can share something that might be related to this which I saw back them.
https://github.com/flutter/flutter/blob/e3429dd35f6441a7562cc6384b24a3a8666eff2a/packages/flutter/lib/src/widgets/scroll_physics.dart#L441
https://github.com/flutter/flutter/blob/e3429dd35f6441a7562cc6384b24a3a8666eff2a/packages/flutter/lib/src/widgets/scroll_physics.dart#L751

@Juliotati, this particular issue needs to be fixed in the engine, not framework.

@knopp is this behaviour present when using an Impeller runtime?

Any statement/news to this? This would be a dealbreaker for our next development.

I’ve had this experience for years, scrolling always feels off, muddy, laggy. It’s the main reason I never got into flutter. I need snappy scrolling 😉

@knopp I tried to repro but I'm seeing a different behavior. The scrolling acceleration seems different between iOS and Flutter. Could you post your workaround so I can try it too?

@knopp I tried to repro but I'm seeing a different behavior. The scrolling acceleration seems different between iOS and Flutter. Could you post your workaround so I can try it too?

This is not about scroll physics. This is just about the tracking phase (following finger while the finger is down). The physics only comes into play after lift (and is indeed slightly different but that's not the point here).
I don't have the changes anymore, it was just a quick and dirty hack to trigger beginning frame immediately after event (instead of scheduling vsync), but i didn't pus it anywhere.

Seeing this behavior on iPad scrolling with the Apple Pencil. You can see that the iOS recognizes the pencil first and triggers the scribble API (which is a Flutter bug), and a few frames after that Flutter would react.
I'm on Flutter 3.7.5
https://user-images.githubusercontent.com/8187501/221723833-12a55e82-1441-401f-b3ef-04bd6d2f41b9.MOV

@ slowmotionbot
Here is your slowed down video with 5x
https://user-images.githubusercontent.com/6442915/221829162-1fdeb20c-7c08-45cf-b52c-83e552f89feb.mp4

@knopp Is there any chance that you could redo that change you made and share it with us so that we might experiment and perhaps make an example repo for the Flutter team to try?
If you dont have time, perhaps you could just point us in the right direction? Any help is appreciated 😁.

Is there any progress on this?

any updates?

Maybe helpful additional info: This issue only applies to IOS running 60hz.
I'll look further into resolving this issue soon.
Some of our customers noticed this issue running our app on "60hz IPhones"

Couldn't it be just that is much less obvious at 120Hz? As far as I can tell the frame cycle should be the same, the latency should be still present just shorter in absolute numbers.


Generate social iconsheet during build

Summary
Simplify management and maintainability of social media icons by using multiple include files and individual SVG files.

This completely breaks backwards compatibility for folks who are using minima as a remote theme, doesn't it?

This completely breaks backwards compatibility for folks who are using minima as a remote theme, doesn't it?

@mattr- Yes, it does. But given the fact that the theme is currently under transition state (from 2.x to 3.0), we are allowed to make breaking changes such as this.
Currently as I see it, the benefits of this change outweighs the drawbacks. I shall update the opening comment to note the pros and cons.
Additionally, consumers of this theme as a remote-theme have the option to lock to a commit prior to a significant breaking change if not ready to move in sync with an unstable branch.

@jekyllbot: merge +major


[20791] Include docker compose in project

This PR adds a docker file:

Vulcanexus iron as based image (so we avoid installing dependencies)
Install other Qt dependencies
Clone all SustainML packages and dependencies, and build them with the selected branches

It also adds a bash script:

Includes a entrypoint for the docker compose with the following node names as arguments: orchestrator, app_requirements, carbon_tracker, hw_constraints, hw_resources, ml_model and ml_model_provider.
Runs the python module based on the given node name.

Finally, it adds a docker compose file:

Create a service per node, including the Front-end.

This PR will be opened in the https://github.com/eProsima/SustainML-Frontend repo

This PR will be opened in the https://github.com/eProsima/SustainML-Frontend repo


Download RDP option is available in plugin when the Remote Desktop port number is not default

Environment: Windows/IntelliJ14
In portal if the Remote desktop port is not 3389, Connect option is not enabled.
PFB the screen shot.

In plugin if the Remote desktop port is not 3389, Connect Remote Desktop option is enabled and it is throwing exception.
PFB the screen shot.

Also this option will not make sense if user is in Mac/Linux. So generating rdp file will only make sense if user is in windows.

This issue is fixed and the functionality is working as expected in the latest build.


Fixed linking with lld

For changelog. Remove if this is non-significant change.
Category (leave one):

Build/Testing/Packaging Improvement

dataparts_500 - bad test @proller


[Bug] RP-1 Shows compatibility with 1.12.3 But can not be installed

Background

Operating System: Windows 10 21H2 19044.1889
CKAN Version: 1.31.0
KSP Version: <IP_ADDRESS>73

Have you made any manual changes to your GameData folder (i.e., not via CKAN)?
No
Problem
Describe the bug
Realistic Progression 1 max version is listed as 1.12.9, however there is no checkbox to install the game and when you try to force it by version it says unsupported
Steps to reproduce

Open CKAN with an instace of the latest game version
In the search box put "RP-1"
No check box is shown

Expected behavior
RP-1 Should be installable
Screenshots (if applicable)

CKAN error code (if applicable):

Luckily the RP-1 team has a wiki with comprehensive instructions on how to install:
https://github.com/KSP-RO/RP-0/wiki/RO-&-RP-1-Express-Installation-for-1.12.3
I'd recommend following those steps carefully.

In case you're curious, the first incompatible dependency is SXTContinued, because it requires Firespitter:

I'll see if we can make that clearer.

That would be great. It is not very clear its not compatible, because a dependency is compatible.


omnipay/common version conflicts

Lots of conflicts when using other omnipay payment gateways with this one, any plans on updating this?

Currently not, but it would be nice when you can update the code and make a PR


Add further models API's  in the extension

[x] Ollama
[ ] ChatGPT
[ ] Gemini
[ ] Claude

Adding ChatGPT, Claude and Gemini was never a part of plan because we will be working with open Source Models


Feature/ap 526 actor versions

Looks good. I would just rename listActVersions to listActorVersions, etc.
Because we will be renaming client.acts to client.actors

Agree with Marek, otherwise looks good to me.


Add comparison methods

This PR

adds specifications for comparison methods.
is derived from comparing API signatures across dataframe libraries.

Notes

Comparison methods are widely implemented across dataframe libraries and are used by downstream libraries.

For all comparison methods, cuDF hangs the methods on a Series object, but not the DataFrame object.

For all comparison methods, cuDF and Koalas do not support the axis keyword. The axis keyword is not currently included in this proposal (see note on broadcasting below).

For all comparison methods, cuDF and Koalas do not support the level keyword.

Pandas, Dask, Modin are liberal in what they accept (e.g., scalar, sequence, Series, DataFrame). This PR limits to DataFrame objects, similar to the array API specification. This choice should be discussed and confirmed.

As a follow-up to the prior note, this PR does not currently address broadcasting semantics. A few notes on this end:

Similar to the array API specification, scalars can be broadcast as an MxN dataframe whose size matches the original dataframe. And similar to the array API specification, how libraries choose to represent a scalar as an MxN dataframe can be implementation-defined.
A dataframe consisting of a single row (size: 1xN) can be broadcast as an MxN dataframe whose size matches the original dataframe.
A dataframe consisting of a single column (size: Mx1) can be broadcast as an MxN dataframe whose size matches the original dataframe.
Otherwise, the dataframe provided as other must have a size equal to the original dataframe (i.e., no broadcasting required).

This has been superceded by other PRs, I'll close this.


enforce complete type annotations

Proposed changes:

fix https://github.com/RasaHQ/rasa/issues/8247

Status (please check what you already did):

[ ] added some tests for the functionality
[ ] updated the documentation
[ ] updated the changelog (please check changelog for instructions)
[ ] reformat files using black (please check Readme for instructions)

@m-vdb Do you think it would be an option to force-merge this if the only failing things are docstrings? Because of the reach of my changes it's quite a lot of failing docstring checks (they wouldn't fail for anyone else as pydocstyle only runs on diffs)

yeah I think that'd be ok 👍🏻

@m-vdb mypy is passing locally and the rest of the test is passing as well. One of our memory leaks checks failed, but I'd increase the value rather in a separate PR (definitely not related to type annotations)

doing a PR for the memory leak test here

@wochinge did the "Test CLI" step pass locally as well?

yep, did that! Works

@wochinge what's up with this test?

Pretty sure it's flakey - will see if I can fix it but not related to this PR

Anca fixed the failing test here . Ready to merge then @m-vdb


Use double quotes for Zig strings

Just changing the strings in the example code to use double quotes for the stings instead of single quotes.

@JosephTLyons Ooops! Thank you for catching this! :+1:


Use arc4random instead of rand

srand() alters global state. This could be problematic in the context of a tweak that injects into another program. Also, the output of arc4random_uniform() doesn't depend on the current time unlike the current implementation.

Lgtm


NOT NULL constraint failed: bayesdb_crosscat_diagnostics.logscore

bayeslite> ANALYZE solvers FOR 15 MINUTES wait;
Traceback (most recent call last):
  File "/home/fsaad/pcp/bayeslite/build/lib.linux-x86_64-2.7/bayeslite/shell/core.py", line 133, in default
    cursor = bql.execute_phrase(self._bdb, phrase)
  File "/home/fsaad/pcp/bayeslite/build/lib.linux-x86_64-2.7/bayeslite/bql.py", line 459, in execute_phrase
    ckpt_seconds=phrase.ckpt_seconds)
  File "/home/fsaad/pcp/bayeslite/build/lib.linux-x86_64-2.7/bayeslite/metamodels/crosscat.py", line 1073, in analyze_models
    'iterations': theta['iterations'],
  File "/home/fsaad/pcp/bayeslite/build/lib.linux-x86_64-2.7/bayeslite/bayesdb.py", line 236, in sql_execute
    self.sql_tracer, self._do_sql_execute, string, bindings)
  File "/home/fsaad/pcp/bayeslite/build/lib.linux-x86_64-2.7/bayeslite/bayesdb.py", line 178, in _maybe_trace
    return meth(string, bindings)
  File "/home/fsaad/pcp/bayeslite/build/lib.linux-x86_64-2.7/bayeslite/bayesdb.py", line 239, in _do_sql_execute
    return bql.BayesDBCursor(self, self.sqlite3.execute(string, bindings))
IntegrityError: NOT NULL constraint failed: bayesdb_crosscat_diagnostics.logscore

Suspicion this happened because I have some columns of all None - but we should be able to do something about that case.

This is a bug in Crosscat: somehow analyze is returning None, or more likely NaN, for some logscore.

Oh boy just got it again, even after nullifying and ignoring columns of all None.

I don't expect any particular Nones in the table have anything to do with this.  It sounds like an invalid floating-point operation in the logscore calculation which yields NaN.
One way to try to debug it would be to trap the invalid-operation exception in Crosscat, if code inspection doesn't find it.
Can you show your table's M_c?  (select metadata_json from bayesdb_crosscat_metadata where generator_id = ?)
It might also be helpful to apply this patch to Crosscat:
assertnotnan.txt

@fsaad, can you take next steps, or provide more info to independently reproduce?

One output from the suggested assertion:
https://docs.google.com/document/d/1fpnbR4AxuWE0NetllswHRUFMD0hk3PqNCdvCnY8u2y8/edit

https://github.com/probcomp/crosscat/commit/f91197c5f7fe24f7c3af47823d187580a1e138b1
https://github.com/probcomp/bayeslite/commit/19b8071ce3dcc6b5669bac0affd94c08fff67af5
as additional diagnostics, and improved guessing in https://github.com/probcomp/bayeslite/commit/8b87fd10cbb7bbb85badbfb15e88ab470757aa81
to help avoid the problem in the first place. It covers all instances I've run across so far, namely all-nulls, and all-null-or-same-value. It's still super easy to get this error in a repeatable way by specifying the column type for such a degenerate column as numeric, rather than letting guess ignore it.

It sounds like there is a bug in Crosscat causing NaN to come out.  If we mask the bug in bayeslite, that will make it harder to find the bug in Crosscat.  Did you trip over the assert you added in Crosscat, or only the one in bayeslite?

Same bug happening on a remote host but not on my local computer
What do you advise?

updating to the latest bayeslite / bdbcontrib, ahead of the releases
(because I haven't released 0.1.6 yet, and it's fixed ahead of 0.1.5)
On Wed, Feb 3, 2016 at 3:55 PM, Belhal KARIMI<EMAIL_ADDRESS>wrote:

Same bug happening on a remote host but not on my local computer
What do you advise?
[image: not null not null constraint failed]
https://cloud.githubusercontent.com/assets/15817944/12796777/9281f7ea-ca8e-11e5-8d85-527ce0752133.png
—
Reply to this email directly or view it on GitHub
https://github.com/probcomp/bayeslite/issues/284#issuecomment-179461208.

--
Gregory Marton
<EMAIL_ADDRESS>617-858-0775
http://csail.mit.edu/~gremio/cal

I just tripped this bug in bayeslite 0.1.8. It sounds like that's unexpected. Do you want me to dump any information?

Fixing this requires finding where Crosscat is getting NaN in the logscore calculation.  Last time I tried to track down sources of this symptom I liberally sprinkled assert(!isnan(...)) throughout Crosscat, starting in State.cpp's State::transition_* functions.
Here is an example of a bug that could cause this symptom (a bug which bayeslite's logic to guess statistical types tries to avoid in some cases, so it may not be the bug you're encountering): https://github.com/probcomp/crosscat/issues/85

Got it. I was just confused by Grem's comment that "...it's fixed ahead of 0.1.5"
Is there a straightforward workaround in the meantime? I could manually ignore the column if it's easy to figure out what column it's choking on, but I don't get much in the ways of debug output. Is that included in the output of that patch?

Probably Grem was referring to the bayeslite statistical type guessing heuristic to avoid modelling constant columns.
Unfortunately, at the time the NaN comes flying out of Crosscat, it is not associated with any particular column.  We don't know all the conditions of column values (if they are isolated to individual columns!) under which Crosscat will compute NaN for a logscore.  We know at least one condition: a constant column.  There are more conditions which have not been fully analyzed, such as https://github.com/probcomp/bayeslite/issues/388, which has a lead you might follow about normal sufficient statistics and update_continuous_hypers.

Of course, if you can discern that you have a constant column and then skip modelling it, that would be the easiest way to work around this!  But if you don't obviously have a constant column then I'm afraid it will take some more work.

@asilversempirical, can you get a backtrace if you recompile crosscat with FP exceptions turned on in State.cpp?

Performance got so bad after enabling FP exceptions that it couldn't initialize the models. I never got to analyze to try to trigger the issue.

Thanks for trying it.  I'll try it on some of the smaller test cases I ran into on the other thread when I get some time.

https://github.com/probcomp/bayeslite/commit/d2376c134942f40b3305cdf972be708e7404f25a


puppeteer.launch() never returns when using --no-startup-window

Steps to reproduce
Tell us about your environment:

Puppeteer version: 1.14.0
Platform / OS version: macOS
URLs (if applicable):
Node.js version: 11.9.0

What steps will reproduce the problem?
Please include code that reproduces the issue.
const browser = await puppeteer.launch({ args: ['--no-startup-window'] })

What is the expected result?
The launch function should return a promise which resolve to a browser.
What happens instead?
The launch call is hanging and never resolve.
An other closed issue already reported this problem (#3630), but I can't use the given workaround in my case. And i think that even if there is a workaround, this is still bug since the behavior is unexpected.

I add here my use case to explain why I can't use the workaround of #3630
I'm using puppeteer to render pdf document from web page. It is a web service and I wanted to allow each instance of my service to have mutliple "renderer" (multiple pages in facts).
To implement this, I'm using a generic pool implementation which allow me to create a pool of page ready to be used.
const puppeteer = require('puppeteer')
const genericPool = require('generic-pool')
const { goToLogin } = require('./utils')

const NB_OF_WORKERS = 5

let pagePool

// Browser singleton
const getBrowser = (() => {
  let memoizedBrowserPromise
  return () => {
    if (!memoizedBrowserPromise) {
      memoizedBrowserPromise = puppeteer.launch({ args: ['--no-startup-window'] })
    }
    return memoizedBrowserPromise
  }
})()

const pageFactory = {
  async create() {
    const browser = await getBrowser()
    const page = await browser.newPage()

    const onError = (error) => pagePool.destroy(page)

    page.on('error', onError)
    page.on('pageerror', onError)

    await goToLogin(page)

    return page
  },
  async destroy(page) {
    if (page.isClosed()) await page.close()
  },
  async validate(page) {
    return page.isClosed()
  },
}

const factoryOptions = {
  min: NB_OF_WORKERS,
  max: NB_OF_WORKERS,
}

pagePool = genericPool.createPool(pageFactory, factoryOptions)

module.exports = pagePool

--no-startup-window is in't even supported by headless chrome. Having an extra blank page uses no CPU and very little memory. I wouldn't worry about it.

I didn't even notice that it wasn't working on headless... It's difficult to see what's happening in headless ^^
Thanks for the advice, if it has no real impact, I will ignore this then.