Wrong query result for one accelerated dataset from pg

Describe the bug
In my testing, the accelerated dataset from pg could be wrong.
To Reproduce
Steps to reproduce the behavior:
this bug can be reproduced stably, by the steps:

start from the the follow spicepod.yaml
watch the log for loaded event
execute the query in spice sql console

Expected behavior
The dataset is one 1.7GB NYC taxi trip dataset which is imported into pg previously. The query to pg and the correct result is like this:
jin=# select avg(total_amount) from rides;
         avg
---------------------
 15.6413944975914241
(1 row)

Time: 414.961 ms
jin=# select count(total_amount) from rides;
  count
----------
 10906860
(1 row)

Time: 307.454 ms

but the spice sql's result is like this:
sql> select avg(total_amount) from rides;
+-------------------------+
| avg(rides.total_amount) |
+-------------------------+
| 15.61769                |
+-------------------------+

15.6413944975914241 is correct result but not 15.61769
More can be seen in the screenshot.
Runtime Details
Spicepod
spicepod.yaml in my testing:
version: v1beta1
kind: Spicepod
name: spice_app
runtime:
  telemetry:
    enabled: false
datasets:
    # - ref: datasets/eth_recent_blocks
    - from: postgres:rides
      name: rides
      params:
        pg_host: <IP_ADDRESS>
        pg_db: jin
        pg_port: 5432
        pg_user: jin
        pg_sslmode: disable
      acceleration:
        enabled: true

Output of the describe table
+-----------------------+-----------------------------+-------------+
| column_name           | data_type                   | is_nullable |
+-----------------------+-----------------------------+-------------+
| vendor_id             | Utf8                        | YES         |
| pickup_datetime       | Timestamp(Nanosecond, None) | YES         |
| dropoff_datetime      | Timestamp(Nanosecond, None) | YES         |
| passenger_count       | Decimal128(38, 0)           | YES         |
| trip_distance         | Decimal128(38, 2)           | YES         |
| pickup_longitude      | Decimal128(38, 15)          | YES         |
| pickup_latitude       | Decimal128(38, 15)          | YES         |
| rate_code             | Int32                       | YES         |
| dropoff_longitude     | Decimal128(38, 15)          | YES         |
| dropoff_latitude      | Decimal128(38, 15)          | YES         |
| payment_type          | Int32                       | YES         |
| fare_amount           | Decimal128(38, 1)           | YES         |
| extra                 | Decimal128(38, 0)           | YES         |
| mta_tax               | Decimal128(38, 1)           | YES         |
| tip_amount            | Decimal128(38, 0)           | YES         |
| tolls_amount          | Decimal128(38, 0)           | YES         |
| improvement_surcharge | Decimal128(38, 1)           | YES         |
| total_amount          | Decimal128(38, 1)           | YES         |
+-----------------------+-----------------------------+-------------+

Output of explain query
explain select avg(total_amount) from rides;
+---------------+---------------------------------------------------------------------------------+
| plan_type     | plan                                                                            |
+---------------+---------------------------------------------------------------------------------+
| logical_plan  | Aggregate: groupBy=[[]], aggr=[[avg(rides.total_amount)]]                       |
|               |   BytesProcessedNode                                                            |
|               |     TableScan: rides projection=[total_amount]                                  |
| physical_plan | AggregateExec: mode=Final, gby=[], aggr=[avg(rides.total_amount)]               |
|               |   CoalescePartitionsExec                                                        |
|               |     AggregateExec: mode=Partial, gby=[], aggr=[avg(rides.total_amount)]         |
|               |       BytesProcessedExec                                                        |
|               |         SchemaCastScanExec                                                      |
|               |           RepartitionExec: partitioning=RoundRobinBatch(20), input_partitions=1 |
|               |             MemoryExec: partitions=1, partition_sizes=[2727]                    |
|               |                                                                                |
+---------------+---------------------------------------------------------------------------------+

spice, spiced, OS info

spice version: spice version
spiced version: spiced --version

2024/10/01 22:41:33 INFO CLI version:     v0.18.3-beta-rc.8ac6fae6
2024/10/01 22:41:33 INFO Runtime version: v0.18.3-beta-rc.8ac6fae6

OS info: uname -a

Linux jin-homesrv 6.10.10-arch1-1 #1 SMP PREEMPT_DYNAMIC Thu, 12 Sep 2024 17:21:02 +0000 x86_64 GNU/Linux

Have you tried this on the latest trunk branch?

[x] Yes
[ ] No

If possible, run spiced with DEBUG log level
n/a
Screenshots
v0.19.0-beta-rc.4277369b (yesterday's trunk head):

v0.18.3-beta-rc.8ac6fae6 #1 (latest trunk head):

v0.18.3-beta-rc.8ac6fae6 #2 (latest trunk head):

Additional context
n/a

Thanks for reporting @mjzk, we'll take a look at this. Which taxi trips dataset did you use?

@mjzk Thanks again for bringing up this issue! We tried to reproduce the reported issue, but failed to do so using the information provided. We will proceed to close this issue for the time being as we have not received any additional information.
Please feel free to revisit the issue and include any more steps or context that might aid in our investigation if you encounter the same issue again. We value your input and contribution!


Other request types not being collected

Hi
I am using WebDriverIO to perform end to end testing of customer journeys, at certain points our tag management system initiates network requests to endpoints such as Google Analytics 4, Floodlights, Facebook, etc.
We would like to be able to test / check the requests emitted contain the correct data however when calling await browser.getRequests() they are not returned.  I believe this is because they are not fetch or XHR request types, is this correct?
And if this is correct is there expected to be any future releases to allow such monitoring / capture to occur.
Below is an example of a request being sent that WebDriverIO is not picking up.

I have a similar requirement.
To be able to intercept the requests that are generated by beacon calls (navigator.sendBeacon() method) tat are of type "ping".
I assume that https://github.com/webdriverio-community/wdio-intercept-service/blob/main/lib/interceptor.js needs to have a replaceSendBeacon implementation likewise fetch and Xhr ones.
Cypress is already supporting it so i was wondering if tere is any plan to add support.

I have a similar requirement. To be able to intercept the requests that are generated by beacon calls (navigator.sendBeacon() method) that are of type "ping".
I assume that https://github.com/webdriverio-community/wdio-intercept-service/blob/main/lib/interceptor.js needs to have a replaceSendBeacon implementation likewise fetch and Xhr ones.
Cypress is already supporting it so I was wondering if there is any plan to add support in the near future.

Indeed, we would need an IE-parsable (does not have to support IE, just be parsed by it) implementation of such. A PR contributing (& testing!) that functionality would be appreciated.

@ctm-adamstribblehill I would need to see the code that triggers the request, not the request details. For example, if the initiating code uses a private reference to fetch or XMLHttpRequest and does not look it up from the window, this library is not able to intercept the request.

@tehhowch This request is not directly initiated / triggered by any code that we develop.
We have a tag management system (Tealium IQ), we configure Tealium to activate the Facebook Pixel tag deployed to the page via Tealium when the appropriate conditions are met and a Tealium event is emitted.  This then results in the Facebook network request to be transmitted as attached above.
I have been able to collect other network requests emitted via Tealium such as a Yahoo Dot request but this is an XHR request.

@ctm-adamstribblehill reading more closely this time, I see you are basically asking about intercepting requests made by the browser itself for e.g. the src attribute of an <image> tag. That is not currently possible with this WDIO library, because those requests do not use the fetch or XMLHttpRequest objects. I do not expect support to be added for non-JS requests, either, since it would (AFAICT) require protocol-level interception and this library provides functionality in all modern browsers, not just those that extend Chromium.
You will probably need to switch to some other interception library that offers tie-ins with the chrome dev tools / puppeteer.
(I welcome any corrections to my assumptions above!)

Hello @tehhowch,
I have raised a PR to add support for intercepting queued beacon requests at https://github.com/webdriverio-community/wdio-intercept-service/pull/651
Would love to assist and get this in the next release. For the time being I unblocked myself using patch-package with the same changes.
On a sidenote, the prettier pre-push check fails with existing code from main you might want to auto format the code base once at some point.
Thanks for the great package!

hey guys, we're seeing the same issue and are unable to retrieve analytics request of type 'ping'.
any news of when will this be supported?
btw thanks for the package.


Review and update RATELIMIT settings for xPro

Description
Verify that ratelimits are as they should be for xpro-edxapp.
They go here: https://github.com/mitodl/ol-infrastructure/blob/main/src/bilder/images/edxapp_v2/templates/edxapp/xpro/lms_only.yml.tmpl#L86
Relates to https://github.com/mitodl/hq/issues/2367
See:
https://docs.openedx.org/projects/edx-platform/en/latest/references/settings.html#setting-RATELIMIT_RATE

Can we do this as part of the Quince upgrade?

Rate limits look good to me. We have set good enough values for all required. We could reduce REGISTRATION_RATELIMIT and REGISTRATION_VALIDATION_RATELIMIT a bit if we want. Here are the details:
RATELIMIT_RATE edX: 120/m, xPro: 600/m
REGISTRATION_RATELIMIT edX: 60/7d, xPro: 1000000/minute
REGISTRATION_VALIDATION_RATELIMIT edX: 30/7d, xPro: 1000000/minute

I'm assuming this is good now


Codecrumbs views don't finish loading

My Codecrumbs browser is in a perpetual loading state. I've added some logging on my end and it seems to be parsing the //cc comments properly, so I can't figure out why there is no visualisation.
Browser

Debug

Hi, can you please show your command how do you run it? (codedrumbs params). Also, which version do you use, please make sure it’s latest. Thanks.

@Bogdan-Lyashenko no errors in the console:

and here's the command to run it. i've tried various entry points, none of them work:

and the version is 1.1.1 (it's in the bottom left of the first screenshot 🙂)

@adityavm for what project are you running codecrumbs?
Let's do this example to see how everything works:

clone/download to your computer this project https://github.com/lucasbento/react-redux-todo (just as simple example)
install codecrumbs - yarn global add codecrumbs
go to the folder react-redux-todo (where you copied project from step 1)
run codecrumbs -d src -e src/index.js
go to the browser url http://localhost:2018 to see if it works

let me know if this works and then we can continue from there.

@Bogdan-Lyashenko it's a personal project. react-redux-todo works fine.

i also thought that it might be because of paths, however logging the paths from the different parser scripts seems to output the correct full path (second screenshot in this thread). i use absolute module paths, like:
import Something from "component/Something";
could that be an issue?

import Something from "component/Something";
could that be an issue?
No, it should be fine, it might not be resolved by dependencies tree builder (Dependencies tab on UI) but it will not affect the entire picture.

So you ran codecrumbs for your project in the exact same way as you just did for  react-redux-todo  and it still doesn't work. That's super interesting... 👀 Alright, I will go through setup and add more error handling and logs. But basically, I assume it doesn't init here
https://github.com/Bogdan-Lyashenko/codecrumbs/blob/master/src/server/project-source/index.js#L75.

@Bogdan-Lyashenko feel free to email me / ping me here if you need me to test out pre-release versions. codecrumbs looks super useful, so happy to help however 👍

@adityavm hi there 🙂 I just pushed version with extra logs. When you have a free minute, can you please update to v1.1.2 (or higher) and run it again? So I covered with logs file structure parsing and also added log to browser to see what you are receiving in the end. Let's see 😸

@adityavm try v1.1.4, should be golden now 😸

@Bogdan-Lyashenko confirmed. works well. thank you!


The mean is not averaged realy.

I noticed that you divide the sum by it sum_size, however the sum_size is not multiplied by the device_ids: https://github.com/vacancy/Synchronized-BatchNorm-PyTorch/blob/81d983920c828cb50398bf6e4f6c9044f85fd568/sync_batchnorm/batchnorm.py#L117

Thank you for your attention! Please see the following lines for your information. We sum up the size of the tensors distributed on different devices at L101.
https://github.com/vacancy/Synchronized-BatchNorm-PyTorch/blob/81d983920c828cb50398bf6e4f6c9044f85fd568/sync_batchnorm/batchnorm.py#L101-L103
Please feel free to reopen this issue if your question remains.


Wildcards in choosers

Hi there,
I saw there is a TODO to add wildcards to choosers as they are not supported at the moment.
i assume this issue is known so i didnt add any example.
I think i know how to implement this feature, so if no one is working at it at the moment ill go ahead and prepare a pull request (and also support message and defaultFile along the way)
Thanks! :>

hi, is there anything still needed to accept the PR?

Nope. Just needed to poke me ^_^ Nights and weekends are eaten up by other projects these days..
I'll try to get all the pending PRs merged/reviewed soon. I'll try to get to them all over the weekend. And, then, hopefully, finally, actually kick 1.0.3 out the door!

Yea, i know the feeling.
Great job on this lib. Very useful for many small things i do. Keep up the good job
Cant wait to port to 1.0.3

Closing. Completed by you! ^_^


Search results link to Hackage, not Stackage.

https://www.stackage.org/lts-6.11/hoogle?q=amazonka

Good catch. Expanding: https://www.stackage.org/lts-6.11/hoogle?q=base
It's the package results that link to Hackage. The output from Hoogle needs a bit more munging.

Looks fixed.


Difference between Free Tier vs. Always Free for Trial Accounts remains unclear to me

Issue description
When I read the documentation it's yet unclear to me what that means for a BTP Trial account:https://help.sap.com/docs/btp/sap-business-technology-platform/trial-accounts-and-free-tier
Feedback Type (Optional)
content gaps
Page Title on SAP Help Portal (prefilled)
Trial Accounts and Free Tier
Page URL on SAP Help Portal (prefilled)
https://help.sap.com/docs/btp/sap-business-technology-platform/trial-accounts-and-free-tier

Thank you for your feedback! We’ll look into it and come back to you if we have any questions.

Hi @ECasio,
You can also read https://help.sap.com/docs/btp/sap-business-technology-platform/using-free-service-plans. Unlike the trial account, you can get CPEA (Cloud Platform Enterprise Agreement) and Pay-As-You-Go commercial model with free tier account. Furthermore, you can just move from a free tier to a paid productive account when you reach the boundaries of the limited resources. On the other side, when the trial account expires, it is automatically deleted and you need to request a new one for productive use.

Hi @nikolasimeonov77,
yes but what is missing in the docu is that a these free tier or always free services have the same meaning for a trial account. That's what I am talking about. The docu doesn't clearly draw the line here. It's free if you have a contracted BTP account, just like certain movies and TV serieses are for free on Amazon Prime (if you paid for a normal amazon prime account).

Hi @ECasio,
Thank you again for your feedback.
At the moment we are still working on adjustments in the documentation for exactly this topic. As soon as we finalize these changes and I have all existing and up-to-date information available, I'm happy to get in touch with you again.
Best regards,
Madeline

Hi all,
As there is nothing more to discuss, I'll close the issue for now.
@Madeline-Schaefer Please notify @ECasio when you have an updated version of the new document. You can also reopen the issue if you need to discuss more about this issue.
Thanks for you collaboration!
Nikola

https://people.sap.com/eddi


Address @jrose-apple's code review comments.

Addressing Jordan's code review feedbacks to apply fixits on broader cases.

Resolved bug number: (SR-)

Before merging this pull request to apple/swift repository:

[ ] Test pull request on Swift continuous integration.

Triggering Swift CI
The swift-ci is triggered by writing a comment on this PR addressed to the GitHub user @swift-ci. Different tests will run depending on the specific comment that you use. The currently available comments are:
Smoke Testing
    Platform     | Comment
    ------------ | -------------
    All supported platforms     | @swift-ci Please smoke test
    All supported platforms     | @swift-ci Please smoke test and merge
    OS X platform               | @swift-ci Please smoke test OS X platform
    Linux platform              | @swift-ci Please smoke test Linux platform

A smoke test on macOS does the following:

Builds the compiler incrementally.
Builds the standard library only for macOS. Simulator standard libraries and
device standard libraries are not built.
lldb is not built.
The test and validation-test targets are run only for macOS. The optimized
version of these tests are not run.

A smoke test on Linux does the following:

Builds the compiler incrementally.
Builds the standard library incrementally.
lldb is built incrementally.
The swift test and validation-test targets are run. The optimized version of these
tests are not run.
lldb is tested.

Validation Testing
    Platform     | Comment
    ------------ | -------------
    All supported platforms     | @swift-ci Please test
    All supported platforms     | @swift-ci Please test and merge
    OS X platform               | @swift-ci Please test OS X platform
    OS X platform               | @swift-ci Please benchmark
    Linux platform              | @swift-ci Please test Linux platform

Lint Testing
    Language     | Comment
    ------------ | -------------
    Python       | @swift-ci Please Python lint

Note: Only members of the Apple organization can trigger swift-ci.

@swift-ci Please smoke test and merge

Thanks! But, um, test cases?

@swift-ci Please smoke test OS X platform

@swift-ci Please smoke test

:-) Thank you!


Pull

Description
A clear and concise description of the features you're adding in this pull request.
Tests
Please describe any tests you've added. If you've added no tests, or left important behavior untested, please explain why not.
Additional context
Add any other context about the problem you're solving.
Metadata

Fixes #[Link to Issue]

hi! please only open PRs against this repo when the intention is to merge the code into our main branch. You can open up a PR against your own fork if you're modify this repo for you own needs. Going to close this out!


fix(readme): add analytics login events

I forgot to add the events to the readme in this PR https://github.com/kiwicom/smart-faq/pull/591LiveURL: https://smartfaq-add-events-readme.surge.sh

:tada: This PR is included in version 1.0.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


fix(native-filters) set currentValue null when empty

SUMMARY
Currently cascading filters evaluate truthiness of the parent filter when determining if the child filter should be rendered in place of the parent. Since the native Select filter returns an empty array for an empty selection, the parent filter is determined to be set, even though it isn't.
Here we change the cascading logic so that only undefined and null are assumed to be unset. This is done because there may be other filters that have falsy values which in fact mean that the filter is set, for instance a lower range being set to 0.
SCREENSHOTS
In the example, I've constructed a cascading filter with "region" as the parent and "country" as the child. As can be seen, the "country" filter show up on the filter tab, despite the selection being empty on the "region" filter:

After the change, an empty selection is set as null, making the parent filter display in the filter tab:

TEST PLAN

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Changes UI
[ ] Requires DB Migration.
[ ] Confirm DB Migration upgrade and downgrade tested.
[ ] Introduces new feature or API
[ ] Removes existing feature or API

Codecov Report

Merging #13000 (3fa9fd3) into master (c440d98) will increase coverage by 8.69%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master   #13000      +/-   ##
==========================================
+ Coverage   53.17%   61.86%   +8.69%
==========================================
  Files         449      545      +96
  Lines       14826    20140    +5314
  Branches     3959     5268    +1309
==========================================
+ Hits         7884    12460    +4576
- Misses       6942     7467     +525
- Partials        0      213     +213

Flag
Coverage Δ

cypress
?

javascript
61.86% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...ponents/nativeFilters/FilterBar/CascadePopover.tsx
20.28% <0.00%> (-76.38%)
:arrow_down:

...src/filters/components/Select/AntdSelectFilter.tsx
0.00% <0.00%> (-96.78%)
:arrow_down:

superset-frontend/src/views/App.tsx
0.00% <0.00%> (-100.00%)
:arrow_down:

superset-frontend/src/views/menu.tsx
0.00% <0.00%> (-100.00%)
:arrow_down:

superset-frontend/src/explore/App.jsx
0.00% <0.00%> (-100.00%)
:arrow_down:

superset-frontend/src/views/index.tsx
0.00% <0.00%> (-100.00%)
:arrow_down:

superset-frontend/src/dashboard/App.jsx
0.00% <0.00%> (-100.00%)
:arrow_down:

superset-frontend/src/setup/setupPlugins.ts
0.00% <0.00%> (-100.00%)
:arrow_down:

superset-frontend/src/setup/setupFormatters.ts
0.00% <0.00%> (-100.00%)
:arrow_down:

superset-frontend/src/explore/reducers/index.js
0.00% <0.00%> (-100.00%)
:arrow_down:

... and 463 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c440d98...3fa9fd3. Read the comment docs.


MixedRealityCommander: UI may still not update the displayed device address if UpdateConnection checked

Looks like the race condition can still impact the commander. Consider checking address updates every heartbeat and responding if there has been a change.

closing since this was checked in already


test

Hi @sperand-io, as part of the monorepo migration, this issue has been moved to new issue. Our engineers have been notified and will prioritize and work on it ASAP. Thank you!
For more information, see README.md.


[BUG] SfIcon doesn't render all sizes and colors correctly

Describe the bug
SfCircleIcon allows the icon size to be set as pixel values (default value: 20px) but this doesn't work with SfIcon which is used internally. Although the SfIcon prop docs say that pixel values are allowed, those are stripped by the computed method in SfIcon. The size you enter is just added as a class, but such classes do not exit.
Also, the color you enter is just appended as a class name. So this works only for Sf-UI values (e.g. primary), too.
In the SfIcon stories you can only choose from options which correspond to Sf-UI colors and sizes. So those stories are fine (but still the SfIcon component docs (comments inside SfIcon.js) are misleading!). But the SfCircleIcon stories allow to enter invalid values.

@EduardDopler it is not really a bug. If you enter a color, it has to be HEX or RGB color since we check if the color follows the HEX and RGB format and SfIcon will check if the color is valid (even in HEX there is invalid color). Otherwise, it will translate as if it is a class color.
About the size, the same, you need to pass it as 20px, or "100%" or "1rem". Check out props [default] in SfIcon. The size there is set as 24px, and not from the list.
I did mention these inside the documentation section of SfIcon.
In the documentation of SfIcon, there is a section for modifying the props - props [default] We need to change the title of this section correctly also. The same goes for SfCircleIcon, you can see there is a section where the icon is defined with 24px size also.

@mayashavin @filrak
I know this is already marked as resolved and the PR merged but I don't get this component, sorry 🙈  I checked the source before opening this bug and I thought I knew where the problem is; I just wanted to wait until the CircleIcon PR (#111) was merged because I think both components need to be addressed for this bug. Anyway, let me clarify the problem a little more:
Prolog
The docs/comments for SfIcon say that you can use

normal CSS values for the color and the size (e.g. red, #ff00ff or 12px, 3rem) or
SfUI values (e.g. primary or xxl).

The template of SfIcon looks like this:
<div :class="['sf-icon', iconColor, iconSize]" ref="icon" v-on="$listeners">
  <slot v-bind="{ viewBox, iconPaths, icon }">
    <svg class="sf-icon-path" :viewBox="iconViewBox" preserveAspectRatio="none">
        <path :d="icon" v-for="(icon, index) in iconPaths" :key="index" style="height: 100%;"/>
    </svg>
  </slot>
</div>

You can see that iconColor and iconSize are inserted as CSS classes.
They are resolved via these computed methods:
iconColor() {
  return !this.isDecimalOrHexColor
    ? `sf-icon--color-${this.color.trim()}`
    : this.color;
}
iconSize() {
  return this.isSFSizes ? `sf-icon--size-${this.size.trim()}` : "";
}

Problem
So when the color prop isn't a SfUI value, it is just inserted as a class as-is.
Similarly, when the size prop is no SfUI value, an empty string gets inserted.
My Interpretation
So I would say: Either the docs aren't correct (we allow only SfUI values, the rest results in garbage CSS classes or nothing) or the implementation isn't correct.
I would favor the latter because imho this could be easily solved: When the size or color is a SfUI value, insert the appropriate class (like now); when it is not, insert them as a :style binding.
wdyt?

And why is this related to CircleIcon?
Because in the SfIcon stories you can only select (from a select input) valid SfUI values. So the problem as described above doesn't show up. But in the SfCircleIcon stories you can enter text values like 24px which are–according to the component docs of SfIcon–correct values but lead to the problem above.

If this is my misinterpretation, please let me now and close the bug again. For now, I'll reopen it because I think it is still broken (cf. Story CircleIcon of this PR's state).

@EduardDopler You didn't look at the component implementation throughout. Before on mounted, we do set the CSS variable of icon-color and icon-style if the color and size passed is not standard (according to SFUI). color and size computed is just the class properties in case it does go according to SFUI. The bug before was that since we only set CSS variable of icon-color and icon-size one time on mounted if the component is updated with different color and size, it won't update the right color and size.

You also need to look at the Storybook, we do have 3 stories for SfIcon, and one of them dedicated for these customized variables.

In this screenshot, as you can see, passing the prop 24px and non-standard color is working still.
Similar in SfCircleIcon, I passed the default value of sizeIcon as 100% and it does show in the component. I don't see why there is a bug as you describe except what I wrote above.
I'm closing it as it's not broken.


Document how to use with multiple Tidbyts

Example .env:
LIVING_TIDBYT_DEVICE_ID="<device id>"
LIVING_TIDBYT_TOKEN="<token>"

OFFICE_TIDBYT_DEVICE_ID="<device id>"
OFFICE_TIDBYT_TOKEN="<token>"

Example apps.yml:
plugins:
  loaders:
  - name: target-tidbyt--living
    inherit_from: target-tidbyt
    config:
      device_id: $LIVING_TIDBYT_DEVICE_ID
      token: $LIVING_TIDBYT_TOKEN
  - name: target-tidbyt--office
    inherit_from: target-tidbyt
    config:
      device_id: $OFFICE_TIDBYT_DEVICE_ID
      token: $OFFICE_TIDBYT_TOKEN

jobs:
# Living
- name: living/revolution
  tasks: [tap-pixlet--revolution target-tidbyt--living]

# Office
- name: office/revolution
  tasks: [tap-pixlet--revolution target-tidbyt--office]

schedules:
# Every minute
- name: living/revolution
  interval: '* * * * *'
  job: living/revolution

- name: office/revolution
  interval: '* * * * *'
  job: office/revolution

Is this actual documentation of how it currently works? Or is this a suggestion about how it should work?

@EricWF This is my actual apps.yml:
plugins:
  loaders:
  - name: target-tidbyt--living
    inherit_from: target-tidbyt
    config:
      device_id: $LIVING_TIDBYT_DEVICE_ID
      token: $LIVING_TIDBYT_TOKEN
  - name: target-tidbyt--office
    inherit_from: target-tidbyt
    config:
      device_id: $OFFICE_TIDBYT_DEVICE_ID
      token: $OFFICE_TIDBYT_TOKEN

jobs:
# Living
- name: living/apple-tv
  tasks: [tap-pixlet--apple-tv target-tidbyt--living]
- name: living/plex
  tasks: [tap-pixlet--plex target-tidbyt--living]
- name: living/revolution
  tasks: [tap-pixlet--revolution target-tidbyt--living]
- name: living/common-misconceptions
  tasks: [tap-pixlet--common-misconceptions target-tidbyt--living]
- name: living/crossword
  tasks: [tap-pixlet--crossword target-tidbyt--living]
- name: living/jeopardy
  tasks: [tap-pixlet--jeopardy target-tidbyt--living]
- name: living/guess-the-flag
  tasks: [tap-pixlet--guess-the-flag target-tidbyt--living]
- name: living/ffmpeg
  tasks: [tap-pixlet--ffmpeg target-tidbyt--living]
- name: living/homebridge-unifi
  tasks: [tap-pixlet--homebridge-unifi target-tidbyt--living]
- name: living/homebridge-unifi--avatars
  tasks: [tap-pixlet--homebridge-unifi--avatars target-tidbyt--living]
- name: living/parcelapp
  tasks: [tap-pixlet--parcelapp target-tidbyt--living]
- name: living/owen-wilson-facts
  tasks: [tap-pixlet--owen-wilson-facts target-tidbyt--living]
- name: living/wedding
  tasks: [tap-pixlet--wedding target-tidbyt--living]
- name: living/dadjoke
  tasks: [tap-pixlet--dadjoke target-tidbyt--living]

# Office
- name: office/revolution
  tasks: [tap-pixlet--revolution target-tidbyt--office]
- name: office/common-misconceptions
  tasks: [tap-pixlet--common-misconceptions target-tidbyt--office]
- name: office/crossword
  tasks: [tap-pixlet--crossword target-tidbyt--office]
- name: office/jeopardy
  tasks: [tap-pixlet--jeopardy target-tidbyt--office]
- name: office/guess-the-flag
  tasks: [tap-pixlet--guess-the-flag target-tidbyt--office]
- name: office/ffmpeg
  tasks: [tap-pixlet--ffmpeg target-tidbyt--office]
- name: office/homebridge-unifi
  tasks: [tap-pixlet--homebridge-unifi target-tidbyt--office]
- name: office/homebridge-unifi--avatars
  tasks: [tap-pixlet--homebridge-unifi--avatars target-tidbyt--office]
- name: office/parcelapp
  tasks: [tap-pixlet--parcelapp target-tidbyt--office]
- name: office/owen-wilson-facts
  tasks: [tap-pixlet--owen-wilson-facts target-tidbyt--office]
- name: office/speedtest
  tasks: [tap-pixlet--speedtest target-tidbyt--office]
- name: office/wedding
  tasks: [tap-pixlet--wedding target-tidbyt--office]
- name: office/dadjoke
  tasks: [tap-pixlet--dadjoke target-tidbyt--office]

schedules:
# Every minute
- name: living/revolution
  interval: '* * * * *'
  job: living/revolution

- name: office/revolution
  interval: '* * * * *'
  job: office/revolution

# Every 5 minutes
- name: living/apple-tv
  interval: '*/5 * * * *'
  job: living/apple-tv

- name: living/plex
  interval: '*/5 * * * *'
  job: living/plex

- name: living/common-misconceptions
  interval: '*/5 * * * *'
  job: living/common-misconceptions

- name: office/common-misconceptions
  interval: '*/5 * * * *'
  job: office/common-misconceptions

- name: living/crossword
  interval: '*/5 * * * *'
  job: living/crossword

- name: office/crossword
  interval: '*/5 * * * *'
  job: office/crossword

- name: living/jeopardy
  interval: '*/5 * * * *'
  job: living/jeopardy

- name: office/jeopardy
  interval: '*/5 * * * *'
  job: office/jeopardy

- name: living/guess-the-flag
  interval: '*/5 * * * *'
  job: living/guess-the-flag

- name: office/guess-the-flag
  interval: '*/5 * * * *'
  job: office/guess-the-flag

- name: living/dadjoke
  interval: '*/5 * * * *'
  job: living/dadjoke

- name: office/dadjoke
  interval: '*/5 * * * *'
  job: office/dadjoke

# - name: living/ffmpeg
#   interval: '*/5 * * * *'
#   job: living/ffmpeg

# - name: office/ffmpeg
#   interval: '*/5 * * * *'
#   job: office/ffmpeg

- name: living/homebridge-unifi
  interval: '*/5 * * * *'
  job: living/homebridge-unifi

- name: office/homebridge-unifi
  interval: '*/5 * * * *'
  job: office/homebridge-unifi

- name: living/homebridge-unifi--avatars
  interval: '*/5 * * * *'
  job: living/homebridge-unifi--avatars

- name: office/homebridge-unifi--avatars
  interval: '*/5 * * * *'
  job: office/homebridge-unifi--avatars

# Every 15 minutes
- name: living/parcelapp
  interval: '*/15 * * * *'
  job: living/parcelapp

- name: office/parcelapp
  interval: '*/15 * * * *'
  job: office/parcelapp

# Every hour
- name: living/owen-wilson-facts
  interval: '0 * * * *'
  job: living/owen-wilson-facts

- name: office/owen-wilson-facts
  interval: '0 * * * *'
  job: office/owen-wilson-facts

- name: office/speedtest
  interval: '0 * * * *'
  job: office/speedtest

- name: living/wedding
  interval: '0 * * * *'
  job: living/wedding

- name: office/wedding
  interval: '0 * * * *'
  job: office/wedding

If you're having problems with that setup, please file issues, because this is what I'm planning to document.


Migrating to v1 as v1beta is deprecated

Starting with k8s 1.22 the v1beta CustomResourceDefinition will be deprecated see https://kubernetes.io/blog/2021/07/14/upcoming-changes-in-kubernetes-1-22/#api-changes
Which means that the code in this repo will no longer be applicable. Is there any plans to migrate the definitions?

Hi @asosnovsky-sumologic, thanks for the question, are you able migrate to using the new SageMaker operator based on AWS controllers for Kubernetes framework? - https://github.com/aws-controllers-k8s/sagemaker-controller
It offers similar functionality and supports more SageMaker features. We have a migration guide as well - https://docs.aws.amazon.com/sagemaker/latest/dg/kubernetes-sagemaker-operators-migrate.html

Hi, We are stopping the development and technical support of SageMaker Operators for Kubernetes in its original version. If you are currently using version v1.2.2 or below of the original version of SageMaker Operators for Kubernetes, we recommend migrating your resources to thelatest SageMaker Operators for Kubernetes, the ACK service controller for Amazon SageMaker based on AWS Controllers for Kubernetes (ACK).
For information about the migration steps, see Migrate resources to the latest Operators.
For answers to frequently asked questions regarding the end of support of the original version of SageMaker Operators for Kubernetes, see Announcing the End of Support of the Original Version of SageMaker Operator for Kubernetes


Can this be used to transform XML files (not .config, but .xml files)?

I need to deploy server specific values in certain XML files in the project.  These XML files are .xml, not web.configs...is it possible to use this to transform non config XML files?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 315ff823-9992-9f9b-62a1-42ce622ed41c
Version Independent ID: 72ec3ffc-1dc9-b2dc-e3d4-4f14deae3720
Content: File transforms and variable substitution - Azure Pipelines
Content Source: docs/pipelines/tasks/transforms-variable-substitution.md
Product: devops
Technology: devops-cicd
GitHub Login: @RoopeshNair
Microsoft Alias: ronai

@mihirdalal  Thank you for the question.
There should be no issues transforming native XML.
Please provide feedback or post below if there are any issues.

Thank you Mathew...I will try it & post an update..when done.

Closing the issue now, please comment below if there are any issues.

Hi Mathew,
I apologize, I was a little late in getting to this..so, finally, I am on it.
Unlike web.configs, I have XML files that are in a subfolder on the project root that need to transformed server specific.
/XML_Templates/file.xml
/XML_Templates/file.deployserver1.xml
/XML_Templates/file
xmlfiles.zip
.deployserver2.xml
I could not get the transformation to work..and am wondering if being in a subfolder this way could be the problem.
The configuration needed in Azure DevOps to make the transformation happen as been set up correctly as such.
I am attaching my xml files as well for review..
Sorry again, for the delay in replying..
Mihir.

I should also let you know, that I am letting the website's web.config transforms happen through the MSBuild step.
I am trying to leverage the Azure DevOps Release step XML Transformation only for some XML files that have to be server specific.
I am guessing, that should not be a problem as such, but just wanted to share this extra piece of information.

Please refer to the XML transformation pre-requisites here. Can you verify if it is applicable in your case?

Yes,
the files are not .config file, but raw .xml files...however on which I am not able to make it work.
As noted in a post few lines below "Not able to prevent Release transforms from running during build", I am not able to fully turn off MSBuild Transformations on web.config.
If I remove the [DependentUpon], I get an error.
2019-12-03T18:23:59.8900142Z ##[error]Error: XML transformation error while transforming C:\agent\vsts-agent-win-x64-2.144.2-TEST_work_temp\temp_web_package_13805296405710688\Content\T_C\agent_01_work\71\s\IVR_Outbound_WebSite\obj\QA\Package\PackageTmp\Web.config using C:\agent\vsts-agent-win-x64-2.144.2-TEST_work_temp\temp_web_package_13805296405710688\Content\T_C\agent_01_work\71\s\IVR_Outbound_WebSite\obj\QA\Package\PackageTmp\Web.Release.config.
2019-12-03T18:23:59.9242257Z ##[section]Finishing: Deploy IIS Website/App:
Once, I leave [DependentUpon]...I am no longer getting any errors/warnings in the build
Similar to the workaround suggested in that previous post "Not able to prevent Release transforms from running during build",
I created a web.release.config
and web.stagename.config
and am able to confirm that the web.release.config & web.stagename.config transformations are now being applied during deployment.
--
I am back to my original question...so besides web.config in the project, I also have some raw xml files..in a folder within the project "xml_templates/file1.xml" which has a corresponding "xml_templates/file1.stagename.xml" file in it.
The deployment transformation however is not running the transformation on the raw .xml files. I tried putting the .xml files on the root, next to web.config and web.stagename.config, but no luck.
In the deployment trace, it only shows that a transformation was attempted on web.config/web.stagename.config, but no trace of transformation attempts on raw xml file file1.xml/file1.stagename.xml.
Can someone confirm, if .xml files should be getting picked up by the transformer? Or is there any explicit setting that can make it pick them up?

Yep...so a final test, I renamed my .xml files to .config files..and sure enough the deployment transformation is firing on them now.
So, its down to this: How to make the deployment transformer do the transformations for raw .xml files..
As a workaround, I can always rename my post deployed transformed file1.config to file1.xml..in some post deploy step.

@mihirdalal -- you may find current answers here:

Azure DevOps Support Bot
Azure DevOps on Stack Overflow

I am closing this issue now.  You are welcome to @ mention me for any followup.
We hope to hear from you again.
My apologies for the delay in responding.


Remove empty constraints-xxx-.txt files after Antares step

After the Antares step, the study output cleaner should remove the constraints files as they are not used (check if this is true)

If we're talking about the files in output directory then they are cleaned up by the lpnamer step. A quick search of "constraints" reveal usage of "additionnal constraints" in lpnamer.
As of now I lack visibility to cleanup files after antares step. We may discuss this to issue to ensure that we are aligned on the files discussed


Updating extents in a scene does not update them in the inspector.

Godot version:

v3.1.alpha.official
Issue description:

If i mutate a CollisionBox2D in the 2D scene, the extents in the inspector should be updated.

Can be reproduced on 13eaded

Duplicate of #21367


DoNoTrackAttribute is inaccessible due to its protection level

It says "DoNoTrackAttribute is inaccessible due to its protection level" whenever I try to use DoNoTrack attribute for my model properties in my project.
And I found out "internal sealed class DoNoTrackAttribute : Attribute", DoNoTrackAttribute is not public. Should it be public instead?

@isaacqc
Yep, you are correct, will have a new release soon.

@isaacqc
I Just released 2.2.17 to nuget, Let me know if you have any more issues.

Thanks for the prompt response! I will try it asap

It works. Thanks a lot!


‘Enable’ property of sap.ui.core.Item cannot control item be disselected

As document said, property 'enabled' is used to control items can be selected.
But it seems after version 1.22, this 'enabled' property is used to show item in the select list.
Is there any other property to control item can be selected?

http://jsbin.com/sikeqa/1/edit?html,console,output
In this demo, you can see 8 items shown in this list.
But after change version of ui5 up to 1.40, we will miss some items in the list.

I guess the question is taken from here https://archive.sap.com/discussions/thread/3792342
In my opinion, disabled items are not rendered to avoid issues with DOM modifications which can modify the selection.
I forwarded to the current team that handles the control via #1880663985

Hello,
I am closing the issued due to existing answer and inactivity afterwards,
Best regards,
Tereza


(maint) Allow Bundler version to float

This commit changes the Bundler version constraint in the gemspec file
from ~> 1.16 to >= 1.16 and allows Bundler 2.x to be used.

this is also in #52


Remove old jquery-migrate files

Fixes #1259

Makes sense to me. I don't see any other jquery-migrate libs in the folder but I assume it isn't needed at all anymore. r+

r+


Run a custom keyword when Setup has failed

Is there anyway for us to run a custom keyword if a [Suite Setup] fails?
We have disabled Selenium2Library's feature to automatically run the Capture Screenshot keyword by setting run_on_failure to Nothing on import.  Instead, in our default test teardown we are using Run Keyword If Test Failed to run the Capture Screenshot keyword from there.
However, that isn't called if there are any Suite Setup failures, is there a way for us to check whether a Setup has failed, and if so, run a keyword?
Currently I'm implementing a workaround where in our Suite Teardown we check for the ${SUITE MESSAGE}, and if it includes Suite setup failed, we run Capture Screenshot, but I'm wondering if there's a more direct way of doing that without relying on having a Suite Teardown

There's no built-in support for something like this, but you could use Run Keyword And Return Status to run certain keyword and then run another only if the first one failed. You can do all this in your suite setup.
If you got further questions, I recommend sending them to robotframework-users mailing list or joining the discussion on our Slack. This tracker is for bug reports and enhancement requests and those actual support forums have a lot bigger audience to help you.


Routes not being registered

Hello,
Just installed a fresh Laravel 5.1 application, proceeded to pull in this package and all is well.
{
    "require": {
        "php": ">=5.5.9",
        "laravel/framework": "5.1.*",
        "dingo/api": "0.10.*"
    },
    "minimum-stability": "dev",
    "prefer-stable": true
}

Obviously, removed the other irrelevant stuff to show I have this set up correctly.
Registered the service provider in config/app.php for Laravel
Dingo\Api\Provider\LaravelServiceProvider::class

Published the config file, again for Laravel (config/app.php)
php artisan vendor:publish --provider="Dingo\Api\Provider\LaravelServiceProvider"

Now, for the routes (app\Http\routes.php)
$api = app('Dingo\Api\Routing\Router');

$api->version('v1', ['namespace' => 'InkedFemales\Http\Controllers'], function ($api) {

	$api->resource('photos', 'PhotosController');

});

As far as I'm aware, this is correct. If my namespacing was incorrect, I know it throws a ReflectionException when running php artisan - which I also tested by misspelling controller and it did exactly that (throwing the exception).
But this is my problem, when running php artisan api:routes I just get

Your application doesn't have any routes.

Why is this? Am I missing something?
Really appreciate what you have done with this package so far!

In addition to this, of course I thought it would be worth noting I have the API_PREFIX value added to my .env file.
API_PREFIX=api

So I'm currently hitting the http://inkedfemales.dev/api/photos route to see the output (404'ing right now)

That command will not show registered API routes. There is an api:routes
command although it's broken at the moment.
As for not seeing anything when you are in the browser, have you tried just
a regular $api-get?
Also you don't have to use the env file. If you've published the config you
can set the prefix there.
On Thu, 23 Jul 2015 07:33 Joe Dawson<EMAIL_ADDRESS>wrote:

In addition to this, of course I thought it would be worth noting I have
the API_PREFIX value added to my .env file.
API_PREFIX=api
So I'm currently hitting the http://inkedfemales.dev/api/photos route to
see the output (404'ing right now)
—
Reply to this email directly or view it on GitHub
https://github.com/dingo/api/issues/533#issuecomment-123873296.

Hey!
I kept running php artisan route:list out of habit, sorry buddy.
I have indeed, I just set up a route:
$api->get('test', function() {
	return ['foo' => 'bar'];
});

When I then hit http://inkedfemales.dev/api/test I still get a 404 - any ideas?

Hm, it all looks good to me and should be working. Must be something
missing though and it's probably something small and obvious.
What kind of 404? A Laravel 404?
On Thu, 23 Jul 2015 07:45 Joe Dawson<EMAIL_ADDRESS>wrote:

Hey!
I kept running php artisan route:list out of habit, sorry buddy.
I have indeed, I just set up a route:
$api->get('test', function() {    return ['foo' => 'bar'];});
When I then hit http://inkedfemales.dev/api/test I still get a 404 - any
ideas?
—
Reply to this email directly or view it on GitHub
https://github.com/dingo/api/issues/533#issuecomment-123876077.

Looks like the usual 404 response.

Could it be that my vendor is empty in my config/api.php? I'm at work all day so can't check until this evening.
Had another peep at the documentation, is this a typo? https://github.com/dingo/api/wiki/Configuration

Vendor
Your vendor name is typically a short name of your application, all lowercase.
You can configure this in your .env file.
API_PREFIX=app

Should this be API_VENDOR?

Double check your configuration. If you enable API_DOMAIN then that domain should be also registered in you http server configuration and available through dns or hosts file.
You can't use both API_DOMAIN and API_PREFIX.

@lightvision I'm just using API_PREFIX

try something like this:
$api->version('v1', function($api) {
            $api->group(['namespace' => $this->namespace], function($api) {
                require app_path('Api/v1/Http/admin_routes.php');
            });
            # other routes here ...
        });

of course the code above assumes that you are using a namespace too.
I am a little bit busy now, but double check your configguration and your api route definition. Everything should be working.

Yeah that documentation is a typo.
I'm not going to be able to look at this properly for a few days as I'm on
the road.
Try another fresh install with the absolute bare minimum.
On Thu, 23 Jul 2015 19:36 Marius Ionel<EMAIL_ADDRESS>wrote:

try something like this:
$api->version('v1', function($api) {
$api->group(['namespace' => $this->namespace], function($api) {
require app_path('Api/v1/Http/admin_routes.php');
});
# other routes here ...
});
of course the code above assumes that you are using a namespace too.
I am a little bit busy now, but double check your configguration and your
api route definition. Everything should be working.
—
Reply to this email directly or view it on GitHub
https://github.com/dingo/api/issues/533#issuecomment-124034003.

No worries, I won't be able to look at it until this evening so will update when I get a chance. Appreciate the help so far!

Guys, please ignore this lol. My vagrant machine hadn't been provisioned.

D'oh!
On Fri, 24 Jul 2015 03:23 Joe Dawson<EMAIL_ADDRESS>wrote:

Guys, please ignore this lol. My vagrant machine hadn't been provisioned.
—
Reply to this email directly or view it on GitHub
https://github.com/dingo/api/issues/533#issuecomment-124172899.

@lightvision , I am trying to migrate API's form site to MAMP. When I run php artisan api:routes show all routes, but when I am try on browser NotFoundHttpException in /Applications/MAMP/htdocs/api-laravel/vendor/laravel/framework/src/Illuminate/Routing/RouteCollection.php line 161:
Please Guide me How I can resolve this issue.

I am facing the same issue.

@punit006 how're you registering your route(s)?

@JoeDawson  Hello Joe. This is my route defination -
$api = app('Dingo\Api\Routing\Router');
Route::get('/', function () {
return view('welcome');
});
$api->version('v1', function ($api) {
$api->get('test', function () {
return 'It is ok';
});
});

That does seem odd, not sure that's something I can help with as my original issue was related to my Homestead server not being correctly configured. (Looks like you're just using php artisan serve)
Hope you can solve it with someones help :)

@punit006 Do you have an API_PREFIX or API_DOMAIN setup in your .env file?

Hi @hskrasek  thanks ! my problem has been solved. Actually i was using API_PREFIX as well as API_DOMAIN. I solved the problem by setting API_DOMAIN value to null. Now i am getting the response but still there seems some problem. My api response -
`[{
"id":1
,"p_name":"Mango",
"p_type":"Fruit",
"p_desc":"ABC",
"p_img":"",
"p_price":"50"
}]
{
"message":"Method gatherRouteMiddlewares does not exist.",
"status_code":500
}`

Can you share your composer.json?
error with gatherRouteMiddlewares has something with laravel 5.3 or a package that expects laravel 5.3

Sure @catalinux , here is my composer.json
{     "name": "laravel/laravel",     "description": "The Laravel Framework.",     "keywords": ["framework", "laravel"],     "license": "MIT",     "type": "project",     "minimum-stability": "stable",     "require": {         "php": ">=5.6.4",         "laravel/framework": "5.3.*",         "dingo/api": "1.0.*@dev"     },     "require-dev": {         "fzaninotto/faker": "~1.4",         "mockery/mockery": "0.9.*",         "phpunit/phpunit": "~5.0",         "symfony/css-selector": "3.1.*",         "symfony/dom-crawler": "3.1.*"     },     "autoload": {         "classmap": [             "database"         ],         "psr-4": {             "App\\": "app/"         }     },     "autoload-dev": {         "classmap": [             "tests/TestCase.php"         ]     },     "scripts": {         "post-root-package-install": [             "php -r \"file_exists('.env') || copy('.env.example', '.env');\""         ],         "post-create-project-cmd": [             "php artisan key:generate"         ],         "post-install-cmd": [             "Illuminate\\Foundation\\ComposerScripts::postInstall",             "php artisan optimize"         ],         "post-update-cmd": [             "Illuminate\\Foundation\\ComposerScripts::postUpdate",             "php artisan optimize"         ]     },     "config": {         "preferred-install": "dist"     } }

@punit006 see #1165
Someone has submitted a PR for a fix. #1152


JSON export broken when using ujson

When ujson is installed, export_set and export_book fail with
# ...
File "/usr/local/lib/python3.6/site-packages/tablib/formats/_json.py", line 30, in export_set
    return json.dumps(dataset.dict, default=date_handler)
TypeError: 'default' is an invalid keyword argument for this function

This is because ujson.dumps does not support the default kwarg and does not plan to in the future.

I have the same issue.

If you need/want to keep the ujson module installed, as I do, you can also monkeypatch tablib:
import json
import tablib

tablib.formats.json.json = json


Unable to control the light

Integration adds the light and it's features, but unable to control it. When toggling switch on it turns off without any changes to the light. No errors that I could see in the logs.

Hi, try to enable strict mode in config and you will see errors in log

Thank you! Now at least I see an error! I'm getting the following
Update for light.bedstrip fails
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/sp110e/controller.py", line 208, in _connect_with_retries
    await self._driver.connect(self._mac, timeout=self._timeout)
  File "/usr/local/lib/python3.10/site-packages/sp110e/driver.py", line 42, in connect
    device = await BleakScanner.find_device_by_address(mac_address, timeout=timeout)
AttributeError: type object 'HaBleakScannerWrapper' has no attribute 'find_device_by_address'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 527, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 722, in async_device_update
    raise exc
  File "/config/custom_components/sp110e/light.py", line 220, in async_update
    self._handle_exception(exception)
  File "/config/custom_components/sp110e/light.py", line 246, in _handle_exception
    raise exception
  File "/config/custom_components/sp110e/light.py", line 217, in async_update
    await self._device.update()
  File "/usr/local/lib/python3.10/site-packages/sp110e/controller.py", line 60, in update
    await self._connect_with_retries()
  File "/usr/local/lib/python3.10/site-packages/sp110e/controller.py", line 211, in _connect_with_retries
    raise Exception
Exception

Hm, is it problem with Bleak version?
Unfortunately I don't have access to my HASS to test it.

get the same error

And one more with this issue.


The REPL is not reporting some parsing errors

We allow definition of type aliases starting with lower case letters, but these cannot be referenced since types with lowercase letters are interpreted as type variables:
>>> type s = int

>>> def g(x: s): s = x

>>> g("arst")
"arst"

You'd expect g to be int => int given the alias, but it is actually typed forall s. s => s.
We should fix the parser so it rejects type defs that don't start with a capital letter.

@bugarela helped me work out that this is not a parsing oversight but just the REPL failing to report the parsing error, if we instead parse that code via the CLI, we get the expected error.


DRIMSEQ_FILTER error

Description of the bug
Dear All, I am testing the pipeline and I am facing the following error,
Not all conditions are included in the contrast file.
Thanks for your support
Error executing process > 'NFCORE_RNASPLICE:RNASPLICE:DRIMSEQ_DEXSEQ_DTU_SALMON:DRIMSEQ_FILTER (1)'

Caused by:
  Process `NFCORE_RNASPLICE:RNASPLICE:DRIMSEQ_DEXSEQ_DTU_SALMON:DRIMSEQ_FILTER (1)` terminated with an error exit status (1)

Command executed:

  run_drimseq_filter.R salmon.merged.txi.dtu.rds q.tx2gene.tsv samplesheet.csv \
      4 \
      2 \
      2 \
      10 \
      0.1 \
      10

  cat <<-END_VERSIONS > versions.yml
  "NFCORE_RNASPLICE:RNASPLICE:DRIMSEQ_DEXSEQ_DTU_SALMON:DRIMSEQ_FILTER":
      r-base: $(echo $(R --version 2>&1) | sed 's/^.*R version //; s/ .*$//')
      bioconductor-drimseq: $(Rscript -e "library(DRIMSeq); cat(as.character(packageVersion('DRIMSeq')))")
  END_VERSIONS

Command exit status:
  1

Command output:
  (empty)

Command error:

  Attaching package: 'DRIMSeq'

  The following object is masked from 'package:base':

      proportions

  Error in .local(x, ...) :
    min_samps_gene_expr >= 0 && min_samps_gene_expr <= ncol(x@counts) is not TRUE
  Calls: <Anonymous> -> <Anonymous> -> .local -> stopifnot
  Execution halted

Command used and terminal output
nextflow run nf-core/rnasplice --input $wd/samplesheet.csv \
							   --contrasts $wd/contrast.csv \
							   --outdir test \
							   --genome GRCm38 \
							   -profile singularity \
							   --aligner star \
							   --dexseq_exon \
							   --edger_exon \
							   --rmats \
							   --rmats_read_len 75 \
							   --save_reference true \
							   --sashimi_plot true \
							   -config config.config \
							   -r dev

Relevant files
nextflow.log
System information
nextflow/22.10.4
HPC - slurm
Operating System: Debian GNU/Linux 10 (buster)

Hi @paolo-kunderfranco
Please could you try to re-run the pipeline with the following dmFilter parameters set to zero:

min_samps_gene_expr
min_samps_feature_expr
min_samps_feature_prop
min_feature_expr
min_feature_prop
min_gene_expr

This can be done in the nextflow.config file - we are trying to ascertain whether there is a bug or the filters are too stringent for your samples.

ok now it seems working, but now I am again as always with different pipelines in STAR segmentantion error.

I believe this is a known issue with STAR 2.7.10a but the nf-core modules have not been updated. In the meantime, if you are able to use conda, you can download the pipeline and replace bioconda::star=2.7.10a with bioconda::star=2.7.10b in each of the main.nf files for the STAR modules.

you mean here?
for both *genomegenerate and *align modules?
`process STAR_ALIGN {
tag "$meta.id"
label 'process_high'
conda "bioconda::star=2.7.10a bioconda::samtools=1.16.1 conda-forge::gawk=5.1.0"
container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?

`

Yes, that's right. Give it a go and report back.

error still persist
[d5/15de98] NOTE: Process NFCORE_RNASPLICE:RNASPLICE:ALIGN_STAR:STAR_ALIGN_IGENOMES (Brain55)` terminated with an error exit status (139) -- Execution is retried (1)
Error executing process > 'NFCORE_RNASPLICE:RNASPLICE:ALIGN_STAR:STAR_ALIGN_IGENOMES (CP28)'
Caused by:
Process NFCORE_RNASPLICE:RNASPLICE:ALIGN_STAR:STAR_ALIGN_IGENOMES (CP28) terminated with an error exit status (139)
Command executed:
STAR
--genomeDir STARIndex
--readFilesIn CP28_1_val_1.fq.gz CP28_2_val_2.fq.gz
--runThreadN 16
--outFileNamePrefix CP28.

--sjdbGTFfile genes.gtf
--outSAMattrRGline ID:CP28 'SM:CP28'
--quantMode TranscriptomeSAM --twopassMode Basic --outSAMtype BAM Unsorted --readFilesCommand gunzip -c --runRNGseed 0 --outFilterMultimapNmax 20 --alignSJDBoverhangMin 1 --outSAMattributes NH HI AS NM MD --quantTranscriptomeBan Singleend
if [ -f CP28.Unmapped.out.mate1 ]; then
mv CP28.Unmapped.out.mate1 CP28.unmapped_1.fastq
gzip CP28.unmapped_1.fastq
fi
if [ -f CP28.Unmapped.out.mate2 ]; then
mv CP28.Unmapped.out.mate2 CP28.unmapped_2.fastq
gzip CP28.unmapped_2.fastq
fi
cat <<-END_VERSIONS > versions.yml
"NFCORE_RNASPLICE:RNASPLICE:ALIGN_STAR:STAR_ALIGN_IGENOMES":
star: $(STAR --version | sed -e "s/STAR_//g")
samtools: $(echo $(samtools --version 2>&1) | sed 's/^.samtools //; s/Using.$//')
gawk: $(echo $(gawk --version 2>&1) | sed 's/^.GNU Awk //; s/, .$//')
END_VERSIONS
Command exit status:
139
Command output:
(empty)
Command error:
.command.sh: line 10: 10242 Segmentation fault      STAR --genomeDir STARIndex --readFilesIn CP28_1_val_1.fq.gz CP28_2_val_2.fq.gz --runThreadN 16 --outFileNamePrefix CP28. --sjdbGTFfile genes.gtf --outSAMattrRGline ID:CP28 'SM:CP28' --quantMode TranscriptomeSAM --twopassMode Basic --outSAMtype BAM Unsorted --readFilesCommand gunzip -c --runRNGseed 0 --outFilterMultimapNmax 20 --alignSJDBoverhangMin 1 --outSAMattributes NH HI AS NM MD --quantTranscriptomeBan Singleend
Work dir:
/home/pkf/gendata2/bioinformatica/pkunderfranco/data/P107_Carloni/nfcore_splicing/work/5d/65bc17ae054eeef25bddcd3039b282
Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out``

any suggestion??

Hi @paolo-kunderfranco are you running with singularity? If so could you try running with conda and the changed versions as @jma1991 suggested? Trying with docker would also give us some useful information.

Hi @bensouthgate , I succesfully align samples with docker, but failed in SUPPA

Hi @paolo-kunderfranco - if you no longer have an issue with DRIMSEQ_FILTER or STAR_ALIGN are you happy for me to close this issue? I believe we are working on another issue you opened for SUPPA #71.


Add WYSIWYG editor on milestone form

Gitea version (or commit ref): 1.13.4
Git version: 2.20.1
Operating system: Debian 10, using binary package from Gitea, running with systemd.
Database:

[ ] PostgreSQL
[x] MySQL
[ ] MSSQL
[ ] SQLite

Can you reproduce the bug at https://try.gitea.io:

[x] Yes (https://try.gitea.io/Cypouz/Test/milestone/1335)
[ ] No

Log gist:

Description
Milestone form doesn't propose a WYSIWYG form like the one proposed by Issue form.
Screenshots

Even github didn't do that. Is this necessary?

I think it's fine for this to stay plaintext. ideally with a markdown preview tab, but that should be part of a future editor component, not added manually to this template.

Since https://github.com/go-gitea/gitea/pull/23876 we default to plaintext editor, so this is not planned.

I guess we can keep this open , and introduce the new editor to the milestone page?
At the moment, there is only a simple textarea, no toolbar, no preview

Yes, all places that allow Markdown and generally hold more than 1 line of content could use the new editor.

Agree, markdown editor could help with long text editing, like comment, release description, so milestone description is similar.
Org/user description is usually not that long, so a simple textarea is enough.

@wxiaoguang v1.23.0 will support markdown editor like a comment?

Yes, I am working on it, there will be a PR

-> Refactor markdown editor and use it for milestone description editor #32688
(actually, there is not real WYSIWYG editor in Gitea, but markdown editor)


Opponent Info plugin: add toggle to only show boss health overlay

Allows the Opponent Info plugin's custom health overlay to be toggled off, so that only the in-game setting 'boss health overlay' is affected.
Use case is I want to have both exact health amounts and percentage displayed on boss health bars (below), but I do not want the custom overlay displayed for non-boss monsters or players.

This is the overlay that is being made toggleable (example of an NPC and a player):

This overlay is unaffected, can still be toggled on/off independent of the new config:

The comment you are editing says it already does the behavior you are adding?
// The in-game hp hud is more accurate than our overlay and duplicates all of the information on it,
// so hide ours if it is visible.

The comment you are editing says it already does the behavior you are adding?
// The in-game hp hud is more accurate than our overlay and duplicates all of the information on it,
// so hide ours if it is visible.

The small health overlay is already hidden when there is a boss health bar visible, yes. However, there is no way to toggle off the small health overlay when interacting with players or non-boss NPCs.
I am adding the ability to always hide the small health overlay, while retaining the ability to customize the boss health overlay.


openstackclient pod dynamic Ansible inventory

This patch implements a dynamic Ansible inventory script
for the openstackclient pod. This inventory queries
the openstacknets API and parses out the relevant
host IPs which are then formatted in json for Ansible.
Docs are updated to recommend the new inventory is used
instead of having to rely on the static playbook generated
inventory we were using.
The motivation for this patch was to eliminate the dependency
on playbook generation to complete before node registration
could be accomplished, thus decoupling the two mechanism. This
dependency was a blocker for further deployment workflow changes.

This is very close but I seem to have broken something last minute as the pod hangs on init. Will have another look in the morning.

added the roles feature, and then ansible vars for ssh. still no dice on this yet. will have a closer look at tripleo-ansible-inventory.yaml tomorrow and see what might be missing

This is a go. Seems to work fine now that I reinstalled OCP.


doc/website: add link to slack channel

@a8m the link does not work. it just takes me to a sign in page where it asks "Enter your workspace’s Slack URL"

Hey @roffe,
I'm new to Slack, so not sure why. We created a channel under the "Gophers Slack" workspace (https://gophers.slack.com) and it works for me.
@cliedeman, @rotemtam any chance do you know what's the reason for that?

The Gophers slack is invite only, you have to first register using https://invite.slack.golangbridge.org/ then you can join the channel


Adding swagger to authoring api

closes #542

What's the swagger path?

The path is /swagger, if you want json then it's /swagger/json.

Let's check author works properly with editor authentication.


Feat  new locking

Test

[x] Config
[x] Context
[x] Control
[x] Dependencies
[x] fsck?
[x] disk init
[x] install cleanup
[x] install update
[x] manager health
[x] Util Logger
[x] Version?

Test Install/ Uninstall
Videos showing a removal and install of service
https://user-images.githubusercontent.com/2364004/164785705-210c1bb3-8bfb-4a4b-b112-e3e162ffdcb9.mp4
https://user-images.githubusercontent.com/2364004/164785712-43b14e47-c4c1-4b97-8e28-7239a7c6e857.mp4
Test - Config
https://user-images.githubusercontent.com/2364004/164756599-24730c8f-14ef-41f2-85aa-dbd079bc2948.mp4
Test - Logs
Snap of the system logs

Snap of an app logs

Test - Version
Snap of version

Reviewed in person.

Why is the OOMkill prevention commit in here? Wasn't #1402 merged a while back?

Merging this without the patchdb change being merged seems sus...

I'm not sure how this got merged - I didn't even look at this today...

Ok, so apparently if you force push a branch that has already been merged to a branch that has an open pr, github thinks that it has already been merged.

Something weird happened; I think I pushed a older version of master up, so this pr was a no-op, and got auto merged or had no protections?
Moved to pr https://github.com/Start9Labs/embassy-os/pull/1422


Logger rename error on Windows file system

When renaming log file it includes reserved character for Windows file system that causes EINVAL error code

Fixed


0.6.0 rc Rebase

Modulus Pull Request
Description
Pull changes from the 0.6.0-rc branch (this was rebased once during the regular release, but we need to rebase again as some changes went in as a part of the SFB process)
Checklist

[ ] I am familiar with the Contributing Guidelines.
[ ] New or existing tests cover these changes.
[ ] The documentation is up to date with these changes.
[ ] The CHANGELOG.md is up to date with these changes.
[ ] An issue is linked to this pull request.

Dependencies

/blossom-ci


:triangular_flag_on_post: Scan cache tool: ability to free up space

Originally from @stas00 in slack (internal link):
for me the main query / need is usually to free up some disk space and so I'd look at the top entries of these 3 groups:

nuking the largest entries (obvious)
nuking the long not accessed entries (sorted by access time)
may be also nuking the oldest entries (probably not using them anymore) (i.e. sorted by file create time) - but most likely 2. would have already caught this category

In general, this is a feature request already discussed in https://github.com/huggingface/huggingface_hub/pull/990. The approach of "pruning" we are aiming is to provide the user information and a tool to delete a specific revision and then make it as easy as possible for the user to define its own strategy.

Related to this: https://github.com/huggingface/huggingface_hub/issues/1013. However for now we are more in an approach of "let the user decide" than automatically deleting stuff.

maybe a combination of --limit 5 and --delete in the CLI to actually rm the corresponding files?

I think a very useful approach would be to be able to sort plus dump ready to delete commands, as in:
rm -rf ~/.cache/huggingface/hub/models--valhalla--t5-base-squad
rm -rf ~/.cache/huggingface/hub/models--nielsr--deformable-detr
rm -rf ~/.cache/huggingface/hub/models--gpt2-large
[...]

That way now it's really trivial for a user to copy-n-paste and also edit the list if they don't want to delete something.
or perhaps this format?
# 12.2GB
rm -rf ~/.cache/huggingface/hub/models--valhalla--t5-base-squad
# 10GB
rm -rf ~/.cache/huggingface/hub/models--nielsr--deformable-detr
# 5GB
rm -rf ~/.cache/huggingface/hub/models--gpt2-large
[...]

or more detailed, but still friendly to be run in shell
# 12.2GB / last used 2 months ago
rm -rf ~/.cache/huggingface/hub/models--valhalla--t5-base-squad
# 10GB / last used 2 days ago
rm -rf ~/.cache/huggingface/hub/models--nielsr--deformable-detr
# 5GB / last used 1 year
rm -rf ~/.cache/huggingface/hub/models--gpt2-large
[...]

so as it'd be already sorted either by size or access I could then dump the output in the editor, massage it and then run it.

@stas00 I really like the idea of outputting a list of commands that the user can edit manually before deleting.
However I'm not sure we can do that with a list of rm -rf ... commands as deleting specific revisions is a bit more complex than deleting an entire folder. The rm -rf .. approach works perfectly fine if someone wants to delete entire repos ("i don't need gpt2-large" for example) but I think a big use case we have is people wanting the delete all past revisions expect main.
To delete a revision, we need to delete the corresponding snapshot folder containing the symlinks but also the blobs that were referred only by this revision (and not the others). So doing it with rm -rf ... commands would need to output potentially a giant command with all blob files listed 1 by 1 which is impossible for a dataset repo with thousands of files.
We can still keep the idea of printing a list to edit but maybe it could be a list of commit_hash:
# 12.2GB
huggingface_cli scan-cache --delete e2983b237dccf3ab4937c97fa717319a9ca1a96d 6c0e6080953db56375760c0471a8c5f2929baf11
# 10GB
huggingface_cli scan-cache --delete 0e4ab15e8865b843b3a348275facfd198cf22f03
# 5GB
huggingface_cli scan-cache --delete 60b8d3fe22aebb024b573f1cca224db3126d10f3
[...]

About the last access time, I am not sure how reliable it is on different OS. Download time can be a more reliable indicator but less interesting (cc @julien-c 's example: gpt2 is a few years old in his cache but still useful to keep). Also, access time can "easily" be determined for a full repo but a lot more difficult for specific revisions as they shared the same blobs.
I'm not 100% against the idea, I'm just concerned how we can make this information useful to the users.

Oh, right! I didn't think of the revisions!
This is super important!
So actually a lot of the times the user will want to wipe out all outdated revisions but the current one.
Should we provide an API for deletion then?
huggingface_cli delete-cache entry-name <revision name | all>

or something like that?
and this also means that the scan should print out the breakdown of each revision's space it's taking, no?

re: access time
We could start with just full repo access - and combined with --keep-last that should work really well.
So if someone asked me to test some code with its repo and I don't normally use it I could easily wipe it out. It'd do it in 2 steps

delete all revisions that aren't current - across all of cache:

huggingface_cli delete-cache --all --keep--last

note: I added another flag --all which I didn't propose in one comment up.

now run scan-cache and sort by last accessed

now I can delete repos I don't really use
How is that?

Thanks for all the ideas :)
I start to think that we should keep your idea of generating editable commands to keep it as simple and flexible as possible. What I'm afraid of with all the --all, --keep-last, --sort, --filter, --limit, etc arguments is that we will end up with a quite complex CLI that doesn't cover absolutely all cases (and therefore we would have feature requests for specific cases).
A realistic possibility I see is to add a --dry-run mode to generate the list of revisions that will be deleted. We can split the work in different parts:
1. --dry-run to output revisions to delete
The --dry-run mode outputs a human-readable/human-editable list of revisions. Every model is listed with download time, last access time and size. For each repo, we list all the revision hashes and their corresponding refs (or (detached)). Revisions that are planned to be kept are commented out while revisions to delete are not.
In a --keep-last example, the revision associated to main is commented out, followed by all revisions that will be deleted:
huggingface_cli delete-cache --keep-last --dry-run

## Model valhalla/t5-base-squad
## Downloaded 1 year ago. Last used 2 months ago
## Total size: 12.2G. Would delete 7.4G.
# 3f4b5290db15d7d3821c190acef6fa7c40069da1 # main, 0.4.1
e2983b237dccf3ab4937c97fa717319a9ca1a96d # refs/pr/1, 0.2.0
6c0e6080953db56375760c0471a8c5f2929baf11 # (detached)

## Model nielsr/deformable-detr
## Downloaded 2 days ago. Last used 2 days ago
## Total size: 10G. Would delete 0.
# 0e4ab15e8865b843b3a348275facfd198cf22f03  # main

## Dataset SLPL/naab
## Download 1 year ago. Last used 5 days ago
## Total size: 7.2G. Would delete 5G.
# 60b8d3fe22aebb024b573f1cca224db3126d10f3 # main
a5ae8a094021fa4094b270166440d4a165939446 # (detached)

[...]

2. Pass a list of revisions to delete-cache
The above list can be manually edited and then passed again to the delete-cache command. The command should be able to parse the list of revisions to delete by splitting the "#" correctly.
eval("huggingface_cli delete-cache --keep-last --dry-run") > revisions.txt
# or "huggingface_cli delete-cache --keep-last --dry-run --export=revisions.txt" ?

(...) # edit revisions.txt manually

cat revisions.txt | huggingface_cli delete-cache

2.b. Eventually pass revision hash manually
# Single revision
huggingface_cli delete-cache a5ae8a094021fa4094b270166440d4a165939446

# Or multiple revisions
huggingface_cli delete-cache e2983b237dccf3ab4937c97fa717319a9ca1a96d 6c0e6080953db56375760c0471a8c5f2929baf11

3. Bonus: more options
I put those options as "bonus" since they don't need to exist to make previous steps work. I think we should start we --dry-run and --keep-last and then discuss more complex options in different issues/PRs.

--keep-last to keep only main revision

huggingface_cli delete-cache --keep-last

--filter="repo_id_pattern" to filter repos

huggingface_cli delete-cache --filter="allenai/*"

--sort=key to sort repos and --limit=5 to truncate the output

huggingface_cli delete-cache --sort="total_size" --limit=5

--filter, --sort and --limit are typically arguments that we want to be common to both scan-cache and delete-cache.
What do you think about this proposition ?

Note:

Revision hashes are unique across repos so no need to provide "repo_type + repo_id + revision hash" to describe a revision; "revision hash" is enough
We would need to have doc for windows users as well. Absolutely no idea how pipes are working there.

That would work, thank you for the super-detailed preview of how it might work, @Wauplin
I think I'd rather --dry-run generate the actual delete commands, so the user can either edit down the output and run it directly or simply copy-n-paste the specific non-dry deletes they want - more versatility, no?
So:
huggingface_cli delete-cache --keep-last --dry-run > cmds.txt
# edit cmds.txt
sh cmds.txt
# or copy-n-paste and execute just specific commands from cmds.txt

--keep-last would of course not generate any delete commands if the cache entry only has the last entry, correct?
and of course eventually it'd be great to have a gui tool where one can quickly select what's wanted/unwanted and delete in one click. something like https://en.wikipedia.org/wiki/Disk_Usage_Analyzer, which also visually shows which is bigger. But of course, this can happen much later (and the difficulty would be to make it cross-platform).

eval("huggingface_cli delete-cache --keep-last --dry-run") > revisions.txt

what is the eval for? won't just huggingface_cli delete-cache --keep-last --dry-run" > revisions.txt work?

We would need to have doc for windows users as well. Absolutely no idea how pipes are working there.

on windows it'd depend on the shell they use - some work like bash, others are more limited.

I think I'd rather --dry-run generate the actual delete commands, so the user can either edit down the output and run it directly or simply copy-n-paste the specific non-dry deletes they want - more versatility, no?

I see the idea but I am not sure how the cmds.txt content would be. Do you see it as a list of huggingface_cli delete-cache commands like this:
## Model valhalla/t5-base-squad
## Downloaded 1 year ago. Last used 2 months ago
## Total size: 12.2G. Would delete 7.4G.
# huggingface_cli delete-cache 3f4b5290db15d7d3821c190acef6fa7c40069da1 # main, 0.4.1
huggingface_cli delete-cache e2983b237dccf3ab4937c97fa717319a9ca1a96d # refs/pr/1, 0.2.0
huggingface_cli delete-cache 6c0e6080953db56375760c0471a8c5f2929baf11 # (detached)

## Model nielsr/deformable-detr
## Downloaded 2 days ago. Last used 2 days ago
## Total size: 10G. Would delete 0.
# huggingface_cli delete-cache 0e4ab15e8865b843b3a348275facfd198cf22f03  # main

## Dataset SLPL/naab
## Download 1 year ago. Last used 5 days ago
## Total size: 7.2G. Would delete 5G.
# huggingface_cli delete-cache 60b8d3fe22aebb024b573f1cca224db3126d10f3 # main
huggingface_cli delete-cache a5ae8a094021fa4094b270166440d4a165939446 # (detached)

?
If yes I'm just worried that it would be sub-optimal as each delete-cache command first require to scan the cache folder so potentially we scan 100 times to delete a 100 revisions (instead of 1 for dry-run and 1 for execution). With the previous snippet from https://github.com/huggingface/huggingface_hub/issues/1025#issuecomment-1236690038, the user would have to build its own huggingface_cli delete-cache a5ae8a... 094021... commands itself for manual deletion. Maybe we could have a comment at the beginning of the generated file/output to explain how to use it ?
(also I find it very verbose in cmds.txt compared to just hashes)

eval("huggingface_cli delete-cache --keep-last --dry-run") > revisions.txt

My bad, I was sure the "> revisions.txt" was parsed by python argparse which we don't want but that's not the case.

and my proposal includes no pipes

True, that's an advantage to generate a file of commands. Maybe we could have an option to take as input a file instead of using a pipe e.g.:
huggingface_cli delete-cache --keep-last --dry-run > revisions.txt

(...)

huggingface_cli delete-cache --from-dry-run=revisions.txt

1.

why do you need to re-scan the cache if you are already given the exact hash to delete?

Even if I know the hash of the revision I want to delete I need to make sure the blob files that I delete are not referenced by another revision in the repo.
repo_A
    blobs
        aaaaaa
        bbbbbb
        cccccc
    snapshots
        123456
             config.json -> ./repo_Ablobs/aaaaaa
             some_data.txt -> ./repo_A/blobs/bbbbbb
        789101
             config.json -> ./repo_A/blobs/aaaaaa
             some_data.txt -> ./repo_A/blobs/cccccc

Let's imagine a huggingface-cli delete-cache 789101 command. I want to delete blob cccccc but not the blob aaaaaa. To know that I need to scan both revisions in order to check which blob is referenced are not. Knowing the revision id to delete is the only information I need from the user but it's not enough to know which blobs to delete, hence the scan.
Just to mention it, it could be somehow possible to count the number of symlinks referrecing a blob on some platforms (Linux for instance) but I prefer to avoid that.
2.

perhaps this is simply the case of a missing indexing component? i.e. adding an index that always gets updated on adding/deleting revisions and never require a manual rescan if the entrypoint is provided by the user - a hash in this case?
of course such index should be able to invalidate entries if a user deleted them manually and not through the API.

Yes, a proper index at the root of the cache would solve this problem. However I feel a bit lazy to invest time building an index component that would need to be robust enough when inserting/deleting files only to get a faster cache scan. If it really starts to be a burden for the users to scan the cache directory then we could think about it but here we just want to ship a "free up some space" feature.
A very rough benchmark on my local "5k-files"-cache takes ~300ms. For a user with 1M cached files (!) we can estimate the scan to ~1min which shouldn't be to much of a hassle for the vast majority of our users, right ?
3.

but you can't act on hashes directly, you have to manually add them to a command that isn't there.
but why not have both ways and let the user choose? --dump-hash-only if you prefer the terse version?

Yes that's true that it's not as convenient/optimal as having a cache index and a cmds.txt file with commands already ready-to-use. However let's remember that it will only be used by "power users" that will run a --dry-run and then analyse the file line by line and then execute only parts of it.
I expect most users to use the huggingface_cli delete-cache --keep-last command and that's it. If you don't see a too big inconvenient, I think I'll start to move forward on this feature. Other options and additions will still be possible later :)

Overall, thanks a lot for the time you took to give feedback on this feature. It really helped a lot !

Even if I know the hash of the revision I want to delete I need to make sure the blob files that I delete are not referenced by another revision in the repo.

OK, so you don't need to rescan the whole cache - just the specific repo - and so on average you'd have just a few entries there - shouldn't really be a burden, no?
I agree that having a proper index can be a nice-to-have one-day feature.

I expect most users to use the huggingface_cli delete-cache --keep-last command and that's it. If you don't see a too big inconvenient, I think I'll start to move forward on this feature. Other options and additions will still be possible later :)

Indeed, but this is only one need.
The other 2 important needs are to be able to identify and purge whole repos that either:
(a) haven't been used in a long time - dead weight
(b) are too large - if one needs urgently a disc space a user can choose to delete a few repos and later restore them. it's easier to nuke the largest repo in this case.

OK, so you don't need to rescan the whole cache - just the specific repo - and so on average you'd have just 2 or few more revisions there - shouldn't really be a big overhead, no? Repos with 1 revision won't even show up in this use case.

Yes makes sense. Big and time-consuming scans will be needed only on large datasets with a lot of revisions. For this we can't do much anyway.

So I propose prune-cache == delete-cache --keep-last

About prune-cache, delete-cache and options, let's keep the idea here. It really makes sense. Once the core-part will be implement, those will be cosmetic adjustments of the CLI.
I think we are good to go know ! :fire:

Another idea popped-up when I started to work on it. Couldn't we just use an external library to make a nice multi-select view in the terminal ?
The user can:

select revisions to delete 1 by 1
select an entire repo to delete

From a process point of view:

We scan the cache once (we cannot get rid of this part)
Given the --keep-last, --filter, --sort=size, --limit,... options we preselect a set of revisions/repos
The user can then choose which ones to delete
Each time the user selects a new entry, we compute how much space would be saved (fast to compute since no re-scan)
Once selected, we prompt a "Are you sure [y/N]" and that's it.

@stas00 @julien-c What do you think about that ? No need for a dry-run mode, no need for an external file/output, no need to copy-paste commands,... We can even have a --skip-confirm to disable the selection menu.
This adds an external dependency but it doesn't seem too problematic to me as it is only cli-related (let's make it optional). I found this package (simple-term-menu) which can easily do the job. I made a small demo that can be improved in term of wording/information displayed :arrow_down:
Screencast from 09-09-2022 12:46:48.webm

I think that's a great idea, @Wauplin, as long as you don't put yourself into a situation of then committing to figure out how to make it work cross-platform.
We can of course make a statement that it'd only work in some cases but not always, and in that case use the manual approach?
If we go with your proposal I'd just suggest to add the memory usage and perhaps last time used time stamp next to each selectable option.

Could idea @stas00. Let's make this a beta-feature that is optional. It seems there are some minor issues on Windows but yeah I definitely don't want to debug that. Previous message was just a demo but I'll do a more careful research to use a package that seems stable enough and that does the job.

Yes definitely. This was a simple example but more information would be better. I also thought that I could display only the first X characters instead of the full revision name. In the background I would still have the long version but the user only needs a short id I think (e.g. 60b8d3fe instead of 60b8d3fe22aebb024b573f1cca224db3126d10f3).

What I wanted to do is:

when all revisions of a repo are selected, automatically select the repo as well
when all revisions and the repo are selected and the user deselects a revision, the repo is automatically de-selected
when a user selects a repo, all revisions are automatically selected
when a user de-selects a repo, all revisions are automatically de-selected

By doing this, we ensure repo/revisions selections are always consistent and the user can understand it. I think it's possible to automatically selects/deselects items when the user selects/deselects one with a callback somewhere.

All sounds great.

that I could display only the first X characters instead of the full revision name. In the background

Yes, that's exactly how git operates, you only need the first X unique characters of the cache - usually 8 is sufficient - great idea on having the caching support short strings.
Also let's make sure that there is the manual version in addition to the GUI, that way if gui doesn't work for a user they always have the fallback.

Also let's make sure that there is the manual version in addition to the GUI, that way if gui doesn't work for a user they always have the fallback.

@stas00  yes indeed. That's now done with the --disable-tui flag. I tried to keep the process as close as possible between the TUI mode and the no-TUI mode. Without TUI, the user have to manually edit a temporary file that has been generated.
See documentation for both mode here.

that's perfect, thank you, @Wauplin

one more addition I have just thought of - should this feature also wipe out clean the downloads dir as it often has all kinds of left-overs and it's not always obvious that it can be safely wiped out.
Thank you!

What is the download directory for you ? Can you run tree on it so that I see what we are talking about?

$ find ~/.cache/huggingface/ -type d -name "downloads"
/home/stas/.cache/huggingface/evaluate/downloads
/home/stas/.cache/huggingface/datasets/downloads

$ du -s /home/stas/.cache/huggingface/datasets/downloads
141G    /home/stas/.cache/huggingface/datasets/downloads

$ tree /home/stas/.cache/huggingface/datasets/downloads
├── 078890d95f804daa5ec1175957649d92c3006dde04e9b584454c42bd78ebc241.991ca2ccf495f2a8f6a30a8acf52a1c41174aaf5054cdab0e373a855b6d0a747.py
├── 078890d95f804daa5ec1175957649d92c3006dde04e9b584454c42bd78ebc241.991ca2ccf495f2a8f6a30a8acf52a1c41174aaf5054cdab0e373a855b6d0a747.py.json
├── 078890d95f804daa5ec1175957649d92c3006dde04e9b584454c42bd78ebc241.991ca2ccf495f2a8f6a30a8acf52a1c41174aaf5054cdab0e373a855b6d0a747.py.lock
├── 11ba2d79cea2e93248b0441726361c9bb586ed8c2b366d3eca0e2a09b09560cf
├── 11ba2d79cea2e93248b0441726361c9bb586ed8c2b366d3eca0e2a09b09560cf.json
├── 11ba2d79cea2e93248b0441726361c9bb586ed8c2b366d3eca0e2a09b09560cf.lock
├── 22cc09b66984393aefb3ad73a828ca57357f3a8401195388305da809d3418088
├── 22cc09b66984393aefb3ad73a828ca57357f3a8401195388305da809d3418088.json
├── 22cc09b66984393aefb3ad73a828ca57357f3a8401195388305da809d3418088.lock
[...]
├── extracted
│   ├── 08b49d88c65e1b539a4c3f5ceed84425b29c5f4b41c8679730a03158c7eb99e3
│   │   └── wikitext-2
│   │       ├── wiki.test.tokens
│   │       ├── wiki.train.tokens
│   │       └── wiki.valid.tokens
[...]
│   ├── ab2462bd81d79dc0ec989ab6c93d61decf0d806ef975a44bddd8a6d7e69c798a
│   │   ├── legal
│   │   │   └── Legal notices.txt
│   │   ├── rapid2019.de-en.de
│   │   └── rapid2019.de-en.en
│   ├── cd100a34a529fcef7abec93d9ba199b79b639c1b908bd7d1686c9e6c2b0e0932
│   │   └── wikitext-103
│   │       ├── wiki.test.tokens
│   │       ├── wiki.train.tokens
│   │       └── wiki.valid.tokens
[....]

as you can see it's full of things that were downloaded from the hub and also things extracted from what was downloaded - a ton of stuff that was temporarily used.

that's datasets (or evaluate) specific though so out of scope from this repo IMO

oh, right!
So to manage datasets caches we should start a similar query at datasets? As this is another huge disc space sinker in addition to models.

I believe the datasets team is also in the process/thinking about integrating the updated cache within their caching system, so let's revisit this once that's taken care of
cc @lhoestq

(and @Wauplin don't hesitate to help the datasets team unify/upgrade their Hub integration 🎉 )

(yes I am in contact with @lhoestq about that. I'm planning to work on that mid-week for having a good integration :) )

Closing this issue as a first version of huggingface-cli delete-cache is implemented :)
I created a new issue to keep track of the ideas we mentionned https://github.com/huggingface/huggingface_hub/issues/1065.


Implement 'hasOwn' routine of Object object

JerryScript-DCO-1.0-Signed-off-by: Csaba Repasi<EMAIL_ADDRESS>

It does not fix any test262 tests, is it a new standard?

It does not fix any test262 tests, is it a new standard?

We need to update the test262 runner to the latest master. cc @ossy-szeged


Need to add a marker depending of what is selected inside the player

Hello ,
I need to add a marker following the scene selected inside the player (not depeding of the javascript time)
$(".add").click(function(){
player.markers.add([{
time: 25, // I don't know wich time I need bu implement it depeding the what I selected inside the player
text: "marker"
}]);
Exemple here 👍
https://www.hostingpics.net/viewer.php?id=252248Sanstitre.png
Is there anyway to do it ?
Thanks

Add a marker to the current time of the player:
player.markers.add([{
      time: player.currentTime()
}]);

Many thanks for your help Borzaka. That's the need !

Thanks for replying @borzaka !


add tests for bfloat16 on HIP

just making this separate PR with the tests I wrote from the #3558 closed PR.

Cool!


Fixed #33872 -- Deprecated django.contrib.postgres.fields.CIText/CICharField/CIEmailField/CITextField.

ticket-33872

This PR look good to me , but checking this PR I discovered that, in the "Model field reference" documentation page, the TextField.db_collation is not declared "Optional." in its description as CharField.db_collation.

Thanks for all reviews :1st_place_medal:


Add alternative smartquotes

First implementation of #657.
For now, this implements the following alternative smartquotes (taken from Wikipedia):

German (Switzerland and Liechtenstein) and the Norwegian languages: The otherwhere regular German quotes (single ‚…‘, double „…“)
Czech, Danish, German, Slovak, Slovene: Inwards pointing angular quotes (single ›…‹, double »…«)
French: English quotes, with french spacing
Finnish, Swedish: Rightwards pointing angular quotes (single still ’…’, double »…»)
Russian: Just the default (English) quotes

I tried to document the new option, but I'm not sure where to put the explanation about alternative quotes, so I'd like some feedback on that.
I'm still on the fence as to whether this is the best design in the long run (as opposed to a more general design that allows for more options per language), but we can always change it again later, and having it is better than not having the option at all, IMO.

(I've already fixed the issue, idk why the commit doesn't show up in the PR yet, we'll probably just have to wait a moment)

I've gone ahead and changed the wording of that as well (All the other docs use "quotes" instead of "quotation marks").
I'll look into rebasing onto main now

Great, just tell me when it's ready to be merged.

That would be now :grin: (rebasing was a trivial ff)

Thank you!


Extmodules workflow misidentifies name/version in certain filesystem hierarchies

The Extmodules workflow assumes that modulefiles are stored in a directory structure where, at the leaf directories, the filename is the version, and the directory in which the file resides is the package name.
This is true in many cases, and irrelevant in many other cases, as IPF will override name/version from the filesystem if it can discover Name and Version key/value pairs inside the files.  However, there are some files, at least on Expanse, such as the Bright Cluster Manager cuda Toolkit, where the file is at:
/cm/shared/modulefiles/cuda11.7/toolkit/11.7.1
so IPF identifies the "Name" as "toolkit"
This could be ameliorated if comments could be added to the module file, including a Name key/value pair.  But it would be good if IPF could figure this situation out itself.
One possibility would be a rework of how the Extmodules flow traverses subdirectories of the directories in the MODULEPATH.  This may require a better understanding of all the ways MODULEPATHs and module file hierarchies are set up in practice.
Another possibility is to try to integrate lmod's "spider" command into the workflow, though I don't think we would be able to rely solely on spider, as it doesn't understand various key/value pairs that XSEDE/ACCESS have defined as extensions for more specific, detailed information.   But it might be possible to get a list of modules from spider and match them up to their module files for extra info extraction.

Some additional details:

Lmod supports the use of both Lua and TCL modulefiles.
Expanse application software stack is built with Spack and generates Lua modules for Lmod.
Some vendor provided modulefiles on Expanse are TCL modulefiles (for example, Bright Cluster Manager CUDA related).

This means IPF needs to be able to collect information about modulefiles in way that will support either Lmod or environment-modules usage on the resource. It should also support both Lua and TCL modulefiles and not necessarily assume either is being used exclusively regardless of which module environment is being used.

I've developed and am currently testing what I think may be a solution to this problem.
Fundamentally, the extmodules workflow of IPF was failing to parse the filesystem hierarchy of TCL and/or Lua modules files in the same way that lmod does, leading to incorrect names/versions for modules.
The obvious way around this is to use lmod itself as the canonical source for name/version, instead of trying to infer from the filesystem hierarchy.  However, the various lmod commands (module avail, spider, etc) didn't return all the needed information, nor did they return what info they did in a particularly useful format.
Then I stumbled upon lmod's Spider System Cache.  One can use "update_lmod_system_cache_files" to create a lua cache file that contains all the info that spider knows about the modules on the system.
Thus, my path forward was to add a "lmod_cache_file" parameter to the ExtendedModApplicationsStep of IPF.  IPF uses the lupa python package to execute the lua code from the lmod_cache_file, then converts the spiderT table to a python dict.  It then goes through the (converted) spiderT dict looking for fileT or metaModuleT structures.
It then goes through each fileT and for each module name in the fileT, adds the "fn" field as a module file that IPF should look at (along with the module name, and "Version" field).  IPF then opens each module file, to check to see if there are keyword overrides for any fields, and publishes the module.
This all appears to work, and appears to solve the issue.  Some care will have to be taken to ensure that the lmod_cache_file is created with the desired MODULEPATH, and is recreated with sufficient periodicity.
If anyone has feedback, specifically with regard to how I am interpreting the spiderT table, (or if it is likely to be substantively different for different versions of lmod), it is appreciated.  So far, I have only run on Expanse, so there may well be variations I haven't encountered yet.

Current code that I am testing is in the lmod_cache branch of this repo

@ericblau Thanks for the update. I will share with our team.

I believe that this issue is fixed, as of commit c31125e, and release 1.7.1. As a note, the lmod_cache code is not part of 1.7.1, because the default behavior (without using lmod_cache) now addresses the issue for both modules and lmod.

Thanks for the update @ericblau. I'll let our team know.