New feature exclude apps from major/minor updates

Proposed Changes
We have some applications where we cannot update the applications without validating them first. But we would like to get patch updates (1.0.2 : 1=Major, 0=Minor, 2=Patch) for security reasons f.e.
This feature implements an option to exclude specific apps from Major updates or Minor Updates.
It has been tested with two applications (Meazure.GuardianBrowser & Mozilla.Firefox), one user app, one system app. Wildcards seem not to work unfortunatly (yet).
Test Log Output:
#################################################################
#  23.09.2024 - CHECK FOR APP UPDATES (System context - No ServiceUI)
#################################################################
15:39:32 - Notification Level: SuccessOnly. Notification Language: Deutsch
15:39:32 - Checking internet connection...
15:39:32 - Connected !
15:39:32 - Winget Version: v1.8.1911
15:39:32 - WAU current version: 1.21.8
15:39:32 - WAU AutoUpdate is Enabled.
15:39:32 - WAU is up to date.
15:39:32 - WAU uses Black List config
15:39:33 - -> Successsfully loaded local excluded apps list.
15:39:33 - -> Successsfully loaded local excluded major update apps list.
15:39:33 - -> Successsfully loaded local excluded patch update apps list.
15:39:33 - Checking application updates on Winget Repository...
-> Available update : Mozilla Firefox (x64 de). Current version : 129.0.2. Available version : 130.
-> Available update : Microsoft Edge. Current version : 127.0.2651.105. Available version : 129.0.2792.52.
15:39:34 - MajorUpdateToSkip: Google.AndroidStudio Meazure.GuardianBrowser Mozilla.Firefox - Detected current Major 129 and new Major:
15:39:34 - MinorUpdateToSkip: Google.AndroidStudio Meazure.GuardianBrowser - Detected current Minor 0 and new Minor:
15:39:34 - Mozilla Firefox (x64 de) : Skipped upgrade because it is a Major update (Current: 129.0.2, Available: 130) and excluded in Major Update list
15:39:34 - Microsoft Edge : Skipped upgrade because it is in the excluded app list
15:39:34 - No new update.
15:39:35 - User logged on, get a list of installed Winget apps in System context...
15:39:36 - Starting WAU in User context...
#################################################################
#  23.09.2024 - CHECK FOR APP UPDATES (User context)
#################################################################
15:39:39 - Notification Level: SuccessOnly. Notification Language: Deutsch
15:39:39 - Checking internet connection...
15:39:39 - Connected !
15:39:40 - Winget Version: v1.8.1911
15:39:40 - WAU current version: 1.21.8
15:39:40 - WAU uses Black List config
15:39:40 - -> Successsfully loaded local excluded apps list.
15:39:40 - -> Successsfully loaded local excluded major update apps list.
15:39:40 - -> Successsfully loaded local excluded patch update apps list.
15:39:40 - Checking application updates on Winget Repository...
-> Available update : Dev Home. Current version : 0.1800.625.0. Available version : 0.1801.625.0.
-> Available update : Guardian Browser 1.14.0. Current version : 1.14.0. Available version : 1.15.2.
15:39:41 - Dev Home : Skipped upgrade because it is in the excluded app list
15:39:41 - MajorUpdateToSkip: Google.AndroidStudio Meazure.GuardianBrowser Mozilla.Firefox - Detected current Major 1 and new Major: 1
15:39:41 - MinorUpdateToSkip: Google.AndroidStudio Meazure.GuardianBrowser - Detected current Minor 14 and new Minor: 15
15:39:41 - Guardian Browser 1.14.0 : Skipped upgrade because it is a Minor update (Current: 1.14.0, Available: 1.15.2) and excluded in Minor Update list
15:39:41 - No new update.
15:39:41 - End of process!

Related Issues

(Github link to related issues or pull requests)

GPOs are now also supported. this PR also adds german translations for the GPOs

Isn't it a bit too complex?
What happens if a new major or minor version is released? Will all the patch versions in between be skipped since they won't be returned as the latest version by Winget, and therefore never applied?
I think this feature should be part of Winget itself I guess

We have the following issue. Our developers use Android Studio and they need to develop on a specific minor version until finished.
Meaning the following scenaros if version 2024.1 is installed and the latest version is 2024.2.2 and version 2024.1.3 exists:

Just default WAU: will be updated to 2024.2.2
Using Exclusion from Major updates: will be updated to 2024.2.2
Using Exclusion from Minor updates: will be updated to 2024.1.3 (if no newer version has been available at that date)
Using Exclusion from Minor updates: will stay at the version

To answer your question. In that scenaro the software will stay on the "old" version.
I think this wont make sense directly at winget, as winget doesnt manage updates, this is what WAU is doing.

Winget does manage updates, but not automatically or in a seamless way. For example, you can manually run winget upgrade --all. However, if the editor updates an app with a Major or Minor tag, your app will no longer receive hotfixes in that case.
That's why I mentioned it as a winget issue. They should introduce a feature in the winget upgrade command that allows users to specify whether they want to upgrade only Major, Minor, or Patch versions (assuming the app follows this kind of versioning).

We use WAU to update apps with Winget upgrade "AppName". It would be great if they could implement an option like Winget upgrade "AppName" --PatchOnly for instance. This way, we could easily manage patch versions since, when the editor releases a new major or minor version, the patched version may no longer be available.

The input from the above MS request was good, I think this makes this PR obsolete, but we still need #746 to be merged for this to work. I will continue my work there

Oh good, I wasn't aware of that feature... That's a very good point, and we can try to address it soon. We'll probably need to review all the code that calls Winget. :/

Maybe you can use Mods to "pin" apps in system context. @KnifMelti, should be possible I guess?
This way, we can pin apps in a static major/minor, and WAU should update patches only. I'll try to test that.

@Romanitho please be sure to check my latest commits at #746 before, in theory it works now, WAU wil be able to handle pinned apps and udate them only if the pinned version is lower than the last available version.
we can pin apps like this:
winget pin add "Microsoft.Office" --version 16.0.*
winget pin list
Name                                      ID                    Version          Quelle Stecknadeltyp Angeheftete Version
-------------------------------------------------------------------------------------------------------------------------
Snagit 2020                               TechSmith.Snagit.2020 20.1.8           winget Pinning
Microsoft 365 Apps for Enterprise - de-de Microsoft.Office      16.0.17928.20216 winget Gating        16.0.*

this will result in WAU updating the app as long as the version is lower than 16.1 for example.
still would be great to somewhat use WAU to pin apps, for the beginning in our company I would have written a small script to pin apps.


Dashboard UX: reference value doesn't properly update when I change time ranges

The reference value does not seem to update correctly when I switch between a short date range and “all time”. This messes with the bars used in the leaderboards.

Most probably related to #630 . We need a better way to run the sync code that runs the queries for time series, big number and leaderboard.


inquirer issue python 3.10

Thanks for creating/packaging/idea!
Thought to test it on my Debian desktop (python 3.10) but got:
$ stapy -u http://localhost:8080/v1.0
/home/richard/venv/lib/python3.10/site-packages/thefuzz/fuzz.py:11: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning
  warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')
Traceback (most recent call last):
  File "/home/richard/venv/bin/stapy", line 5, in <module>
    from stapy.cli.main import main
  File "/home/richard/venv/lib/python3.10/site-packages/stapy/cli/main.py", line 3, in <module>
    from stapy.cli.parser import Parser
  File "/home/richard/venv/lib/python3.10/site-packages/stapy/cli/parser.py", line 10, in <module>
    from stapy.cli.cli import CLI
  File "/home/richard/venv/lib/python3.10/site-packages/stapy/cli/cli.py", line 1, in <module>
    from PyInquirer import prompt
  File "/home/richard/venv/lib/python3.10/site-packages/PyInquirer/__init__.py", line 6, in <module>
    from prompt_toolkit.token import Token
  File "/home/richard/venv/lib/python3.10/site-packages/prompt_toolkit/__init__.py", line 16, in <module>
    from .interface import CommandLineInterface
  File "/home/richard/venv/lib/python3.10/site-packages/prompt_toolkit/interface.py", line 19, in <module>
    from .application import Application, AbortAction
  File "/home/richard/venv/lib/python3.10/site-packages/prompt_toolkit/application.py", line 8, in <module>
    from .key_binding.bindings.basic import load_basic_bindings
  File "/home/richard/venv/lib/python3.10/site-packages/prompt_toolkit/key_binding/bindings/basic.py", line 9, in <module>
    from prompt_toolkit.renderer import HeightIsUnknownError
  File "/home/richard/venv/lib/python3.10/site-packages/prompt_toolkit/renderer.py", line 11, in <module>
    from prompt_toolkit.styles import Style
  File "/home/richard/venv/lib/python3.10/site-packages/prompt_toolkit/styles/__init__.py", line 8, in <module>
    from .from_dict import *
  File "/home/richard/venv/lib/python3.10/site-packages/prompt_toolkit/styles/from_dict.py", line 9, in <module>
    from collections import Mapping
ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/__init__.py)

Tried to update 'inquirer' to latest version... but that dit not help yet...

Mmm, the requirements.txt says to install 'inquirer', but 'PyInquirer' is imported (in cli.py).
Maybe a mixup of the two projects?
https://github.com/CITGuru/PyInquirer/tree/master/PyInquirer
and
https://github.com/magmax/python-inquirer/tree/master/src/inquirer ?
The last one seems to be most 'alive'?

Some time ago I moved this project from using PyInquirer to inquirer since the former got abandoned.
Unfortunately I can't confirm the wrong import you stated, in the current master branch as well as the last shipped version v0.3.0 there is the import line from inquirer import prompt in cli.py


Windows build not working on windows xp

Hello,
I downloaded the windows build from here:
http://rg3.github.com/youtube-dl/windows.html
It works perfectly on Windows7 but not on Windows XP.  I made sure Python 3.3 is installed on my xp machine.  When I launch (double click) the "youtube-dl" exe file I get the following error:
"This application has failed to start because the application configuration is incorrect".
When I try to use the command line "youtube-dl "AnyYouTubeUrl"" I get:
"The system cannot execute the specified program"
What am I missing?
Thanks,
Derek

I can confirm that I just received the same error message on windows xp (with the last set of updates available). Installing python 2.7 fixed it.


Update ignoring-files.md

Further Reading

.gitignore in the man pages for Git

man > main
spelling error?

Why:

Closes [issue link]
What's being changed:

Check off the following:

[ ] I have reviewed my changes in staging. (look for the deploy-to-heroku link in your pull request, then click View deployment)
[ ] For content changes, I have reviewed the localization checklist
[ ] For content changes, I have reviewed the Content style guide for GitHub Docs.

Hi there,
I don't think this is a spelling error. A man page for a terminal tool is a "manual" for the tool.

Thanks so much for opening a PR @colinthedev! @ajs256 is correct, this isn't a typo. 'main' in this case is referring to the primary pages. I'm going to close this now 💛
Take a look at the contributing.md to learn how to contribute in this repo. You can open a discussion if you have any questions or check out issues with the help wanted label. :sparkles:


Fix EclipseModelBuilder compatibility with Gradle 5.5 ToolingApi clients

EclipseModelBuilder needs to handle Gradle 5.5 ToolingAPI clients. These use EclipseWorkspaceProject instances that do not have the new isOpen() method.
This change also adds the safety against duplicate projects names that is already present in EclipseModelBuilder to RunBuildDependenciesTaskBuilder.
When running an gradle 5.5 tooling model against a 5.6 daemon this exception is thrown:
Unsupported method: EclipseWorkspaceProject.isOpen().
The version of Gradle you connect to does not support that method.
To resolve the problem you can change/upgrade the target version of Gradle you connect to.
Alternatively, you can ignore this exception and read other information from the model.
* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Exception is:
org.gradle.tooling.model.UnsupportedMethodException: Unsupported method: EclipseWorkspaceProject.isOpen().
The version of Gradle you connect to does not support that method.
To resolve the problem you can change/upgrade the target version of Gradle you connect to.
Alternatively, you can ignore this exception and read other information from the model.
    at org.gradle.tooling.model.internal.Exceptions.unsupportedMethod(Exceptions.java:33)
    at org.gradle.tooling.internal.adapter.ProtocolToModelAdapter$InvocationHandlerImpl.invoke(ProtocolToModelAdapter.java:362)
    at com.sun.proxy.$Proxy52.isOpen(Unknown Source)
    at org.gradle.plugins.ide.internal.tooling.EclipseModelBuilder.buildAll(EclipseModelBuilder.java:124)
    at org.gradle.plugins.ide.internal.tooling.EclipseModelBuilder.buildAll(EclipseModelBuilder.java:84)
    at org.gradle.tooling.provider.model.internal.DefaultToolingModelBuilderRegistry$ParameterizedBuildOperationWrappingToolingModelBuilder$1$1.create(DefaultToolingModelBuilderRegistry.java:140)

The Gradle error message in this case is actually misleading/wrong - Gradle does support that particular method, its the ToolingApi that is missing it.
Context

This PR fixes an incompatibility introduced by https://github.com/gradle/gradle/pull/9405
Contributor Checklist

[x] Review Contribution Guidelines
[x] Make sure that all commits are signed off to indicate that you agree to the terms of Developer Certificate of Origin.
[x] Check "Allow edit from maintainers" option in pull request so that additional changes can be pushed by Gradle team
[x] Provide integration tests (under <subproject>/src/integTest) to verify changes from a user perspective
[ ] Provide unit tests (under <subproject>/src/test) to verify logic
[x] Update User Guide, DSL Reference, and Javadoc for public-facing changes
[ ] Ensure that tests pass locally: ./gradlew <changed-subproject>:check

Gradle Core Team Checklist

[x] Verify design and implementation
[x] Verify test coverage and CI build status
[x] Verify documentation
[x] Recognize contributor in release notes

Thank you for the fix!


Fixup executable path when running tests

The changes using cmake generator expressions in the MPI tests are required when using the Ninja Multi-Config generator.

Note to self.
CMake add_test
The COMMAND argument works likes this:
    Specify the test command-line. If <command> specifies an executable target (created by [add_executable()](https://cmake.org/cmake/help/latest/command/add_executable.html#command:add_executable)) it will automatically be replaced by the location of the executable created at build time.

It seems to expand the path properly when it does COMMAND <TargetName> <args>... that's why I did not use a generator expression in that case

It seems to expand the path properly when it does COMMAND <TargetName> <args>... that's why I did not use a generator expression in that case

Yes, I double checked that too. As long as the proper target name is the first argument, it's fine.

Two builds ran out of time (HIP, SYCL), ignoring.


High latency for Chat responses, stuck in "Thinking…"

Copilot Cat had a performance regression. We are working on an issue that fixes long delays for responses being stuck in "Thinking …".

Same problem here but after switching internet connections the issue solved for me, probably a network issue

Fix was released yesterday. Please file a new issue if you see latency issues.

I'm still having this issue, though not idea if it's related to a network issue. It certainly gets stuck at Thinking… and it never comes up with a response


Fix issue 20425: Remove Proxy.opCmp constraints

They fail with types that have an opCmp defined and are value
types.

Issue number?

There’s isn’t one I could find. Just went off this:
<EMAIL_ADDRESS>On Mon, 2 Dec 2019 at 00:15, Nicholas Wilson<EMAIL_ADDRESS>wrote:

Issue number?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dlang/phobos/pull/7298?email_source=notifications&email_token=AAUEDTFK6ZT45CMZX63ZPS3QWRASTA5CNFSM4JTMGDYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFRY4JI#issuecomment-560172581,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAUEDTDYVJK7P3E6N3NM3G3QWRASTANCNFSM4JTMGDYA
.

Please either make one or add a changelog entry so this is recorded.
Thanks!

@thewilsonator made and added to title, is there a way to trigger the dlang bot to pick it up?

You need it to be part of the commit message. The title isn't picked up at all.


Added missing unit tests in config module

Description
Added missing unit tests in config module
Type of change

[ ] Refactor
[ ] New feature
[ ] Bug fix
[ ] CVE fix
[ ] Optimization
[ ] Documentation Update
[ ] Configuration Update
[ ] Bump-up dependent library
[x] Unit tests

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (9bf0be5) 94.59% compared to head (89f1940) 95.14%.
Report is 20 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #454      +/-   ##
==========================================
+ Coverage   94.59%   95.14%   +0.54%
==========================================
  Files          32       34       +2
  Lines        1092     1092
==========================================
+ Hits         1033     1039       +6
+ Misses         59       53       -6

see 1 file with indirect coverage changes

/lgtm

/approve


Close button isn't easily discoverable on hover

The close button is only visible on hover, which isn't easily discoverable for users. They may well not know the functionality exists if they never hover over the component.
Further to this, on touch devices, there is no obvious way to trigger the functionality - I thought it was a bug when trying on an iPhone - as there's no icon to hit.

I can resolve that issue if it wasnt assigned yet, I will add that as custom option for toast 😉

Nice! Let's make it optional and disabled by default @andrewbarnesweb.

Nice! Let's make it optional and disabled by default @andrewbarnesweb.

Not sure I follow 🙂. The close button is already optional? Just when it's passed in you can't see it, unless you hover - so I raised a PR for having the close button visible if passed.
Let me know what your expectation is here and I'll update the PR 👍🏻

@andrewbarnesweb is it possible to change the position of the closeButton?
I would love to justify it to the right after the text, it should be discoverable without the hover effect.

@emilkowalski is there any way to change the position of the close button without having to make a full blown custom toast?

The close button is discoverable without the hover effect now. To change its position you can target data-close-button="true" element in tailwind or css and change the top/bottom/right/left values.


Not working as expected with the latest mdl lib

Hi,
This template is not working as expected with the latest mdl lib. Menus disappear.
Could you update? Or even better could you update and also make it look like the newer android.com website?

Can you give us a few more information about your Browser and OS version? Menus are working fine for me on the Android template.

Hi Surma
My apologies, it was a local caching issue, works as intended - even with the latest versions. Kindly close this issue. Thanks for the prompt response.


Android Simulator opening properly but getting stuck on runnnig of my main dart file!!

Steps to reproduce
help me to solve this problem as soon as possible!!!

Actual results
I specified the right path of simulator in Environment variables of Advanced System Properties and also in VSCODE simulator path everything was fine upto now . When I am opening the simuator it was opening but when I tried to run my main dart file it was opening flutter app in that android and getting stuck there .. no output was showing up . I am suffering from this problem for the last two weeks . I don't understand two weeks back everything was good .
Logs

Logs

<!-- Paste your logs here -->

Flutter Doctor output
I am not getting any output on running of this command.
Intial I am able to see some output but now  , there is no output .

Hi @GantaVenkataKousik
Looking at the screenshot above which shows your project code, it's hard to say which could be the cause of the issue.
The goal of this Github repo is not to assist in personal code or the whole package/plugin project but to improve Flutter framework reporting bugs or feature requests. If you encounter behavior that you believe could be a Flutter framework bug, please provide a completed and minimal reproducible code sample that doesn’t include 3rd party plugins or complex production code.
You should narrow down the issue by removing the usage of 3rd party plugins/packages one by one, unrelated/unaffected pages, and isolating the affected page into a minimal sample code that can produce the issue. And please share it here when you have such an example.

I am not getting any output on running of this command.

Does it hang or produce any error when running flutter run -v command?

--->>That's the problem brother , I am not sharing my code I am trying to explain on execution it was running the main file , the flutter app is running in android studio and nothing showing up as shown in the snapshot .
--->>Intially when I installed it was good , everything was great, but now I can't understand where was this problem is araising .  I wan't a solution to this!!!!

I am not sharing my code

Please take a look at my comment above, it's not recommended sharing production/your project code because it's hard to debug the issue. It would be ideal if you could dig down the issue (as I shared above) to get a minimal sample of code causing this.
Otherwise, please share the output of flutter run -v which may contain information (crash stack trace for e.g) to further investigate the issue.


Add -j $J_CORES after all "make"

Add -j $J_CORES after all "make"

Please wait for full test before merging it.
Thanks.

That's now full tested.
You can merge it.
Thanks.
Good night !

@RespawnDespair Could you please revert this PR.
I was silly to make it, as it was not needed, ... and it was the reason why OSD was not working anymore (not compiled) !!
I'm testing a new PR which should fix some other issues.


APIGateway - Generate API key value when no value provided

On AWS when creating an API key if a value is not provided then AWS generates an value for you.
Example:
aws apigateway create-api-key --name "Test"
Returns:
{
    "name": "Test",
    "enabled": true,
    "value": "dsasDDsn7qweQWE2r327lq44nLRNq0AsdAd",
    "stageKeys": [],
    "lastUpdatedDate":<PHONE_NUMBER>,
    "createdDate":<PHONE_NUMBER>,
    "id": "twvchui76ac"
}

On moto the value was only generated when the --generate-distinct-id flag was passed, which is a bug.
This PR fixed this.

Coverage decreased (-0.1%) to 92.139% when pulling 95e4c96c38c6a9384458d2fa35edc1c1d2ec46b6 on fewstera:api-key-value-fix into bb6da93891482ddcb87c23e6abddc0e451f72d30 on spulec:master.

@terrycain thank you for the review. I've re-added generateDistinctId  and all checks are passing now.


icon text display issue

I used date_picker in my project, but those icons don't display correctly. Is the font file not configuring in Cargo.toml via git?

Are you importing your own icon font? I recently had the same issue. My problem was that I have imported an other font under the same name as the icon font of iced_aw. The icons of iced_aw are named "Icons". If you import an other font under the same name they infer with each other.
pub const ICON_FONT: Font = Font::External {
    name: "Icons", // Use a different name here
    bytes: include_bytes!("../font/icons.ttf"),
};

Thank you very much, it solved my problem!

I've reopened this issue as a reminder to rename the icon font of iced_aw to something more unique in the future. A more unique name would save a headache for the next person struggling with this issue.

Hi. I've got the same issue. I'm using tabbar (which uses icons) and date_picker. I'm not importing my own icons, but the date_picker has the buttons without icons as above. If I wanted to use the solution above, where do I make changes?

If you are not importing your own icons and the icons are visible on the tabbar but not on the date_picker, this seems like a different kind of bug as there is no second icon font messing around with the one of iced_aw itself :thinking:

Thanks for leaving that open - had the same issue  :)

this is no longer relevant on how iced uses fonts.


Docu update 240623

Small docup updates for upcoming code review

Welcome to Codecov :tada:
Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.
:information_source: You can also turn on project coverage checks and project coverage reporting on Pull Request comment
Thanks for integrating Codecov - We've got you covered :open_umbrella:


refactor/artifacts

refactor(artifact-file-type): remove backward compatibility code, as everything is a zip or npm
refactor(artifact-migrator): remove need for artifact migrator, as now everything has migrated to v2
refactor(artifact-installation): refactor Artifact Record keeping into something owned by a Org

Codecov Report

Merging #865 (f3a6bb2) into develop (2c14c54) will increase coverage by 1.33%.
The diff coverage is 58.24%.

:exclamation: Current head f3a6bb2 differs from pull request most recent head 190c0d4. Consider uploading reports for the commit 190c0d4 to get more accurate results

@@             Coverage Diff             @@
##           develop     #865      +/-   ##
===========================================
+ Coverage    65.41%   66.75%   +1.33%
===========================================
  Files           47       46       -1
  Lines         1573     1522      -51
  Branches       342      327      -15
===========================================
- Hits          1029     1016      -13
+ Misses         544      505      -39
- Partials         0        1       +1

Impacted Files
Coverage Δ

...ages/core/src/artifacts/ArtifactFilePathFetcher.ts
34.02% <0.00%> (+3.46%)
:arrow_up:

...org/packageQuery/InstalledPackagesQueryExecutor.ts
40.00% <ø> (ø)

packages/core/src/org/SFPOrg.ts
54.66% <54.66%> (ø)

packages/core/src/utils/ObjectCRUDHelper.ts
85.71% <85.71%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9169127...190c0d4. Read the comment docs.


Scroll document to element?

Is it possible to have something like the following:
const offset = element.getBoundingClientRect().top;
anime({
	target: document,
	scrollTop: offset,
	duration: 1000
});

The idea being we can scroll to the element using some kind of easing function.

Yes, but the document itself do not have the scrollTop property. You may want to animate document.body and document.documentElement.

Here is an other solution including scrolling position.
const scrollTo = (element) => {
  const elementSelector = document.querySelector(element)
  document.querySelector('.btn-scroll').addEventListener('click', () => {
    const elementOffset = elementSelector.getBoundingClientRect().top
    const scrollPosition = window.scrollY
    const documentTop = document.documentElement.clientTop

    const scrollOffset = elementOffset + scrollPosition - documentTop

    anime({
      targets: [document.documentElement, document.body],
      scrollTop: scrollOffset,
      duration: 800,
      easing: 'easeInOutQuad'
    })
  })
}

@jimblue This was helpful! Thanks.


Change handling and registration of asset routes

Currently, the routes to serve assets are tied to the framework. This is not a good place because it doesn't allow any changes. The task is to find a better place and handling for assets and the related routes

Finished with https://github.com/superchargejs/framework/commit/206bd8347d4baf2c2c11495070eeb5bf2b1e5a43


Add sh script tool to consume historical offsets

The parent issue is:https://github.com/apache/incubator-inlong/issues/2203
Through the sh script method, running the script can consume the historical offset of the specified topic and output it to the screen in text form

I don't know who is interested, you can participate in the development of this piece according to the issue-related implementation of this parent class, thanks very much


PoS Decouple Validator Jailing from Validator Snapshotting

Current dependencies on/for this PR:

feature/proof-of-stake

PR #579

PR #581   👈

This comment was auto-generated by Graphite.


Gulp serve after  file changes not getting reflected

Category

[x] Bug

Describe the bug
Not able to make file changes and test it in workbench
Steps to reproduce

Edit SCSS file of a react component
gulp serve as usual in sharepoint environment , using ?loadSPFX=true&debugManifestsFile=https://localhost:4321/temp/manifests.js  appended to the page in SharePoint. (workbench.aspx works fine if we load up the webpart in there)

Also tried this ? debug=true&noredir=true&debugManifestsFile=https://localhost:4321/temp/manifests.js as suggested when we do a serve.
Expected behavior
Can see the changes in styles right away in the SharePoint page
Developer environment

OS: Windows 10
Framework: Node 10.15.3
Browser(s): Chrome, Edge Chromium
Tooling: VS Code, SPFx v1.10.0 (REACT component)

Just saw that the changes have no impact since today, was working fine before
Any help would be appreciated.

SCSS is unrelated to this. The issue is that when you debug with the query string you will be asked the first time:

Then you accept and the first load everything is loading normally

.
After you change something in your files the compilation kicks in as is should.

On the second load the popup do not show up anymore and only the manifest is loaded but not anything else.
The workaround is to open every time a new browser tab. This issue happens in Firefox too

Yes @StfBauer noticed this in any changes as well and have updated the issue title. Previously the changes were reloaded in the same tab.

@rabwill There is an intermediate dirty fix to this.
When you add for development the following line of code:
sessionStorage.clear();

Then everything is back to normal and the web part loads as it always has been.

This is happening also on SPFx 1.9.1, the clearing the sessionStorage works, thx @StfBauer.
It seems that something broke in SPO and not in SPFx.

This is working now today . I am getting the prompts and changes reflected
Browser : MS edge (chromium)

Dumping sessionStorage is not working for me. Really impacts rapid development when you cannot see changes reflected on the workbench. Seems like this is a persistent (pun intended) issue.

Yes its not working even with sessionStorage.


[Access Request] report 7391 "Whse. Get Bin Content" - Change Access Scope to protected

Could you please change the scope of the following procedure to protected?

//+Change
//-local procedure GetLocation(LocationCode: Code[10]; var Location: Record Location)
protected procedure GetLocation(LocationCode: Code[10]; var Location: Record Location)
//-Change
begin
   ...
end;

//+Change
//-local procedure InsertTempTrackingSpecification(WarehouseEntry: Record "Warehouse Entry"; QtyOnBin: Decimal; var TempTrackingSpecification: Record "Tracking Specification" temporary)
protected procedure InsertTempTrackingSpecification(WarehouseEntry: Record "Warehouse Entry"; QtyOnBin: Decimal; var TempTrackingSpecification: Record "Tracking Specification" temporary)
//-Change
begin
    ...
end;

//+Change
//-local procedure CalcQtyUOM(QtyBase: Decimal; QtyPerUOM: Decimal): Decimal
protected procedure CalcQtyUOM(QtyBase: Decimal; QtyPerUOM: Decimal): Decimal
//-Change
begin
  ...
end;

//+Change
//-local procedure GetQtyToEmptyBase(ItemTrackingSetup: Record "Item Tracking Setup"): Decimal
protected procedure GetQtyToEmptyBase(ItemTrackingSetup: Record "Item Tracking Setup"): Decimal
//-Change
var
    BinContent: Record "Bin Content";
begin
   ...
end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 45938.


[nightly] Trigger Integ test before pushing to dockerhub

Description
This PR includes the following changes

push image to ECR in each nightly build
call integration workflow before pushing the newly-built images to djl docker hub
add skip_nightly_integ_test arg so we can manually start a nightly build without running integ test
remove cron schedule in integration workflow
add ECR login in integration workflow

If this change is a backward incompatible change, why must this change be made?
Interesting edge cases to note here

Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] New feature (non-breaking change which adds functionality)
[ ] This change requires a documentation update

Feature/Issue validation/testing
I have created one run here
https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#pull_request_review_comment
The integration test is not passing right now. But it is not related to this change

Pushing and Pulling the built container in itself takes >10 mins. I would like to avoid this if possible.
I added a persistent FSX cache volume to all the self hosted runners. Can you try to use the cache to persist data between runners instead of using ECR ?  I assume we would run into the same issue with PR sanity tests.

Pushing and Pulling the built container to transfer it between hosts in itself takes >10 mins. I would like to avoid this if possible.
I added a persistent FSX cache volume to all the self hosted runners. Can you try to use the cache to persist data between runners instead of using ECR ?  I assume we would run into the same issue with PR sanity tests.

I have done some experiment today. And I found it is not possible to use Lustre right now for docker volume storage. By checking https://docs.docker.com/engine/storage/drivers/select-storage-driver/#supported-backing-filesystems, Lustre as a file system is not supported. The only way that we can start the dockerd service with the Lustre folder is to use vfs.
according to doc,
The vfs storage driver is intended for testing purposes, and for situations where no copy-on-write filesystem can be used. Performance of this storage driver is poor, and is not generally recommended for production use.
And it throws error during the docker pull when we use vfs.

Pushing and Pulling the built container to transfer it between hosts in itself takes >10 mins. I would like to avoid this if possible.
I added a persistent FSX cache volume to all the self hosted runners. Can you try to use the cache to persist data between runners instead of using ECR ?  I assume we would run into the same issue with PR sanity tests.

I have done some experiment today. And I found it is not possible to use Lustre right now for docker volume storage. By checking https://docs.docker.com/engine/storage/drivers/select-storage-driver/#supported-backing-filesystems, Lustre as a file system is not supported. The only way that we can start the dockerd service with the Lustre folder is to use vfs.
according to doc, The vfs storage driver is intended for testing purposes, and for situations where no copy-on-write filesystem can be used. Performance of this storage driver is poor, and is not generally recommended for production use.
And it throws error during the docker pull when we use vfs.

We don't have to start docker on the FSX volume. We just have to save & load from the FSX volume. Can you try - docker save followed by docker load.
Ref:
https://docs.docker.com/reference/cli/docker/image/save/
https://docs.docker.com/reference/cli/docker/image/load/
You don't even need to compress images. Instead of saving & loading to ECR, you will be using a FSX volume.


Docker image -alpine size unneccssary increase since 1.1.2

Since 1.1.2-alpine, the step 11 will double the size of /bin which is unnecessary. Need to avoid it.

https://stackoverflow.com/a/49993241/541462
This may help


fixed css of footer in signup page

Related Issue
[Cite any related issue(s) this pull request addresses. If none, simply state "None”]
closes #1228
Description
[Please include a brief description of the changes or features added]
fixed

Type of PR

[ ] Bug fix
[X] Feature enhancement
[ ] Documentation update
[ ] Other (specify): _______________

Screenshots / videos (if applicable)
[Attach any relevant screenshots or videos demonstrating the changes]
Checklist:

[ ] I have performed a self-review of my code
[ ] I have read and followed the Contribution Guidelines.
[ ] I have tested the changes thoroughly before submitting this pull request.
[ ] I have provided relevant issue numbers, screenshots, and videos after making the changes.
[ ] I have commented my code, particularly in hard-to-understand areas.

Additional context:
[Include any additional information or context that might be helpful for reviewers.]

@Suchitra-Sahoo please merge PR

Your screenshot is not visible

@Suchitra-Sahoo  check now done

check now @Suchitra-Sahoo


CA for Partnership

Talking to Geri about partnering. bringing AShley back in.

Sent Geri an email in Denver.

CA/Rally is in for $1000. Needs to go to a non-profit so funding through CfA.


[App]: Internal wandb error: file data was not synced

Current Behavior
Model training stops at an arbitrary point when the process is synced to the online app. Processes run fine when done 'offline'. The error has occurred at various times, eg ~1750 epochs, ~2000 epochs, ~4000 epochs. The error message is clear but the cause is not; why would a local log file suddenly become unavailable?
<Normal logging...>
epoch: 1500, loss:  2.184e+06, time:   934.3, log_vars: 14.7473, -0.3149
epoch: 1750, loss:  9.470e+05, time:  1091.9, log_vars: 14.4531, -0.3113
Thread SenderThread:
Traceback (most recent call last):
  File "C:\Users\xxxx\AppData\Local\Programs\Python\Python39\lib\site-packages\wandb\sdk\internal\internal_util.py", line 54, in run
    self._run()
  File "C:\Users\xxxx\AppData\Local\Programs\Python\Python39\lib\site-packages\wandb\sdk\internal\internal_util.py", line 105, in _run
    self._process(record)
  File "C:\Users\xxxx\AppData\Local\Programs\Python\Python39\lib\site-packages\wandb\sdk\internal\internal.py", line 312, in _process
    self._sm.send(record)
  File "C:\Users\xxxx\AppData\Local\Programs\Python\Python39\lib\site-packages\wandb\sdk\internal\sender.py", line 237, in send
    send_handler(record)
  File "C:\Users\xxxx\AppData\Local\Programs\Python\Python39\lib\site-packages\wandb\sdk\internal\sender.py", line 830, in send_summary
    self._update_summary()
  File "C:\Users\xxxx\AppData\Local\Programs\Python\Python39\lib\site-packages\wandb\sdk\internal\sender.py", line 842, in _update_summary
    with open(summary_path, "w") as f:PermissionError: [Errno 13] Permission denied: 'C:\\Users\\Iblah\\blah\\blah\\wandb\\run-20220324_140524-2zt53myz\\files\\wandb-summary.json'
wandb: ERROR Internal wandb error: file data was not synced

And then tracebacks from a particular log command:
run.log({"train/loss":loss.item()**.5})
leading to
Exception: The wandb backend process has shutdown
Expected Behavior
Expectation is for model training to continue through the prescribed number of epochs, with pretty standard logging statements. Instead it breaks at arbitrary times.
Steps To Reproduce
No response
Screenshots
No response
Environment
OS: Windows
Browsers: MS Edge
Version: MS Edge Version 99.0.1150.46 (Official build) (64-bit)
Additional Context
Earlier I'd run up to 10k epochs with pretty much the same code. I changed the loss function and the models a bit, but not sure why it would affect logging, in particular why anything I did in the model building code would affect access to a log file on my local file system.

Hi @heintzcode,
I'm sorry you're getting this issue. Is the directory that your wandb folder is in a network synced directory? Such as One Drive or Dropbox?
It seems that Python lost permissions to this folder for some reason which is usually related to network synced folders. If this isn't the case is there any other changes that have been made that could have cause this interruption in permissions?
Thank you,
Nate

Hi, yes, it is a synced folder. Are you suggesting that I move the log directory to a folder that is not network synced?

@heintzcode that would be my recommendation but if running scripts from your network directory is necessary you should be able to specify wandb.init(dir=<path/to/non-synced/directory>) and W&B will create the run folder outside of the network drive. This will allow you to run from the network synced drive but your runs metadata will be saved elsewhere.
Another option would be to adjust the permission settings of your network directory to allow Python write access to your network directory.

I did the first and that seems to have addressed the problem. Thanks for your response.


NameError: name 'ChromeDriverManagerException' is not defined

I am trying to do as specified in docs.
This is the only code I have
First did this:
pip install ChatGPTAutomation

then in python:
from chatgpt_automation.chatgpt_automation import ChatGPTAutomation
chat_bot = ChatGPTAutomation()
chat_bot.send_prompt_to_chatgpt("Hello, ChatGPT!")

My python version is 3.8.0

Sorry for the long delay in replying, it seems like there was a problem with libary and I didn't know it was fixed.


Add option to continuously transform visuals in point cloud plugin

Hello,
I made a new pull request, which is essentially the same as #1652, which was merged but then reverted as it broke the visualization of other users. But now the changes are less invasive and do not affect old visualizations as long the option is unchecked (the default). The new option triggers the same behavior as the activated frame_locked flag in the Marker Message.

Dear @AndreasR30,
in principle, I support the idea to enable your new feature optionally with a corresponding checkbox. However, the feature should be very well motivated and described. I would like @Martin-Oehler to chime in. Please discuss together, whether this new feature is really necessary. As far as I interpret https://github.com/ros-visualization/rviz/pull/1652#issuecomment-920823254, he wasn't convinced yet.
I won't have cycles to dive into this topic any time soon. So, please discuss between the two of you and come up with a consensus.

Hello everyone,
I still don't think, that this feature is necessary. The problem described by @AndreasR30 occurs, because they publish data in the map frame but visualize it in the base link. Therefore, they select a fixed frame for their visualization which is not fixed in regards to the data. You could simply fix your visualization by selecting "map" as your fixed frame and lock the camera to your base link.
The frame_locked parameter in the marker message is valid, because markers don't necessarily visualize data and a moving marker might be useful. The same is not the case for data, where a wrong transform invalidates its meaning.
Best regards,
Martin

Hello @Martin-Oehler ,
I understand your first point. However, we (and I think also many others) often want to visualize data at the same time, which is not all in the same frame. For example, I also added an analog pull request for Odometry Display visualizing the Odometry message.
Here we have one Odometry message (red), which represents wheel odometry (dead reckoning) and lives in the odom frame. It describes the pose of base_link in odom frame. At the same time, we have another Odometry message (blue), which represents our ground truth from a inertial navigation system (IMU + GPS) and lives in utm frame. It describes the pose of base_link in utm (or map) frame:

With your description it would be possible to visualize one, but not both at the same time in Rviz. However, this is quite useful if you want to compare how well dead reckoning is working w.r.t. ground truth.
Even in the case where your description works because you only want to visualize messages in the same frame, I don't find it optimal when the user has to think about, which fixed frame he has to use to make the visualization work. In my opinion the visualization should work always irrespective of the fact what is the fixed frame. So, for me, the fixed frame option should just change where the camera is fixed to but not whether a visualization works or not.
I don't understand your second point:

The frame_locked parameter in the marker message is valid, because markers don't necessarily visualize data and a moving marker might be useful. The same is not the case for data, where a wrong transform invalidates its meaning.

Can you make an example scenario where you want the marker to move but not the data? Why do you think the transform is "wrong"? I think the transforms used in this pull request are correct.
I admit that this pull request for PointCloud2 is less important, than the one for Odometry display. Because in our scenario it just makes the visualization smoother, when the TF w.r.t. to fixed frame is updated more frequently than the message rate. But I still think it is important for many users. Maybe we can at least agree that it is very useful for Odometry message?
Kind regards,
Andreas

You could do the same in the UTM frame, as you can see from the screenshot here, that I took from your bag.

Not sure what you were doing but in my Rviz the visualization is wrong. It does not show the drift of the odom frame over time. . I also used utm in global options and  in Views:

However, rviz does not handle these high numbers very well. You could zero your utm frame at the starting position and publish a separate NavSatFix message. See also rviz_satellite.

My pull request handles these high numbers automatically. Why not simply use it without having to introduce new intermediate frames to avoid large TFs. In this scenario I want to compare Odometry Messages against each other and not aerial image against Odometry Message.

Well, with your change, only data that is already transformed to the correct fixed frame will be displayed correctly. All other data is transformed wrong, which is much worse, I think.

No that is wrong. You can visualize the point cloud when the fixed frame != message frame. Message in odom frame and fixed frame is base_link and it still works perfectly:
https://user-images.githubusercontent.com/4062443/146984992-942e5e79-3b6b-4761-b5a4-4c9e5bd6ca6d.mp4
It does not matter which fixed frame you choose. Unlike in the old implementation:
https://user-images.githubusercontent.com/4062443/146985147-0f8ae112-aff1-4ed5-b14f-f07abb558ace.mp4
Furthermore, I want to stress that this is just an option now, which can be activated if one wants to.

It is correct for the special case where your data is already transformed to its fixed frame. In all other cases, old data will be transformed with new transforms, leading to wrong results. I can add the equations, if you are interested.

I would prefer to have a minimal rosbag available of your Gazebo Simulation, where you show problems with the new implementation. I want to understand where the problem comes from.

No idea honestly. I don't think it makes much sense in either case.

I can tell you where this feature is useful. For example we recorded UTM coordinates while where were driving on our track. Now we want to visualize this static track by means of markers. By using the frame_locked feature it is enough to publish the marker once because its pose w.r.t. fixed frame is continuously re-computed. Here is the code:
#!/usr/bin/env python
import rospkg
import rospy
import scipy.io
import time
from geometry_msgs.msg import Point, Vector3
from std_msgs.msg import ColorRGBA
from visualization_msgs.msg import Marker

rospy.init_node('visualize_test')

rospack = rospkg.RosPack()

path = rospack.get_path('mpc_path_follower') + '/paths/cpg_fast_lap_BagT.mat'
mat = scipy.io.loadmat(path)

x = mat['x'][0]
y = mat['y'][0]

marker = Marker()
marker.header.stamp = rospy.Time.now()
marker.header.frame_id = "utm"
marker.ns = "path_points"
marker.id = 1
marker.type = Marker.SPHERE_LIST
marker.action = Marker.ADD
marker.pose.position = Point(x[0], y[0], 0)
marker.pose.orientation.w = 1
marker.scale = Vector3(.2, .2, .2)
marker.color = ColorRGBA(1, 0, 0, 1)
marker.lifetime = rospy.Duration(0)
marker.frame_locked = True  # important: tells Rviz plugin to continuously transform into 'fixed_frame' -> no need to publish message periodically

for i in range(x.shape[0]):
    marker.points.append(Point(x[i] - marker.pose.position.x, y[i] - marker.pose.position.y, 550))

pub = rospy.Publisher('path_points', Marker, queue_size=10)
time.sleep(1)

pub.publish(marker)

rospy.spin()

https://user-images.githubusercontent.com/4062443/146992197-4389ed72-86c4-4476-ac81-e5c446cd66bb.mp4
We think such a feature would be essential for nearly any visual not only Markers.
Finally, I want to show you that there are multiple Displays, which constantly re-transform into fixed frame in the update step with the latest stamp available:

(As already mentioned) Markers with frame_locked flag.
Robot model display, see here and here.
Axes display: see here
Effort display: see here
Grid display: see here
Map display: While searching I found an option in Map Display "Use Timestamp", which does exactly the same as I do for other visuals, see here.
And much more...

Would you also say that their transforms are wrong/bad?

The displays, you mention @AndreasR30, don't display data recorded at a specific timestamp, but they visualize data w.r.t. to the current time. An exception might be the Effort display, which displays effort data at some timestamp but using the current/latest joint pose.

@rhaschke and @Martin-Oehler Please give me a rosbag (pointcloud topic, /tf, /tf_static) or a synthetic example, where the new visualization option fails. I am quite sure that there is an issue with the messages' frame_id or stamp. Or with your TF tree.
It's a pity that you simply ignore the issues you can definitely see in the synthetic example above. The point cloud there is in map frame and should clearly stay at the same position w.r.t. the map frame. This means the parent frame has to move in the screen because the camera is moving w.r.t. to map frame. The visualization is only correct at the moments, when there is a new message but in between it is wrong.

@rhaschke Also the Map display shows data from a specific frame and stamp.  E.g. you can generate a OccupancyGrid message from a single laser scan. With the option the grid's visualization remains valid even when some time elapsed since the last published. In general OccupancyGrid and a PointCloud2 are a quite similar thing, when you publish a PointCloud2 in a world fixed frame. Why then not make their visualization similar?

Not sure what you were doing but in my Rviz the visualization is wrong. It does not show the drift of the odom frame over time. . I also used utm in global options and in Views:

From your screenshot, I am not sure what you are doing differently. But since both odom paths will end at the current robot position by definition and they are not the same path, you are bound to see the drift.

My pull request handles these high numbers automatically. Why not simply use it without having to introduce new intermediate frames to avoid large TFs. In this scenario I want to compare Odometry Messages against each other and not aerial image against Odometry Message.

That's a nice addition then. You should probably split these changes into a separate PR. However, it won't fix the rviz camera view, which for me did not work properly anymore.

No that is wrong. You can visualize the point cloud when the fixed frame != message frame. Message in odom frame and fixed frame is base_link and it still works perfectly:

The odom frame is still a fixed frame with regards to the data. Your changes break visualization, when the data is in the base link, e.g. data from a lidar or depth camera.

I would prefer to have a minimal rosbag available of your Gazebo Simulation, where you show problems with the new implementation. I want to understand where the problem comes from.

Okay, here you go: https://drive.google.com/drive/folders/17E2ejoBJm8qcFWe0rZeAEmz0XQKXY8ql?usp=sharing
The bag contains a lidar scan of a rotating VLP-16, published in the laser frame. An rviz config is provided.
This video shows the visualization with the correct behavior previous to your path:
https://user-images.githubusercontent.com/7110154/147231100-af8a31cd-703d-4753-af39-c8fb38912645.mp4
This video shows the broken behavior with the "Continuous Transform" enabled:
https://user-images.githubusercontent.com/7110154/147231286-bd23a7ba-94ad-4292-a159-b760154e2bc8.mp4

Also the Map display shows data from a specific frame and stamp.

The map is different because it does not represent data from a specific sensor. It is expected to be published in a world-fixed frame, unlike point cloud data.

Not sure what you were doing but in my Rviz the visualization is wrong.

From your screenshot, I am not sure what you are doing differently.

Maybe, provide your rviz config, @Martin-Oehler?

Not sure what you were doing but in my Rviz the visualization is wrong. It does not show the drift of the odom frame over time. . I also used utm in global options and in Views:

Oh well, I did some more investigation and that one is on me. For some reason, our build server delivered on out-dated version of rviz which actually still had the proposed change from #1686. I agree, that this change makes sense for odometry, but we should discuss this in the other PR.
The main difference is that point cloud data can be from a fixed frame (map, odom) or from a frame attached to the robot body. Odom is typically published in some fixed frame.
I put a bit more thought into this and think that the new behavior is fine as an option. Using the timestamp is still the more general way, since we don't know the origin of the point cloud, but if the user knows, that the cloud is already given in a fixed frame, this feature can be useful. However, I would change the name of the option to something like "Ignore timestamp" to make the naming in line with the option in the Map display (which is "Use timestamp" and does the opposite).

@Martin-Oehler Thank you very much for providing your rosbag. It looks fine. Very interesting application of the lidar sensor by the way.
I think our discussion mainly comes from the fact that we follow different philosophies for visualization:
In my opinion, sensor data with a frame, which is attached to the robot, should move with the robot. And only sensor data with a world fixed frame should stick to the world.
This, of course, can lead to strange behavior (as one see in your videos). In my philosophy, it would be necessary to transform the sensor data into a world fixed frame based on its time stamp. (That is why the time stamp can be ignored afterwards). I understand it can be annoying to do this extra step. However, I think this is the more powerful and flexible approach as it can handle multiple (pseudo-) fixed frames like odom and utm frames in the odometry example above. Also a multi robot scenario, where each has its own lidar sensor and odom frame, can be handled. Furthermore, I guess, in order to generate an accumulated point cloud, as in the video above, which can be used in a downstream perception task, you have to transform the individual scans to a world fixed frame anyway (because of sensor motion).
However, I also understand your approach, because one gets a nice visualization quite quickly without having to transform sensor data before. It works well when there is only one fixed frame to consider, which is the typical use case with point clouds. A very big reason, why we do not work with Views -> Current View -> Target Frame option (we always set it to <Fixed Frame>), which is an important part in @Martin-Oehler's approach, is that we observe jittering in our robot model, when the Target Frame != Fixed Frame:
https://www.dropbox.com/s/aefs56f6zx0nt9g/jittering.mp4?dl=0
vs:
https://www.dropbox.com/s/fp0odav15knkz4c/no_jittering.mp4?dl=0
However, this is not a problem of @Martin-Oehler's approach itself but rather because of problems with robot model display. We are already working on a pull request, which fixes this.
I would be happy to get get this pull request accepted, so the user can decide whether he or she wants to transform the sensor data into world fixed frame by him-/herself in order to be more flexible or whether rviz should do this.

I put a bit more thought into this and think that the new behavior is fine as an option. Using the timestamp is still the more general way, since we don't know the origin of the point cloud, but if the user knows, that the cloud is already given in a fixed frame, this feature can be useful. However, I would change the name of the option to something like "Ignore timestamp" to make the naming in line with the option in the Map display (which is "Use timestamp" and does the opposite).

I am happy that you understand why this option can be useful for some applications. I also agree that your approach is the more typical one for point clouds. So the option should be off by default.
I'm not completely happy with "Ignore timestamp" /  "Use timestamp" naming, but I can accept it to be consistent with other displays.
@rhaschke What do you think?


Return metadata for each table version on queryTable with startingVersion

Return metadata for each table version, if seen in delta log, on queryTable with startingVersion.
Also updated the share config for integration test, added share8, so that when adding new tables later, we don't need to update the test for share1.

Do we need a one-decision before moving forward with this?

Added in the doc: https://docs.google.com/document/d/1BFAaZ4h5llE_7YzVO6cbROvOdtIA0KM4KVJbUY9Gs4s/edit#heading=h.nytfc746dvw8


at f.waitForHeartbeat (c:\Users\myuser\.vscode\extensions\ms-python.python-2020.9.243884588-dev\out\client\extension.js:56:574699)]
Info 2020-09-11 09:49:15: KernelProcess output: Content-Length: 132
Content-Type: application/vscode-jsonrpc; charset=utf8

{"jsonrpc": "2.0", "method": "log", "params": {"level": "INFO", "msg": "Execute rpc method kill_kernel in DS Daemon", "pid": 12428}}
Info 2020-09-11 09:49:15: Python Daemon (pid: 12428): Execute rpc method kill_kernel in DS Daemon
Info 2020-09-11 09:49:15: KernelProcess output: Content-Length: 196
Content-Type: application/vscode-jsonrpc; charset=utf8

{"jsonrpc": "2.0", "method": "log", "params": {"level": "INFO", "msg": "Execute rpc method kill_kernel from C:\\Users\\myuser\\AppData\\Local\\Programs\\Python\\Python38\\python.exe", "pid": 12428}}Content-Length: 129
Content-Type: application/vscode-jsonrpc; charset=utf8

{"jsonrpc": "2.0", "method": "log", "params": {"level": "INFO", "msg": "Kill kernel in DS Kernel Launcher Daemon", "pid": 12428}}Content-Length: 43
Content-Type: application/vscode-jsonrpc; charset=utf8

{"jsonrpc": "2.0", "id": 2, "result": null}
Info 2020-09-11 09:49:15: Python Daemon (pid: 12428): Execute rpc method kill_kernel from C:\Users\myuser\AppData\Local\Programs\Python\Python38\python.exe
Info 2020-09-11 09:49:15: Python Daemon (pid: 12428): Kill kernel in DS Kernel Launcher Daemon
Info 2020-09-11 09:49:15: KernelProcess Exit Exit - null
Info 2020-09-11 09:49:52: Cached data exists getEnvironmentVariables, extension-output-#6

You're getting this error now (not related to Ian's change):
"C:\\Users\\myuser\\AppData\\Local\\Programs\\Python\\Python38\\python.exe: No module named ipykernel_launcher\r\n"}

Did you use a different python environment? I'd try installing ipykernel again.

@rchiodo Hi,
I'm using the same python environment. And I've always the same error. The first version of @IanMatthewHuff  was working but now none works...

The error states that ipykernel_launcher is not installed. Can you verify that it is? It might be something else that's wrong.

The error states that ipykernel_launcher is not installed. Can you verify that it is? It might be something else that's wrong.

@rchiodo please could you give me the correct way to check this point ?

You can do this to check (pip won't list it).

Run "C:\Users\myuser\AppData\Local\Programs\Python\Python38\python.exe"
import ipykernel
See what error it gives (if any).

If it's not there, you can reinstall ipykernel.
For me this is what I get:
Python 3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:57:54) [MSC v.1924 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ipykernel_launcher
>>> ipykernel_launcher.__file__
'C:\\Users\\Rich\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python38\\site-packages\\ipykernel_launcher.py'
>>>

@rchiodo
Here is what I get :
Python 3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:57:54) [MSC v.1924 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ipykernel
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'ipykernel'

But if I remember good, this module should normally installed automatically with the Python extension when I click on the button for Interractive window but it's not the case.
I have the following :

Then I click "Yes" and I have:
C:/Users/myuser/AppData/Local/Programs/Python/Python38/python.exe c:\Users\myuser\.vscode\extensions\ms-python.python-2020.8.109390\pythonFiles\pyvsc-run-isolated.py c:\Users\myuser\.vscode\extensions\ms-python.python-2020.8.109390\pythonFiles\shell_exec.py C:/Users/myuser/AppData/Local/Programs/Python/Python38/python.exe c:\Users\myuser\.vscode\extensions\ms-python.python-2020.8.109390\pythonFiles\pyvsc-run-isolated.py pip install -U ipykernel --user C:/Users/myuser/AppData/Local/Temp/tmp-25376D6YREkp6KW61.log
C:\Users\myuser\AppData\Local\Programs\Python\Python38\python.exe: can't open file 'c:Usersmyuser.vscodeextensionsms-python.python-2020.8.109390pythonFilespyvsc-run-isolated.py': [Errno 2] No such file or directory

@craph Looks like your extension version doesn't have the fix from the private VSIX that I made. I'm not fully sure on this but I believe that when we release a new version of the extension it will replace existing private builds if you have your extensions set to auto update (which is the default). Could you try installing the previous vsix that worked for you and checking the number on it? I think the issue might be that we released a new public version (which doesn't have the fix for that) and it replaced the working private vsix with a new public version.

Hi @IanMatthewHuff Thank you for the update.
Here is the link of the vsix that was working for me : https://1drv.ms/u/s!AtMGc9apDXrG20TSPBLSo2zQ7Xkj?e=1GEFue
But now it doesn't work anymore and I have the following error:
C:/Users/myuser/AppData/Local/Programs/Python/Python38/python.exe c:/Users/myuser/.vscode/extensions/ms-python.python-2020.9.0-dev/pythonFiles/pyvsc-run-isolated.py c:\Users\myuser\.vscode\extensions\ms-python.python-2020.9.0-dev\pythonFiles\shell_exec.py C:/Users/myuser/AppData/Local/Programs/Python/Python38/python.exe c:/Users/myuser/.vscode/extensions/ms-python.python-2020.9.0-dev/pythonFiles/pyvsc-run-isolated.py pip install -U ipykernel --user C:/Users/myuser/AppData/Local/Temp/tmp-160968TF8J7mo0Ugh.log
Traceback (most recent call last):
  File "c:/Users/myuser/.vscode/extensions/ms-python.python-2020.9.0-dev/pythonFiles/pyvsc-run-isolated.py", line 24, in <module>
    runpy.run_path(module, run_name="__main__")
  File "C:\Users\myuser\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 264, in run_path
    code, fname = _get_code_from_file(run_name, path_name)
  File "C:\Users\myuser\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 234, in _get_code_from_file
    with io.open_code(decoded_path) as f:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Usersmyuser.vscodeextensionsms-python.python-2020.9.0-devpythonFilesshell_exec.py'

The version is : v2020.9.0-dev
And I have this error : No module named ipykernel_launcher  for the version: v2020.9.243884588-dev
No more version are working for me now 😭
I have uninstalled all the version and restart vscode none are working

@craph sorry about the delay here. I hadn't commented back here as we believed that we were rolling out the fix for you in the most recent version of the extension, which we just released a day or two ago. Have you tried the newest version of the extension?
However we just found that this fix might actually be causing issues for other customers. I looked back through the logs now and I do actually see what the root cause for you probably is. Sorry for missing it before. Looks like you are using git bash as your default terminal. This terminal (git bash on windows) is actually not supported by the python extension. You'll run into other issues using it. I'd recommend using command prompt or powershell instead when on windows.

Hi @IanMatthewHuff,
Thank you very much for the update.
You're right, I have installed git-scm on my Windows laptop for my job and I've installed it with git-bash.
When I launch Vs-code I haven't did any specific settings about terminal.
How can I deal with this ? Your advise is to remove git-scm ? Or can I change it in vs-code only ?
When I install the Python extension, i havent specify to use git-bash ... it's really weird
I don't know what to do. I use git-bash as a command line interface to run admin git commands on various projets...

@craph so the docs for the integrated terminal are here.
https://code.visualstudio.com/docs/editor/integrated-terminal
For the python extension we use this default integrated terminal, so you would just have to change a setting in vs code to change this, you don't need to change anything else about your system. On windows there is a convenient way to change the terminal setting here:
https://code.visualstudio.com/docs/editor/integrated-terminal#_windows
Changing it from git bash to either powershell or normal command prompt should unblock you.

@IanMatthewHuff I confirm that by selecting PowerShell 7 as the default Shell the interactive Window is now working. I have been able to install ipykernel successfully.
Sorry for the inconvinience about this issue and my git-bash. Is it planned to support Git-bash ?
Many thanks for your help.

To answer this question:

Is it planned to support Git-bash ?

I believe the answer is maybe? Looks like this issue here covers it:
https://github.com/microsoft/vscode-python/issues/13005


Add implementation 2 d model

Make simulations compatible with 2D models.

Some comments: You have two .dll files for the 2D model. Please remove one

Good catch! Both of them were in fact outdated examples. I replaced them by the gait1018.osim, which directly relates to an example model provided with OpenSiminstallations.

With relation to PR #71, will this PR still continue to be merged with master-dev?


concise information

I believe that https://github.com/FedeClaudi/Term.jl/blob/613502b5ca4cea2cacdf4dac0d81c5e24a3a8f05/src/renderables.jl#L39
should be renamed to something like info(renderable::AbstractRenderable), or maybe it's definition should be swapped with Base.show(io::IO, renderable::AbstractRenderable), allowing to render the panel in the repl instead of something akin to Python's repr for debugging.
This (definitions swapped) feels more natural when working withing the repl:
julia> Panel("foo")
╭──────────────────────────────────────────────────────────────────────────────────────╮
│  foo                                                                                │
╰──────────────────────────────────────────────────────────────────────────────────────╯
julia> show(io, Panel("foo"))
Panel <: AbstractRenderable size: Measure (w: 88, h: 3)

Has this been discussed somewhere ?

Great timing, I just made a comment about this in the PR you've just opened.
It took me a while to wrap my head around it (see here: https://github.com/FedeClaudi/Term.jl/issues/82#issuecomment-1133418461) but now it works as you suggested (from the next release in a couple weeks):
<img width="410" alt="image" src="https://user-images.githubusercontent.com/17436313/170836849-41c739d6-2dcd-4b7a-b66e-00ae705b23f0.png">

Thanks, I checked out v4 and using display(Panel("foo")) is outputting exactly what I was expecting (from a code not evaluated in the REPL).

Awesome, the next version will have a couple breaking changes (which shouldn't affect you thought) but also a lot of new/improved things like these.
If you're working on layout stuff for unicode plots I'd be curious to see what comes out of it! Currently Term is less than ideal in e.g. accounting for the terminal width etc. It might also be nice to have a grid(renderables... function that creates a grid layout from a bunch of renderables!

If you're working on layout stuff for unicode plots I'd be curious to see what comes out of it!

I am currently trying to remove most of the Plots calls in my old codes, and replacing them with UnicodePlots + Term for the layout.
Here on a simple cavity flow, it looks nice, although I'd be nice if the Term.renderables.Renderable <: AbstractRenderable (w:119, h:40) was made optional, but I can live with it ;)

Ha yes, grid would be awesome, I'll try something out now.

That looks great! The line at the bottom comes up when using display?
Admittedly it's not very useful in this case, but it can be when prototyping a layout with multiple renderables and has a didactic value when it comes up in the documentation.
What would be the best way to make it optional? I'm guessing display calls show here, but which version ?
Perhaps there's a way to remove the line when using display vs just showing a renderable in the REPL.

The line at the bottom comes up when using display?

Yes, because REPLDisplay uses show with "text/plain" mime when defining display: https://github.com/JuliaLang/julia/blob/eb230a55d2bb075b6c77391866bf4816fac718f5/stdlib/REPL/src/REPL.jl#L267.

I'm guessing display calls show here, but which version ?

It's this one https://github.com/FedeClaudi/Term.jl/blob/27837e44bf3e7fa37df86a23719fcc3fc377315c/src/renderables.jl#L68
, because "text/plain" specializes the generic one.

What would be the best way to make it optional?

I'd have to think about that. Everything boils down to show(...) so I feel we're a bit stuck here, unless using a dedicated function for that purpose. I always reason in terms on simplicity for the end user. So the minimalistic version is to let the REPLDisplay handle the display call under the hood in the REPL, and use the high level display in scripts otherwise.

thank's for the info. Yeah I don't think another function is necessary. perhaps just a global flag (set to false by default) which determines if we should print the extra line? Passing arguments to show might be hard...

Yes, something like const DEBUG = Ref(false), in order to be able to toggle the flag if needed.

https://github.com/FedeClaudi/Term.jl/pull/102/ adds this behavior:

where const TERM_DEBUG_ON = Ref(false). I understand the motivation for using Ref, but I have to admit I'm not a big fan of the syntax since its value is then accessed/modified with TERM_DEBUG_ON[], but I'll live with it for now.


When i  use shared-data StarRocks, and my data be stored in HDFS,I found many small files on HDFS，Can it be merged automatically?

i load data from kafka to starrocks and my table is a a large width table。
Steps to reproduce the behavior (Required)
CREATE ROUTINE LOAD tableName ON table
PROPERTIES
(
"desired_concurrent_number" = "3",
"format" = "json",
"strict_mode" = "false"
)
FROM KAFKA
(
"kafka_broker_list" = "xxxxx:9092",
"kafka_topic" = "xxxx",
"kafka_partitions" = "0,1,2"
);
Expected behavior (Required)
When load data into hdfs, small files can be merged automatically on a scheduled basis
Real behavior (Required)
I am using a large wide table with over 900 columns
I found that over 10000 small files were generated within two days, which will affect HDFS performance in the future
StarRocks version (Required)

starrocks-3.0.4

yes, the files will be compacted into large ones. the final number of files depends on the bucket number for the table and number of partitions, linear to total number of tablets.


calculate_outcomes, bin='all': all the input arrays must have same number of dimensions

Here is a new error in calculate_outcomes (log is at /Genomics/argo/users/ms0270/git/CLM.snakemake/slurm_logs/rule_calculate_outcomes/6059528.log):
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Provided resources: mem_mb=256000, mem_mib=244141, disk_mb=1000, disk_mib=954
Conda environments: ignored
Select jobs to execute...
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Provided resources: mem_mb=256000, mem_mib=244141, disk_mb=1000, disk_mib=954
Conda environments: ignored
Select jobs to execute...

[Tue Sep  3 23:13:12 2024]
rule calculate_outcomes:
    input: /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/samples/structures_SMILES_2_unique_masses.csv.gz, /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/samples/known_structures_SMILES_2_unique_masses.csv.gz, /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/samples/invalid_structures_SMILES_2_unique_masses.csv.gz, /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/inputs/train0_structures_SMILES_2.smi
    output: /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/model_evaluation/30/structures_SMILES_2_calculate_outcomes.csv.gz
    jobid: 0
    reason: Forced execution
    wildcards: output_dir=/Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1, enum_factor=30, dataset=structures, repr=SMILES, fold=2
    resources: mem_mb=256000, mem_mib=244141, disk_mb=1000, disk_mib=954, tmpdir=/tmp, slurm_partition=main,hoppertest,skinniderlab, runtime=1000

reading NP model ...
model in
(INFO) (__main__.py) (03-Sep-24 23:13:17) CLM vsrc
(WARNING) (functions.py) (03-Sep-24 23:13:21) Not enough molecules for frequency bin '2-2'. Using 92868 molecules.
(WARNING) (functions.py) (03-Sep-24 23:13:21) Not enough molecules for frequency bin '3-10'. Using 88529 molecules.
(WARNING) (functions.py) (03-Sep-24 23:13:21) Not enough molecules for frequency bin '11-30'. Using 22206 molecules.
(WARNING) (functions.py) (03-Sep-24 23:13:21) Not enough molecules for frequency bin '31-100'. Using 9273 molecules.
(WARNING) (functions.py) (03-Sep-24 23:13:21) Not enough molecules for frequency bin '101-'. Using 33300 molecules.
(INFO) (calculate_outcomes.py) (03-Sep-24 23:13:23) Reading training smiles from /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/inputs/train0_structures_SMILES_2.smi
(INFO) (calculate_outcomes.py) (03-Sep-24 23:22:30) Reading sample smiles from                                                     smiles  ...   bin
0                Brc1cc(Br)c(Oc2c(Br)cc(Br)c(Br)c2Br)cc1Br  ...  3-10
1                  Brc1cc(Br)c(Oc2cc(Br)c(Br)cc2Br)c(Br)c1  ...  3-10
2                    Brc1cc(Br)c(Oc2cc(Br)c(Br)cc2Br)cc1Br  ...  3-10
3                      Brc1ccc(Oc2c(Br)cc(Br)cc2Br)c(Br)c1  ...   1-1
4                                             C#CC#CC#CCCO  ...  3-10
...                                                    ...  ...   ...
1246171                      O=C(O)CNC(=O)CCCCCCCCCc1ccco1  ...   all
1246172    C(C[N+]1(C=O)c2ccc(O)cc2)(CNc2nc(C(N)=O)nc2)CC1  ...   all
1246173                         C=C(C)C1CCC2(C)CCC(=O)C2C1  ...   all
1246174  C1(CO)OC(c2c3c(c(=O)cc(-c4ccc(O)cc4)o3)c(O)c2)...  ...   all
1246175                   CC1CCC2(O)C(C(C)(C)O)CC3CCCC12C3  ...   all

[1246176 rows x 5 columns]
 15%|█▌        | 159563/1030768 [30:14<2:49:02, 85.90it/s][23:52:45] WARNING: could not find number of expected rings. Switching to an approximate ring finding algorithm.
[23:52:45] WARNING: could not find number of expected rings. Switching to an approximate ring finding algorithm.
[23:52:45] WARNING: could not find number of expected rings. Switching to an approximate ring finding algorithm.
[23:52:45] WARNING: could not find number of expected rings. Switching to an approximate ring finding algorithm.
[23:52:45] WARNING: could not find number of expected rings. Switching to an approximate ring finding algorithm.
 81%|████████▏ | 838053/1030768 [2:37:00<31:14, 102.83it/s][01:59:31] WARNING: could not find number of expected rings. Switching to an approximate ring finding algorithm.
[01:59:31] WARNING: could not find number of expected rings. Switching to an approximate ring finding algorithm.
[01:59:31] WARNING: could not find number of expected rings. Switching to an approximate ring finding algorithm.
[01:59:31] WARNING: could not find number of expected rings. Switching to an approximate ring finding algorithm.
[01:59:31] WARNING: could not find number of expected rings. Switching to an approximate ring finding algorithm.
 99%|█████████▉| 1022461/1030768 [3:11:10<01:18, 106.08it/s][02:33:41] WARNING: could not find number of expected rings. Switching to an approximate ring finding algorithm.
[02:33:41] WARNING: could not find number of expected rings. Switching to an approximate ring finding algorithm.
[02:33:41] WARNING: could not find number of expected rings. Switching to an approximate ring finding algorithm.
[02:33:41] WARNING: could not find number of expected rings. Switching to an approximate ring finding algorithm.
[02:33:41] WARNING: could not find number of expected rings. Switching to an approximate ring finding algorithm.
100%|██████████| 1030768/1030768 [3:22:13<00:00, 84.95it/s]
/Genomics/argo/users/ms0270/git/CLM/src/clm/commands/calculate_outcomes.py:160: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.
  valid_df = pd.concat(valid_data.to_list())
/Genomics/argo/users/ms0270/git/CLM/src/clm/commands/calculate_outcomes.py:160: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.
  valid_df = pd.concat(valid_data.to_list())
/Genomics/argo/users/ms0270/git/CLM/src/clm/commands/calculate_outcomes.py:160: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.
  valid_df = pd.concat(valid_data.to_list())
/Genomics/argo/users/ms0270/git/CLM/src/clm/commands/calculate_outcomes.py:160: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.
  valid_df = pd.concat(valid_data.to_list())
/Genomics/argo/users/ms0270/git/CLM/src/clm/commands/calculate_outcomes.py:160: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.
  valid_df = pd.concat(valid_data.to_list())
/Genomics/argo/users/ms0270/git/CLM/src/clm/commands/calculate_outcomes.py:160: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.
  valid_df = pd.concat(valid_data.to_list())
/Genomics/argo/users/ms0270/git/CLM/src/clm/commands/calculate_outcomes.py:160: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.
  valid_df = pd.concat(valid_data.to_list())
/Genomics/argo/users/ms0270/git/CLM/src/clm/commands/calculate_outcomes.py:160: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.
  valid_df = pd.concat(valid_data.to_list())
/Genomics/argo/users/ms0270/git/CLM/src/clm/commands/calculate_outcomes.py:160: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.
  valid_df = pd.concat(valid_data.to_list())
/Genomics/argo/users/ms0270/git/CLM/src/clm/commands/calculate_outcomes.py:160: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.
  valid_df = pd.concat(valid_data.to_list())
(INFO) (calculate_outcomes.py) (04-Sep-24 02:57:43) 1030768 valid SMILES out of 1246176
(INFO) (calculate_outcomes.py) (04-Sep-24 02:57:43) 1176066 novel SMILES out of 1246176
(INFO) (calculate_outcomes.py) (04-Sep-24 02:58:29) Calculating outcomes
(INFO) (calculate_outcomes.py) (04-Sep-24 02:58:31) Calculating outcomes for bin 1-1
(INFO) (calculate_outcomes.py) (04-Sep-24 03:52:56) Calculating outcomes for bin 101-
(INFO) (calculate_outcomes.py) (04-Sep-24 04:01:04) Calculating outcomes for bin 11-30
(INFO) (calculate_outcomes.py) (04-Sep-24 04:09:21) Calculating outcomes for bin 2-2
(INFO) (calculate_outcomes.py) (04-Sep-24 04:26:59) Calculating outcomes for bin 3-10
(INFO) (calculate_outcomes.py) (04-Sep-24 04:44:06) Calculating outcomes for bin 31-100
(INFO) (calculate_outcomes.py) (04-Sep-24 04:50:39) Calculating outcomes for bin all
Traceback (most recent call last):
  File "/Genomics/skinniderlab/PED-generation/env-clm/bin/clm", line 8, in <module>
    sys.exit(main())
  File "/Genomics/argo/users/ms0270/git/CLM/src/clm/__main__.py", line 81, in main
    args.func(args)
  File "/Genomics/argo/users/ms0270/git/CLM/src/clm/commands/calculate_outcomes.py", line 346, in main
    calculate_outcomes(
  File "/Genomics/argo/users/ms0270/git/CLM/src/clm/commands/calculate_outcomes.py", line 336, in calculate_outcomes
    out = calculate_outcomes_dataframe(sample_df, train_df)
  File "/Genomics/argo/users/ms0270/git/CLM/src/clm/commands/calculate_outcomes.py", line 292, in calculate_outcomes_dataframe
    if _out := handle_bin(
  File "/Genomics/argo/users/ms0270/git/CLM/src/clm/commands/calculate_outcomes.py", line 192, in handle_bin
    np.concatenate(bin_df["elements"].to_numpy()), return_counts=True
ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 1 dimension(s) and the array at index 270620 has 0 dimension(s)
[Wed Sep  4 04:51:03 2024]
Error in rule calculate_outcomes:
    jobid: 0
    input: /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/samples/structures_SMILES_2_unique_masses.csv.gz, /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/samples/known_structures_SMILES_2_unique_masses.csv.gz, /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/samples/invalid_structures_SMILES_2_unique_masses.csv.gz, /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/inputs/train0_structures_SMILES_2.smi
    output: /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/model_evaluation/30/structures_SMILES_2_calculate_outcomes.csv.gz
    conda-env: clm
    shell:
        clm calculate_outcomes --train_file /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/inputs/train0_structures_SMILES_2.smi --sampled_file /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/samples/structures_SMILES_2_unique_masses.csv.gz --known_smiles_file /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/samples/known_structures_SMILES_2_unique_masses.csv.gz --invalid_smiles_file /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/samples/invalid_structures_SMILES_2_unique_masses.csv.gz --max_molecules 500000 --seed 12 --output_file /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/model_evaluation/30/structures_SMILES_2_calculate_outcomes.csv.gz
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
srun: error: argo-10: task 0: Exited with exit code 1
srun: launch/slurm: _step_signal: Terminating StepId=6059528.0
[Wed Sep  4 04:51:05 2024]
Error in rule calculate_outcomes:
    jobid: 0
    input: /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/samples/structures_SMILES_2_unique_masses.csv.gz, /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/samples/known_structures_SMILES_2_unique_masses.csv.gz, /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/samples/invalid_structures_SMILES_2_unique_masses.csv.gz, /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/inputs/train0_structures_SMILES_2.smi
    output: /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/model_evaluation/30/structures_SMILES_2_calculate_outcomes.csv.gz
    conda-env: clm
    shell:
        clm calculate_outcomes --train_file /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/inputs/train0_structures_SMILES_2.smi --sampled_file /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/samples/structures_SMILES_2_unique_masses.csv.gz --known_smiles_file /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/samples/known_structures_SMILES_2_unique_masses.csv.gz --invalid_smiles_file /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/30/prior/samples/invalid_structures_SMILES_2_unique_masses.csv.gz --max_molecules 500000 --seed 12 --output_file /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/model_evaluation/30/structures_SMILES_2_calculate_outcomes.csv.gz
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

TODO: Make sure that canonical_smile in tabulate_molecues.py:62 can be passed on to rdkit to generate a valid molecule (one instance where it doesn't work is "c1c2c(OC3C(O)C(O)C(O)C(C)O3)c3c4c(cc(C)oc(=O)c4c31)c1c(cc3c(C(C)(C=C)C)c4c(c(O)c1oc(=O)c2c(OC)c43)cccc4)C(=O)C=C(C)C4").

Fixed in #248. For reference, command was:
snakemake --configfile /Genomics/skinniderlab/food-clm/clm/database=FooDB-representation=NA-metric=NA-optimizer=NA-rarefaction=1/config.yaml --slurm --jobs 50 --default-resources slurm_partition=main,hoppertest,skinniderlab --latency-wait=60 --rerun-incomplete


Cookies not deleted properly

Qwik Version
0.12.1
Operating System (or Browser)
Mac OS. Chrome.
Node Version (if applicable)
16.14.2
Which component is affected?
Qwik City
Expected Behaviour
cookie.delete(name) should delete cookie.
Actual Behaviour
If the cookie was created with a domain and path, the cookie isn't deleted. The delete method of Cookie doesn't take into account domain/path when setting the cookie to 'deleted'.
Additional Information
No response

Great catch! Thanks!


geoserver: add extensions and update script

Description of changes
This PR adds the  possibility to run geoserver with extensions. This is achieved by adding the function geoserver.[passthru.]withExtensions that expects a selector function that selects extensions among a given list and returns a geoserver derivation with the selected extensions installed. Example:
geoserverWithImporterExtension = pkgs.geoserver.withExtensions (ps: with ps; [ importer ]);

All extensions are listed in a new file geoserver/extensions.nix. They are fed to withExtensions upon invocation. The standard procedure to install extensions is followed (i.e. unpack extension content into geoserver/webapps/geoserver/WEB-INF/lib). My initial implementation used symlinkJoin. While the server starts, trying to access it results in java.io.FileNotFoundException: Could not open ServletContext resource [/WEB-INF/dispatcher-servlet.xml]. Besides, the wrappers (geoserver-startup, geoserver-shutdown) would have to be rewritten. So the current implementation copies the files from the extension's $out into geoserver's $out. Some extensions have dependencies. This is coped with. Some extensions do not work currently. Those are commented out for now.
Since there are >50 extensions, an update.sh script has been added to automatically update both geoserver as well as all extensions. The script is designed to work by either fetching the latest Geoserver version from GitHub or by providing specific versions. Example:
cd nixpkgs
export DEBUG=1
export UPDATE_NIX_OLD_VERSION=2.24.1
export UPDATE_NIX_NEW_VERSION=2.24.2
./pkgs/servers/geospatial/geoserver/update.sh # update geoserver + all plugins
./pkgs/servers/geospatial/geoserver/update.sh importer # update geoserver + importer plugin

passthru.updateScript is set to this new script and things should work as intended (see Nixpkgs manual section on passthru.updateScript)
nixos/tests/geoserver.nix has been adapted to test several geoserver installations. Crucially, a test is run with (nearly) all extensions enabled. This test checks for known issues in the log file.
I think there is room for improvement in this PR and I'm happy to work in comments. Overall, I think the approach is sound.
ToDos:

[ ] Add entry to "Other Notable Changes" in the 24.05 release notes?

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Looks like a good job @rollf ! I'll review it soon.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/week-in-geospatial-team/37035/7

Okay, thank you @imincik . I had seen before the label of borg internal error. I think this was me when I polished the PR (I don't remember correctly what it was, maybe I added a syntax error or so). I will remove this label now by hand because the ofborg-eval has been "pending" now for several days.

With the release of nix 2.20.0, the invocation of nix hash to-sri ... could be modified to nix hash convert --to sri ... (see release docs). Should I do this right now? I wouldn't know how, though. I'd need to force the script to run nix >= 2.20.0.

ofborg-eval doesn't seem to run at all.

ofborg-eval doesn't seem to run at all.

Can you try to force-push to this PR ? It might help.

With the release of nix 2.20.0, the invocation of nix hash to-sri ... could be modified to nix hash convert --to sri ... (see release docs). Should I do this right now? I wouldn't know how, though. I'd need to force the script to run nix >= 2.20.0.

Oh, I missed this command. I think we shouldn't use new/experimental CLI in nixpkgs. I suggest to replace it with nix-hash --to-sri. This is actually a good example of the reason why we should stick with old CLI until the new one is stable.

@rollf , thanks for your work !

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/week-in-geospatial-team/37035/8


Allow any ISelector to be used as jQuery context

Since we can determine the DOM path of every element located with ISelector we could have the option to use any ISelector instances as a context for jQuery helper method calls. Currently only JQuerySelector type is accepted.

ISelector doesn't contain enough information in order to make it possible to initialize jQuery context with any ISelector instance. On the other hand jQuery selections are meant to be used with jQuery selectors so this issue is invalid.


support for QuerySet.annotate()

It seems using QuerySet.annotate() breaks chartit. the problem lies at validation.py:30 where it validates that the fields used are part of the model. meta.get_all_field_names()  doesn't return annotated fields, for instance a Count() or an Avg() which are very useful in the case of a chart.
here's the offending code:
    model_fields = model._meta.get_all_field_names()
    if terms[0] not in model_fields:
        raise APIInputError("Field %r does not exist. Valid lookups are %s."
                         % (terms[0], ', '.join(model_fields)))
    if len(terms) == 1:
            return model._meta.get_field(terms[0]).verbose_name

as a temporary fix I changed it to this but it's just a dirty workarround and effectively breaks the validation:
    model_fields = model._meta.get_all_field_names()
    if terms[0] not in model_fields:
        pass
    if len(terms) == 1:
        try:
            return model._meta.get_field(terms[0]).verbose_name
        except:
            return str(terms[0])

rberiot
Thank you very much for this tip!


Element.createShadowRoot is deprecated and will be removed in M80

Hi,

content.min.js:2 [Deprecation] Element.createShadowRoot is deprecated and will be removed in M80, around February 2020. Please use Element.attachShadow instead. See https://www.chromestatus.com/features/4507242028072960 and https://developers.google.com/web/updates/2019/07/web-components-time-to-upgrade for more details.

I want to share with you, the warning message displayed in my console on Chrome.

I made as mistake issue is not link to instascan


add version to txInfoSigned

createSigned inside of Tx._hashAndSignAll was not passing the version provided in the txInfo param and therefor could not adjust the version.
This is required if wanting to use DashTx.js with other cryptocurrencies.

Version bumped, and pulled in.