Update silo data archive and tool to gen visit_data_files

Description
Resolves #5931
This updates the make_visit_data_files script and adds CThead.silo to pdb test data.
The broken link in #5931 is resolved with 1) update to our largedata site to house visit_data_files.tar.gz and 2) an update to our website to point at the data hosted on largedata site.
Type of change

[ ] Bug fix
[ ] New feature
[ ] Documentation update
[x] Other: data file update

How Has This Been Tested?
Checked links in various sites

I saw that you changed the behavior of "make_visit_data_dist", which I believe is fine since I couldn't find any references to it by doing

Yes, correct. That is the tool used to produce visit_data_files.tar.gz. That tool had never been updated since migration to GitHub. I also added a few Silo/HDF5 formatted files to it. CThead.silo was nowhere to be found here. I had to use Google to find an instance of it and then I copied it into our silo_pdb data archive.
Now, you can find visit_data_files.tar.gz here

I suggest regenerating visit_dist_data since there are new files that would be added to the script.

Hm....visit_dist_data or visit_data_files. These are two different beasts.
The issue at hand here is broken links for visit_data_files.tar.gz. And, yes, that was regenerated.
visit_dist_data gets generated for packaging a release and has handled inside the CMakeList.txt logic.

Can you show me where in the CMakelists.txt that visit_dist_data gets regenerated?

Can you show me where in the CMakelists.txt that visit_dist_data gets regenerated?

I think I can. But...is that related to this ticket for visit_data_files.tar.gz though?

I saw the name of the script: make_visit_data_dist and my mind interpreted this as the script that generates visit_dist_data tarball (housed in the data dir), even though the script clearly says it creates visit_data_files tarball.  My apologies.

I saw the name of the script: make_visit_data_dist and my mind interpreted

Well, I think your observation does beg the question...why do we have what appears to be two different data archives with largely the same purpose? Maybe we can get by with one?

And, I was recalling incorrectly. The logic to gen visit_dist_data.tar.gz is here...
https://github.com/visit-dav/visit/blob/d1fbae6caeb3364954f3c2b54eb0a27ae69fd889/src/tools/data/datagen/CMakeLists.txt#L391-L400

that code section you mention doesn't regenerate visit_data_data tarball, it simply untars it. In fact the code is wrong, because it is still looking for 'tar.gz' but the extension is actually '.7z'

but the extension is actually '.7z'

I don't think so. visit_dist_data should be .tar.gz because its something we expect our users to be able to deal with. We agreed that its ok for developers to have to deal with 7z archives (and possibly missing 7z tooling) but not users. So, we agreed externally released tarballs like visit_dist_data should be .tar.gz.
Regarding my reversing the meaning...for some reason I think that same CMakeLists.txt file includes logic to produce that tarball too. Let me go look again.

So, I disagree that section of that file is only about UNtaring that file. There is logic in that file to generate visit_dist_data.tar.gz too.


Encoding: option for 24bpp input support (fixes #9)

There are several kinds of possible input formats for the encoding. However, non-8bpc colors would require an overhaul to the BMP header writing.
I briefly considered an implementation that would automatically derive from the input buffer size and image dimensions the presence of the fourth byte; and throw an error if the dimensions and size are incompatible. While the added safety would probably have been a good thing for many users, I guess there might be some funkier uses of buffers which would break. Plus, the parameter at least makes it clear what is supported.
So, here is the ”put new functionality behind a flag” approach to avoid changing anything for past users.

Please update the package on npm with the fix!


Need UI / UX enhancement

The current UI needs improvement, open to ideas

Hi @DhanushNehru , could you assign this issue to me? I'd like to work on this

Hi Dhanush. Can you assign this issue to me?

@DhanushNehru assign me under hacktoberfest


Add rank_zero functions to the graveyard

What does this PR do?
Adds these deprecations https://github.com/Lightning-AI/lightning/blob/dbb5ca8d436a917fc9c2bdd6e9c00e4fd6187735/src/pytorch_lightning/utilities/distributed.py#L407-L422 to the graveyard
Closes #15536
Closes #15509
Does your PR introduce any breaking changes? If yes, please list them.
None

I was mistaken here. The problem is not rank_zero_{info,debug} but rank_zero_only which was accidentally removed without a deprecation in https://github.com/Lightning-AI/lightning/commit/d2459df2ff79184c596b2f5c865ffb17c3541307#diff-f0e5bea63aa4d620b1480af939a7d0218ede26d39f889112f4ed5a87e2b41f2dL26


UNIQUE_REG_ID setting doesn't work with MySQL

If I set UNIQUE_REG_ID to True and then run migration 0007_uniquesetting.py, I get an error MySQLdb._exceptions.OperationalError: (1170, "BLOB/TEXT column 'registration_id' used in key specification without a key length"). Django won't fix it according to https://code.djangoproject.com/ticket/2495. And there is a note about the issue in docs https://docs.djangoproject.com/en/2.2/ref/databases/#textfield-limitations
registration_id is TextField in GCMDevice, WNSDevice and WebPushDevice

Any clue about this?

Given that this is a known Django issue,  I'm not sure what we can do other than update the docs to detail this issue in the docs.


Not able to connect maestro on android lower OS version like 5, 6, 7. Can you please tell from which version it is supported?

Hi Team,
Can you please tell from which android version maestro is supported ?

Hey @VaishaliY - the minimum Android OS version is Android 8 at the moment.  We'll clarify this in our documentation. Thanks for surfacing!


[ASM] Block IP ranges

Description
This PR adds a check for blocking IP ranges. The blocked IP list sent with Remote Configuration is updated to include:
{ "value": "<IP_ADDRESS>/24", "expiration":<PHONE_NUMBER> },
{ "value": "<IP_ADDRESS>", "expiration": 1 }

<IP_ADDRESS> won't be blocked because the expiration time is in the past and because it is more specific than <IP_ADDRESS>/24.
Workflow

⚠⚠ Create your PR as draft (we're receiving lot of PR, it saves us lot of time) ⚠⚠
Follows the style guidelines of this project (See how to easily lint the code)
Work on you PR until the CI passes (if something not related to your task is failing, you can ignore it)
Mark it as ready for review

NOTE:  By default in PR only default scenario tests will be launched. Please refer to the documentation to run all scenarios in your PR if needed.

Once your PR is reviewed, you can merge it ! :heart:

Will submit another PR with a new check.


Added the option to clear errors after retrieving them

When using express-validator in multiple chained middlewares, it is useful to optionally clear the validation errors after retrieving them, so that errors from the last middleware don't interfere with the current middleware.
I have added an optional second parameter to the expressValidator.validationErrors method, that specifies whether the current errors should be cleared after retrieving them.
Example usage:
var firstMiddleware = function(req, res, next) {
    req.checkParams("something", "No valid something specified").isInt();
    var errros = req.validationErrors(false, true); // this clears the errors
    if(errors) {
        // do stuff to handle validation error
        return next();
    }
    next();
};

var secondMiddleware = function(req, res, next) {
    req.checkQuery("p", "No valid p specified").notEmpty().isInt();
    var errors = req.validationErrors();
    // without this, errors would contain all the errors that have occurred in earlier middleware,
    // even though they have been handled
    if(errors) {
        // return some kind of error to the user
    }
    // do useful stuff
};

app.get("/:something", firstMiddleware, secondMiddleware);

Too far gone from master -- please reopen if concern is still valid.


fix: Fix typing of include_spark parameter

Description
This pull request addresses some feedback received in our ApiView review that correctly pointed out that the include_spark parameter should be type bool and not Optional[bool]
All SDK Contribution checklist:

[x] The pull request does not introduce [breaking changes]
[x] CHANGELOG is updated for new features, bug fixes or other significant changes.
[x] I have read the contribution guidelines.

General Guidelines and Best Practices

[x] Title of the pull request is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[x] Pull request includes test coverage for the included changes.

API change check
APIView has identified API level changes in this PR and created following API reviews.
azure-ai-ml


MESA community codes need individual pip packages

Currently, we have one amuse-mesa package (which doesn't work). This is not ideal, since it is unclear which MESA this will install.
I think it's probably best to have individual packages for mesa_r15140 and mesa_r2208, and also a package amuse-mesa which will depend on mesa_r15140 (I don't think it's possible to have an "or" dependency, so I'd choose the newer one).

suggested package names:
amuse-mesa-r15140 (installs amuse.community.mesa_r15140)
amuse-mesa-r2208 (installs amuse.community.mesa_r2208)
amuse-mesa (depends on amuse-mesa-r15140, installs amuse.community.mesa)

@ipelupessy do you agree with the above?

Is the plan here to bundle a fully compiled MESA with all its data files into a python package?

No, that would not work.


Add support for backslash-escaped multiline strings

All JS engines (as far as I know; tested in V8, SpiderMonkey, and JSC) support backslash-escaped multiline strings, but this still chokes on them.  Really hoping for support, as it's hard to edit userscripts when your inlined CSS blows everything up.
Example of syntax:
var things = "this is a\
really long\
multiline string";

backlash-escpaed multiline strings are not working. The changes referenced in the pull are not applied in the file.


Have plan for VSCode?

有，目前正观察atom中使用情况，下载量超过100就开始~~^-^

Hi, I want to migrate this plugin to https://github.com/octref/vetur.
Vetur is the most popular extension for Vue in vscode. It is extensible enough to support Element out of box.  It is also MIT licensed. No license issue should occur.
@ElementFE What's your opinion?

@HerringtonDarkholme
Hi, I quite approve of your plan,  just do it!

Pull Request is here! https://github.com/octref/vetur/pull/234

@ShowJoy
After a discussion with @HerringtonDarkholme, I have published element-tags.json and element-attributes.json to npm, so you can use directly instead of maintaining these two files here. Usage: https://github.com/octref/vetur/pull/234#issuecomment-309277127

@Leopoldthecoder Great!  thanks.

No problem.👏

@liuyatao Although a little late...., VSCode version has done! see vscode-element-helper


Upgrade @parameter1 base-cms dependencies to 2.8.6

 Add website section event context to Leaders Data Card > Open events


update(css-nth-child-of): update spec status

add link to MS Edge platform status
update spec to reflect published standard

Relates to #1058

If you set "cssselectorslevel4" as "ie_id" value that should fetch the status automatically.

@Fyrd: Is that all I need to change? What about spec or the new link?


Add Renovate config

We currently use Dependabot for managing updates, but it seems that a lot of teams are migrating to Renovate now, so I thought we’d give it a go. We get a lot more control over what dependencies we can update, and can be a lot smarter about rebasing etc.

We should probably remove dependabot.yml at the same time as adding this

Tru dat. Lemme sort it


feat(cli): add logging modes, show version, add bootstrap config

add --log option for different logging modes
add --version option to show version
add init subcommand to bootstrap a default config file
Signed-off-by: Daniel Richter<EMAIL_ADDRESS>Closes #16, Closes #15, Closes #7
Purpose
Describe the problem or feature in addition to a link to the issues.
Approach
How does this change address the problem?
Open Questions and Pre-Merge TODOs

[ ] Use github checklists. When solved, check the box and explain the answer.

Learning
Describe the research stage
Links to blog posts, patterns, libraries or addons used to solve this problem
Blog Posts

How to Pull Request Github Repo with Learning focused Pull Request Template.

Codecov Report

Merging #29 into master will increase coverage by 0.22%.
The diff coverage is 92.85%.

@@            Coverage Diff             @@
##           master      #29      +/-   ##
==========================================
+ Coverage   97.87%   98.09%   +0.22%
==========================================
  Files           7        7
  Lines          94      105      +11
  Branches       16       15       -1
==========================================
+ Hits           92      103      +11
  Misses          2        2

Impacted Files
Coverage Δ

plugins/bump-package-json/src/bump.ts
94.44% <85.71%> (+0.69%)
:arrow_up:

plugins/type-git/src/type.ts
98.11% <93.75%> (+0.19%)
:arrow_up:

plugins/determine-semver/src/determine.ts
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 76244de...2afc715. Read the comment docs.


Automate building of Docker images

This adds automation of building docker containers to ease development.  The build infrakit/bundle image is for development only and not meant for production deployment.

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "circle_ci" git@github.com:chungers/infrakit.git somewhere
$ cd somewhere
$ git rebase -i HEAD~842354357944
editor opens
change each 'pick' to 'edit'
save the file and quit
$ git commit --amend -s --no-edit
$ git rebase --continue # and repeat the amend for each commit
$ git push -f

Amending updates the existing PR. You DO NOT need to open a new one.

Current coverage is 68.83% (diff: 100%)

Merging #339 into master will increase coverage by 0.06%

@@             master       #339   diff @@
==========================================
  Files            30         30
  Lines          1569       1569
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           1079       1080     +1
+ Misses          383        382     -1
  Partials        107        107

Powered by Codecov. Last update 159cfe0...c1663c1

I need to merge this in so that circle can build images on master.


⚠ ClamAV has degraded performance

In 09c998b, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 223 ms

Resolved: ClamAV performance has improved in db82be5.


Adding codesandbox of react-hotkeys

Thanks for the suggestion, atomicpages.
I've not used codesandbox.io before. Can you tell me, if we include a link in the Readme, will it automatically update whenever the example project is updated - or must a new sandbox be generated and a new link included in the readme each time that happens?

@greena13 you can fork the project and use your own URL 👍


Feature Requests

I have a couple of feature requests for ng-aside:

Allow the inside side to be optionally resizeable, ideally with a custom bit of HTML to act as a handle. For example, if the aside is anchored to the right side then the left has a widget to allow of resizing horizontally. The template could have a aside-resize-handle section to act as the anchor for the interaction.

I would love if there could be an option where the underlying content behind the aside (the main page) was still able to be interacted with. I understand that having based the implementation on the modal, might present some inherent difficulties, but it would be a great option to have.

Thanks!

Also, it'd be great to have the menu render above the content, instead of pushing it either way. I doubt it'd be tough, but it'd be cool to have out-of-the-box.


Added self.API.FeatureFlags:GetFeatureFlag("AllowDroppingSwords") to the documentation

The updated doc is now up: https://thenexusavenger.github.io/Nexus-Admin/included-feature-flags/#allowdroppingswords-default-true


there is any way to setup this in subdomain?

i don't know if this is directaly related to the alexa integration,or to nginx setup,but there is any way to integrate with subdomain?
i've tinking about it and the only way i can figure out to do it is to redirect all the requests to the port 80 with a specific patter related to the alexa api

Which is the reason for that, can you please make an example? Thanks

i use some sub domains into my server,to isolate every docker container , but my nginx reverse proxy automaticaly redirect all the 80 requests to the 443 port,except by those to the node-red and home assistant subdomains,but i need to redirect to the api url instead of it,since this does not work,wich is this subfolder url? to redirect the requests to this url to the port 80 only

Hi. I am curious... why are you going via nginx?
Surely you are just adding node-red-contrib-amazon-echo palette to your node-red running in a docker container?

Closing as It is not related to the functionality of this module.


[MWPW-158314 : NALA] Add nala tests for additional marquee variants

Update console error steps
Add test scripts for new marquee variants.
- Marquee (large, standard)
- Marquee (large, compact)
- Marquee anchors
- Marquee anchors (transparent)
- Marquee background image focal point)

Resolves: MWPW-158314
Test URLs:

Before: https://main--milo--adobecom.hlx.page/?martech=off
After: https://marquee-test-updates--milo--adobecom.hlx.page/?martech=off

There's a bunch of video marquees https://github.com/adobecom/milo/pull/2915
and CC has an "interactive marquee" https://github.com/adobecom/cc/pull/413 should we add tests for those as well?

There's a bunch of video marquees #2915 and CC has an "interactive marquee" adobecom/cc#413 should we add tests for those as well?

@mokimo : The video marquee (as part of cc#413) is in progress, and I will add them separately. (will add it as part of the video block )


Font Awesome 5 (free)

Update resources for Font Awesome 5.0.8 free and remove Font Awesome 4.5.
Update references to Font Awesome in index.html and preferences.html.
Update to hain-commands package.json to use #fas fa-cog. This is the only icon syntax difference I could find with default hain plugins.

Sorry for the late response, out of town for summer break.

No worries! Thank you for the merge 😀


Can't generate project

Issue details
I can't generate project like it's described in this tutorial

project generate problem. see stack trace below
I can't see ios-moe project
this is my seetings.gradle content include 'desktop', 'android', 'ios', 'core'

Reproduction steps/code
follow steps from project setup wiki
Version of LibGDX and/or relevant dependencies
from build.gradle
        appName = "my-gdx-game"
        gdxVersion = '1.9.10'
        roboVMVersion = '2.3.7'
        box2DLightsVersion = '1.4'
        ashleyVersion = '1.7.0'
        aiVersion = '1.8.0'

Stacktrace
Generating app in C:\workspace\game
Executing 'C:\workspace\game/gradlew.bat clean --no-daemon idea'
To honour the JVM settings for this build a new JVM will be forked. Please consider using the daemon: https://docs.gradle.org/5.4.1/userguide/gradle_daemon.html.
Daemon will be stopped at the end of the build stopping after processing

FAILURE: Build failed with an exception.

* What went wrong:
Task 'idea' not found in root project 'game'.

* Try:
Run gradlew tasks to get a list of available tasks. Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 8s
Done!
To import in Eclipse: File -> Import -> General -> Existing Projects into Workspace
To import to Intellij IDEA: File -> Open -> YourProject.ipr

Please select the affected platforms

[x] Android
[ ] iOS (robovm)
[x] iOS (MOE)
[ ] HTML/GWT
[x] Windows
[x] Linux
[x] MacOS

I've use following generator configuration

FYI:
OS: windows 10
JAVA_HOME points to openjdk 11
IDEA 2019.2

@badlogic could you please check this?

See #5781

@SimonIT thanks. this help me go generate project without error. but what about missed ios-moe project?

Please see https://github.com/libgdx/libgdx/pull/5700#issuecomment-514636568


xDS: add api version fallback from v3alpha -> v2

Signed-off-by: shikugawa<EMAIL_ADDRESS>Description: If specified xDS api version is v3alpha but management server supports only v2, version fallback is required to reduce user troubles to switch v3alpha -> v2.
Risk Level: Mid
Testing: Unit / Integration
Docs Changes: maybe required
Release Notes:
[Optional Fixes #Issue] #9617
[Optional Deprecated:]

Thanks @Shikugawa. I'm going to punt this one until after 1.13.0, since it's a potential wire change that we should avoid while stabilizing.


pip install does not work on Ubuntu 22.04 LTS

Describe the bug
pip install does not work on Ubuntu 22.04 LTS
To Reproduce
Steps to reproduce the behavior, e.g.:

Install Ubuntu 22.04 LTS Server
Run: pip install --force-reinstall git+https://github.com/ericaltendorf/plotman@main

Expected behavior
Should install plotman successfully.
System setup:

OS: Ubuntu 22.04 LTS Server

Config
None yet

Additional context & screenshots
$ pip install --force-reinstall git+https://github.com/ericaltendorf/plotman@main
Defaulting to user installation because normal site-packages is not writeable
Collecting git+https://github.com/ericaltendorf/plotman@main
  Cloning https://github.com/ericaltendorf/plotman (to revision main) to /tmp/pip-req-build-jt3m0is9
  Running command git clone --filter=blob:none --quiet https://github.com/ericaltendorf/plotman /tmp/pip-req-build-jt3m0is9
  Resolved https://github.com/ericaltendorf/plotman to commit 1b7dfba139ed03d3c32a2f04f6011e03bfb1f442
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting desert==2020.11.18
  Downloading desert-2020.11.18-py3-none-any.whl (9.5 kB)
Collecting packaging~=20.9
  Downloading packaging-20.9-py2.py3-none-any.whl (40 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.9/40.9 KB 1.4 MB/s eta 0:00:00
Collecting marshmallow~=3.12
  Using cached marshmallow-3.20.1-py3-none-any.whl (49 kB)
Collecting pyyaml~=5.4
  Using cached PyYAML-5.4.1.tar.gz (175 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [43 lines of output]
      running egg_info
      Traceback (most recent call last):
        File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 130, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 162, in get_requires_for_build_wheel
          return self._get_build_requires(
        File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 143, in _get_build_requires
          self.run_setup()
        File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 158, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 271, in <module>
          setup(
        File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 153, in setup
          return distutils.core.setup(**attrs)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 148, in setup
          return run_commands(dist)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 163, in run_commands
          dist.run_commands()
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 967, in run_commands
          self.run_command(cmd)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 986, in run_command
          cmd_obj.run()
        File "/usr/lib/python3/dist-packages/setuptools/command/egg_info.py", line 299, in run
          self.find_sources()
        File "/usr/lib/python3/dist-packages/setuptools/command/egg_info.py", line 306, in find_sources
          mm.run()
        File "/usr/lib/python3/dist-packages/setuptools/command/egg_info.py", line 541, in run
          self.add_defaults()
        File "/usr/lib/python3/dist-packages/setuptools/command/egg_info.py", line 578, in add_defaults
          sdist.add_defaults(self)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/sdist.py", line 228, in add_defaults
          self._add_defaults_ext()
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/sdist.py", line 312, in _add_defaults_ext
          self.filelist.extend(build_ext.get_source_files())
        File "setup.py", line 201, in get_source_files
          self.cython_sources(ext.sources, ext)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 103, in __getattr__
          raise AttributeError(attr)
      AttributeError: cython_sources
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Same here.


[Linux] Wrong permissions on downloaded /native/ libs.

I started using the nightly updater and kept seeing a problem with mediainfo issues. The files were being downloaded but the issue ended up being that they did not have the execute permission bit set so they were not able to be loaded. After manually using chmod, the errors went away and using the mediainfo function worked again.
There is also a discrepency with the 64bit reporting in that they are downloaded to /native/linux-amd64/  but the application says it is trying to load from /native/linux-x86-64/
Also, a request, would it be possible to have the application use the system libraries if they exist, perhaps with a config option on whether to prefer native over system? It would also be nice to print out the library version in the log when it loads it.

which linux distribution (and arch) do you have? I did not see any similar problem at home (using manjaro linux)

Using Fedora 23 x86_64. During my various tests, I had deleted the native folder a few times and each time it downloaded the files, they didn't have the execute bit set. I tried it again just now with the same result.
I'm not able to access the machine to launch the actual gui, but I expect it will give a load failure when lauching. I'll check in a few hours when I get home and update.

Tested, and fails to load the library on freshly download libzen and libmediainfo files.

09:11:11.079 ERROR org.tinymediamanager.core.Settings - could not parse proxy port: null
09:11:11.202 ERROR org.tinymediamanager.thirdparty.MediaInfo - Failed to load mediainfo
java.lang.UnsatisfiedLinkError: Unable to load library 'mediainfo': Native library (linux-x86-64/libmediainfo.so) not found in resource path

Manually doing a chmod 755 on the files has it working as expected.

just checked it at home
a) we do not ship a linux-x86-64 folder inside native (only amd64; I did not even know that some linux distros name it as x86-64.. - will put in a patch for loading)
b) I definitely do not need the executable bits set for loading the native libs (644); please try to rename the native/linux-amd64 folder to native/linux-x86-64 and have a look if it is loading or not

ad loading native mediainfo:
try to create a file called extra.txt in the tmm dir with the following content:
-Djna.nosys=false
this should force Java to load the native lib

This is already set in getdown.txt.  I tried adding it anwyays with the same result. The problem is the lack of executable bit. Fedora must be doing something different, or it could be some toggle in how I build my kernel, that prevents the library from loading if this isn't set. RedHat does apply a lot of extra security hardening bits to the packages they compile that nobody else does yet, so that could be the difference.

Maybe it's just something with my system. Did some more experiments and when using the bfd linker I can't get the libraries to load at all. When switching back to my default of the gold linker, they load, but only if executable.
Strange stuff, feel free to close since this only seems to be affecting me.

Definitely seems to be something wonky on my end. After trying lots of random things, it now works as expected. I don't get why it is acting so randomly, but will close this issue. Thanks for helping!

no need for closing it. I've just done some research and there are several sites on the net which suggest that the native libs should have the executable bit set (maybe that has something to do with SELinux or similar).
I will create a patch for it

a new build is up.

libs under /native/linux-* will get +x at start up of tmm to make sure they are loadable
unified loading paths of libmediainfo

during my tests I've recognized that jna loads the distribution installed mediainfo whenever the
provided lib under /native/linux-* is not loadable (missing or incompatible).
We ship a mediainfo lib which has been built for ubuntu, but at my arch linux this was not loadable -  thus tmm loads the system library

I updated and did a bunch of testing and everything is working as you described, @mlaggner. Thanks for the updates and your patience!


Cant access https://yajrabox.com/docs/laravel-datatables/7.0/upgrade

Summary of problem or feature request
Unable to access https://yajrabox.com/docs/laravel-datatables/7.0/upgrade. Times out.

The website is currently down.

You can still access the website through web.archive.org

Or google cache: http://webcache.googleusercontent.com/search?q=cache%3Ahttps%3A%2F%2Fyajrabox.com%2Fdocs%2Flaravel-datatables%2F7.0%2Fupgrade&ie=utf-8&oe=utf-8&client=firefox-b-ab

It's up now, not sure why it went offline. Thanks!


ERROR: Could not build wheels for rapidfuzz, which is required to install pyproject.toml-based projects

BACKGROUND
I am trying to pip install openbb
PROBLEM
I get following ERROR:
Could not build wheels for rapidfuzz, which is required to install pyproject.toml-based projects
Please note, that I am using rapidfuzz 2.9.0 version, but the issue remains also by using 3.3.1
My software is
System Version: macOS 13.6 (22G120)
Kernel Version: Darwin 22.6.0
Python 3.11.4

Took me a bit to get down to the bottom of this. The latest versions of openbb do no longer claim support for Python 3.11 since https://github.com/OpenBB-finance/OpenBBTerminal/pull/4695, so pip install openbb actually installs openbb version 2.3.1. This old version of openbb did depend on rapidfuzz version 1.9.1 which fails to compile.
So either openbb does not support Python 3.11 or it incorrectly states that it doesn't support it. Either way this is an issue with openbb.

Thank you very much for your efforts and information.
I will follow your advice.


Miner's Backpack: Shift from iron to bronze?

Would make it more accessible, alternately any ore?

How about either copper or tin ingots, or bronze nuggets?


Make it easy to use custom styles

Part of the way there (toggleable themes) but needs a better way to add user-created themes than 'edit the source code'.

Themes should come in subfolders in an 'extras' folder (eg 'extras/some-theme-name/style.css').
try-mesh.html should load the default, embedded theme, then override it with a style.css file in the same folder (if there is one). This allows users to just copy across themes from the themes directory instead of having to rename them.
It also simplifies the custom theme code, because theme selection and browsing is now handled via the file system UI.
But if we did include Mesh themes - it would be cool to apply the theme as soon as the mouse is hovered over. Maybe something a separate themes website could do.


support to csproj

Hello ,
Are you will be planning add support to csproj projects ?

Hi biasmey,
The csproj based tooling actually supports directly setting the version prefix by passing the msbuild property /p:VersionPrefix=1.2.3 to any relevant dotnet commands (namely pack, publish, and build). Therefore, this tool is no longer needed once you migrate to csproj tooling, and I will not be continuing development.
Paul

Thank you so much


bug: cannot setup without providing empty table

Setting up the plugin without providing arguments, like so:
require('refjump.nvim').setup()

results in the following error:
E5108: Error executing lua vim/shared.lua:0: after the second argument: expected table, got nil
stack traceback:
        [C]: in function 'error'
        vim/shared.lua: in function 'validate'
        vim/shared.lua: in function 'tbl_deep_extend'
        .../site/pack/rocks/start/refjump.nvim/lua/refjump/init.lua:45: in function 'setup'
        [string ":lua"]:1: in main chunk

It would be nice if the argument was optional, as this plays nicer with some plugin managers.

Fixed in c630682f02d06552947578367034d9f9db500526! Thanks for reporting this. It's ironic because this annoys me a lot when plugins require an argument to .setup() but I completely forgot about when making this plugin lol.


Values from metrics not getting pushed by pusher in combination with AWS Lambda

Hello,
I'm having an AWS Lambda in which I like to add some custom metrics and push the data to Prometheus.
The Lambda is very simple. It calls an API endpoint and receives data.
The problem I'm having is that no values appear of metrics inside Prometheus, but the metrics did get created. I create a pushserver, add a couple of counter metrics, start the server and finally stop the server when the AWS Lambda is done doing what it needs to do.
Output in Prometheus:
https://ibb.co/brvzT33
My code is looking like this:
private IMetricPushServer _metricsPushServer;

public async Task<T> FunctionHandler(ILambdaContext context)
{
    try
    {
        var registry = new CollectorRegistry();
        var factory = new MetricFactory(registry);
        var pusher = new MetricPusher(registry, "some endpoint", "some name", "some id", labels, null);
        _metricsPushServer = new MetricPushServer(new IMetricPusher[] { pusher });
        var testCounter = factory.CreateCounter("test", "this is a test");

         _metricsPushServer.Start();

        //calls API endpoint

        testCounter.Inc();
    }
    catch
    {
        //error handling
    }
    finally
    {
         _metricsPushServer.Stop();
    }
}

Anyone has any idea what I could do differently or try out etc.?

Metrics are not appears on Prometheus, because pushserver does not have a chance to push them... as it pushes metrics with some timeout, but you stops the server immediately when your unit of work is done.
For such use cases it might makes sense to use IPusher directly:

Create registry and factory (as you already done)
Create metrics and use it
Create MetricPusher by providing registry and Prometheus push gateway
When your unit of work is done - use PushAsync to send the metrics

Makes sense. The Lambda only takes a couple of seconds to complete it's run. So timing may be an issue here.
Unfortunately I can't verify if your possible solution works. Due to changes in project setup we don't need the Lambda anymore. Therefore I'm closing this issue.
Thanks for your time!


New version: ClimaCorePlots v0.2.0

Registering package: ClimaCorePlots
Repository: https://github.com/CliMA/ClimaCore.jl
Created by: @simonbyrne
Version: v0.2.0
Commit: 41106f0f9043a3d66c2ed8367f509f59b199c034
Reviewed by: @simonbyrne
Reference: https://github.com/CliMA/ClimaCore.jl/commit/41106f0f9043a3d66c2ed8367f509f59b199c034#commitcomment-65563442
Description: CliMA model dycore

@simonbyrne please see the message above.  Once you fix the automerge issues you can register again the new revision without changing the version number and this pull request will be automatically updated.
[noblock]


mbMAP: webGL detection

[ ] Check that a browser has webgl enabled
[ ] if not,

show a dismissable message (store dismissal in localstorage) that tells the person how to update their browser
hide the map and the legend

related:
https://www.mapbox.com/mapbox-gl-js/api/#supported
https://www.mapbox.com/mapbox-gl-js/example/check-for-support/


Completely is not working

When i add the dependency and follow the rest of the steps to add this library, it doesn't show in my android studio I can't call it, the code is not recognized.

please read #858


[BUG]: How to save model when utilizing pipeline parallel?

🐛 Describe the bug
When I tried the pipeline_parallel demo in the link: https://github.com/hpcaitech/ColossalAI/tree/main/examples/language/gpt/experiments/pipeline_parallel, I found that the parameters of the model did not change after the "forward_backward" step. I guess the model has been splitted into submodules in each worker, and I should save the submodule in each worker. Are there some example guidelines to save the splitted submodules?
Environment
No response

We will expose a user api to save the split modules.


GUI : see all file status on main interface & Ressource Loading

Maybe expandable

Now includes the finalization of ExternalRessource loading


Installing package refused

Problem encountered on https://dotnet.microsoft.com/en-us/platform/upgrade-assistant/tutorial/install-sdk
Operating System: windows
Provide details about the problem you're experiencing. Include your operating system version, exact error message, code sample, and anything else that is relevant.

I was trying to install Microsoft.net SDK 8.0.302 but it refused saying that the installation package is not supported  by the processor

For issues with the SDK itself please raise an issue at https://github.com/dotnet/sdk/issues


C++/MFC resources files (.rc) syntax highlighting

Not sure if newer VS is still using .rc files, but would be nice to have syntax highlighted for them for opening old projects in VSC. For example Notepad++ does support rc-files (as a "Resource file" language). Just basic coloring would be enough for me.
I guess tokenization would be similar to C++ (at least in terms of quotes and comments), with a different set of keywords (DIALOG, MENU, STRINGTABLE, BEGIN, END, CONTROL, LTEXT, RTEXT, EDITTEXT, GROUPBOX, ...). If BEGIN-END would create a collapsible context then that would be great but not required.

We try to keep VS Code lean and we think the functionality you're asking for is great for a VS Code extension. Maybe you can already find one that suits you in the VS Code Marketplace. Just in case, in a few simple steps you can get started writing your own extension. See also our issue reporting guidelines.
Happy Coding!


Publishing on the store

I'm currently developping an application that uses your library.
When it'll be finished i'd like to publish it on the play store.
Is it possible ?

It is possible, why not?

I'm guessing they wanted to know the licence details in  summary not the licence jargon.
As in the you can do this but you can't do that....

Library is published on Apache License, may be used in free open source and in commercial closed source apps.


Automatically adjust heading levels for the newly created note

In the selected note content, the highest level of heading may be Level 4 header. However, in the newly created note, the desired level may be Level 2 header. Therefore, it would be great if this can be automatically converted.
Specifically, the old note content to be split may be:
#### Header Level 4

Note.

##### Header Level 5

Note.

The new note content may be:
# Header Level 1 (this may be created by using the template)

## Header Level 2

Note.

### Header Level 3

Note.

Added in 1.8.1


ldshell: command "logs create"

Pull Request Template
Description
close #214
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Manual testing
Checklist:
Must:

[x] I have run format.sh under script
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] New and existing unit tests pass locally with my changes

Semi-Must

[ ] I have added new tests that prove my fix is effective or that my feature works
[ ] I have checked my code and corrected any misspellings

Optional:

[ ] My code follows the style guidelines of this project
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] Any dependent changes have been merged and published in downstream modules

close via https://github.com/hstreamdb/hstream/pull/236


Custom Items

Hello, I am trying to use this plugin for a custom currency on my server. But when i setup a trade sign with a custom token, i used a slime ball with specific meta values on it, it will also take regular slime balls.
Is there a fix?

Are you configuring the items.yml file correctly? You basically need to give your special slimeball an alternative name and use that one on the sign.
(Like “coin”, or just”slimes” - note the “s” st the end)

@slimeguy24 Here's a link to the wiki page on custom items, make sure you're using them for it to work properly. If it still does not seem to be working and you are on the newest version then give us an ip if possible and I can get on and try to figure out whats not working.
https://github.com/SparklingComet/TradeShop/wiki/Custom-Items

@sparklingcoment I gave myself a slime ball with the name Slime Token and a special lore and I type /ts additem SlimeToken
And it adds the item just fine but when a user goes to a sign where u use the slime token to pay for an item they can just have normal slime balls in their inventory, it dosent take the normal slime balls but it still puts a slimetoken in the chest and gives them the item for free

@slimeguy24 can you paste your customitem. Yml file here?

@slimeguy24 and you're on the newest version of the plugin and at least 1.12?

yes I am

@slimeguy24 do you have an ip for the server?

wrong button...woops :P

Closing. If this is not fixed, reopen the issue.


PSCustomObject does not work with Select-Object -Unique and Sort-Object -Unique

Prerequisites

[X] Write a descriptive title.
[X] Make sure you are able to repro it on the latest released version
[X] Search the existing issues.
[X] Refer to the FAQ.
[X] Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce
Given any collection of PSCustomObjects with different fields, they will not compare equal but still collapse into a single item under Select-Object -Unique or Sort-Object -Unique. Notable cmdlets outputting PSCustomObject collections are:

Select-Object itself (-Unique works when used immediately, but not later on)
ConvertFrom-Csv

This behavior has been mentioned in https://github.com/PowerShell/PowerShell/issues/15806 and https://github.com/PowerShell/PowerShell/issues/12059 , but neither are for this exact problem, and it affecting Select-Object transformations and ConvertFrom-Csv makes it very prominent.
Expected behavior
> $files = Get-ChildItem | Select-Object Name

> $files
Name
----
dotnet-sdk-5.0.408-linux-arm64
dotnet-sdk-6.0.400-linux-arm64

> $files[0] -eq $files[1]
False

> $files | Select-Object -Unique
Name
----
dotnet-sdk-5.0.408-linux-arm64
dotnet-sdk-6.0.400-linux-arm64

Actual behavior
> $files | Select-Object -Unique
Name
dotnet-sdk-5.0.408-linux-arm64

Error details
No response
Environment data
Name                           Value
----                           -----
PSVersion                      7.3.0-preview.3
PSEdition                      Core
GitCommitId                    7.3.0-preview.3-304-gd02c59addc24e13da3b8ee5e1a8e7aa27e00c745
OS                             Linux 5.15.0-1013-raspi #15-Ubuntu SMP PREEMPT Mon Aug 8 06:33:06 UTC 2022
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

Visuals
No response

> $files = Get-ChildItem | Select-Object Name
> $files  | Select-Object * -unique

Works , but if the second doesn't have a property parameter it fails.
The two linked items suggest that select-object uses the hash code, but this casts doubt on that.
I've never understood why -ExcludeProperty doesn't work unless property is specified, I wonder if both could be solved if Property defaulted to "*"

It seems we already have such issue...

@iSazonov It's not a duplicate because #15806 is just about better hashing, which may improve for example storing PSCustomObject in dicts, but is never enough for equality testing. Just because two objects have same hash does not mean they are equal. I quickly found two guid strings which have same hash codes but do not have this problem:
> $x = 'f8c9ced6-86ee-432d-925a-7fcc6efeb881'
> $y = '4696130c-891d-4b66-9ab8-d1d92948155e'
> $x -eq $y
False
> $x.GetHashCode() -eq $y.GetHashCode()
True
> $x, $y | Select-Object -Unique
f8c9ced6-86ee-432d-925a-7fcc6efeb881
4696130c-891d-4b66-9ab8-d1d92948155e

@dkaszews I guess hashcode is a root of the issue. You could try @SeeminglyScience's proposal https://github.com/PowerShell/PowerShell/issues/15806#issuecomment-883825587 and either confirm or disprove this.

@iSazonov Please look at my example, it proves that hashcode is not the root of the issue because it's not a problem for other types. I can try that suggestion, but I doubt it will work. Can you at least remove the duplicate tag until I try it, so this issue will not get closed automatically? If it does indeed help, I will close it myself.

@dkaszews I cannot reproduce the issue. Please create repro steps like Pester tests (with creating demo files).

@iSazonov Here is a minimal isolated test case using ConvertFrom-Csv. Unsure why pester thinks the first does not match, may be another bug, but you can clearly see difference in behavior
> $data = "a`n1`n2" | ConvertFrom-Csv
> $data | Select-Object -Unique * | Should -Be $data
InvalidResult: Expected @(@{a=1}, @{a=2}), but got @(@{a=1}, @{a=2}).
> $data | Select-Object -Unique | Should -Be $data
InvalidResult: Expected @(@{a=1}, @{a=2}), but got @{a=1}.

The first failing test is due to array oddity, piping both expected and actual through ConvertTo-Json - Compress fixes it

I cannot reproduce the issue.
@iSazonov  try this simple one in any directory with multiple files

$files = Get-ChildItem | Select-Object Name
$files  | Select-Object  -unique
$files  | Select-Object  -unique *

The second line only returns one item. The third returns all of them.   The same happens if you use Sort-Object -unique
But if the first line is $files = Get-ChildItem | % name or  just $files = Get-ChildItem   both lines return all items
I don't think it is hashing
I think the code ( https://github.com/PowerShell/PowerShell/blob/master/src/Microsoft.PowerShell.Commands.Utility/commands/utility/Select-Object.cs  line 628 onwards)  compares base objects without note properties - and guess what... PSCustomObjects are ALL note properties -and ONLY looks at note properties if -Properties is specified. So
$a = get-item .
$b = get-item .
$a, $b | Select-Object -unique

Correctly returns ONE item
Add-Member -NotePropertyName "foo" -NotePropertyValue "bar" -InputObject $a
$a, $b | Select-Object -unique

Should make a difference between a and b and return two items, but Select-Object ignores it because it is a note property
 $a, $b | Select-Object -unique name

Correctly only returns one object because it is not told to look at the note property that is different
 $a, $b | Select-Object -unique name,foo

Now returns two objects because it IS looking at the note property.
This looks to be by design  two objects are not considered different if they only differ in note properties, UNLESS those note properties are specified. (Which is why adding * fixes the problem in the initial example). But that design doesn't allow for PSCustomObjects

The example with csv is simpler and does not depend on the current directory, clearly demonstrating the issue. I agree with @jhoneill that NoteProperties being omitted by default are to blame. Defaulting -Properties to * sounds like a good solution. Alternatively, changing Select-String to consider NoteProperties when the objects would otherwise be "empty" can fix PSCustomObject, but would still be unexpected for objects which have a mix of different prop types.
> $data = "a`n1`n2" | ConvertFrom-Csv
> $data
a
1
2
> $data | Select-Object -Unique *
a
1
2
> $data | Select-Object -Unique
a
1

$files | Select-Object -Unique

But in OP Select-Object -Unique doesn't work as expected, in the example Select-Object -Unique *.
I'd want to get exact repro steps to understand that we investigate.

In mine it is there to show that it doesn't do what is expected too ,
$files = Get-ChildItem | Select-Object Name    # Gets something
$files  | Select-Object  -unique               # Only outputs the first item not expected
$files  | Select-Object  -unique *             # Outputs all items

What is a result we expect for [pscustomobject] @{ foo=1 } -eq [pscustomobject] @{ foo=1 }?
For [System.Management.Automation.LanguagePrimitives]::Compare([pscustomobject] @{ foo=1 }, [pscustomobject] @{ foo=1 })?

I would obviously expect value semantics, so those should compare equal. Otherwise we run into issues when putting them into associative containers. As they are currently compared by reference you can never retrieve a key without iteration. Also this means that two objects converted from the exact same JSON or CSV are not equal to each other.
But then, PowerShell has broken comparison semantics for primitive containers anyways. @{ 'foo'=1 } -eq @{ 'foo'=1 } also fails, while @( 1, 2, 3 ) - eq @( 1, 2, 3 ) returns null because it is stupid: 3 - eq 1..3 is false, while 1..3 -eq 3 is 3.

I think pscustomobject equality and comparison semantics is a key here. We need to describe then exactly and also we need to investigate whether it is possible to change this without breaking other PSObjects.

PowerShell has broken comparison semantics for primitive containers anyways. @{ 'foo'=1 } -eq @{ 'foo'=1 } also fails, while @( 1, 2, 3 ) - eq @( 1, 2, 3 ) returns null because it is stupid: 3 -eq 1..3 is false, while 1..3 -eq 3 is 3.

I explained this somewhere else like this .

9 -lt 10  is true:  9 is less than 10
11 -lt 10  is false:  11 is Greater than 10
so what does @(11,9) -lt 10 return. True? Because something the array IS less than 10. Or False? because something IS NOT less than 10. Or True, and False for the two members ? And you and I both know what it does do, it returns a subarray with the items that meet the condition.

@iSazonov  I said above the problem with this case is note properties being ignored in select and sort.
I wonder if many objects (e.g. directoryInfo) dont work with the -eq because their type doesn't have a CompareTo method.

 Incomparable types were compared by typename instead. So all integers were smaller than all strings, which was really funny when you forgot a cast somewhere. I spent a week debugging issue whose root cause was upper_limit parameter being left as a string, meaning it never discarded elements.
Arrays using comparison operators for selection instead of comparison was such a bad idea, the very fact that a comparison operator can return anything other than bool. For selection, you can just use the Where-Object or, if you really wanted to have an operator for that, create a separate one like ?lt. I'm half tempted to write a proposal to have comparison operators throw a StrictMode error with arrays, to force everyone to use Where-Object or Compare-Object.

$输出数组 = @()
$a = [PSCustomObject]@{
name = '张三'
age = 18
}
$输出数组 += $a
$a = [PSCustomObject]@{
name = '李四'
age = 18
}
$输出数组 += $a
$输出数组 | format-table
$输出数组 | Get-Unique
Get-Unique -InputObject $输出数组

this is by design ?
both powershell v5.1 and powershell v7.3 return sam
Get-Unique specifying Property is not supported,
xxx |  Get-Unique
Get-Unique -InputObject
These two usages return different results, which is easy to be troubled by.

@SeeminglyScience What do you think about the issue? What about adding IEquatable in PSCustomObject and IComparable if PSCustomObject is [ordered]?

so what does @(11,9) -lt 10 return

It should return an error

That's one of multiple valid views.
There is problem which I was taught about in my CS degree in the mid 1980s where it was called "overloaded operators".
In some languages = is both the test for equality and the assignment operator (others use := or == to split them up)

is both arithmetic addition and concatenation, really it should not be.
In PowerShell % can be alias for ForEach-Object or the modulo operator, and ? and be a wildcard, the alias for where-object or the ternary operator.   "." can be "run in current scope" or the member operator.

One way to think about -eq etc is as overloaded operators which do different jobs in different contexts.  I'm a lot more bothered that (get-item .) -eq (get-item .)  returns false than I am that the original design making it a filter when the first operand is an array - that does have ring of "quick and dirty, but still useful" to it.

@jhoneill the problem is not overloading, it's unexpected overloading. When you're implementing a list, having a plus do element-wise addition is a valid implementation (like in Matlab), as is it doing contatenation (like in Python). But there are certain contracts that should be obeyed, especially for comparison operators, and PowerShell arrays returning anything but booleans when using comparison is flounting those rules.:

broken typing: $x -eq $y may return anything depending on whether $x is an array and what types it holds, though at least it always returns an array without unwrapping single elements. Here's a fun one: play around with @( $false )
broken symmetry: 1 -eq @( 1, 2 ) returns false, but @( 1, 2 ) -eq 1 returns false
broken identity: $x -eq $x always return null for arrays, because it means $x | Where-Object { $_ -eq $x }, and items will never be equal entire array, especially when the array is on left side. This is even true for single-element and empty arrays and I don't think you can ever construct an array where it will return anything other than null without cheesing it with a custom type.

I'm a lot more bothered that (get-item .) -eq (get-item .) returns false

This stems directly from broken identity. So we cannot fix it without changing how equality operators behave on arrays, which would be a major breaking change. The only thing I can think of is a StrictMode warning forcing use of Compare-Object, or going the way of Javascript and introducing a new operator like !eq which means "compare, but really compare". At least == in JS always returns a bool, so PS somehow managed to break its comparison even more.

Just on (2) -eq isn't symmetrical.
$false -eq  ""  returns true because the Boolean for "" converts to $false. but  "" -eq $false  is false  because $false converts to the string "false".
So just when the novice user says "aha so the type one right is converted to match the one the right so 3 -eq [math]::pi will convert PI to an [int] " we find that the rule doesn't apply when comparing numbers.
Or try this
$x = 1L
$y = 1s

 $x -eq $y

It's true even though they store 1 as long int and short int.
We also know that this  $hash = @{1 = "Long"; 1 = "Short"} will give a duplicate keys error
 $hash = @{$x = "Long"; $y = "Short"}
 $hash

Name                           Value
----                           -----
1                              Long
1                              Short

$hash[1]
# nothing

So a hash table allows two keys which test as equal, but if we provide a key which is equal to both it doesn't match either of them.  It also works for char and string but at least "A" matches the string later. What different things consider to be "equal" is not consistent.

Can we circle back to @jhoneill 's original solution to simply default -Property to *? It fixes the issue and I cannot see any flows it could break. If nobody can think of a reason that would be a breaking change, I suggest we go with it, as it is trivial to implement.

This cmdlet is so complex that any workaround is bound to break something. The solution must be fundamental to be approved. But first we need the approval of the WG as a whole.

@PowerShell/wg-powershell-cmdlets reviewed this and agree that the current behavior for PSCustomObject is incorrect.  We believe this is likely a bucket 3 breaking change that users are relying on the current behavior.  We considered an option to inform the user to use -Property * if we detect that the objects are PSCustomObjects, however, it seems more useful to fix this behavior that future users can rely upon.  So the change is only if the first object is a PSCustomObject and -Property isn't specified, then it gets set to -Property *.

@SteveL-MSFT Great to hear! So the fix should be something like this?
if (object is PSCustomObject && Properties.IsNullOrEmpty())
{
    Properties = "*";
}

No warning after all? Just to confirm before I sit down to implement it.

@dkaszews No warning. Check only first object being PSCustomObject.

@SteveL-MSFT @iSazonov From code review: how should we handle Sort-Object? Should the change of default Property = "*" still be triggered based on first element? For Select we don't have much choice, as we cannot inspect extra elements without breaking streaming, but for Sort, the output is deferred anyway and just looking at first object makes it order dependent, which should never happen for a sorting function:
> $dict = @{ a = 0 }
> $obj = [PSCustomObject] @{ a = 1 }
> $obj2 = [PSCustomObject] @{ a = 2 }
> $obj, $obj2, $dict | Sort-Object -Unique
a: 0
a: 1
a: 2
> $dict, $obj, $obj2 | Sort-Object -Unique  # fix doesn't kick in
a: 0
a: 1

So we should probably go with Any or All instead - but which? I can't find a good argument for one over another other than the first will be quicker to evaluate for cases with PSCustomObject in the objects, while the latter will be quicker for cases without. I think the Any will be safer option to make the fix more prone to kick in than not.
This also circles back to the Select - if you have a list of just PSCustomObject but prepend it with some other type, perhaps a string or something else (maybe your parser is inconsistent with nulls), then the fix won't kick in. Changing the Property mid run is a no-go, but maybe a warning in this case? So if you are not at first element but it's PSCustomObject and Property is still default with the -Unique, issue a single warning.

Looking at the results of the proposed fix, I am not satisfied. While the main scenario is fixed by this fix (usually all objects of the same type in the pipeline), I'm afraid that we'll immediately get feedback that it doesn't work if the first object isn't a PSCustomObject. Since we're doing a slow accumulation of objects from the pipeline anyway, there's nothing stopping us from checking their type until we encounter a PSCustomObject.
This in turn makes me think that this fix is more of a workaround and the main problem is somewhere deeper and should be fixed, although it's not trivial anymore.

@iSazonov I think the problem is in the design of the cmdlets that the unspecified property ignores NoteProperty for reasons I don't understand. The "proper" fix in my opinion would be to always take them into account, equivalent to setting default to star, regardless of the type of the objects.

ignores NoteProperty for reasons I don't understand

Performance, I guess.

@iSazonov You say "performance", I say give me the benchmarks 😉 . I had a programming 101 professor who would dock point for writing loops in C as for (size_t i = 0; i < n; i++) { foo(a[i]; } because he thought for (int* p = array, end = array + n; p != end; ++p) { foo(*p); } would be more "performant" even when I showed him they compile to exact same assembly.
Is there an easy way to enumerate all types currently loaded and list their properties of each kind? That would allow to see if there might be some reasonably common case where comparing NoteProperties will indeed introduce a performance hit, then construct a test and get some real numbers.

Really PSObject does lazy initialization internal property collections. So we have no need a benchmark - it is obvious slowly.
To be precise, I am not against the suggested correction at all, since this cmdlet is already full of contradictions. However, I would rather follow the fundamental concepts of PowerShell than mask problems.

Looking at the results of the proposed fix, I am not satisfied. While the main scenario is fixed by this fix (usually all objects of the same type in the pipeline), I'm afraid that we'll immediately get feedback that it doesn't work if the first object isn't a PSCustomObject.

There are many cases where if the first object is of one type and the later objects have different properties you don't get all the fields. (Formatting being the most obvious), another one is not a problem IMHO.
Fixing the problem when the objects are all similar is a net gain, not knowing what to do when they are dissimilar is a problem which doesn't have an easy or obvious solution.

@iSazonov I think the problem is in the design of the cmdlets that the unspecified property ignores NoteProperty for reasons I don't understand. The "proper" fix in my opinion would be to always take them into account, equivalent to setting default to star, regardless of the type of the objects.

It's not that it ignores a particular type of property, but instead that equality (and comparison) defaults to reference equality when the target type does not implement it. Defaulting to * doesn't really solve anything unless you also make it recursive.
It would also be a breaking change as some objects only have a single instance per identity. System.Type for example, if you try to compare all of it's properties you'll run into a lot of issues, but otherwise [type] -eq [type] works because the ref identity is the same.

@SeeminglyScience My main perplexity is that PSCuscomObject behaves unpredictably in many cases, and yet is one of the key concepts. Although it appeared as a way to overcome the limitations of .Net in the absence of flexible dynamic.
I think it would be much easier for everyone to understand and work with PowerShell if there were simple rules and they were enforced.

I can sympathize with that, but as with many things in PowerShell, the amount of freedom afforded does limit how consistent some features can be. For example, some selective equality and comparability can be implemented but it can't work in dictionaries due to mutability. We also can't implicitly recurse an object to compare it so at best we'd be solving some of the problems and making the rest harder to troubleshoot. Maybe that middle ground is acceptable, but I'm not sure a simple set of rules that are inherently understandable is realistic.
#15806 should be fixed for sure though.

I wonder if it's worth discussing a new operator that explicitly attempts structural comparison.

I temporarily closed the proposed fix because checking the just first object resulted in some odd results, if you had a mix of types. While I believe heterogenous collections are a Bad Thing™ on their own, they should be considered. In particular, this could cause some odd behavior where changing Sort-Object | Select-Object -Unique to Sort-Object -Unique changed the behavior.
Simple solution would be to produce warning/error when the -Property is changed from default to * because the first object is PSCustomObject but later we see and object which is not.
Alternatively, I would like to push for a more elegant solution I already proposed before, which causes PSCustomObjects to always compare structurally as with -Property * but does not depend on the order:

look at all NoteProperty if and only if the object contains no Property. Not much more complicated and even less likely to break existing flows in case there are some objects which store their real values as Property and use NoteProperty only for metadata.

@doctordns Could you please discuss this at the next Cmdlet workgroup meeting and decide on the new direction?


[lbrynet-daemon latest] Modernize latest binary container.

Needed to get everything up to speed with Chainquery and Lbrycrd

Completed


Make fetch operation more robust

[x] Ensure we do not duplicate data in database if fetch is run multiple times
[x] Make fetch an option to the run command so that the server can listen and fetch at the same time (make this play nice with the above feature)

Implemented very simple semantics to keep things simple.

If you run a fetch with an empty database it will create the database and fetch everything
If a database exists, it will fetch anything newer than the last entry added to the database


Getting distorted audio playback from stream

Hi all, I am quite new to audio processing and I am working on a project that another dev created that is using sounddevice to stream some audio. We want the latency to be quite minimal between when the audio is generated and when it is streamed. However, it seems like the current method is distorting the audio playback, because if we save the audio instead of streaming it the audio locally it seems to be much better quality
We believe this is due to how we are copying the buffer in the callback function but I am not sure what is being done wrong. I've read the docs and I tried copying the approach in this example https://python-sounddevice.readthedocs.io/en/0.3.15/_downloads/47cc99d1f4d4b5d914d6a05da8e75553/play_long_file.py from the docs instead but I was unable to get it to work. Any insight would be appreciated, thank you!!
python3 -m sounddevice
  0 pulse, ALSA (32 in, 32 out)
  1 default, ALSA (32 in, 32 out)

 >>> import sounddevice as sd
sd.get_portaudio_version())>>> print(sd._libname)
libportaudio.so.2
>>> print(sd.get_portaudio_version())
(1246720, 'PortAudio V19.6.0-devel, revision 396fe4b6699ae929d3a685b3ef8a7e97396139a4')

import time
import scipy.io.wavfile
import requests
import sounddevice as sd
import numpy as np

def run(app: App, input_queue: queue.Queue):

    log = logging.getLogger("tts")
    buffer = []

    def buffer_duration():
        rv = 0.0
        for b in buffer:
            rv += len(b) / 24000
        return rv

    def callback(outdata, frames, time, status):
        # audio frame callback, play from buffer or silence if buffer is empty
        if len(buffer) == 0:
            outdata[:] = np.zeros((frames, 1))
            return
        # fill up outdata with samples in buffer array
        while len(buffer) > 0 and frames > 0:
            # if we have enough frames in the buffer chunk, fill outdata with and truncate the buffer chunk
            if len(buffer[0]) > frames:
                outdata[:frames] = buffer[0][:frames]
                buffer[0] = buffer[0][frames:]
                frames = 0
            # if we don't have enough frames in the buffer chunk, fill outdata with the buffer chunk and remove it from the buffer
            else:
                outdata[: len(buffer[0])] = buffer[0]
                outdata = outdata[len(buffer[0]) :]
                frames -= len(buffer[0])
                buffer.pop(0)
        # if we still have frames left to fill, fill them with silence
        if frames > 0:
            outdata[:frames] = np.zeros((frames, 1))

    def generate(text: str, voice_settings: dict):
        data = get_audio()
        sample_rate, samples = scipy.io.wavfile.read(io.BytesIO(data))
        assert sample_rate == 24000
        assert len(samples.shape) == 1
        samples = samples.reshape((len(samples), 1))
        buffer.append(samples)

    stream = sd.OutputStream(
        channels=1, callback=callback, samplerate=24000, blocksize=2000,
    )
    stream.start()

    text_buffer = []
    stop = False

    while True:
        try:
            item = input_queue.get(timeout=0.1)
            if item is None:
                log.info("Got stop signal.")
                stop = True
            else:
                text_buffer.append(item)
        except queue.Empty:
            pass

        dur = buffer_duration()
        app.is_speaking = dur > 0.5

        if len(text_buffer) > 0 and (dur < 4 or stop):
            # get all text buffer chunks as long as the settings are the same
            text = ""
            voice_settings = text_buffer[0][1]
            while len(text_buffer) > 0 and text_buffer[0][1] == voice_settings:
                text += text_buffer.pop(0)[0].strip() + " "
            generate(text.strip(), voice_settings)

        if stop:
            # wait for buffer to be empty
            while len(buffer) > 0:
                time.sleep(0.1)
            stream.stop()
            break

Switched to using RawOutputStream and this fixed the issue


Update gh.js - updated quitePeriodSeconds from 30 to 300 as this short time always fails to wait for Instance running state

isn't "timeoutMinutes = 5" the real timeout? what you try to update is the interval between attempts or something. did this work for you, to fix timeouts on registration? (i have no timeouts in current version)


Usable with GDScript

Title says all, it is possible to get a usable version for GDScript?
Thanks

I haven't tried it, but someone else made a C++ module for this for Godot 3.x:
https://github.com/Blackdrop-Interactive-AB/godot-dear-imgui/
Making GDScript bindings currently requires a C++ module or GDExtension, which I'm not doing because it's more complicated for everyone to use.
However, C# GDExtension support is planned for Godot 4:

One of the big changes that we are still working on is support for writing GDExtensions in C#. With GDExtension, C# classes will be registered in the engine and work as the built-in classes do, which should improve the support of C# nodes and resources throughout the engine.

With that, I think I could make a good GDScript interface. Though it's not simple, because GDScript doesn't really have pointer/reference types for parameters, which the Dear ImGui API uses a lot.
So I'm leaving this issue open, but I have to wait for that. Not sure if that will be in Godot 4.0, or 4.1 or later.

Implemented in v5.0.0


browser.mock executed too late

Describe the bug -> This is not really a bug but a question regarding wdi5 and WebdriverIO internals
In my local free-form UI5 test application I have a JSON model in my manifest.json configured like this:
  "sap.app": {
    "dataSources": {
      "myjsonSource": {
          "type": "JSON",
          "uri": "http://<IP_ADDRESS>/myjsonapi/"
      }
  }

When I test this application using wdi5, I want to mock the call to myjsonapi using the standard WebdriverIO method documented here, so in mytest.e2e.js I have this:
describe('On the main page', async () => {
    before( 'test something', async () => {

        const mock = await browser.mock('http://<IP_ADDRESS>/myjsonapi/');
        mock.respond({
            title: 'This is mocked data',
            completed: false
        })

    });
});

The problem is that the await browser.mock call is executed too late. By the time the test gets to this call, UI5 has already made the call to myjsonapi, therefore, my application will be executed with the production data and not the mocked data.
I'm not sure if it matters, but in wdio.conf.js I don't have a url parameter in the wdi5 object and baseUrl is set to http://<IP_ADDRESS>/myui5app/webapp/ where I have my index.html file.
Expected behavior
I would like to know where shall I put my call to await browser.mock so that it is executed before UI5 calls myjsonapi. This would have the effect that when UI5 actually makes the call WebdriverIO would supply the mocked response to my application.
I'm not a WebdriverIO expert but I guess the question boils down to how wdi5 starts the UI5 application. This is not explained in the wdi5 docs. It seems to me that in WebdriverIO you have to specifically call browser.url() to start your web application but wdi5 seems to navigate to baseUrl automatically.
Runtime Env (please complete the following information):

wdi5/wdio-ui5-service-version: 1.3.1
UI5 version: 1.107.1
wdio-version (output of wdio --version): 7.26.0
node-version (output of node --version): 18.13.0
OS: Linux Kubuntu
Chrome 111

hi, what you're experiencing is most likely due to the preload behavior of models in UI5. We're only able to inject the UI5 Test API at wdi5 runtime after the UI5 bootstrap has happened - eventually including the call to preload any models.
Have you tried setting preload: false in the manifest.json's model definition (https://ui5.sap.com/sdk/#/topic/26ba6a5c1e5c417f8b21cce1411dba2c)?

I previously did not have the preload setting in my manifest.json so I added it like this:
  "sap.ui5": {
    "models": {
      "myjsonModel": {
        "type": "sap.ui.model.json.JSONModel",
        "dataSource": "myjsonSource",
        "preload": false
      }
    }
  }

but the model is still preloaded.
My application starts with a basic page which does not use the model. I also checked and I don't have any JS code which uses the model during application startup.
The strange thing is that the documentation you referenced says that the default is actually preload: false, so preload should not have happened even before I added this setting.
In Chrome Dev Tools I took a screenshot of the "Initiator" tab of the network request to the JSON source. It seems that the model is preloaded without touching my code at all.
Do you have any other hints how to make preload: false work? Alternatively, shall I maybe try using skipInjectUI5OnStart? So far I tried to avoid skipInjectUI5OnStart because I hoped there is a cleaner solution (since my app is pure UI5), but maybe I have no other choice.

This runs successful.
I guess since the model is configured in manifest.json the init method runs quite before the test execution -> use the before hook to create your mock data and/ or use the late inject.
in wdio-ui5-late.conf.ts
baseConfig.before = async () => {
    const mock = await browser.mock("http://localhost:8080/V2/Northwind/Northwind.svc/Customers('TRAIH')")
    mock.respond({
        d: {
            __metadata: {
                uri: "https://services.odata.org/V2/Northwind/Northwind.svc/Customers('TRAIH')",
                type: "NorthwindModel.Customer"
            },
            CustomerID: "TRAIH",
            CompanyName: "Trail's Head Gourmet Provisioners",
            ContactName: "This is mocked data",
            ContactTitle: "Sales Associate",
            Address: "722 DaVinci Blvd.",
            City: "Kirkland",
            Region: "WA",
            PostalCode: "98034",
            Country: "USA",
            Phone: "(206) 555-8257",
            Fax: "(206) 555-2174",
            Orders: {
                __deferred: {
                    uri: "https://services.odata.org/V2/Northwind/Northwind.svc/Customers('TRAIH')/Orders"
                }
            },
            CustomerDemographics: {
                __deferred: {
                    uri: "https://services.odata.org/V2/Northwind/Northwind.svc/Customers('TRAIH')/CustomerDemographics"
                }
            }
        }
    })
}

in ui5-late.test.js
    it.only("wdi5 should work regularly with mocked data", async () => {
        // native wdio functionality - navigates to the wdi5 github page
        await browser.$("#user-content-wdi5-").waitForDisplayed()
        // open local app
        await browser.url("http://localhost:8080/index.html")
        // do the late injection
        await wdi5.injectUI5()

        const inputText: wdi5Selector = {
            selector: {
                id: "mainUserInput",
                viewName: "test.Sample.tsapp.view.Main"
            }
        }

        const webcomponentValue = await (browser.asControl(inputText) as unknown as Input)

        expect(await webcomponentValue.getValue()).toEqual("This is mocked data")
    })

Happy to if you can work out and contribute a best practice recipe using the browser.mock funtionality.

With your suggestions I was able to make mocking work. However, I find this method quite complicated.

Do I really have to declare my mocked data at the config-file level?
Does this mean that if I want to use different mocked data for different tests, I'm forced to create many config-files, because I'm unable to declare the data at the test level?

So far, my idea was that to test one application I will create one high-level config-file. Then I will test different behaviors in different describe-it tests, using the same config-file. But this seems to be impossible with mocked data.
I admit, my hope with creating this issue was that you might have insider information about the initialization process of UI5 (or have colleagues whom you can ask) and you can come up with a finely-targeted point of time where the mocked data can be injected, without messing up the test by introducing per-test config-files, late-injecting UI5, waiting for an #id on an external page (so now the test depends on the correct #id), etc.

Hi @mocsa,
why don't you use the help of the mockserver? With the V2 MockServer you also have the possibility to intercept simple REST calls and return data as you like.

I haven't thought of that. I tried to remain within the realms of WebdriverIO and wdi5, and not adding another dependency unless absolutely necessary. Since WebdriverIO already has mocking capabilities, I tried to make that work. Also, I have zero experience with MockServer.
I guess, you mean sap.ui.core.util.MockServer. I tried to briefly check the documentation but I could only find examples for mocking OData, however I'm using JSON.
Also, is it easy in MockServer to have different mocked data for different tests?
Could you perhaps give a brief overview of what settings I need to put into what files to make your suggestion work?

for most of your questions I would need a look at your setup and use-case to give you concrete guidance here.
For me it is also unclear if you want to write integration tests or real e2e tests? Maybe you can have a look at this blog post
We are soon able to offer sponsored support where we then could explain you everything in more detail but for now I would close this issue.

Sorry, Simon, but I'm not getting much help from your comments. Why have you closed this issue? It is not solved. I also don't know what you mean by "sponsored support". Is that paid support? It sounds a bit out-of-place in an open source project. I'm just helping this project in my free time for no money. But sorry if I misunderstood you.
Anyways...
After implementing @dominikfeininger's suggestion I tried to bring it closer to my original expectation and I partly succeeded. I'm putting it here for the benefit of others.
I wanted to have different mock responses for each of my tests, so I changed @dominikfeininger's above code by moving the setup-code of the mock response from the wdio.conf.js file to the ui5.test.js file and it actually worked. Basically, I had to implement the late injection of the wdi5 framework and this allowed me to set up the mock response before UI5 loaded the data of my model configured in manifest.json.
I did the following:

In my wdio.conf.js I changed baseUrl from the index page of my UI5 app to an arbitrary other webpage which does not start my UI5 application.

I added skipInjectUI5OnStart=true in wdio.conf.js.

In my test.e2e.js file I did something like this:

const { default: _ui5Service } = require("wdio-ui5-service")
const ui5Service = new _ui5Service()
/* Other require statements */

describe('On the subpage', async () => {
    before( 'when something is done', async () => {
        const mock = await browser.mock('http://<IP_ADDRESS>/myjsonapi/');
        mock.respond({
            title: 'This is mocked data',
            completed: false
        })
    // The following line should wait for an element on the page in your baseUrl
    await browser.$("#wrapper").waitForDisplayed()
    // open your UI5 app
    await browser.url("http://<IP_ADDRESS>/myui5app/webapp/index.html")
    // late inject the wdi5 framework into your app
    await ui5Service.injectUI5()

    });
    it('ordering of table should work', async () => {

        // Do your test here, interact with your UI5 app

        // assert your test result
        expect(value1 <= value2).toBeTruthy

    });
});


Dev hybridcompute microsoft.hybrid compute 2022 12 27 preview

ARM API Information (Control Plane)
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Azure 1st Party Service can try out the Shift Left experience to initiate API design review from ADO code repo.  If you are interested, may request engineering support by filling in with the form https://aka.ms/ShiftLeftSupportForm.
Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[x] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month.
By default, Azure SDKs of all languages (.NET/Python/Java/JavaScript for both management-plane SDK and data-plane SDK, Go for management-plane SDK only ) MUST be refreshed with/after swagger of new version is published. If you prefer NOT to refresh any specific SDK language upon swagger updates in the current PR, please leave details with justification here.

Contribution checklist (MS Employees Only):

[x] I commit to follow the Breaking Change Policy of "no breaking changes"
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[x] Check this box if any of the following apply to the PR so that the label "ARMReview" and "WaitForARMFeedback" will be added by bot to kick off ARM API Review.  Missing to check this box in the following scenario may result in delays to the ARM manifest review and deployment.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you copy the existing version into the new directory structure for first commit and then push new changes, including version updates, in separate commits. You can use OpenAPIHub to initialize the PR for adding a new version. For more details refer to the wiki.

[ ] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If you have any breaking changes as defined in the Breaking Change Policy, request approval from the Breaking Change Review Board.
Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Additional details on the process and office hours are on the Breaking Change Wiki.
NOTE: To update API(s) in public preview for over 1 year (refer to Retirement of Previews)
Please follow the link to find more details on PR review process.

@shubhammalhotra28  - Can you please review and fix failed checks such as PrettierCheck, Lintdiff etc

FYI those API are intentionally deleted

Signed off from ARM side with minor comment

@shubhammalhotra28 Please fix the Avocado errors, details can be found via https://github.com/Azure/azure-rest-api-specs/pull/22034#issuecomment-1370250196

/azp run

/azp run

@tadelesh Could you help to review the breaking change of go? Thanks!

@qiaozha Could you help to review the breaking change of JS? Thanks!

/azp run


Thoughts on reorganizing BEIR main landing page

tl;dr - IMO the current README landing page tries to do too much... I think we should trim down.
For example, why do we have these?

https://github.com/beir-cellar/beir#beers-course-material-on-ir
https://github.com/beir-cellar/beir#beers-multilingual-datasets

People come to the page for the dataset itself? So these should come first:

https://github.com/beir-cellar/beir#beers-quick-example
https://github.com/beir-cellar/beir#beers-available-datasets

All the examples and baselines should come later?
Thoughts @NThakur20 ?

Hi @lintool,
Sounds good to me. I think we can reposition the quick example and available datasets on top and push course material and multilingual datasets further down on the landing page.
I kept the course material on the IR page, in case someone is not familiar with retrieval or dense retrieval, they can refer to these course materials available online. Similarly, BEIR can also support the evaluation of multilingual datasets, that's why we have a multilingual datasets section.
What do you think @nreimers?
Kind Regards,
Nandan Thakur

The README has been organized accordingly in the latest BEIR version v1.0.1.
I am closing the issue.
Kind Regards,
Nandan Thakur


Closes #34  |  Create dataset loader for MKQA

Please name your PR after the issue it closes. You can use the following line: "Closes #ISSUE-NUMBER" where you replace the ISSUE-NUMBER with the one corresponding to your dataset.
Checkbox

[x] Confirm that this PR is linked to the dataset issue.
[x] Create the dataloader script seacrowd/sea_datasets/my_dataset/my_dataset.py (please use only lowercase and underscore for dataset naming).
[x] Provide values for the _CITATION, _DATASETNAME, _DESCRIPTION, _HOMEPAGE, _LICENSE, _URLs, _SUPPORTED_TASKS, _SOURCE_VERSION, and _SEACROWD_VERSION variables.
[x] Implement _info(), _split_generators() and _generate_examples() in dataloader script.
[x] Make sure that the BUILDER_CONFIGS class attribute is a list with at least one SEACrowdConfig for the source schema and one for a seacrowd schema.
[x] Confirm dataloader script works with datasets.load_dataset function.
[x] Confirm that your dataloader script passes the test suite run with python -m tests.test_seacrowd seacrowd/sea_datasets/<my_dataset>/<my_dataset>.py.
[ ] If my dataset is local, I have provided an output of the unit-tests in the PR (please copy paste). This is OPTIONAL for public datasets, as we can test these without access to the data files.

Hi @fhudi, I'm revising the PR init message as follows:

Please name your PR after the issue it closes. You can use the following line: "Closes #ISSUE-NUMBER" where you replace the ISSUE-NUMBER with the one corresponding to your dataset.

to

Closes #34.

@sabilmakbar
Do note that the edit is recorded automatically, i.e. you do not need to explicitly detailing in comment.

@jamesjaya Please don't forget to review this ticket.

Hi @fhudi @jen-santoso @jamesjaya, let's only include SEA indigenous and non-indigenous languages stated in the issue ticket for both source* and seacrowd schemas.
If you find an error in the language set (e.g., if there are some SEA languages forgotten to be included or there are some non-SEA languages included), please let @SamuelCahyawijaya and me know.

source schema is used to adapt the original data format, not the entire set of the supported languages.

Thanks for the quick response @fhudi! LGTM
@jamesjaya please review

Changing the reviewer from @jamesjaya to @holylovenia for faster responses.

https://github.com/SEACrowd/seacrowd-datahub/pull/177#pullrequestreview-1871811766

Sorry for this error.
commit #bc1f606 should have fixed it.

Hi @fhudi, I encountered an error when trying to run python -m tests.test_seacrowd seacrowd/sea_datasets/mkqa/mkqa.py.
Here is the full log of the error:
INFO:__main__:args: Namespace(path='seacrowd/sea_datasets/mkqa/mkqa.py', schema=None, subset_id=None, data_dir=None, use_auth_token=None)
INFO:__main__:self.PATH: seacrowd/sea_datasets/mkqa/mkqa.py
INFO:__main__:self.SUBSET_ID: mkqa
INFO:__main__:self.SCHEMA: None
INFO:__main__:self.DATA_DIR: None
INFO:__main__:Checking for _SUPPORTED_TASKS ...
module seacrowd.sea_datasets.mkqa.mkqa
INFO:__main__:Found _SUPPORTED_TASKS=[<Tasks.QUESTION_ANSWERING: 'QA'>]
INFO:__main__:_SUPPORTED_TASKS implies _MAPPED_SCHEMAS={'QA'}
INFO:__main__:schemas_to_check: {'QA'}
INFO:__main__:Checking load_dataset with config name mkqa_source
Downloading and preparing dataset mkqa/mkqa_source to /Users/holylovenia/.cache/huggingface/datasets/mkqa/mkqa_source/1.0.0/e340f291853bc02c788449e1e2007f5c356bcb21b224218e0729650d660fcb09...
E
======================================================================
ERROR: runTest (__main__.TestDataLoader.runTest)
Run all tests that check:
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/holylovenia/anaconda3/lib/python3.11/site-packages/datasets/builder.py", line 1625, in _prepare_split_single
    example = self.info.features.encode_example(record) if self.info.features is not None else record
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/holylovenia/anaconda3/lib/python3.11/site-packages/datasets/features/features.py", line 1824, in encode_example
    return encode_nested_example(self, example)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/holylovenia/anaconda3/lib/python3.11/site-packages/datasets/features/features.py", line 1223, in encode_nested_example
    {
  File "/Users/holylovenia/anaconda3/lib/python3.11/site-packages/datasets/features/features.py", line 1224, in <dictcomp>
    k: encode_nested_example(sub_schema, sub_obj, level=level + 1)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/holylovenia/anaconda3/lib/python3.11/site-packages/datasets/features/features.py", line 1223, in encode_nested_example
    {
  File "/Users/holylovenia/anaconda3/lib/python3.11/site-packages/datasets/features/features.py", line 1223, in <dictcomp>
    {
  File "/Users/holylovenia/anaconda3/lib/python3.11/site-packages/datasets/utils/py_utils.py", line 302, in zip_dict
    yield key, tuple(d[key] for d in dicts)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/holylovenia/anaconda3/lib/python3.11/site-packages/datasets/utils/py_utils.py", line 302, in <genexpr>
    yield key, tuple(d[key] for d in dicts)
                     ~^^^^^
KeyError: 'en'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/holylovenia/Projects/seacrowd-datahub/tests/test_seacrowd.py", line 134, in setUp
    self.dataset_source = datasets.load_dataset(
                          ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/holylovenia/anaconda3/lib/python3.11/site-packages/datasets/load.py", line 1797, in load_dataset
    builder_instance.download_and_prepare(
  File "/Users/holylovenia/anaconda3/lib/python3.11/site-packages/datasets/builder.py", line 890, in download_and_prepare
    self._download_and_prepare(
  File "/Users/holylovenia/anaconda3/lib/python3.11/site-packages/datasets/builder.py", line 1649, in _download_and_prepare
    super()._download_and_prepare(
  File "/Users/holylovenia/anaconda3/lib/python3.11/site-packages/datasets/builder.py", line 985, in _download_and_prepare
    self._prepare_split(split_generator, **prepare_split_kwargs)
  File "/Users/holylovenia/anaconda3/lib/python3.11/site-packages/datasets/builder.py", line 1487, in _prepare_split
    for job_id, done, content in self._prepare_split_single(
  File "/Users/holylovenia/anaconda3/lib/python3.11/site-packages/datasets/builder.py", line 1644, in _prepare_split_single
    raise DatasetGenerationError("An error occurred while generating the dataset") from e
datasets.builder.DatasetGenerationError: An error occurred while generating the dataset

----------------------------------------------------------------------
Ran 1 test in 3.115s

FAILED (errors=1)

cc: @jen-santoso, did you not encounter this error?

Apparently, I didn't encounter this, or I might have accidentally skipped a test (in this case, I apologize 🙏). lemme check the newest commit again.

In what condition does a data instance have [] instead of None for a value of answer_aliases?

@holylovenia
I also noticed this pattern long time ago, IMHO both are interchangeable.
(Seems like it has same semantic of no spelling aliases for a known entity ID)
Nevertheless, shall we try to make an enquiry on this matter to the authors?

In what condition does a data instance have [] instead of None for a value of answer_aliases?

@holylovenia I also noticed this pattern long time ago, IMHO both are interchangeable. (Seems like they have the same meaning of no spelling aliases for a known entity ID) Nevertheless, shall we try to make an enquiry on this matter to the authors?

Hi @shayne-longpre, as the author/dataset owner, could you please help us understand the difference between [] and None for answer_aliases in MKQA? Thank you in advance!

In what condition does a data instance have [] instead of None for a value of answer_aliases?

@holylovenia I also noticed this pattern long time ago, IMHO both are interchangeable. (Seems like they have the same meaning of no spelling aliases for a known entity ID) Nevertheless, shall we try to make an enquiry on this matter to the authors?

Hi @shayne-longpre, as the author/dataset owner, could you please help us understand the difference between [] and None for answer_aliases in MKQA? Thank you in advance!

It's been a long time since I looked at this, but yes it looks like a minor parsing bug, and can be treated interchangeably. Thanks for working on this!

It's been a long time since I looked at this, but yes it looks like a minor parsing bug, and can be treated interchangeably. Thanks for working on this!

Noted, @shayne-longpre! Thanks for helping us.
@fhudi, could we convert the [] to None in the seacrowd schemas for consistency's sake?

It's been a long time since I looked at this, but yes it looks like a minor parsing bug, and can be treated interchangeably. Thanks for working on this!

Noted, @shayne-longpre! Thanks for helping us. @fhudi, could we convert the [] to None in the seacrowd schemas for consistency's sake?

@holylovenia Surebeans.
Confirming, just in case, you understand that both method have same semantics, right?
i.e.: print([] or None or "seacrowd") prints "seacrowd"

@holylovenia Surebeans. Confirming, just in case, you understand that both values have same semantics, right? i.e.: print([] or None or "seacrowd") prints "seacrowd"

Yes yes, that's why I said just for consistency's sake.

Yes yes, that's why I said just for consistency's sake.

@holylovenia Done. Please have a look 🙏


chore(main): release 1.0.0

:robot: I have created a release beep boop
1.0.0 (2024-07-05)
Features

add release action (d17af08)
add release action (cab926a)
add release action (9230e2e)
add release action (4707c27)
add release action (e19a6aa)
add release action (4cd6d07)
init (e0844d8)

This PR was generated with Release Please. See documentation.

:robot: Created releases:

v1.0.0
:sunflower:


Bug 1740035: Azure should create custom container

Azure should create a custom container with name provided by user
Also fixed up a few other parts of the Azure CreateStorage flow

@dmage ptal, comments addressed
I decided to go the route of adding a boolean for whether it was found or not in addition to the error

/retest
[INFO] Acquiring a lease ...
[INFO] Lease acquired!
[INFO] Leased resource: {"type":"aws-quota-slice","name":"b99b9549-e9f7-4d4d-997f-adcd06c3a234","state":"leased","owner":"ci-op-i0fc0kqf-12abb","lastupdate":"2019-08-29T15:52:27.085648238Z","userdata":{}}
[INFO] Sending heartbeats to retain the lease...
failed to send heartbeat for resource "b99b9549-e9f7-4d4d-997f-adcd06c3a234": Post http://boskos.ci/update?name=b99b9549-e9f7-4d4d-997f-adcd06c3a234&owner=ci-op-i0fc0kqf-12abb&state=leased: dial tcp: lookup boskos.ci on <IP_ADDRESS>:53: no such host
Another process exited
[INFO] Releasing the lease on resouce b99b9549-e9f7-4d4d-997f-adcd06c3a234...
failed to release resource "b99b9549-e9f7-4d4d-997f-adcd06c3a234": status 401 Unauthorized, statusCode 401 releasing b99b9549-e9f7-4d4d-997f-adcd06c3a234

/test e2e-azure-operator

@dmage @adambkaplan ptal
The e2e-azure-operator test failures look unrelated to this pull request.

/test e2e-azure

/retest

@dmage @adambkaplan ptal, updates have been made per your suggestions

/test e2e-azure e2e-azure-operator

/retest

/test e2e-azure e2e-azure-operator

/test e2e-azure

/test e2e-aws-upgrade

@dmage @adambkaplan bump

/approve

@adambkaplan ptal, comments have been addressed.