Custom Field Data Searching

Searching for Custom content data seems to fail.
For example:
Create a custom field for IP Addresses
Add data to that custom field on an IP address object.
Go to the Front Page and search for this data or part of this data in "Search IPs" field.
The resulting IP object will not be found.
Similarly in IP Addresses page, you can search for the Custom Field data by using the Custom Field Search box on the right side of the page.  But you cannot enter partial data in this field and get a result.  It has to be all data contained in that field.
Can the search functions be modified to include searchung custom fields from the front page and also partial data? (for example searching for "url1" when the data in the field may contain "URL1, URL2, URL3, ULR4" etc.

Similarly in IP Addresses page, you can search for the Custom Field data by using the Custom Field Search box on the right side of the page. But you cannot enter partial data in this field and get a result. It has to be all data contained in that field.

This should be resolved in - https://github.com/digitalocean/netbox/commit/583830c652c2b46c44df21ff0779dad6101932cc
However this request also wants global search to return Custom Fields in global search results (rather than the Custom Field Search under IP Address).
I think enough people want Global search to search through Custom Fields that we should open a new discussion on how to make this happen. It was discussed briefly in https://github.com/digitalocean/netbox/issues/1190#issuecomment-306029330
As such this issue can probably be closed now and further discussion to the topic of Searching Custom Fields can take place in issue 1190.

@t0i agreed. Closing this one out because #1190 has more discussion attached to it.


BUG - EXT - Keyboard navigation through GA returned list when filter is applied

Related Feature
Feature request: issue number
Description
Users are unable able to navigate through the returned GA list when a filter has been applied to the list.
It is possible when no filter is applied
When the clear filter link is selected /highlighted  the tab control doesn't give focus to the returned list it tabs out of the application.
Steps to Reproduce
Steps to reproduce the behaviour:

Open GA extension
Enter search criteria and apply a filter
Using the keyboard tab from search input field to the returned GA list

Expected results
User can tab to the returned list
Actual results
User cannot tab to the returned list when a filter has been applied.
Screenshots
If applicable, add screenshots to help explain the problem.
Version/Components/Environment
Add any other context about the problem here
OS:

[ ] Mac OS
[ ] Windows
[ ] Other

Root Cause Analysis
Problem
{describe the problem}
Fix
{describe the fix}
Why was it missed
{Some explanation why this issue might have been missed during normal development/testing cycle}
How can we avoid this
{if we don’t want to see this type of issues anymore what we should do to prevent}

@carwelsh fixed & merged


How to construct querybank probe matrix?

I have some questions about the algorithm of qb-norm described as follows.
The process described in paper is

Considering that in MSRVTT-1ka, |G| = 1000, N=5000, given a query, then the process will be (1000 x 5000) x (5000 x 1) = (1000 x 1). Does the |G| is the  number of videos in testing set and N is the number of text queries sampled from training set? If so, why the matrix with the shape of (5000 x 1) is obtained instead of (1000 x 1) when a query is given. Does it means the query video is used to calculated similarity with all querybank text?
How to construct the querybank? I follow the paper to random 5000 samples from training set, but the result is really bad and is unstable when the querybank is changed.
Could you provide the script of constructing query bank and computing querybank probe matrix?

Hi,

The code works with similarities matrices which in our case have the shape 'queries x videos'. So, the code expects the following:

sims_train_test_path : a path to a similarity matrix containing nr_queries_from_training x nr_videos_testing similarities (this is the transpose of P probe matrix described in the paper)
sims_test_path: a path to a similarity matrix containing nr_queries_from_testing x nr_videos_testing similarities
If you don't obtain the right shape of the matrix, please pay attention to the input matrix (it should be 'queries x videos', so maybe you should transpose it).

In our final experiments we used all the captions from the training set. In practice, for MSRVTT we observed that using 5000 queries we obtained good results, however better results are obtained by using the whole dataset (as observed in Fig 3a). We only used 5000 for some ablations, but we obtained the best results using all the queries.

Ioana

Thanks for you reply!
Another question is why the shape of the train_test matrix of msvd is (97200, 670)?
I count the number of queries in train set and the result is 48774.

Hi,
Thanks for pointing that out! The CE code outputs a (max_query_per_video*nr_videos)*nr_videos matrix. In the case of MSVD the max_query_per_video=81. The additional entries are not valid numbers and they should not be used. Due to the fact that we forgot to take that into account when extracting the training_testing matrix for TT-CE+ the numbers now increased for QB-Norm. Thanks again for pointing that out!
We updated the sims matrix (https://www.robots.ox.ac.uk/~vgg/research/teachtext/QB-Norm/msvd-sims.tar.gz) and also the results from the table.
Ioana

Thanks for your update! I will test it with the new similarity matrix.


[GHSA-f522-ffg8-j8r6] Regular Expression Denial of Service in is-my-json-valid

Updates

Details
Affected version range
Fixed version

Comments
This GHSA-ID corresponds to NSWG-ECO-76.
When I compare it with NSWG-ECO-76, the details, affected version range, and fixed version appear to be incorrect.

Hi @matsumokei, thanks for the PR, but it looks to me like the fix got merged into the 2.17.2 tag.
https://github.com/mafintosh/is-my-json-valid/pull/159/commits/767c6c0ee5e55c506ccd49141e7f31cb1910c47f
I can't find anything for version1.4.1 so perhaps that is incorrect, but where does 2.12.4 come from?

This may have been one that we corrected based on the evidence at hand. I can see how the description would lead you to your conclusion, but looking at the actual code change and the tags associated I think the description might be a typo. Perhaps it might be worth raising this with the node security working group?

I'll raise an issue with the node security working group.
When I have finished investigating the vulnerabilities of the is-my-valid-json and find something, I may send the PR.
I'll close the PR.
Thanks for discussing with me.

Happy to have the conversation and thank you again for effort :)
I'll go ahead and update this advisory to drop the 1.4.1 version as a fix given the lack of evidence and extend the >= 2.0.0, < 2.17.2 range to < 2.17.2.
Let me know how your investigation goes and if feel free to follow up in this thread if you feel like the results don't warrant a new PR. 👍


Assembly signing for .net core 3.x

when i use global tool for .net core 3.1 app:
get message:
Hiding strings...
Renaming:  fields...Parameters...Properties...Events...Methods...Types...Done.
Saving assemblies...
Unhandled Exception: System.PlatformNotSupportedException: Strong-name signing is not supported on this platform.
   at System.Reflection.StrongNameKeyPair.get_PublicKey()
   at Obfuscar.Obfuscator.SaveAssemblies(Boolean throwException) in F:\obfuscar\Obfuscar\Obfuscator.cs:line 242
   at Obfuscar.Obfuscator.RunRules() in F:\obfuscar\Obfuscar\Obfuscator.cs:line 137
   at Obfuscar.Program.Main(String[] args) in F:\obfuscar\GlobalTools\Program.cs:line 115

BTW, Whether .net core 3.1 is supported in PostBuildEvent mode  (use nuget packages)？
Hope to be able to clearly indicate the currently supported .net platform in the documentation.

You should avoid strong naming .NET Core assemblies. That exception is expected.
PostBuildEvent is fully supported even in SDK based projects.
Unless explicitly stated, you should assume this tool works for all cases (as Mono Cecil does).

Alright, thanks !


FFmpeg: use inline metrics

Writing raw data to file, then reading into internal metric function adds unnecessary runtime overhead.
Instead, we can have FFmpeg calculate the MD5,SSIM, or PSNR concurrently.

@Bin-CI can you verify this patch works on Windows, please?

@Bin-CI can you verify this patchset works on Windows, please?

OK, I will test it

@Bin-CI can you verify this patchset works on Windows, please?

OK, I will test it

test with windows TGL for d3d11 part, it can be worked on the windows


Fixes for the packageFile option returning a boolean, and an incorrect exit code being returned on exit.

Having the option set for just '-p, --packageFile' will only return a true/false response for the option, making the require throw an error.
Also, the hasErrors function returns a value rather than a boolean of the existence of an error, so the current bin would return 1 when there were no errors, and 0 when there were.

Fixed by https://github.com/zackehh/dep-validate/commit/d90eb8ba382501bbc64c9ece05ab62e22adbbc8b.


docs: add Jie Ding to the people list

add Jie Ding to the people list and upload his image
Signed-off-by: Jie Ding<EMAIL_ADDRESS>

Hi Jie. Do you have a better quality image? This one looks a bit pixelated: https://www.cncf.io/people/technical-oversight-committee/?p=jie-ding

Hi Jie. Do you have a better quality image? This one looks a bit pixelated: https://www.cncf.io/people/technical-oversight-committee/?p=jie-ding

Yep, I'll reload another one now


[PE-D][Tester D] UG Inaccurate ScreenShot for mailtg

The telegram link was not shown. Given that the purpose of this image "...shows the email template containing the group telegram link generated.", I have marked this bug as medium severity.

Labels: type.DocumentationBug severity.Medium
original: tanjieling0/ped#10

completed


Item: Dark Matter Pickaxe

A Dark Matter Pickaxe is a pickaxe made of Dark Matter

Textures merged in https://github.com/pahimar/Equivalent-Exchange/pull/54


vimPlugins: add friendly-snippets

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[x] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 121006 at 4309b387 run on aarch64-linux 1

4 packages built successfully:

spacevim
vimPlugins.telescope-fzy-native-nvim
vimPlugins.vim-clap
vimPlugins.vim-hexokinase

2 suggestions:

warning: missing-phase-hooks
configurePhase should probably contain runHook preConfigure and runHook postConfigure.
Near pkgs/misc/vim-plugins/build-vim-plugin.nix:28:14:
   |
28 |       inherit unpackPhase configurePhase buildPhase addonInfo preInstall postInstall;
   |              ^

warning: missing-phase-hooks
buildPhase should probably contain runHook preBuild and runHook postBuild.
Near pkgs/misc/vim-plugins/build-vim-plugin.nix:28:14:
   |
28 |       inherit unpackPhase configurePhase buildPhase addonInfo preInstall postInstall;
   |              ^


Combining bold and italics like this won't compile

It's worth noting _ **text** _ works

Where is the compiler?


Update ZAP to fix attribute default endian-ness.

The default values were broken for little-endian.
Problem
Default attribute values are output as big-endian even on little-endian platforms.
Change overview
Switch the order for little-endian.
Testing
Manual examination of the generated defaults.  More tests will be created as people start using nonzero defaults.
Fixes https://github.com/project-chip/connectedhomeip/issues/11413

@jmeg-sfy

/rebase


:no_good: (EARLY, DNM, UNFIN) Initial Implementation

It might be cool to simplify some of the logic by leaning on typing_inspect a bit - https://github.com/ilevkivskyi/typing_inspect

Use pytype as a reference: https://github.com/Stewori/pytypes/blob/3846f1be3e44892c8e60f750f7581cbddd8be383/pytypes/type_util.py#L1812

Another potential reference: https://github.com/bojiang/typing_utils


Add feature gates

Currently the binary includes everything, though not all use cases utilize every feature. It might be beneficial to gate these. The default can then enable all of them if needed. For instance, the message API should be able to be removed fairly cleanly

This will require some additional code restructuring


Reclaim more space (about 3GB) & optimize steps

https://github.com/P3TERX/Actions-OpenWrt/commit/8330db576b78d8e6e7331138fbb7b17e3941e24c#commitcomment-53657226


Introduce chat memory

Depending on which LLM provider you use (size of window context), we can enable/disable chat memory.

The "challenge" is that the chat memory is across LLM providers.
Because the user can select for example first "OpenAI GPT-3" and for the following up question "Claude Opus".


algorithm skeleton

Build workflow diagram for organizational purposes

Ultimately, for each step, we want

process_name
description
software
input
output

load_data
read in fcs files
custom python scripts
fcs
txt


SofwareSerial

Is there SofwareSerial library on nanpy?

Currently not a feature but you can alter the com class to look for software serial instead of the hardware serial for your project. This would have to be done on the firmware side.


Using datatable with Metronic admin theme (bs 3)

[ ] bug
[ ] feature request
[ ] enhancement
[x] question

Environment details

Angular version: 2.4.0

**Clarity version: 0.8.7

**OS and version: Win 10

Hi.
Is it possible to use Clarity components (especially datagrid) inside existing SPA application
based on bootstrap 3  and Metronic template ?.
There is no problems with ts code and icons.
But using both Metronic.css and Clarity-UI.css is not working perfect.
Is it possible somehow override styles just for datatable component ?
Thanks
Edvin

Hi, @sebitsi: we cannot handle specific conflicts with the Metronic template, however, you should be able to use the Clarity datagrid inside of it if you do the modifications needed yourself (by simply overriding CSS styles although be careful that this means supporting the datagrid moving forward with updates from Clarity is going to be difficult and time consuming on your part) or if you work with the Metronic team to add support for Clarity.

If you want to use an external theme with Clarity, you will have to handle CSS conflicts on your side, mostly by overriding the conflicting parts on both side in your own stylesheet with whichever value produces the result you want.
Sadly, Datagrid is an extremely complex component for which the internal template and classes are not documented, on purpose. We need to be able to update and modify the internal styles without it being a breaking change every time.
Overriding one style on your side for a very specific reason would probably be fine, but overriding almost all the styles to get an entirely different look and feel would definitely not be maintainable.

Hi,
thanks for both answeres.
I like Clarity UX very much. Concepts how components are created to.
If we start with empty project based on clarity clarity will bi great option.
But we have project based in Metronic and some jquery libraries.
If we want migrate to Clarity this is not possible because of conflicts with css.
I have 2 more questions:

In project roadmap i noticed baclog item "Move off Bootstrap".
What this mean ? If you remove dependices to bootstrap will you change classes names and there
will be no conflicts with bootstrap in same project ?
If so this could be solution for my problem.

Clarity-UI.min.cs has 737 KB size. Why is so big ? Clarity-Angular.js has only 164 KB.
BS 3 on other hand is "only" 191 KB.

Regards.
Edvin


frontend - add unit tests for no Vue components

Follow up of #384 and #584
Simply add all the missing unit tests for all modules that are not a Vue component. This includes the state management, the Ethereum providers, all composables, possible route guards, etc. pp. While doing so it should be made sure that the tested code gets adapted to make it become better testable. Usually, when there are no unit tests from the very beginning, the written code becomes hard to get into a test-harness. This needs to be resolved in this phase now. A typical case is the dependency inversion by removing direct dependencies of a modules and replacing it with a dependency injection (e.g. via constructor argument).
The unit tests should be very focused and stripped down to the unit. Everything that is not part of this unit should be strictly mocked. Unit tests of a component should not fail by changes to another component. This becomes very simple with good testable code due to included refactoring. Everything else goes into the direction of integration tests, which will follow, but focus on different parts of code properties.

A very interesting question here becomes the decision Mohammed and I took in terms of the dependencies between composables. We need to investigate how easy it is to mock them properly and if we want to keep rather the composables clean and independent than the Vue components.

For testing the composables, please consider this section of the vue documentation: https://vuejs.org/guide/scaling-up/testing.html#testing-composables


This adapter shouldn't work

I didn't check your code direct, but i made some code based on your code and it has bug - not passsing headers to response
In this line
https://github.com/gofiber/adaptor/blob/master/main.go#L74
according to https://golang.org/pkg/net/http/#ResponseWriter
// Changing the header map after a call to WriteHeader (or
// Write) has no effect unless the modified headers are
// trailers.

You should move WriteHeader call to the end

Nice catch @mr-tron, fixed in https://github.com/gofiber/adaptor/releases/tag/v0.1.2 👍

Also I recommend to use Set() instead Add().

I don't have much time this week, I will accept a PR's if you want to help us ❤


add Dat

Just noticed Dat was missing. Gotta have it.
@juliangruber @Kriesse @yoshuawuyts @karissa

❤  thank you so much for adding it!

Nice! :D
On Sat, Oct 21, 2017, 11:19 Julian Gruber<EMAIL_ADDRESS>wrote:

❤ thank you so much for adding it!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/electron/electron-apps/pull/338#issuecomment-338377802,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACWlep-gTQCePXdtUsH7J1EFu_WQR7Roks5subcxgaJpZM4QBa6i
.

Yay thanks!!


what is different from fabric-sdk-py

Hi Bao hua,
As I can see the difference is that this repos is out of date, right? But the sdk repo is updated regularly.
The three branches in this repo are not updated since last Sep. Does that mean this repo is
dead now? If I want to learn something about the hyperledger python sdk, should I just go for fabric-sdk-py repo?

hi, wandy.
the repo (only support fabric v0.6) has been migrated to the fabric-sdk-py (which will support fabric 1.0 and more new features)


Root rotations test (v2)

This contains Adams commit from #91 rebased on current main and an additional commit that looks big but is really mostly changed indent:

Use @pytest.mark.parametrize() to create 15 tests out of the one
function, tweak test so there is no loop
use request.node.name for the repository dump directory name
(this contains the parametrized test name). The repository dumping now
works for the parametrized tests
Use bool instead of UnsignedMetadataError for expected success/failure

I think this looks good:
jkukkonen@jkukkonen-l:~/src/tuf-conformance$ ./env/bin/pytest tuf_conformance --entrypoint "./clients/go-tuf/go-tuf" --repository-dump-dir /tmp/root-tests -k test_root_rotation
========================================= test session starts ==========================================
platform linux -- Python 3.11.9, pytest-8.3.2, pluggy-1.5.0
rootdir: /home/jkukkonen/src/tuf-conformance
configfile: pyproject.toml
collected 43 items / 28 deselected / 15 selected

tuf_conformance/test_updater_key_rotations.py ...............                                    [100%]

================================== 15 passed, 28 deselected in 0.27s ===================================
jkukkonen@jkukkonen-l:~/src/tuf-conformance$ ls /tmp/root-tests/test_root_rotation\[3-of-5-one-key-rotate-fails-not-signed-with-3-old-keys\]/refresh-1/
1.root.json  2.root.json  snapshot.json  targets.json  timestamp.json

huh interesting: go client is fine but python client fails to find the repository now that the url contains []...
Example http://<IP_ADDRESS>:36493/test_root_rotation[1-of-1-key-rotation]/metadata/
Will have to take a look

huh interesting: go client is fine but python client fails to find the repository now that the url contains []

The http libraries seem to have different defaults: python url encodes automatically and then the lookup fails when SimulatorServer tries to use a fragment of  the encoded URL to lookup the corret repository. Fixed now.


Don't exclude 'New Tab Page' and 'Messaging System' from autoclosing of intermittent bugs

Checklist

[ ] Type annotations added to new functions
[ ] Docs added to functions touched in main classes
[x] Dry-run produced the expected results
[ ] The to-be-announced tag added if this is worth announcing

This will mean closing ~120 bugs.


[error] line[23] column[10], Identifier unexpected "static.nss"

Hi
I like your program and installed it on my surfacebook 2 which is windows 11 and works fine.
but when i tried to install it on windows 8.1 (my main machine) becaues i saw in your site that said Compatibility with 8.1 .
after installation it just not work
and shell.log :
`Nilesoft Shell
version 1.8.1 64-bit
<EMAIL_ADDRESS>https://nilesoft.org
Windows 8.1 Pro 64-bit, version 6.3.9600
2023-07-21 17:05:29 [info] Register completed successfully.
2023-07-21 17:05:30 [info] Restart Explorer.
2023-07-21 17:05:40 [error] line[23] column[10], Identifier unexpected "static.nss"
2023-07-21 17:05:45 [info] Register completed successfully.
2023-07-21 17:05:46 [info] Restart Explorer.
2023-07-21 17:05:49 [error] line[23] column[10], Identifier unexpected "static.nss"`
default context menu contain fences 3 and 7-zip, em editor, nvidia, intel entries

There is an bug in the "imports/static.nss" file that occurs when using Windows 8.1 or earlier.
To avoid this bug, remove this identifier "id.give_access_to" from the "imports/static.nss" file


Issue with docs.zowe.org/stable/user-guide/start-zowe-zos/

Description

This page should probably not state how to start or stop zowe via SDSF (or SYSVIEW).  Instead, the doc should describe the MVS commands that you use to start or stop.  Users could then choose to issue those commands via tools like SDSF.
If the SDSF references are removed, the preceding / should be consistently removed from the commands listed in this doc, e.g. /S -> S or /P -> P.
Pages to Update

Screenshots

Expected behavior

Additional context

closed with https://github.com/zowe/docs-site/pull/3609


fix "target" typo in Sankey Diagram Hover Text

Is it possible to remove the extra "target" word that is added in the hover text of sankey diagrams?

The new position and style of the hover text is great. However, depending on the what is below the sankey chart, it can be hidden. Is there a way to fix this?
Eg. without adding extra space

Adding extra space below sankey diagram

@lilila Hi, can you please help in how to create the multi-level sankey diagram. I have my table in the format
source, target, count
step-1, step-2, 10
step-1, step-3, 30
step-2, step-4, 10
step-2, step-5, 30
But still end up with single level diagram. Thanks.


Latest version incompatible with latest Sanic version

In Python, when trying to install the most recent version of slack_bolt (1.14.3) alongside the latest version of Sanic (22.6.2) with the async extras enabled, there is a dependency conflict with websockets. Currently, slack_bolt specifies "websockets>=8,<10" but Sanic requires "websockets>=10". To support the newest version of Sanic, support for websockets and sanic should be upgraded.
The last upgrade to the Sanic adapter was in https://github.com/slackapi/bolt-python/pull/432/files, and so these versions probably just need updating.
Reproducible in:
pip freeze | grep slack
python --version
sw_vers && uname -v # or `ver`

The slack_bolt version
$ poetry run pip freeze | grep slack
slack-bolt==1.14.3
slack-sdk==3.18.3

Python runtime version
$ poetry run python --version
Python 3.10.6

OS info
sw_vers && uname -v
ProductName:	macOS
ProductVersion:	12.6
BuildVersion:	21G115
Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64

Steps to reproduce:
(Share the commands to run, source code, and project settings (e.g., setup.py))

Set the following package versions in a package manager (I'm using poetry)

slack-bolt = { version = "^1.14.3", extras = ["async"] }
sanic = "^22.6.2"

Try to install both packages (poetry install)
Get error

Expected result:
Successful install
Actual result:
> poetry install
Updating dependencies
Resolving dependencies... (1.2s)

Because no versions of sanic match >22.6.2,<23.0.0
 and sanic (22.6.2) depends on websockets (>=10.0), sanic (>=22.6.2,<23.0.0) requires websockets (>=10.0).
And because slack-bolt (1.14.3) depends on websockets (>=8,<10)
 and no versions of slack-bolt match >1.14.3,<2.0.0, sanic (>=22.6.2,<23.0.0) is incompatible with slack-bolt (>=1.14.3,<2.0.0).
So, because <redacted> depends on both slack-bolt (^1.14.3) and sanic (^22.6.2), version solving failed.

Requirements
Please read the Contributing guidelines and Code of Conduct before creating this issue or pull request. By submitting, you are agreeing to those rules.

Hi @JWZepf, thanks or sharing this! We will improve this in future versions.

Hey @JWZepf! If you want to run the cutting edge version of bolt-python (the main branch, with the above PR) while using poetry, you could try the following:

Clone this very bolt-python repo in the parent directory of your current project
Build the package locally (step 1 in this section of the maintainer's guide)
Update your Bolt import in your pyproject.toml file to use the local package:

slack-bolt = {path = "../bolt-python/dist/slack_bolt-1.14.3-py2.py3-none-any.whl", extras = ["async"] }

Update your Poetry package imports for your app

$ poetry lock
$ poetry install

Start your app as you normally do! Not sure how you do this, but I use ngrok and..

$ uvicorn async_app:api --reload --port 3000 --log-level debug

Poetry is a really beautiful package manager. Thanks for sharing this!

@e-zim Thanks for fixing this!
@JWZepf We will release a new version including the fix within a few business days. Thanks again for taking the time to report this!

Checked the fix locally and everything looks good on my end. Thanks for the prompt response and fix!


cmd/go: allow 'mod verify' to use multiple CPUs

For example, on a moderately sized module of mine:
$ /usr/bin/time go mod verify
all modules verified
13.33user 2.39system 0:15.78elapsed 99%CPU (0avgtext+0avgdata 24472maxresident)k
1558152inputs+0outputs (0major+4086minor)pagefaults 0swaps

Given that I have four physical CPU cores, and that the bottleneck seems to be hashing moderately sized zip files on disk, I think there's plenty of room for improvement.
For example, if we verified up to GOMAXPROCS concurrently, I expect the process could easily get two or three times faster on my laptop.
I realise that this command doesn't need to be super fast, but I make use of it for "module sanity checks" in CI along with others like go mod tidy, so it's unfortunate that it doesn't make proper use of the resources available.
/cc @bcmills @jayconrod @matloob

I also wonder if there are other commands which are similarly bottlenecked on a single CPU and could be improved. I haven't really noticed any others; for example, go mod tidy regularly uses at least three of my cores.

I also wonder if there are other commands which are similarly bottlenecked on a single CPU and could be improved.

Definitely. @matloob is looking at adding tracing that would help identify bottlenecks.

Yeah I'm hoping tracing will help us find more of those commands.

This was marked as "help wanted", and I think it's a safe and easy change to make a week before the freeze, so I'll mail a CL shortly.

😊


Support for Mink framework false detection.

I think this maybe the wrong repo to send this request to. I've resubmitted over here: https://github.com/Behat/CommonFormatters/pull/7


Update README.md

Add version number zu CC0 and fix mailing list link

The mailing list is public :)


allow DisableThrottlingAttribute on a class

fixes #14

hold on quick and i'll add to the example project 🍵

alrighty!

Looks great! thanks

I've published v2.1.1, will be available once NuGet is done indexing.
Thanks again


Add UK2GTFS

Add link to the UK2GTFS converter

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Update logging

As part of #35 I have been trying to make sure and double check that all the middlewares have roughly consistent logging. This PR has the base changes for that, adding MiddlewareName to the base class, expanding the logger interface to allow a couple extra useful methods, extension method to allow a list of errors to be converted into an error string.
Additionally it contains the first few middlewares from the pipeline (except ExceptionHandlerMiddleware)

@MarcDenman this is really good, thank you very much!
What do you think about doing something to not have to add {Middleware} everywhere on the logging calls? I don't know if it works in .NET core but in good old fashioned .NET we have CallerMemberName attribute. I don't know if you are familiar with it so forgive me if I'm being patronising.
We could maybe do this
 CreateLogger([CallerMemberName]string memberName = "")
 {
      //store member name, which should be constructor name, then use it for subsequent logging.
 }

What do you think? I'm not sure if this helps but throwing it out there also no idea if it will work!

@TomPallister Sorry for the delay in replying, only just seen this.
Nope, not being patronising at all :) I have never come across it, good to learn about hidden gems like that. I will take a look at it and try to see if it helps.
Probably won't get around to it till Saturday, got a busy couple of evenings ahead of me.

@MarcDenman no worries no rush! Have to fit this stuff in around real life.

Started looking at it, [CallerMemberName] will unfortunately not work, as the logger is created within the constructor of the middleware so the value will end up being ".ctor". However, the CreateLogger function has the signature CreateLogger<T>, where T is the type you are logging for. I tried to get the type out of the logger, however it doesn't appear to be exposed easily through Microsoft.ILogger.  Instead, I have altered the IOcelotLogger to have a Name property (I want to think of a better name) and altered the constructor for the AspDotNetLogger so the name can be set when it is initialized.
I have also hung a few extension methods(not ideal I know) off IOcelotLogger for a couple common logging statements, i.e. MiddlewareEntered, MiddlewareCompleted
Not sure what to do with the MiddlewareName property, tempted to leave it as it could still be useful  for logging purposes.
Thoughts?

@MarcDenman looks good. Are you happy to merge this?

Yep. happy if you are.


UTF8 and fix newlines in CSV on windows

See footnote in https://docs.python.org/3.3/library/csv.html#id3 - need newline='' when opening a file for use with CSV libraries, especially on Windows. Adding explicit UTF 8 which will help on some environments.

@huoqif - I think this might be a bit better here - using the newline=''. Check it out and let me know what you think. I think this will fix the issue you had with resuming.
I still need to run a test with spaces in the export path. I believe that could fail on Windows as you say, I'm just skeptical the only fix is to move to the older path libraries.


Propagated positions are wiped out by Transformers

The mechanism of calling the default method on the Transformer recreates tree items, wiping out any line and column numbers if you have propagated_positions on. This only calls the default method if it exists; otherwise it fixes the tree in place (preserving properties).

Hi, thanks for submitting the PR, but unfortunately I can't merge it, because it breaks the tests.
I'm planning to do an overhaul of how transformers work internally,  and hopefully I'll be able to fix this problem too.
Meanwhile, you can use Transformer_NoRecurse, which doesn't alter the tree unless you tell it to.

Yeah, I was in a hurry and saw that afterwards - sorry about that. It was as easy as raising an issue anyway and I needed the workaround for now. You can leave here until you rewrite the Transformers or I can close if you want.


feat(AccessCode): add /access_codes/delete

Required for https://github.com/seamapi/react/issues/327.

@mikewuu Can you update from main?

@razor-x done! 👍


feat(e2e): shielding and unshielding

Description
part of #878

re-rolls noir sha256 stuff, as loops dont harm compile time anymore!
Moves public -> private stuff into the non native token as shielding and unshielding
Moves misc test stuff in public -> private into the test contract
Creates e2e test for shielding and un shielding again

Please provide a paragraph or two giving a summary of the change, including relevant motivation and context.
Checklist:

[ ] I have reviewed my diff in github, line by line.
[ ] Every change is related to the PR description.
[ ] I have linked this pull request to the issue(s) that it resolves.
[ ] There are no unexpected formatting changes, superfluous debug logs, or commented-out code.
[ ] The branch has been merged or rebased against the head of its merge target.
[ ] I'm happy for the PR to be merged at the reviewer's next convenience.

Btw this might help reduce some copy pasta - https://github.com/AztecProtocol/aztec-packages/pull/916


[Test] check_style and run_tests do not work on Python 2.

Related to https://github.com/Azure/azure-cli/issues/3883
When I run these scripts I get:
Traceback (most recent call last):
  File "C:\Users\trpresco\Documents\github\env2\Scripts\check_style-script.py", line 11, in <module>
    load_entry_point('azure-cli-dev-tools', 'console_scripts', 'check_style')()
  File "C:\Users\trpresco\Documents\github\env2\lib\site-packages\pkg_resources\__init__.py", line 561, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "C:\Users\trpresco\Documents\github\env2\lib\site-packages\pkg_resources\__init__.py", line 2626, in load_entry_point
    raise ImportError("Entry point %r not found" % ((group, name),))
ImportError: Entry point ('console_scripts', 'check_style') not found

Traceback (most recent call last):
  File "C:\Users\trpresco\Documents\github\env2\Scripts\run_tests-script.py", line 11, in <module>
    load_entry_point('azure-cli-dev-tools', 'console_scripts', 'run_tests')()
  File "C:\Users\trpresco\Documents\github\env2\lib\site-packages\pkg_resources\__init__.py", line 561, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "C:\Users\trpresco\Documents\github\env2\lib\site-packages\pkg_resources\__init__.py", line 2626, in load_entry_point
    raise ImportError("Entry point %r not found" % ((group, name),))
ImportError: Entry point ('console_scripts', 'run_tests') not found

Environment summary
Install Method (e.g. pip, interactive script, apt-get, Docker, MSI, edge build) / CLI version (az --version) / OS version / Shell Type (e.g. bash, cmd.exe, Bash on Windows)
dev_setup.py / azure-cli (2.0.21) / Win 10 / CMD.exe

This actually looks related to https://github.com/Azure/azure-cli/commit/b5e53a8d44977337aeb85d52bbf0833f4756b3b9#diff-2071ffe9914306221414bd4300c8a8a3R52.
I'm surprised as it's fully documented at http://python-packaging.readthedocs.io/en/latest/command-line-scripts.html#the-console-scripts-entry-point.
What's the full Python version & pip version you're using?

Python 2.7.11 pip 9.0.1


Implement SelectVariants tool

this is need to address https://github.com/icgc-argo/gatk-mutect2-variant-calling/issues/9

addressed by #56


KeyError?

Hello,
I am looking forward to running graphmb and have prepped my input files according to the instructions.
My install was of graphmb version 0.2.3 using conda:
conda create -n graphmb -c bioconda graphmb
My command was:
conda activate graphmb
graphmb --assembly GraphMB_ownDepths/ --outdir GraphMB_ownDepths/graphMB --cuda
It seemed to start running ok but has now given a Key Error? (log copied below). Do you have any suggestions how I can fix this?
Running GraphMB 0.2.3
using cuda: True
setting seed to 1
Cache not found on GraphMB_ownDepths/graphMB
processing sequences GraphMB_ownDepths/assembly.fasta
read 32772 seqs
processing GFA file (edge nodes) GraphMB_ownDepths/assembly_graph.gfa
skipped contigs 4156 < 1000
read 34831, edges
reading depths
reading labels
Saved cache to GraphMB_ownDepths/graphMB
==============Running VAE model=====================
******* Running model: CCVAE **********
***** using edge weights: True ******
***** cluster markers only: False *****
***** self edges only: False *****
***** Using raw kmer+abund features: True
***** SCG neg pairs: (0,)
***** input features dimension: (32772, 104)
Uncaught exception
Traceback (most recent call last):
File "/srv/scratch/z2194795/miniconda3/envs/graphmb/bin/graphmb", line 10, in
sys.exit(main())
File "/srv/scratch/z2194795/miniconda3/envs/graphmb/lib/python3.10/site-packages/graphmb/main.py", line 497, in main
vae_embs, _ = train_ccvae.run_model_ccvae(dataset, args, logger, 0,
File "/srv/scratch/z2194795/miniconda3/envs/graphmb/lib/python3.10/site-packages/graphmb/train_ccvae.py", line 170, in run_model_ccvae
cluster_labels, stats, _, hq_bins = compute_clusters_and_stats(
File "/srv/scratch/z2194795/miniconda3/envs/graphmb/lib/python3.10/site-packages/graphmb/evaluate.py", line 367, in compute_clusters_and_stats
unresolved_contigs_with_scgs = np.array([n for i,n in enumerate(node_names)
File "/srv/scratch/z2194795/miniconda3/envs/graphmb/lib/python3.10/site-packages/graphmb/evaluate.py", line 368, in
if labels[i] not in positive_clusters and len(dataset.contig_markers[n]) > 0])
KeyError: 'edge_1'

Hi, sorry for the delay but I think the issue might be with your input data. However can you retry with the latest version (0.2.5) ? it fixes some bugs

I also get the same error using v0.2.5, so unfortunately this bug doesn't seem to be fixed. Any help would be much appreciated! (see below for log)
Running GraphMB 0.2.5
using cuda: False
setting seed to 1
Cache not found on mag_script_test/bins/QIA_MUTT_4/graphmb_results/
processing sequences mag_script_test/metaflye_assemblies/QIA_MUTT_4/polished.fasta
read 22785 seqs
processing GFA file (edge nodes) mag_script_test/metaflye_assemblies/QIA_MUTT_4/assembly_graph.gfa
skipped contigs 59 < 1000
read 55, edges
reading depths
reading labels
Saved cache to mag_script_test/bins/QIA_MUTT_4/graphmb_results/
Reading assembly info file
==============Running VAE model=====================
******* Running model: CCVAE **********
***** using edge weights: True ******
***** cluster markers only: False *****
***** self edges only: False *****
***** Using raw kmer+abund features: True
***** SCG neg pairs: (0,)
***** input features dimension: (22785, 104)
Uncaught exception
Traceback (most recent call last):
File "/home/harry/miniconda3/envs/graphmb/bin/graphmb", line 8, in
sys.exit(main())
File "/home/harry/miniconda3/envs/graphmb/lib/python3.8/site-packages/graphmb/main.py", line 499, in main
vae_embs, _, _ = train_ccvae.run_model_ccvae(dataset, args, logger, 0,
File "/home/harry/miniconda3/envs/graphmb/lib/python3.8/site-packages/graphmb/train_ccvae.py", line 170, in run_model_ccvae
cluster_labels, stats, _, hq_bins = compute_clusters_and_stats(
File "/home/harry/miniconda3/envs/graphmb/lib/python3.8/site-packages/graphmb/evaluate.py", line 367, in compute_clusters_and_stats
unresolved_contigs_with_scgs = np.array([n for i,n in enumerate(node_names)
File "/home/harry/miniconda3/envs/graphmb/lib/python3.8/site-packages/graphmb/evaluate.py", line 368, in
if labels[i] not in positive_clusters and len(dataset.contig_markers[n]) > 0])
KeyError: 'edge_1'

I have solved this issue, which was a user error. It appears to be caused by inputting an incorrectly formatted --markers file. I was previously generating this using checks on the individual fasta file containing all my contigs, where it requires a file from checkm on all contigs in separate fasta files


"Oh, this is not even categorized!" on image with 1 category

Summary
Despite https://commons.wikimedia.org/w/index.php?title=File:Stamp_of_Ajman_-1968-Colnect_682825-Ferdinand_of_Austria-by_Velasquez_and-_Return_from_hunting.jpeg&action=history having a category (and having it since it was first created), I got the following screen in Review:

Steps to reproduce
Open Review
Review until you get Oh, this is not even categorized!
If it actually does not have categories, try again
Expected behaviour
should not display this message if there is at least one category
Actual behaviour
displays message despite 1 category
Device name
No response
Android version
9
Commons app version
master
Device logs
No response
Screen-shots
No response
Would you like to work on the issue?
Prefer not

Thanks for showing interest @devarsh-mavani-19, we recommend solving each issue in a different PR.

Thanks for showing interest @devarsh-mavani-19, we recommend solving each issue in a different PR.

@4D17Y4 sorry for delay in responding. Actually adding this feature
https://github.com/commons-app/apps-android-commons/pull/4874
Will automatically solve this issue as well.

Despite https://commons.wikimedia.org/w/index.php?title=File:Stamp_of_Ajman_-1968-Colnect_682825-Ferdinand_of_Austria-by_Velasquez_and-_Return_from_hunting.jpeg&action=history having a category (and having it since it was first created), I got the following screen in Review:

May be this is related to https://github.com/commons-app/apps-android-commons/issues/4901 ? I say that because the picture linked above has a single category ("1968 stamps of Ajman") that starts with a date. If that's the case, #4902 [ I hope to get around to fixing those tests soon :) ].


Evaluation based on SPLADE++ ED on dl21 and dl22

Is it currently not supported to evaluate based on SPLADE++ ED on dl21 and dl22？
How were the results reported in RankZepher obtained？

Hey @melonking32 thanks for your interest.
Prebuilt index of SPLADE++ ED for MS MARCO v2 (dl21/22) was not yet provided in pyserini at the time of RankZephyr. I built the indexes myself in Anserini, cached the results https://github.com/castorini/rank_llm_data/tree/main/retrieve_results/SPLADE_P_P_ENSEMBLE_DISTIL and moved them over (both files for each of dl21/dl22) to retrieve_results/SPLADE_P_P_ENSEMBLE_DISTIL to automatically hit it. You can just copy these files over to achieve similar results. We plan to make this automatic, it is the high priority PR so when that's sorted, you can automatically get the cache when exists. In the meantime, we will also try to add more pyserini indexes. But if you need this quick, reuse the cache :)

Thank you for your reply! I can now read directly from the file.
I have another question, does the current code not support dl22 evaluation?

The QRELS were not added to pyserini during RankZephyr's time so that would fail, you can just score them with trec_eval, it still will reorder the results


Minimal Maven version

Quickstarts currently enforce Maven 3.1.1 by the enforcer plugin and 3.2.1 by <prerequisites> in the main pom.xml.
Which one is correct ?

3.2.1 :)


Add eslint + prettier

Could you add eslint + prettier?

sure I can add it, I'm just very busy with work so I will probably not add it very soon :/
If you have time you can add it of course ;)

Added in new version


setuptools requirement

this package has a hidden dependency of setuptools <=66.1.1. With 66.2.0 "pkg_resources is deprecated as an API".

Just noting that I hit this as well.  For others, the error looks like
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/awheeler/Simons Foundation Dropbox/Adam Wheeler/science/rotating_giants/.venv/lib/python3.12/site-packages/sdss_semaphore/__init__.py", line 6, in <module>
    from pkg_resources import resource_filename
ModuleNotFoundError: No module named 'pkg_resources'

Workaround of uv pip install 'setuptools<=66.1.1' worked.

the other option seems to be to use importlib
import importlib.resources as resources
try: #python 3.9+
path = resources.as_file(resources.files(name).joinpath(f'etc/{self.MAPPING_BASENAME}'))
except: #python 3.7 and 3.8
path = resources.path(name, f'etc/{self.MAPPING_BASENAME}')


Feature:  Custom committees

Fee-backed assets, or FBA's, will require several pieces.  This ticket describes one of those pieces, custom committees.
In current Graphene release, there are two accounts, GRAPHENE_COMMITTEE_ACCOUNT and GRAPHENE_RELAXED_COMMITTEE_ACCOUNT.  The semantics of GRAPHENE_COMMITTEE_ACCOUNT are as follows:

Every maintenance interval, the account's authority is updated to the top N committee members voted for by the proxies of holders of CORE asset.
Account can only act through proposals which go through a review period of t seconds.

This ticket proposes making committee_size = N, committee_asset = CORE and review_period_seconds = t fields in a new object, called committee_object.
Every account would then have an optional< committee_id_type > which would be set during account creation.  If this field is non-optional, the account's authority
will be set every maintenance interval using the same algorithm on the asset specified in committee_asset.
There are some details to fill in:

An account must now specify which committee(s) it is willing to serve on.  I think a separate committee_member_object
for each committee you are willing to serve on, so a single LTM account can serve on multiple committees (thus well-known
community members can use their account's existing reputation to help bootstrap trust in a new committee.)  This also allows
voters to specify preferences of the form "I support Alice for committee C, but not committee D."
The committee_member_object now needs a committee field to indicate for which committee it's running.
By previous point, we need to refer to committee ID's, so the committee_member_object committee_object will be a protocol object and its ID's will also become part of the protocol.
Reserve some low-numbered committee_member_object committee_object for future system-level expansions.
Index committee_object by committee_asset field, then compute votes for each asset at most once, regardless of how many committees exist for the asset.
Spam potential limited by account creation fees, LTM fees for committee members, and voting fees / limits.  Spam potential of custom committees (with or without custom asset) not worse than spam potential of CORE voting.
Committee members are no longer unique on committee_member_account, but unique on the composite key (committee_member_account, committee).

Things to think about:

How do we do things like COMMITTEE_ACCOUNT and RELAXED_COMMITTEE_ACCOUNT as separate account objects, but not separate committee objects?
One idea:  Have a committee object which clones membership of another committee object.  Allows committees and accounts to be 1-1 (every account has own committee object).  This is useful because we can add a field to the committee specifying the account it is for, allowing us to go from committee to account without a new index on the large account set.
Another idea:  Store the review period in the account object, not the committee object.
We need a new operation to create a committee member for a committee other than the core committee, as the committee operations don't have extensions field defined.
Specifying the committee when an account is created can be done by adding an extension to the account_create_operation
Ideally we should make the COMMITTEE_ACCOUNT and RELAXED_COMMITTEE_ACCOUNT simply exercise the new committee code, so we only do vote counting, etc. in one place.

Actually I am having second thoughts about committee objects being protocol objects.  There is little reason to have multiple accounts with the same committee as their authority, so we can attach the committee to the account instead.


Warning using debug requests method

✨ Feature proposal

Add console warning when we use .debugRequests() method.
🎯 Motivation

We always forget to remove the method after debugging
👀 Example

💬 Comments

What do you think about this proposal?

i LOVE this 😍


HBASE-24153. Remove unnecessary super() in MultiVersionConcurrencyControl#MultiVer…

…sionConcurrencyControl()
Signed-off-by: sunlisheng<EMAIL_ADDRESS>

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 43s
Docker mode activated.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
6m 37s
the patch passed

+1 :green_heart:
compile
1m  8s
the patch passed

+1 :green_heart:
javac
1m  8s
the patch passed

+1 :green_heart:
shadedjars
5m 42s
patch has no errors when building our shaded downstream artifacts.

-0 :warning:
javadoc
0m 39s
hbase-server in the patch failed.

_ Other Tests _

+1 :green_heart:
unit
173m  0s
hbase-server in the patch passed.

204m 43s

Subsystem
Report/Notes

Docker
Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1468/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/1468

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 52fd740bff7d 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 1fb693b998

Default Java
2020-01-14

javadoc
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1468/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt

javadoc
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1468/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1468/1/testReport/

Max. process+thread count
3224 (vs. ulimit of 10000)

modules
C: hbase-server U: hbase-server

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1468/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 31s
Docker mode activated.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
8m  5s
the patch passed

+1 :green_heart:
compile
1m 23s
the patch passed

+1 :green_heart:
javac
1m 23s
the patch passed

+1 :green_heart:
shadedjars
6m 28s
patch has no errors when building our shaded downstream artifacts.

-0 :warning:
javadoc
0m 47s
hbase-server in the patch failed.

Do you have a justification for changing this? It is just being explicit about the super call.  Is it bad?  Thanks.

@saintstack
current code is no problem. But
this way of writing seem not to conform to the habit of java code.
Subclass constructor calls the default constructor of the parent class are implicit calls.
Thank you.

@saintstack Yes I agree with @leosunli  super() will be implicitly added if not any super is called, I think it makes sense just to remove it here.
The parametrized constructor
public MultiVersionConcurrencyControl(long startPoint)

does not have super() and we know it's fine. Adding super() one place but not the other seems a bit inconsistent.


bug where iNat tries to graft taxa from external sources to the wrong branches

I've noticed this over the years but never really investigated. Was brought up again here: https://www.inaturalist.org/flags/232667#comment-1754921
worth looking into

Pretty sure this is a duplicate of #1602


add some gio overrides

This adds some overrides from gio which are based on: https://gitlab.gnome.org/GNOME/gjs/-/blob/master/modules/core/overrides/Gio.js. But not complete yet.
There is actually a lot of interesting stuff in the overrides dir which should be ideally included in this repo.

I also added now github actions. For that you once more need to add your npm secret as decribed here: https://github.com/Gr3q/types-cjs/pull/9#issue-1086975155

Please verify that they typings are correct with CJS instead, that's what Cinnamon is using

Also I'm thinking to just use a single package for cjs and move these to cjs because even these are becoming Cinnamon specific.


Docs.MS Release Updates for azure-template

Update docs metadata and targeting for release of azure-template

Docs Build status updates of commit ffe2883:
:warning: Validation status: warnings

File
Status
Preview URL
Details

docs-ref-services/preview/template-readme.md
:warning:Warning
View (azure-python-preview)
Details

:warning:Warning

Details

ci-configs/packages-preview.json
:white_check_mark:Succeeded

docs-ref-services/preview/template-readme.md

Line 2, Column 1: [Warning-ms-prod-and-service] Only one of the following attributes can exist: 'ms.prod', 'ms.service'. Use ms.prod for on-premise products, or ms.service for cloud services.
Line 9, Column 16: [Warning-ms-prod-technology-invalid] Invalid value for 'ms.technology': 'azure' is not valid with 'ms.prod' value 'azure'.
Line 11, Column 13: [Warning-ms-service-subservice-invalid] Invalid value for 'ms.service': 'template'.
Line 2, Column 1: [Suggestion-description-missing] Missing required attribute: 'description'.

Line 0, Column 0: [Warning-validation-incomplete] Failed to get the validation ruleset and validation was not completed. This happens when there's an issue with the service and continuing to retry the call could cause build delays. You might have content issues that were not reported. To retry validation, close and re-open your PR, or rebuild your branch via Docs Portal (requires admin permissions). If you need admin help or if you continue to see this message, file an issue via https://SiteHelp.
Line 0, Column 0: [Warning-xref-not-found] Cross reference not found: 'azure.storage.filedatalake.DataLakeFileQueryError'.
Line 0, Column 0: [Warning-xref-not-found] Cross reference not found: 'azure.keyvault.certificates.CertificateIssuer'.
Line 0, Column 0: [Warning-xref-not-found] Cross reference not found: 'azure.keyvault.certificates.IssuerProperties'.
Line 0, Column 0: [Warning-xref-not-found] Cross reference not found: 'azure.mgmt.monitor.v2019_11_01_preview.models.ErrorResponseError'.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Update xgboost LTR examples for 2.x

Apparently 2.x only allows integer labels
Check failed: label_is_integer: When using relevance degree as target, label must be either 0 or positive integer.
Stack trace:

Should add notebooks in sample apps under testing similar to what we do for pyvespa
https://github.com/vespa-engine/sample-apps/blob/master/commerce-product-ranking/notebooks/Train-xgboost.ipynb

Fixed in https://github.com/vespa-engine/sample-apps/pull/1595


feat: validate chunk size is a multiple of 64 bytes

Closes https://github.com/celestiaorg/rsmt2d/issues/235

If we expect codecs to only support one chunk size (i.e. 64 in this case), wouldn't be better if the interface was instead ChunkSize() int and we compared that value with the chunk size provided.

We expect codecs to support more than one chunk size. In this case, leopard supports chunks of size 64, 128, 192, etc.


jslib.k6.io/k6-summary cannot be imported

Hi Team
I have k6 tests with import like below:
import { textSummary } from 'https://jslib.k6.io/k6-summary/0.0.1/index.js'
When I run docker command with docker image ghcr.io/grafana/xk6-client-prometheus-remote:v0.2.2 I see:
ERRO[0000] Module specifier "https://jslib.k6.io/k6-summary/0.0.1/index.js" was tried to be loaded as remote module by prepending "https://" to it, which didn't work. If you are trying to import a nodejs module, this is not supported as k6 is _not_ nodejs based. Please read https://k6.io/docs/using-k6/modules for more information. Remote resolution error: "Get "https://jslib.k6.io/k6-summary/0.0.1/index.js": x509: certificate signed by unknown authority"         at go.k6.io/k6/js.(*InitContext).Require-fm (native)         at file:///mnt/c/D/HM/Repos/common-cart/tests/k6/full.test.js:1:0(33)  hint="script exception"
I checked with flag --insecure-skip-tls-verify=true but it doesn't help
BR
Mario

Hi @marioosh11 sorry for the long delay :(
Does this error happen for you with k6 not build with xk6? Preferably again from inside docker?
--insecure-skip-tls-verify is only about k6 script requests, so like http.get and co. not about requests k6 does on your behalf. Relevant issue https://github.com/grafana/k6/issues/2697


UX: Clarify "no content" message.

If a user opens a flow, he will often be greeted with a message stating "no content". We should rephrase that to "No request content" or "No request content (press tab to view response)".

When you become familiar with the concept of "tab" allowing to switch between request and response, it is clear that "no content" is referring to the request body. Instead, I suggest adding "tab:toggle between request and response" guide, next to "?:help q:back [*:8080]", because these guides are the first thing I look as a novice user.

If you can fit "tab:toggle between request and response" into <10 chars, I'm all in. For now, we'll display
No request content (press tab to view response) in the UI if there's no request content.
Cheers,
Max


AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentity

I have the csi driver and aws provider installed on my eks cluster. I am attempting to follow the instructions for usage, but my test pod is getting the following error trying to mount the secret:
  Warning  FailedMount  41m  kubelet  MountVolume.SetUp failed for volume "secrets-store-inline" : rpc error: code = Unknown desc = failed to mount secrets store objects for pod default/nginx-deployment-74d68b559b-lsfr9, err: rpc error: code = Unknown desc = Failed fetching parameters: WebIdentityErr: failed to retrieve credentials
caused by: AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentity
           status code: 403, request id: 74069505-0a96-4068-a6ea-4e37e1bf9f9d

Here is the corresponding event from Cloud Trail:
{
    "eventVersion": "1.08",
    "userIdentity": {
        "type": "WebIdentityUser",
        "principalId": "arn:aws:iam::000000000000:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/00000000000000000000000000000000:sts.amazonaws.com:system:serviceaccount:default:vitotest-sa",
        "userName": "system:serviceaccount:default:vitotest-sa",
        "identityProvider": "arn:aws:iam::000000000000:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/00000000000000000000000000000000"
    },
    "eventTime": "2021-05-24T20:16:19Z",
    "eventSource": "sts.amazonaws.com",
    "eventName": "AssumeRoleWithWebIdentity",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "x.x.x.x",
    "userAgent": "aws-sdk-go/1.37.0 (go1.15.8; linux; amd64)",
    "errorCode": "AccessDenied",
    "errorMessage": "An unknown error occurred",
    "requestParameters": {
        "roleArn": "arn:aws:iam::000000000000:role/eksctl-my-cluster-name-addon-iamserviceaccou-Role1-000000000000",
        "roleSessionName": "secrets-store-csi-driver-provider-aws"
    },
    "responseElements": null,
    "requestID": "74069505-0a96-4068-a6ea-4e37e1bf9f9d",
    "eventID": "e3f6f433-c6e5-45fa-a970-f629aca1d037",
    "readOnly": true,
    "resources": [
        {
            "accountId": "000000000000",
            "type": "AWS::IAM::Role",
            "ARN": "arn:aws:iam::000000000000:role/eksctl-my-cluster-name-addon-iamserviceaccou-Role1-000000000000"
        }
    ],
    "eventType": "AwsApiCall",
    "managementEvent": true,
    "eventCategory": "Management",
    "recipientAccountId": "000000000000",
    "tlsDetails": {
        "tlsVersion": "TLSv1.2",
        "cipherSuite": "ECDHE-RSA-AES128-SHA",
        "clientProvidedHostHeader": "sts.us-east-1.amazonaws.com"
    }
}

Steps I've taken:

Associated the oidc provider with my cluster via eksctl:

eksctl utils associate-iam-oidc-provider --cluster my-cluster-name --region="us-east-1" --approve

Created an IAM policy with SSM permissions.

POLICY_ARN=$(
    aws --region us-east-1 --query Policy.Arn --output text iam create-policy \
        --policy-name vitotest-policy \
        --policy-document '{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": ["ssm:DescribeParameters", "ssm:GetParameters"],
            "Resource": ["arn:aws:ssm:us-east-1:xxx:parameter/vitotest/*"]
        },
        {
            "Effect": "Allow",
            "Action": [
              "kms:Describe*",
              "kms:List*",
              "kms:Encrypt",
              "kms:Decrypt",
              "kms:ReEncrypt*",
              "kms:GenerateDataKey*"
            ]
            "Resource": ["arn:aws:kms:us-east-1:xxx:key/xxx"]
        }
    ]}'
)

Created IAM service account via eksctl:

eksctl create iamserviceaccount \
    --name vitotest-sa \
    --region="us-east-1" \
    --cluster "my-cluster-name" \
    --attach-policy-arn "$POLICY_ARN" \
    --approve \
    --override-existing-serviceaccounts

Applied the following manifest to my k8s cluster:

---
apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
  name: vitotest-aws-secrets
spec:
  provider: aws
  parameters:
    objects: |
      - objectName: "vitotest/mykey"
        objectType: "ssmparameter"
      - objectName: "vitotest/mysecret"
        objectType: "ssmparameter"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
    owner: vito
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
        owner: vito
    spec:
      serviceAccountName: vitotest-sa
      volumes:
      - name: secrets-store-inline
        csi:
          driver: secrets-store.csi.k8s.io
          readOnly: true
          volumeAttributes:
            secretProviderClass: "vitotest-aws-secrets"
      containers:
      - name: nginx-deployment
        image: nginx
        ports:
        - containerPort: 80
        volumeMounts:
        - name: secrets-store-inline
          mountPath: "/mnt/secrets-store"
          readOnly: true

We found the root cause of the issue, which was two-fold.

We hit this scenario with eksctl, which was confusing at best.
The objectNames in my SecretProviderClass resource were missing the leading slash for ssm param names. (I thought I read that the leading slash should be left off, but I can't find where I saw that now). The correct format should be:

---
apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
  name: vitotest-aws-secrets
spec:
  provider: aws
  parameters:
    objects: |
      - objectName: "/vitotest/mykey"
        objectType: "ssmparameter"
      - objectName: "/vitotest/mysecret"
        objectType: "ssmparameter"


[TEVA-2069] Automate generation of offline site

Jira ticket URL
https://dfedigital.atlassian.net/browse/TEVA-2069
Changes in this PR:

Change to use the Gov.UK Frontend
Automate the generation of the offline site with a GitHub Actions workflow

Review app deployed to https://teaching-vacancies-review-pr-2967.london.cloudapps.digital


Implement the "Show API request" endpoint

This PR implement the endpoint which, given in input the processing request, return a string containing the equivalent CADS API request.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Why did this method I used not speed up?

Why did this method I used not speed up? Did anyone encounter this situation?Are there any restrictions on this method

@wnntju could you clarify "speed up" here?

close as no response so far. feel free to reopen if needed.


How to set message.timeout.ms for Producer

I want to get error faster than after 5min if brokers are down. I found that options message.timeout.ms is for it. But when I set this options when I creating Producer instance and call ProduceAsync I get result with error after 5min.
I create producer like this
var config = new KeyValuePair<string, string>[]{
new KeyValuePair<string, string>("message.timeout.ms", "1000"),
new KeyValuePair<string, string>("bootstrap.servers", "server:9092"),
}
_producer = new Producer(config, false, false);
Thanks
Checklist
Please provide the following information:

[ ] Confluent.Kafka nuget version: 0.11.0

try setting message.timeout.ms under default.topic.config, and also note that message.send.max.retries will also affect how long it takes before you get an error. eg:
            var pConfg = new Dictionary<string, object>()
            {
                { "bootstrap.servers", "localhost:9092" },
                { "default.topic.config", new Dictionary<string, object>()
                    {
                        { "message.timeout.ms", 5000 }
                    }
                },
                { "message.send.max.retries", 0 }
            };

In 0.11.0, my understanding was that librdkafka should now recognize topic level configuration properties at the top level (as you are doing), however I just tested this and it didn't appear to work here @edenhill .

@mhowlett It does, and it creates an implicit default topic configuration object internally for those properties, but this config object is then replaced when the .NET bindings call set_default_topic_config().

ahh ok. I think the way to solve this is promote any default.topic.config settings to the top level in the producer constructor and then never call set_default_topic_config. a warning would be good - though we don't really have a good mechanism for doing it. We could maybe warn by writing to stderr or firing and event on OnError, but neither are ideal.

I meant a documentation warning.

@mhowlett thanks your config works as I expected.

@edenhill I am currently using Confluent V0.11.4-RC2. Iam able to set the timeout directly at the top level. But if my Broker is unavailable I still get the error message as "timeout". Here is th code:
var config = new Dictionary<string, object>
{
{ "bootstrap.servers", brokerTestNoSecurity },
{"message.timeout.ms", 1000},
};
using (var producer = new Producer<Null, string>(config, null, new StringSerializer(Encoding.UTF8)))
{
Console.WriteLine($"{producer.Name} producing on {topicName}. q to exit.");
    string text;
    while ((text = Console.ReadLine()) != "q")
    {
				var deliveryReport = producer.ProduceAsync(topicName, null, text);

				deliveryReport.ContinueWith(task =>
				{
					Console.WriteLine($"Partition: {task.Result.Partition}, Offset:
                                            {task.Result.Offset}");
				});
				var x = deliveryReport.Result;
    }
    producer.Flush(TimeSpan.FromSeconds(10));

there is no support for Dictionary<string, object>() in 1.0.0 ... How could we be able to use "message.timeout.ms"?

there is no support for Dictionary<string, object>() in 1.5.0 ... How could we be able to use "message.timeout.ms"?

Would suggest using ProducerConfig in Confluent.Kafka. It has a settable property "MessageTimeoutMs".


Add telemetry metric tests

Description
Adds a test for telemetry metrics. Currently only enabled for .NET, as I'm not sure what other teams have implemented these yet.
This test is necessarily quite slow, as we need to make sure we have had at least one aggregation period (10s). Waiting for 2 periods to avoid flake. Unfortunately the period is fixed at 10s by design (as this is a global setting per-metric in the backend) so can't be safely reduced for testing purposes.
There were already some ASM tests for metrics, so this extracts the logic already used there and shares it in the library interfaces
Motivation
We want to add system tests for telemetry features
Workflow

⚠⚠ Create your PR as draft
Follow the style guidelines of this project (See how to easily lint the code)
Work on you PR until the CI passes (if something not related to your task is failing, you can ignore it)
Mark it as ready for review

Once your PR is reviewed, you can merge it! :heart:
Reviewer checklist

[ ] Check what scenarios are modified. If needed, add the relevant label (run-parametric-scenario, run-profiling-scenario...). If this PR modifies any system-tests internal, then add the run-all-scenarios label (more info).
[ ] CI is green

[ ] If not, failing jobs are not related to this change (and you are 100% sure about this statement)

if any of build-some-image label is present

is the image labl have been updated ?
just before merging, locally build and push the image to hub.docker.com

@andrewlock Lint was failing, I pushed the fix (run ./format.sh).


Argument name shadows return field name: is this a feature?

First, a concrete failing test: one cannot store an object with field "i" in an array, because it is impossible to read the field back (because the first argument of array.get is also called "i", so (arr.get 0).i returns 0, not 11):
[] > array-get-arg-name-shadows-field-name
  [] > a
    11 > i
  (* a) > arr
  (arr.get 0).i.eq 11 > @

Same test with "j" instead of "i" passes (as expected).
Second, a more general test (also fails for a similar reason: (f z).x equals z for all z):
[] > arg-name-shadows-field-name
  [x] > f
    [] > @
      123 > x
  (f 567).x.eq 123 > @

Is this a bug or a feature?

@dours this definitely is a feature. This is how decoration works. What's in the object is more important vs. what's in the decoratee.


update icon default colour and ButtonIcon disabled opacity

💬 Description
Icon default colours now use the new neutral primary colour, unfortunately I've had to do this the old fashioned way as a sass variable. Once everything is tailwindy we can make this nicer.
Also a new opacity setting and applying that to ButtonIcons

 Update our base neutral colour to match GCDS

 Make disabled icon buttons 30% opacity


Infer the jsx property type

Bug Report
🔎 Search Terms
jsx, infer property
🕗 Version & Regression Information

This is the behavior in every version I tried, and I reviewed the FAQ for entries about JSX

⏯ Playground Link
import React from 'react'

declare function A(props: {
  children:
    | React.ReactElement<{ test: 'aaaa' | 'bbbb' }>
}): React.ReactElement

declare function B<P extends Record<string, any>>(props: ...

Playground Link
💻 Code
import React from 'react'

declare function A(props: {
  children:
    | React.ReactElement<{ test: 'aaaa' | 'bbbb' }>
}): React.ReactElement

declare function B<P extends Record<string, any>>(props: P): React.ReactElement

const C =

  // ^? (property) test: string
  {B({
    test: 'aaaa'
  // ^? (property) test: 'aaaa'
  })}

🙁 Actual behavior
Infer test property is string type in JSX.
🙂 Expected behavior
Infer test property is 'aaaa' type in JSX.

Type-checking this won't work without something like https://github.com/microsoft/TypeScript/pull/29818 .
The contextual types here could maybe be improved to aid autocompletions and stuff like that, but it likely wouldn't solve your underlying problem, see this playground

Type-checking look like works. In fact, I need provide correct keywords hints when typing 'a', such as:

When using function not JSX, call a function which is return React.ReactElement type. It provide the correct hints.

The example is very easy. Actually, my case is use a function and infer the function arguments type.


Proxy Failure on WS Proxy Server Connection Close

HTPM just got merged into webpack(webpack/webpack-dev-server#359) which is awesome because it enables things like native websockets which didn't work previously.
I was just taking it for a spin and everything was working great until I took down the backend server before shutting down the dev-server. The logic in my front-end is such that after the socket dies it will recheck if it is logged in (hence the /api/users/me check). I can work on an example but it may take me a little while, wondering if it might be obvious from these logs + stack trace:
[HPM] Upgrading to WebSocket
[HPM] Client disconnected
[HPM] Proxy error: ECONNREFUSED. <IP_ADDRESS> -> "<IP_ADDRESS>:9020/api/users/me"
[HPM] Upgrading to WebSocket
./node_modules/webpack-dev-server/node_modules/http-proxy-middleware/lib/handlers.js:8
        res.writeHead(500);
            ^

TypeError: res.writeHead is not a function
    at ProxyServer.proxyError (./node_modules/webpack-dev-server/node_modules/http-proxy-middleware/lib/handlers.js:8:13)
    at ProxyServer.emit (./node_modules/webpack-dev-server/node_modules/http-proxy-middleware/node_modules/http-proxy/node_modules/eventemitter3/index.js:117:27)
    at ClientRequest.onOutgoingError (./node_modules/webpack-dev-server/node_modules/http-proxy-middleware/node_modules/http-proxy/lib/http-proxy/passes/ws-incoming.js:153:16)
    at emitOne (events.js:77:13)
    at ClientRequest.emit (events.js:169:7)
    at Socket.socketErrorListener (_http_client.js:259:9)
    at emitOne (events.js:77:13)
    at Socket.emit (events.js:169:7)
    at emitErrorNT (net.js:1257:8)
    at doNTCallback2 (node.js:441:9)
    at process._tickCallback (node.js:355:17)

Im totally expecting the proxy to fail and even loudly complain, but I wouldn't expect it to crash the dev-server. If there's some hardening that must be done on the webpack-dev-server itself instead we can get a PR there.

Looks like the default error handler doesn't take care off WebSocket errors... (only http)
I'll have to take a look how to catch those properly.

The main issue is when a WebSocket is established and an error occurs in the scenerio you described; the error-handler currently responds by returning a http 500 response. Which is fine in case of a http(s) error, but it will not work because the writeHead method doesn't exist for WebSockets.
I'm hoping @mjrussell might be more knowledgeable on how to handle WebSocket errors properly, since I'm not really sure.
===
Summing up the information I've encountered:
karma
https://github.com/karma-runner/karma/blob/ae05ea496b8fff1a316387f0b5919de673c5e274/lib/middleware/proxy.js#L60
They just call res.destroy() and log different messages for http(s)- and websocket errors

Destroy any sockets that are currently in use by the agent.
https://nodejs.org/dist/latest-v4.x/docs/api/http.html#http_agent_destroy

w3 / mdn
http://tools.ietf.org/html/rfc6455#page-41
https://www.w3.org/TR/websockets/#closeWebSocket
https://developer.mozilla.org/en-US/docs/Web/API/CloseEvent#Status_codes
close(1006) WebSocket with code 1006.
mdn

A WebSocket server that abruptly closed the connection after successfully completing the opening handshake.

rfc6455

 1006 is a reserved value and MUST NOT be set as a status code in a
 Close control frame by an endpoint.  It is designated for use in
 applications expecting a status code to indicate that the
 connection was closed abnormally, e.g., without sending or
 receiving a Close control frame.

@chimurai nice summary! I would think that the proxy should just let the connection terminate, which I guess would be res.destroy().
I wouldn't expect the middleware to send an extra code or anything in the socket that the server itself didn't send, especially because sometimes people may run their own protocol/codes inside the socket.
The client side should be able to handle this as an unexpected error in the onError callback.
I do also like the extra logging done in the karma proxy log.debug('failed to proxy %s (browser hung up the socket)', req.url)

@mjrussell can you verify if this patch fixes the issue?

@chimurai will do tomorrow, thanks for working this!

I kept the changes tiny, so I won't disturb to functionality too much.
Tried destroying the response onError, but some tests were failing. Need to investigate why it is happening.
Maybe I'll refactor error handling for version 1.0, so it will have proper error handling and logging.

published fix in v0.9.1
https://github.com/chimurai/http-proxy-middleware/releases/tag/v0.9.1

"http-proxy-middleware": "0.19.1",

Same here.
After the server is shut down, the proxy will not crash immediately. Crash after trying to reconnect several times.


extend error message for all services when cannot set user configurat…

…ion options

Some test failures

For some reason, go mod sum was outdated for go-client. That is why some tests were failing, unable to download go-client.


Backend overhaul fetch media by filename

Migrated fetchMediaByFilename to retrofit as per #3026
Also, the classes MwParseResponse and MwParseResult really belong to wikipedia dataclient. Should we submit a pull request or keep the classes within the app?

Codecov Report

Merging #3081 into backend-overhaul will decrease coverage by <.01%.
The diff coverage is 7.69%.

@@                 Coverage Diff                 @@
##           backend-overhaul   #3081      +/-   ##
===================================================
- Coverage               5.5%    5.5%   -0.01%
===================================================
  Files                   268     269       +1
  Lines                 11794   11781      -13
  Branches                961     961
===================================================
- Hits                    649     648       -1
+ Misses                11089   11077      -12
  Partials                 56      56

Impacted Files
Coverage Δ

...rw/commons/mwapi/ApacheHttpClientMediaWikiApi.java
0% <ø> (ø)
:arrow_up:

.../java/fr/free/nrw/commons/review/ReviewHelper.java
86.48% <ø> (ø)
:arrow_up:

...in/java/fr/free/nrw/commons/media/MediaClient.java
66.66% <0%> (-5.34%)
:arrow_down:

...ava/fr/free/nrw/commons/media/MwParseResponse.java
0% <0%> (ø)

.../java/fr/free/nrw/commons/media/MwParseResult.java
0% <0%> (ø)

...n/java/fr/free/nrw/commons/MediaDataExtractor.java
94.73% <100%> (-0.27%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 75f3098...aa21599. Read the comment docs.

Migrated fetchMediaByFilename to retrofit as per #3026
Also, the classes MwParseResponse and MwParseResult really belong to wikipedia dataclient. Should we submit a pull request or keep the classes within the app?

We should submit a PR in the data-client

Migrated fetchMediaByFilename to retrofit as per #3026
Also, the classes MwParseResponse and MwParseResult really belong to wikipedia dataclient. Should we submit a pull request or keep the classes within the app?

They can belong in our app. :)

They can belong in our app. :)

However, most of the migrations I have left require me to add fields to MwQueryResult instead of implementing new classes. It seems somewhat wasteful to copy and paste classes just to change little things. On the other hand, submitting PR's to wikipedia dataclient might make everything take a lot longer.

submitting PR's to wikipedia dataclient might make everything take a lot longer

I don't know what you guys are talking about, but just a tip:
You can easily fork dataclient, modify it as you like, and use that version in build.gradle while waiting for the PRs to be merged. We already do that for some other libraries.

Does Travis failing with this error have anything to do with my PR?
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000eefef000, 38871040, 0) failed; error='Cannot allocate memory' (errno=12)

There is insufficient memory for the Java Runtime Environment to continue.
Native memory allocation (mmap) failed to map 38871040 bytes for committing reserved memory.
An error report file with more information is saved as:
/home/travis/build/commons-app/apps-android-commons/app/hs_err_pid6656.log

Task :app:testProdDebugUnitTest FAILED
FAILURE: Build failed with an exception.

and use that version in build.gradle

How can I do that? Do I deploy my own maven repository?

No need, just open your build.gradle and see how I did for com.github.nicolas-raoul:Quadtree:ac16ea8035bf07, I did not deploy anything, I just pushed to my GitHub :-)

Ooh, that's cool :)

They can belong in our app. :)

However, most of the migrations I have left require me to add fields to MwQueryResult instead of implementing new classes. It seems somewhat wasteful to copy and paste classes just to change little things. On the other hand, submitting PR's to wikipedia dataclient might make everything take a lot longer.

Yes, for any class that already exists in the data client, we should add attribute to the library itself instead of copying the class to our app. But if we are adding a completely new response, we can add the response in the app itself. :)

Please add test cases for the PR. :)

Please add test cases for the PR. :)

Done :)

@ilgazer Can you resolve merge conflicts so that the PR can be merged :)

@ilgazer reminder:)

I'm sorry for the delay, I am starting at a university this fall, and the preparation, the prep exam etc. along with other personal projects took up all my time. The conflicts are resolved :)


Appstudio update test-custom-default-branch-dsdj

Pipelines as Code configuration proposal
To start the PipelineRun, add a new comment with content /ok-to-test
For more detailed information about running a PipelineRun, please refer to Pipelines as Code documentation Running the PipelineRun
To customize the proposed PipelineRuns after merge, please refer to Build Pipeline customization

Pipelines as Code CI/test-custom-default-branch-dsdj-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
5 seconds
init

✅ Succeeded
13 seconds
clone-repository

✅ Succeeded
8 seconds
sast-snyk-check

✅ Succeeded
24 seconds
build-container

✅ Succeeded
49 seconds
clamav-scan

✅ Succeeded
8 seconds
sbom-json-check

✅ Succeeded
46 seconds
inspect-image

✅ Succeeded
9 seconds
clair-scan

✅ Succeeded
9 seconds
deprecated-base-image-check

✅ Succeeded
6 seconds
show-summary

✅ Succeeded
6 seconds
show-sbom


-exportOptionsPlist doesn't support the 'none' method

It seems -exportOptionsPlist doesn't support the 'none' method, and an absense of an explicit method defaults to 'development', so we have to use the older, deprecated style in that case. A bug report has been filed for this.

Yes, I think that's why I skipped this option in the first place. But it makes sense to have a workaround. Thanks @Rheeseyb!

Fixes #3


Import Carbon components

Once the Gatsby project has been created, Carbon React components and styles must be imported to be used in creating a Gatsby site.
This issue is complete when:

[ ] carbon-components-react is a dependency in this repo
[ ] Carbon's stylesheet from Unpkg are imported into the demo page
[ ] Importing Carbon React components is straight forward without having to dive deep into node_modules for import paths (Import path resolving)

Styles are here: https://github.com/ibm-frontend/gatsby-starter-carbon/blob/master/src/layouts/index.scss#L13-L17
We should probably import the Carbon styles from the package, not unpkg.

Each component imports the relevant MD components directly. We'll need to either drop examples (where irrelevant) or update to use Carbon.
https://github.com/ibm-frontend/gatsby-starter-carbon/blob/master/src/components/About/About.jsx#L2-L3


Moving resource loses associated shares

This is using the OC storage I haven't tested with EOS
Works fine on EOS.
Scenario: tests/acceptance/features/apiSharePublicLink2/multilinkSharing.feature:180
Basically:
Steps

Create folder "test"
Create public links for "test"
Rename "test" to "test_renamed"
Check the public link list for "test_renamed"

Expected
Links are still there
Actual
Links disappeared and are likely still linked to the other path ?!
Or maybe a cache issue ?
I thought shares were associated by resource id.
@refs

Leaving open and tagging as belonging to the oc storage.

Leaving open and tagging as belonging to the oc storage.

OC storage is not encouraged anymore.


Add support for "source" document folder

First off, very cool project!
I think it would be extremely useful if there was an option to generate.py to simply include a folder that contains user source files, simliar to what privateGPT/localGPT supports? While the UI is certainly nice, I've found that it can't handle large number of files (in my case 1500+. I was hoping it would attempt to batch upload and then process the all the files ... right now, I've had to break it up into chunks of 25 files and using the UI.
I saw in a few comments there's an API but not sure what would be the minimal client code to simply upload files which are .html. Any assistance would be great on top of the suggested feature enhancement

I think you just mean this, no?
https://github.com/h2oai/h2ogpt/blob/main/docs/README_LangChain.md#database-creation

Thank you @pseudotensor, I hadn't looked at that page thinking it might have been referenced in main README
Once the files are loaded into DB, would I simply backup db_dir_UserData for future uses, say on another system without having to re-process all files?

Yes, in general that's enough for things to work.  If you want links to source files to still work, then the user_path directory should also be copied along the with database.  The Chroma database only keeps the text after parsing, not the original source files that are technically not required for anything except staring at them in UI.

🙏 Thanks for all the help! First time trying out the solution today


[New API] : Food Delivery Service API

Brief Description 🤓
food service API! Build apps for recipes, meals, & restaurants. Access free, public data on ingredients, nutrition, menus.
it will have endpoints like:

Find recipes by ingredients: Enter your ingredients (like eggs, milk, flour) and get recipe ideas.
Search restaurants: Look for restaurants by type of food (Italian, Mexican) near you.
Food details: Look up a specific food (apple, chicken) to see its nutrients (vitamins, calories).
Create meal plans (advanced): Save your meals for the week (breakfast, lunch, dinner).

How will you develop it?🤩

i will use node-express for backend and mongodb database
and check all of the end points
create all required models
will make proper MVC model
and make system smooth

Deployment
No
Select the open source program you are contributing to
Other
Code of Conduct

[X] I follow CONTRIBUTING GUIDELINES of this project.

Hey @Yashgabani845 ! I wanted to inform you that we have closed issue.Thank you for your understanding, and we look forward to your continued engagement with our repository🤗.
In case of any issues, you can contact us on Discord.
Thank you! ❣
Happy Coding! ✨
Will See you soon❣


Move frontend env variables into publicRuntimeConfig

Problem

Because of the way Nuxt's build works, process.env.API_URL in the env.ts file is "baked" at build time, so that it statically references the value at build time:
https://github.com/WordPress/openverse/blob/697f62f01a32cb7fcf2f4a7627650a113cba40da/frontend/src/utils/env.ts#L1
Despite this seeming not to be the case, when you investigate the built code:
// .nuxt/dist/server/server.js
"use strict";
/* harmony export (binding) */ __webpack_require__.d(__webpack_exports__, "a", function() { return env; });
var _a;
const apiUrl = (_a = process.env.API_URL) !== null && _a !== void 0 ? _a : "https://api.openverse.engineering/";

It is actually baked into the Nuxt context:
// .nuxt/dist/server/server.js
async function setContext(app, context) {
  // If context not defined, create it
  if (!app.context) {
    app.context = {
      isStatic: false,
      isDev: false,
      isHMR: false,
      app,
      payload: context.payload,
      error: context.error,
      base: app.router.options.base,
      env: {
        "apiUrl": "https://api.openverse.engineering/",
        "filterStorageKey": "openverse-filter-visibility",
        "savedSearchCount": "4",
        "providerUpdateFrequency": "3600000"
      }
    };

// ...

const createApiService = ({
  baseUrl = "https://api.openverse.engineering/",
  accessToken = undefined,
  isVersioned = true
} = {}) => {

This is complicated by the fact that we do not use Nuxt context to retrieve the default value for baseUrl, meaning we necessarily rely on static definitions, that Nuxt will overwrite.
This documented and intentional Nuxt behaviour: https://v2.nuxt.com/docs/configuration-glossary/configuration-env/#processenv--
Description

To fix this, we could switch to using publicRuntimeConfig, which will read the environment variable at runtime. However, that makes things a big more complicated, because we'd be forced to rely on the Nuxt context $config to retrieve the variable! That's very hard with the way we've written the api-service.ts module and created top-level calls to createApiService that cannot reference runtime context.
While that would be the "correct" solution, we can bypass the static injection of the variable at build time by using process.env.API_URL instead of the nuxt env setting's apiUrl. That would require a one-line change, and define the default where it's used, rather than in the global "env" context.
The following patch would do that:
diff --git a/frontend/src/data/api-service.ts b/frontend/src/data/api-service.ts
index f67bc1024..6797f236e 100644
--- a/frontend/src/data/api-service.ts
+++ b/frontend/src/data/api-service.ts
@@ -92,8 +92,15 @@ export interface ApiService {
   ): Promise<AxiosResponse<T>>
 }

+function getDefaultApiUrl() {
+  const defaultApiUrl = process.env.API_URL || "https://api.openverse.engineering/"
+  return defaultApiUrl.endsWith("/") ? defaultApiUrl : `${defaultApiUrl}/`
+}
+
+const DEFAULT_API_URL = getDefaultApiUrl()
+
 export const createApiService = ({
-  baseUrl = process.env.apiUrl,
+  baseUrl = DEFAULT_API_URL,
   accessToken = undefined,
   isVersioned = true,
 }: ApiServiceConfig = {}): ApiService => {
diff --git a/frontend/src/utils/env.ts b/frontend/src/utils/env.ts
index 8d089ce14..ba439f330 100644
--- a/frontend/src/utils/env.ts
+++ b/frontend/src/utils/env.ts
@@ -1,4 +1,3 @@
-const apiUrl = process.env.API_URL ?? "https://api.openverse.engineering/"

 /**
  * Default environment variables are set on this key. Defaults are fallbacks to existing env vars.
@@ -6,7 +5,6 @@ const apiUrl = process.env.API_URL ?? "https://api.openverse.engineering/"
  * providerUpdateFrequency - how often we should re-fetch provider statistics.
  */
 export const env = {
-  apiUrl: apiUrl.endsWith("/") ? apiUrl : `${apiUrl}/`,
   filterStorageKey: "openverse-filter-visibility",
   savedSearchCount: "4",
   providerUpdateFrequency: `${60 * 60 * 1000}`, // 1 hour

It (a) removes the env.apiUrl definition, which has the "baked in" feature, as described in the above section, and (b) changes the createApiService function to use a new default that reads from the environment at runtime, falling back to the currently established default.
This gets around the build-time injection of the value in the code, and makes it possible to point the built frontend to any API URL desired, just by using the API_URL environment variable.
Alternatives

I mentioned the publicRuntimeConfig alternative. It is better in that it is more "nuxt-y" and would force us into a more maintainable pattern for our API service instance creation.
However, I think the best and most flexible option, would be the following:

Create a Nuxt plugin that defines the API service instances (or, better yet, create a single API client instance based on something like the JS client https://git.sr.ht/~sara/openverse-api-client generates or https://git.sr.ht/~sara/openverse-api-client-js defines)
In that plugin, at request time, resolve the API URL to use based on the following logic, in order:

If the requester has manually set a session variable apiUrl, use that.
Otherwise, if process.env.API_URL is defined, use that.
Otherwise, use https://api.openverse.engineering/ as the absolute default.

This is the best solution because it allows us to change the API url at any time we want for any environment, in order to test whatever API url we like. If we want to point the production frontend to the staging API, we can do so for ourselves without affecting anyone else. Likewise, if we want to default the staging frontend to point to the staging API, we can do that, while allowing developers to easily point the staging frontend to the production API by setting the session variable. Or if we don't want that default, we can individually point the staging frontend to the staging API whenver we want.
It's tremendously flexible, but would require a significant overhaul in our API request handling on the frontend. It's something we should do, but difficult to argue needs to be done, considering the effort it would take, and the fact that there is a usable workaround the achieves the main, immediate needs.

I'd prefer the second solution because we have discussed moving away from the api service anyway.
Using the nuxt context is problematic because it's not always available when we need it. Namely, on the server, when we do async fetch or call a function that makes an api request inside the async fetch or in the middleware. At least, that was the problem I had in the Nuxt 3 branch.

This has been fixed in #4705


METRON-658:  Updated Grammar to Handle More Uses of in/not in Expressions

This PR allows for the use of in/not in expressions in more places. For example, the following expressions now work:
('grok' not in 'foobar') == true
'grok' not in ('foobar' == true)
'grok' not in 'foobar' == true
('grok' not in 'foobar') AND 1 < 2

It is important to note that github's diff of Stellar.g4 isn't very nice. A vast majority of the changes are just whitespace.

Is there a way to split out the core changes that you made from the whitespace changes?  Were the whitespace changes required?  Or should those be a separate "clean-up" PR?

@nickwallen personally I don't think it should be a separate PR. To me it is just one of those things that should be cleaned up as we go. My reasoning is that I think we should encourage clean up as we go with minimal overhead. However, I understand that it is difficult to read on github. I do agree that I could have been more conscious of the way I made the commits though. I changed this PR to have two commits,1 with the changes and 1 with the cleanup. If this is still hard to review, I can split them into two PRs.

I'm good with this split as separate commits.

+1 by inspection, good contribution.  I like the refactoring of the grammar too.  Stellar is getting cleaner and cleaner with every contribution like this.


build and upload both arm64 and amd64 images

Hi guys! Mac user here 👋
Currently it's not possible to run the Quinn image on my machine:
❯ docker run stammw/quinn-interop
Unable to find image 'stammw/quinn-interop:latest' locally
latest: Pulling from stammw/quinn-interop
docker: no matching manifest for linux/arm64/v8 in the manifest list entries.
See 'docker run --help'.

This means I can't test against Quinn when running the interop runner locally.
With this PR, a multi-platform build for both arm64 and amd64 is performed. I tested this workflow in my fork, and here's how the result looks: https://hub.docker.com/layers/martenseemann/quinn-interop
The only downside I can see is that the build now takes really long, almost 25 minutes(see here), compared to 2 minutes before. Coming from Go, I find this quite surprising. I suspect this is because the Rust compiler is super slow, and even more so when cross-compiling. I don't think it ultimately matters if the CI job is taking a few minutes more though.

Thanks! That compile time is definitely longer than it should be. Is docker running the build itself inside qemu, rather than properly cross-compiling? I don't have much grasp of how docker machinery works, but I can imagine emulating the CPU would make things very slow indeed.
I agree it's not a blocker in any case, so I'm happy to merge if you're confident this is the right approach.

This is exactly what quic-go is doing. I'm not sure if there's a better (faster?) way of doing this, but this approach works.

Running the compiler on a native CPU should be much faster, then. Good to know we can explore that if buildtime becomes too annoying.


Add ColorPalette support

Adds support for loading and saving ColorPalette assets. There is only one version of color palettes so this should be universal across RB1 and later. These seem to mostly be used for allowing the player to choose colors for instruments, clothes, etc. in character creation, but they are also used on the crowd models in certain games to add variation (see TBRB crowd milos) and likely in other places too. I plan on eventually adding support for tinting textures to palettes in the preview UI, but that is for a future PR.
I tried to stick to the coding style/naming convention present in the rest of grim but feel free to point out anything you would want changed here and I will adjust accordingly.

Thanks. Looks good to me!