UnicodeEncodeError on windows

Got the UnicodeEncodeError on Windows for latest master branch (at the moment 537dc51880ee0cc636eb721b1b8fe9cab003bff6):
PS E:\models\Stable-diffusion\sdxl\realism-engine-sdxl> civitdl --verbose --limit-rate 5M https://civitai.com/models/152525?modelVersionId=293240 .
Active code page: 65001
Arguments:  {'source_strings': ['https://civitai.com/models/152525?modelVersionId=293240'], 'rootdir': '.', 'sorter': 'basic', 'max_images': 3, 'api_key': '', 'with_prompt': True, 'limit_rate': '5M', 'retry_count': 3, 'pause_time': 3.0, 'cache_mode': '1', 'model_overwrite': False, 'verbose': True}
Chosen Sorter Description:  This is useful when you only just want all of the lora/checkpoint/etc. in the same root directory. Metadata + images are downloaded inside a directory under the parent directory.
Requesting model metadata.
Metadata API Request URL: https://civitai.com/api/v1/models/152525
Finished requesting model metadata.
Requesting model metadata.
Metadata API Request URL: https://civitai.com/api/v1/model-versions/293240
Finished requesting model metadata.
Now downloading "Realism Engine SDXL"...
            - Model ID: 152525
            - Version ID: 293240

Preparing to send download model request...
Model Download API URL: https://civitai.com/api/download/models/293240
---------
Traceback (most recent call last):
  File "C:\Users\username\projects\civitdl\venv\Lib\site-packages\civitdl\batch\batch_download.py", line 32, in batch_download
    download_model(
  File "C:\Users\username\projects\civitdl\venv\Lib\site-packages\civitdl\batch\_get_model.py", line 257, in download_model
    _download_metadata(sorter_data.metadata_dir_path, metadata)
  File "C:\Users\username\projects\civitdl\venv\Lib\site-packages\civitdl\batch\_get_model.py", line 156, in _download_metadata
    write_to_file(model_dict_path, [dumps(
  File "C:\Users\username\projects\civitdl\venv\Lib\site-packages\helpers\utils.py", line 171, in write_to_file
    raise e
  File "C:\Users\username\projects\civitdl\venv\Lib\site-packages\helpers\utils.py", line 165, in write_to_file
    write_contents(file, content_chunks,
  File "C:\Users\username\projects\civitdl\venv\Lib\site-packages\helpers\utils.py", line 103, in write_contents
    file.write(content)
  File "C:\Users\username\AppData\Local\Programs\Python\Python311\Lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode characters in position 2923-2924: character maps to <undefined>
'charmap' codec can't encode characters in position 2923-2924: character maps to <undefined>

---------
Pausing for 3 seconds...
Waking up!
PS E:\models\Stable-diffusion\sdxl\realism-engine-sdxl>

Changing the environment to UTF-8 (chcp 65001 for windows) did not fix this.
Maybe it'd be a good idea to write everything as downloaded, in binary mode?
Or even better, encode all text files to utf8 by default?

Hi, I managed to fix the issue in my windows virtual machine in bug/utf-write branch just now.
For now I have merged it into master since it is a pretty critical bug. Can you give it a go? New version is v2.0.1 on pypi :)

So, just pulled the master branch and installed from git with pip install --upgrade .
Now sample images and metadata are downloaded
New error is division by 0, so I suppose that this fix worked and we've got to the next (probably separate) error :D
PS E:\models\Stable-diffusion\sdxl\realism-engine-sdxl> civitdl --verbose --limit-rate 5M https://civitai.com/models/152525?modelVersionId=293240 .
Active code page: 65001
Arguments:  {'source_strings': ['https://civitai.com/models/152525?modelVersionId=293240'], 'rootdir': '.', 'sorter': 'basic', 'max_images': 3, 'api_key': '', 'with_prompt': True, 'limit_rate': '5M', 'retry_count': 3, 'pause_time': 3.0, 'cache_mode': '1', 'model_overwrite': False, 'verbose': True}
Chosen Sorter Description:  This is useful when you only just want all of the lora/checkpoint/etc. in the same root directory. Metadata + images are downloaded inside a directory under the parent directory.
Requesting model metadata.
Metadata API Request URL: https://civitai.com/api/v1/models/152525
Finished requesting model metadata.
Requesting model metadata.
Metadata API Request URL: https://civitai.com/api/v1/model-versions/293240
Finished requesting model metadata.
Now downloading "Realism Engine SDXL"...
            - Model ID: 152525
            - Version ID: 293240

Preparing to send download model request...
Model Download API URL: https://civitai.com/api/download/models/293240
Now requesting images...
Finished requesting images...
Images: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3.00/3.00 [00:00<00:00, 285iB/s]
---------
Traceback (most recent call last):
  File "C:\Users\username\projects\civitdl\venv\Lib\site-packages\civitdl\batch\batch_download.py", line 32, in batch_download
    download_model(
  File "C:\Users\username\projects\civitdl\venv\Lib\site-packages\civitdl\batch\_get_model.py", line 268, in download_model
    _download_prompts(sorter_data.prompt_dir_path, prompts_basenames,
  File "C:\Users\username\projects\civitdl\venv\Lib\site-packages\civitdl\batch\_get_model.py", line 146, in _download_prompts
    write_to_files(dirpath, basenames, [dumps(
  File "C:\Users\username\projects\civitdl\venv\Lib\site-packages\helpers\utils.py", line 182, in write_to_files
    write_contents(file, content_chunks)
  File "C:\Users\username\projects\civitdl\venv\Lib\site-packages\helpers\utils.py", line 107, in write_contents
    speed = bytes_downloaded / download_time
            ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~
ZeroDivisionError: float division by zero
float division by zero

---------
Pausing for 3 seconds...
Waking up!
PS E:\models\Stable-diffusion\sdxl\realism-engine-sdxl> ls ".\Realism Engine SDXL\extra_data-vid_293240\"

    Directory: E:\models\Stable-diffusion\sdxl\realism-engine-sdxl\Realism Engine SDXL\extra_data-vid_293240

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---          24.01.2024    16:14          25209 5342532.jpeg
-a---          24.01.2024    16:14          42749 5344273.jpeg
-a---          24.01.2024    16:14             16 5344273.json
-a---          24.01.2024    16:14          32632 5344319.jpeg
-a---          24.01.2024    16:14         128759 model_dict-mid_152525-vid_293240.json

PS E:\models\Stable-diffusion\sdxl\realism-engine-sdxl>

Your computer/network is pretty fast :)
I made somes changes so that when download_time is 0, it won't divide (branch is bug/download-time).
Would you be able to check that out?
# for convenience reasons
git pull
git checkout bug/download-time
pip install .

Apparently unicode bug has been in windows version of civitdl in v1 of the script... I can't believe I never found out about this bug...

Sorry, took some time to return to my desktop.

branch is bug/download-time

It's working, everything looks correct now! Thank you.

Your computer/network is pretty fast :)

I'd suspect time resolution on Python Windows vs Python Linux as it looks like error was triggered in pretty small file, 5344273.json accompanying the sample image was broken before - just 16 bytes was saved in the listing above when division by 0 occured and 1258 bytes now (still smaller than typical ethernet MTU so probably was downloaded in single packet but can't be sure) after branch bug/download-time run through all of metadata and sample images and started the download of the actual 6.5Gb checkpoint:
PS E:\models\Stable-diffusion\sdxl\realism-engine-sdxl> ls ".\Realism Engine SDXL\extra_data-vid_293240\"

    Directory: E:\models\Stable-diffusion\sdxl\realism-engine-sdxl\Realism Engine SDXL\extra_data-vid_293240
Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---          24.01.2024    18:49          25209 5342532.jpeg
-a---          24.01.2024    18:49           1609 5342532.json
-a---          24.01.2024    18:49          42749 5344273.jpeg
-a---          24.01.2024    18:49           1258 5344273.json
-a---          24.01.2024    18:49          32632 5344319.jpeg
-a---          24.01.2024    18:49           1648 5344319.json
-a---          24.01.2024    18:49         128759 model_dict-mid_152525-vid_293240.json

Apparently unicode bug has been in windows version of civitdl in v1 of the script... I can't believe I never found out about this bug...

There's some pretty broken HTML in Realism Engine SDXL model description:
F7ZE0hKIGXtkR4qfjKEw￼￼You\">You can also add hires_fix_refiner_pass

Apparently unicode bug has been in windows version of civitdl in v1 of the script... I can't believe I never found out about this bug...

There's some pretty broken HTML in Realism Engine SDXL model description:
F7ZE0hKIGXtkR4qfjKEw￼￼You\">You can also add hires_fix_refiner_pass

As can be seen there are two U+FFFC characters before You\">

Hmm looking at the html in civitai.com/models/152525, it also gave the same broke url with two U+FFFC for some reason. I tried changing to utf-32 encoding for the metadata, but still the same issue.
I think this link / anchor element was suppose to display a url or image that leads to some kind of example image on how add hires_fix_refiner_pass to some quick list setting.

<a target=\"_blank\" rel=\"ugc\" href=\"https://private-user-images.githubusercontent.com/655287/268127943-b3b3177b-29f5-4013-a984-33e19c8bc13b.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MDQzOTUwMTMsIm5iZiI6MTcwNDM5NDcxMywicGF0aCI6Ii82NTUyODcvMjY4MTI3OTQzLWIzYjMxNzdiLTI5ZjUtNDAxMy1hOTg0LTMzZTE5YzhiYzEzYi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwMTA0JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDEwNFQxODU4MzNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT02NGFlYzE0YjNmMmY5ZDdjYzYyNjE0MDY1YjUyOTMyYTE5YmY1YWM5N2NlNjdhOWM1NTA3YmExYmI4OTY0NGQ1JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.oH2ZYe6EYUC84DKAa0ovqnxF7ZE0hKIGXtkR4qfjKEw￼￼You\">You

So looks most probably like the result of partially HTML formatted copy/paste of the description by model author, and that's the reason why this error never surfaced before.
Although I'm slightly surprised too that no one tried to download some models with e.g. Japanese or Chinese language descriptions on the Windows box with the default Latin locale.

And I suppose there's nothing to fix here on client side?

Unfortunately so if the author copy pasted their html then only the author themselves probably have the correct html.

Unless I am missing anything else here

Description is downloaded as is and is probably best left for web browsers to interpret as is.
If someone would like to process the JSON file further it's simple to pipe the HTML parts through html-tidy or something similar.

Will piping the html part will fix the link?
I was curious what html-tidy did so I tried to run the description of Realism Engine SDXL on this site (not sure if this site does what html-tidy is supposed to do).
Don't know if it actually looked prettier.
Before:

Version 3.0 of the Realism Engine

In this version we improved the skin realism, the eyes. We also improved the Male anatomy.

Keep in mind that this is not a NSFW model. For that I recommend SDXXXL.

Introducing Version 2.0 of the Realism Engine

We are excited to announce that Version 2.0 of the Realism Engine is now available, featuring significant enhancements.

This updated model exhibits superior prompt responsiveness and offers markedly improved overall coherence, including more facial expressions, a greater variety of faces, more poses, and improved hands.

Additionally, we have addressed several quirks from the previous version, such as the challenges in generating nighttime scenes and the issue where increasing the CFG Scale enhanced color vividness without proportionally adjusting contrast. While being capable of producing NSFW content it is less likely to produce it without asking.

This version include the VAE to help newcomers with their setup.

Happy Prompting

Good news for the fans of the original Realism Engine

Realism Engine SDXL is here

Version 1.0 is officially out.

Here's the recommended setting for Auto1111

Sampler: DPM++ 2S a,

CFG scale range: 5-9,

Hires sampler: DPM++ SDE Karras,

Hires upscaler: ESRGAN_4x,

Refiner switch at: 0.9 (ON THE FIRST PASS ONLY)

Look for this in Setting -> Stable Diffusion
<a target=\"_blank\" rel=\"ugc\" href=\"https://private-user-images.githubusercontent.com/655287/268127943-b3b3177b-29f5-4013-a984-33e19c8bc13b.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MDQzOTUwMTMsIm5iZiI6MTcwNDM5NDcxMywicGF0aCI6Ii82NTUyODcvMjY4MTI3OTQzLWIzYjMxNzdiLTI5ZjUtNDAxMy1hOTg0LTMzZTE5YzhiYzEzYi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwMTA0JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDEwNFQxODU4MzNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT02NGFlYzE0YjNmMmY5ZDdjYzYyNjE0MDY1YjUyOTMyYTE5YmY1YWM5N2NlNjdhOWM1NTA3YmExYmI4OTY0NGQ1JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.oH2ZYe6EYUC84DKAa0ovqnxF7ZE0hKIGXtkR4qfjKEw￼￼You\">You can also add hires_fix_refiner_pass to you quick list setting to have it on top of you Ui

While this model hit some of the key goals I was reaching for, it will continue to be trained to fix the weaknesses.

I appreciate all the good feedback from the community.

After:

<h2 id=\"heading-167\"><span style=\"color:rgb(34, 139, 230)\">Version 3.0 of the Realism Engine</span></h2><p>In this version we improved the skin realism, the eyes. We also improved the Male anatomy.</p><p>Keep in mind that this is not a NSFW model. For that I recommend SDXXXL.</p><p></p><h2 id=\"heading-167\"><span style=\"color:rgb(34, 139, 230)\">Introducing Version 2.0 of the Realism Engine</span></h2><p>We are excited to announce that Version 2.0 of the Realism Engine is now available, featuring significant enhancements.</p><p></p><p>This updated model exhibits superior prompt responsiveness and offers markedly improved overall coherence, including more facial expressions, a greater variety of faces, more poses, and improved hands.</p><p></p><p>Additionally, we have addressed several quirks from the previous version, such as the challenges in generating nighttime scenes and the issue where increasing the CFG Scale enhanced color vividness without proportionally adjusting contrast. While being capable of producing NSFW content it is less likely to produce it without asking.</p><p></p><p><em>This version include the VAE to help newcomers with their setup.</em></p><p></p><p><strong>Happy Prompting</strong></p><p></p><h3 id=\"heading-13\"><span style=\"color:rgb(34, 139, 230)\">Good news for the fans of the original Realism Engine</span></h3><p><strong>Realism Engine SDXL is here</strong></p><p></p><p>Version 1.0 is officially out.</p><p></p><p><strong>Here's the recommended setting for Auto1111</strong></p><p>Sampler: DPM++ 2S a,</p><p>CFG scale range: 5-9,</p><p>Hires sampler: DPM++ SDE Karras,</p><p>Hires upscaler: ESRGAN_4x,</p><p>Refiner switch at: 0.9 (<strong>ON THE FIRST PASS ONLY)</strong></p><p><img src=\"https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/51dd7ec2-b64d-4c63-b7e0-7aea6f4413c3/width=525/51dd7ec2-b64d-4c63-b7e0-7aea6f4413c3.jpeg\" /></p><p><span style=\"color:rgb(193, 194, 197)\">Look for this in Setting -&gt; Stable Diffusion</span><br /><a target=\"_blank\" rel=\"ugc\" href=\"https://private-user-images.githubusercontent.com/655287/268127943-b3b3177b-29f5-4013-a984-33e19c8bc13b.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MDQzOTUwMTMsIm5iZiI6MTcwNDM5NDcxMywicGF0aCI6Ii82NTUyODcvMjY4MTI3OTQzLWIzYjMxNzdiLTI5ZjUtNDAxMy1hOTg0LTMzZTE5YzhiYzEzYi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwMTA0JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDEwNFQxODU4MzNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT02NGFlYzE0YjNmMmY5ZDdjYzYyNjE0MDY1YjUyOTMyYTE5YmY1YWM5N2NlNjdhOWM1NTA3YmExYmI4OTY0NGQ1JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.oH2ZYe6EYUC84DKAa0ovqnxF7ZE0hKIGXtkR4qfjKEw￼￼You\"><span style=\"color:rgb(193, 194, 197)\">You</span></a><span style=\"color:rgb(193, 194, 197)\"> can also add hires_fix_refiner_pass to you quick list setting to have it on top of you Ui</span></p><p></p><p>While this model hit some of the key goals I was reaching for, it will continue to be trained to fix the weaknesses.</p><p></p><p>I appreciate all the good feedback from the community.</p>

I don't think you miss anything, html content is probably the result of failed (or partially successful) copy/paste.
I think that this tool should not try to fix the errors on civitai site.
Re: tidy, it's an html (and xml if I recall correctly) sanitizer.
Here is the source https://github.com/htacg/tidy-html5
Docs are here: https://www.html-tidy.org/documentation/
That was just an example of if I would e.g. sometime in the future need a nice clean text description extracted from the json metadata then it's much simpler to do this:
$ sudo apt install jq tidy html2text
$ jq .description ./extra_data-vid_293240/model_dict-mid_152525-vid_293240.json \
 | tidy \
   -indent \
   --indent-spaces 2 \
   -quiet \
   --tidy-mark no \
   2>/dev/null \
   | html2text

Than to try to process all possible variants during download?

It does look better. Maybe the program could also create a description.txt file that is sanitized after download.

Than to try to process all possible variants during download?

What do you mean by variants?

Keeping this open in case I missed a bug


Problems translating Java code to use with SharpNL

I'm trying to port https://github.com/mlehman/nlp-intent-toolkit to C#, but some objects from opennlp seems not to be available in SharpNL.
For example:
var combinedDocumentSampleStream = ObjectStreamUtils.createObjectStream(categoryStreams.ToArray());
There are no ObjectStreamUtils available in SharpNL =\

The commit works fine! AppVeyor had an issue, that's why the build did not work!
:+1:


Allow usage of custom formatters

Added fmt_args and allowed fmt_exe to handle other formatters such as scadformat: https://github.com/hugheaves/scadformat

handled typos
addressed clippy warnings
added documentation for new formatting

If it is better，we should migrate to this dedicated formatter, clang-format is just a workaround.

Yes scadformat is better in my opinion

Could you please remove the code related to clang-format and update document ?

Are you sure you want to completely remove support for clang-format? scad-format looks like it lacks any configurability (even for something basic like line length), and there has not been any commits to main for 8 months.

@Lenbok yeah scad-format has no configurability but I still found it better than using clang-format, barring any alternative suggestions I would suggest https://topiary.tweag.io/ be looked into, I've used tree-sitter-opecad to get topiary fmt to run on my file:
https://github.com/mkatychev/topiary/blob/feat/openscad/topiary-queries/queries/openscad.scm
More rules to be added but the design doc provides an easy in:
https://github.com/tweag/topiary/blob/main/README.md#design

@Lenbok @mkatychev Ok, we can keep both formatter for a while, until scad-format is more refined.

@Lenbok @mkatychev Ok, we can keep both formatter for a while, until scad-format is more refined.

We can note in the documentation that scad-format is recommended for use.

@Leathong  I have built my own formatter that is quite customizable:
https://github.com/mkatychev/topiary/blob/feat/openscad/topiary-queries/queries/openscad.scm
I have not fleshed out all the rules but it can compile and format dynamically. This will be better than clang-format or scad-format but I need time/help to flesh out the rules.


Loading module throws errors (webpack)

How to reproduce:

Basic webpack set up
Importing exceljs as import ExcelJS from 'exceljs/dist/es5/exceljs.browser';
Create new workbook like var wb = new ExcelJS.Workbook();

I get the error that module 'fs' was not found - which gets loaded straight away at the beginning of almost every file.

My bad,
I should have set the node option in webpack config.
{
  node: { fs: 'empty' },
}


fix for issue-40

fix for issue-40: generate actor ids deterministically so that if the web service is restarted we end up with same set of visual objects.

Looks good to me.


ValueError: invalid vertex: 3; condense-edges.py error.

Hi,
I was trying to run Superpang on a 50 MAGs (supposedly 1 species) . It returned the following error:
Traceback (most recent call last):
  File "/lustre/BIF/nobackup/hrab001/micromamba/envs/superpang/lib/python3.8/site-packages/superpang/scripts/condense-edges.py", line 278, in <module>
    main(parse_args())
  File "/lustre/BIF/nobackup/hrab001/micromamba/envs/superpang/lib/python3.8/site-packages/superpang/scripts/condense-edges.py", line 222, in main
    predecessors, successors = G2dicts(GS, name2vertex, vertex2name)
  File "/lustre/BIF/nobackup/hrab001/micromamba/envs/superpang/lib/python3.8/site-packages/superpang/scripts/condense-edges.py", line 238, in G2dicts
    for pv in GS.get_in_neighbors(v):
  File "/lustre/BIF/nobackup/hrab001/micromamba/envs/superpang/lib/python3.8/site-packages/graph_tool/__init__.py", line 2441, in get_in_neighbors
    vertices = libcore.get_in_neighbors_list(self.__graph, int(v),
ValueError: invalid vertex: 3

There was an error running condense-edges.py. Please open an issue

Can you advice on resolving it?
Best,
Pavlo

Is this happening with the latest version of SuperPang?
Can you share those 50 genomes with me so I can try to replicate?

I just released version 1.3.0.post1, which adds compatibility with newer graph-tool versions.


Revert "Use the puppet from $PATH"

Reverts alphagov/govuk-puppet#4247
Breaks the puppet run for the dev vm with the error
"Error: Could not find a suitable provider for augeas"

\cc @boffbowsh

@alexmuller what should we be using instead? At present a new VM isn't bootstrappable from start it seems, as the original /usr/bin/puppet doesn't exist.

Hm, I'm a little confused because I can't reproduce that behaviour. If I destroy my VM and provision a new one, Puppet is installed because the puppet-common package is on the base box as part of the packer build process:
vagrant@development:~$ apt-cache policy puppet-common | grep Installed
  Installed: 3.8.5-1puppetlabs1
vagrant@development:~$ dpkg -L puppet-common | grep /usr/bin/puppet
/usr/bin/puppet

Which means that the initial Puppet run is fine.

There was a bug which I fixed in 801e82889b3b0c0c6eb75fbe181bd19990aa8e92 where some boxes didn't have Puppet installed, maybe that's the problem you're seeing?

Oh, I've not experienced this myself, it was @tvararu that had the issue. If it's not an issue normally, I'll merge this revert.

I had issues provisioning my machine with /usr/bin/puppet as the path.
Slack ate my messages, but it was along the lines of (recounted from memory):
mac$ vagrant destroy
mac$ vagrant up --provision
ERROR: /usr/bin/puppet does not exist
mac$ vagrant ssh
dev$ which puppet
/usr/local/bin/puppet

# replace path with /usr/local/bin/puppet in tools/puppet-apply

mac$ vagrant provision
Works fine, once. (timed out on some Go stuff, let's run it again)
mac$ vagrant provision
ERROR: Could not find a suitable provider for augeas
mac$ vagrant ssh
dev$ which puppet
/usr/lib/rbenv/shims/puppet

# replace path with /usr/lib/rbenv/shims/puppet in tools/puppet-apply

mac$ vagrant provision
I can't remember if this worked or not.

So yeah, it would seem that the box didn't have puppet installed correctly to begin with.


Windows 10 Support

All my code runs on an Ubuntu based OS, however I do the majority of my prototyping on a Windows machine. I am currently unable to use said Windows machine, but I am wondering what the kind of support is like if all I am doing it fetching a webpage with dryscrape and then parsing with BeautifulSoup4. Can I expect problems?

Windows 10 is not officially supported. Feel free to try it out and let me know how it goes. I imagine that your best bet is the new Ubuntu subsystem.


push container image to private harbor registry failed

What happened:
I update virt-launcher source code, and make  bazel-build-images success.
I set DOCKER_PREFIX=harbor-local.xxx.com/kubevirt , but make push failed.
make push
hack/dockerized "export BUILD_ARCH=x86_64 && hack/bazel-fmt.sh && DOCKER_PREFIX=harbor-local.xxx.com/kubevirt DOCKER_TAG=v1.2.0-5 DOCKER_TAG_ALT= IMAGE_PREFIX= IMAGE_PREFIX_ALT= KUBEVIRT_PROVIDER= PUSH_TARGETS='virt-launcher' ./hack/multi-arch.sh push-images"
go version go1.22.2 linux/amd64

go version go1.22.2 linux/amd64
Sandbox is up to date
INFO: Build option --define has changed, discarding analysis cache.
INFO: Analyzed target //vendor/mvdan.cc/sh/v3/cmd/shfmt:shfmt (0 packages loaded, 11314 targets configured).
INFO: Found 1 target...
Target //vendor/mvdan.cc/sh/v3/cmd/shfmt:shfmt up-to-date:
  bazel-bin/vendor/mvdan.cc/sh/v3/cmd/shfmt/shfmt_/shfmt
INFO: Elapsed time: 1.143s, Critical Path: 0.11s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/vendor/mvdan.cc/sh/v3/cmd/shfmt/shfmt_/shfmt -i 4 -w /root/go/src/kubevirt.io/kubevirt/hack/INFO: Build completed successfully, 1 total action
INFO: Analyzed target //:gazelle (0 packages loaded, 1132 targets configured).
INFO: Found 1 target...
Target //:gazelle up-to-date:
  bazel-bin/gazelle-runner.bash
  bazel-bin/gazelle
INFO: Elapsed time: 0.598s, Critical Path: 0.09s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
INFO: Analyzed target //:goimports (0 packages loaded, 98 targets configured).
INFO: Found 1 target...
Target //:goimports up-to-date:
  bazel-bin/goimports.bash
INFO: Elapsed time: 0.538s, Critical Path: 0.08s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
INFO: Analyzed target //:buildifier (0 packages loaded, 579 targets configured).
INFO: Found 1 target...
Target //:buildifier up-to-date:
  bazel-bin/buildifier.bash
INFO: Elapsed time: 0.641s, Critical Path: 0.09s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
Sandbox is up to date
INFO: Build option --define has changed, discarding analysis cache.
INFO: Analyzed target //:push-virt-launcher (0 packages loaded, 18455 targets configured).
INFO: Found 1 target...
Target //:push-virt-launcher up-to-date:
  bazel-bin/push-virt-launcher.digest
  bazel-bin/push-virt-launcher
INFO: Elapsed time: 2.157s, Critical Path: 0.14s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
2024/08/13 15:13:19 Error pushing image to harbor-local.xxx.com/kubevirt/virt-launcher:v1.2.0-5: unable to push image to harbor-local.xxx.com/kubevirt/virt-launcher:v1.2.0-5: Get "https://harbor-local.xxx.com/v2/": tls: failed to verify certificate: x509: certificate signed by unknown authority

In kubevirt-bazel-server docker:
docker ps
CONTAINER ID   IMAGE                                           COMMAND                  CREATED          STATUS          PORTS     NAMES
1485f085a5ce   quay.io/kubevirt/builder:2408081318-d0866a61b   "/entrypoint.sh hack…"   14 minutes ago   Up 14 minutes             kubevirt-bazel-server
(base) ➜  ~ docker exec -it 1485f085a5ce bash
go version go1.22.2 linux/amd64
[root@docker-desktop kubevirt]# cat ~/.docker/config.json
{
	"auths": {
		"harbor-local.xxx.com": {
			"auth": "bXV5ZTp6Z3lDbG91ZDIwMjBAQkpNWQ=="
		}
	},
	"currentContext": "desktop-linux",
	"plugins": {
		"debug": {
			"hooks": "exec"
		},
		"scout": {
			"hooks": "pull,buildx build"
		}
	},
	"features": {
		"hooks": "true"
	}
}[root@docker-desktop kubevirt]# cat ~/.docker/daemon.json
{
 "builder": {
  "gc": {
   "defaultKeepStorage": "20GB",
   "enabled": true
  }
 },
 "experimental": false,
 "insecure-registries": ["harbor-local.xxx.com"]
}

What you expected to happen:
push container image to local harbor registry success.
Environment:

KubeVirt version (use virtctl version): 1.2.0
Kubernetes version (use kubectl version): 1.27.6
OS (e.g. from /etc/os-release): centos8

works fine, i am using
kubectl version
Client Version: v1.30.2

@wywself is it a private registry? From the error message it seems that the certificate of the registry is self-signed. So, this isn't really a bug in kubevirt, but you need to configure your setup correctly.

@wywself is it a private registry? From the error message it seems that the certificate of the registry is self-signed. So, this isn't really a bug in kubevirt, but you need to configure your setup correctly.

yes, I tried to set insecure-registries in ~/.docker/daemon.json,  but not work in kubevirt-bazel-server container.
cat ~/.docker/daemon.json
{
 "builder": {
  "gc": {
   "defaultKeepStorage": "20GB",
   "enabled": true
  }
 },
 "experimental": false,
 "insecure-registries": ["harbor-local.xxx.com"]
}

we don't use docker but bazel for pushing the image, so this isn't going to work


this is mad

just some attempt of mine stealing and learning...
now i need to learn how to filter out the noise

🤘 🤘 🤘 ⚡ ⚡ ⚡


update react-testing-library and remove toBeInDom

Issue Number
#431
Overview of PR
remove deprecated toBeInDom
Provide an overview of the changes in this PR.

remove toBeInDom and replace it with toBeInDocument
update react-testing-library to version 5

@Ilaiwi Thanks for this! I will no longer have to see tests as fails locally.


Streaming API for LOBs

Fixes Issue #1, hopefully.
Not full, as only sequential reading and writing is implemented.
But at least it is simple and understandable.
Maybe documentation needs some adjustment!

Simple and understandable sounds good


Fixed README header rendering

Four headers were missing a space after the hashes and are not rendered in markdown then making a less easy way of viewing.

I see this was fixed in #27


Create release to include -legacy support

@heaths,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

:white_check_mark: Build vswhere 1.0.50+g1c4b3be64c completed (commit https://github.com/Microsoft/vswhere/commit/1c4b3be64c by @heaths)


Update the version number and release notes for v2.11.0

Description

Closes #3688
Link to documentation: https://esmvaltool--3695.org.readthedocs.build/en/3695/changelog.html

Before you get started

[x] ☝ Create an issue to discuss what you are going to do

Checklist
It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

[x] 🛠 This pull request has a descriptive title
[x] 🛠 Code is written according to the code quality guidelines
[x] 🛠 Documentation is available
[x] 🛠 All checks below this pull request were successful

To help with the number of pull requests:

🙏 We kindly ask you to review two other open pull requests in this repository

I have just reviewed ESMValTool discussion: ESMValTool v2.11.0 release and I hadn't updated the version of ESMValCore in the environment(_osx).yml files, so I have done that in 191e501b1 😊

You'll need to unpin the release candidate channel to make that work.

I have update the release date and the zenodo file, so once the checks have passed then I think this is ready! 🥳


re-add npm test

once https://github.com/workbenchapp/solana-workbench/pull/52 lands we need to make some decisions (and do some hacking) about IPC, then we'll have the foundations to get the test(s) running

#56 landed, and then we dao-netes'd . still needs work


[FEA] Enable CUDA UVM (Unified/Universal) Memory Support from Numba

Feature request
NVIDIA's embedded GPU line (TX2, Xavier, Nano, to name a few) feature a shared memory space between CPU and GPU. Typically handled in CUDA with unified memory, data access between host and device involves a zero-copy.
On these embedded platforms (I'm using the TX2), I'm seeing different pointer addresses when analyzing the pointer from __array_interface__ and __cuda_array_interface__.
>>> import numpy as np
>>> import cupy as cp
>>> from numba import cuda
>>>
>>> arr = np.arange(100)
>>>
>>> arr.__array_interface__['data']
(94169420899168, False)
>>>
>>> grr = cuda.to_device(arr)
>>>
>>> grr.__cuda_array_interface__['data']
(140351520112640, False)

numba.cuda.to_device() doesn't seem to have awareness that the memory allocated via np.arange(), in this case, is also accessible to the GPU.
Instead, we see a performance overhead due to a physical memory copy and different pointer address.
Test Setup
NVIDIA Jetson TX2
Numba 0.45 via conda4aarch64

Seems like we could look at the device properties to determine whether a copy is needed.  This will also affect the CUDA Python kernels, as Numba automatically calls to_device() when host arrays are passed to the GPU for processing.

It would be nice if the property we use to detect this situation also worked when the future support for automatic paging of host memory to the device appears in Linux x86_64.

Also, do you know if zero-copy works on Linux POWER8/9 when the GPU is connected to the CPU over NVLink?  There may be multiple platforms affected by this, now that I think about it.

I don't think POWER CPU/GPU is a shared memory space via NVLINK, but don't quote me on that one. What you do get is massively increased BW over a PCIe Gen3 data movement. Each NVLINK lane is 50GB/s bi-directional.
Happy to see some great engagement on this feature request, @seibert!

CC @andih, you may be interested.

Did some hunting, and I think I was remembering this talk from GTC:
http://on-demand.gputechconf.com/gtc/2018/presentation/s8430-everything-you-need-to-know-about-unified-memory.pdf
Specifically slide 47.  However the requirements are pretty strict: POWER9, Volta, and NVLINK2.  Also not sure if the Address Translation Service is enabled by default.

Looping in @kkraus14
I can do some digging too, but unified memory is certainly also supported on Pascal architectures. This is just a reminder that I don't know nearly as much about GPU memory hierarchy as I thought :).

OK, looking at these docs:
https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__DEVICE.html#group__CUDART__DEVICE_1gb22e8256592b836df9a9cc36c9db7151
I think the attribute we need to check is cudaDevAttrUnifiedAddressing?  Not sure if cudaDevAttrConcurrentManagedAccess is also relevant.  There a lot of device attributes since the last time I looked.  😅

actually, I think we want cudaDevAttrIntegrated, at least for the Tegra chips.

I sent out the call for help, Stan. I'm over my skis on this one!

Okay, did some asking around and got the following information:
It's probably desirable (at some level) to have three different types of GPU memory allocators:

normal cudaMalloc
zeroCopy (cudaHostAlloc)
cudaMallocManaged

You could have these different allocations be either user-defined or check the device for which allocator to use.
For Tegra, the 'easiest' is cudaManaged (Managed Memory, aka Unified Memory) as the Tegra driver is smart enough not to move data.

@mnicely found a nice work-around which may give some ideas for general handling of this feature request:
from numba import cuda
import numpy as np
import cupy as cp

# Number of elements in array
N = 10

arr = np.arange(N)
gpu_arr = cuda.mapped_array(N, dtype=np.float, strides=None, order='C', stream=0, portable=False, wc= True)

# copy contents of arr to a
gpu_arr[0:10] = arr

# print gpu_arr
gpu_arr
>>> MappedNDArray([0., 1., 2., 3., 4., 5., 6., 7., 8., 9.])

# look at a pointers
gpu_arr.__array_interface__['data']
>>><PHONE_NUMBER>36288, False)

gpu_arr.__cuda_array_interface__['data']
>>><PHONE_NUMBER>36288, False)

# cupy zero copy works too
cupy_arr = cp.asarray(gpu_arr)
cupy_arr.__cuda_array_interface__['data']
>>><PHONE_NUMBER>36288, False)

I think these suggestions are going at the problem in the other direction from where we started.
Originally, we were trying to avoid an unnecessary copy of a NumPy array when processing it on a Tegra GPU.  NumPy always uses the system allocator to allocate memory, so there isn't a way to have NumPy use one of the CUDA managed memory allocators in the first place.  If Numba can determine when it is OK to pass a pointer allocated by the OS directly to the GPU, then we should be able to treat NumPy arrays as if they were device arrays and avoid any copies.

@seibert -- my point was if the memory is originally allocated with something like cuda.mapped_array(), then you're already at zero copy. I don't know if this is actual doable with the numba internals and how it interacts with numpy arrays, however.
For more context, I'm essentially trying to take cupy FFTs off of ~16384 complex samples.
# allocate memory
sig_array = cuda.mapped_array(16384, dtype=np.complex128, strides=None, order='C', stream=0, portable=False, wc=True)

# bring data into shared-mem
sig = np.random.rand(16384) + 1j*np.random.rand(16384)
sig_array[0:16384] = sig

# Operate numpy fft
numpy_fft = np.fft.fft(sig_array)

# Operate cupy fft
cupy_array = cp.asarray(sig_array)
cupy_fft = cp.fft.fft(cupy_array)

Ultimately, in my example here, we should probably be able to get rid of this: sig_array[0:16384] = sig.

Late to the party, but thanks, @stuartarchibald for the mention.
I'd consider the Tegra memory behaviour a special case (albeit the most convenient one of all 😃). Usual Unified Memory does involve copies, but they are handled by the driver on-demand (via page faults).
That is also the case for POWER8, where NVLink does exist between CPU and GPU, but it's comparable dumb. NVLink2 on POWER9 CPU↔GPU does support ATS, which omits the page-faulting mechanism. I'm not sure how the exact behaviour with ATS is, as I don't have proper access to a POWER9 machine right now; but the memories are physically distinct so some means of copy does need to happen. @LenaO* does have access, and she's doing quite a lot of experiments with Numba on the machine currently – she might be willing to test-run and help out with things.
That said, doesn't it all boil down to enabling Unified Memory in Numba such that in the case of Tegra, Numba can use this automatically (with the CUDA API methods mentioned above), and have it in place for (future) smarter CPU-GPU memory management (ATS, HMM)? Added benefit: Have it available for people wanting to use it over usual PCIe busses (oversubscription, …).
*: I hope that's actually her. Will update if not.

That said, doesn't it all boil down to enabling Unified Memory in Numba such that in the case of Tegra, Numba can use this automatically (with the CUDA API methods mentioned above), and have it in place for (future) smarter CPU-GPU memory management (ATS, HMM)?

I agree with this. Providing the option for Unified Memory access within Numba and specifically defaulting to that option if the user is on a Tegra + Arm SoC (so, in this case, the driver knows not to migrate memory via page-faults) would be great!

Thanks to @AndiH  for mention.
From my experience, also on Power9 unified memory most of the time involves a memory transfer - however, the difference to other systems is  that is not necessarily required.
You can use "hints" to make a more precise suggested, were the memory should be located  - in this case, page faults can be avoided.   On a GPU - on Power9 Systems - the CPU can also access it without a copy (the main difference) - but I have to redo some experiments.
Talking form a C perspective, the GPU can now also access memory allocated with malloc.
Btw. I have an early stage Bachelor-Thesis working on integrating unified memory into Numba - and we want to evaluate it on both kinds of machines (with and without Hardware Support for unified memory) - probably a Jetson would be an interesting extension...
(And I am actually willing to  help)

Just to follow up: Can someone confirm that cudaDevAttrIntegrated is the correct device attribute to look at to determine if the device functions like a Tegra?

@seibert -- yep, cudaDevAttrIntegrated is the correct device attribute (confirmed by folks on the Jetson team).

I'm going to do a bit of cleanup here.
The mapped_array trick I was using should theoretically not migrate data from CPU->GPU on a Jetson. There will be some PCIe activity on a dedicated GPU (e.g. V100) - but you've got mapped and pinned memory, so it's really just DMA traffic. Fast!
I think the best way to address this feature request is to enable -- and we've discussed this -- unified memory support. On the Jetsons, the driver should be smart enough not to double-copy memory. As far as I know, the mapped_array functionality takes memory pages away from the OS, which could be dangerous on the Jetson.
Plus, the community may find unified memory valuable when creating custom CUDA kernels with Numba.

Hi everyone -- I'm updating this feature request since first writing. My original title,

NVIDIA Jetson SoC (TX2, Xavier, Nano) Failure to Zero-Copy from CPU to GPU with numba.cuda.to_device()

Is more aptly described with enabling NVIDIA UVM (Unified/Universal memory) within Numba. The basics of UVM can be found here, but I'm interested in enabling support as the NVIDIA Jetson GPU line share GPU/CPU memory space. Enabling UVM would allow for zero-copy between the GPU and CPU as the driver "knows" the memory space can be addressed by both processors.
My current work-around in cuSignal is to use cuda.mapped_array, but this removes memory pages from the CPU and probably isn't the best long term solution.

Closing this as it is now implemented via #5900.


Gradle build fails for ZIP file

The build fails when building the downloaded zip file:
$ unzip eureka-master.zip
$ cd eureka-master
$ ./gradlew clean build

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/eureka-master/build.gradle' line: 2

* What went wrong:
An exception occurred applying plugin request [id: 'nebula.netflixoss', version: '2.2.4']
> repository not found: /tmp/eureka-master

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

The problem seems to be the missing .git folder. Building the clone with .git removed results in the same problem.
Tested on Ubuntu 12.04.5 LTS and Archlinux.

Hi @teeratpitakrat unfortunately our new build process requires git pull if you are building from source. Is this a blocking issue for you w.r.t. using eureka?

Hi @qiangdavidliu, I was trying to add eureka as a submodule inside another git project. Then, git modified .git folder of eureka and caused this problem.
My temporary solution is to clone eureka and put it inside that git project without committing (which is a little inconvenient).
It would be great if eureka can be built without the dependency to git.

@teeratpitakrat do you need to edit source code? If not, why not just depend on the published eureka artifacts?

@qiangdavidliu Unfortunately, I need to edit the source code.

@teeratpitakrat unfortunately it looks our new build process does not allow eureka to be used as a git submodule inside another project. If your changes to eureka are not domain specific for your case, feel free to send us pull requests. If you would like to make changes to the build files to enable submoduling, that would be appreciated also.

As there was no follow up on this issue for long time, I am closing it.


Assign number of ports after initialization is complete

I confirm that this contribution is made under the terms of the license found in the root directory of this repository's source tree and that I have the authority necessary to make this contribution on behalf of its copyright owner.

@bjorncs, @andreer fyi

Close in favor of #10500


[AutoRelease] t2-loganalytics-2022-12-09-74137(can only be merged by SDK owner)

https://github.com/Azure/sdk-release-request/issues/3551
Live test success
https://dev.azure.com/azure-sdk/internal/_build?definitionId=1004
BuildTargetingString
azure-mgmt-loganalytics
Skip.CreateApiReview
true

issue link:https://github.com/Azure/sdk-release-request/issues/3551


Character [  unusable in text, keymap fr_BE, because of Keybinding

Prerequisites

[x] Are you running the latest version of Atom?
[x] Are you reporting to the correct repository?
[x] Did you perform a cursory search to see if your bug or enhancement is already reported?

Description
Hi, in Atom-core, when I write text (such as pseudo code or markdown), I can't have access to the [ character, because the keybinding is defined as such :

Keystroke
Command
Source
Selector

ctrl-[
editor:outdent-selected-rows
Core
atom-workspace atom-text-editor:not([mini])

Here is the Belgian Keyboard Layout

As you can see, the [  character in on level 3 on Belgian layout, where on English, it's on the first level. As we need ctrl-alt (or alt-gr) to write ´[´, nothing can happen because of the keybinding in the same time. This happens on Windows and it might happen on Linux too. I am not sure on macOS / OS X, however.
Steps to Reproduce

Change keyboard layout to Belgian (fr_BE)
Try to use [ in ATOM

Expected behavior: Get [.
Actual behavior: Nothing happens.
Versions
Atom: 1.10.2
Electron: 0.37.8
Chrome: 49.0.2623.75
Node: 5.10.0

On Windows 10.
Greetings !

Duplicate of https://github.com/atom/atom-keymap/issues/35.

Will be fixed starting with Atom 1.12.0.

Ho yes, sorry. I did not make a search within closed issues, nor took more time for the good repo.
Lesson learned.
Thank you!


Fabric Dependency not met even though fabric is installed

Minecraft says that the fabric mod isn't installed even though I have it installed I have tried different versions of fabric. I disabled the mod in multiMC and it loaded and showed fabric was installed. but as soon as I enable modern beta it shows the error.
Incompatible mod set!
net.fabricmc.loader.impl.FormattedException: Mod resolution encountered an incompatible mod set!
A potential solution has been determined:
- Install fabric, any version.
Unmet dependency listing:
- Mod 'Modern Beta Fabric' (modern_beta) 5.1.1+1.19.3 requires any version of fabric, which is missing!
at net.fabricmc.loader.impl.FabricLoaderImpl.load(FabricLoaderImpl.java:190)
at net.fabricmc.loader.impl.launch.knot.Knot.init(Knot.java:148)
at net.fabricmc.loader.impl.launch.knot.Knot.launch(Knot.java:68)
at net.fabricmc.loader.impl.launch.knot.KnotClient.main(KnotClient.java:23)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.multimc.onesix.OneSixLauncher.launchWithMainClass(OneSixLauncher.java:214)
at org.multimc.onesix.OneSixLauncher.launch(OneSixLauncher.java:249)
at org.multimc.EntryPoint.listen(EntryPoint.java:143)
at org.multimc.EntryPoint.main(EntryPoint.java:34)

You need Fabric API: https://www.curseforge.com/minecraft/mc-mods/fabric-api

whoops, sorry I thought that was added automatically.


Send POST for Chrome app

Fixes webrtc/apprtc#96 in conjunction with server side changes.
@jiayliu

lgtm
On Tue, Mar 3, 2015 at 1:56 PM, Chuck Hays<EMAIL_ADDRESS>wrote:

Fixes webrtc/apprtc#96 https://github.com/webrtc/apprtc/issues/96 in
conjunction with server side changes.
@jiayliu https://github.com/jiayliu
You can view, comment on, or merge this pull request online at:
https://github.com/webrtc/apprtc/pull/101
Commit Summary

Send POST for Chrome app

File Changes

M src/web_app/js/util.js
https://github.com/webrtc/apprtc/pull/101/files#diff-0 (6)

Patch Links:

https://github.com/webrtc/apprtc/pull/101.patch
https://github.com/webrtc/apprtc/pull/101.diff

—
Reply to this email directly or view it on GitHub
https://github.com/webrtc/apprtc/pull/101.

--
Jiayang Liu


Query and store all moderation actions going back to date N

Merged into master at feeade3ae95a5c32f608c38d6974a405cd2267b1

TODO: add a daily scheduled task to do this job

Resolved with 25ae6b30f67f0fe3c26f0aa4ee1b41a3864b0275


Refactoring EVM Authentication (WalletConnect / Metamask)

Fixes #256
WIP: this still needs a lot of testing.
Known bugs/issues that need to be addressed

when you log in using WalletConnect it takes a couple of seconds to actually log in. We need some loading feedback.
If you change the Metamask network the site does not identify that situation immediately as before.
This was not tested on Android yet (https://github.com/telosnetwork/telos-wallet/issues/452)
I can't test this on iOS so I need someone to volunteer (when it's ready).
In the case where you are not logged in but still on the Collectibles page, if you try to login from inside the site (not home page) it should work but was not tested yet.

Description
Diagram

Test scenarios
Checklist:

[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have cleaned up the code in the areas my change touches
[ ] My changes generate no new warnings
[ ] Any dependent changes have been merged and published in downstream modules
[ ] I have checked my code and corrected any misspellings
[ ] I have removed any unnecessary console messages
[ ] I have included all english text to the translation file and/or created a new issue with the required translations for the currently supported languages
[ ] I have tested for mobile functionality and responsiveness
[ ] I have added appropriate test coverage

I'm closing this PR because it is included in this newer PR


Android - The archived room opened empty and did not disappear from the LHN page

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Issue found when executing PR https://github.com/Expensify/App/pull/11631
Action Performed:

Go to URL https://staging.new.expensify.com/ and login
Have multiple pinned reports
Have some chats which are not pinned that have unread messages
Change the focus view mode in preferences
Find the archived room in LHN (Note: the workspaces of those rooms have already been deleted)
Tap on any of this rooms

Expected Result:
The archived room opens with conversation in past and should disappear from the LHN page when tap on it
Actual Result:
The archived room opened empty and did not disappear from the LHN page
Workaround:
Unknown
Platform:

Where is this issue occurring?

Android

Version Number: <IP_ADDRESS>.
Reproducible in staging?: Yes
Reproducible in production?: No
Email or phone of affected tester (no customers): any
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://user-images.githubusercontent.com/93399543/194581888-44784229-3e57-42df-89e2-e0046b992038.mp4
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
View all open jobs on GitHub

@kbecciv what happens if you do a clean sign out and come back in, does it still occur?

Also is that archived chat still pinned in this case? Or is it unpinned?

If the chat is still pinned in this case, this is expected functionality and not a regression!

Will wait for your response before closing this out

Closing this out


Primer commit

👷 Deploy Preview for managecrm processing.

Name
Link

🔨 Latest commit
271824bb26de5bd18b2c50b961d4d11247698187

🔍 Latest deploy log
https://app.netlify.com/sites/managecrm/deploys/631d3b3c0cb67d000a8e5139


Issue with android 24.0.0-alpha1

Error:Execution failed for task ':app:processDebugResources'.

Error: more than one library with package name 'com.afollestad.materialdialogs.commons'

Use com.github instead of com.afollestad.

Sorry, com.github. afollestad instead of com.afollestad

Thanks it worked

https://github.com/afollestad/material-dialogs/releases/tag/<IP_ADDRESS>


Update dependencies and Xcode

Update to Xcode 9.2

Codecov Report

Merging #146 into development will increase coverage by 0.28%.
The diff coverage is n/a.

@@               Coverage Diff               @@
##           development     #146      +/-   ##
===============================================
+ Coverage        63.79%   64.08%   +0.28%
===============================================
  Files               50       50
  Lines             2453     2453
  Branches            46       46
===============================================
+ Hits              1565     1572       +7
+ Misses             887      880       -7
  Partials             1        1

Impacted Files
Coverage Δ

Watch Extension/Generated/L10n.swift
0% <ø> (ø)
:arrow_up:

MyWeight/Services/MassService.swift
48.48% <0%> (+4.24%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 027aa56...34fede1. Read the comment docs.


[e2e]: go workflow_dispatch branch1 config-ldflags slsa3

Repo: https://github.com/slsa-framework/example-package/tree/branch1
Run: https://github.com/slsa-framework/example-package/actions/runs/6387665157
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.go.workflow_dispatch.branch1.config-ldflags.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.go.workflow_dispatch.branch1.config-ldflags.slsa3.yml
Trigger: workflow_dispatch
Branch: branch1
Date: Tue Oct  3 02:00:15 UTC 2023

Repo: https://github.com/slsa-framework/example-package/tree/branch1
Run: https://github.com/slsa-framework/example-package/actions/runs/6400579986
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.go.workflow_dispatch.branch1.config-ldflags.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.go.workflow_dispatch.branch1.config-ldflags.slsa3.yml
Trigger: workflow_dispatch
Branch: branch1
Date: Wed Oct  4 02:01:37 UTC 2023
Tests are passing now. Closing this issue.


Refactor job dict object as Class

Problem
We've been living dangerously for the past couple years since our humble beginings, and we're still passing a dictionary around as our job object. This isn't great. We don't get any type-validation or intellisense that comes with classes. And, ignoring the FP vs OOP debate, we don't get any instance methods or shared state between very related functions. Most of our handlers can join the job class. We're iterating the same jobs over and over again to filter, validate and prompt based on different criteria of the same data, which is very inefficient. We could do most of this on __init__.
Dataclasses
Dataclasses do away with the __init__ section to make working with lots of variables easy. They also implement some common class methods like _repr_ and __eq__ to allow comparing classes. They keep data-centric classes tidy, helping keep bugs to a minimum. Implementing dataclasses will win us:

static type safety
a single object to structure our data in instead of a loosely defined dict, meaning:

easier comparisons
more solid definitions

Unfortunately we have some hurdles to make them work.
Mutability
Currently, jobs have data that can be treated as immutable, mostly in the form of clip properties and project metadata. Some of this metadata we manipulate to reflect user choices and detected paths, notably:

proxy - Set to None to rerender offline
proxy_media_path - set to determined output path

We use the existing keys to save us from extra complications later – these are big dictionaries, and potentially a lot of them to iterate through!
Unfortunately this is slow 🐌 since we're constantly handling mutable data, and there's no static type safety. Just KeyError, TypeError or worst of all NoneType has no attribute ___
Parsing
We need to parse certain values before we use them in certain places. That's exactly what the __init__ section is for, handling the instance variables... but that's a lot of handling! We don't need to handle them all. To avoid handling them all, we could update only the ones we need using the __post_init__ dunder.
Clip properties form the main 'chunk' of a job.
clip_properties = {

    "clip_name": cp["Clip Name"],
    "file_name": cp["File Name"],
    "file_path": cp["File Path"],
    "duration": cp["Duration"],
    "resolution": list(cp["Resolution"]).split("x"),
    "frames": int(cp["Frames"]),
    "fps": float(cp["FPS"]),
    "h_flip": True if cp["H-FLIP"] == "On" else False,
    "v_flip": True if cp["H-FLIP"] == "On" else False,
    "proxy_dir": proxy_dir,
    "start": int(cp["Start"]),
    "end": int(cp["End"]),
    "start_tc": cp["Start TC"],
    "proxy_status": cp["Proxy"],
    "proxy_media_path": cp["Proxy Media Path"]

    if not len(cp["Proxy Media Path"])
    else cp["Proxy Media Path"],

    "end_tc": cp["End TC"],
    "media_pool_item": media_pool_item,

}

Then there's project properties that we add later.
        project=project_name,
        timeline=timeline_name,
        proxy_settings=settings["proxy"],
        paths_settings=settings["paths"],

    )

Proposal
A traditional class, Job,  with immutable, SourceMetadata and ProjectMetadata dataclasses; and dynamic variables to easily update  mediapoolitem , proxy status and any changing paths.
All types are set to target types, like bool for h_flip and v_flip, list for horizontal and vertical resolution, instead of str for everything. This means the worker gets what it expects, pre-parsed.
Instead of passing proxy and path settings, we pass the entire settings object, then we can validate the object as type SettingsManager
Any handlers that don't prompt the user can be become class methods, like getting a collision-free output path.

from typing import Any, Union

@dataclass(frozen=True)
class SourceMetadata:

    clip_name: str
    file_name: str
    file_path: str
    duration: str
    resolution: list
    frames: int
    fps: float
    h_flip: bool
    v_flip: bool
    proxy_dir: str
    start: int
    end int
    start_tc: str
    proxy_status: str
    proxy_media_path: str
    end_tc: str

@dataclass(frozen=True)
class ProjectMetadata:
	project_name: str
	timeline_name: str
	[etc...]

# Create a list of MediaPoolItems that stay in the queuer.
# We look up our items for post-encode link using file-name.
class MediaPoolRefs():
	def __init__(self):
		self.media_pool_items = list()

	def add_ref(self, filename:str, media_pool_item:PyRemoteObj):
		"""filename should reference source media media_pool_item belongs to"""
		if filename not in self.media_pool_items:
			if media_pool_item not in self.media_pool_items:
				self.media_pool_items.append({filename, media_pool_item})
				return
			raise ValueError(f"{media_pool_item} already registered!")
		raise ValueError(f"{filename} already registered!")

	def get_ref(self, filename:str, all:bool=False):

		if all:
			return dict(self.media_pool_items)

		mpi = self.media_pool_items.get(filename)
		if mpi == None:
			raise ValueError(f"{filename} not registered!")
		return mpi

	def drop_refs(self):
		self.media_pool_items = []

class Job():
	def __init__(
		self,
		src_meta: SourceMetadata,
		proj_meta: ProjectMetadata,
		settings: SettingsManager,
	):

		# Get data
		self.source_metadata = source_metadata
		self.project_metadata = project_metadata

		# Get dynamic vars
		self.output_path = self._get_output_path()
		self.online = _is_online()
		self.orphan = self._is_orphan()
		self.linkable = self._is_linkable()

	def is_online(self):
		status = self.source_metadata.proxy_status
		switch = {
			"Offline": None,
			"None": False,
		}
		# Status is resolution (depends on source-res) when online
		return switch.get(status, True)

	# Refactor 'handle_file_collisions'
	def _get_output_path(self):
		# increment_file_if_exist_blah...
		return output_path

	def is_orphan(self)
		if not self.proxy_status:
			# check if orphaned blah...
		return True

media_pool_refs = MediaPoolRefs()

# approximately like so
def get_proxy_jobs(r_, media_pool_ref, settings):
	for mpi in mpis:
		cp = mpi.GetClipProperty()
		# bunch of filtering...

		project_metadata = ProjectMetadata(r_.project, r_.timeline)

		# Some picking and parsing, need to get hands dirty
		source_metadata = SourceMetadata(

            clip_name = cp["Clip Name"],
            file_name = cp["File Name"],
            file_path = cp["File Path"],
            duration = cp["Duration"],
            resolution = list(cp["Resolution"]).split("x"),
            frames = int(cp["Frames"]),
            fps = float(cp["FPS"]),
            h_flip = True if cp["H-FLIP"] == "On" else False,
            v_flip = True if cp["H-FLIP"] == "On" else False,
            proxy_status = cp["Proxy"],
            proxy_media_path = cp["Proxy Media Path"]
            proxy_dir = proxy_dir,
            start = int(cp["Start"]),
            end = int(cp["End"]),
            start_tc = cp["Start TC"],
            end_tc = cp["End TC"]
        )

		# Define job
		job = Job(project_metadata, source_metadata, settings)

		# Add media pool item reference
		media_pool_ref.add_mpi(source_metadata.file_name, mpi)

Moving forward from here
At some point we'll want to introduce chunking. Since the resolve module actually instantiates all the Jobs with get_proxy_jobs(), it returns them to the queuer class. If we want them to be chunked, we need to re instantiate each job, since every job should be a chunk. We don't want to re-init everything again by making new classes...
We can merge  project_metadata dataclass with source_metadata. That way we don't have to iterate each  source_metadata object and add the same project_metadata data.
We can make MediaPoolRefs a Singleton class. That way we can register all the media_pool_items from the Resolve class and just import MediaPoolRefs in the queuer module. The reason why we can't ship them with source_metadata is that they are unhashable, referencing a pointer in memory to a proprietary API, and will fail transport in Celery. We also don't want to remove them, and we want easy access to them from other modules. A singleton provides that.
There seems to be some discouragement of singletons in Python. Many call them an anti-pattern. We may be able to substitute them for a plain module? All global code and functions. We'd just have to be careful that we are indeed importing the same module instance whenever we do.
If that works out well for us, it may be an idea to re-evaluate SettingsManager as a plain module.
Perhaps instead of returning jobs when we call get_proxy_jobs, we can add get_proxy_jobs and other sister functions to the Resolve class. Then we can save any variables we need to that self and call them like so:
# job module

class Job():
	# All that job stuff from the
	# Job class code block before

def construct_jobs(
    self,
	settings:SettingsManager,
    source_metadata: SourceMetadata,
    chunky:bool=True
)

	if chunky:
		sm = sm.i_like_em_chunky(seconds=30)

	jobs = [Job(x, settings) for x in sm]
	self.jobs = jobs
	return jobs

from resolve import Resolve
from SettingsManager import settings
from job import Job, MediaPoolRefs

r = Resolve(settings)
sm = r.get_source_metadata(make_mpi_refs=True)
jobs = job.construct_jobs(settings, sm, chunky=True)

mpr = MediaPoolRefs()
all_refs = mpr.get_ref(all=True)

/cib


dev mode with pnpm

I was dreaming of a UI framework with vueUse and unoCSS and you have done. Awesome! Thanks.
I followed the file contributing to install in my computer and fix some issue.
But it was a little bit painful. I had to do ni (pnpm i) everywhere. In the root folder, in packages/anu-vue and packages/documentation.
Also, from the root folder, nr dev and nr dev:docs don't work. I had to start two terminal instances not from the root folder but from both packages to prevent vite and vitepress errors.
I wonder how is your dev workflow?

I was dreaming of a UI framework with vueUse and unoCSS and you have done. Awesome! Thanks.

I am glad we are building the framework together everyone wanted ❤
Here's TL;DR

Clone the forked repo
Run ni (It installs all the dependencies required)
Open new terminal in vscode and run nr dev (It will start the anu in dev mode)
Open new terminal in vscode and run nr docs:dev (It will start the docs in dev mode)
Open the shown URL after running docs command and we are done 🥳

The rest of the things are mentioned in contributing, Please let me know if you have any further queries.
BTW, how did you find Anu? 🤔

Thanks for your response.
Below, I relate my installation experience :
At step 2, running ni at the root folder doesn't install all dependencies for me.
After running ni, the command nr dev return vite: command not found. Therefore, I installed vite globally, but I get another error Cannot find package<EMAIL_ADDRESS>So, I run the command ni directly in each packages, but in packages/documentation, ni fail with Unsupported URL Type "workspace:": workspace:*. But pnpm i didn't fail !
After struggled with this installation, I can run nr dev and nr docs:dev from the root folder.
I found Anu from the unocss readme :rocket:

That's strange 🙄
Anyways, as we are using pnpm you can use pnpm commands directly instead of ni.
However, if possible please try cloning the repo again (just for checking) and follow the steps as mentioned above, IG there might be some glitch.
If you already have pnpm installed make sure it's not too old.

I found Anu from the unocss readme 🚀

Great!
If still find this installation issue after a fresh clone please let me know.
Happy coding 🙌🏻

@ManUtopiK If this issue is resolved can we close this?

I tried again and I confirm that I can get everything installed with pnpm i but not with ni from the root folder.
You can close this issue at anytime.

Do you need to run pnpm i in every dir or just in root?

Just in root.

And with ni this doesn't work. You have to go to each dir, right?

Exactly!

Since I updated pnpm and did a corepack enable command, everything works as expected!
I close this issue.
Thanks.


Refactoring: Adjust api imports for new api version

What would you like to be added:
Replace
import "github.com/kudobuilder/kudo/pkg/apis/kudo/v1beta1"

with
import kudoapi "github.com/kudobuilder/kudo/pkg/apis/kudo/v1beta1"

and usage of v1beta1.Instance with kudoapi.Instance
Why is this needed:
Currently the code in KUDO imports the API packages like this:
import (
    "github.com/kudobuilder/kudo/pkg/apis/kudo/v1beta1"
)

the types imported are therefore used as v1beta1.Instance, v1beta1.OperatorVersion, etc.
When we introduce a new API version v1beta2 we need to adjust all these usages, which will introduce a lot of changes in unrelated code, making it hard to review.
I propose to alias the import of a specific api to a generic "kudoapi" and only use this in the code itself. When we change the used API version, we only need to run a global replace to use the new API and the PR change will be limited to imports and actual code changes that is required because of the new API.

I'm a fan for revving the version to v1beta1.   I don't see this proposal doing anything meaningful (unless I'm missing something).   what we need to decide is if we will support v1beta1 and v1beta2 concurrently... if not we bump to v1beta2 and move on.   if we support both... then we need an internal model and the use of beta1 and beta2 where appropriate will likely require the aliasing of the import and it's usage in code.
The only thing I see here with the current proposal is the aliasing... IMO we shouldn't prescribe low level usage of imports... that is on an as need basis while coding... unless we are creating a standard by which we can make code generally easier to read.
I also don't think we need to proactively make import alias changes in prep for new versioned apis.
Perhaps your suggestion is to have an alias kudoapi which is swappable from one to the other... with a perceived value of swapping what the alias is pointing to without changing it's usage.   This too seems off IMO... 1. we may need both 2. it would be expected that a version change has changes that happen in code as well.
I'm of the opinion lets make import and alias changes in the process of coding.

I'm a fan for revving the version to v1beta1.

I'm not sure what you mean with this sentence?

if we will support v1beta1 and v1beta2 concurrently
Depends on the definition of support :) At least he api packages needs both versions for a while for migration purposes, although the rest of KUDO will use only one version at a time - Conversion will be handled completely by the conversion webhook.

then we need an internal model
There will be an internal model in any case, to comfortably convert between the different versions - this is described here pretty nicely: https://book.kubebuilder.io/multiversion-tutorial/conversion-concepts.html

There is an option to only use the internal model for KUDO code, but on the other hand the KUDO CLI and the controller receive specific versions of the resources. Using the internal model would require conversion every time on read and write. Not necessarily a blocker, but something to keep in mind.

Perhaps your suggestion is to have an alias kudoapi which is swappable from one to the other... with a perceived value of swapping what the alias is pointing to without changing it's usage. This too seems off IMO... 1. we may need both 2. it would be expected that a version change has changes that happen in code as well.

Yes - my suggestion is to have an alias kudoapi which is swappable. I wish this was possible on a gobal scope and not on a per-file base. The reason for this is that the actual change between the versions is relatively small.
For 1.: I don't think there will be a case where we need to use both in non-generated code outside of the api package, but I agree, in these cases the kudoapi alias should not be used.
For 2.: A version change will need to have code changes as well. The proposed kudoapi alias simply limits the amount of unneccessary changes.
For comparison: We currently have 63 files which import the v1beta1 api and could use the kudoapi alias. These 63 files have 997 references to v1beta1. I think it makes a big difference for a review to look at 997 changes plus the actual code adjustments or 63 (one import per file) plus the actual code adjustments.

hmm... first the description in the kubebuilder book is about APIs (and operators) and seems different.  management of API versions across components with externalized (non-encapsulated) interfaces with a need to convert from one version to another and back is completely different than internal versioning IMO.
I'm convinced but I'm not against it.   I currently don't see the value.
What we need is the ability for 1 version of kudo to support potentially different versions of the API types (which conversion WH doesn't help with)... why?     The CLI needs to be able to work with operator packages which are v1beta1 and v1beta2.  If not now (based on priorities) at some point.   At some point it will need to be able to parse and manage operators from different API versions.
In order to manage this, I believe we need to have an internal model (perhaps it is versioned) and the external models (currently defined in the api package).   Then we translate external versions (v1beta1 AND v1beta2) to the internal model.   Much of the code which you are looking to modify today should be switched to an internal model and NOT to an external model at all... which is another way of solving the problem you are looking at.

I'm not convinced but I'm not against it. I currently don't see the value.

I've created a draft PR for this. These changes would be mixed in with actual code changes when we switch to v1beta2. I think the value of reviewing this vs a much smaller PR is quite a bit.

What we need is the ability for 1 version of kudo to support potentially different versions of the API types (which conversion WH doesn't help with)... why? The CLI needs to be able to work with operator packages which are v1beta1 and v1beta2 at the same time. At some point it will need to be able to parse and manage operators from different API versions.

There are two things here:

KUDO CLI does not need to understand two CRD versions at the same time - at least not at the moment. This might be a goal for the future, but currently the CLI either talks v1beta1 or v1beta2, not both - at least I don't see any reason why it would.
The KUDO package format. I agree that KUDO must be able to work with both versions here - but the package format is different from the CRDs. The structs for the the package format is in pkg/kudoctl/packages/types.go and isn't even versioned yet. The pkg/kudoctl/packages/convert package contains the code to convert from package format to a specific CRD.

In order to manage this, I believe we need to have an internal model (perhaps it is versioned) and the external models (currently defined in the api package). Then we translate external versions (v1beta1 AND v1beta2) to the internal model. Much of the code which you are looking to modify today should be switched to an internal model and NOT to an external model at all... which is another way of solving the problem you are looking at.

I've mentioned this in my previous comment, and I think using only the internal (hub) model would be a viable solution - it also solves the problem where the "helper" functions should be defined. This PR would still be helping with this switch, as it separates the tedious search-and-replace work that doesn't need much review from actual code changes.


Host data on an external data repository

The GitHub repository currently occupies ~650 MB, mainly due to hosting data. git is not well suited for large data storage.
Some possibilities:

GIN
Zenodo

From @atyson:

We can then either implement our own solution for downloading data (e.g. https://github.com/brainglobe/bg-atlasapi) or use an existing tool such as pooch or possibly a more involved data-management solution such as DataLad (this may be overkill though).

One thing to think about at this early stage is whether or not it's worth re-writing the git history. Even if the data is hosted elsewhere, the git repo will stay the same size. Even on a fast university connection, git clone takes an annoyingly long time.

I've created a GIN repository which could be used to host this data. We use it for all data associated with the BrainGlobe project and it works well. I can give everyone access if you create a GIN login.
Before the data can be moved, do we have details on the licensing for the three initial datasets? Both to see if we can indeed host them ourselves, but also to determine which license we can use going forward.


Implement the id_token Implicit Flow

At present only the id_token token flow is supported

https://github.com/damienbod/angular-auth-oidc-client/issues/1


add get_descriptions parameter to gettradeoffer request

PR for issue: Trade Offer Item descriptions not getting loaded in TradeOfferWebAPI.cs #1100

Can someone verify this fixes the issue?  If so I'll merge it

This is weird. Here are the API docs: https://developer.valvesoftware.com/wiki/Steam_Web_API/IEconService
get_descriptions is not mentioned in the docs for GetTradeOffer/v1.
However, without the parameter, you won't receive the descriptions.


lowercase not yet properly implemented

issues with keyboard giving uppercase, but needing lowercase when shift not pressed. may be fixed with a tolowercase function

moving to z80screentest repo


Template sensors - noisy on startup

Home Assistant release (hass --version):
0.20.0.dev0
Python release (python3 --version):
Python 3.4.3
Component/platform:
sensor/template
binary_sensor/template
switch/template
Description of problem:
These template devices can be very noisy generate and many warnings on startup until the devices they depend on load.
Easy for this to hide real issues.
The warnings can be useful if unexpected - or when first developing a template. But are less useful once it's working
Suggestion:
Add a 'warnings: off` config option on each device which you can turn on when you're happy your template is correct and working.

@balloob Happy to implement if you agree this makes sense.

Closed via #2144


WebUI界面图标不显示

NNI 学生项目问题概述 / General Question of Student Program
请简要概述您的问题 / 观点 :
Short summary about the question / idea :
WebUI界面图标不显示 浏览器为Chrome 92.0.4515.107(64位， win7系统)

请提供 NNI 环境信息 :
nni Environment :

nni version: 2.7
nni mode(local|pai|remote): local
OS: Ubuntu 18.04
python version: 3.8.13
is conda or virtualenv used?: conda
is running in docker?: no

其他建议 / Other Advice
是否需要更新文档（是 / 否）:
Need to update document ( yes / no ) : no
其他分享内容 :
Anything else we need to know :
Log message / 日志信息 :

nnimanager.log and dispatcher.log :

# path: nni/examples/nas/multi-trial/mnist
 python search.py

nnictl stdout and stderr :

Please make sure your network connection is normal (not stuck), and try to use ctrl+F5 to refresh the page

@xiayouran - does this answer resolved your problem? thanks.

@scarlett2018 hi，I tried this method, but it didn't work. Then I cleaned up the cache, but it still didn't work

nice！Thank you for your reply！

try to use other browser(Microsoft edge) to test these icon issue
You could run rm -rf node_modules yarn and yarn build in nni/ts/webui folder. It's possible that the dependence doesn't install right.

@xiayouran - does this work for you?

hi @xiayouran nni v2.9 had been released! And your issue maybe had been fix by these proposal. If you still have this issue, please let me know or reopen this issue. Thanks for using nni.


Custom status code support

I'd like to be able to set arbitrary status codes (201, 202, 204, 302, etc.) and currently, that is impossible because Response::status_code is private and there is no method to create a custom response or modify a response like with_status(self, code: u16) (similar to with_headers.)
This will go away with #13.

Totally agree @jakeswenson. Upgrade to worker v0.0.4 and you'll have this new method available.
https://github.com/cloudflare/workers-rs/blob/2427771d0ae54bd9fa47fc6195fc959d3e849e7c/worker-sandbox/src/lib.rs#L334-L335


Fitbit Missing access token parameter

Make sure you are running the latest version of Home Assistant before reporting an issue.
You should only file an issue if you found a bug. Feature and enhancement requests should go in the Feature Requests section of our community forum:
Home Assistant release (hass --version):
0.24.1
Python release (python3 --version):
Python 3.4.2
Component/platform:
Fitbit
Description of problem:
It was working ok , but since the update 0.24 i'm not able to use the fitbit component
Additional info:
homeassistant.components.sensor: Error while setting up platform fitbit
Traceback (most recent call last):
File "/srv/hass/hass_venv/lib/python3.4/site-packages/homeassistant/helpers/entity_component.py", line 98, in _setup_platform
discovery_info)
File "/srv/hass/hass_venv/lib/python3.4/site-packages/homeassistant/components/sensor/fitbit.py", line 233, in setup_platform
authd_client.client.refresh_token()
File "/home/hass/.homeassistant/deps/fitbit/api.py", line 162, in refresh_token
auth=requests.auth.HTTPBasicAuth(self.client_id, self.client_secret)
File "/home/hass/.homeassistant/deps/requests_oauthlib/oauth2_session.py", line 309, in refresh_token
self.token = self._client.parse_request_body_response(r.text, scope=self.scope)
File "/home/hass/.homeassistant/deps/oauthlib/oauth2/rfc6749/clients/base.py", line 409, in parse_request_body_response
self.token = parse_token_response(body, scope=scope)
File "/home/hass/.homeassistant/deps/oauthlib/oauth2/rfc6749/parameters.py", line 376, in parse_token_response
validate_token_parameters(params)
File "/home/hass/.homeassistant/deps/oauthlib/oauth2/rfc6749/parameters.py", line 386, in validate_token_parameters
raise MissingTokenError(description="Missing access token parameter.")
oauthlib.oauth2.rfc6749.errors.MissingTokenError: (missing_token) Missing access token parameter.

Hello @carlosmgr, did you try to remove the fitbit.conf and restart the HASS to see if it will work?

Sorry for the delay, but my RPI was crashing for some reason. so fresh install....
I tried your ideia, and now i'm getting:
16-08-07 16:31:09 homeassistant.components.sensor: Error while setting up platform fitbit
Traceback (most recent call last):
File "/srv/hass/lib/python3.4/site-packages/homeassistant/helpers/entity_component.py", line 98, in _setup_platform
discovery_info)
File "/srv/hass/lib/python3.4/site-packages/homeassistant/components/sensor/fitbit.py", line 208, in setup_platform
config_file = config_from_file(config_path)
File "/srv/hass/lib/python3.4/site-packages/homeassistant/components/sensor/fitbit.py", line 128, in config_from_file
return json.loads(fdesc.read())
File "/usr/lib/python3.4/json/init.py", line 318, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.4/json/decoder.py", line 343, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.4/json/decoder.py", line 361, in raw_decode
raise ValueError(errmsg("Expecting value", s, err.value)) from None
ValueError: Expecting value: line 1 column 1 (char 0)

Hello @carlosmgr, it seems the fitbit.conf file under your HASS configuration directory is not a JSON file.
Try to run the steps below:
-- Stop HASS service
# cd /home/hass/.homeassistant/
# mv fitbit.conf fitbit.conf.old
-- Start HASS service

Then, modify the fitbit.conf and make sure to keep the correct syntax. For example:
{ "client_id": "AA..[snip]....W", "client_secret": "919..[snip]....xxx"}

After the changes, you can also test if the JSON file is correct by trying this on your PI console:
# ipython3
# import
In [1]: import json
In [2]: print (json.load(open('/home/hass/.homeassistant/fitbit.conf', 'r')))
{ "client_id": "AA..[snip]....W", "client_secret": "919..[snip]....xxx"}

Then finish the OAuth2 authorization process using the HASS frontend and it should work as expected.

Obrigado @tchellomello  ;)
You were right it was not a JSON file, because it was empty!!!
I was doing every thing right, the fitbit.conf kept getting blank every time i modified the file to save the initial configuration.
I resolved that by deleting the fitbit.conf created by HA and creating a new one my self.


Replace examples of calls to the rest API

The current examples use a rather obtuse configuration syntax which dates back to before the az staticwebapp appsettings commands were available this updates the examples to use the newer commands.

@stimms : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Hi @stimms - Thank you for contributing. The content has been moved or reconfigured since the pull request was opened. The pull request can't be merged in, so we'll need to close it. If the proposed update still needs to be added to the content, please submit it in a new pull request. Thanks.


Documented some ToString Templates

This is just a documentation quickie.
Added an assertion to the non-obvious template for flag sets
that should never be called. It isn't currently called.

CI gfxreconstruct build # 2204 failed.

CI gfxreconstruct build queued with queue ID 34443.

CI gfxreconstruct build # 2210 running.

CI gfxreconstruct build queued with queue ID 34460.

CI gfxreconstruct build # 2211 running.

CI gfxreconstruct build # 2211 passed.

CI gfxreconstruct build queued with queue ID 34923.

CI gfxreconstruct build queued with queue ID 34936.

CI gfxreconstruct build # 2217 running.

CI gfxreconstruct build # 2217 passed.


update README for manual oc install instructions

This is as much for confirming if 3.11 image pushes to dockerhub are working

the 3.11 image was updated
see https://ci.openshift.redhat.com/jenkins/view/All/job/push_jenkins_images/266/parameters/
from the console logs:
+ docker push docker.io/openshift/jenkins-2-centos7:v3.11
openshift/jenkins-2-centos7                   v3.11               4ec935568f4d        2 hours ago         1.62GB


No terminal available. "New Terminal" or "New Terminal with Profile" buttons does nothing

Version: 1.0.112
Platform: win32 10.0.18363
Plugins: none
The program starts with no terminal opened and it is impossible to create one using "New Terminal" or "New Terminal with Profile". The others buttons works.
I tried to debug and an error occurs several times => "TypeError: Cannot read property 'value' of undefined" in the file core.umd.js at line 4302| console.error.apply(console, __spread(values));
Then there is a lot of warnings telling "DevTools failed to parse SourceMap" on different .js.map files. That can be not related to the problem.
Maybe the program has a compatibility issue with something installed on my system as it works correctly on an older Windows 7 machine. But maybe it is a compatibility issue with my version of Windows 10 64-bit (1909 version 10.0.18363.959). I hope you will be able to do something about that and help me to make it work.
Thank you.
Regards.

Please try the latest nightly build - this has likely been already fixed.

Please try the latest nightly build - this has likely been already fixed.

Ok, Indeed with version 0.113 all is in order. Thank you for your fast answer and for making this software ;)


Refactor to allow for unique ids

🚧 work in progress 🚧

ok not everything's done, but im doing new stuff that wasnt done before so gonna merge


update go-control-plane

Please provide a description of this PR:
update experimental-ambient branch to use same version of envoy go-control-plane library
To help us figure out who should review this PR, please put an X in all the areas that this PR affects.

[ X ] Ambient
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

Is there a desire to add this to the automation? I believe it was recently added in the main branch.

@ericvn I was going to raise it at the next ambient WG meeting


docs: add syntax highlighting to HOWTO.md

Looks awesome. Minor nit: In the git commit, can it be add instead of adding ? We don't follow our own advice, but it's better to be consistent with our verbage :)

@iph Done :) you could squash merge and change the message


Added comment about replacing backslashes in compile step

Since a standardized solution to our problem might arrive at a later time, we want to include some details about that in the code. See #169

Thanks!


Introduce ES module entry point

This was little bit tricky because vanilla-cookie-consent was exported as main in package.json and webpack prioritize main over module.
So the cascade is browser > module > main for webpack to import right file.
I've also added an example with simple webpack configuration and usage for this package.
See:

https://github.com/webpack/webpack/issues/5756
https://github.com/webpack/webpack/issues/5673
https://github.com/defunctzombie/package-browser-field-spec/blob/master/README.md
• https://developpaper.com/browser-module-main-field-priority-you-dont-know-about-in-package-json/

Example in examples/module will be working with new version published. For simulating the current behaviour do these steps:

run yarn build
go to dist
run yarn pack which creates *.tgz export of package
go to examples/module
install tgz using yarn add <path-to-tgz>
run yarn build
open index.html in your browser and you should see cookie consent

This was little bit tricky because vanilla-cookie-consent was exported as main in package.json and webpack prioritize main over module.

Isn't this something which could be adjusted in the upstream vanilla package? Or they are doing it properly and the issue is purely ours?

This was little bit tricky because vanilla-cookie-consent was exported as main in package.json and webpack prioritize main over module.

Isn't this something which could be adjusted in the upstream vanilla package? Or they are doing it properly and the issue is purely ours?

Well, using main in package.json for this type of package is not entirely wrong. As you can see also jQuery use it same way. Problem is that both packages are client-side package and they do not have default module.export, eg. they are not CommonJS (Node.js, server-side) packages. So they should technically use browser field in package.json, see https://docs.npmjs.com/cli/v7/configuring-npm/package-json#main
But problem with ESM export is purely ours.

However - I don't think it worth it adding the webpack example in examples/module/ and rearranging all the other examples into examples/browser/ folder.

Well, now we have multiple examples and I think that it is fine.

It makes accessing basic examples a bit less obvious, as they are hidden in on more subfolder

Ok, but what is "basic" example? Browser one or with import?

It includes code which will need constant maintenance (constant yarn.lock and package.json.updates, for instance), but the benefit is not clear

Not constant, IMO, only when necessary. It is an example based on dist files.

It IMO does not include anything actually interesting worth showing as an example - it is is just using default webpack.config.js, the same include statement as in readme etc. Just purely basic stuff, which IMHO don't need an example...

Well, I want to add a readme to those examples as well with some additional information which we can link form main readme. Also this basic stuff is also some kind of integration test, so we know that something is not broken.

So it is not obvious to me, example of what it actually showcases and why should someone use this example...
=> I think just an example of webpack integration (ie. extending the part "Via npm") in readme may be enough.

If someone in company uses another tool we should add an example for that as well. Just for simple "It's working, dude!" ;-)

[ ] Remove yarn.lock
[ ] Remove bundle.js
[ ] Add Readme
[ ] Update root readme links
[ ] Better directory structure

The examples would be for me as a developer using this library still a bit confusing.
We basically want to present two types of examples:

configuration examples (how to adjust various configuration, including theming adjustments)
integration examples (esm/cjs).

User of the library would seek each example in different moment. It could (and often would) even be  a different developer / coder. When I'm in the middle of setting the configuration, I'd like to have different examples to be side-by-side, and don't want to be confused with integration examples.
Because of this, and also from systematic point of view, I don't think we should have each specific example from type 1. (configuration examples) on the same level as those of the type 2. (integration examples).
To make it usable and simple for the developers, I suggest two (three) options:
A) Keep configuration examples in the examples/ root, and just add integration examples
examples/
├── index.html
├── functions.js
├── personalization.js
├── configuration.html
├── callbacks.html
├── ...
├── webpack-with-cjs
│   ├── index.html
│   ├── index.js
│   ├── package.json
│   ├── README.md
│   └── webpack.config.js
├── webpack-with-esm
    ├── index.html
    ├── index.js
    ├── package.json
    ├── README.md
    └── webpack.config.js

Main pros of this is: simple access to the most used examples (configuration), so basically discoverability is the key here .
The con is obviously mixing example cases 1. with 2. in the same structure (although on different level).
However they are still separated - configuration examples right in the directory, integration examples in sub-directories. So they don't confuse with each other.
B) Separate configuration examples to configuration/ directory
examples/
├── configuration
│   ├── callbacks.html
│   ├── configuration.html
│   ├── functions.js
│   ├── index.html
│   ├── personalization.js
│   └── theming.html
├── webpack-with-cjs
│   ├── index.html
│   ├── index.js
│   ├── package.json
│   ├── README.md
│   └── webpack.config.js
└── webpack-with-esm
    ├── index.html
    ├── index.js
    ├── package.json
    ├── README.md
    └── webpack.config.js

This is more systematic. However the con is that the configuration examples are more hidden in the directory structure and hard to be found.
C) Encapsulate integration with own directory
It would mean this structure:
examples/
├── configuration
└── integration
    ├── webpack-with-cjs
    └── webpack-with-esm

Although maybe the most systematic, I think this is a bit over-complicated, so this variant is here just for for completeness' sake :)

What do you think? I would be happy with variants A) and B), with A) a bit preferred because of the discoverability point.

I've just taken the A path to refactor these examples.

Maybe add link to those two examples to readme.md?

Please, do a quick check. I've just added them to readme.


Is it possible to aggregate stats by author name instead of author+email

In my project, the most prolific author changed his email during the course of the project (but not his commit name) skewing the stats altogether.
Could it be possible to aggregrate the author only by name?

Hi,
I am currently using the Python-based gitstats and do this with a .mailmap file in the repo. Unfortunately, git_stats apparently ignores it.

Remove email here https://github.com/tomgi/git_stats/blob/cb0af46e187ce718d78ced6d3b13540ca874897b/lib/git_stats/git_data/author.rb#L62


ReferenceManyField not rendering data after first load (cache issue?)

Porting issue #1749 from react-admin, initially reported by evancorl
What you were expecting:
When my restClient returns a response, the ReferenceManyField should re-render with the fetched data.
What happened instead:
When my restClient returns a response, the ReferenceManyField occasionally doesn't re-render with the fetched data. Instead, it continues to display the loading bar. This can be directly traced to the ids prop being undefined.
Steps to reproduce:
The pattern I've noticed: the first attempt at loading a ReferenceManyField renders properly. Any subsequent loads of that ReferenceManyField will not be rendered properly.
In other words, I first load Dealer 1 with referenced salespeople. Loading Dealer 2 (or Dealer 3, Dealer 4, etc.) will not render the salespeople. Going back and loading Dealer 1 again, however, will render the salespeople.
My guess is this is some sort of issue related to the AOR cache. Notice how the id prop here is undefined-undefined-objectObject-63822:

Related code:
Here is my Edit component for Dealers:
export const DealerEdit = props => (
  <Edit {...props} title="Edit Dealer">
    <SimpleForm>
      <TextInput source="name" validate={required} />
      <ReferenceManyField
        reference="salespeople"
        target="dealerId"
        label="Salespeople"
      >
        <Datagrid>
          <TextField source="firstName" label="First Name" />
          <TextField source="lastName" label="Last Name" />
          <EmailField source="email" />
          <DateField source="createdAt" label="Created" />
          <DateField source="updatedAt" label="Updated" />
          <EditButton />
        </Datagrid>
      </ReferenceManyField>
    </SimpleForm>
  </Edit>
);

And here is the data returned from my restClient (using Apollo under the hood):
// GET_ONE (Dealer)
{
  "data": {
    "_id": "5ac7df3b5bc8ae1454ebd549",
    "id": "5ac7df3b5bc8ae1454ebd549",
    "name": "Dealer 1",
    "__typename": "Dealer",
  }
}

// GET_MANY_REFERENCE (Salespeople)
{
  "data": [
    {
      "_id": "5ad3b59fc7d879946ca7e5bc",
      "id": "5ad3b59fc7d879946ca7e5bc",
      "firstName": "John",
      "lastName": "Salesman",
      "email"<EMAIL_ADDRESS>      "createdAt": "2018-04-15T20:27:11.924Z",
      "updatedAt": "2018-04-16T21:35:31.259Z",
      "__typename": "DealerUser",
    }
  ],
  "total": 1
}

Environment

Admin-on-rest version: 1.4.0
React version: 16.3.1
Browser: Chrome 65.0.3325.181 (Mac)

@fzaninotto This can be closed. The issue was that the rest client was unintentionally mutating the rest params filter, which seemed to break the cache.

@evancorl Can you elaborate regarding your findings. I'm having a similar issue and trying to crack the problem.


Could someone explain to me how this can be self-hosted and bridge matrix.org homeserver ?

As the title says. The documentation trips me up when I read this:
Modify your HSs appservices config so that it includes the generated file.
Which makes me think I have to also host a homeserver in order to use this bridge, so that the bot can run on it, and then connect to matrix.org homeserver and discord at the same time (so 3 connections).
Is this a correct understanding?
Thank you!

All appservices have to run with HS colaboration.
And Appservice Puppets are just users that are controlled by the Appservice and they federate like normal users.
So your question doesnt rly make sense to me but thats because im thinking at way to high a level. Yes you would have your homeserver talk to matrix.org and your homeserver would talk to your Appservice discord instance and that Appservice discord instance would talk to discord.com and the bot user on discord.com

Ok cool, thank you :) This could be made more clear in the README.


issue credits

issue 10000 credits to @ctessum

 Follow up
On Monday, 26-Feb-24 14:40:09 UTC by Gov4Git dev
Issued 10000 credits to member @ctessum.


Client Allocation Request for: Ipfsapp

Client Allocation Questions
Core Information

Name: Ipfsapp
Website / Social Media: http://ipfsapp.io
Region: Greater China
DataCap Requested: 500TiB
Addresses to be Notarized: f08229
Notary Requested: ozhtdong

@ipfslab - @ozhtdong is currently out of DataCap to allocate. Would recommend you reach out to another notary that still has DataCap available or wait 2-3 weeks till the next round of notary elections is completed. Thank you!