Image issue

When someone join welcome message is showing but Image is not

Please join the discord server (https://discord.gg/NBWrqtmArQ) for support like this.


Add new functions from 1.1

This change is

@MaikKlein This should be ready to be merged. However you may take a look, if unsafe is used correctly. All Device functions are unsafe. Also, all Instance function, where a non-exhaustive type is passed (like indices or info-structs), are marked unsafe.

Because of the recent implementation of the default trait, I am inclined to believe that everything might need to become unsafe, but I'll open a separate issue for that and merge this PR now.


Cannot connect to Frigidaire

This morning, my dehumidifier showed up as "unavailable" in HA. I can still connect to and control it via the Frigidaire iOS app.
I deleted and tried to re-add the Integration in HA, and receive a "Failed to connect" error upon entering username/password.

Looks like it's back now.


add new languages and technologies

Related Issue

Add new languages and technologies

Closes: #2
Describe the changes you've made

Created separate modules for storing the languages and technologies. This way, it will be easy to add new stuff in future.
Added following languages

     "Ruby",
    "Perl",
    "C++",
    "C#",
    "GoLang",
    "Deno",
    "R",
    "Swift",
    "Matlab"

Added the following technologies

"Full-Stack",
"MEAN"

Sorted the languages and technologies, so that it will be easier for the user to find what he wants.

Describe if there is any unusual behaviour of your code(Write NA if there isn't)
NA

@all-contributors please add @avinash-tiwari  for code


gnomeExtensions.systemd-manager: package manually

Description of changes
Closes https://github.com/NixOS/nixpkgs/issues/279638
https://github.com/hardpixel/systemd-manager/issues/19

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Is this ready?

Is this ready?

As I said here, I pushed 2 commits that fix the polkit files dilemma. If you think these changes are good @linsui then yes I think we can merge this.

Your changes are good to me. :)

How can I override this package? I thought this makes manually packaged extensions not overridable.
https://github.com/NixOS/nixpkgs/blob/153864b4ef7cfaa89ec5386028aa3ac51089f07d/pkgs/desktops/gnome/extensions/default.nix#L75
Changing this to
    (extensions: extensions // (import ./manuallyPackaged.nix { inherit callPackage; }))

works.

You are right... Also .overrideScope is not available. What do you think about @linsui's suggestion @jtojnar ?

I think @linsui's suggestion makes sense.


Added button type to component list buttons

This is a fix for issue #435
Added the type="button" to the component-list buttons

👍

👍

@yoshokatana would this qualify as a bug fix and need a np patch?

@jonwinton I think a patch would be fine.


BUG: add missing //plugin to //include

🦟 Bug fix
Partially addresses #641 . It adds the missing element, but doesn't do anything about additional namespaced elements.
Checklist

[x] Signed all commits for DCO
[ ] Added tests
[ ] Updated documentation (as needed)
[ ] Updated migration guide (as needed)
[ ] codecheck passed (See contributing)
[ ] All tests passed (See test coverage)
[ ] While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use *Squash-Merge

@azeey Done. There does not seem to be a v1.8 folder on sdf9, so I only applied the changes to v1.7 and below.


feat(modal): replace function names by good ones

Summary
Type

Enhancement

Summarise concisely:
What is expected?
Replace function names by good ones (close/show)
The following changes where made:

Change function names

Relevant logs and/or screenshots

Page
Before
After

url
screenshot
screenshot

url
screenshot
screenshot

Some unit testing is broken but then it's green for me 🚀


WFLY-7728  Increased Memory footprint in recent builds

fix memory leaks in testsuite
fix memory leak when using expression language in non web deployments

Retest this please


detect faces when head rotates different angles

Hi, i extracting face rects using AVMetaDataObject, and pass the rects to dlibWrapper to get landmark points, which are then drawn on the video image(rendered by either OpenCV or openGL) .
It works perfectly fine when detected face's rollAngle is approximately less than 30 degrees ( or more than 330 degrees, saying i'm holding my iphone in landscape with home button on the left and front camera on).  HOWEVER,  all points stop moving along with face when rollAngles is out of the above range. Why is this happening? Please help me! Thanks.

The model isn’t trained on data with roll angles that large. You need to train a new model using appropriate data.

Got it, Thank you so much, I'll give it a try. Btw, getting immediate response from davisking feels like making handshakes with the creator of DLIB Kingdom....wowww....exciting.....

One more question: it's possible to train my personal model to meet customed requirement, isn't it? I need your official confirmation. Thanks again!

Since it's mentioned somewhere in the documents that "for any input rect where left > right and top > bottom, it represents an empty value". BUT quite unexpectedly, I'm able to produce very accurate landmarks when camera roates 180 degrees (or face upside down ), by arbitrarily redefining left = rect.origin.x + rect.size.width,  right = rect.origin.x,  top = rect.origin.y + rect.size.height,  and bottom = rect.origin.y. According to your documents, this should be wrong, but it just works.  Am I doing this by chance or if there's actually a mathematical equation that governs the transformation so I don't have to retrain the predictor model to get what I want???

There is no origin field of the dlib.rectangle.  So you must be doing something else.

The rect i'm using is extracted from AVMetaDataObject (ios), expressed as a rectangle, where an origin point of (0.0, 0.0) represents the top-left corner of an unrotated picture and (1.0, 1.0) represents the bottom-right corner of an unrotated picture.

Ok, well, I'm not familiar with that.  In dlib a rectangle isn't anything complex.  It's just an axis aligned box.

Hi davisking,
i'm very sorry but i have to ask you the question i know you hate answering over and over again. Why my jpeg_loader not working?
i'm using dlib in my ios app for getting face landmarks, and it works pretty well. But errors of jpeg_loader reading images occur when I'm trying to use dlib's frontal_face_detector for better face detection results instead of extracting face bound out of AVMetaDataObject.
ERROR#1: when <dlib/gui_widgets.h> is include, it reports ""DLIB_NO_GUI_SUPPORT is defined so you can't use the GUI code.  Turn DLIB_NO_GUI_SUPPORT off if you want to", "Also make sure you have libx11-dev installed on your system";
ERROR#2: when <dlib/gui_widgets.h> is NOT include, it reports "libc++abi.dylib: terminating with uncaught exception of type dlib::image_load_error: jpeg_loader: error while reading /private/var/mobile..."
And yes, DLIB_NO_GUI_SUPPORT:BOOL=OFF, and DLIB_JPEG_SUPPORT:BOOL=ON in cmakeCache.txt;
And yes, i do have x11 installed on my mac (xQuartz);
and yes, I did add DLIB_PNG_SUPPORT and DLIB_JPEG_SUPPORT and other flags, even though I built dlib through cmake;
THIS IS MY DLIB.MM FILE IN XCODE:
/////////////////////////////////////////////////////////////////
#import <dlib/image_loader/jpeg_loader.h>
#import <dlib/image_processing.h>
#import <dlib/gui_widgets.h>
#import <dlib/image_io.h>
#import
#import <dlib/image_processing/frontal_face_detector.h>
#import <dlib/array2d.h>
#import <dlib/matrix.h>
#import "FaceDetectionDlibWrapper.h"
#import <UIKit/UIKit.h>
#import <CoreImage/CoreImage.h>
@implementation FaceDetectionDlibWrapper {
dlib::shape_predictor predictor;
dlib::frontal_face_detector detector;
}

(instancetype)init
{
self = [super init];
if (self) {
NSString *modelFileName = [[NSBundle mainBundle] pathForResource:@"shape_predictor_68_face_landmarks" ofType:@"dat"];
NSLog(@"modelFileName: %@", modelFileName);
  std::string modelFileNameCString = [modelFileName UTF8String];
  dlib::deserialize(modelFileNameCString) >> predictor;

  detector = dlib::get_frontal_face_detector();
  [self faceDetector];

}
return self;
}

(void)faceDetector
{
dlib::frontal_face_detector detector = dlib::get_frontal_face_detector();
NSString *imagePath = [[NSBundle mainBundle] pathForResource:@"hillary_clinton" ofType:@"jpg"];
dlib::array2ddlib::rgb_pixel img;
std::string fileName = [imagePath UTF8String];
dlib::load_jpeg(img, fileName);
dlib::pyramid_up(img);
std::vectordlib::rectangle dets = detector(img);
if (dets.size() == 0) {
return;
}
for (int i = 0; i < dets.size(); i++) {
draw_rectangle(img, dets[i],dlib::rgb_pixel(0, 255, 255));
}
}

BELOW IS HOW I COMPILE DLIB DOWNLOADED FROM GITHUB:
/////////////////////////////////////////////////////////////////
~ cd Downloads/dlib-master/examples
~ mkdir build
~ cd build
~ cmake -G Xcode .. -DUSE_SSE4_INSTRUCTIONS=ON
~ cmake --build . --config Release
BELOW IS CMAKE STATUS INFO:
/////////////////////////////////////////////////////////////////
-- The C compiler identification is AppleClang <IP_ADDRESS>01145
-- The CXX compiler identification is AppleClang <IP_ADDRESS>01145
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using CMake version: 3.14.0
-- Compiling dlib version: 19.17.99
-- Enabling SSE4 instructions
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - found
-- Found Threads: TRUE
-- Found system copy of libpng: /usr/local/lib/libpng.dylib;/usr/lib/libz.dylib
-- Found system copy of libjpeg: /usr/local/lib/libjpeg.dylib
-- Searching for BLAS and LAPACK
-- Searching for BLAS and LAPACK
-- Found PkgConfig: /usr/local/bin/pkg-config (found version "0.29.1")
-- Checking for module 'cblas'
--   No package 'cblas' found
-- Checking for module 'lapack'
--   No package 'lapack' found
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of void*
-- Check size of void* - done
-- Found LAPACK library
-- Found CBLAS library
-- Looking for cblas_ddot
-- Looking for cblas_ddot - found
-- Looking for sgesv
-- Looking for sgesv - found
-- Looking for sgesv_
-- Looking for sgesv_ - found
CUDA_TOOLKIT_ROOT_DIR not found or specified
-- Could NOT find CUDA (missing: CUDA_TOOLKIT_ROOT_DIR CUDA_NVCC_EXECUTABLE CUDA_INCLUDE_DIRS CUDA_CUDART_LIBRARY) (Required is at least version "7.5")
-- DID NOT FIND CUDA
-- Disabling CUDA support for dlib.  DLIB WILL NOT USE CUDA
-- Building a C++11 test project to see if your compiler supports C++11
-- C++11 activated.
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/bo/Downloads/dlib-master/examples/build
please help me out with this, looking forward to your reply. thanks!
bo
------------------ 原始邮件 ------------------
发件人: "Davis E<EMAIL_ADDRESS>发送时间: 2019年5月6日(星期一) 凌晨0:31
收件人<EMAIL_ADDRESS>抄送: "陈波"1132756202<EMAIL_ADDRESS>主题: Re: [davisking/dlib] detect faces when head rotates different angles(#1757)
Ok, well, I'm not familiar with that.  In dlib a rectangle isn't anything complex.  It's just an axis aligned box.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.


[BUG] Hovering and Editing an unidentified object in a container causes the Sheet to become unresponsive.

Hi, during my game my player discovered, that when checking an unidentified magic item in their bag, their sheet became unresponsive and buggy.
It seems to have to do with hovering/editing an unidentified item that doesn't have a description.
System: Foundry V12, D&D 5e 3.3.1
Tidy5e: 5.10.1
Steps that caused the issue for us:
Have a container with an item. (The item doesn't have an unidentified description)
Unidentify it, and hover over it in the container's menu.
Try to edit it and click on the description and details tab.
Currently the window becomes buggy and I cannot seem to close it, it requires page refresh to remove it.

Hey friend! 👋
Thanks for reporting.
I ran this test in the same Foundry / dnd5e / Tidy version. I'm able to hover over an unidentified item in a bag.
From a container on a sidebar:

From a PC:

I'll need some more help with reproducing this error.
Is there a specific item that triggers it?
Does it trigger consistently?
Does it trigger when Tidy is the only module enabled? If it does not trigger in this scenario, let's figure out which module Tidy is having issues integrating with, and we can probably figure something out.
If you are willing, can you export the actor that's experiencing the issue. You can send to me privately on discord, if you'd like. I might be able to trigger it based on data. If it is a data issue, then that's something I can probably fix in Tidy 🤞

Note: I did test with empty descriptions, also:

I believe the error is unrelated to description. When the item card tried to pull the description to show it, the item itself was undefined, which triggered the error.

Hey there, thank you for investigating the issue!
After further testing, and disabling every other module I noticed a thing.
Have you tried to replicate the issue as a GM or as a player? This seem to affect me only if I login as a player of mine, and it's very consistent, no matter the container/item.

My tests were as a player.

Can you try this on a fresh world?


[FEATURE] add /project and /leaderboard endpoints

Is this a unique feature?

[X] I have checked "open" AND "closed" issues and this is not a duplicate

Is your feature request related to a problem/unavailable functionality? Please describe.
Add an endpoint for:

[ ] A /project/{owner}/{repo} endpoint
[ ] A /leaderboard endpoint

Proposed Solution

Screenshots
No response
Do you want to work on this issue?
No
If "yes" to above, please explain how you would technically implement this
No response

I think issue should be separate in 2.
I created a PR for the leaderboard https://github.com/EddieHubCommunity/RepoRater/pull/65, so this PR could be for that. Then the project endpoint could be another issue


bump Makie compat

docs build is erroring because Makie no longer ignores invalid kwargs. I haven't looked more closely yet but I suspect we're just forwarding all kwargs to all plots even when some kwargs only apply to some plots.

Invalid attribute makersize 🙃

now markerstrokewidth 😞


Add Huggingface model zoo from community

According to [Feature Request: "Model Zoo" for quantization #1591],
this is our initial effort to create the Model Zoo.
The first model uploaded is Llama3-70b, AWQ Quantized.
I have identified several opportunities within the Model Zoo. I encountered a variety of configurations including PP_size, TP_size, KV_cache_type (fp16, fp8, int8), Group_size (64, 128), and Quantization algorithms (AWQ, SQ, FP8).
I will try to figure out the "proper" base configuration.
I have decided to use the lowest possible Group_size (to prevent the degradation of quantization) and set PP_size to 1.
Let's discuss if we can determine the "proper" configurations.

Thank you for the PR. We will merge it soon.

Hi @matichon-vultureprime ,thanks for your contributing. We've merged your contribution into code base and will add you into contributor list.


Wbnf grammar fixes

Multiple small fixes on the grammar. I have divided them into multiple commits for easier review. I can divide them into multiple PRs but the fixes are very small.
Changes proposed in this pull request:

union rule has working COMMENT statement.
union rule for user_defined_type is changed to NAME. type_spec allows sized types which isn't allowed in the original grammar.
param rule now allows newline inside parameters. Current sysl validate doesn't allow this, but this needs to be allowed as one of the allowed argument is an field with multiple attributes. For example:

field <:
      attribute1 <: int
      attribute2 <: string

field multiple attributes now work like the example above.
alias restricts to one type line (it allowed multiple type line before)
annotation now allows full stop.

@anz-bank/sysl-developers

added a test case for fields with multiple attributes


ITVR - deploy spring service for cra file encryption/decryption

Describe the task
Deploy this: https://github.com/bcgov/itvr/pull/640
Purpose
This automates, to an extent, the encryption/decryption of CRA files
Acceptance Criteria

[ ] The code contained in https://github.com/bcgov/itvr/pull/640 is deployed successfully to production.

Hey team! Please add your planning poker estimate with Zenhub @emi-hi @JulianForeman @tim738745


auro-skeleton: WCAG 2.1 Graphics Contrast Ratio

Describe the feature
Since WCAG 2.1 adds a contrast ratio of 3.1 for graphics and user interface components against adjacent colors, does this mean that the auro-skeleton component (with a ratio of Contrast Ratio 1.03:1) is not WCAG 2.1 compliant when used on light backgrounds? or does this qualify as an inactive UI component?
The scope of this feature is to revisit this component, assess its a11y support and recommend feature enhancements if applicable.
To Reproduce
Steps to reproduce the behavior:

Visit Webaim Contrast Checker
Use the Webaim.org contrast ratio checker to compare #f9fbfc (a.k.a. --auro-color-brand-neutral-100) on a background of #ffffff

For reference, here is an example from our current project.


[BUG] Reading Dataset from S3 Results in Error

Describe the bug
Hello! I was experimenting with reading a parquet dataset directly from S3 and ran into a problem.
Steps/Code to reproduce bug
import pandas as pd
import nvtabular as nvt

s3_save_path = "s3://my-s3-bucket/my_folder/nvt_test.parquet"
num_rows = 30000000

df = pd.DataFrame({
    "cat_col": [f"val_{i}" for i in range(num_rows)],
    "int_col": list(range(num_rows)),
})

df.to_parquet(
    s3_save_path
)
nvt.Dataset(
    s3_save_path,
    engine="parquet",
)

yields the error
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [1], in <cell line: 16>()
      8 df = pd.DataFrame({
      9     "cat_col": [f"val_{i}" for i in range(num_rows)],
     10     "int_col": list(range(num_rows)),
     11 })
     13 df.to_parquet(
     14     s3_save_path
     15 )
---> 16 nvt.Dataset(
     17     s3_save_path,
     18     engine="parquet",
     19 )

File /core/merlin/io/dataset.py:304, in Dataset.__init__(self, path_or_source, engine, npartitions, part_size, part_mem_fraction, storage_options, dtypes, client, cpu, base_dataset, schema, **kwargs)
    302 if isinstance(engine, str):
    303     if engine == "parquet":
--> 304         self.engine = ParquetDatasetEngine(
    305             paths, part_size, storage_options=storage_options, cpu=self.cpu, **kwargs
    306         )
    307     elif engine == "csv":
    308         self.engine = CSVDatasetEngine(
    309             paths, part_size, storage_options=storage_options, cpu=self.cpu, **kwargs
    310         )

File /core/merlin/io/parquet.py:312, in ParquetDatasetEngine.__init__(self, paths, part_size, storage_options, row_groups_per_part, legacy, batch_size, cpu, **kwargs)
    309 self.dataset_kwargs = self.read_parquet_kwargs.pop("dataset", {})
    311 if row_groups_per_part is None:
--> 312     self._real_meta, rg_byte_size_0 = run_on_worker(
    313         _sample_row_group,
    314         self._path0,
    315         self.fs,
    316         cpu=self.cpu,
    317         memory_usage=True,
    318         **self.read_parquet_kwargs,
    319     )
    320     row_groups_per_part = self.part_size / rg_byte_size_0
    321     if row_groups_per_part < 1.0:

File /core/merlin/core/utils.py:488, in run_on_worker(func, *args, **kwargs)
    486     return dask.delayed(func)(*args, **kwargs).compute()
    487 # No Dask client - Use simple function call
--> 488 return func(*args, **kwargs)

File /core/merlin/io/parquet.py:1215, in _sample_row_group(path, fs, cpu, n, memory_usage, **kwargs)
   1213         _df = cudf.io.read_parquet(path, row_groups=0, **kwargs)
   1214     else:
-> 1215         _df = _optimized_read_remote(path, 0, None, fs, **kwargs)
   1216 _indices = list(range(n))
   1217 if memory_usage:

File /core/merlin/io/fsspec_utils.py:158, in _optimized_read_remote(path, row_groups, columns, fs, **kwargs)
    149 else:
    150     # Get byte-ranges that are known to contain the
    151     # required data for this read
    152     byte_ranges, footer, file_size = _get_parquet_byte_ranges(
    153         path, row_groups, columns, fs, **user_kwargs
    154     )
    156     return cudf.read_parquet(
    157         # Transfer the required bytes with fsspec
--> 158         io.BytesIO(
    159             _fsspec_data_transfer(
    160                 path,
    161                 fs,
    162                 byte_ranges=byte_ranges,
    163                 footer=footer,
    164                 file_size=file_size,
    165                 add_par1_magic=True,
    166                 **user_kwargs,
    167             )
    168         ),
    169         engine="cudf",
    170         columns=columns,
    171         row_groups=row_groups,
    172         strings_to_categorical=strings_to_cats,
    173         **read_kwargs,
    174     )

TypeError: a bytes-like object is required, not '_io.BytesIO'

Funnily enough, if I tune the num_rows to 28000000, this above snippet succeeds. So I am assuming this is something with the file size. I have tried playing around with the part_size and part_mem_fraction parameters but saw no changes.
Expected behavior
nvt.Dataset should be able to read entire datasets from S3.
Environment details (please complete the following information):
Using docker image built from merlin-training:22.04, so this is using NVT version 1.0.0. I am running this in a Jupyter Notebook running on a single V100 GPU instance.

Thanks for the simple reproducer @shoyasaxa !
I didn't try running the code yet, but it looks like that container may be picking up a pretty old version of fsspec. Would you mind trying to run with the latest fsspec and s3fs libraries installed? You should be able to get the latest releases with:
pip install<EMAIL_ADDRESS>

We're basing that container off the DLFW/Rapids container nvcr.io/nvidia/tensorflow:22.03-tf2-py3 - and I've just verified that that container has cudf 22.02 and fsspec 2021.07.0 :
In [1]: import fsspec
In [2]: fsspec.__version__
Out[2]: '2021.07.0'

In [3]: import cudf
In [4]: cudf.__version__
Out[4]: '22.02.00a+309.gdad51a548e'

Ah yep looks like doing pip install<EMAIL_ADDRESS>fixes the issue!

Okay -  I now always use the _optimized_read_remote code path for row-group/schema sampling (even with cudf>=22.02).  I suppose that's fine as long as we are still using the fsspec.parquet module there.

Do we need a newer version of fsspec than whats provided by the unified DLFW/Rapids container? If so - I think we should probably list this as a dependency in merlin-core, and specify the fsspec version we require.

Yes. For best performance, we need the fsspec.parquet module from fsspec>=2021.11.1.  We also need s3fs>=2021.11.1 (because earlier versions contain a bug that only shows up when a file is opened through fsspec.parquet).

Hi @rjzamora, I have a quick follow up question. So upon instantiating nvt.Dataset(path_to_my_dataset_in_s3) as well as while fitting a workflow, I see a constant stream of warning messages saying
/usr/local/lib/python3.8/dist-packages/fsspec/caching.py:503: UserWarning: Read is outside the known file parts: (222230609, 223104838). IO/caching performance may be poor!

with the number used for the file parts different for each warning message.
Are these warning messages expected? Is there something I could do to not get these?

Are these warning messages expected? Is there something I could do to not get these?

These messages mean that the optimized data transfer is not working correctly (and performance will likely be terrible).  If you are sure that you are using s3fs>=2021.11.1, then this may be a bug.

My apologies - upon a second look it looks like s3fs's version is not what it should be. I will clean this up a little more and get back to you!

@shoyasaxa is this still an ongoing issue?

My apologies - confirmed this works with pip install<EMAIL_ADDRESS>Closing this comment - thank you!


add hyperbatching

Essential feature request: Emulate TurkPrime's (now "Cloud Research") HyperBatch feature to avoid accruing an extra 20% MTurk fee for having more than 9 subjects / HIT. I've used TurkPrime quite a bit. The feature is tried-and-true, but there's a fee for it (smaller than the 20% amazon fee but still worth avoiding).
See:
https://gist.github.com/deargle/f454a156d3a72adf62869e2ef09101d8
https://github.com/ddbourgin/mturk_utils

This would be nice, but it should be implemented in psiturk itself. Note that you can achieve this effect quite easily by simply copying the hit creation command many times. In a text editor, create a file like this, where it repeats as many times as necessary to get to the total number of assignments you want. Then copy the whole thing and paste it into the psiturk prompt. Elegant? No. Effective? Yes.
hit create 9 1.25 0.5
y
hit create 9 1.25 0.5
y
hit create 9 1.25 0.5
y
...


Initial work on updating status bar

Description
This PR makes the following changes:

active layer is now the topmost visible layer, regardless of selection. (See below for discussion.)
printing of status messages based on value is now handled centrally, and much improved. I haven't changed over the obvious (integer) prints. In a future PR, I think this machinery can be used to print shape info, rather than just the shape index.
if the cursor is out of range of the data, no value is shown, rather than showing the value at the nearest edge.
a new keyboard shortcut is added, toggling visibility of the topmost layer.

Type of change

[x] Bug-fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update

How has this been tested?

[x] example: all tests pass with my change

Final checklist:

[x] My PR is the minimum possible work for the desired functionality
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works

Discussion points
I'm still not sure about the active layer behavior. Now, it's a bit more cumbersome to mess with a layer's settings: for example, if I have a points layer and an image layer underneath it, I have to hide the points layer before I can adjust the clims on the image. However, this does solve the problem of adjusting settings on an invisible layer.
My suggestion, which I can implement here or in a followup PR depending on preferences, is that the active layer be defined as:

if no layers are selected, the topmost visible layer.
if a layer is selected, it is the active layer if it is at all visible, which means that only the following layers are in front of it:

image, labels, and pyramid with opacity < 1
points, shapes, or vectors, with any opacity (since these are generally sparse, visually)

I'll wait for thoughts before working on that logic.
Regarding the keyboard shortcut, topmost is kinda clunky, it turns out. My suggested improvement, which I can implement here or in a followup, is:

Ctrl-V: hide the next visible layer (ie the active layer)
Ctrl-Shift-V: show the next not-visible layer (ie the layer above the active layer), if any
V: undo the last visibility change

Thoughts? And I just realised the shortcuts will have to change because Ctrl-V is taken. Suggestions welcome!

Also, after this PR is resolved, I do want the status bar to show the status from all visible layers in a sequence. But that can come in a followup, and I could definitely use some guidance from @sofroniewn about how to go about this!

This is still WIP, but @sofroniewn @kne42, note that when I move my @Viewer.bind_key to @ViewerModel.bind_key, I can't get the keybindings to trigger at all... Is this expected?

Also, @sofroniewn, I believe you were going to give me some pointers about where to start for concatenating status messages from multiple layers together? Apologies if you already did and I have lost track of it.

@jni i would do the concatenation of status messages inside this for loop - https://github.com/napari/napari/blob/ebc5474a59a89496c99c43129b65e63afa7cc662/napari/components/viewer_model.py#L705
and then do self.status = concat_status at the end instead of self.status = active_layer.status and that will trigger all the updates needed

@jni with the merging of #485 which incorporates some of the code from this PR I recommend we close this PR. Also I added similar keybindings from this PR in #472.
You can then make a new PR that makes all the status bar printing from many layers happen at the same time.

Yep, that was the impression I had. Will deal with unifying status bars mid-september.

Sounds good. I bumped unifying status bars to 0.2 so it's on our list in #301 now :-)


Error compiling example for Arduino DUE

Issue template

Hardware description: Arduino DUE
RTOS: NO
Installation type: using static library
Version or commit hash: micro_ros_arduino-2.0.3-foxy into Arduino 1.8.19 IDE's libraries directory.

Steps to reproduce the issue
Install micro_ros_arduino-2.0.3-foxy into Arduino 1.8.19 IDE's libraries directory.
Go to File > Examples > micro_ros_arduino > micro-ros_publisher
Try to compile the project.
Expected behavior
To finish compilation successfully, and get uploaded. Run the micro-ros_publisher as it is.
Actual behavior
Arduino: 1.8.19 (Linux), Board: "Arduino Due (Programming Port)"
Library micro_ros_arduino-2.0.3-foxy has been declared precompiled:
Using precompiled library in /home/borgesjvt/Arduino/libraries/micro_ros_arduino-2.0.3-foxy/src/cortex-m3
The platform does not support 'compiler.libraries.ldflags' for precompiled libraries.
sketch/micro-ros_publisher.ino.cpp.o: In function `timer_callback(rcl_timer_t*, long long)':
/home/borgesjvt/Arduino/libraries/micro_ros_arduino-2.0.3-foxy/examples/micro-ros_publisher/micro-ros_publisher.ino:36: undefined reference to `rcl_publish'
sketch/micro-ros_publisher.ino.cpp.o: In function `set_microros_transports':
/home/borgesjvt/Arduino/libraries/micro_ros_arduino-2.0.3-foxy/src/micro_ros_arduino.h:32: undefined reference to `rmw_uros_set_custom_transport'
sketch/micro-ros_publisher.ino.cpp.o: In function `setup':
/home/borgesjvt/Arduino/libraries/micro_ros_arduino-2.0.3-foxy/examples/micro-ros_publisher/micro-ros_publisher.ino:49: undefined reference to `rcutils_get_default_allocator'
/home/borgesjvt/Arduino/libraries/micro_ros_arduino-2.0.3-foxy/examples/micro-ros_publisher/micro-ros_publisher.ino:52: undefined reference to `rclc_support_init'
/home/borgesjvt/Arduino/libraries/micro_ros_arduino-2.0.3-foxy/examples/micro-ros_publisher/micro-ros_publisher.ino:55: undefined reference to `rclc_node_init_default'
/home/borgesjvt/Arduino/libraries/micro_ros_arduino-2.0.3-foxy/examples/micro-ros_publisher/micro-ros_publisher.ino:58: undefined reference to `rosidl_typesupport_c__get_message_type_support_handle__std_msgs__msg__Int32'
/home/borgesjvt/Arduino/libraries/micro_ros_arduino-2.0.3-foxy/examples/micro-ros_publisher/micro-ros_publisher.ino:58: undefined reference to `rclc_publisher_init_default'
/home/borgesjvt/Arduino/libraries/micro_ros_arduino-2.0.3-foxy/examples/micro-ros_publisher/micro-ros_publisher.ino:66: undefined reference to `rclc_timer_init_default'
/home/borgesjvt/Arduino/libraries/micro_ros_arduino-2.0.3-foxy/examples/micro-ros_publisher/micro-ros_publisher.ino:73: undefined reference to `rclc_executor_init'
/home/borgesjvt/Arduino/libraries/micro_ros_arduino-2.0.3-foxy/examples/micro-ros_publisher/micro-ros_publisher.ino:74: undefined reference to `rclc_executor_add_timer'
sketch/micro-ros_publisher.ino.cpp.o: In function `loop':
/home/borgesjvt/Arduino/libraries/micro_ros_arduino-2.0.3-foxy/examples/micro-ros_publisher/micro-ros_publisher.ino:81: undefined reference to `rclc_executor_spin_some'
collect2: error: ld returned 1 exit status
exit status 1
Error compiling for board Arduino Due (Programming Port).

This report would have more information with
"Show verbose output during compilation"
option enabled in File -> Preferences.
Additional information
I have put the  SAM patch like mentioned:
export ARDUINO_PATH=[Your Arduino path]
cd $ARDUINO_PATH/hardware/sam/1.6.12/
curl https://raw.githubusercontent.com/micro-ROS/micro_ros_arduino/foxy/extras/patching_boards/platform_arduinocore_sam.txt > platform.txt

I've also seen other issues like #427 and #429 looking for insights of the problem but seems to be all like instructed.

Hello @BorgesJVT, this is a problem with the SAM patch because the build process is complaining of The platform does not support 'compiler.libraries.ldflags' for precompiled libraries..
In fact, those patches includes this ldflags and add them here also.
So make sure that you are replacing the correct file, should modify this one

Hello @pablogs9, I checked again and the platform.txt file is correct, but still the same result.
That's my path /home/borgesjvt/Downloads/arduino-1.8.19-linux64/arduino-1.8.19/hardware/arduino/avr/platform.txt.
At first I had installed Arduino with snap, but I was not able to edit the platform.txt file since there was some kind of mounted file system with snap.
I've also tested with foxy-2.0.0 version.
I don't know if it gives some insights of the problem. Any suggestions?

Let us replicate the environment.

I am trying to do the same thing, and run into this issue as well. My code now compiles correctly, although I have not uploaded or tested the actually application yet.
The path you mention contains the AVR core files, but not the SAMD ones (for whatever reason). You can find the correct path through the arduino IDE:

Open the Arduino IDE
File>Prefernces
At the bottom of this window you will see a note "More preferences can be edited directly in the file", followed by a link to a folder on your computer.
Click the link. A folder will open. It should contain a folder named "Packages", among other things.
Navigate to Packages>Arduino. This is the actual path you need to give to the above commands to properly apply the patch. You can verify it by navigating to Packages>Arduino>Hardware. You should see a folder named SAMD inside.

There might be a better way to figure out the correct path, but this should do the trick.

@BorgesJVT, does the @SpMakris solutions work for you?

Thank you @SpMakris and @pablogs9. Yes, it worked. When changing the platform.txt file inside this path (/home/borgesjvt/.arduino15/packages/arduino/hardware/sam/1.6.12) it works fine.

Thank you @SpMakris and @pablogs9. Yes, it worked. When changing the platform.txt file inside this path (/home/borgesjvt/.arduino15/packages/arduino/hardware/sam/1.6.12) it works fine.

hi , could you please share the exact file that you used and got no error
i have the same problem

Better switch to vscode and platformio.
Follow the wiki, https://github.com/hippo5329/micro_ros_arduino_examples_platformio/wiki
pio run -e due

thank you thomas
it worked fine
On Sat, Nov 2, 2024 at 6:19 PM Thomas Chou @.***> wrote:

Better switch to vscode and platformio.
Follow the wiki,
https://github.com/hippo5329/micro_ros_arduino_examples_platformio/wiki
pio run -e due
—
Reply to this email directly, view it on GitHub
https://github.com/micro-ROS/micro_ros_arduino/issues/819#issuecomment-2453013503,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AR6DPSXRWOWD3LGTXZUIYNLZ6TRBFAVCNFSM6AAAAABRBY3M2GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJTGAYTGNJQGM
.
You are receiving this because you commented.Message ID:
@.***>


Append ApplyFunction to math functions

Append a <mo></mo> to the MathML of math functions such as \sin or \tan.
Ref:  https://www.w3.org/TR/REC-MathML/chap3_2.html#sec3.2.4

LGTM! Thanks for the pointers.

@kevinbarabash, I think I will shut this PR down and re-submit after PR #871 lands. That will be easier than trying to coordinate the two PRs.

@ronkok thanks.  I'll see what I can do to expedite that review.


Add snap builds to CI

#1198 added the snapcraft config to the repo but it hasn't been added to the CI yet.
We want to have a setup similar to that of our current docker builds. A push to the snapstore on every master and release build. This should be fairly easy to do based on the CI stuff we already have.

I’m a little worried about pushing on master builds. If we accidentally check in a bad commit then there’s a window where a downstream user gets a bad build, right?
IIRC (not at computer right now) there’s not automatic updates so until the user updates their snap image they could just have a broken build for a while.
On Mon, Mar 16, 2020, at 5:19 PM, bas smit wrote:

#1198 https://github.com/iovisor/bpftrace/pull/1198 added the
snapcraft config to the repo but it hasn't been added to the CI yet.
We want to have a setup similar to that of our current docker builds. A
push to the snapstore on every master and release build. This should be
fairly easy to do based on the CI stuff we already have.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/iovisor/bpftrace/issues/1216, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAGJKJ6O5N6LGQW3MIZCISLRH262TANCNFSM4LM2DIKQ.

--
Daniel Xu
<EMAIL_ADDRESS>

I’m a little worried about pushing on master builds. If we accidentally check in a bad commit then there’s a window where a downstream user gets a bad build, right?

I think we should keep it similar to how the docker/quay setup works. Release builds go to the release channel and the master builds to edge/latest. That way the user can decide what is best for them.
https://snapcraft.io/docs/channels

@danobi Can we close this?

Yeah, appimages supersede this


Flicker / Render issue with roundPixels AND cacheAsBitmap turned on

I have a chunk that generates tiles as sprites, the sprites are added to a container which is then translated around the player. There originally was a weird flicker issue going on around the edges of each tile but setting roundPixels: true fixed this (NOTE: all sprite coordinate values were already getting rounded before this so I'm not sure how this fixed that issue)
My issue now is that when I set this.sprite.cacheAsBitmap = true; on the container filled with tiles, a weird rendering issue occurs

NOTE: the border line is much thinner in the center and this ONLY occurs when you move your character towards the right side of the map (isn't like that always)
Now the weird part is setting roundPixels to false and keeping cacheAsBitmap true fixes this, or turning off cacheAsBitmap to false and keeping roundPixels true fixes it too. My issue is that I would like to keep both roundPixels and cacheAsBitmap but have no idea what the problem being caused is currently.
Thanks in advance!

I, personally, cant give guarantees about cacheAsBitmap. If you make a fiddle, we'll fix it.
Workaround: please use https://github.com/pixijs/pixi-tilemap , tutorial is here: https://github.com/Alan01252/pixi-tilemap-tutorial

Am trying to replicate it on fiddle, will message you soon. I would rather not use more libraries if possible

@ivanpopelyshev jsfiddle
very strange behavior, the render issue seems to have disappeared without me changing anything but now there's a weird flicker that occurs horizontally (halfway across the bitmap) and vertically (halfway across the bitmap) almost as if it's being split up into 4 quadrants. I replicated the code as best as I can but none of those things are occurring on jsfiddle. There is smoothing code on my copy that I haven't added to the fiddle, which takes the expected sprite location - current sprite location / variable adjusted for time between frames (this gives float values for the sprite) but they should be getting rounded through the roundpixel game setting. Turning the cacheAsBitmap to false makes all of this go away.

Closing as looks like you have a handle on a solution :)


Dark mode extension or flag dark

Hello,  the dark mode saves energy and battery and sometimes we watch long videos on streaming sites which sometimes do not have a dark theme mode, so it will be great to implement this light extension in Quark player to dark web pages in streaming sites :
https://github.com/m-khvoinitsky/dark-background-light-text-extension
it is an extension for firefox but we can install addons extensions from the firefox store in google chrome through these extensions:
https://foxified.org/
https://crosspilot.io/
Chameleon
https://chrome.google.com/webstore/detail/chameleon/dmpojjilddefgnhiicjcmhbkjgbbclob
https://sereneblue.github.io/chameleon/
or just add an option to enable chromium's dark mode flag by default, thank you very much !

@trimechee Added. There is now a setting in "Settings" on the menu bar to toggle light/dark mode.

Great ! 🥇 and sorry Mr Alex @Alex313031 for having riddled you and flooded you with my many requests and questions! you are a very brilliant developer and your multiple projects and outstanding revolutionary software are a source of inspiration and motivation which improves our lives and makes it more joyful and comfortable, thank you very much :)


Spec: Update examples in spec documentation to make it clear what is within API/Implementation and what is user code

Within examples of the spec, there are sometimes code snippets which are user written (not defined within API and implementation)
This can be confusing for the reader. Use comments to indicate what should/can be performed in these statements instead of real code.

+1

-1 If it's not clear just include a comment - code snippets make the examples more readable

I wonder if there are precedents from the other Microprofile specs?

Should not we talk about specific cases? I think having a code snippet could be more readable at some places, e.g. when a good naming makes method calls readable as it would be a comment. But if the code snippet is not obvious for the reader then it's really confusing. When looking into the spec there are parts that, by my point of view, would deserve clarification.

For example what is the getActivity(lraId)? That would be better to be changed for comment.
isFinished(lraId) the method functionality is not defined elsewhere and it would be nice to clarify with a comment but then the following code return Response.ok().entity(ParticipantStatus.Compensated).build(); seems reasonable for me to be included in the example
etc.

There are code snippets in other specifications. But it seems they are trying to be as simple as possible. I agree that code showing required protocol actions (as returning correct response with correct status code etc.) is necessary.

I do agree that any pseudo code should be clearly marked as such. @ochaloup do you not think the comment "// Business logic, not provided by spec or Implementation" is clear enough? IMO it is and I think both the getActivity and isFinished (and indeed isFailed) all have a note close enough that someone could not confuse it to be defined within spec.

I meant ragher that e.g. for the getActivity or isFinished would be good to be accompanied with the comment text of kind // search of the LRA id in a storage managed by business logic and // business logic method assuring the LRA action was already finished.
I mean this being added per case. Informing the user that it's a business logic responsibility and what's happening.

Personally I would see that as quite a stylistic choice. I think when it is not clear from a comment that the code is business logic that should be changed but the examples using getActivity, isFinished seemed pretty clear, at least in my opinion.

Personally I think we should close this as the examples that I can see do make it clear that they are business logic and not defined/provided by the spec. I do agree with the sentiment of making it clear but as the examples appear to be clear in this regard I think we are able to mark it as resolved? If there are code examples where this is not clear though we should fix them but I cannot spot them.

@rdebusscher Can this issue be closed as already done (in PR https://github.com/eclipse/microprofile-lra/pull/165)?

@mmusgrov If the rest of the group is ok with the current state of the examples in the spec doc, it is fine to close this issue.

@mmusgrov When everyone is happy with the current indication of code examples in the spec, this can be closed for me.
@ochaloup Do we need more comments what each statement does?

@rdebusscher I think there was not clear agreement on what would be the best way to proceed with this issue.
I expressed my feelings that I think some of the code snippets are valuable for user as it shows how to code with the LRA. I mean things like Response.ok().entity(ParticipantStatus.Compensated).build(). Other parts are not so obvious but it's only my point of view.
In summary, I think I'm fine to be left as it is as I can't think about some new proposal that would be with alignment with all what was discussed here.
Feel free to close this. Thanks you for the idea and discussion.

+1 - thanks for the idea. My preference would be to close it or at least agree it is not necessary for 1.0 . If we spot specific instances where code that is not marked as business logic is detected each of those are corrected prior to 1.0 as separate issues to align with the current approach which is to add the comment along the lines of // business logic method <BRIEF DESCRIPTION>.

So everyone has agreed it is okay to close this issue and if anyone finds a code snippet where it is not clear then they should raise a specific PR for it.


feat: publish to NPM

Outside the range of being able to leave review comments but just noticed .github/workflows/ci.yml:4 wasn't updated with default branch naming change (still has master tho the branch is main)
https://github.com/signalfx/splunk-otel-js-browser/blob/16c41be9f46d340854445ea8a25a68b70411605f/.github/workflows/ci.yml#L4


getting-started new-project doc error on windows

from https://biffweb.com/docs/get-started/new-project/
On windows 11 PowerShell:
❯  clj -M -e '(load-string (slurp ""https://biffweb.com/new.clj""))'
Syntax error compiling at (REPL:1:14).
No such namespace: https:

Full report at:
C:\Users\mail\AppData\Local\Temp\clojure-366610206050372560.edn

This work fine:
❯ clj -M -e '(load-string (slurp "https://biffweb.com/new.clj"))'

Interesting--I just tested this on Windows 10 again and I still get the exact reverse behavior. Maybe MS changed the way powershell escaping works in Windows 11? What version of clj are you on/how did you install it? (Not sure it would make any difference, just curious).
If the single-quote works on Windows 11, that sounds great to me--I'd love to only list one command on the create-a-project page anyway. I guess anyone using powershell (on Windows 10 at least?) will hopefully be familiar already with the need to add quotes to clj commands, so if they do run into problems maybe it won't be a big deal anyway. I'll go ahead and update the docs, and if someone opens another issue maybe I'll add a note or something.

Sorry, I must have missed a notification on this.
I'm using PowerShell 7.4.5.

ah--just did a little googling on that; I didn't even realize there were two different versions of PowerShell. I've updated the website to say that the escaping may be necessary; that oughtta cover all the bases:
clj -M -e '(load-string (slurp "https://biffweb.com/new.clj"))'

# In some versions of PowerShell, you may need to escape the quotes:
clj -M -e '(load-string (slurp ""https://biffweb.com/new.clj""))'


Partner A/B (re-ordering)

I thought we had this feature (drag and drop partner order) but I'm not sure what happened to it. It might have been removed due to some other bug

Mhm. I tried it locally and couldn't reproduce. Were you unable to drag, or did it error when you attempted to save? Either way, could you include the traceback? I can't really think of anything that would affect this.

Happens when assignment is past lock_date - need a message to clarify this


tabelle in pagine web: estrarre autore e numero delle righe

In queste pagine web al paragrafo  Notable Fixes, ci sono delle tabelle, una sotto l'altra, con varie righe e colonne; ogni tabella è caratterizzata da un numero di righe, da un autore e da chi ha finanziato la risoluzione dei bug, sotto un esempio:

come estrarre, per ogni tabella, il numero di righe, autore e finanziatore?
sotto un esempio di output

Bugs fixed by
These bugfixes were funded by
numero righe

Even Rouault
QGIS.ORG (through donations and sustaining memberships)
15

Alessandro Pasotti
QGIS.ORG (through donations and sustaining memberships)
18

Puoi iniziare a esplorare la pagina con VisiData
vd https://www.qgis.org/en/site/forusers/visualchangelog326/index.html#notable-fixes

Non appena possibile, torno con soluzione più pensata e cucita

vd https://www.qgis.org/en/site/forusers/visualchangelog326/index.html#notable-fixes

conosco VisiData e anche il comando: spettacolare!
manca solo l'autore e sponsor

Ciao @pigreco,
per questo tipo di task, devi imparare a fare query XPATH o CSS Selector.
Poi devi guardare la struttura della pagina, capire se c'è qualche elemento utile per distinguere la parte di tuo interesse da tutto il resto.
La parte di tuo interesse è dentro un tag section con id="notable-fixes".
La query XPATH per selezionare quella parte è //section[@id="notable-fixes"], che vuol dire: trovami un tag section ovunque nella pagina, ma che abbia come id il valore notable-fixes.
Queste query le puoi testare anche nel browser.

Un altro elemento interessante di questa struttura HTML è che per ogni user, c'è una sub sezione con id uguale al nome dell'user.

Ho fatto uno script bash, che per grandi linee fa questo:

scarica la pagina;
scarica l'elenco degli id degli user

bug-fixes-by-even-rouault
bug-fixes-by-alessandro-pasotti
bug-fixes-by-alex-bruy
bug-fixes-by-sandro-santilli
bug-fixes-by-nyall-dawson

per ognuno di questi id

estrae il nome dello user
conta il numero di righe di ogni tabella
estrare la frase sul chi ha finanziato
crea una linea in formato JSON

{"nome":"Even Rouault   ","numeroRighe":"16","funded":"These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)"}

e infine converte il tutto in CSV.

nome,numeroRighe,funded
Even Rouault,16,These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)
Alessandro Pasotti,19,These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)
Alex Bruy,12,These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)
Sandro Santilli,12,These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)
Nyall Dawson,39,These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)

Come tool uso scrape (per le query XPATH), miller, e xq.

#!/bin/bash

set -x
set -e
set -u
set -o pipefail

folder="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

URL="https://www.qgis.org/en/site/forusers/visualchangelog326/index.html#notable-fixes"

# scarica pagina
curl -kL "$URL" >"$folder"/tmp.html

# estrai id persone
scrape <"$folder"/tmp.html -be '//section[@id="notable-fixes"]/section' | xq -r '.html.body.section[]."@id"' >"$folder"/toto-id.txt

if [ -f "$folder"/toto.jsonl ]; then
  rm "$folder"/toto.jsonl
fi

# per ogni utente estrai dati
while read id; do
  nome=$(scrape <"$folder"/tmp.html -e '//section[@id="'"$id"'"]/h3/a[1]/text()' | sed -r 's/^.+by *//')
  numeroRighe=$(scrape <"$folder"/tmp.html -be '//section[@id="'"$id"'"]/table//tr' | xq '.html.body.tr|length')
  funded=$(scrape <"$folder"/tmp.html -be '//section[@id="'"$id"'"]//p[contains(.,"funded")]' | xq -r '(.html.body.p."#text")+""+(.html.body.p.a."#text")')
  echo '{"nome":"'"$nome"'","numeroRighe":"'"$numeroRighe"'","funded":"'"$funded"'"}' >>"$folder"/toto.jsonl
done <"$folder"/toto-id.txt

mlr --j2c clean-whitespace "$folder"/toto.jsonl >"$folder"/toto.csv

lista delle pagine web da cui scaricare i dati:
https://www.qgis.org/en/site/forusers/visualchangelog36/index.html
https://www.qgis.org/en/site/forusers/visualchangelog38/index.html
https://www.qgis.org/en/site/forusers/visualchangelog310/index.html
https://www.qgis.org/en/site/forusers/visualchangelog312/index.html
https://www.qgis.org/en/site/forusers/visualchangelog314/index.html
https://www.qgis.org/en/site/forusers/visualchangelog316/index.html
https://www.qgis.org/en/site/forusers/visualchangelog318/index.html
https://www.qgis.org/en/site/forusers/visualchangelog320/index.html
https://www.qgis.org/en/site/forusers/visualchangelog322/index.html
https://www.qgis.org/en/site/forusers/visualchangelog324/index.html
https://www.qgis.org/en/site/forusers/visualchangelog326/index.html

ecco il risultato

date
version
nome
numeroRighe
funded

2019-02-22
QGIS 3.6
Alessandro Pasotti
29
This feature was funded byQGIS.ORG donors and sponsors

2019-02-22
QGIS 3.6
Alexander Bruy
28
This feature was funded byQGIS.ORG donors and sponsors

2019-02-22
QGIS 3.6
Even Rouault
7
This feature was funded byQGIS.ORG donors and sponsors

2019-02-22
QGIS 3.6
Hugo Mercier
10
This feature was funded byQGIS.ORG donors and sponsors

2019-02-22
QGIS 3.6
Julien Cabieces
10
This feature was funded byQGIS.ORG donors and sponsors

2019-02-22
QGIS 3.6
Jürgen Fischer
21
This feature was funded byQGIS.ORG donors and sponsors

2019-02-22
QGIS 3.6
Loïc Bartoletti
6
This feature was funded byQGIS.ORG donors and sponsors

2019-02-22
QGIS 3.6
Martin Dobias
9
This feature was funded byQGIS user group Germany

2019-02-22
QGIS 3.6
Nyall Dawson
21
This feature was funded byQGIS.ORG donors and sponsors

2019-02-22
QGIS 3.6
Peter Petrik
9
This feature was funded byQGIS.ORG donors and sponsors

2019-02-22
QGIS 3.6
Victor Olaya
11
This feature was funded byQGIS.ORG donors and sponsors

2019-06-21
QGIS 3.8
Alessandro Pasotti
34
This feature was funded byQGIS.ORG donors and sponsors

2019-06-21
QGIS 3.8
Alexander Bruy
16
This feature was funded byQGIS.ORG donors and sponsors

2019-06-21
QGIS 3.8
Denis Rouzaud
2
This feature was funded byQGIS.ORG donors and sponsors

2019-06-21
QGIS 3.8
Even Rouault
10
This feature was funded byQGIS.ORG donors and sponsors

2019-06-21
QGIS 3.8
Loïc Bartoletti
5
This feature was funded byQGIS.ORG donors and sponsors

2019-06-21
QGIS 3.8
Peter Petrik
8
This feature was funded byQGIS.ORG donors and sponsors

2019-06-21
QGIS 3.8
Victor Olaya
11
This feature was funded byQGIS.ORG donors and sponsors

2019-10-25
QGIS 3.10
Alessandro Pasotti
41
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2019-10-25
QGIS 3.10
Alexander Bruy
20
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2019-10-25
QGIS 3.10
Even Rouault
14
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2019-10-25
QGIS 3.10
Matthias Kuhn
5
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2019-10-25
QGIS 3.10
Nyall Dawson
75
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2019-10-25
QGIS 3.10
Paul Blottiere
6
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2019-10-25
QGIS 3.10
Peter Petrik
9
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2019-10-25
QGIS 3.10
Sandro Santilli
10
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-02-21
QGIS 3.12
Alessandro Pasotti
31
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-02-21
QGIS 3.12
Alexander Bruy
5
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-02-21
QGIS 3.12
Bertrand Rix
10
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-02-21
QGIS 3.12
Denis Rouzaud
8
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-02-21
QGIS 3.12
Even Rouault
10
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-02-21
QGIS 3.12
Julien Cabieces
10
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-02-21
QGIS 3.12
Loïc Bartoletti
12
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-02-21
QGIS 3.12
Nyall Dawson
23
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-02-21
QGIS 3.12
Paul Blottiere
8
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-02-21
QGIS 3.12
Sandro Santilli
6
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-02-21
QGIS 3.12
Sebastien Peillet
8
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-02-21
QGIS 3.12
Stephen Knox
2

2020-06-19
QGIS 3.14
Alessandro Pasotti
32
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-06-19
QGIS 3.14
Alexander Bruy
16
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-06-19
QGIS 3.14
Audun Ellertsen
3
This feature was funded byKongsberg Digital

2020-06-19
QGIS 3.14
Bertrand Rix
5
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-06-19
QGIS 3.14
Denis Rouzaud
7
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-06-19
QGIS 3.14
Even Rouault
18
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-06-19
QGIS 3.14
Julien Cabieces
14
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-06-19
QGIS 3.14
Loïc Bartoletti
6
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-06-19
QGIS 3.14
Nyall Dawson
67
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-06-19
QGIS 3.14
Paul Blottiere
9
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-06-19
QGIS 3.14
Sebastien Peillet
7
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-10-23
QGIS 3.16
Alessandro Pasotti
45
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-10-23
QGIS 3.16
Denis Rouzaud
9
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-10-23
QGIS 3.16
Even Rouault
21
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-10-23
QGIS 3.16
Julien Cabieces
24
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-10-23
QGIS 3.16
Matthias Kuhn
5
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-10-23
QGIS 3.16
Nyall Dawson
84
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-10-23
QGIS 3.16
Olivier Dalang
2
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-10-23
QGIS 3.16
Paul Blottiere
12
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2020-10-23
QGIS 3.16
Peter Petrik
49
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2021-02-22
QGIS 3.18
Alessandro Pasotti
24
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2021-02-22
QGIS 3.18
Even Rouault
12
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2021-02-22
QGIS 3.18
Julien Cabieces
10
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2021-02-22
QGIS 3.18
Nyall Dawson
32
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2021-02-22
QGIS 3.18
Peter Petrik
15
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2021-06-21
QGIS 3.20
Alessandro Pasotti
30
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2021-06-21
QGIS 3.20
Denis Rouzaud
10
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2021-06-21
QGIS 3.20
Even Rouault
15
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2021-06-21
QGIS 3.20
Julien Cabieces
9
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2021-06-21
QGIS 3.20
Loïc Bartoletti
8
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2021-06-21
QGIS 3.20
Nyall Dawson
47
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2021-06-21
QGIS 3.20
Paul Blottiere
8
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2021-06-21
QGIS 3.20
Peter Petrik
7
This feature was funded byQGIS.ORG (through donations and sustaining memberships)

2021-10-22
QGIS 3.22
Alessandro Pasotti
27
These bug fixes were funded byQGIS.ORG (through donations and sustaining memberships)

2021-10-22
QGIS 3.22
Denis Rouzaud
2
These bug fixes were funded byQGIS.ORG (through donations and sustaining memberships)

2021-10-22
QGIS 3.22
Even Rouault
16
These bug fixes were funded byQGIS.ORG (through donations and sustaining memberships)

2021-10-22
QGIS 3.22
Julien Cabieces
12
These bug fixes were funded byQGIS.ORG (through donations and sustaining memberships)

2021-10-22
QGIS 3.22
Loïc Bartoletti
10
These bug fixes were funded byQGIS.ORG (through donations and sustaining memberships)

2021-10-22
QGIS 3.22
Nyall Dawson
25
These bug fixes were funded byQGIS.ORG (through donations and sustaining memberships)

2021-10-22
QGIS 3.22
Peter Petrik
9
These bug fixes were funded byQGIS.ORG (through donations and sustaining memberships)

2021-10-22
QGIS 3.22
Sandro Santilli
11
These bug fixes were funded byQGIS.ORG (through donations and sustaining memberships)

2022-02-18
QGIS 3.24
Alessandro Pasotti
28
These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)

2022-02-18
QGIS 3.24
Alex Bruy
22
These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)

2022-02-18
QGIS 3.24
Damiano Lombardi
2
These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)

2022-02-18
QGIS 3.24
Denis Rouzaud
4
These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)

2022-02-18
QGIS 3.24
Even Rouault
9
These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)

2022-02-18
QGIS 3.24
Matthias Kuhn
2
These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)

2022-02-18
QGIS 3.24
Nyall Dawson
30
These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)

2022-02-18
QGIS 3.24
Paul Blottiere
6
These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)

2022-02-18
QGIS 3.24
Sandro Santilli
8
These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)

2022-06-18
QGIS 3.26
Alessandro Pasotti
19
These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)

2022-06-18
QGIS 3.26
Alex Bruy
12
These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)

2022-06-18
QGIS 3.26
Even Rouault
16
These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)

2022-06-18
QGIS 3.26
Nyall Dawson
39
These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)

2022-06-18
QGIS 3.26
Sandro Santilli
12
These bugfixes were funded byQGIS.ORG (through donations and sustaining memberships)

statistiche

name
nroBugFix

Nyall Dawson
433

Alessandro Pasotti
329

Even Rouault
137

Peter Petrik
99

Julien Cabieces
82

Alexander Bruy
80

Paul Blottiere
43

Sandro Santilli
42

Loïc Bartoletti
41

Denis Rouzaud
35

Alex Bruy
32

Jürgen Fischer
20

Victor Olaya
20

Bertrand Rix
13

Sebastien Peillet
13

Hugo Mercier
9

Matthias Kuhn
9

Martin Dobias
8

Audun Ellertsen
2

Damiano Lombardi
1

Olivier Dalang
1

Stephen Knox
1

@aborruso
grazie mille per l'esaustiva spiegazione, sembra tutto facile quando spieghi le cose; ti invidio tanto perché sono strumenti che mi piacerebbe molto saper usare, ma qui ci vuole molta esperienza e creatività per capire cosa cercare e filtrare.
grazie mille per il tempo che ci hai dedicato

@aborruso
non riesco a fare il ciclo FOR su un insieme di link, questo script non funziona
#!/bin/bash

set -x
set -e
set -u
set -o pipefail

folder="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

LINK="https://www.qgis.org/en/site/forusers/visualchangelog36/index.html#notable-fixes
https://www.qgis.org/en/site/forusers/visualchangelog38/index.html#notable-fixes
https://www.qgis.org/en/site/forusers/visualchangelog310/index.html#notable-fixes
https://www.qgis.org/en/site/forusers/visualchangelog312/index.html#notable-fixes
https://www.qgis.org/en/site/forusers/visualchangelog314/index.html#notable-fixes
https://www.qgis.org/en/site/forusers/visualchangelog316/index.html#notable-fixes
https://www.qgis.org/en/site/forusers/visualchangelog318/index.html#notable-fixes
https://www.qgis.org/en/site/forusers/visualchangelog320/index.html#notable-fixes
https://www.qgis.org/en/site/forusers/visualchangelog322/index.html#notable-fixes
https://www.qgis.org/en/site/forusers/visualchangelog324/index.html#notable-fixes
https://www.qgis.org/en/site/forusers/visualchangelog326/index.html#notable-fixes"

# crea ciclo con le pagine web
for lista in $LINK;do
# scarica pagina
curl -kL "$lista" >"$folder"/tmp.html

# estrai id persone
scrape <"$folder"/tmp.html -be '//section[@id="notable-fixes"]/section' | xq -r '.html.body.section[]."@id"' >"$folder"/toto-id.txt

if [ -f "$folder"/toto.jsonl ]; then
  rm "$folder"/toto.jsonl
fi

# per ogni utente estrai dati
while read id; do
  versione=`echo "$LINK" | sed -E s/\\d+//`
  nome=$(scrape <"$folder"/tmp.html -e '//section[@id="'"$id"'"]/h3/a[1]/text()' | sed -r 's/^.+by *//')
  numeroRighe=$(scrape <"$folder"/tmp.html -be '//section[@id="'"$id"'"]/table/tbody/tr' | xq '.html.body.tr|length')
  funded=$(scrape <"$folder"/tmp.html -be '//section[@id="'"$id"'"]//p[contains(.,"funded")]' | xq -r '(.html.body.p."#text")+""+(.html.body.p.a."#text")')
  echo '{"versione":""'"$versione"'",nome":"'"$nome"'","numeroRighe":"'"$numeroRighe"'","funded":"'"$funded"'"}' >>"$folder"/toto.jsonl
done <"$folder"/toto-id.txt

mlr --j2c clean-whitespace "$folder"/toto.jsonl >>"$folder"/toto.csv
done

@aborruso
ora funziona il ciclo, ma ho errore in Miller:
+ mlr --j2c clean-whitespace /mnt/c/Users/pigre/Desktop/featureQGIS/toto.jsonl
mlr: Unable to parse JSON data: Line 1 column 5: Unexpected `h` in object

script
#!/bin/bash

set -x
set -e
set -u
set -o pipefail

folder="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

LINK="https://www.qgis.org/en/site/forusers/visualchangelog36/index.html#notable-fixes
https://www.qgis.org/en/site/forusers/visualchangelog38/index.html#notable-fixes"

# crea ciclo con le pagine web

for lista in $LINK
do

# scarica pagina
	curl -kL "$lista" >"$folder"/tmp.html

# estrai id persone
	scrape <"$folder"/tmp.html -be '//section[@id="notable-fixes"]/section' | xq -r '.html.body.section[]."@id"' >"$folder"/toto-id.txt

# per ogni utente estrai dati
	while read id; do
		versione=$lista
		nome=$(scrape <"$folder"/tmp.html -e '//section[@id="'"$id"'"]/h3/a[1]/text()' | sed -r 's/^.+by *//')
		numeroRighe=$(scrape <"$folder"/tmp.html -be '//section[@id="'"$id"'"]/table/tbody/tr' | xq '.html.body.tr|length')
		funded=$(scrape <"$folder"/tmp.html -be '//section[@id="'"$id"'"]//p[contains(.,"funded")]' | xq -r '(.html.body.p."#text")+""+(.html.body.p.a."#text")')
		echo '{"versione":""'"$versione"'",nome":"'"$nome"'","numeroRighe":"'"$numeroRighe"'","funded":"'"$funded"'"}' >>"$folder"/toto.jsonl
	done <"$folder"/toto-id.txt

	if [ -f "$folder"/toto-id.txt ]; then
	rm "$folder"/toto-id.txt
	fi
done

mlr --j2c clean-whitespace "$folder"/toto.jsonl >>"$folder"/toto.csv

@aborruso
ho trovato gli errori, sono qui:
echo '{"versione":""'"$versione"'",nome":"'"$nome"'","numeroRighe":"'"$numeroRighe"'","funded":"'"$funded"'"}' >>"$folder"/toto.jsonl
ci sono " messi male
ora funziona!!!

questo script cicla su tutti le pagine web e crea unico file csv:
#!/bin/bash

set -x
set -e
set -u
set -o pipefail

folder="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

LINK="https://www.qgis.org/en/site/forusers/visualchangelog36/index.html
https://www.qgis.org/en/site/forusers/visualchangelog38/index.html
https://www.qgis.org/en/site/forusers/visualchangelog310/index.html
https://www.qgis.org/en/site/forusers/visualchangelog312/index.html
https://www.qgis.org/en/site/forusers/visualchangelog314/index.html
https://www.qgis.org/en/site/forusers/visualchangelog316/index.html
https://www.qgis.org/en/site/forusers/visualchangelog318/index.html
https://www.qgis.org/en/site/forusers/visualchangelog320/index.html
https://www.qgis.org/en/site/forusers/visualchangelog322/index.html
https://www.qgis.org/en/site/forusers/visualchangelog324/index.html
https://www.qgis.org/en/site/forusers/visualchangelog326/index.html"

# crea ciclo con le pagine web

for lista in $LINK
do

# scarica pagina
	curl -kL "$lista" >"$folder"/tmp.html

# estrai id persone
	scrape <"$folder"/tmp.html -be '//section[@id="notable-fixes"]/section' | xq -r '.html.body.section[]."@id"' >"$folder"/toto-id.txt

# per ogni utente estrai dati
	while read id; do
		versione=`echo "$lista" | sed -e 's/[^0-9]//g'`
		nome=$(scrape <"$folder"/tmp.html -e '//section[@id="'"$id"'"]/h3/a[1]/text()' | sed -r 's/^.+by *//')
		numeroRighe=$(scrape <"$folder"/tmp.html -be '//section[@id="'"$id"'"]/table/tbody/tr' | xq '.html.body.tr|length')
		funded=$(scrape <"$folder"/tmp.html -be '//section[@id="'"$id"'"]//p[contains(.,"funded")]' | xq -r '(.html.body.p."#text")+""+(.html.body.p.a."#text")')
		echo '{"versione":"'"$versione"'","nome":"'"$nome"'","numeroRighe":"'"$numeroRighe"'","funded":"'"$funded"'"}' >>"$folder"/toto.jsonl
	done <"$folder"/toto-id.txt

	if [ -f "$folder"/toto-id.txt ]; then
	rm "$folder"/toto-id.txt
	fi
done

mlr --j2c clean-whitespace "$folder"/toto.jsonl >>"$folder"/toto.csv

Ciao @aborruso

Ma questa perché non è chiusa, con ricetta?
non ricordo perché è ancora aperta, ma non credo di aver fatto ricetta; le farò, ma non so quando :-(

ricetta fatta e pubblicata:
https://tansignari.opendatasicilia.it/ricette/bash/tabelle_in_pagine_web_estrarre_autore_e_nro_righe/
grazie mille(r) @aborruso


Wwang talend/tdi 39964 revert

What is the current behavior? (You can also link to an open issue here)
revert the fix for https://jira.talendforge.org/browse/TDI-39964
What is the new behavior?
revert the fix for https://jira.talendforge.org/browse/TDI-39964
Please check if the PR fulfills these requirements

[x] The commit message follows Talend standard
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features) ?
[ ] The code coverage on new code >75%
[ ] The new code does not introduce new technical issues (sonar / eslint)

What kind of change does this PR introduce?

[x] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build / CI related changes
[ ] Other... Please describe:

Does this PR introduce a breaking change?

[ ] Yes
[x] No

please don't merge this


Add UBSan kokoro configuration

Adds a kokoro configuration for UBSan, to allow this sanitizer to be
used during continuous integration.

Thanks for the approval, @ben-clayton - I also just added you as a reviewer to the CL for the internal config stuff.


Refreshing live video causes audio backchannel button to be disabled

Refreshing any live video window (even ones which don't have the audio backchannel button enabled) will cause the backchannel button to be disabled when you go to a live stream which includes it. Clicking the home button and going back to the video page will enable the backchannel button again.

Caused by isGuest state. isGuest in the video component was copied from isGuestAccount in the utilsService before its value was set there.


Add gRPC load test example

This PR has two commits:

The first one changes the call to the stop() instance method in the runner so that it can be overridden.
The second one (which needs the first commit) adds an example on how to implement a gRPC load test to avoid #1676; please note that I created a PR (https://github.com/grpc/grpc/pull/26058) to solve the root cause in grpcio, but I'm not sure if/when will be merged.

Looks good! Just auto-format the files using Black and I can merge.

thanks!

@beandrad Please ask weather support grpc BidirectStrem interface? I used Generator to grpc stream method, but the test case was stuck


Godot 4.0 Rework

I'm porting this to Godot 4.0 and cleaning some things up on the way.
Here's what still needs to be done:

[x] Add button Audio playback
[x] Make sure it can find redguardunity.exe
[x] Fix popup window disappearing when clicked away or file dialog opens
[x] (optional) Make popup window a new scene
[x] Disable main buttons when the options window is open
[x] Fix font size in options window

Some more minor fixes that could be done:

[ ] "Play Redguard" button plays not audio because launcher shuts itself down too fast


copied project from trails

-Added  trails projects
-added linters

The project is well done to go.
You have followed all the required changes recommended by your code reviewers.
It is now ready to


PSCIS Phase 1 duplicates Bulkley

Hi Simon, here is a table showing all the pscis ids for our phase 1 sites. It was entered into the system twice. We'd like to retain everything from stream_crossing_id 198109 down to 198117, and get rid of the duplicates which are from 198108 up to 198100. In reference to issue here: https://github.com/NewGraphEnvironment/fish_passage_bulkley_2022_reporting/issues/41

Thanks Mateo! Any chance you could post the ids as text so I can copy/paste?

csv is attached as well
pscis_phase1_dups.csv


border: fix accessing union bug

Normally, field accessing like task_struct.sched_entity.load is correctly
handled in an AST tree like,
     ComponentRef
target |  field  \
       |          \
ComponentRef     load
       |    \
target |field\
       |      \
task_struct  sched_entity

However, when accessing field through a unnamed union, for example
sched_class[.unamed_union].pick_next_task. This is introduced in
RHEL/CentOS with the RH_KABI_REPLACE, which creates unions wrapper
for fields. Developers omit [.unamed_union] in the code because
they're unamed. We get an AST tree like,
     ComponentRef
target |  field  \
       |          \
ComponentRef     pick_next_task
       |    \
target |field\
       |      \
sched_class (unamed KABI replacing union)

Before this patch, collect.py finds

ComponentRef1(sched_class, [unamed_union])
ComponentRef2(CompnentRef1, pick_next_task)

This patch compress these two operations into one,

ComponentRef(sched_class, pick_next_task)

Signed-off-by: Yihao Wu<EMAIL_ADDRESS>

/retest


Make cmd_lists_count public in DrawData struct

Also updated comments with latest from imgui.
Relates to https://github.com/Gekkio/imgui-rs/issues/325

This should probably be a getter method instead. Making it public allows the value to be mutated which makes draw_lists() unsafe:
    pub fn draw_lists(&self) -> DrawListIterator {
        unsafe {
            DrawListIterator {
                iter: self.cmd_lists().iter(),
            }
        }
    }
    pub(crate) unsafe fn cmd_lists(&self) -> &[*const DrawList] {
        slice::from_raw_parts(
            self.cmd_lists as *const *const DrawList,
            self.cmd_lists_count as usize,
        )
    }

Changed to a setter as requested.

cc @Gekkio

Thanks! :smile:
Would it make more sense to return usize instead of i32? The public count fields are i32 to match the underlying memory layout, but in a getter function we have the option of casting the value to something more appropriate for Rust code. Note also that you can already get the count with draw_lists().count(), which returns usize.

Changed getter to return usize instead of u32.


Module doesn't work when strict vars are on

Hiya. When strict variables are on the module doesn't work.
The first place that gets hit is https://github.com/pcfens/puppet-filebeat/blob/master/manifests/init.pp#L93
$::filebeat_version fact doesn't exist and with strict variables it blows up.
Evaluation Error: Unknown variable: '::filebeat_version'.
You might want to run some acceptance tests with strict vars enabled and fix some things ;-)

Thanks for finding this - I thought I was testing using strict variables so I'll dive in a little bit deeper.
Aside from better testing, this should be fixed in ede4c4a. I'll push an updated release to the forge soon.

Thanks or fixing it so quick! Will report if i find anything else.


x/tools/gopls: add mode/option/mechanism to allow automatic updates to go.mod

What version of Go are you using (go version)?

$ go version
go version devel +dfdc3880b0 Thu Sep 10 02:22:19 2020 +0000 linux/amd64
$ go list -m golang.org/x/tools
golang.org/x/tools v0.0.0-20200915031644-64986481280e
$ go list -m golang.org/x/tools/gopls
golang.org/x/tools/gopls v0.0.0-20200915031644-64986481280e

Does this issue reproduce with the latest release?
n/a
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/myitcv/.cache/go-build"
GOENV="/home/myitcv/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/myitcv/gostuff/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/myitcv/gostuff"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/myitcv/gos"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/myitcv/gos/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/myitcv/gostuff/src/github.com/myitcv/govim/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build336384284=/tmp/go-build -gno-record-gcc-switches"

What did you do?
By default gopls does not not automatically update a user's go.mod file, a setting established via the TempModfile approach:
https://github.com/golang/tools/blob/64986481280e06c512dcd7674532f4f7f4cfc0c6/internal/lsp/source/options.go#L101
In this mode, in order to make changes to their go.mod file, a user either needs to:

accept the suggestions from gopls
run go mod tidy
run go test or similar

Alternatively, with TempModfile turned off, go.mod files are updated as a side effect of cmd/go's various actions, and both gopls and cmd/go use and see the same source of truth.
https://github.com/golang/go/issues/40728 however moves us to a situation where -mod=readonly effectively becomes the default for many cmd/go commands which appears to rule out us turning off TempModfile. Therefore the only option is 1 or 2 from above, i.e. manually intervening.
This makes the following workflow quite painful:

accept unimported completion candidate
run go test which now fails under https://github.com/golang/go/issues/40728
have to manually accept change to go.mod ~because gopls is maintaining a shadow go.mod where the change has been made but the real go.mod is as yet unaltered (this is effectively a variant of the concern raised in  https://github.com/golang/go/issues/32394#issuecomment-689542541)~ (edit: see https://github.com/golang/go/issues/41397#issuecomment-692833408)

The first step, is, to my mind an implicit acceptance of the fact that a dependency has been added. Indeed, given the import statement is added to the file in question I would suggest the requirement should also also be added. But I can see that some people might want/prefer the explicit step.
Hence, this issue is a request that some mode/option/mechanism be maintained whereby the user can continue to opt in to automatically having changes applied to their go.mod file. The example presented involves accepting unimported completions, but generally speaking I do want/need gopls to make changes to my go.mod file as a side effect of what I do via gopls.

cc @stamblerre @heschik @findleyr
FYI @leitzler

I'm not sure if this will be necessary because the Go command's behavior is changing too: https://github.com/golang/go/issues/40728.
The -mod=mod flag that gopls is currently relying on will likely be deprecated in 1.16, so I think that the only mechanism gopls would have of providing these updates would be something like automatically running go mod tidy on every Go file save (which would be too expensive IMO).
BTW, regarding:

gopls is maintaining a shadow go.mod where the change has been made but the real go.mod is as yet unaltered (this is effectively a variant of the concern raised in #32394 (comment))

gopls doesn't actually maintain a temporary go.mod file that it updates and reuses. Any call to the go command gets its own new temporary go.mod file that is a copy of whatever is currently in the existing go.mod file.
/cc @jayconrod

We have previously discussed modifying goimports to provide the specific module paths and versions for the imports it suggests (particularly if they are not already in the build list). Does gopls currently modify go.mod when it runs goimports?

gopls doesn't actually maintain a temporary go.mod file that it updates and reuses.

Perhaps I'm misunderstanding something then. With TempModfile: true, what happens when I accept an unimported completion that requires a new dependency? The real go.mod file is not changed (by definition), but a change must be made/maintained somewhere, not least because it gives rise to the suggested edit to the real go.mod file?

Does gopls currently modify go.mod when it runs goimports?

It uses -modfile with a temporary go.mod so that any go list commands it runs will modify that.

but a change must be made/maintained somewhere, not least because it gives rise to the suggested edit to the real go.mod file?

The two are separate actually. Like @bcmills said, unimported completions don't have module paths and versions associated with them, so it only add imports to Go files. gopls also runs go mod tidy any time an import in any Go file is changed (on disk), which is where the go mod tidy diagnostics come from.

Thanks for the clarification on the implementation details, @stamblerre. I've edited my misunderstanding in the description, linking to your explanation in the previous comment.

gopls also runs go mod tidy any time an import in any Go file is changed (on disk), which is where the go mod tidy diagnostics come from.

So this issue is ultimately about auto-applying these diagnostics. I guess options here range from a gopls mode/option, to the client being able to precisely identify them as "go mod tidy diagnostics" and auto-apply them.
But it sounds like we also need the functionality @bcmills describes. Is that covered elsewhere?

to the client being able to precisely identify them as "go mod tidy diagnostics" and auto-apply them.

This is what VS Code does. Any go mod tidy diagnostics are classified as source.organizeImports, so VS Code can apply them on save.

But it sounds like we also need the functionality @bcmills describes. Is that covered elsewhere?

I'm not sure about this. @heschik would probably know though.

This is what VS Code does. Any go mod tidy diagnostics are classified as source.organizeImports, so VS Code can apply them on save.

Interesting. So is there a manual step involved at all in VSCode? Are these diagnostics automatically applied, or is there some config option that enables/disables this?

This is the configuration in the VS Code:
	"[go.mod]": {
		"editor.codeActionsOnSave": {
			"source.organizeImports": true,
		},
	},

Note that I actually have it disabled because I find it to be too slow, especially when you're typing in the go.mod file, and it might be in a bad state.

Interesting, thank you very much.
So it looks like exactly the same could/should be achieved in govim. i.e. we switch to TempModfile: true and then use an option/mode equivalent to the above to enable/disable this.

Note that I actually have it disabled because I find it to be too slow, especially when you're typing in the go.mod file, and it might be in a bad state.

Can you expand on this? Is gopls too slow when enabled? How does typing in the go.mod affect something that happens on save?

So it looks like exactly the same could/should be achieved in govim. i.e. we switch to TempModfile: true and then use an option/mode equivalent to the above to enable/disable this.

Sure, but I just want to reiterate the default behavior of the Go command will be changing. So it may be worth matching that because otherwise you'll have the reverse issue--gopls will introduce edits, but go test, etc. won't.

Can you expand on this? Is gopls too slow when enabled? How does typing in the go.mod affect something that happens on save?

I tend to save as I type a lot, which is why I was running into this. Probably much less of an issue with Vim. VS Code also has removed the codeActionOnSaveTimeout feature which used to prevent slow saves.

So it may be worth matching that because otherwise you'll have the reverse issue--gopls will introduce edits, but go test, etc. won't.

For a default I think you're probably right. As I said above, I fall into the "using my editor means I'm implicitly giving the LSP the right to make changes to files I'm working with" camp. But I can quite see why others would prefer the more explicit behaviour this new default introduces, with manual accepts of suggested edits.

I tend to save as I type a lot, which is why I was running into this. Probably much less of an issue with Vim. VS Code also has removed the codeActionOnSaveTimeout feature which used to prevent slow saves.

Thanks, that's useful context.
And thanks again for taking the time to explain this.
So, on the basis I think we have a solution here, I'm tempted to suggest we close this?
Unless you want to leave it open, @stamblerre, perhaps to track your question to @heschik in https://github.com/golang/go/issues/41397#issuecomment-693153430?

I think closing is fine--if there was additional work needed to improve goimports in some way, I'm sure that Heschi would open an issue, so I'm guessing this isn't actually necessary.

I just tried this within govim against<PHONE_NUMBER>0e (Go version eaa97fbf20). The setup was as follows:
-- main.go --
package main

func main() {
}
-- go.mod --
module mod.com

go 1.16

I then changed main.go as follows (simulating an unimported completion) and saved that file:
package main

import (
	"fmt"

	"github.com/kr/pretty"
)

func main() {
	fmt.Printf("%v", pretty.Sprint(5))
}

With TempModfile: false

go.mod changes happen as a side effect of gopls' normal operation
no diagnostics received for go.mod
diagnostics received for main.go in the period before the file save, after which they disappear

All as expected.
With TempModfile: true

no automatic changes to go.mod
diagnostics received for main.go in the period before the file save, after which they resort to github.com/kr/pretty is not in your go.mod file
diagnostics received for go.mod: github.com/kr/pretty is not in your go.mod file

It seems I misunderstood/interpreted something in our exchange above. Because saving main.go does not trigger  gopls to send the client edits to go.mod.
With TempModfile: true the only means by which I can make changes to my go.mod is by triggering a save of go.mod itself. As I outlined in the description, that is the kind of break in workflow I'm looking to avoid because I've already accepted that import and by definition the requirement it implies.
What I was expecting is that instead of the diagnostic github.com/kr/pretty is not in your go.mod file being sent for the go.mod file we have a workspace edit to apply. The beauty of this approach being that it is async from the original save of the main.go file, hence you don't end up blocking on a go mod edit on each save.
Re-opening because it doesn't appear the current setup supports the workflow detailed in the description.

What I was expecting is that instead of the diagnostic github.com/kr/pretty is not in your go.mod file being sent for the go.mod file we have a workspace edit to apply.

You need to request code actions for the diagnostic in the go.mod file in order to get the associated suggested fix. There is also the go mod tidy code lens which appears whenever your module is not tidy, and you could use that to always apply edits.

So it sounds like to automate this we (govim) need to either:

match diagnostics for a go.mod file against "is not in your go.mod file", then request code actions for those diagnostics, then filter the code actions that are returned (the quickfix kind? Anything more specific), and then apply them all
repeatedly ask for the go mod tidy code lens

Option 2 doesn't seem feasible because the go.mod file might not be open, and in any case govim doesn't know which go.mod file it should use in the general case.
Option 1 sounds a bit brittle, but better in that we are using the diagnostic edge as a trigger point for working out whether there is something to do. Can this option be refined in some way?

We could use the diagnostic from option 1 to know where the go.mod is located that we want to ask for in a CodeLens call.
It would still be fragile to do string matching on diagnostics. Even if we should go for string matching, the response from the CodeLens call contains several commands and we wouldn't know which one to pick (since the command names aren't public/stable).
The last part could be solved by adding tidy as an explicit CodeAction, so we can request it with a known kind (i.e. tidy/go.tidy/etc). That would align with the suggestion Heschi had at the previous tools call.

match diagnostics for a go.mod file against "is not in your go.mod file", then request code actions for those diagnostics, then filter the code actions that are returned (the quickfix kind? Anything more specific), and then apply them all

They will be the "source.organizeImports" kind - all of the go mod tidy diagnostics are categorized this way. We are considering making them errors rather than warnings in the future, so you could also get code actions for all of the error level diagnostics.
The VS Code config I shared earlier is doing effectively this - basically saying apply any organize import code actions to the go.mod file on save (though not on an Go file save).

Oh, sorry I missed that part. A source.organizeImports request would do it.
I don't quite follow the part about diagnostics being of a certain kind, are you referring to the Source in the diagnostic?
e.g.
PublishDiagnostics callback: &protocol.PublishDiagnosticsParams{
    URI:         "file:///private/var/folders/j4/l2j99h6d5qd6knjlllql0bb80000gn/T/tmp.H5PUVQQb/go.mod",
    Version:     0,
    Diagnostics: {
        {
            Range: protocol.Range{
                Start: protocol.Position{},
                End:   protocol.Position{Line:0, Character:8},
            },
            Severity:           2,
            Code:               nil,
            Source:             "go mod tidy",
            Message:            "github.com/kr/pretty is not in your go.mod file",
            Tags:               nil,
            RelatedInformation: nil,
        },
    },
}

Ah sorry, for kind, I was referring to the CodeActionKind that is part of the Only field in the request.
For error vs. warning, I was referring to the Severity field.

I see. Then I can't really see a way to automatically update go.mod as things look today since:

The client doesn't know where/what go.mod is (unless string-matching URI)
The client doesn't know if a diagnostic is about missing deps (unless string-matching message/source)

Apologies if I missed why earlier in the conversation, but why can't the code action response for my .go-file not only have an edit that adds the import, but also updates go.mod? Is it because -mod=readonly will be default?
In that case, should it do things automatically if the user specifies -mod=autoupdate? (or was that just a part of Russ original post that won't happen in reality?)

Apologies if I missed why earlier in the conversation, but why can't the code action response for my .go-file not only have an edit that adds the import, but also updates go.mod?

This should be the case. When a dependency is missing from the go.mod, you get a diagnostic for both the Go file and the go.mod file. You can then send those diagnostics back to the server as part of a code action, which will return, in this case, the same fix - adding the dependency to the go.mod file. You can specify a CodeActionKind of source.organizeImports to make sure that these are only go mod tidy diagnostics (or match on the go mod tidy source) and then apply these fixes.

In that case, should it do things automatically if the user specifies -mod=autoupdate? (or was that just a part of Russ original post that won't happen in reality?)

I don't think that gopls will obey -mod flags provided by the user, but will rather obey the TempModfile configuration. With TempModfile: false, changes will be applied directly to the files.

I don't think that gopls will obey -mod flags provided by the user, but will rather obey the TempModfile configuration. With TempModfile: false, changes will be applied directly to the files.

My assumption on opening this issue was that, even with TempModfile: false, this wouldn't be the case, precisely because of the changes being made to cmd/go in 1.16. Is gopls doing something special that causes the side effects of go list etc to result in go.{mod,sum} changes? Or have I totally missed something?

Sorry, I was speaking about the state pre-1.16 and as of right now. With gopls and Go at tip, you will actually still get the previous -mod=mod behavior, but we are working on switching to use -mod=readonly by default.

Ok, thanks for clarifying

I don't think that gopls will obey -mod flags provided by the user, but will rather obey the TempModfile configuration. With TempModfile: false, changes will be applied directly to the files.

...

but we are working on switching to use -mod=readonly by default.

At which point I think the TempModfile solution becomes redundant, correct?
I ask that only as an observation to clarify, because I think the better solution here is effectively the one we've arrived at in the course of this discussion (thanks for your patience). Namely: match diagnostics for a go.mod file against "is not in your go.mod file", then request code actions of the source.organizeImports kind for those diagnostics on that file, and then apply them all. Arguably, the matching against "is not in your go.mod file" diagnostics is not necessary, we could simply request source.organizeImports code actions whenever there are diagnostics for any go.mod file.
This has the added benefit of changes to a go.mod being initiated by gopls as opposed to happening as a side effect of a cmd/go command.

Thanks again for helping to talk this through, @stamblerre. I'll close this for now because I think we've reached agreement on a way forward. I've raised https://github.com/govim/govim/issues/956 to capture actually implementing this change.


Add a way to combine Uni and discard the emitted items

This is useful when combining Uni instances, and even various Unis where we are only interested by the completion of the operation and not the result.

Waiting a bit to merge, to see if a better name emerges.

We decided to keep discardItems.


Refactor node and npm middleware

[x] Rename all instances of Node and Npm to JavaScript
[x] Refactor middleware AddJavaScriptApp
[x] Refactor middleware AddJavaScriptCLIApp
[x] Refactor test
[ ] Refactor other files

Opening as draft because I have not refactored some satellite files I'm not very sure the mechanism of
This is an attempt and must be run
Microsoft Reviewers: Open in CodeFlow

cc @radical I was asked to help to redirect it

What is the motivation for this change?

@mitchdenny - see #1448