StochasticAlgorithm with Gradient Descent and FISTA example

Initial implementation of StochasticAlgorithm with and example of StochasticGradientDescent.
The design is as follows:

after each iteration, or any number of iteration, the Stochastic algorithm notifies that it's ready to operate on a new subset
the concrete implementation of the stochastic algorithm must forward this message to the underlying SubsetOperator

in the case of GradientDescent the operator is wrapped in the objective_function object, so the algorithm calls a method in the StochasticNorm2Sq.
The fact that I created a different class for the stochastic Norm2Sq can be discussed. I did not want to add methods here and there.
The whole machinery can be tested with these 2 files:

stocastic.py  is self contained and will work on a current 19.10 release of the framework with ccpi-astra and tomophantom
stochastic_gd.py as above, but will require you to run python setup.py install.

The second file shows how the user will be using the new machinery.

closes #124

I've updated the branch addressing #449
Let's try to resume how this implementation works starting from an example. A minimisation with GradientDescent is done as follows
# im_data  is the ground truth
# A is an Operator
# data is a sinogram with N angles (e.g. 180)

# Let's use least squares
l2 = Norm2Sq(A=A, b=data)

gd = GradientDescent(x_init=im_data*0.,
                     objective_function=l2,
                     step_size=None,
                     update_objective_interval=10, max_iteration=100)
gd.run(100)

The current code allows usage of subsets with very few changes:
# OS_A is the Subset version of the A operator before
nsubs = 10
data.generate_subsets(nsubs, 'uniform')

sl2 = StochasticNorm2Sq(A=OS_A, b=data)

sgd = StochasticGradientDescent(x_init=im_data*0.,
                                objective_function=sl2,
                                number_of_subsets=nsubs,
                                update_objective_interval=10, max_iteration=100)
sgd.run()

I will try to resume the key points of the implementation now, don't be scared, it's a bit complex.
I've added to the AcquisitionGeometry the faculty to generate indices for subsetting the entire dataset. The subset concerns only angles currently.
Now, once this subsets (or indices) have been created, the geometry will have one new members (subsets) and will have populated the members subset_id and number_of_subsets with non trivial values, especially for the latter.

data.generate_subset(number_of_subsets=10, method='uniform')
# to select a specific subset, e.g. 8
data.geometry.subset_id = 8

The call data.generate_subset is actually forwarded to the geometry of the AcquisitionData class. Once this happens it will be possible to perform all the algebra and methods available to AcquisitionData on the actual subset (number 8 in the example).
This is done transparently for the user because as_array will check if a subset is selected and return the actual subset to operate on.
This means that if the data has 180 angles and the number of subsets is 10 the size in the angle dimension is 18.
The rest of the design is the same as described before:

The Stocastic algorithm will call its method notify_new_subset after one iteration on a subset is finished
The concrete class of the stochastic algorithm must pass the call to the appropriate objects. For instance StochasticGradientDescent will pass the call to the objective function.
In this example. the objective function will need to pass the info to the data and to the Operator. This is the reason for creating a new class, which I called StochasticNorm2Sq.
The AstraSubsetOperatorSimple inherits from the AstraOperatorSimple and adds only the method notify_new_subset

A few things to notice are that

the AstraSubsetOperatorSimple holds a reference to the data.geometry, therefore it gets updated of the changes of the subsets immediately.
I modified the method clone of the AcquisitionGeometry, which now will return the appropriate geometry for a subset if the data is being subsetted. Also I added copy as an alias of clone
The implementation of the clone method is the reason that this line works.
Notice that the order of the calls matters. The data must be notified first otherwise the Operator will operate on the wrong angles.

And now an image

closes #449

closes #450

As @evelinaametova pointed out in #577, the subsetting assumes that the slicing of the data is done on the first dimension. Instead of an integer, we should pass a slice object.

The previous comment has been addressed.
Currently this requires some additional testing/unittest and adding examples. There is a directory wip which should be removed, but other than that it seems in good order to me.

Closing as superseded by #1345


Difference in drop in challenge behaviour between 3ds checks for instant payment methods and inline card numbers

Describe the bug
We have found that using the same card number 4917 6100 0000 0000 when entering in cards will display the challenge window inline. When this card is used from an instant payment method (the test card suite using a card with 3ds). It will force a full browser redirect vs displaying the challenge window within the drop in component.

To Reproduce
Steps to reproduce the behavior:

Ensure you have the test suit of cards from google to have access to the 3ds card
Load a session in the drop in  and enter in the card 4917 6100 0000 0000 - notice that the challenge window is displayed inline
Load a new session and this time select google pay and the 3ds card. This will force the browser to do a full redirect rather than display inline. (We have attempted to use authorization data native 3ds on the initial session but its does not work)

Expected behavior
Challenge behavior for both should be handled within the iframe that is loaded
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: Mac
Browser: Chrome
Version 124.0.6367.201

Additional context
We are attempting to handle all 3ds checks within the drop in component and prevent the full page redirect. We have tried the docs on providing native3ds as preffered but the instant paymentMethod (e.g. google) seem to always try the full page redirect

@terrymccartan-toast - I'm referring this question to our Authentication team.
I'll let you know when I know more.
In the meantime can you tell us what version of Dropin/Components you are using? And against which version of the CheckoutAPI?

Hi @sponglord  - its checkout api version 70 and the drop in component 5.53.3

Thanks, and can you just confirm that you are setting this: nativeThreeDS: 'prefered? re.  https://docs.adyen.com/api-explorer/Checkout/70/post/payments#request-authenticationData-threeDSRequestData-nativeThreeDS

Yep confirmed we are setting it on the /sessions request and it seems to have no affect when paying with the google pay instant payment button. The full redirect still occurs.
Our question is whether the redirect is valid or should the challenge window be displayed within the drop in component
Thanks!

@terrymccartan-toast - do you have a pspReference?

Sure thing. We have a couple
HG6H5C9XXQDG96F6
HDVW438TTVBVDND3
HDVW438TTVBVDND3
P4VTD49RXNXLV3X3
All of these ended up in a full page redirect - where we have seen with cards the challenge window is displayed inline

@terrymccartan-toast - thanks for the refs
To answer your question the redirect is valid - it's a legitimate flow.
I'm just trying to establish (from out backend & Authentication teams) if there is anything you can do, via config props, to influence this flow for Instant payment methods, and bring it inline

@terrymccartan-toast - turns out it was an issue on our side.
The GooglePay component wasn't passing on a property necessary for our backend to decide whether to route the 3DS2 through a native or redirect flow.
This has been fixed in #2690 and will be available in our next release

Fantastic - thanks @sponglord for seeing this through


ERROR when benchmarking TFLite model "ERROR: Given shapes, [1, 40, 23, 256] and [1, 23, 40, 256], are not broadcastable."

System information

Have I written custom code: Yes
OS Platform and Distribution: Ubuntu 20.04/Android 10
Mobile device: Xiaomi Redmi Note 7
TensorFlow installed from: pip
TensorFlow version: v2.4.0-49-g85c8b2a817f 2.4.1
Python version: Python 3.8.8

Describe the current behavior
I am training a model and converting it to TFLite but when benchmarking it on TFLite I got an unexpected error shown below.
ERROR: Given shapes, [1, 40, 23, 256] and [1, 23, 40, 256], are not broadcastable.
ERROR: Node number 80 (ADD) failed to prepare.

Describe the expected behavior
Since the training works and that the shapes are good when I visualize the model in netron, I expect it to work.

Standalone code to reproduce the issue
I am running the benchmark using the native tool which includes TFOps and Flex Delegate that can be downloaded here.
My model can be downloaded here.
I run the benchmark on an android device using the following lines:
adb push android_arm_benchmark_model_plus_flex /data/local/tmp/benchmark
adb shell chmod +x /data/local/tmp/benchmark
adb push model.tflite /data/local/tmp/model.tflite
adb shell "/data/local/tmp/benchmark"  --graph="/data/local/tmp/model.tflite" --use_gpu=true --input_layer=input,scale_input --input_layer_shape=1,640,360,3:1,4

I do not understand where the error could be coming from since can train and run inference on the Tensorflow model and that the graph in Netron is good.
Thanks in advance!

@yann-pourcenoux could you try TF -> TFLite model conversion with the tf-nightly version to see the above issue already fixed? If possible, please share a minimal reproducible step for the TFLite conversion.

Thanks for your quick reply @abattery, I just tried with tf-nightly 2.6.0-dev20210516 and the error is still the same. I convert my model by doing:
mobile_model = create_model(...) # I'd rather not share this
converter = tf.lite.TFLiteConverter.from_keras_model(mobile_model)

# Set the optimizations of quantization
converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_SIZE]

# Experimental environment
converter.experimental_new_converter = True
converter.experimental_new_quantizer = True
converter.target_spec.supported_ops = [
    tf.lite.OpsSet.TFLITE_BUILTINS,  # enable TensorFlow Lite ops.
    tf.lite.OpsSet.SELECT_TF_OPS,  # enable TensorFlow ops.
]

# Converting model
tflite_model = converter.convert()
with open(tflite_filepath, "wb") as f:
    f.write(tflite_model)

Sorry. It is really hard to reproduce your issue at our end. Please consider creating the simple and minimal reproducible steps as a gist and share it to us if possible.

While creating a script to reproduce the issue, I realized I swapped the axis when specifying the input_shape for the benchmark. The error was then totally on my side.
Would adding something to ensure that the model is fed the expected shape be possible to prevent such mistakes?
Anyway, thanks again for your quick reply!

Thanks for the confirmation!
@multiverse-tf could you take a look at https://github.com/tensorflow/tensorflow/issues/49216#issuecomment-842566632?

While creating a script to reproduce the issue, I realized I swapped the axis when specifying the input_shape for the benchmark. The error was then totally on my side.
Would adding something to ensure that the model is fed the expected shape be possible to prevent such mistakes?
Good suggestion! Will add more input-shape-related checks asap.

Anyway, thanks again @abattery for your quick reply!


Building scikit-learn-intelex with openblas/lapack math library with openmp

Hi,
Is it possible to build scikit-learn-intelex with openblas or lapack in place of onemkl and openmp in place of onetbb? I could not find any config file for building the library with different math and threading library.

oneMKL and oneTBB are only supported math and threading libraries by underlying ML library (oneDAL).

I would say it's not feasible to replace threading as far TBB and openmp use different approach and code in oneDAL assumes external threading capability such as oneTBB
But for MKL functionality - this is more realistic
Here is functions export from FPK lib - https://github.com/oneapi-src/oneDAL/blob/master/cpp/daal/src/threading/export.def
They can be mapped to non MKL implementations

Hi @Alexsandruss @napetrov,

After installing scikit-learn-intelex, numpy, if i check numpy.show_config() i find its config to be build using openblas. So how MKL is actually linked in intelex? Intelex does not use numpy, scipy on top of MKL? or it uses them in different way?
I tired approaches suggested here (https://github.com/IntelPython/composability_bench) where it gives a way to use TBB and mkl based numpy to run scikit learn algorithms. I did not observe any performance gain in execution time on scikit-learn's kmeans algorithm with tbb monkey patching and intelnumpy(which uses MKL) over normal kmeans algorithm.
I have installed intelnumpy, tbb, smp, numba. I have following doubts:
3. do TBB monkey patching mimics 100% of oneTBB which is used in onedal(scikit-learn-intelex)? TBB monkey patching is
proposed in this paper ( https://conference.scipy.org/proceedings/scipy2016/pdfs/anton_malakhov.pdf)
4. Based on results can we conclude that oneMKL and oneTBB does not contribute at all for performance gain for kmeans
which are major building blocks in scikit-learn-intelex?
As pointed by @napetrov
mapping to non MKL implementations means, i understand that we need to use openblas or lapack in place of it which means we replace full MKL with openblas or lapack. Am i correct? Also what is full form of fpk?

@napetrov thanks for the reply.

So i understand that numpy and scipy dont use MKL when intelex scikit-learn(oneDAL) is used, but oneDAL's algorithms directly use MKL calls. is this correct? Also what is fpk library(full form of fpk)?
You point out that there is no difference in openmp and tbb threading. Is tbb in the paper and oneTBB of onedal are same? if same then what are benefits of oneTBB over openmp? if there is no benefit why openmp is not used in place of oneTBB?
Paper refers as intel TBB which i assume it as oneTBB only, my doubt is that intel TBB which is being referred in paper is same as oneTBB in onedal or different?
From your answer, i understand that mkl based numpy and TBB monkey patching is not correct way to use mkl and TBB on stock scikit. Intelex oneDAL does this entirely different without linking MKL to numpy.
Ok. Can i know the potential files or areas which needs to be changed to replace all MKL functions with normal BLAS/LAPACK? I wanted to build oneDAL from source on ARM architecture by replacing oneMKL functions as oneMKL wont work on arm.
Also can u help me in what extra code changes that needs to be done to compile and build oneDAL on ARM?

Hi @napetrov ,
4. you are saying openmp being more performant than oneTBB? Can we expect any performance gain if use BLAS/LAPACK + oneTBB combination ? can u suggest some resources if possible to know gains and shortcommings of using openmp or oneTBB? Also resources to know the entire ecosystem of oneTBB.
5.
b.) I did profiling (gprof) on intelex's random forest, and i found its using fpk_vsl_kernel_z0_iRngUniform and  fpk_vsl_sub_kernel_z0_dBRngMT19937. Though it dont use any GEMM functions, since it uses "random number generator" function from mkl fpk, so how to compile this on ARM? Since mkl fpk "random number generator" is not available on ARM? Any alternative to this function we can use to replace mkl fpk RNG ?
Also can i know how to compile only random forest(intelex's) on ARM?

Also in stock sklearn, joblib is used as parallel backend which calls openMP. Does oneTBB replaces joblib+openmp in intelex sklearn?
Observation: when i used intelnumpy (numpy+mkl), profiling (gprof) output of kmeans shows it uses mkl_blas_avx512_dgemm_kernel_nocopy_TN_b0 which is actually different one which oneDAL uses which is fpk_blas_avx512_xdgemm which is fpk version of mkl. So this difference of using mkl's dgemm and fpk's dgemm  would effect the performance gain? Do fpk's dgemm as more optimizations leading to more performance gain?

It would be helpful if you could connect folks who are able to port parts to ARM here.  Thanks.

@vineel96
4. https://community.intel.com/t5/Intel-oneAPI-Threading-Building/TBB-vs-OpenMP/td-p/1103173  In general there are other articles there. but key point - those are different programming models. openMP can deliver higher top perf but at cost of productivity. But again we are talking about < 10% geomean i guess. As an end user you shouldn't bother much about backend threading solution being used.   https://github.com/oneapi-src/oneTBB   TBB is quite popular threading approach and there are millions of monthly downloads.

b) Yes, there would be some use of FPKs libraries.  Here is sort of idea of what you have to replace.
BLAS/LAPACK -> openBLAS
RNGs -> STD lib generators
VML/VSL -> there are open implementations available

So for RNGs you have to use alternative implementation.  Basically as you see this is standard MT19937 which have both generic and ARM optimized implementations.

joblib sits at higher level and would be calling multiple estimators in parallel. scikit-learn-intelex would do threading on it's onw based on oneDAL without involving joblib. So there is no replacement.

In general there are same level of optimizations between MKL and FPK. At the end for this case FPK just have different prefix to avoid conflicts. then the difference is that oneDAL have different algorithm implementation and likely would be calling different MKL function with different params.

Yes, would connect once they would be ready.  Should be some update in upcoming weeks if they would be ready to participate.  But bottom line with set of changes on FPK functions replacement and build targets changes they get things running on non Intel platform.

@napetrov Thanks for the detailed explanations.
How to compile and build only intelex's random forest on ARM? If we replace mkl RNG to with open source RNG, which file needs to be changed, as RNG used in intelex is from libonedal_core.so file? Like can i know the detailed steps and files needs to be changed for building intelex's random forest on ARM as a initial start or atleast where to start changing things?

I would suggest to first test steps on x86
Here are steps for build - https://github.com/oneapi-src/oneDAL/blob/master/INSTALL.md#installation-steps
make -f makefile daal PLAT=lnx32e CORE.ALGORITHMS.CUSTOM="dtrees/forest" REQCPU=avx2 -j16
in terms of RNGs -
https://github.com/oneapi-src/oneDAL/tree/d21ed3261dc603788a54c07bb943de4ecf9346a1/cpp/daal/src/algorithms/engines/mt19937

@napetrov You've mentioned that few folks were able to compile few parts on arm, any idea what did they replace the functions in /oneDAL/cpp/daal/src/externals/service_service.h with?

@KulikovNikita  can comment here

@rakgb-fujitsu Yes, we have an opened PR to support various backends in the oneDAL repository. It can be adjusted to support ARM CPUs.
The steps needed to be able to run it:

Replicate dispatching done in the cpp/daal/src/algorithms/kernel_inst_x86.h file with the corresponding ARM CPU families
Adjust makefiles to support linking with an ARM specific compilation options
Modify build scripts in scikit-learn-intelex

What have been done in this PR:

Introduced dispatching by backend
Implemented reference algorithms for some of the primitives
Implemented a draft implementation of infrastructure to build the library with such backend

Known issues:

There will be no bit-to-bit compatibility between results computed on x86 and other backend. It is due to a different RNG backends
Be aware about used OpenBLAS library. It can have issues supporting high number of cores

Opened questions:

CI support
Build distribution

@KulikovNikita this looks good, thanks for sharing this PR. I have a couple of questions:

I can see that the backend can choose between "mkl" or "ref", here the "ref" is openblas or something else?
Any changes to the build command in order to build this PR? I want to first try this out on x86 with "ref" as backend and openblas support.
Since this backend functionality is added, do we still need to handle replacements for RNG, DFT, VSL, SPBLAS calls?

Indeed. "Ref" backend options correspond to OpenBLAS calls. I also believe that you need to set up environment variables with path to OpenBLAS installation, etc.
Simple versions of these functions/wrappers are already in the library. Feel free to optimize them further with an appropriate substitutions (optimized x-platform libraries).

Yes, I was able to build it with openblas backend. However the regular fpkmkl backend is breaking, I've commented on the PR regarding the same.

Hi @KulikovNikita
Can you please help with explanation of below code in Makefile -
CPUs.files := nrh neh hsw skx Makefile:Ln53
USECPUS.files := $(subst sse2,nrh,$(subst ssse3,,$(subst sse42,neh,$(subst avx,,$(subst avx2,hsw,$(subst avx512,skx,$(subst avx512_mic,,$(USECPUS))))))))Ln132
https://github.com/oneapi-src/oneDAL/blob/1143b38864ee82e0d80ffcb05d9009272726e3b3/makefile#L136C1-L140C71
https://github.com/oneapi-src/oneDAL/blob/1143b38864ee82e0d80ffcb05d9009272726e3b3/makefile#L514C1-L520C69`

Hi @KulikovNikita Can you please help with explanation of below code in Makefile - CPUs.files := nrh neh hsw skx Makefile:Ln53 USECPUS.files := $(subst sse2,nrh,$(subst ssse3,,$(subst sse42,neh,$(subst avx,,$(subst avx2,hsw,$(subst avx512,skx,$(subst avx512_mic,,$(USECPUS))))))))[Ln132]

According to the GNU Make official documentation:

$(subst from,to,text)
Performs a textual replacement on the text text: each occurrence of from is replaced by to. The result is substituted for the function call.

So it temporarily renames files substituting the most advanced allowed vector extension (sse2, ssse4, avx2, avx512) to the proprietary CPU generation naming (nrh, neh, hsw, skx). As far as I understand it was done to be able to target very specific generation.
This step should affect both automatically populated files as well as manually created ones.


MySQL 8 package

Can we have a separate feedstock for MySQL 8?

why?

@beckermr there doesn't seem to be any mysql package available for conda.

there is mysql 5 here
we can upgrade this feedstock to 8
prs welcome!

Ohh okay, I thought since mysql 5.7 is still in use, this was intentional. Will create a PR

we can make another branch for 5.7 if people want updates

coordinate with @nehaljwani as he had some great ideas for making this build better

sure, @beckermr. Thanks for the pointer.

So, building is not that hard. Splitting it is.
On Fedora, we have:
community-mysql | MySQL client programs and shared libraries
community-mysql-common | The shared files required for MySQL server and client
community-mysql-devel | Files for development of MySQL applications
community-mysql-errmsg | The error messages files required by MySQL server
community-mysql-libs | The shared libraries required for MySQL clients
community-mysql-server | The MySQL server and related files
community-mysql-test | The test suite distributed with MySQL

My intent is to split the package like this:
mysql-plugins -> all plugins
mysql-libs -> all shared libraries
mysql-common -> shared files b/w server and client
mysql-devel -> header files and dependency on mysql-libs
mysql -> server binaries with a dependency on mysql-plugins and mysql-libs and mysql-common

Then mysql-connector-python and other mysql (C/C++/python/whatnot) clients can then start depending on mysql-devel (during build) which will have a run_exports on myql-libs. If these want plugins, they can optionally depend on that too.
Refer: https://src.fedoraproject.org/rpms/community-mysql/blob/f32/f/community-mysql.spec for the contents for splitting


Buildtrees folder and Winpcap pdb file

I have removed the content of Buildtrees folder. Previously, I installed through vcpkg the following package in this way:
vcpkg install pcl[pcap,tools,cuda]:x64-windows --featurepackages
because I was running out of space in C: as buildtrees, downloads and packages were taking 25 GB, I decided to clean the content of those folders.
I would like now to debug some code and it is needed the Winpcap pdb that was supposed to be located in the Buildtrees folder.
In order to construct again the respective folder in Buildtrees I ran:
vcpkg install pcl[pcap]:x64-windows --featurepackages
And what I was expecting did not happen.
Do you know how to build again the respective folder for winpcap in   Buildtrees?

Ok.... You can install separately winpcap through vcpkg. I thought there was a special modification between PCL-winpcap and winpcap.


Roblox's new SharedTable is an undefined global.

Roblox added a new global SharedTable but it wasn't added as a global to roblox LSP yet.

Please Merge This Request 🙏

please dont forget the new Vector methods

This is unrelated to this issue.

Fixed


WIP Improved Install and Linux support

Please test this branch by installing as follows:
curl -fsSL https://raw.github.com/zachcurry/emacs-anywhere/release_1.1/install | bash

What's new

Fixes 18
Fixes 25
Fixes 35

The install implementation has been modified to more closely resemble that of cknadler/vim-anywhere. This fixed the issue where the scripts fail when run from outside the root of the repo.
The install script also checks for dependencies now, which I hope will give some users instant feedback as opposed to having to open an issue here and wait for an answer.
Also this branch should fix most of the "Not working in my Linux" issues.
Soon I'll add the Emacs command to abort Emacs Anywhere mid-session.
I will be force pushing squashed commits to this branch until I'm finished, so stay tuned!
Let's try to get as many people testing and sanity checking this code as possible!

This still doesn't work for me on Linux. I think a better way to solve the problems for Issue 18 is PR#23  by @psibi. This branch still uses the idea that Ctrl-V is a universal way to paste, but even when I use an application that has Ctrl-V as its paste binding I don't get the buffer contents, but instead whatever was on the clipboard before. I agree with @psibi that clipboards in Linux are somewhat tricky and not as idiomatic as using files.

@DavidLarsson Do you have xclip installed? If so, can you verify that it is capable of putting content in your clipboard? If so, can you verify that Emacs is integrated with the system clipboard.
Bypassing the clipboard could be a configuration option in the future, but it will need to be implemented for both osx and  linux, otherwise it's not getting merged. Also, I worry that having xdotool attempt to type arbitrary text of arbitrary length could be it's own source of problems, especially if it needs to add a delay to work with large pieces of text. Any PR that introduces an element of time/timing will not be merged.

@zachcurry Going via clipboard path is messy in Linux so I'm not supportive of this PR.

@psibi Thanks for your feedback! I'm open to adding an option to bypass the clipboard but, I'm not going to remove clipboards from the repository! Clipboards are a part of this repository. Period.

@zachcurry If you can give some emacs option to configure it from init.el file, that would solve the issue.

@psibi I was thinking of adding a bash config file to the repo instead. That way we don't require the user to edit their init.el.  That environment variables are accessible in elisp.

This didn't work because of a simple typo 😆 Not the most productive code review i've participated in...


bug: "regression" in 1.1.0 regarding "Multiple URLs found for"

[!WARNING]
This report might look like:

After writing all of this, this probably should have been a feature request instead...
TL;DR: Could an option be added to disable the "Multiple URLs found for" warning and just let references to silently resolve locally as before and/or have an option to resolve links with multiple URLs found to a specific page?
Description of the bug
After doing an update from 1.0.1 to 1.1.0 I started to get many warnings in the form:

Multiple URLs found for 'x.y.z': ['reference/x/y/z/#x.y.z', 'user-guide/x/y/z#x.y.z']. Make sure to use unique headings, identifiers, or Markdown anchors (see our docs).

And since we are using strict mode, this means an error and docs that won't build.
To Reproduce
The thing is we are using this pattern to try to have as much documentation as possible as part of docstrings, but still have a nice and more organized "user guide": we use some internal modules docstrings as the user guide section and then include it in the markdown documentation. For example:
$ cat docs/user-guide/formula-engine.md
# Formula Engine

::: frequenz.sdk.timeseries.formula_engine.FormulaEngine
    options:
        members: None
        show_bases: false
        show_root_full_path: false
        show_source: false

::: frequenz.sdk.timeseries.formula_engine.FormulaEngine3Phase
    options:
        members: None
        show_bases: false
        show_root_full_path: false
        show_source: false

And we also generate API reference documentation using some script based on what's suggested in the docs, which will create a file like:
$ cat docs/reference/frequenz/sdk/timeseries/formula_engine/index.md
::: frequenz.sdk.timeseries.formula_engine

Hence, we have the same anchor in 2 different pages.
Expected behavior
Before this upgrade all worked as more or less fine, links just being resolved locally when possible, and externally when not.
But I think things could be even improved for this use case, as before in the example above FormulaEngine in the user guide will resolve to the user guide, but ideally it should resolve to the API reference.

Add an option to ignore multiple URLs for warnings.
a. Globally in mkdocs.yaml
b. Locally in the `:: x.y.z`` block
Example:
# Formula Engine

::: frequenz.sdk.timeseries.formula_engine.FormulaEngine
    options:
        members: None
        show_bases: false
        show_root_full_path: false
        show_source: false
        disable_multiple_urls_warning: true

A way to tell where to resolve multiple references to a specific page. For example:
For example something like:
# Formula Engine

::: frequenz.sdk.timeseries.formula_engine.FormulaEngine
    options:
        members: None
        show_bases: false
        show_root_full_path: false
        show_source: false
        resolve_multiple_urls:
            "frequenz.sdk.timeseries.formula_engine.FormulaEngine": reference/frequenz/sdk/timeseries/formula_engine.md

Additional context

Failing CI job trying to generate the docs: https://github.com/frequenz-floss/frequenz-sdk-python/pull/1043
Link to the working documentation generated previous to the update: https://frequenz-floss.github.io/frequenz-sdk-python/v1.0-pre/
Script generating the reference docs: https://github.com/frequenz-floss/frequenz-repo-config-python/blob/v0.x.x/src/frequenz/repo/config/mkdocs/api_pages.py#L43

Thanks for the report! I was expecting to get one but still thought this warning had to be enabled by default. I definitely didn't anticipate this use-case you have though.

But I think things could be even improved for this use case, as before in the example above FormulaEngine in the user guide will resolve to the user guide, but ideally it should resolve to the API reference.

That's exactly the issue the warning is aiming to solve. With non-unique heading ids across pages, cross-references might link you to one or the other, non-deterministically (or deterministically, but depending on the order the pages are built, and this might be more complex than that).
I agree that this could be a feature request indeed: the feature I think we need here is a way to change the heading ids of the ones if your Guide pages. By the way, I can't seem to find any mkdocstrings-generated headings in your Guide pages, could you link me to some examples? I'd like to see how you render them.
It might be possible to workaround this right now, by doing the following:

manually write the headings in your guide pages: ### MyClass

but change their id: ### MyClass {#guide-MyClass} or ### MyClass {#guide-complete.path.to.MyClass}

render the object by omitting its heading:
::: complete.path.to.MyClass
    options:
        show_root_heading: false

By the way, I can't seem to find any mkdocstrings-generated headings in your Guide pages

Sorry, I don't know what this means. :disappointed:, so I don't know how to point you to examples. Not sure if you still need them given the EDITs.

It might be possible to workaround this right now, by doing the following:
...
Maybe not afternall. Will need to think about it.

Yeah, no, sadly it doesn't.

@llucax I'd like to inspect a bit more your docs sources, is there a particular branch I should look at? One that triggers the warnings.

The default branch should have the issue. We just pinned the dependency to avoid CI failures until we can find a solution, but if you manually update the dependency it should break.
All non-generated docs are in docs/. The doc you are looking for is here: https://github.com/frequenz-floss/frequenz-sdk-python/blob/v1.x.x/docs/user-guide/formula-engine.md.
https://github.com/frequenz-floss/frequenz-sdk-python/blob/v1.x.x/docs/user-guide/microgrid-concepts.md also have the issue.
Thanks a lot for looking at it! ❤

Thanks, I wasn't looking into the right repo.
So, by adding the following options:
        show_root_heading: false
        show_root_toc_entry: false

(which you already have in actors and microgrid-concepts)
...that reduces the number of warnings. The four remaining ones come from subheadings in the corresponding objects' docstrings (### The run() Function): these subheadings are rendered and their id is prefixed with the object path, whether we've rendered the root heading or not, so whether we're in the guide or the reference. In the end: duplicated ids.
A few solutions that come to mind:

(in mkdocstrings) detect that the root heading (and toc entry) are not rendered, and therefore avoid prefixing the subheadings. But the issue would pop its head again with more than two places where we render the docstring.
(in mkdocstrings) add an option to configure the prefixing behavior for subheadings: no prefix, default behavior (prefix with object path), custom prefix. We do something similar in Markdown Exec :+1:

Obviously, we could also consider reverting, and putting these warnings behind an option. But that means a lot of maintenance, to maintain incorrect/un-deterministic behavior a few projects rely on. If many projects are impacted by this and come here to add their voice, we'll do something, but for now it seems only two projects are impacted.
WDYT? Would an option to configure (sub)headings prefixes work for you? I imagine something like this:
# Actors

::: frequenz.sdk.actor
    options:
        members: []
        headings_prefix: ""
        show_bases: false
        show_root_heading: false
        show_root_toc_entry: false
        show_source: false

There's an alternative solution by the way. You could use pymdownx.snippets instead of mkdocstrings to inject your docstrings into your "Guide" Markdown pages. See https://facelessuser.github.io/pymdown-extensions/extensions/snippets/#snippet-sections. Snippets lets you declare sections in your sources, whathever the language (and comment syntax). Then you can inject just these sections into pages.
In src/frequenz/sdk/actor/__init__.py:
# License: MIT
# Copyright © 2022 Frequenz Energy-as-a-Service GmbH

# Please note this docstring is included in the getting started documentation in docs/.
"""Actors are a primitive unit of computation that runs autonomously.

<!-- --8<-- [start:docstring] -->

## Actor Programming Model
...

<!-- --8<-- [end:docstring] -->
"""

I'm using HTML comment syntax since we're in a Markdown docstring. Note that the docstring summary is not included in the snippet section because it would be weird. So here you'd have to repeat just this summary in the Markdown page:
# Actors

Actors are a primitive unit of computation that runs autonomously.

--8<-- "src/frequenz/sdk/actors/__init__.py:docstring"

Well, it's not perfect, just wanted to list this alternative :slightly_smiling_face:

Hi, thanks again for having a deep look at this. I also agree that just disabling the warning is not ideal, as there is a potential issue, it would be much better to have a way to disambiguate something ambiguous instead.
About the snippets solution, I prefer to avoid it, another goal of us is to keep docstrings as readable as possible, same reason we try to keep most docs in docstrings, they are accessible via the IDE and code editors, which is in our experience the main way our users look at docs.
So your proposed solution of headings_prefix means that these headings will get a different prefix depending on where they were "included", so for example we could leave no prefix for the reference docs and have a headings_prefix: "user-guide" in the user guide for example? I guess that could work.
So we are currently linking to that "The run() method" using:
[_run]: #the-_run-method

In this case, if we leave the link as is it will point to the reference, and if we change it to user-guide-the-_run_method it will point to the user guide, right?
I guess we can live with that, but ideally the link should resolve "locally" when possible, it would be best if it resolved to the user guide when rendered in the user guide and to the reference when rendered in the reference.
This is why I was thinking if it would be possible to have some sort of "resolve multiple URL using this page" kind of option I suggested before. So in this case mkdocstrings will find 2 locations for #the-_run-method, but we'll have something like this:
::: frequenz.sdk.timeseries.formula_engine.FormulaEngine
    options:
        members: None
        show_bases: false
        show_root_full_path: false
        show_source: false
        resolve_multiple_urls:
            "#the-_run-method": reference/frequenz/sdk/actor.md

So it will use the link to that page without giving any warnings.
I can imagine it might be a bit of work and maybe it is not justified if we one of a kind using this documentation scheme, but it would be good to know if that would be even doable :)

About the snippets solution, I prefer to avoid it, another goal of us is to keep docstrings as readable as possible

Completely agree :+1:

so for example we could leave no prefix for the reference docs and have a headings_prefix: "user-guide" in the user guide for example?

That's the idea :slightly_smiling_face:

In this case, if we leave the link as is it will point to the reference, and if we change it to user-guide-the-_run_method it will point to the user guide, right?

Yes!

ideally the link should resolve "locally" when possible, it would be best if it resolved to the user guide when rendered in the user guide and to the reference when rendered in the reference.

Then you shouldn't use autorefs for this. Autorefs will always resolve references to one and only one location, be it on the same page or somewhere else. Instead, you should use the native reference syntax [some title](#some-anchor-on-this-very-page): this will always be local to the page, wherever the docstring is rendered.

[...] I can imagine it might be a bit of work and maybe it is not justified [...]

Such a solution would have to be implemented in autorefs, but would only be supported through mkdocstrings, which would be a bit weird. Autorefs will definitely have more features in the future (and this might be one), but we're not ready for that yet. We need solid foundations for this type of features first :slightly_smiling_face:

Then you shouldn't use autorefs for this. Autorefs will always resolve references to one and only one location, be it on the same page or somewhere else. Instead, you should use the native reference syntax [some title](#some-anchor-on-this-very-page): this will always be local to the page, wherever the docstring is rendered.

I see, so if all the anchors pointing to symbols are fixed by using:
        show_root_heading: false
        show_root_toc_entry: false

(not sure if the later is desirable or not, will need to check it out)
And for regular anchors we can avoid autorefs and it does exactly what we want (resolve locally) without any warnings, I think that's all we need. I'm not sure why we used autorefs there, but I wouldn't surprised if it was just a mistake.
I will try it out.

Actually, even prefixing ids might be complicated to implement 🤔 If the-run-method gets prefixed as user-guide-the-run-method, then what are we supposed to do with [the run() method][the-run-method]? Should we prefix the id there too? But we don't know yet that user-guide-the-run-method exists. And even if it does, how do we know we don't really want to use the-run-method untouched, because it could exist somewhere else later? 🤔 🤔 🤔

If the above works out, then I don't think I need prefixes at all. I'm not sure if we are using deep linking to link into a section of a docstring, but if we do I guess we could live without those.
Will try to give this a try (next week) and report back.
Once again, thanks a lot for your support! :heart:

I'm not sure why we used autorefs there, but I wouldn't surprised if it was just a mistake.

Possibly for consistency. For example, the Actor class also uses an autorefs link to the-_run-method IIRC.
It's possible that there are places where you cannot use a "local reference" (direct link using # instead of autorefs), and using an autoref will emit warnings.

If the above works out, then I don't think I need prefixes at all.

Yes please, don't hesitate to share your progress with me, I'll gladly chime in again and try to help :smile: I'll live this open in the meantime.

OK, I'm having a look at this and now I know why we used the [the run() method][_run] format, this was just to have shorter links I think, and to only having to care about slugification in one place. I'm not sure if it is 100% standard markdown, but it seems most parsers accepts [text][ref] as a link and then you can declare the link later like [ref]: https://mylink.com. We are using it like that, so the intention was not to use autorefs for this link, just to make it more concise and avoid duplication.

https://github.com/frequenz-floss/frequenz-sdk-python/blob/091628e326995f91f6741201661b16d467e3dced/src/frequenz/sdk/actor/__init__.py#L39
https://github.com/frequenz-floss/frequenz-sdk-python/blob/091628e326995f91f6741201661b16d467e3dced/src/frequenz/sdk/actor/__init__.py#L161
https://github.com/frequenz-floss/frequenz-sdk-python/blob/091628e326995f91f6741201661b16d467e3dced/src/frequenz/sdk/actor/__init__.py#L247
(etc.)
https://github.com/frequenz-floss/frequenz-sdk-python/blob/091628e326995f91f6741201661b16d467e3dced/src/frequenz/sdk/actor/__init__.py#L596

So in this case the intention here was to explicitly define the link, which happens to point to a HTML fragment/ID instead of a full URL.
Is there any way for autorefs to detect when we are actually explicitly defining where [ref] points to in [text][ref]?
On the other hand, we do are using some autoref links for "deep linking" into some other sections:

https://github.com/frequenz-floss/frequenz-sdk-python/blob/091628e326995f91f6741201661b16d467e3dced/src/frequenz/sdk/timeseries/formula_engine/_formula_engine.py#L67 pointing to https://github.com/frequenz-floss/frequenz-sdk-python/blob/091628e326995f91f6741201661b16d467e3dced/src/frequenz/sdk/microgrid/init.py#L13
https://github.com/frequenz-floss/frequenz-sdk-python/blob/091628e326995f91f6741201661b16d467e3dced/src/frequenz/sdk/actor/_actor.py#L20-L21 pointing to https://github.com/frequenz-floss/frequenz-sdk-python/blob/091628e326995f91f6741201661b16d467e3dced/src/frequenz/sdk/actor/init.py#L212

This doesn't seem to be fixed by just adding:
        show_root_heading: false
        show_root_toc_entry: false

(these options are actually already there)
For this former we could get around with removing the --frequenz-sdk-microgrid-model and just linking to the whole module, that seems to work, but in the actor it would really hurt the cross-linking in the docs.
So it seems all our problems are not solved by the solution above, and some are only partially solved, producing worse quality docs (less precise linking).
I'll await in case you have any new ideas, and otherwise I guess we'll have to link to modules instead of module's sections and just tell the user "go find this section inside that doc yourself" basically, which I guess it kind of goes against the goals of this tool (to make the best docs ever possible :) ).
I created a draft PR with my attempt to solve this:

https://github.com/frequenz-floss/frequenz-sdk-python/pull/1051

I would be awesome if the last 2 commits were not necessary somehow...

Is there any way for autorefs to detect when we are actually explicitly defining where [ref] points to in [text][ref]?
I would be awesome if the last 2 commits 2 commits prior to the last one were not necessary somehow...

Good news, commit b0996a3 shouldn't be necessary. Bad news, I'm not sure why links weren't rendered naturally. Let me explain.
autorefs only tries to fix references that weren't already rendered naturally. All these [`_run()`][_run] should be rendered naturally without autorefs because you define [_run]: #the-_run-method later in the document. When these references do not work through mkdocstrings, it's usually because the docstring uses sections (parameters, note, whatever), which means the docstring is split in different sections which have to be converted independently, which means that [`_run()`][_run] and [_run]: #the-_run-method get separated. But in your case, I don't see any sections: you're using plain markup, so the whole docstring should be rendered in one go, and these links should be rendered, and autorefs should never have to fix them later :thinking: I'll investigate.
 Generally speaking, rendering the same contents in two different places, and the same autorefs linking to this contents twice too, makes it impossible to not get the warnings. Well at least autorefs is not equipped to deal with this. The only way to fix this would be to update the link syntax to be able to provide additional metadata to help decide how to resolve the link (in addition to a way of specifying id prefixes, as previously discussed).

Another option would be to add an option to mkdocstrings to say "don't register any heading generated here against autorefs", similarly to https://github.com/mkdocstrings/mkdocstrings/issues/671 and https://github.com/mkdocstrings/mkdocstrings/issues/672.

I tried again in your repo, and the _run links are correctly rendered. I don't see any change in your PR that would prevent them for working. Maybe you confused warnings somehow?

With or without this commit, I only get three seemingly unrelated warnings (that I cannot trace to their origin, but it's another story).

A lot was written here, let me try to summarize.

Unless a lot of projects show that they are impacted, we won't revert this to put the warnings behind an option as it means a lot of maintenance.
Supporting the use-case where the same headings and autorefs are added more than once to the site seems a lost cause.
For this reason @llucax initial suggestion of resolving links "locally" seems appealing. While it wouldn't work in all cases, we could alleviate warnings with an option to say "resolve to the URL that most closely matches the current page". Example: if we're in page user-guide/actors/ and we have two URLs for identifier the-run-method which are reference/sdk/etc/#actor--the-run-method and user-guide/#actor--the-run-method, use the second one and don't emit a warning. If we're in page /something/else/entirely, none of the two URLs remotely matches, so we emit a warning. Here, "matching" URLs would mean "start with the same thing". We could set this option to true by default (bringing us back closer to previous state, but providing a bit more correct link resolution), and allow users to set it to false for even stricter checks.

WDYT?

Thanks for the summary, I was a bit busy and didn't have time to go through all of it properly.
The last 2 points sound very reasonable to me, and it seems like they should cover all our use-cases. The "local resolution" seem like the most important to me, it should bring us on par to what we had before.
The second one would be a nice add-on, and I think it could actually improve our current documentation, as we could avoid adding some undesirable self-references in the user guide and just always point the users to the reference.
I will update the draft PR to remove the unnecessary commit when I have some time. Thanks for checking that!

Not really relevant, but there's another change coming up that can potentially make CI fail, see our announcement here: https://github.com/mkdocstrings/autorefs/discussions/56. Users who saw their CI fail because of the duplicated headings warnings and who pinned mkdocs-autorefs to avoid latest version might want to wait or test the upcoming version before unpinning/upgrading :slightly_smiling_face: I'll let you know when the change is merged, and again when it is released.

Thanks for the heads up, I think (and hope!) we will not be impacted by that one.
I updated my draft PR and I can confirm that the changes to [run()][_run] where not necessary. :+1:

Do you have any ideas of a time frame for implementing the local references, if you plan to do it eventually at all? Just to know if I should wait for a solution or if I should merge that PR as it and live with the "degraded" documentation for now.

I'm currently focusing on mkdocs-autorefs so these changes and features will be released in the upcoming week(s).

OK, just pushed the "resolve closest" feature commit. The option is false by default, because I don't want to encourage its use, since it can be problematic for object inventories.
plugins:
- autorefs:
    resolve_closest: true

Just waiting to see how CI behaves, and I'll publish a new release :slightly_smiling_face:

I updated the PR and using the new option worked flawlessly! Also no regressions detected with 1.2.0 :tada:
Thanks a lot for taking care of this :heart:

Great! Thanks for your help and patience!


API's search feature doesn't handle Japanese

It whould be nice to be able to search for anime using parts of their real title.
A request like
https://anilist.co/api/anime/search/月刊?access_token=w1x257Q1gWTPCUSxQqXb...
or rather (url encoded)
https://anilist.co/api/anime/search/%E6%9C%88%E5%88%8A?access_token=w1x257Q1gWTPCUSxQqXb...
yields no results.
Whereas
https://anilist.co/api/anime/search/gekkan?access_token=w1x257Q1gWTPCUSxQqXb...
does.

Anime/Manga search now supports Japanese characters.

awesome! :3


Dropdown multiselect Select all option and DateRange month/year selectors

Description
Fixes #1776, #1777
Select all option:

Old DateRange:

New DateRange:

Checklist

[x] For UI or styling changes, I have added a screenshot or gif showing before & after
[x] I have added a changeset

Date Range:

The button treatment on the two dropdowns is too heavy w/ the shadows in an existing popover - try the "ghost" variant?
You get visual shift in the calendar popover when you toggle to months of different lengths.
Font size too big relative to the next / pervious icons

Select all

This is great. Instead of removing the "clear selection" button when a selection is made, can we just disable it? It's jumpy to have it disappear.


Enhancing the "export of services" by choosing to which group of peers the service will be exported

SHOULD WRITE DOCUMENTATION


IOS browser position: fixed causes elements in the swiper to blink

Check that this is really a bug

[X] I confirm

Reproduction link
https://github.com/rick-liruixin/body-scroll-lock-upgrade/tree/swiper/examples/react
Bug description
IOS browser position: fixed causes elements in the swiper to blink
When I use swiper, I use the css property position: fixed. My swiper is used in the page flow, but my page has a popup/dialog, when the popup/dialog pops up, I will disable page scrolling. I use position: fixed attribute, but I found position: fixed will cause swiper elements flashing, after investigation, there is no problem in the elements outside swiper, this situation occurs at the bottom of the page, after entering the page to scroll down a part of the distance to have such a situation, just enter the page can see the large elements will not have blinking problems. React components are not re-rendered.
Recurring website: https://www.alexandermcqueen.cn/categories/women/bags.html?page=2
Recurrence condition:

Open the link in any browser of IOS mobile /IPAD device, such as Safari and google.
2, need to scroll down beyond the screen height distance.
3, click "filter" will appear pop-up window, when opened and closed because of the use of position: fixed will cause swiper flash

The sample code warehouse: https://github.com/rick-liruixin/body-scroll-lock-upgrade/tree/swiper/examples/react
https://github.com/nolimits4web/swiper/assets/50647350/e1889f30-c038-4c70-b232-3d883ab73165
Expected Behavior
When swiper is used, the css property position: fixed, swiper does not blink
Actual Behavior
No response
Swiper version
11.0.6
Platform/Target and Browser Versions
iOS safari 17.1.1
Validations

[X] Follow our Code of Conduct
[X] Read the docs.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.
[X] Make sure this is a Swiper issue and not a framework-specific issue

Would you like to open a PR for this bug?

[ ] I'm willing to open a PR

It is issue with Safari, not with Swiper


Write blog post -- eggs data published on NepalMap

@roshghimire has assigned to Namrata Karki

Published: http://beta.codefornepal.org/2018/07/08/eggs-in-nepal/


User request: add hand grabbing animation as a theme

can you change the cursor animation when you try to move a piece, like there it is in chess.com? I mean the grabbing hand animation

Added as part of the Nicer piece drag theme in 2.3.176


Fix chip-tool logging to work for various interesting data types.

Optional values, nullable types, etc, etc.
Fixes https://github.com/project-chip/connectedhomeip/issues/11214
Fixes https://github.com/project-chip/connectedhomeip/issues/11215
Problem
Command-line chip-tool received value logging can't handle all values: has trouble with structs and lists when they're nested deeply..
Change overview
Use helper functions to implement logging of arbitrary-nesting data structures.
Testing
Manually ran some chip-tool commands and observed the output.

Fast tracking, given this is adding logging for testing.

/rebase

/rebase


Correct docs for Phaser.Boot.Config#audio

This PR

Updates the Documentation

Boot.Config#audio was the wrong type (I think).

This change is not correct.
SoundConfig is used internally and for passing sound objects their configuration.
AudioConfig is used for passing sound manager config to the game and should be defined as a separate type.

There's still a definition for SoundConfig in https://github.com/photonstorm/phaser/blob/v3.15.1/src/sound/index.js#L15.
I think the one I removed was a duplicate, so it should be safe.

This PR is correct, cheers.


"Publish to lbry://xxx" button doesn't pre-populate publish address?

When I click the the "Publish to" button above, I was expecting that the Publish Page would have pre-populated the "new channel" or address field with the said value, since the text above it says "claim this spot".  Otherwise, it's no different than having to manually copy the claim URI (which we can't at the moment) and going to the publish page as normal.

If the intention is not to pre-populate the field in the first place, perhaps we can just remove/replace the button to become a short link.  This should indirectly solve the space problem in #4434 [Mobile] Reflow issues for abandoned channels in Blocked Page as well.

It should definitely pre-populate. Maybe we should cut it off after 20 characters or something to help with the overflow issues.

This is no longer valid after #4433 Option to remove abandoned claims from Blocked Channels page.  The component in the screenshot above is not supposed to appear in the blocked list, which probably explains why it didn't populate the URL field.  In the normal cases (outside of Blocked Page), that button does populate the URL field.


[connectivity_plus] Opening the application causes onConnectivityChanged to emit two events

System info
Platform the Issue occurs on: iOS
Plugin name: connectivity_plus
Plugin version: 2.2.1
Steps to Reproduce

Demo code increments counter when connection is changed to Wifi.
Link to gist

Run the gist code on a simulator or real device

When the app is opened the onConnectivityChanged  emits two events instead of just one. On Android it correctly emits just one event. I'm guessing this can be a bug on the iOS.
The video attached is the release build of the demo gist on a iPhone 8
Since, there are 2 events emitted the counter increments 2 times if WiFi is turned on and 2 is displayed when the app is opened. There are 2 events emitted even if Wifi and Data is turned off.
Expected: Only one event should be emitted.
https://user-images.githubusercontent.com/98470469/158479544-e5a1fa9a-45b5-47db-8bf0-578ad3ab67fd.mp4

Running in debug mode on iOS Simulator.
"Got a new connectivity status!  ::wifi" is printed twice.
Logs

Launching lib/main.dart on iPhone SE (3rd generation) in debug mode...
Running Xcode build...
 └─Compiling, linking and signing...                         3.5s
Xcode build done.                                            9.6s
flutter: initState
flutter: Got a new connectivity status!  ::wifi
flutter: Got a new connectivity status!  ::wifi
Syncing files to device iPhone SE (3rd generation)...              214ms

Flutter run key commands.
r Hot reload. 🔥🔥🔥
R Hot restart.
h List all available interactive commands.
d Detach (terminate "flutter run" but leave application running).
c Clear the screen
q Quit (terminate the application on the device).

💪 Running with sound null safety 💪

Flutter doctor output

✓] Flutter (Channel stable, 2.10.3, on macOS 12.2.1 21D62 darwin-x64, locale en-CA)
    • Flutter version 2.10.3 at /Users/calvin.gonsalves/Desktop/development/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 7e9793dee1 (13 days ago), 2022-03-02 11:23:12 -0600
    • Engine revision bd539267b4
    • Dart version 2.16.1
    • DevTools version 2.9.2

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/calvin.gonsalves/Library/Android/sdk
    • Platform android-31, build-tools 31.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7590822)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7590822)

[✓] VS Code (version 1.65.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.37.20220301

[✓] Connected device (1 available)
    • Chrome (web) • chrome • web-javascript • Google Chrome 99.0.4844.51

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

I stumbled on this issue by chance and tested it with https://github.com/fluttercommunity/plus_plugins/pull/791
Same result, first event is ConnectivityResult.none and second is ConnectivityResult.wifi.

closing, if the issue reappears please say so here and I will reopen the issue

I have tested the recently published version v2.3.0 and I also tested PR 791 and I'm still facing the same issue.
The commit I specifically tested from PR 791 was:
connectivity_plus: #^2.3.0
    git:
      url: https://github.com/kuhnroyal/plus_plugins
      path: packages/connectivity_plus/connectivity_plus
      ref: d7b83c425f63a049d5195df99848c33a9eb1e894

@luis901101 On which iOS version did you test?
The line which can potentially cause double emit is still there: https://github.com/fluttercommunity/plus_plugins/blob/main/packages/connectivity_plus/connectivity_plus/ios/Classes/SwiftConnectivityPlusPlugin.swift#L67
I remember this also being a timing issue. In my test app on iOS 15 I only had one event.
Removing the line cause sometimes no initial event to be fired.
Will try to testing this again.

Also this may play a role here: https://github.com/fluttercommunity/plus_plugins/pull/791#issuecomment-1069385002

Tested in iOS 15.3.1 and 15.4.1

Reopened. Just to be sure, is this a big issue if you receive two events? How is that affecting your code? (just want to understand better the scope of this problem)

@838 Which iOS version and did the same happen before the 2.3.0 release?

@miquelbeltran  In my case, I'm checking the connectivity state on every AppLifecycleState.resumed and for a very short time the app will show a offline page if I don't skip the first emit.
@kuhnroyal The iOS version is 15.4.1. I have tested on 2.2.2 and 2.2.1 I have the same issue, but if I'm connected to WIFI then onConnectivityChanged emits ConnectivityResult.wifi twice. In 2.3.0 the first emit is always ConnectivityResult.none.

@838 This is likely due to https://github.com/fluttercommunity/plus_plugins/pull/791#issuecomment-1069385002
My suggestion in that comment would probably fix this.
What I do to circumvent this is an await checkConnection() and start the stream afterwards, discarding the first event.

@kuhnroyal calling await checkConnection() before start the stream doesn't change anything with double emit 😟 for iOS.
Also check on Android and there onConnectivityChanged emits one event as expected.

Just fallback to old ReachabilityConnectivityProvider by commented PathMonitorConnectivityProvider within plugin source code and event emit one time as expected.
Also create stub SwiftUI app with next initial block:
        let monitor = NWPathMonitor()
        monitor.pathUpdateHandler = { path in
            print(path)
        }
        let queue = DispatchQueue.global(qos: .background)
        monitor.start(queue: queue)

and also get double handle calls with identical path objects (at last from its public fields view).
My current iOS version is 15.4.1.
🤷🏻♂

Same issue. [2.3.5]

So it seems the NWPathMonitor .pathUpdaterHandler can be called multiple times with the same path. According to discussions on the apple community board, your app needs to be resilient against multiple invocations.
This can happen on app start or even while the app is running and you lose connectivity.
There may still be ways to reduce the chance of this happening on app start but I don't think it can be prevent completely.

What's a fantastic Apple API. Does this board explain what the reason for these multiple invocations is? Does it indicate something valuable (some interface updated, but I can't see any property changes when debugging it)?
In any case, what do you think about adding multiple invocations handling in connectivity_plus plugin and putting only different ConnectivityResult in onConnectivityChanged?

In any case, what do you think about adding multiple invocations handling in connectivity_plus plugin and putting only different ConnectivityResult in onConnectivityChanged?

I thought about this but that is rather hard to do correct. Like changing from one wifi to another wifi network will result in 2 wifi events.
Maybe it can be done with a very limited timeframe check but it will not completely guard you from those events. You need to handle the meaning of this on your side, especially if you want to do more than just a connectivity check.

Does it indicate something valuable (some interface updated, but I can't see any property changes when debugging it)?

Yes I have noticed the same thing, but the objects are not comparable so that I can for them out. Could probably try to compare it on a field by fiel basis. Maybe someone else knows more about swift ^^

I thought about this but that is rather hard to do correct. Like changing from one wifi to another wifi network will result in 2 wifi events.

Have you checked that the wifi network switch emits two wifi results without none between them?

This is how I fixed this issue using RxDart's debounceTime.
late StreamSubscription connectivitySubs;
PublishSubject<ConnectivityResult> onConnectionChange =
      PublishSubject<ConnectivityResult>();

      onConnectionChange
        .debounceTime(const Duration(
      seconds: 1,
    ))
        .listen((event) {
      log.d('connectionOnChange $event');
      if (event == ConnectivityResult.none) {
        Get.dialog(
          const NoInternetModal(),
          barrierDismissible: false,
        );
      }
    });

    connectivitySubs = Connectivity()
            .onConnectivityChanged
            .listen((ConnectivityResult result) {
          connectionOnChange.add(result);
        });

    // don't forget to dispose connectivitySubs
    connectivitySubs.cancel();

I know it's redundant to use 2 listeners, but it works for me.

Same issue happened to me


country code -> name conversion is extremely slow

i'd recommend using country-list as it's a lot faster (tested both with bpy-web, country-list was noticeably faster)

resolved with 970477d


Optional usages of the sites framework

The django sites framework- django.contrib.sites has been removed from the INSTALLED_APPS by default beginning with django 1.6.
This pull request will make django-feedback work with django.contrib.sites as well as without django.contrib.sites.
Cheers.

Is it necessary to remove site field from the model? I consider it wrong, because it lead to data loss when the django.contrib.sites are removed from INSTALLED_APPS.


Verify DocFX works on Mac

@spboyer  can you verify DocFX builds work on Mac. Currently DocFX is not creating the TOC but is creating all the HTML files.

Getting is setup using Mono now.

Install mono via Homebrew - brew install mono
Download Latest version of DocFX - https://github.com/dotnet/docfx/releases/tag/v2.7.2
Extract to \bin\docfx
created alias for docfx

function docfx {
  mono $HOME/bin/docfx/docfx.exe
}

function docfx-serve {
  mono $HOME/bin/docfx/docfx.exe serve _site
}

run docfx in the Docs\aspnetcore directory to build the site, and docfx-serve to view the site at http://localhost:8080

@Rick-Anderson - do you want to add this information to the README?

@tdykstra  can you add these instructions to the appropriate guide?

Will do
Get Outlook for iOShttps://aka.ms/o0ukef

From: Rick Anderson<EMAIL_ADDRESS>Sent: Friday, November 4, 2016 12:18:42 PM
To: aspnet/Docs
Cc: Tom Dykstra; Mention
Subject: Re: [aspnet/Docs] Verify DocFX works on Mac (#2104)
@tdykstrahttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftdykstra&data=02|01|tdykstra%40microsoft.com|97be81be33f24ce481d308d404e764f3|72f988bf86f141af91ab2d7cd011db47|1|0|636138839252822699&sdata=jcEZ5w9h5oAdFINzO9mOFVPsuAZoHW6rRNc67oTcJV4%3D&reserved=0 can you add these instructions to the appropriate guide?

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Faspnet%2FDocs%2Fissues%2F2104%23issuecomment-258524054&data=02|01|tdykstra%40microsoft.com|97be81be33f24ce481d308d404e764f3|72f988bf86f141af91ab2d7cd011db47|1|0|636138839252822699&sdata=CVHN2fDsAjKC3ExvoFl0HxkxKhXYk4hKKgl%2F70U4ajU%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABfzY7CQeZSjSTufYaiF1UapeYUUFdsTks5q64USgaJpZM4KpxYW&data=02|01|tdykstra%40microsoft.com|97be81be33f24ce481d308d404e764f3|72f988bf86f141af91ab2d7cd011db47|1|0|636138839252822699&sdata=os2RiOVYYKVQuyhfPN3TjROZ1Tr7Kz02JiLySRcxTi8%3D&reserved=0.


Implement cloud-init for multiple nics

Implement support for configuration of multiple NICs within cloud-init and keep it backward compatible with previous approach.
Following are some details. NICs definition is a list of hashes and is passed to OVIRT::VM#cloudinit via the opts[:nicsdef] option.
The format should be the following:
opts[:nicsdef] = [
  {
    :nicname => 'eth0',
    :boot_protocol => 'STATIC',
    :ip => '<IP_ADDRESS>',
    :netmask => '<IP_ADDRESS>',
    :gateway => '<IP_ADDRESS>'
  },
  {
    :nicname => 'eth1',
    :boot_protocol => 'STATIC',
    :ip => '<IP_ADDRESS>',
    :netmask => '<IP_ADDRESS>',
  },
  {
    :nicname => 'eth2'
  }
]

opts[:nicsdef] = [
  {
    :nicname => 'eth0',
    :boot_protocol => 'DHCP'
  }
]

Each NIC is configured to:

default to 'NONE' boot protocol if :boot_protocol is not specified,
default to 'true' if :on_boot is unspecified.

The :gateway is optional for a NIC definition thus letting the user to specify which interface should be used for a default gateway.
Please note that opts[:nicsdef] is evaluated if opts[:nicname] is also defined, i.e. definition of opts[:nicname] has a higher precedence than opts[:nicsdef].
@abenari, @jhernand: Would it be possible to merge this into upstream?

@jhernand: Thanks :+1


cmd/cgo: invalid flag in #cgo CFLAGS: -fno-plt

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
1.10
Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/puka/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/puka/go"
GORACE=""
GOROOT="/usr/lib/go"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build840200638=/tmp/go-build -gno-record-gcc-switches"
What did you do?
i installed qt and the bindings for go and i tried to run an example from github but i got this error.
If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.
Code for the code i was trying to run:
https://pastebin.com/RWK3aeEw
What did you expect to see?
My qt application
What did you see instead?
go build github.com/therecipe/qt/core: invalid flag in #cgo CFLAGS: -fno-plt

seems CGO_CFLAGS_ALLOW enviroment variable (https://github.com/golang/go/blob/master/src/cmd/go/alldocs.go#L1038) can help.

cc @ianlancetaylor

(See also https://github.com/golang/go/issues/23749.)

You can leave this open if you want it to be added to the list so you don't have to use CGO_CFLAGS_ALLOW

no-plt is ok


Run custom actions during feature update link is not working

The Run custom actions during feature update link is broken.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: c880203c-6a5b-28e7-c74d-152890baa955
Version Independent ID: 59c9eb13-dd44-358a-130d-ca1e020573ba
Content: What's new in Windows 10, version 1803
Content Source: windows/whats-new/whats-new-windows-10-version-1803.md
Service: unspecified
GitHub Login: @greg-lindsay
Microsoft Alias: trudyha

Hi, Sergio - can you check again? We had a delay between content publishing, but it's working for me now. Thanks!

Yes, I did a pull request for another link that is wrong, should remove "review" in hyperlink,see #815, they only merged our request and closed my pull request.

I also edited the document to fix the link. I suppose they will approve it soon.


Test Issue created at Mon, 06 Dec 2021 02:19:56 GMT

Test issue body  Mon, 06 Dec 2021 02:19:56 GMT

Comment on issue at Mon, 06 Dec 2021 02:19:57 GMT


Performance bug in version 2.0.0 with clang compiler (UTF-16 to UTF-8)

The new UTF-16 to UTF-8 transcoding function with AVX-512 has terrible performance when built with clang. I consider it to be a bug.
 ./benchmarks/benchmark -P convert_utf16_to_utf8+icelake -P convert_utf16_to_utf8+haswell -F ../unicode_lipsum/lipsum/Chinese-Lipsum.utf16.txt -I 10000
We define the number of bytes to be the number of *input* bytes.
We define a 'char' to be a code point (between 1 and 4 bytes).
===========================
Using ICU version 70.1
testcases: 1
input detected as UTF16 little-endian
current system detected as icelake
===========================
convert_utf16_to_utf8+haswell, input size: 46922, iterations: 10000, dataset: ../unicode_lipsum/lipsum/Chinese-Lipsum.utf16.txt
   5.282 GB/s (2.4 %)    2.641 Gc/s     2.00 byte/char
convert_utf16_to_utf8+icelake, input size: 46922, iterations: 10000, dataset: ../unicode_lipsum/lipsum/Chinese-Lipsum.utf16.txt
   2.297 GB/s (0.7 %)    1.148 Gc/s     2.00 byte/char

I should be able to fix it quickly.

Fix in 2.0.1