Clearing the /var/www/html/ folder without warning

Someone who doesn't know any better is gonna run install.sh and wipe out files already stored in /var/www/html with no warning.

I have added a check of how many files are in the /var/www/html/ folder, and if there is more than 1, a notice will popup about clearing the folder.


Improve UI Layout

Problem
The current layout leaves a lot of wasted real estate and also means I cannot see the data and the graph at the same time.
Proposed Solution

move the graph under the sequence list
have the data pane on the right take the whole side. We can then subdivide the data pane more easily (e.g. to handle large state objects).

More things:

[ ] want top bar to stay like you can do in a spreadsheet, so you can toggle raw/render and stay in place.
[ ] also want how github shows what function you're in when things are collapsed so you know where you are/what you're looking at -- e.g. that could be in the top bar.
[ ] when the left is minimized, you can't see what the name of the action is that you're on.
[ ] render forms components should be collapsable -- could even allow people to specify a config as to what to/not to render.

Most of these are solved by: https://github.com/DAGWorks-Inc/burr/pull/121

Closing this initial ticket. We can create more.


Unrecognized Argument in Sensitivity analysis

After updating the repo, I attempted to run a sensitivity analysis and encountered a new error:
'SA_setup_and_run.py: error: unrecognized arguments: /data/workflows/newtest2/output/initial_params_run_dir'
It seems somewhere in the setup, the initial_params_run_dir is being created and not recognized when running setup_multi which causes the run to fail. All subdirectories are being created, but fails before creating sample folders due to this error. Can't seem to find where exactly the creation of the folder is being called and unrecognized

Fixed with 2d8e1e5 (straggling changes that should have been committed with 8f21898cd98).


Support removing "_keys" from the HEC event

When specifying source_key, index_key and host_key the keys are also included in the event which duplicates the data since they are also added to the "source", "index" and "host" respectively.
There should be an option to remove the keys from the event. For example like the plugin:
https://github.com/fluent-plugins-nursery/fluent-plugin-cloudwatch-logs

Have created PR with suggested feature request


Rolling window umap

TLDR: Is there a way to implement rolling time windows in umap, where new samples are embedded and old ones  (and their influence on the embedding) are removed?
Thank you so much for umap, it has been very, very useful for me.
I am mainly using it to process bibliograpic data of scientific papers.
Example:
https://homepage.univie.ac.at/noichlm94/full/structphil/index.html
I made visualizations with embeddings in two dimensions, and am currently working on temporal embeddings, as shown below. (Every point represents a paper, their distance is computed from co-citations.)

The second embedding was made by reducing to one dimension which was then spread out over several years. This is not really satisfactory but, we can already see how clusters fade out after some time.
I have now experimented with doing one embedding (_fit) of the whole data, and then embedding every year seperately (_transform).
This has greatly improved the results, but is not totally satifying, as we see every year from the perspective of the whole history. Example from a different project (please ignore the heading):

I think it would be better (and epistemically more sound) if I could embed every year in the year(s) directly before it. For that I would have to embedd, lets say, the first five years, transform the sixth according to them, and then somehow remove the first, and so on.
Is there any way of doing this, while keeping the whole embedding constant?

First of all, thanks for the link to your work -- it looks awesome!
As to your question, I am not sure I have a solution that completely fits your needs, but it may be sufficient.
You could embed the first 5 years, then transform the 6th. Then take the embedding of the points for all but the first year and save it off. The do an embedding for the next window (years 2-6) but use the ability to provide an initialisation in the form of pre-specified points (you can pass init=<numpy array>) to initialise that embedding with the positions for those points that you saved off. You can then transform in the 7th year, save off the appropriate embedding data and repeat the process, sliding along. That's rather compute intensive, so you might want to consider larger windows and longer slides if it is getting to be too much, but it will mostly do the job.
So the downside is that it won't hold the embedding fixed throughout -- it will evolve over time. On the other hand by setting the initialisation to the previous embedding you should have fairly consistent embedding from window to window so it will drift rather than change dramatically each time.
I'm not sure if it meets your needs, but hopefully it might work.
Please let me know how your project ends -- it looks intriguing!

Thank you so much for the encouragement and the comment. I came up with this code that I think implements what you describe. Weirdly my Jupyter-Kernel (and nteract, and running as .py) dies somewhere in the loop around 70% of the time for  window_size=1, and 95% of the time for window_size<1.
Code:
import umap
from scipy import stats

n_neighbors = 10
window_size = 2
l = list(np.where(drc["year"] - drc["year"].min() <= window_size)[0]) #drc is where I keep the bibliographic data
L = M[l] #M is the sparse Matrix where I keep the vectors

trans = umap.UMAP(n_components=1, n_neighbors=n_neighbors, min_dist=0.01, metric='cosine').fit(L.toarray())

coordinates = []
for year in range(int(drc["year"].min()+window_size),int(drc["year"].max()-window_size)):
    l = list(np.where(drc["year"] == year)[0])
    L = M[l]

    z = []
    for w in range(0,window_size):
        k = list(np.where(drc["year"] == year+w-1)[0])
        z = z + k
    new_list = M[z]

    if len(l) > n_neighbors:
        #Transform data on trans
            old_embedding = trans.embedding_
            #old_embedding = old_embedding[~np.isnan(old_embedding)]
            embedding = stats.zscore(trans.transform(L.toarray()))

            emb = pd.DataFrame(embedding)
            emb.columns = ['xI']

            emb["year"] = drc.iloc[l,:]["timestamp"].tolist()
            coordinates.append(emb)
            coordinates_full_df = pd.concat(coordinates, ignore_index=True)
            coordinates_full_df.to_csv('yearly_output.csv')
        #Renew trans:
            while True: #We have to do this, as umap sometimes fails and only returns NaNs
                trans = umap.UMAP(n_components=1,
                          n_neighbors=n_neighbors,#small => local, large => global: 5-50
                          min_dist=0.01, #small => local, large => global: 0.001-0.5
                          metric='cosine',
                          init = old_embedding).fit(new_list.toarray())
                if np.isnan(trans.embedding_).any() ==True:
                    continue
                else:
                    break

coordinates_full_df = pd.concat(coordinates, ignore_index=True)

plt.figure()
plt.scatter(coordinates_full_df["xI"], coordinates_full_df["year"], s= 5)
plt.show()

Without the z-transformation this gives me something like this:

Or this:

When I z-transform the embedding for every year, I end up with this:

So at the moment it is not as satisfying as the previous version, but it seems to work in principle. If you have any ideas about the code I would love to hear them. Otherwise thank you again for your help, and I will update if I make further progress.

The z-transform seems to be helping, but I can't help but feel something is going astray here -- I would not in any way have expected the marching blocks effect seen in your first plot there. I don't see anything obviously wrong in your code, so now I'm a little suspicious in general of the approach I suggested. I'll try to think about it some more and see if I can come up with anything better.

So, I never got the rolling-window-thing to work. I have tried various different combinations of standardization techniques to keep the window from shifting, but I never got a nice and coherent structure out of it. But the problem was not that central to the project, which I just finished. I have produced the graphic below as a final version.
Thank you  for your time, if you want to utilize it as a use-case for umap, please do.


Add some more newb friendly local dev blurb to DEVELOPMENT.MD

Release Note

Add more verbose instructions for local development to DEVELOPMENT.MD

/lgtm
/approve


Ubuntu 20.04 not working

I'm currently running Ubuntu 20.04 and having a hard time getting microk8s to show the nodes. After running microk8s inspect, I get the following error:
(base) vinhdiesal@vinhdiesal-Blade-Stealth:~$ microk8s inspect
Inspecting Certificates
Inspecting services
Service snap.microk8s.daemon-cluster-agent is running
Service snap.microk8s.daemon-flanneld is running
FAIL:  Service snap.microk8s.daemon-containerd is not running
For more details look at: sudo journalctl -u snap.microk8s.daemon-containerd
Service snap.microk8s.daemon-apiserver is running
Service snap.microk8s.daemon-apiserver-kicker is running
Service snap.microk8s.daemon-proxy is running
Service snap.microk8s.daemon-kubelet is running
Service snap.microk8s.daemon-scheduler is running
Service snap.microk8s.daemon-controller-manager is running
Service snap.microk8s.daemon-etcd is running
Copy service arguments to the final report tarball
I looked at the containerD files and they're in the /var/snap/microk8s/current/args directory, so does this mean it doesn't work with Ubuntu 20.04?

Nevermind, it turns out that I had some kubernetes left lying around, after I performed a snap remove it resolved the issues.

@vinhdiesal microk8s works on ubuntu 20.04 in your case?
Thanks for confirming.


Adapter reboot when receiving message

The adapter is configured in "server" mode. If I send a message to the bot, the adapter reloads with an error. Messages from the adapter come to me without any problems.

time
debug
message

2018-11-30 19:41:44.140
- error:
telegram.0 uncaught exception: msg is not defined

2018-11-30 19:41:44.141
- error:
telegram.0 ReferenceError: msg is not defined

---
---
at IncomingMessage.req.on.e (/opt/iobroker/node_modules/iobroker.telegram/main.js:216:31)

---
---
at emitNone (events.js:86:13)

---
---
at IncomingMessage.emit (events.js:185:7)

---
---
at endReadableNT (_stream_readable.js:978:12)

---
---
at _combinedTickCallback (internal/process/next_tick.js:80:11)

---
---
at process._tickCallback (internal/process/next_tick.js:104:9)

2018-11-30 19:41:44.646
- info:
telegram.0 terminating

2018-11-30 19:41:44.663
- error:
Caught by controller[1]: ReferenceError: msg is not defined

2018-11-30 19:41:44.665
- error:
Caught by controller[1]: at IncomingMessage.req.on.e (/opt/iobroker/node_modules/iobroker.telegram/main.js:216:31)

2018-11-30 19:41:44.665
- error:
Caught by controller[1]: at emitNone (events.js:86:13)

2018-11-30 19:41:44.665
- error:
Caught by controller[1]: at IncomingMessage.emit (events.js:185:7)

2018-11-30 19:41:44.665
- error:
Caught by controller[1]: at endReadableNT (_stream_readable.js:978:12)

2018-11-30 19:41:44.665
- error:
Caught by controller[1]: at _combinedTickCallback (internal/process/next_tick.js:80:11)

2018-11-30 19:41:44.665
- error:
Caught by controller[1]: at process._tickCallback (internal/process/next_tick.js:104:9)

2018-11-30 19:41:44.665
- error:
host.iobroker instance system.adapter.telegram.0 terminated with code 0 (OK)

2018-11-30 19:41:44.666
- info:
host.iobroker Restart adapter system.adapter.telegram.0 because enabled

2018-11-30 19:41:45.705
- info:
host.iobroker instance system.adapter.telegram.0 started with pid 17078

2018-11-30 19:41:46.788
- info:
telegram.0 starting. Version 1.3.5 in /opt/iobroker/node_modules/iobroker.telegram, node: v6.14.4

2018-11-30 19:41:48.440
- info:
telegram.0 https server listening on port 8443

Please check 1.3.6 from Github and give feedback

Thank you, it works.

Ich bekomme das Update nicht hin, liegt es an mir?

Update telegram from @1.3.5 to @1.3.6
NPM version: 6.4.1
npm install<EMAIL_ADDRESS>--production --save --prefix "/opt/iobroker" (System call)
Killed
host.ioBroker-RasPi Cannot install<EMAIL_ADDRESS>137
root@ioBroker-RasPi:/home/pi# iobroker url https://github.com/ioBroker/ioBroker.telegram
install https://github.com/ioBroker/ioBroker.telegram/tarball/master
npm install https://github.com/ioBroker/ioBroker.telegram/tarball/master --production --save --prefix "/opt/iobroker" (System call)
Killed
host.ioBroker-RasPi Cannot install https://github.com/ioBroker/ioBroker.telegram/tarball/master: 137

System call killed ... Speicher voll? Mal bitte reboot


Multiple improvements and fixing most open issue

Adopting this pull request would be cool because it would make the library nicer at no expense and resolve pretty much all of the existing open issues. Given the popularity of this library and as a fellow user of it and nanoid contributor (I wrote zig-nanoid) I think it would be cool to reach 0 open issues and improve the library like this.
Below I detail everything I did.
Summary

Added useful alphabets from nanoid-dictionary so users don't have to maintain their own in most cases.

Made the default id size a public constant so people can easily reference it, this is constant across all nanoid implementations.

Improved or added missing docs to all public members.

Fixed various typos and exception instantiations.

Fixed issue #12 by using Int32.LeadingZeroCount and adding .net7 as a target for unit tests.

Fixed issue #9 by making CryptoRandom._r a non-static field.

Fixed issue #5 by adding more information in readme.md

Minor changes to formatting.

Remarks

If this is merged, I also recommend fixing issue #13. It seems like while the license file is specified in the csproj, nuget doesn't just pick it up from there. The MS docs from what I've seen suggest using a .nuspec file but I never published on nuget before so I am not sure how that works.

I also tried looking into issue #7 but it is not clear to me on that website what the problem with this library is. Only one vendor seems to tag the library as malicious but I can't see any rationale for it. I suggest either asking for more info from the person who opened the issue (assuming they have more knowledge on the topic and care about it) and see if there is a fix. If not, probably fine to just close.

If any of the formatting or name changes are undesirable I can also change those, I tried not to change too much in that regard though, so I think it should be ok.

Breaking changes
The way in which CryptoRandom works is technically a breaking change, as _r is no longer static, but I would argue that no one should have been abusing that implementation detail anyways, as #9 mentions, so in practice this shouldn't break anyone's code.
Testing
I compiled and ran the tests using .net8 and they all passed.

Not sure why the CI fails, reading the logs it seems to be because it can't find an appropriate .net version or something? Haven't used appveyor for this before so not sure, maybe a contributor that worked with this can look into it.

I'm picking up Nanoid for a project I'm building and a lot of these improvements are exactly what I'm looking for (in particular, an accessible alphabet).
Thank you for you work on this!

I identified an additional problem and documented it here: #15
Also implemented a fix and added it as part of this pull request.
If there is some detail that I missed which makes this issue irrelevant let me know but it seems like a real problem from everything I read and my solution I believe fixes it.

@SasLuca Thank your PR, I'll merge this PR first, releasing the new version may take a few more days. Thanks you, @SasLuca.

@SasLuca Thank your PR, I'll merge this PR first, releasing the new version may take a few more days. Thanks you, @SasLuca.

Hey @codeyu
is a new release scheduled any time soon? I'm currently trying out NanoIds and could really use these changes in your awesome lib.


TypedQuery#setFirstResult(int) isn't supported in HBase

When I create a TypedQuery
TypedQuery<...> query = ...

and call:
query.setFirstResult(4);

it doesn't have any effect.
Looking into the sources, it's clear that this method isn't taken into account.
#384 claims that pagination is currently supported for HBase. What does it mean? Is there any other way to do pagination?
Do you plan to support Query#setFirstResult method? (Is it even possible?)

@mhorejsi
query.setFirstResult() is not supported by kundera-hbase. Although you can limit number of records using query.setMaxResults() as it is supported.
-Devender

Do you plan to support it in the future? Is it even possible to implement it? I'm interested in query.setFirstResult() for kundera-hbase and kundera-oracle-nosql.

@mhorejsi
Hbase internally does not support setting initial page limit and hence is not supported by Kundera as well.
Chhavi

Closing with Kundera-3.2


l293d output should be annotated

When the library prints something, it should be prefixed by '[l293d]: '

Hi @jamesevickery would this involve updating this function to add that prefix?
Would you like the code refactored to use python's logging lib? That would make these kind of changes easier and also mean any exceptions would be logged in the same way.
I would be happy to create a pr #hacktoberfest

@crispg72 Yes, that's the right method.
I haven't used the logging lib before, although it looks more suited to what I'm using print for - feel free to make those changes. A PR would be perfect thanks

Ok great, I'm hoping to get this done later today.

Created https://github.com/jamesevickery/l293d/issues/72


[latex] remove svg from latex priority

This PR:

removes image/svg+xml as a latex image option.

@choldgraf I am not entirely sure how the solution xml is generated here. Do you run locally and visually inspect the solutions to update it?
E       AssertionError: FILES DIFFER:
39
E       /tmp/pytest-of-runner/pytest-0/test_complex_outputs_latex0/test_transform/test_complex_outputs_latex.xml
40
E       /tmp/pytest-of-runner/pytest-0/test_complex_outputs_latex0/test_transform/test_complex_outputs_latex.obtained.xml
41
E       HTML DIFF: /tmp/pytest-of-runner/pytest-0/test_complex_outputs_latex0/test_transform/test_complex_outputs_latex.obtained.diff.html
42
E       ---
43
E       +++
44
E       @@ -124,7 +124,6 @@
45
E                                plt.ylabel(r'a y label with latex $\alpha$')
46
E                                plt.legend();
47
E                        <CellOutputNode classes="cell_output">
48
E       -                    <image candidates="{'*': '_build/jupyter_execute/complex_outputs_17_0.png'}" uri="_build/jupyter_execute/complex_outputs_17_0.png">
49
E            <section ids="tables-with-pandas" names="tables\ (with\ pandas)">
50
E                <title>
51
E                    Tables (with pandas)

@chrisjsewell @choldgraf I don't see any docs on how to generate the xml representation to update the preferred solution for the latex test.
https://myst-nb.readthedocs.io/en/latest/develop/contributing.html#how-to-write-tests
I'll submit a docs PR to document.

Ah - so in this case, file_regression checks whether or not the data structure you give it differs from something on disk. If it differs at all then an error is raised.
If you want to regenerate the XML file on disk, just delete it, and re-run the tests. The test will fail the first time, saying the file doesn't exist, and it will create the file in the process. You should then inspect the file and make sure it looks correct. If so, re-run the test and it should pass.

If you want to regenerate the XML file on disk, just delete it, and re-run the tests.

 Does what its says, regenerates failing files

:-O

If you want to regenerate the XML file on disk, just delete it, and re-run the tests.

Easier than that, just run pytest --force-regen. Does what its says, regenerates failing files

Thanks guys. I can't seem to get it to work though. Do I need to issue this before the file is committed?
> pytest --force-regen
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --force-regen
  inifile: /Users/matthewmckay/repos-collab/ebp/MyST-NB/pytest.ini
  rootdir: /Users/matthewmckay/repos-collab/ebp/MyST-NB

@choldgraf it looks like the complex_outputs.ipynb file needs to be re-executed as the output block for the matplotlib figure is empty when I regenerate the xml representation.  However the example.jpg file is missing. Do you know where I can find it to commit?

Is it an embedded file from markdown? Hmm weird. We could just point it to some other image file that's already in the repo

Is it an embedded file from markdown? Hmm weird. We could just point it to some other image file that's already in the repo

Roger that. I'll just replace it -- seems only to be used in the one jupyter notebook.

[ ] @choldgraf when I re-executed the complex_outputs.ipynb and then used pytest --force-regen the xml for non-latex builds also updated. Would you mind to case a quick look at the diff. It looks ok to me with replacement binary blobs -- but some of the changes may be version specific such as pandas latex representations.

hmm - they look OK to me (though tests aren't passing). If the tests end up being brittle or platform dependent, then we can run the tests on a subset of the generated XML if need be

Btw, should we be entirely removing SVG from the priority list or just moving it to the bottom? This would be useful when the only output is SVG, in conjunction with sphinx.ext.imgconverter  or sphinxcontrib-svg2pdfconverter.

Btw, should we be entirely removing SVG from the priority list or just moving it to the bottom? This would be useful when the only output is SVG, in conjunction with sphinx.ext.imgconverter  or sphinxcontrib-svg2pdfconverter.

@chrisjsewell I have removed it from latex. @choldgraf do you mind if I demote the svg output for html in this PR?

I have removed it from latex

Yes but that's the point; these extensions convert SVGs to PNG/PDF during the creation of latex outputs.

do you mind if I demote the svg output for html in this PR?

What would you be demoting it behind? SVG should most definitely be preferential to PNG/JPEG for HTML

ah sorry @chrisjsewell I misunderstood your previous comment. I see what you're saying. Still learning how this mime stuff works. I'll add it to the end of latex. So sphinx can convert svg to png behind the scenes for latex? I was having issue with latex trying to \includegraphics[{{file.svg}] which won't work in LaTeX.

So sphinx can convert svg to png behind the scenes for latex?

Yep that's the idea; they generally call on an external program like imagemagik, libRSVG or inkscape to convert them (that you obviously also need to have installed). Whether they are currently compatible with myst-nb is something I think should definitely be tested at some point; in terms of the stage in the sphinx process at which both are called. I know that I certainly use packages that only output SVG (like graphviz), so it would be good to still have a way to use these output when creating PDFs, and we should document the preferential way to do this somewhere in the documentation.
The main thing with graphics though is that, where possible, you should be looking to use vectorised (SVG/PDF) graphics, which have lossless quality with scaling, over rasterized (PNG/JPEG) ones.

thanks @chrisjsewell -- that's helpful info. How do you trigger the conversion though. When svg was the preferred output the latex generated was including the svg as an image and then failing as \includegraphics couldn't parse the file type. Should latex be converting these through an update to preamble or should sphinx convert svg -> png prior to writing latex?

How do you trigger the conversion though

For SVG in standard rST files the conversion is just triggered by adding the extension to conf.py.
Obviously without either of these extensions loaded, having only SVG will definitely fail at present.
The two sphinx extensions I mentioned do it on the sphinx end (prior to writing latex): this could be an issue, given that we inject the image into the doctree at quite a late stage (hence why I am not certain of compatibility). In the last few years there is now also a latex package which does similar on the latex end: https://ctan.org/pkg/svg?lang=en, which may be another option.
This should be raised as an issue to look into as a long-term goal

Looking over the code again; it looks like we "hard-code" in the priority lists: https://github.com/executablebooks/MyST-NB/blob/afba1fe9ccae394e9b13621e113ab96a5266c6fe/myst_nb/transform.py#L18
As opposed to in juoyter-sphinx, where it is a configuration variable (albeit only allowing for a single list and not per builder): https://github.com/jupyter/jupyter-sphinx/blob/6a0df8575fd1fc76e1258aab300ec415eee92569/jupyter_sphinx/init.py#L155
I think the priority lists should be made configurable, to cope with use cases like this; whereby in general SVG should not be in the priority list for latex, since it will fail, but then you may want to add it in if you are also using an extension that makes it compatible.

thanks @chrisjsewell I agree with configurability. Looking at sphinxcontrib-svg2pdfconverter seems like a good option to support svg so I will open an issue to track this discussion for future support.

Just opened an issue 👍

Regenerating the support test files is failing. It seems like there is a mismatch in the set of files for test test_complex_output but don't understand how to debug the issue.
>       assert filenames == {
            "complex_outputs_17_0.svg",
            "complex_outputs.ipynb",
            "complex_outputs_17_0.pdf",
            "complex_outputs.py",
            "complex_outputs_24_0.png",
            "complex_outputs_13_0.jpg",
        }
E       AssertionError: assert {'complex_out...uts_24_0.png'} == {'complex_out...uts_24_0.png'}
E         Extra items in the left set:
E         'complex_outputs_17_0.png'
E         Extra items in the right set:
E         'complex_outputs_17_0.pdf'
E         'complex_outputs_17_0.svg'
E         Use -v to get the full diff

/Users/matthewmckay/repos-collab/ebp/MyST-NB/tests/test_parser.py:61: AssertionError
---------------------------------------------------------------- Captured stdout call -----------------------------------------------------------------
{'complex_outputs.ipynb', 'complex_outputs_24_0.png', 'complex_outputs_17_0.png', 'complex_outputs.py', 'complex_outputs_13_0.jpg'}
=============================================================== short test summary info ===============================================================
FAILED tests/test_parser.py::test_complex_outputs - AssertionError: assert {'complex_out...uts_24_0.png'} == {'complex_out...uts_24_0.png'}
============================================================ 1 failed, 34 passed in 21.13s ============================================================

but don't understand how to debug the issue.

Before we were getting pdf and svg outputs from cell 17, now we are only getting png.
I imagine that is the matplotlib outputs. I don't see why that has only just changed in this PR, but I known that in a previous commit, I removed the ipypublish code from the test notebooks, which was calling this function to set up matplotlib to output PDF and SVG, and also another to set up pandas to output latex: https://github.com/chrisjsewell/ipypublish/blob/43c6ff4be0b8eac3c7d2062cec6d1e459d5baa0c/ipypublish/scripts/nb_setup.py#L177.
They're actually very useful functions for getting these packages to create optimal outputs for sphinx, and would be good if we could incorporate them somewhere in the executablebook stack

I don't see why that has only just changed in this PR

Agreed. Don't understand why the change is in this PR from looking at what has changed.

@choldgraf @chrisjsewell I made this change to the test_parser set of tests but not 100% sure why it needed to be made.
For some reason we are generating the png file only for complex_.._17 and not the svg and pdf versions of that image.
Tests should pass now.

[x] this fixes latexpdf builds for https://github.com/executablebooks/myst-nb.example-project

ping for @mmcky - it this is all ready to go then let's get it in, just need to hear from y'all if that's the case!

sorry @choldgraf -- didn't see first message -- good to go.

once this is merged I will test more widely

ok let's do it! :-D


Optimizing Image Compression

Is your feature request related to a problem? Please describe.
Devices capture images larger than 5MB, resulting in processing errors within the SDC library. Users encounter obstacles as the current library lacks a dedicated attachment feature to efficiently handle and compress images that surpass this size threshold. The absence of a tailored solution restricts users from seamlessly proceeding with larger image files. I've attached a sample screenshot below.

Describe the solution you'd like
I'd like to create a function that will compress the image to a disired size.
Describe alternatives you've considered
N/A
Additional context
Would you like to work on the issue?
Yes

@Itskiprotich thanks for raising this! what exact APIs are you proposing here? does it need to live in the sdk?

@jingtang10 I intend to incorporate the Android built-in Bitmap compression capabilities. Leveraging the native Bitmap.compress() function.


how to get image and send message

Hello sigalor,
i started my panel and login successfully .
but how can find my image and send message. please guide me
thank Manoj Rao

Please refer to #18 for a similar issue. The project does not yet support actually looking at your message like regular WhatsApp Web. If the decoding is successful, you may be able to see the contents of some of your messages in the JSON trees.
Sending data is not yet implemented at all, but I have started investigating and documenting it.

Hey, solved the image one here issue#30 now I want to get voice notes and videos


Game crashing with the mod

Before reporting, I have confirmed that

[X] This bug does not appear to be reported on GitHub Issues before.
[X] This bug still exists in the latest version of the Minecraft Transit Railway mod.
[X] I have tested this in the official version of Minecraft Transit Railway, not a modified, custom, or unofficial build.

Describe the Bug
My game crashes after updating the mod to the latest version.
Reproduction Steps
1.Update the mod from 3.2.2 to 4.0.0 beta 8
2.Start the game
3.
In-game Log and Crash Report
https://mclo.gs/sUYfe8C
Mod Loader
Forge
Minecraft Version
1.19.2
Minecraft Transit Railway Version
4.0.0 Beta 8
Operating System
Windows 10
List of Mods or Modpack You Were Using
	FarmersDelight-1.19.2-1.2.3.jar
	TerraBlender-forge-1.19.2-<IP_ADDRESS>.jar
	jei-1.19.2-forge-<IP_ADDRESS>6.jar
	simplylight-1.19.2-1.4.5-build.42.jar
	exlinecopperequipment-forge-1.19.2-v2.0.6.jar
	Mekanism-1.19.2-<IP_ADDRESS>.jar
	flywheel-forge-1.19.2-0.6.10-20.jar
	Patchouli-1.19.2-77.jar
	create-1.19.2-0.5.1.f.jar
	Xaeros_Minimap_23.6.0_Forge_1.19.1.jar
	Croptopia-1.19.2-FORGE-2.2.2.jar
	worldedit-mod-7.2.12.jar
            Filters API
	cfm-7.0.0-pre35-1.19.2.jar
	architectury-6.6.92-forge.jar
	MTR-forge-1.19.2-3.2.2-hotfix-1.jar
	Tianjin-Metro-1.0.0-beta-1+1.19.2.jar
	Joban-Client-Mod-1.19.2-1.2.2.jar
	MekanismGenerators-1.19.2-<IP_ADDRESS>.jar
	PuzzlesLib-v4.4.3-1.19.2-Forge.jar
	Oh_The_Biomes_You'll_Go-forge-1.19.2-<IP_ADDRESS>.jar
	CorgiLib-forge-1.19.2-<IP_ADDRESS>.jar
	forge-1.19.2-43.4.2-universal.jar
	MTR-NTE-0.4.1+1.19.2.jar
	geckolib-forge-1.19-3.1.40.jar
	MTR-MSD-Addon-forge-1.19.2-3.2.2-1.3.4-enhancement

Extra Information
Forge 43.4.2

new log after deleting NTE mod:https://mclo.gs/856QHdQ
delete nte mod doesn't work

Please update all MTR addon (Joban Client Mod, MTR MSD) to a version that supports MTR 4

Please update all MTR addon (Joban Client Mod, MTR MSD) to a version that supports MTR 4

I updated them.

The latest version of JCM is 2.0.0-beta-3
Please re-capture the list of mods you are using right now that is causing the crash

The latest version of JCM is 2.0.0-beta-3 Please re-capture the list of mods you are using right now that is causing the crash

Oh, thank you, I will try.

The latest version of JCM is 2.0.0-beta-3 Please re-capture the list of mods you are using right now that is causing the crash

I updated it, and I believe that the situation has eased somewhat, but the game still crashes: https://mclo.gs/HC6IUDn

This is likely a bug, please open a new issue with the log above, thanks.
Please note that MTR 4 is still in Beta. If you need a stable experience, you are advised to stay in MTR 3.2.2 until MTR 4 is officially released.


Fix consistency in test naming, fix issue where clicking share button…

… then clicking a side menu will incorrectly give  share menu option

Welcome to Codecov :tada:
Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.
:information_source: You can also turn on project coverage checks and project coverage reporting on Pull Request comment
Thanks for integrating Codecov - We've got you covered :open_umbrella:


feat: application emojis

Summary
Implements https://github.com/discord/discord-api-docs/pull/7010
This PR is about the Application Emoji feature for Discord.
It's my first time submitting a related PR, and although I've ensured that the functionality should be working, there are probably still some issues. I'm also not very familiar with the docs.
I'm sorry for any inconvenience this may cause. If you have any suggestions or questions, please let me know, and I'll address them as soon as possible!
Checklist

[ ] If code changes were made, then they have been tested

[ ] I have updated the documentation to reflect the changes
[x] I have formatted the code properly by running pdm lint
[ ] I have type-checked the code by running pdm pyright

[ ] This PR fixes an issue
[x] This PR adds something new (e.g. new method or parameters)
[x] This PR is a breaking change (e.g. methods or parameters removed/renamed)
[ ] This PR is not a code change (e.g. documentation, README, ...)

Hi, don't want to make a big deal out of this or anything but I'd appreciate it if you didn't blatantly copy-paste the code from https://github.com/Pycord-Development/pycord/pull/2501 (including the mistakes in the first version), thanks 🥰

Hi,  don't want to make a big deal out of this or anything but I'd appreciate it if you didn't blatantly copy-paste the code from Pycord-Development/pycord#2501 (including the mistakes in the first version), thanks 🥰

Hi @NeloBlivion, Don't get me wrong, I understand your concern I believe the person who submitted the MR was trying to help, not being malicious. both projects use the MIT License, and using and sharing code like this is allowed. It’s part of what makes open-source great!
Considering that both pycord and disnake are API wrappers for the same platform, it's natural that similar solutions might emerge, so I don't believe reinventing the wheel is necessary.

Hi @NeloBlivion, Don't get me wrong, I understand your concern I believe the person who submitted the MR was trying to help, not being malicious. Both projects use the MIT License, and using and sharing code like this is allowed. It’s part of what makes open-source great!
Considering that both pycord and disnake are API wrappers for the same platform, it's natural that similar solutions might emerge, so I don't believe reinventing the wheel is necessary.

Oh yeah, I get it - there's only so many differences you can get between libraries serving the same purpose, that were originally based off of the same project. I also enjoy looking at other library implementations to see how other people tackle new features (especially in updates like these where it can have greater implications on other structures).
It's not really a big issue and I don't expect this to close or anything, that being said i just found it strange that someone would directly copy a draft PR to pass off as their own work (obviously it's still under MIT license where this is perfectly fine, but still felt off to me)
anyway havefun i guess

Hi, don't want to make a big deal out of this or anything but I'd appreciate it if you didn't blatantly copy-paste the code from Pycord-Development/pycord#2501 (including the mistakes in the first version), thanks 🥰

Thank you for your reply. I apologize for making you think that I copied and pasted your code written in Pycord. In fact, I admired your work so much when I saw your code. Therefore, I studied some of your code and combined it with my own ideas to optimize it. If this has caused you any discomfort, I am truly sorry and ask for your understanding.

Since this code will be a breaking change for Disnake, I believe it will differ from Pycord's code, and I will continue to modify it. It is undeniable that my code was indeed inspired by yours, which may have unintentionally included some of your errors. For this, I apologize again. Your code is truly excellent.

Since this code will be a breaking change

What's the part that makes it a breaking change?

@Mantouisyummy in future, if you more or less copy code, at least disclose the original source and author in a footnote

Pycord is licensed under the MIT License, which gives anyone free reign over its code. It's also inevitable that 2 forks of discord.py will share similar code/implementation details due to the design of the original library and the fact that they are both adding the same features from the Discord API (duh). So disclosing the "original" author/source is unnecessary here, it was just a matter of timing between this PR and the Pycord one.

Hi there, thanks for the PR. After some internal deliberation, I'm making the decision to close this.
While other commenters may not be wrong that the MIT license technically/legally allows this, and that both libraries themselves are naturally quite similar by virtue of being forks, I'm not comfortable with merging this PR given the circumstances, sorry.
The base of this PR is quite clearly copied (at least partially) from the original linked PR - looking at a diff of diffs (select "Character" change highlighting) between this PR's base commit and the original PR's first commits up to the same date/time, some sections (e.g. is_usable or _remove_emoji) line up too well for them to just be "inspired" by another PR.
Aside from pre-existing differences between the two libraries, most of the other changes seen in that diff are formatting changes or renames, as far as I can tell. You've put in some work into this PR beyond that, which I don't want to just dismiss here, but it doesn't change the fact that the groundwork is the same.
Even if the license allows it, the fact that there wasn't a single reference to the original PR or any sort of attribution somewhat goes against the open source spirit, imho. The original author has made it pretty clear that they ultimately don't really approve of this - if this was coordinated with them, it would've probably been fine.
While maintaining an ideal "clean room" design for a library is probably unfeasible, I think everyone would still appreciate it if PR authors at least majorly author their own PRs.

In terms of implementation, it seems there's still some bikeshedding to be done. Before someone (you, or anyone else) gives implementing this another try, it might be worth talking about some design specifics first, either in a separate issue or #bikeshedding.


cp: incorrectly copies directory when path ends in dot

cp -R incorrectly copies a directory when the path ends in a dot.
GNU cp v8.30:
mkdir x y
cd y
cp -R ../x/. .  # succeeds and does *not* copy x/ into y/

uutils cp:
mkdir x y
cd y
../target/debug/cp -R ../x/. .  # succeeds and *does* copy x/ into y/

The corresponding GNU test case is tests/cp/src-base-dot.sh. This test is incorrectly marked as passing on the main branch because the GNU test case doesn't actually verify that the directory x/ is not copied to y/.
Originally mentioned by @jfinkels in https://github.com/uutils/coreutils/issues/3894#issuecomment-1236041009

I'm working on this!

I had a quick look into this. The issue is that rust normalizes out of existence most occurrences of '.' in paths whenever API methods are called (i.e. lazily) which interacts incorrectly with the logic that's in place. In particular,
let root_parent = if target.exists() {
    root_path.parent()
} else {
    Some(root_path.as_path())
};

produces the wrong answer in the case of the code in cp. If we take the example, then
cp -R ../x/. .

leads to root_path = <cwd>/y/../x/. and root_path.parent() = <cwd>/y/.. and this is incorrect; the x has been lost from the root path because parent() normalized the trailing dot out of existence before it did anything. This then leads to it being treated as if it were contained in the target, and everything stems from this.
I don't see an easy fix here; it's probably going to come down to a custom implementation running over the OsStr representation of the path I guess.

Maybe this is a duplicate of #3886? If so it may be fixed by pull request #3954.

Closing this ticket as it seems to be fixed and the directory is not copied when running the example.


Implement structured logging

HI,
thanks for you work! This PR implements structured logging. In kubernetes, the json log format can be used and passed to a ELK stack.

@martin-helmich what did you thing about this, after I would rebase this?

rebased


Azure Function Timer trigger doesn't invoke accordingly to cron expression

Issue description
I have Azure Function written in F# (dotnet 8) deployed to Container App Environment with function trigger as below
module TimerFunctions =
    [<Function("TimerFunction")>]
    let Run ([<TimerTrigger("0 0 * * * *")>] timer: TimerInfo, log: ILogger) =
        Console.WriteLine("================== Hey, I am a 'TimerFunction'. I am really lazy, so I only print to console ==================")

Sometimes this function is not invoked.
My function is scaled to 0, but this should not be an issue for this trigger

While all triggers can be used, only the following triggers can dynamically scale (from zero instances) when running in a Container Apps environment:

HTTP
Azure Queue Storage
Azure Service Bus
Azure Event Hubs
Kafka
Timer

Expected behavior [What you expected to happen.]
I expected to have this function executed according to the cron expression.
Screenshots

Additional context
Here are version of packages I use:
	<PackageReference Include="Microsoft.Azure.Functions.Worker" Version="1.23.0" />
	<PackageReference Include="Microsoft.Azure.Functions.Worker.Sdk" Version="1.18.1" OutputItemType="Analyzer" />
	<PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Http" Version="3.2.0" />
	<PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Http.AspNetCore" Version="1.3.2" />
	<PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Timer" Version="4.3.1" />
        <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.ServiceBus" Version="5.22.0" />

@kmadof thank you for reaching out. When you mention that executions got skipped you meant 17/10 6,7 and 9am as examples?

hi @kmadof  - can you confirm if the function is deployed using Azure functions on Azure Container Apps hosting as mentioned in the link - https://learn.microsoft.com/en-us/azure/azure-functions/functions-deploy-container-apps?tabs=acr%2Cbash&pivots=programming-language-csharp

Hi @kmadof , I am sorry about the issue you have faced. This is not expected behaviour of the trigger. It can scale from and to zero. May I ask you raise an issue here in the functions github repo - https://github.com/Azure/azure-functions-on-container-apps/issues

@kmadof thank you for reaching out. When you mention that executions got skipped you meant 17/10 6,7 and 9am as examples?

Yes, that's my point.

Hi @kmadof , I am sorry about the issue you have faced. This is not expected behaviour of the trigger. It can scale from and to zero. May I ask you raise an issue here in the functions github repo - https://github.com/Azure/azure-functions-on-container-apps/issues

Yes of course!

Here is link to new issue

We'll track this in the other issue. Closing this one.


the random example don't work when you update to 1.2.4

It's an angular issue, need to re-write the example

I had to write the *ngFor differently nothing to do with the package. Maybe upgrading to the latest angular/typescript had a change that I wasn't aware of


[v0.13] Release Tracker

We cut a release branch for the 0.13 release. (common commit  with main is 69ce452)
This issue is for tracking cherry-picks to the release branch.
The POCs for the TorchVision team are @nicolashug  and @YosuaMichael
Cherry-Pick Criteria
Phase 1 (until 06/06/22):
Only low-risk changes may be cherry-picked from master:

Fixes to regressions against the most recent minor release (e.g. 0.12 for 0.13 release);
Critical fixes for: silent correctness, backwards compatibility, crashes, deadlocks, (large) memory leaks
Fixes to new features introduced in the most recent minor release (e.g. 0.12 for 0.13 release)
Test/CI fixes
Documentation improvements
Compilation fixes or ifdefs required for different versions of the compilers or third-party libraries
Release branch specific changes (e.g. change version identifiers)

Phase 2 (after 06/06/22):
Note that changes here require us to rebuild a Release Candidate and restart extended testing (likely delaying the release). Therefore, the only accepted changes are Release-blocking critical fixes for: silent correctness, backwards compatibility, crashes, deadlocks, (large) memory leaks
Changes will likely require a discussion with the larger release team over VC or Slack.
Cherry-Pick Process

Ensure your PR has landed in master. This does not apply for release-branch specific changes (see Phase 1 criteria).

Create a PR against the release branch.

# Find the hash of the commit you want to cherry pick
# (for example, abcdef12345)
git log

git fetch origin release/0.13
git checkout release/0.13
git cherry-pick abcdef12345

# Submit a PR based against 'release/0.13' either:
# via the GitHub UI
git push my-fork

# via the GitHub CLI
gh pr create --base release/0.13

Make a request below with the following format:

Link to landed master PR (if applicable):
*

Link to release branch PR:
*

Criteria Category:
*

the POC will reply with approved / denied or ask for more information.
If approved, the PR can be merged once the tests pass.

NOTE: Our normal tools (ghstack / ghimport, etc.) do not work on the release branch.
Please note HUD Link with branch CI status and link to the HUD to be provided here.
HUD
Versions
0.13

PR for main: https://github.com/pytorch/vision/pull/6082
PR for release/0.13: https://github.com/pytorch/vision/pull/6084
category: bugfix

Handling edge cases on test_backbone_utils when the tracer sample nodes that is buffer with integer dtype, in this case the function Tensor.mean() will raise error since it does not accept integer. We add .float() before .mean() with this PR.
PR for main: https://github.com/pytorch/vision/pull/6090
PR for release/0.13: https://github.com/pytorch/vision/pull/6091
category: bugfix

Refactor swin_transformer so it can be compatible with the 3d version (hence we dont introduce BC changes in the future release)
PR for main:  https://github.com/pytorch/vision/pull/6088
PR for release/0.13: https://github.com/pytorch/vision/pull/6100 (waiting the CI to be green before merging)
category: backward compatibility

Link to landed master PR (if applicable):

NA

Link to release branch PR:

https://github.com/pytorch/vision/pull/6101

Criteria Category:

Critical CI fixes

Fix the CI error unsafe repo
PR for main: https://github.com/pytorch/vision/pull/6106
PR for release/0.13: https://github.com/pytorch/vision/pull/6109 (MERGED)
category: CI bugfix

Add M1 CI Build setup
PR for main: https://github.com/pytorch/vision/pull/5948 and https://github.com/pytorch/vision/pull/6110
PR for release/0.13: #6112 (Wait until CI is green)

Add missing _version to the MLPBlock
PR for main: https://github.com/pytorch/vision/pull/6113
PR for release/0.13: https://github.com/pytorch/vision/pull/6114
category: bugfix

https://github.com/pytorch/vision/pull/6117 might need to be cherry-picked as well. Waiting for @malfet 's input.

#6135 might also need to be cherrypicked. Pretty much all M1 related commits + the BE work that Nicolas highlighted above need to be cherrypicked. I think we should do it once @malfet gives us the OK that all M1 related work is completed.

https://github.com/pytorch/vision/pull/6140 as well

Also https://github.com/pytorch/vision/pull/6147

Fix ViT and Resnext docs
PR on main: https://github.com/pytorch/vision/pull/6150
PR on release branch: https://github.com/pytorch/vision/pull/6150
Category: docs

Update _pil_constants.py to fix PIL SIMD
PR on main: https://github.com/pytorch/vision/pull/6154
PR on release branch: https://github.com/pytorch/vision/pull/6156
Category: bug fix

Fix minor typo in optical flow gallery example
Category: docs
main: https://github.com/pytorch/vision/pull/6163
release branch: https://github.com/pytorch/vision/pull/6164

Add new .. betastatus:: directive to document Beta APIs
Category: docs
main: https://github.com/pytorch/vision/pull/6115
release branch: https://github.com/pytorch/vision/pull/6165

Adding missing _log_api_usage_once to Swin's reusable components
Category: minor bugfix
main: #6174
release branch: #6175
@atalman this cherrypick comes after the final RC cut and it's not a critical bugfix, rather an omission on the logging side. I'll leave it up to you if it can be merged and included in the release.

I think we're done with this one, thanks everyone for your help! Closing now.


Create quick steps document for new tab in chatbot

[ ] Create word document - Quick steps instruction of the new chatbot feature (Ask GPT-4").


pex commands not working in game

ok so i've been researching for most of the day now but i have not been able to find a solution to my problem. first i'll give what details i know.
The server service i use is mcpro hosting, is compatible with sponge and loads pex without errors.
using mc version 1.8.9 with the latest version of forge and sponge
when i log into the game and try to use the pex commands, it says i do not have permission to use that command. when i go into the server console to use the commands it says it works fine for example i put in " pex group GM def true "  and i get the lind that i set the default permission to 1 for group GM in global context. then i add my username into the group with " pex user Naeren parent add group GM " and i get the response Added parent group GM for user Naeren in Global context. however when i am in game, i still cannot use any of the pex commands for editing or creating new groups, and my permissions dont seem to update at all as i dont get access to the other commands i should have in that case. i have tried numerous different suggestions posted in the replies to the main thread and the issues page, but i have had no luck. i would greatly appreciate it if someone could help me out with this.

Run /pex user Naeren delete, and then recreate the user using your UUID -- currently the username is what the account is stored by, rather than the UUID that PEX and MC use internally.


In 8.0.0-preview.3.24105.21 project links to non-Aspire projects are now ignored for build

In 8.0.0-preview.3.24105.21 and VS Version 17.9.0 when I add a link to a project library for supporting services it is ignored by the build system.
Steps to Reproduce:

Create an Aspire app using the standard VS template
Create a support library project and add any class to it
Reference that project in the Aspire apphost
Attempt to use the namespace for that support class from the external library

The build process refuses to accept those references when building (saying the namespace does not exist) while this is set:
 <PropertyGroup>
   <OutputType>Exe</OutputType>
   <TargetFramework>net8.0</TargetFramework>
   <ImplicitUsings>enable</ImplicitUsings>
   <Nullable>enable</Nullable>
   **<IsAspireHost>true</IsAspireHost>**
 </PropertyGroup>

If you remove the IsAspireHost setting though it will see the reference/namespace and build successfully.
Why is this important?

If you are using EF core and need to migrate the db created in the AppHost before the services come up you can't reference it.
You can't create any supporting libraries for Aspire to handle your internal processes

In 8.0.0-preview.3.24105.21 and VS Version 17.9.0 when I add a link to a project library for supporting services it is ignored by the build system.
Steps to Reproduce:

Create an Aspire app using the standard VS template
Create a support library project and add any class to it
Reference that project in the Aspire apphost
Attempt to use the namespace for that support class from the external library

The build process refuses to accept those references when building (saying the namespace does not exist) while this is set:
 <PropertyGroup>
   <OutputType>Exe</OutputType>
   <TargetFramework>net8.0</TargetFramework>
   <ImplicitUsings>enable</ImplicitUsings>
   <Nullable>enable</Nullable>
   **<IsAspireHost>true</IsAspireHost>**
 </PropertyGroup>

If you remove the IsAspireHost setting though it will see the reference/namespace and build successfully.
Why is this important?

If you are using EF core and need to migrate the db created in the AppHost before the services come up you can't reference it.
You can't create any supporting libraries for Aspire to handle your internal processes

So if I understand correctly you're referencing a library?
Can you try appending IsAspireProjectResource="false" ?
 <ProjectReference Include="MyLib.csproj" IsAspireProjectResource="false" />

That works! :) I should have thought of that. Thank you.
Is this going to be a long term way of dealing with that, or is this a short term fix?

That works! :) I should have thought of that. Thank you. Is this going to be a long term way of dealing with that, or is this a short term fix?

I don't know I'm not a contributor
#1726 and https://github.com/dotnet/aspire/pull/1726#discussion_r1465440286
Maybe help here

In 8.0.0-preview.3.24105.21 and VS Version 17.9.0 when I add a link to a project library for supporting services it is ignored by the build system.
Steps to Reproduce:

Create an Aspire app using the standard VS template
Create a support library project and add any class to it
Reference that project in the Aspire apphost
Attempt to use the namespace for that support class from the external library

The build process refuses to accept those references when building (saying the namespace does not exist) while this is set:
 <PropertyGroup>
   <OutputType>Exe</OutputType>
   <TargetFramework>net8.0</TargetFramework>
   <ImplicitUsings>enable</ImplicitUsings>
   <Nullable>enable</Nullable>
   **<IsAspireHost>true</IsAspireHost>**
 </PropertyGroup>

If you remove the IsAspireHost setting though it will see the reference/namespace and build successfully.
Why is this important?

If you are using EF core and need to migrate the db created in the AppHost before the services come up you can't reference it.
You can't create any supporting libraries for Aspire to handle your internal processes

Btw I on the other hand don't seem to can resolve project references. All Microsoft.NET.Sdk.Web projects are red to me on v1 with latest tooling VS 17.10 and 8.0.200
The Microsoft.NET.Sdk projects are fine

In 8.0.0-preview.3.24105.21 and VS Version 17.9.0 when I add a link to a project library for supporting services it is ignored by the build system.
Steps to Reproduce:

Create an Aspire app using the standard VS template
Create a support library project and add any class to it
Reference that project in the Aspire apphost
Attempt to use the namespace for that support class from the external library

The build process refuses to accept those references when building (saying the namespace does not exist) while this is set:
 <PropertyGroup>
   <OutputType>Exe</OutputType>
   <TargetFramework>net8.0</TargetFramework>
   <ImplicitUsings>enable</ImplicitUsings>
   <Nullable>enable</Nullable>
   **<IsAspireHost>true</IsAspireHost>**
 </PropertyGroup>

If you remove the IsAspireHost setting though it will see the reference/namespace and build successfully.
Why is this important?

If you are using EF core and need to migrate the db created in the AppHost before the services come up you can't reference it.
You can't create any supporting libraries for Aspire to handle your internal processes

Btw I on the other hand don't seem to can resolve project references. All Microsoft.NET.Sdk.Web projects are red to me on v1 with latest tooling VS 17.10 and 8.0.200
The Microsoft.NET.Sdk projects are fine
Got ideas?

You may have a similar issue I just fixed. I updated the Aspire workload but I also needed to update the nuget packages too. Once I did the Projects namespace fixed itself.

In 8.0.0-preview.3.24105.21 and VS Version 17.9.0 when I add a link to a project library for supporting services it is ignored by the build system.
Steps to Reproduce:

Create an Aspire app using the standard VS template
Create a support library project and add any class to it
Reference that project in the Aspire apphost
Attempt to use the namespace for that support class from the external library

The build process refuses to accept those references when building (saying the namespace does not exist) while this is set:
 <PropertyGroup>
   <OutputType>Exe</OutputType>
   <TargetFramework>net8.0</TargetFramework>
   <ImplicitUsings>enable</ImplicitUsings>
   <Nullable>enable</Nullable>
   **<IsAspireHost>true</IsAspireHost>**
 </PropertyGroup>

If you remove the IsAspireHost setting though it will see the reference/namespace and build successfully.
Why is this important?

If you are using EF core and need to migrate the db created in the AppHost before the services come up you can't reference it.
You can't create any supporting libraries for Aspire to handle your internal processes

Btw I on the other hand don't seem to can resolve project references. All Microsoft.NET.Sdk.Web projects are red to me on v1 with latest tooling VS 17.10 and 8.0.200
The Microsoft.NET.Sdk projects are fine
Got ideas?

You may have a similar issue I just fixed. I updated the Aspire workload but I also needed to update the nuget packages too. Once I did the Projects namespace fixed itself.

I don't want to because there are other issues linked with v3 for me.
https://github.com/dotnet/aspire/issues/2226
My services take up ports on their own. Aspire creates proxy with same port and that won't work.
I'd like to use v1 the same way as before.
However, new version of VS / Workload / Dotnet break v1 for me (Sdk.Web projects can't be found)

This may be the same issue as in https://github.com/dotnet/aspire/issues/2229 .

Can you try appending IsAspireProjectResource="false" ?
Is this going to be a long term way of dealing with that, or is this a short term fix?

That is the current plan of record for enabling this. There are a lot of discussion here at:

https://github.com/dotnet/aspire/issues/1074
https://github.com/dotnet/aspire/issues/1563

We tried a few different approaches, but the current behavior is what we landed on. MSBuild and NuGet have some issues that we can't tell if you are referencing a library or an exe early enough to be able to default this value at the right time.
Closing as I believe this issue is now addressed with setting IsAspireProjectResource="false".


Add asv benchmark jobs to CI

I found 3 alternatives:

Benchmark - Triggered using the run-benchmark label. Official version from scikit-image.
Benchmark cron - Triggered per push. Tests pre-PR to scikit-image I imagine.
Benchmark PR - Triggered per push. Tests pre-PR to scikit-image I imagine.

If someone has some deeper understanding how this works. Feel free to stitch together a version that fits xarray.
Notes:

https://github.com/scikit-image/scikit-image doesn't have the same benchmark folder setup, for example config file is in root directory, other folder names.
https://github.com/numpy/numpy has same folder name as sckit-image. config file is in the folder however.

References:

https://labs.quansight.org/blog/2021/08/github-actions-benchmarks/
https://github.com/scikit-image/scikit-image/pull/5424
https://github.com/jaimergp/scikit-image/pull/1

Good commits to look at:

9f79f15
5630aa2

[x] Related to #4648
[x] Tests added
[x] Passes pre-commit run --all-files
[ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
[ ] New functions/methods are listed in api.rst

Awesome!! Thanks a lot @Illviljan !
On how we run these — I would be fine with the labeled approach that scikit-learn uses.
What else do you think is needed for this? It already looks excellent, thank you.

Here's some things @max-sixty:

Long times, +240 minutes. scikit had it down to like 13min somehow. Which tests are super slow?
A lot of printed errors which makes the report very messy, go through each test and fix those. Not necessary for this PR though.
asv.conf.json has been moved. If you have any idea how to trigger this job inside asv_bench that would be nice.
Should we align benchmark folder names? Numpy?

Ah one major problem IIUC is that we run a lot of benchmarks with and without bottleneck even if bottleneck isn't involved in the operation being benchmarked.

Some of the following should be fixed

ImportError: Pandas requires version '0.12.3' or newer of 'xarray' (version '0.0.0' currently installed).

rolling.Rolling.time_rolling_construct
xarray.core.merge.MergeError: conflicting values for variable 'x_coords' on objects to be combined. You can skip this check by specifying compat='override'.

IOWriteNetCDFDaskDistributed.time_write
Looks like we're spinning up multiple dask clusters.
UserWarning: Port 8787 is already in use.
Perhaps you already have a cluster running?
Hosting the HTTP server on port 33423 instead
warnings.warn(

Do you mean skipping those that require bottleneck rather than relying on asv to handle the crash? If so, sounds good

Hmm, I was just thinking installing all possible dependencies. But I think I've simply misunderstood the tests and why they were crashing.
The Benchmark no label triggered seems to have succeded but still failed? Anyone understands what the error means?
 [100.00%] ··· unstacking.UnstackingDask.time_unstack_slow             32.7±0.3ms

BENCHMARKS NOT SIGNIFICANTLY CHANGED.
+ grep 'Traceback \|failed\|PERFORMANCE DECREASED' benchmarks.log
+ exit 1
++ '[' 1 = 1 ']'
++ '[' -x /usr/bin/clear_console ']'
++ /usr/bin/clear_console -q
Error: Process completed with exit code 1.

@dcherian did you have something specific in mind when you linked to https://github.com/jaimergp/scikit-image/blob/main/.github/workflows/benchmarks-cron.yml in #4648? I think I'll remove it otherwise and just focus on

Benchmark label triggered - from scikit
Benchmark no label triggered - from scikit modded by me.

There we go. I think the workflow works as intended now we'll see in 3-4 hours.
Only thing left is to improve the tests and maybe rewrite that README slightly.

Hi @Illviljan — we'd be interesting in seeing whether you wanted to join the core developers group — but we don't have a way of contacting you! So a comment on your PR was the best I could think of. If you'd be interested in discussing this, drop me an email at m at maximilian roos dot com . Thanks!

Hello @Illviljan! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file asv_bench/benchmarks/pandas.py:

Line 4:1: F401 'xarray as xr' imported but unused

Down to 101 minutes now. :)
I thinks it's mainly from the tests that I haven't checked yet. If someone wants to take a stab at reducing the times and removing errors for dataset_io.py and indexing.py.

@dcherian did you have something specific in mind when you linked to https://github.com/jaimergp/scikit-image/blob/main/.github/workflows/benchmarks-cron.yml in #4648? I think I'll remove it otherwise and just focus on

Nothing specific. Just wanted to link their configuration.

Down to 30 mins now. dataset_io is being skipped however.
I think this is finished and is ready for review.

Getting a weird error now.

Run set -x
+ asv machine --yes
· No information stored about machine 'fv-az231-522'. I know about nothing.

I will now ask you some questions about this machine to identify it in the benchmarks.

1. machine:  A unique name to identify this machine in the results.
   May be anything, as long as it is unique across all the machines
   used to benchmark this project.  NOTE: If changed from the default,
   it will no longer match the hostname of this machine, and you may
   need to explicitly use the --machine argument to asv.
machine [fv-az231-522]: 2. os:  The OS type and version of this machine.  For example,
   'Macintosh OS-X 10.8'.
os [Linux 5.8.0-1042-azure]: 3. arch:  The generic CPU architecture of this machine.  For example,
   'i386' or 'x86_64'.
arch [x86_64]: 4. cpu:  A specific description of the CPU of this machine, including
   its speed and class.  For example, 'Intel(R) Core(TM) i5-2520M CPU
   @ 2.50GHz (4 cores)'.
cpu [Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz]: 5. num_cpu:  The number of CPUs in the system. For example, '4'.
num_cpu [2]: 6. ram:  The amount of physical RAM on this machine.  For example,
   '4GB'.
+ echo 'Baseline:  ebfc6a3db0580cc11418e906766805ff4bf36455 (Illviljan:main)'
ram [7120800]: Baseline:  ebfc6a3db0580cc11418e906766805ff4bf36455 (Illviljan:main)
+ echo 'Contender: 7af19db79493412bddc6ad9db768ce05c75d7297 (Illviljan:asv-benchmark-cron)'
++ which mamba
Contender: 7af19db79493412bddc6ad9db768ce05c75d7297 (Illviljan:asv-benchmark-cron)
+ export CONDA_EXE=/usr/share/miniconda3/condabin/mamba
+ CONDA_EXE=/usr/share/miniconda3/condabin/mamba
+ ASV_OPTIONS='--split --show-stderr --factor 1.5'
+ tee benchmarks.log
+ sed '/Traceback \|failed$\|PERFORMANCE DECREASED/ s/^/::error::/'
+ asv continuous --split --show-stderr --factor 1.5 ebfc6a3db0580cc11418e906766805ff4bf36455 7af19db79493412bddc6ad9db768ce05c75d7297
· Creating environments
· Discovering benchmarks
·· Uninstalling from conda-py3.8-bottleneck-dask-distributed-netcdf4-numpy-pandas-scipy
·· Building 7af19db7 for conda-py3.8-bottleneck-dask-distributed-netcdf4-numpy-pandas-scipy
·· Installing 7af19db7 into conda-py3.8-bottleneck-dask-distributed-netcdf4-numpy-pandas-scipy
·· Error running /home/runner/work/xarray/xarray/asv_bench/.asv/env/06e0f5dba81d6db545c0b3d92fe94a49/bin/python /home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py discover /home/runner/work/xarray/xarray/asv_bench/benchmarks /tmp/tmp3sm5q3yv/result.json (exit status 1)
   STDOUT -------->

   STDERR -------->
Error:    Traceback (most recent call last):
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 1315, in <module>
       main()
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 1308, in main
       commands[mode](args)
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 1004, in main_discover
       list_benchmarks(benchmark_dir, fp)
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 989, in list_benchmarks
       for benchmark in disc_benchmarks(root):
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 887, in disc_benchmarks
       for module in disc_modules(root_name, ignore_import_errors=ignore_import_errors):
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 869, in disc_modules
       for item in disc_modules(name, ignore_import_errors=ignore_import_errors):
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 857, in disc_modules
       module = import_module(module_name)
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/06e0f5dba81d6db545c0b3d92fe94a49/lib/python3.8/importlib/__init__.py", line 127, in import_module
       return _bootstrap._gcd_import(name[level:], package, level)
     File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
     File "<frozen importlib._bootstrap>", line 991, in _find_and_load
     File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
     File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
     File "<frozen importlib._bootstrap_external>", line 843, in exec_module
     File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
     File "/home/runner/work/xarray/xarray/asv_bench/benchmarks/combine.py", line 3, in <module>
       import xarray as xr
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/06e0f5dba81d6db545c0b3d92fe94a49/lib/python3.8/site-packages/xarray/__init__.py", line 3, in <module>
       from . import testing, tutorial, ufuncs
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/06e0f5dba81d6db545c0b3d92fe94a49/lib/python3.8/site-packages/xarray/tutorial.py", line 13, in <module>
       from .backends.api import open_dataset as _open_dataset
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/06e0f5dba81d6db545c0b3d92fe94a49/lib/python3.8/site-packages/xarray/backends/__init__.py", line 9, in <module>
       from .h5netcdf_ import H5NetCDFStore
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/06e0f5dba81d6db545c0b3d92fe94a49/lib/python3.8/site-packages/xarray/backends/h5netcdf_.py", line 25, in <module>
       from .netCDF4_ import (
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/06e0f5dba81d6db545c0b3d92fe94a49/lib/python3.8/site-packages/xarray/backends/netCDF4_.py", line 34, in <module>
       import netCDF4
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/06e0f5dba81d6db545c0b3d92fe94a49/lib/python3.8/site-packages/netCDF4/__init__.py", line 3, in <module>
       from ._netCDF4 import *
   ImportError: /home/runner/work/xarray/xarray/asv_bench/.asv/env/06e0f5dba81d6db545c0b3d92fe94a49/lib/python3.8/site-packages/netCDF4/../../../libnetcdf.so.19: undefined symbol: H5Pset_fapl_ros3

·· Failed: trying different commit/environment
·· Uninstalling from conda-py3.8-bottleneck-dask-distributed-netcdf4-numpy-pandas-scipy
·· Building ebfc6a3d for conda-py3.8-bottleneck-dask-distributed-netcdf4-numpy-pandas-scipy
·· Installing ebfc6a3d into conda-py3.8-bottleneck-dask-distributed-netcdf4-numpy-pandas-scipy
·· Error running /home/runner/work/xarray/xarray/asv_bench/.asv/env/06e0f5dba81d6db545c0b3d92fe94a49/bin/python /home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py discover /home/runner/work/xarray/xarray/asv_bench/benchmarks /tmp/tmpsjxq903z/result.json (exit status 1)
   STDOUT -------->

   STDERR -------->
Error:    Traceback (most recent call last):
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 1315, in <module>
       main()
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 1308, in main
       commands[mode](args)
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 1004, in main_discover
       list_benchmarks(benchmark_dir, fp)
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 989, in list_benchmarks
       for benchmark in disc_benchmarks(root):
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 887, in disc_benchmarks
       for module in disc_modules(root_name, ignore_import_errors=ignore_import_errors):
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 869, in disc_modules
       for item in disc_modules(name, ignore_import_errors=ignore_import_errors):
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 857, in disc_modules
       module = import_module(module_name)
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/06e0f5dba81d6db545c0b3d92fe94a49/lib/python3.8/importlib/__init__.py", line 127, in import_module
       return _bootstrap._gcd_import(name[level:], package, level)
     File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
     File "<frozen importlib._bootstrap>", line 991, in _find_and_load
     File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
     File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
     File "<frozen importlib._bootstrap_external>", line 843, in exec_module
     File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
     File "/home/runner/work/xarray/xarray/asv_bench/benchmarks/combine.py", line 3, in <module>
       import xarray as xr
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/06e0f5dba81d6db545c0b3d92fe94a49/lib/python3.8/site-packages/xarray/__init__.py", line 3, in <module>
       from . import testing, tutorial, ufuncs
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/06e0f5dba81d6db545c0b3d92fe94a49/lib/python3.8/site-packages/xarray/tutorial.py", line 13, in <module>
       from .backends.api import open_dataset as _open_dataset
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/06e0f5dba81d6db545c0b3d92fe94a49/lib/python3.8/site-packages/xarray/backends/__init__.py", line 9, in <module>
       from .h5netcdf_ import H5NetCDFStore
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/06e0f5dba81d6db545c0b3d92fe94a49/lib/python3.8/site-packages/xarray/backends/h5netcdf_.py", line 25, in <module>
       from .netCDF4_ import (
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/06e0f5dba81d6db545c0b3d92fe94a49/lib/python3.8/site-packages/xarray/backends/netCDF4_.py", line 34, in <module>
       import netCDF4
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/06e0f5dba81d6db545c0b3d92fe94a49/lib/python3.8/site-packages/netCDF4/__init__.py", line 3, in <module>
       from ._netCDF4 import *
   ImportError: /home/runner/work/xarray/xarray/asv_bench/.asv/env/06e0f5dba81d6db545c0b3d92fe94a49/lib/python3.8/site-packages/netCDF4/../../../libnetcdf.so.19: undefined symbol: H5Pset_fapl_ros3

·· Failed: trying different commit/environment
·· Uninstalling from conda-py3.8-dask-distributed-netcdf4-numpy-pandas-scipy
·· Building 7af19db7 for conda-py3.8-dask-distributed-netcdf4-numpy-pandas-scipy
·· Installing 7af19db7 into conda-py3.8-dask-distributed-netcdf4-numpy-pandas-scipy
·· Error running /home/runner/work/xarray/xarray/asv_bench/.asv/env/3aabcb0950276df5ae33a267529abdce/bin/python /home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py discover /home/runner/work/xarray/xarray/asv_bench/benchmarks /tmp/tmpnxh5svbe/result.json (exit status 1)
   STDOUT -------->

   STDERR -------->
Error:    Traceback (most recent call last):
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 1315, in <module>
       main()
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 1308, in main
       commands[mode](args)
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 1004, in main_discover
       list_benchmarks(benchmark_dir, fp)
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 989, in list_benchmarks
       for benchmark in disc_benchmarks(root):
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 887, in disc_benchmarks
       for module in disc_modules(root_name, ignore_import_errors=ignore_import_errors):
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 869, in disc_modules
       for item in disc_modules(name, ignore_import_errors=ignore_import_errors):
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 857, in disc_modules
       module = import_module(module_name)
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/3aabcb0950276df5ae33a267529abdce/lib/python3.8/importlib/__init__.py", line 127, in import_module
       return _bootstrap._gcd_import(name[level:], package, level)
     File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
     File "<frozen importlib._bootstrap>", line 991, in _find_and_load
     File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
     File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
     File "<frozen importlib._bootstrap_external>", line 843, in exec_module
     File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
     File "/home/runner/work/xarray/xarray/asv_bench/benchmarks/combine.py", line 3, in <module>
       import xarray as xr
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/3aabcb0950276df5ae33a267529abdce/lib/python3.8/site-packages/xarray/__init__.py", line 3, in <module>
       from . import testing, tutorial, ufuncs
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/3aabcb0950276df5ae33a267529abdce/lib/python3.8/site-packages/xarray/tutorial.py", line 13, in <module>
       from .backends.api import open_dataset as _open_dataset
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/3aabcb0950276df5ae33a267529abdce/lib/python3.8/site-packages/xarray/backends/__init__.py", line 9, in <module>
       from .h5netcdf_ import H5NetCDFStore
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/3aabcb0950276df5ae33a267529abdce/lib/python3.8/site-packages/xarray/backends/h5netcdf_.py", line 25, in <module>
       from .netCDF4_ import (
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/3aabcb0950276df5ae33a267529abdce/lib/python3.8/site-packages/xarray/backends/netCDF4_.py", line 34, in <module>
       import netCDF4
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/3aabcb0950276df5ae33a267529abdce/lib/python3.8/site-packages/netCDF4/__init__.py", line 3, in <module>
       from ._netCDF4 import *
   ImportError: /home/runner/work/xarray/xarray/asv_bench/.asv/env/3aabcb0950276df5ae33a267529abdce/lib/python3.8/site-packages/netCDF4/../../../libnetcdf.so.19: undefined symbol: H5Pset_fapl_ros3

·· Failed: trying different commit/environment
·· Uninstalling from conda-py3.8-dask-distributed-netcdf4-numpy-pandas-scipy
·· Building ebfc6a3d for conda-py3.8-dask-distributed-netcdf4-numpy-pandas-scipy
·· Installing ebfc6a3d into conda-py3.8-dask-distributed-netcdf4-numpy-pandas-scipy
·· Error running /home/runner/work/xarray/xarray/asv_bench/.asv/env/3aabcb0950276df5ae33a267529abdce/bin/python /home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py discover /home/runner/work/xarray/xarray/asv_bench/benchmarks /tmp/tmp_8pq_qyl/result.json (exit status 1)
   STDOUT -------->

   STDERR -------->
Error:    Traceback (most recent call last):
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 1315, in <module>
       main()
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 1308, in main
       commands[mode](args)
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 1004, in main_discover
       list_benchmarks(benchmark_dir, fp)
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 989, in list_benchmarks
       for benchmark in disc_benchmarks(root):
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 887, in disc_benchmarks
       for module in disc_modules(root_name, ignore_import_errors=ignore_import_errors):
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 869, in disc_modules
       for item in disc_modules(name, ignore_import_errors=ignore_import_errors):
     File "/home/runner/.local/lib/python3.8/site-packages/asv/benchmark.py", line 857, in disc_modules
       module = import_module(module_name)
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/3aabcb0950276df5ae33a267529abdce/lib/python3.8/importlib/__init__.py", line 127, in import_module
       return _bootstrap._gcd_import(name[level:], package, level)
     File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
     File "<frozen importlib._bootstrap>", line 991, in _find_and_load
     File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
     File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
     File "<frozen importlib._bootstrap_external>", line 843, in exec_module
     File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
     File "/home/runner/work/xarray/xarray/asv_bench/benchmarks/combine.py", line 3, in <module>
       import xarray as xr
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/3aabcb0950276df5ae33a267529abdce/lib/python3.8/site-packages/xarray/__init__.py", line 3, in <module>
       from . import testing, tutorial, ufuncs
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/3aabcb0950276df5ae33a267529abdce/lib/python3.8/site-packages/xarray/tutorial.py", line 13, in <module>
       from .backends.api import open_dataset as _open_dataset
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/3aabcb0950276df5ae33a267529abdce/lib/python3.8/site-packages/xarray/backends/__init__.py", line 9, in <module>
       from .h5netcdf_ import H5NetCDFStore
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/3aabcb0950276df5ae33a267529abdce/lib/python3.8/site-packages/xarray/backends/h5netcdf_.py", line 25, in <module>
       from .netCDF4_ import (
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/3aabcb0950276df5ae33a267529abdce/lib/python3.8/site-packages/xarray/backends/netCDF4_.py", line 34, in <module>
       import netCDF4
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/3aabcb0950276df5ae33a267529abdce/lib/python3.8/site-packages/netCDF4/__init__.py", line 3, in <module>
       from ._netCDF4 import *
   ImportError: /home/runner/work/xarray/xarray/asv_bench/.asv/env/3aabcb0950276df5ae33a267529abdce/lib/python3.8/site-packages/netCDF4/../../../libnetcdf.so.19: undefined symbol: H5Pset_fapl_ros3

·· Failed to build the project and import the benchmark suite.

Here's how long dataarray_missing.py takes in this workflow with different shapes:
321a761 - shape=(100, 25, 25), 3 minutes
8f262f9 - shape=(365, 50, 50), 3m 28s
0b7b1a0 - shape=(365, 75, 75), 4m 8s
8f08506 - shape=(365, 100, 100), 5m 47s
d1b908a - shape=(365, 200, 400) , 12m 38s
56556f1 - shape=(3650, 100, 100) 19m 55s and crashes
1eba65c - shape=(3650, 200, 400), 20 minutes and crashes


Add the possibility to define several processes on an action

On complex workflows, we may have the need to define 2 (or more) processes on an action but, with Tradista Flow 4.0.0, we can only define one process on a given action.
It would be very useful to be able to define several processes on a given action. Moreover, we should be able to define the order in which processes are executed.
A workaround could have been to group several processes in a single process but if we do that, we lose the possibility to configure individually one of these processes elsewhere in the workflow (or in another workflow).

This ticket will be included in v5.0.0


Not working on Browser

After some debugging it seems that the following line in src\providers\image-loader.ts is never called on the Browser
Observable.fromEvent(document, 'deviceready').first().subscribe(function (res) {

Browser as in Cordova Browser or ionic serve?

ionic serve

Hmm, good call. I think we need to go back to platform.ready(). Or maybe add something like:
if (!platform.is('cordova')) {
  this.init = true;
} else {
  Observable.fromEvent(....);
}

That seems more in line of what Ionic does, let me try to submit a PR

Just released 4.0.1 with your fix included.

Please Reopen the Issue i get the Error Message with v4.0.1
Uncaught (in promise): cordova_not_available

Sorry I just realized this whole thing is because of #61 ... I didn't review the PR properly. Reverting now.

Thank you for the fast Help 👍


Back slashes added?

Any ideas on what could be causing this? I'm not sure it's this plugin, to be honest.

 postcss: function () {
      return [
        require('postcss-import')({
          addDependencyTo: webpack,
        }),
        customProperties({
          preserve: true,
        }),
        cssApply,
        autoprefixer,
      ]
    },

I even removed :root and still the back slashes were added.

There seems to be some issue with colons as well. When I don't add them to the "mixins" declared under :root, the rules below :root are not eval'd.

Hi,
are you sure the @apply plugin is executed actually?
Because property sets declarations are supposed to be removed from final output.
Or most probably it's because the slash is added to the declaration before the plugin runs, which breaks its "detection", so nothing happens.
I would advise to always add colons at the end of the declaration.
I will add more tests for this.
Can you set-up a small repo to be able to reproduce/debug?

The plugin is running, because the styles are being applied. Let me see if I can set up a small test repo.

@pascalduez Here you go: https://cloudup.com/cLsnx9RtW82 See README for repro steps.

The plugin is running, because the styles are being applied.

Odd. I noticed that they're being applied in Chrome, but not Safari.

Yeah, something's up for sure, because changing the buttons.css file doesn't change anything in Chrome, even though Webpack is "rebuilding".

Maybe it's this?
https://github.com/postcss/postcss-custom-properties/issues/48

@kevinSuttle Is this issue still relevant?

I tried fiddling with the provided repo for an hour or so, I could not get any progress.
Looks like the postcss plugins are not even applied/called.
By default storybook will use its own webpack config, so no surprise the plugins aren't used.
I added a custom webpack config for storybook but same result.
Next step would be to try without storybook, just "plain" webpack and an index.html.
I'm using cssnext on a projet with webpack  n' stuff and have no issue BTW.

@pascalduez I haven't seen it reproduced. Thanks for checking into it. I appreciate you taking the time.

Seeing this again now.
I'm now using:
https://github.com/sebastian-software/postcss-smart-import
https://github.com/postcss/postcss-custom-properties/
:root {
  \--ThemeButtonStyles: {
    @apply --ThemeAnimationStyles;
...
}

postcss: function() {
      return [
        atImport({
           addDependencyTo: webpack,
           path: './src/Index.css',
           plugins: [
              customProperties({preserve: true}),
              cssApply,
              autoprefixer
          ]
        }),
      ]
    },

This is concerning enough, even if Webpack 2 is still in beta. Re-opening.

Let me elaborate on the structure:
./Index.css
@import "./Themes/Vars.css";
@import "./Themes/Colors/Kratos.css";
@import "./Themes/Colors/Prometheus.css";
@import "./Themes/Mixins.css";

WARNING in ./~/css-loader!./~/postcss-loader!./src/Index.css
postcss-custom-properties: /Users/kevinSuttle/Code/platform-ui-primitives/src/Themes/Mixins.css:15:5: variable '--PrimaryThemeColor' is undefined and used without a fallback

[repeats]
This is inaccurate. --PrimaryThemeColor is defined in the 2 color files listed before Mixins.css.
BUT, something is adding these slashes before they can get there.

Confirmed. Removing that :root declaration in Button/Buttons.css removed that \, but the warnings still appear. Separate issue it seems.

Hm. The top-level styles are getting added last, which some of it, but not all.

I did this in my top-level npm module export:
require("./Themes/Vars.css");
require("./Themes/Colors/Kratos.css");
require("./Themes/Colors/Prometheus.css");
require("./Themes/Mixins.css");

And got the slashes back on top-level :root declarations.
I removed the smart-import plugin and the slashes went away.
No warnings, no errors, no slashes.

Hi , I got the same issue with the plugin, it was adding backslashes when I try to import  custom property sets from an external file.
I aim to keep all of my sets in separate file as my project is  build with multiple CSS files (files per component).
I succeed to import my file and use custom property sets by using postcss-apply in conjonction of postcss-import.
There's 2 downside.

I have to import the file into my current file anytime I want to use a custom property set.
A waring is triggered saying "No custom properties set declared " , even though  the styles are actually being applied.

I there a better way to achieve this, without those two downside ?
Thx
part of my webpack config :
 var pcssNesting       = require('postcss-nesting') var pcssNext          = require('postcss-cssnext') var cssApply          = require('postcss-apply') var pcssCustomProps   = require('postcss-custom-properties') var pcssReporter      = require('postcss-reporter') var atImport          = require("postcss-import")
    postcss: [       pcssCustomProps({preserve: 'computed', strict: false}),       pcssNext,       atImport(),       cssApply,       pcssNesting,       pcssReporter({clearMessages: true})     ]

@Memoyr Just a quick remark on what you posted above, postcss-custom-properties and postcss-apply are already included within postcss-cssnext. So apparently you're running them twice.
Also the plugins order is important, postcss-import should come first.

I have to import the file into my current file anytime I want to use a custom property set.

That's how postcss-import and most preprocessors works.
Only alt solution I can see for now is waiting for #15.

Same is happening here. apply can't do its job because something is adding a \ before the variables. My best guess is that it's css-loader doing it, but I can't understand why.
For now to get unblocked I've created this postcss plugin to clean the extra backslash.

@ericsaboia Basically postcss-loader should comes before the css-loader so the apply plugin should already have done it's job.
I once tried to debug, but the thing is quite dense, there's so many actors involed...
Re-opening anyway?

thanks for looking into this, even though it doesn't seem to be an issue caused by postcss-apply, @pascalduez.
I'm executing postcss-apply as a plugin using extract-text-webpack-plugin + optimize-css-assets-webpack-plugin, so it's only called after all the loaders are executed.
When I execute it inside the loaders it works well, meaning the \ isn't there yet. That's why I believe it might be happening inside css-loader.
I can set up a minimal reproducible environment to help debug this.

Also hitting this issue, and also because of css-loader. If I switch to the simple string-loader it works (although imports break), so it seems css-loader does some kind of validation while loading the CSS, and automatically comments invalid CSS out. I can't find a way to turn this off though.

Okay, I spotted the culprit. This happens when used together with CSS modules.
It's css-selector-tokenizer stringify function.

@pascalduez no way around this then?

@yebrahim One solution, MR that lib to prevent the escaping. But since custom property sets are not going anywhere, it might be more difficult to get the change accepted, dunno. I'll try, but feel free to.
Currently custom property sets  don't have any browser support and will most likely won't get any.
So it does not make much sense to use the preserve option, so they basically won't get escaped (problem solved).
That might be mitigated by the order the plugin are run though.


cask tty removed

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[x] Have you successfully run brew tests with your changes locally?

Removed cask's tty. (#777)
HB's tty does not support double arguments, fixed them as I could, feel free to correct.
cc @MikeMcQuaid @jawshooah @vitorgalvao @reitermarkus

LGTM other than @reitermarkus's comments.

The only thing I don't understand is why it's called Tty.white, I'm pretty sure 39 is default, not white.

Confused me as well when I tried to replace bold

Probably the same reason why Tty.em isn't called Tty.underline: History and nobody going to the trouble of cleaning up this part of the code. 39 is indeed the default foreground color, which tends to be white (or some shade of it) when using a dark terminal background.

that tends to be white (or some shade of it) when using a dark terminal background.

Well, considering Terminal's default theme is black on white, this does make even less sense.

True, but I believe most maintainers and many seasoned command-line users prefer a dark background. I haven't been around long enough and haven't paid too much attention to defaults, but maybe the default Terminal theme also changed between Homebrew's creation and now?

Yes, I think the default changed from Mavericks to Yosemite.

I use a white background and have for a while and it looks fine here.

I'm happy with this as-is. @reitermarkus any final comments/objections to merging?

No, but let's not forget renaming Tty.white to something more fitting afterwards.


OSPP 22 opentelemetry addon

@yue9944882 are we ok to merge this first, this seems pretty big and won't be easy to review.

/lgtm
/approve