setup.py deprecation warning

Hi, when building Python packages in ROS 2 Galactic the following  warning is produced
--- stderr: examples_rclpy_pointcloud_publisher
/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
---
Finished <<< examples_rclpy_pointcloud_publisher [1.28s]

is there a plan to change the installation logic to get rid of it?
I'm not expert at all of python packaging, but would it be sufficient to replace the setup.py calls with python -m build ?

Looks like the Python build process should be moved over to something that follows PEP 517 (https://www.python.org/dev/peps/pep-0517/)
build is probably a good choice.

I'm testing some colcon extensions on Ubuntu Jammy 22.04, but noticed the same spammy deprecation notice:

```
Starting >>> colcon-ros
--- stderr: colcon-cd
/usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
---
```

Guessing we should stop directly executing setup.py files and be call a dedicated python build backend instead?
https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html

At our project we try to follow strategy "no warnings during builds/treat all warnings as errors". Is there a way to change our python packages structure/setup.py content/whatever, to remove those warnings? Do we need to change something in our packages or is this an issue only with colcon and we need to live with those warnings till a fix in colcon is implemented?

You could try and install the master branch of https://github.com/colcon/colcon-python-setup-py via pip and see if that solves the issue.

AFAIK I already have the latest version (0.2.7):
root@af77d7851fa2:/ws# pip3 install colcon-python-setup-py -U
Requirement already up-to-date: colcon-python-setup-py in /usr/lib/python3/dist-packages (0.2.7)
Requirement already satisfied, skipping upgrade: colcon-core>=0.6.1 in /usr/lib/python3/dist-packages (from colcon-python-setup-py) (0.8.2)
root@af77d7851fa2:/ws# colcon build --packages-up-to as_ros_license
Starting >>> as_ros_license
--- stderr: as_ros_license
/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
---
Finished <<< as_ros_license [2.51s]

Summary: 1 package finished [2.78s]
  1 package had stderr output: as_ros_license
root@af77d7851fa2:/ws#

This issue doesn't occur for setuptools 45.2.0 (which are rather old but still used in the latest official ros:galactic image), however it occurs for setuptools >= 60.

What's on pypi is behind the master branch.

Ok, so another try:
root@23e85568dac8:/ws# pip3 install -U git+https://github.com/colcon/colcon-python-setup-py.git@master
Collecting git+https://github.com/colcon/colcon-python-setup-py.git@master
  Cloning https://github.com/colcon/colcon-python-setup-py.git (to revision master) to /tmp/pip-req-build-mt1uugi5
  Running command git clone -q https://github.com/colcon/colcon-python-setup-py.git /tmp/pip-req-build-mt1uugi5
Requirement already satisfied, skipping upgrade: colcon-core>=0.6.1 in /usr/lib/python3/dist-packages (from colcon-python-setup-py==0.2.7) (0.8.2)
Requirement already satisfied, skipping upgrade: setuptools in /usr/local/lib/python3.8/dist-packages (from colcon-python-setup-py==0.2.7) (62.1.0)
Building wheels for collected packages: colcon-python-setup-py
  Building wheel for colcon-python-setup-py (setup.py) ... done
  Created wheel for colcon-python-setup-py: filename=colcon_python_setup_py-0.2.7-py3-none-any.whl size=11692 sha256=7757e6012eb1146ff7fc3af8077e94a73aba82f21fcf22854dcf7932e2bfc832
  Stored in directory: /tmp/pip-ephem-wheel-cache-dmm0bbmb/wheels/bc/46/d5/da5ce83806c2ab78193e6ee022996337a912af8447706ea89a
Successfully built colcon-python-setup-py
Installing collected packages: colcon-python-setup-py
  Attempting uninstall: colcon-python-setup-py
    Found existing installation: colcon-python-setup-py 0.2.7
    Uninstalling colcon-python-setup-py-0.2.7:
      Successfully uninstalled colcon-python-setup-py-0.2.7
Successfully installed colcon-python-setup-py-0.2.7
root@23e85568dac8:/ws# colcon build --packages-up-to as_ros_license
Starting >>> as_ros_license
--- stderr: as_ros_license
/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
---
Finished <<< as_ros_license [2.70s]

Summary: 1 package finished [3.03s]
  1 package had stderr output: as_ros_license
root@23e85568dac8:/ws#

I'm probably missing something.

I'm probably missing something.

No probably not, it was just a suggestion. Good to know it still isn't fixed.

IIUC, we can continue to use setuptools as the build backend though we should stop invoking the setup.py CLI (which is deprecated, hence the warning).
I suppose since setuptools supports PEP 517, we could have colcon make appropriate calls to the setuptools library, acting as a build frontend. I think this would require packages built by colcon to supply most of the project metadata in a setup.cfg, instead of the setup.py (reference).
However, it appears the Python community is moving towards using pyproject.toml files for describing package metadata (PEP 621). Also, I think a pyproject.toml may be necessary to leverage modern build front-ends like build, which may make colcon's job easier (e.g. if we replaced setup.py CLI calls with build's CLI). Switching to pyproject.toml files everywhere would probably cause a lot more churn than some other solution, I don't know. It might be worth noting setuptools supports pyproject.toml (instead of setupcfg), however it "experimental" (reference). At the very least, it'd probably be nice to have colcon plugin to support packages using a pyproject.toml. It looks like someone has worked on one to support poetry.
Whatever the ultimate solution is, it looks like it may involve updating downstream Python packages still relying on invoking the setup.py CLI. We can keep colcon's current implementation for building Python packages as fallback behavior if the downstream package is not updated to use a static description (e.g. setup.cfg or pyproject.toml).

In the meantime, users can explicitly quiet the deprecation warning if it bothers them with a filter. For example, by setting the following environment variable:
PYTHONWARNINGS=ignore:::setuptools.command.install

In addition, I get /usr/lib/python3/dist-packages/setuptools/command/easy_install.py:158: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools on ROS Humble @ Ubuntu 22.04. I can't find a place where we directly use easy_install though. It this also a problem with colcon?

I've noticed the EasyInstallDeprecationWarning deprecation warning, along with PkgResourcesDeprecationWarning, if you use colcon's --symlink-install option. If you want, you could also quiet those warnings by appending to the warnings filter, e.g.
PYTHONWARNINGS=ignore:::setuptools.command.install,ignore:::setuptools.command.easy_install,ignore:::pkg_resources

however, ultimately I think these warnings should all be resolved when colcon can build packages without a setup.py.

Okay, thanks for the confirmation! We'll silence them an wait for a patched colon. 😃

I believe @cottsay is working on this issue.

any news on this? :)

Quite a similar discussion is at ament_cmake repo:
https://github.com/ament/ament_cmake/issues/382

I tried to downgrad setuptools to 58.2.0 by
pip3 install setuptools==58.2.0
then colcon build --symlink-install works fine on all examples

I tried to downgrad setuptools to 58.2.0 by pip3 install setuptools==58.2.0 then colcon build --symlink-install works does not show error but ros2 test still has the following error
Finished <<< examples_rclpy_pointcloud_publisher [0.96s]
--- stderr: launch_testing_examples

=============================== warnings summary ===============================
launch_testing_examples/check_msgs_launch_test.py: 2 warnings
launch_testing_examples/check_multiple_nodes_launch_test.py: 2 warnings
launch_testing_examples/check_node_launch_test.py: 2 warnings
launch_testing_examples/hello_world_launch_test.py: 2 warnings
launch_testing_examples/record_rosbag_launch_test.py: 2 warnings
launch_testing_examples/set_param_launch_test.py: 2 warnings
  Warning: There is no current event loop

test/test_flake8.py::test_flake8
test/test_flake8.py::test_flake8
  Warning: SelectableGroups dict interface is deprecated. Use select.

-- Docs: https://docs.pytest.org/en/stable/warnings.html
---
Finished <<< launch_testing_examples [16.5s]

Summary: 22 packages finished [18.6s]
  10 packages had stderr output: examples_rclpy_executors examples_rclpy_guard_conditions examples_rclpy_minimal_action_client examples_rclpy_minimal_action_server examples_rclpy_minimal_client examples_rclpy_minimal_publisher examples_rclpy_minimal_service examples_rclpy_minimal_subscriber examples_rclpy_pointcloud_publisher launch_testing_examples

yeah i have the same issue with Humble with ubuntu 22.04.1 even after trying what you did

I tried to downgrad setuptools to 58.2.0 by pip3 install setuptools==58.2.0 then colcon build --symlink-install works does not show error but ros2 test still has the following error
Finished <<< examples_rclpy_pointcloud_publisher [0.96s]
--- stderr: launch_testing_examples

=============================== warnings summary ===============================
launch_testing_examples/check_msgs_launch_test.py: 2 warnings
launch_testing_examples/check_multiple_nodes_launch_test.py: 2 warnings
launch_testing_examples/check_node_launch_test.py: 2 warnings
launch_testing_examples/hello_world_launch_test.py: 2 warnings
launch_testing_examples/record_rosbag_launch_test.py: 2 warnings
launch_testing_examples/set_param_launch_test.py: 2 warnings
  Warning: There is no current event loop

test/test_flake8.py::test_flake8
test/test_flake8.py::test_flake8
  Warning: SelectableGroups dict interface is deprecated. Use select.

-- Docs: https://docs.pytest.org/en/stable/warnings.html
---
Finished <<< launch_testing_examples [16.5s]

Summary: 22 packages finished [18.6s]
  10 packages had stderr output: examples_rclpy_executors examples_rclpy_guard_conditions examples_rclpy_minimal_action_client examples_rclpy_minimal_action_server examples_rclpy_minimal_client examples_rclpy_minimal_publisher examples_rclpy_minimal_service examples_rclpy_minimal_subscriber examples_rclpy_pointcloud_publisher launch_testing_examples

yeah i have the same issue with Humble with ubuntu 22.04.1 even after trying what you did

This is an unrelated flake8 issue that is tracked in https://bugs.launchpad.net/ubuntu/+source/python-flake8/+bug/1968072, so it has nothing to do with this ticket.

We'll be discussing this topic at the Infrastructure Community Meeting on May 17: https://discourse.ros.org/t/infrastructure-community-meeting-2023-05-17/31268

I've issued a Call For Testing on a prototype package based on the PEPs discussed here: https://discourse.ros.org/t/call-for-testing-standards-based-python-packaging-with-colcon/32008
If you're interested in seeing these deprecation warnings disappear, please take a moment to provide feedback.

Whats the current state of this issue?

The prototype is available and working, but there are problems with symlink installs and general performance. PEP 660 with setuptools is currently ignoring data_files, which is the mechanism that ROS packages use to install things like package manifests and launch files. That alone is a blocker.
If you're interested in this landing sooner, please consider trying the prototype package and providing feedback.

The prototype is available and working, but there are problems with symlink installs and general performance. PEP 660 with setuptools is currently ignoring data_files, which is the mechanism that ROS packages use to install things like package manifests and launch files. That alone is a blocker.
If you're interested in this landing sooner, please consider trying the prototype package and providing feedback.

Is there any way to download this prototype via pip?

Is there any way to download this prototype via pip?

You can target the git branches directly using pip, but I don't think it's a good idea at all. Pip has a tendency to tangle up your system and it might make it difficult to revert back to the non-prototype packages.


v0 pdf rendering

Tasks to be done:

cleaning up the template itself
build out the RESTful endpoints for getting the pdf
much more testing. I'm not sure this will work for all the texts we've got. I'm making a lot of assumptions here.
font selection
deployment details-- (including latex packages, etc)
multiple templates; perhaps an alternate template with maybe 3 verses at most per page, so i can annotate while i translate

Thanks, this is a start!
One way to put off thinking about deployment etc is to simply generate a .tex file that can be downloaded, and leave turning it into a PDF (by running xelatex or whatever) to the user: this won't be as useful to most users as having an actual PDF, but it's a start, and keeps the initial problem minimal.
In fact, we could put it off for quite a while: could simply generate the .tex and .pdf files offline (doesn't have to run as part of Ambuda), and have them stored in some repo / S3 / whatever.

Ah yes, that makes sense w.r.t. deployment. I think we can think about the S3 bucket + Overleaf API to do the compilation. But that's for later. I'll start testing this weekend. See if I can get a good number of the text we have up working.

Ohhh I didn't realize jinja could be used for this. Thank you!

@shreevatsa I've converted the code to a jinja template.

How do I suppress those god awful halting latex compile messages? I've tried xelatex and lualatex. Couldn't figure it out...
I support two of the texts on ambuda now. kumarasambhava and the śivopanishad. There's so much variation in texts. Right now I'm just writing a bunch of try escapes to get the fields to parse. Not sure if that's sustainable...

Should I focus on:

adding texts, or
getting the pdf to look nicer?

@skmnktl About supporting more texts v/s getting the pdf to look nicer, it's up to you (whatever you're more motivated to do), though if you're neutral between the two options, then I'd say that getting the PDF to look nicer can be done indefinitely, after a first version gets in. Also consider a third option, that of cleaning up the PR for production.
I could think of a sequence of PRs like, say:

A Python script that takes in an XML file and generates a .tex file (OK if it doesn't look great or works only on some XML files).
Extending that script to more files.
Offline tasks for generating PDF files by running the scripts, and storing those PDF files somewhere.
A route in the app to point to / download the corresponding (previously generated) PDF file, given a text.

But could also think of doing it in other orders.
BTW what did you mean by "halting latex compile messages"?

@shreevatsa For the tasks outlined:

A Python script that takes in an XML file and generates a .tex file (OK if it doesn't look great or works only on some XML files).

This is what make_latex.py does. I'm writing the resultant .tex file as output.tex.

Offline tasks for generating PDF files by running the scripts, and storing those PDF files somewhere.

Do we want to wait on this? Should I use AWS? Or perhaps a Google Cloud Storage Bucket? Or perhaps a mongo instance? If the main app will move to a more permanent storage, I can just piggyback off that. If you have preferences on what that storage will be in the future, we can do that as well.

A route in the app to point to / download the corresponding (previously generated) PDF file, given a text.

I'm hesitant to write this since the storage decisions have yet to be made. We could go the render on the fly every time route, and for that I can make a FastAPI endpoint. There's not that many texts right now anyway.

@skmnktl Hi, to be clear, by "sequence of PRs" I meant doing each of those things in a separate PR (so to answer your latter two questions, yes definitely wait on storing in the cloud, and don't write the app now) — I was actually talking about removing stuff from this PR (or moving to another one), to keep it minimal and ready to merge. :-) So this PR would have only 3 files:

make_latex.py
parser.py
template.tex

if I understand correctly. (Because output.tex is generated by make_latex.py, and output.pdf is generated by running xelatex/lualatex on output.tex, and tufte-handout.cls / tufte-common.def are already part of TeX Live so don't need to be committed.)
In the meantime I'll download and compile output.tex on my computer, so that I can see what error messages you were talking about…

@skmnktl I looked into output.tex locally. I see what you mean about the errors — there are indeed a lot of them, and even if hitting Enter (or putting TeX in batchmode) will proceed with those errors ignored, it's not a good state to get into :)
Let's start with a template where we understand why each line is needed, and which compiles without any errors or warnings, and add things as needed (e.g. the current template has a line or two of math-related stuff that I'm sure we won't need for a long time if ever).
The file compiles fine without any errors or warnings if I replace lines 1–53 of output.tex (or template.tex originally) with the following:
\documentclass{article}

\title{Kumārasaṃbhava}

\author{apauruṣeya}

\usepackage[parfill]{parskip}
\usepackage{fontspec}

\newenvironment{sanskrit}{}{}

or, if you want to keep tufte-handout (from looking around online, it seems to be not a very well-maintained package and has quite a few bugs unfortunately…), this works:
\documentclass[nobib]{tufte-handout}

\title{Kumārasaṃbhava}

\author{apauruṣeya}

\usepackage[parfill]{parskip}
\usepackage{fontspec}

% Fix for bug: see https://tex.stackexchange.com/q/200722
% and https://github.com/Tufte-LaTeX/tufte-latex/issues/64
% Set up the spacing using fontspec features
\renewcommand\allcapsspacing[1]{{\addfontfeature{LetterSpace=15}#1}}
\renewcommand\smallcapsspacing[1]{{\addfontfeature{LetterSpace=10}#1}}

\newenvironment{sanskrit}{}{}

Feel free to add your font choices back in later.
There were also some errors from Babel's Sanskrit setup; we don't seem to need it for this output.tex, and we can also consider using polyglossia instead of babel. (Sorry I haven't looked into the Devanagari version yet; I didn't run make_latex.py and just tried working with the output.tex that was already in the PR.)

@shreevatsa I took your comments above and did a few things with my last commit:

I removed the tufte package entirely.

I changed the template block and variable markers for jinja.
I have a script in src now that toggles pdf creation, and then moves the pdf to a render_pdf/pdf_outputs titled <text_title>_<author>.pdf.
And all the intermediate files are deleted (including the tex file).
For now, I've suppressed the "sanskrit" environment that I had earlier for babel. But once we start supporting documents transliterated into other scripts, we can set those up I think.
The runscript now uses latexmk rather than xelatex, though it does still call the xelatex engine in the background.

@skmnktl Great! I still see the tex_files with output.tex etc in the PR; could you take another look? Maybe they didn't get removed properly…

By the way, if it helps, feel free to consider splitting this PR further, and checking in just one of the two:

just the parsing code to get "data" out of the TEI XML file,
just the LaTeX template and build scripts, with clear documentation on what data it needs as input.

(E.g. it's ok to merge just (2) for now, and work on (1) in parallel.)

Do you want to meet this weekend to finalize this? I can wrap it up in the
next couple of days?
On Thu, Apr 6, 2023 at 9:32 PM Arun Prasad @.***> wrote:

Returning to this PR -- I suggest we scope this PR down to the simplest
thing that adds new functionality then iterate on it with follow-up PRs.
—
Reply to this email directly, view it on GitHub
https://github.com/ambuda-org/ambuda/pull/364#issuecomment-1499824428,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AE2BJBATJKU2CSUW6P4PON3W75VDFANCNFSM6AAAAAAQSTANOE
.
You are receiving this because you were mentioned.Message ID:
@.***>


add ssh start command to readme

Even it's no hard command it took me minutes to figure out why my code wasn't working
:ssh.start was the command I was looking for

Merged!  Thanks!


ACME-clientsetup-retrial

adding retry to acme setup client, since it can experience timeouts  or other types of Connection Errors

Coverage increased (+0.004%) to 59.525% when pulling b5ab87877b34e760de6c1d0abef166301bd9618a on hosseinsh:acme-client-setup-retrial into 356d3ee9f3c3f7e1026fe70cb946afb86b6c77d3 on Netflix:master.


Feature request: late review reminders

Would we be able to have auto-late review reminders? I suggest that they occur at 1 and 3 weeks after review date, and they tag the editor as well as the reviewer so the editor is reminded that it happened.  Example relevant issue: https://github.com/ropensci/software-review/issues/554

I suspect this would mean editors would need to never forget to submit reviews otherwise the bot wouldn't know the review exists. :thinking:

@noamross @mpadge thoughts?

I think this would be a useful thing to have. The reminders would also serve as a reminder to editors to remember to log the reviews, which should also just be a :heavy_plus_sign:, right?


pod.yaml file is missing in app-a/chp15/troubleshooting/

missing a file here!

Thanks for pointing me to it. This has been corrected with https://github.com/bmuschko/ckad-study-guide/commit/f61a4b7b8e6aec9e4924dddb93a58c4576e57f2c.


docker-compose.yaml

Feature Description
Has someone discussed whether or not to have a docker-compose.yaml included with the project? It is my step 2 for testing once I get the project checked out.  I can add, but I figured you had a reason for not adding?
Screenshots
No response

Using this https://docs.gitea.com/installation/install-with-docker

Maybe not related, I did some work for deploying Gitea based on docker-compose, which likes using helm chart in k8s.
But I have no enough time to maintain it now.
This tool can deploy Gitea with DB and Runner Ready without writing docker-compose file.
Of cause, it can do more things, not only deploying Gitea.
see
https://github.com/TeaChart/teachart
and
https://github.com/TeaChart/gitea


Add JSON adapter

Would it be possible to add a JSON adapter? This could potentially be useful to situations where data files are shared amongst libraries in Ruby, Node, Python, etc.

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/11327920-add-json-adapter?utm_campaign=plugin&utm_content=tracker%2F155789&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F155789&utm_medium=issues&utm_source=github).

What would it do specifically?

Persist object data to JSON files, as an alternative to YAML or CSV formats.

Ah OK so we're talking about a file-based adapter targeting json. Yes this
sounds good. It would be very similar to yaml/csv adapters.
On Sat 18 Apr 2015 at 18:40 Drew Dara-Abrams<EMAIL_ADDRESS>wrote:

Persist object data to JSON files, as an alternative to YAML or CSV
formats.
—
Reply to this email directly or view it on GitHub
https://github.com/rom-rb/rom/issues/205#issuecomment-94179659.

Great, thank you for considering this.

I will work on this issue.

@lukeses I've done some initial work in https://github.com/rom-rb/rom-json, but writing is still missing. Is that something you'd care to look into?

Think it would be better to have a rom-file adapter, then plugin an encoder/decoder each would take a string and convert to the required format, kind of like how rom-http will work

Yeah that is the plan actually. Think we're getting there with csv yaml and
now json adapters having so much in common.
On Fri 7 Aug 2015 at 12:10 Andy Holland<EMAIL_ADDRESS>wrote:

Think it would be better to have a rom-file adapter, then plugin an
encoder/decoder each would take a string and convert to the required
format, kind of like how rom-http will work
—
Reply to this email directly or view it on GitHub
https://github.com/rom-rb/rom/issues/205#issuecomment-128663845.

Closing this one as we have rom-json now :boom:


feat(neocmakelsp): use release assets

Describe your changes

Use release assets since neocmakelsp provide the binary file, and removes dependency on cargo
Issue ticket number and link

Checklist before requesting a review

[x] I have successfully tested installation of the package.
[x] I have successfully tested the package after installation.

Screenshots

I think we've had that discussion somewhere else.
But it can have a downside to use release assets:

You are dependent on the release asset names and also architectures
So if the names change, this change has to be reflected here. And if they decided to drop or add support for a platform, it has to be done here too.
So using cargo makes it a bit longer for the end user to have the binary compiled, but it will work on their machine no matter the release assets.

But ultimately it's up to the maintainer to approve or deny it.

I think we've had that discussion somewhere else. But it can have a downside to use release assets:

Would you mind giving me the link to the discussion. I want to see the conclusion of discussion.

You are dependent on the release asset names and also architectures
So if the names change, this change has to be reflected here. And if they decided to drop or add support for a platform, it has to be done here too.
So using cargo makes it a bit longer for the end user to have the binary compiled, but it will work on their machine no matter the release assets.

That's absolutely true. The changes will increase the burden of maintainer.
IMHO, as this project is used by many people, these changes not only save the network bandwidth, but also save the power consumption when compiling binary files.

Would you mind giving me the link to the discussion. I want to see the conclusion of the discussion.
https://github.com/mason-org/mason-registry/pull/6057

That's absolutely true. The changes will increase the burden on maintainer.
IMHO, as this project is used by many people, these changes not only save the network bandwidth, but also save the power consumption when compiling binary files.

Yeah, you can make that argument but I do think most end users don't really care about these that much. They most likely care about it taking a long time to update and such.
But yeah. It's ultimately up to the maintainer to make the decision

Okay, I didn't notice there's already a PR. Close this PR and track here.

Can mason use fallback way to download asserts when the target platform asserts are not provided?


Tutorial on https://hub.qovery.com/guides/getting-started/deploy-your-first-application/ failed

The tutorial on:
https://hub.qovery.com/guides/getting-started/deploy-your-first-application/
Here's what went wrong:

Hi @Niyaz041 ,
while the videos are indeed referring to our previous interface, almost 100% of the content is still valid. can you tell me more about which part we should fix?


show a table if the user does not pick a location when replicating

Andreas (chronark) points out that DX could be better if the user is presented with the list of locations right away. We will also take the opportunity to teach them that it is possible to pass a --location next time.

@CodingDoug for your review.

This table (and likely the other location table introduced recently) need to be able to scroll its contents in order to accommodate shorter terminal windows.  If the window is shorter than the list (by some margin), the user can't see the topmost options and the selection cursor. This is obviously problematic.
The intro text is also part of this issue. Consider if there is intro text, and it's not allowed to stay in view along with the table, it's effectively invisible for shorter windows.

have to, no. But it's certainly nice, no?
I think in practice this will work for almost any terminal size, plus this is contained to helper paths.

( I just force-pushed a version that actually fixes the table size )

have to, no. But it's certainly nice, no?

Fancy, yes.  Bu tnice can mean different things to different people.  For a CLI, my objective judgement of "nice" is that it gets me to accomplish a task as fast as possible, which is why I mention number of keystrokes as a metric to optimize. Tapping the down arrow 20+ times to get to yyz is not "nice".

@CodingDoug I added code to the main table implementation (so should cover all interactive tables) that lets you search the id list by typing.
How about that?

(it's a bit buggy and will need another pass, but just as a PoC)


multi-word exact matches at the start of a title aren't always first in the list of search results

Hi gang, search ranking question - I was trying to find the book "Information Law", by searching on those words:  https://wellcomecollection.org/works?query=information+law. As you can see the book doesn't show up until about half way down the second page of hits, even though the words are an exact match to the first two words of the title. Is that to be expected (because the book's works page isn't very popular or something?). If I add the author name to the search, then it shows up as the only hit (the book is https://wellcomecollection.org/works/zkg7xqm7).

the current query is prioritising matches in the search.relations, which has no positional awareness. we don't currently prioritise proximity to the start of a title (or any other field)

The 'correct' result for this query can be brought back by adding a span_first query:
{
  "query": {
    "bool": {
      "should": [
        {
          "span_first": {
            "match": {
              "span_term": {
                "data.title.shingles": "information law"
              }
            },
            "end":1,
            "boost":1000
          }
        },
        {
          "multi_match": {
...

I'd like to investigate the effect of

turning down the boost on the relations field vs
adding the span_first to title.shingles
using the standard set of rank tools before making any changes to the prod query

Issue first reported on Slack at https://wellcome.slack.com/archives/C8X9YKM5X/p1653293886888919


Added error message for prompt-based loss and num_variants=1

The loss functions ccs_prompt_var and prompt_var_squared cause NaN errors when --num_variants is set to 1, because of the variance calculation.
I added an error message in train.py for this.
As ccs_prompt_var and num_variants=1 are set by default, this means an error is thrown by default, which is not ideal.

Ah it looks like torch.var by default computes the sample variance, which divides by n-1, rather than n. The previous code was dividing by n so returned the equivalent of CCS loss. But your solution seems fine too, since we don't really care about being able to run prompt_var with num_variants=1

@AlexTMallen yeah you can also do unbiased=False but I think it's better to just error probably


Issue: Types not found.

When importing the library in TypeScript, the type declaration isn't found.
I think this issue is caused by package.json not linking to the correct file. It's pointing to a types file in the root of the build folder, but the NPM modules doesn't come bundled with that file.
It's also worth noting that the types for ESM and CJS are identical. You can remove the types from the export fields and point to a shared types file in types. VSCode cannot read the types inside export anyway.

@terkelg  Thanks for the report! The fixed version 0.0.14 published!


image build is broken

Currently main branch head is 823fe0bb11973af6c4efdccea32a9a814dcf77b1 and the image build is broken
docker build -t limitador:latest .
...
error: failed to run custom build command for `openssl-sys v0.9.72`

Caused by:
  process didn't exit successfully: `/usr/src/limitador/target/release/build/openssl-sys-4a2b2d25d671da9a/build-script-main` (exit status: 101)
  --- stdout
  cargo:rustc-cfg=const_fn
  cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_MUSL_OPENSSL_LIB_DIR
  X86_64_UNKNOWN_LINUX_MUSL_OPENSSL_LIB_DIR unset
  cargo:rerun-if-env-changed=OPENSSL_LIB_DIR
  OPENSSL_LIB_DIR unset
  cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_MUSL_OPENSSL_INCLUDE_DIR
  X86_64_UNKNOWN_LINUX_MUSL_OPENSSL_INCLUDE_DIR unset
  cargo:rerun-if-env-changed=OPENSSL_INCLUDE_DIR
  OPENSSL_INCLUDE_DIR unset
  cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_MUSL_OPENSSL_DIR
  X86_64_UNKNOWN_LINUX_MUSL_OPENSSL_DIR unset
  cargo:rerun-if-env-changed=OPENSSL_DIR
  OPENSSL_DIR unset
  cargo:rerun-if-env-changed=OPENSSL_NO_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS_x86_64-unknown-linux-musl
  cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS_x86_64_unknown_linux_musl
  cargo:rerun-if-env-changed=TARGET_PKG_CONFIG_ALLOW_CROSS
  cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-unknown-linux-musl
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_unknown_linux_musl
  cargo:rerun-if-env-changed=TARGET_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-unknown-linux-musl
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_unknown_linux_musl
  cargo:rerun-if-env-changed=TARGET_PKG_CONFIG_SYSROOT_DIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
  run pkg_config fail: "pkg-config has not been configured to support cross-compilation.\n\nInstall a sysroot for the target platform and configure it via\nPKG_CONFIG_SYSROOT_DIR and PKG_CONFIG_PATH, or install a\ncross-compiling wrapper for pkg-config and set it via\nPKG_CONFIG environment variable."

  --- stderr
  thread 'main' panicked at '

  Could not find directory of OpenSSL installation, and this `-sys` crate cannot
  proceed without this knowledge. If OpenSSL is installed and this crate had
  trouble finding it,  you can set the `OPENSSL_DIR` environment variable for the
  compilation process.

  Make sure you also have the development packages of openssl installed.
  For example, `libssl-dev` on Ubuntu or `openssl-devel` on Fedora.

  If you're in a situation where you think the directory *should* be found
  automatically, please open a bug at https://github.com/sfackler/rust-openssl
  and include information about your system as well as this message.

  $HOST = x86_64-unknown-linux-gnu
  $TARGET = x86_64-unknown-linux-musl
  openssl-sys = 0.9.72

  ', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/openssl-sys-0.9.72/build/find_normal.rs:180:5
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed
The command '/bin/sh -c cargo build --release --target=x86_64-unknown-linux-musl --all-features' returned a non-zero code: 101

Isn't this fixed in #58?


massive config parsing improvement that improves the parsing of confi…

…g?!?

This can be simplified by removing it.

removingh what?

Hello syrice,
submitting troll pull requests (pull requests with intentionally problematic or irrelevant code changes) is not a productive or professional behavior in the software development community. It can have negative consequences for both the individual who submitted the pull request and the project as a whole.
First and foremost, submitting troll pull requests wastes the time and resources of the project maintainers and other contributors. Reviewing and addressing these requests takes time and effort away from more meaningful work, and can even slow down the progress of the project. This can also cause frustration for other contributors who are trying to make valuable contributions to the project.
 The software development community is built on trust, collaboration, and mutual respect, and submitting troll pull requests goes against these core values. This can hurt the individual's credibility and make it harder for them to participate in other projects or be taken seriously in the future.
Finally, submitting troll pull requests can also harm the reputation of the project as a whole. Projects that are known for tolerating this kind of behavior can be seen as unprofessional and less credible, which can make it harder for them to attract new contributors or users.
 It wastes the time and resources of others, damages the reputation of the individual and the project, and goes against the core values of the software development community. Instead of submitting troll pull requests, individuals should focus on making meaningful contributions that advance the project and the community as a whole.
Sincerely yours,
Ender

Hello syrice,
submitting troll pull requests (pull requests with intentionally problematic or irrelevant code changes) is not a productive or professional behavior in the software development community. It can have negative consequences for both the individual who submitted the pull request and the project as a whole.
First and foremost, submitting troll pull requests wastes the time and resources of the project maintainers and other contributors. Reviewing and addressing these requests takes time and effort away from more meaningful work, and can even slow down the progress of the project. This can also cause frustration for other contributors who are trying to make valuable contributions to the project.
 The software development community is built on trust, collaboration, and mutual respect, and submitting troll pull requests goes against these core values. This can hurt the individual's credibility and make it harder for them to participate in other projects or be taken seriously in the future.
Finally, submitting troll pull requests can also harm the reputation of the project as a whole. Projects that are known for tolerating this kind of behavior can be seen as unprofessional and less credible, which can make it harder for them to attract new contributors or users.
 It wastes the time and resources of others, damages the reputation of the individual and the project, and goes against the core values of the software development community. Instead of submitting troll pull requests, individuals should focus on making meaningful contributions that advance the project and the community as a whole.
Sincerely yours, Ender

ily frfr

What he said. Lol.

I mean I didn't read it but yeah what he said.


added html

I finished with week 1 and went on to week 2 and started working on week 2 project before I realized I didn't as for a pull request for week one. So the pull request shows multiple files because I'm working on the week 2 challenge (interactive game with an index.html, main.js and main.css).

Your pull request needs further work because of the following reason(s):

More than one file has been added/removed/modified.

Once you have resolved the issues, comment on this request to be re-judged
This is an automated response, beep boop.
if you think there is a mistake, contact a maintainer

32f7274877e75231d873439287878516360a56fe

You made modifications to files in README. this is not a part of any challenge. Remove those two files and i will be happy to merge

I'm sorry I didn't realize I saved filed to the README. I removed those files and made sure all the files are where they should be. Everything should be good now. If not let me know!

You have a file "games/.../untitled-1" please remove this file.

I

Sorry about that. Im looking through these on my phone and its hard to press the tiny buttons.

Your pull request needs further work because of the following reason(s):

More than one file has been added/removed/modified.

Once you have resolved the issues, comment on this request to be re-judged
This is an automated response, beep boop.
if you think there is a mistake, contact a maintainer

db8d965bfa7cd10b5c766c037b096fb619b6b8d2

I removed that file! No worries! I appreciate your feedback and apologize for the files showing up in random areas. I'll make sure when I finish week 2 that I double check all files and changes I've made so there's no untitled files anywhere or in the wrong place.

Looks good for now, ill keep this pr here under attention (rather than needs work) until your game is completed. Basically preventing the bot from autoclosing

This PR has been idle for more than 24 hours, and will now be closed. Feel free to re-open.
This is an automated response, beep boop.
if you think there is a mistake, contact a maintainer

58064ca03a93aecf9c22db2e5723c782cd34ac46

Whenever you've completed the challenge either open a new PR, or tell me to reopen this one for you.


filament-renderer: disable bloom and update golden images.

I don't see any significant differences.

Maybe I should trim this PR to include only the first commit, since the second is a no-op.

@prideout If you've verified that, that's probably a good idea.

Done.

The FIlament team has released 1.5.0, 1.5.1, and 1.5.2 in a slew so I've updated this PR to 1.5.2.
NOTE: while examining screenshots I noticed that the model-viewer screenshot of SimpleMorph needs to be updated, but the FIlament screenshot does not need to be updated.  This seems unrelated to this PR so should perhaps be updated separately.

I've now reviewed the screenshots so I'll mark this as ready for review.  As previously noted, the filament-specific screenshots have remained as-is.


edupont04 from OPS portal: Merge from master to live

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.publish.config.json
:white_check_mark:Succeeded

README.md
:white_check_mark:Succeeded

dynamics-gp/docfx.json
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


-db2u-restore-morph-job Errors - The default database not found in system catalog

global:
  dbType: db2wh
images:
  db2u:
    replicas: 1
mln:
  distribution: '2:1'
  total: 2
limit:
  cpu: 5000m
  memory: 5Gi
storage:
  storageClassName: nfs-client
  useDynamicProvisioning: true
  persisted: true
subType: mpp
dedicated: true
runtime: ICP4Data

Using this chart: https://github.com/IBM/Db2/tree/develop/deployment/ibm-db2u
This is our nfs-client storageclass
IsDefaultClass:        No
Annotations:           <none>
Provisioner:           cluster.local/nfs-nfs-client-provisioner
Parameters:            archiveOnDelete=true
AllowVolumeExpansion:  True
MountOptions:
  nfsvers=4
  context=system_u:object_r:container_file_t:s0
ReclaimPolicy:      Delete
VolumeBindingMode:  Immediate
Events:             <none>

This happened after we uninstall and re-installed db2 several times.
Note that we label our nodes with "icp4data=database-db2wh"

This happens to me as well! And do not know, what's going wrong...


linux error

An error occurred during the installation and operation of Linux. The cross source request has been intercepted: the same source policy prohibits reading at http://localhost:8080/sockjs -Node / info? T =<PHONE_NUMBER>007. (reason: CORS request failed)

You need to run the Node.js server AND running the development server.


add rebus

Associated Issue: #
Summary of Changes

change 1
change 2

Pull Request Test Coverage Report for Build 743

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 87.143%

Totals

Change from base Build 514:
0.0%

Covered Lines:
158

Relevant Lines:
180

💛  - Coveralls


upload the updated sql file

e_dukaan(1).sql sent. please Check and confirm if it works with current code. If not I will send from work system.


Initial fixup of the VSCode extension

This PR is just fixing up and cleaning up the old VSCode extension with an at least initial attempt to fix #309 and #112
This will make the extension a bit more useful than it was before.
allthough debugging capabilities do not work right now.
Much more work in the future will be required to properly hook up hovering and debugging.

Thanks for the changes. Sorry we didn't notice them sooner - the notifications went to someone on holiday.

happens

As for the changes, it seems there is a switch from spaces to tabs - I thought the expectation was spaces everywhere now? It also seems to use buck2 rather than starlark as the driver. I'm honestly not sure what to do about that - we deliberately don't want to couple the starlark-rust project to Buck2, but it is the biggest user. Maybe there should be a switch?

There were some talks about this in the mentioned issue, thus I thought that was the right direction. I obviously do not have access to any internal communication that happen.

Maybe it should try and find if buck2 or starlark is on the path?

Right now, it uses the one that is already in the PATH variable.

Is the nix thing entirely optional? By removing the .lock directories, doesn't that risk the breakage as packages upgrade?

The nix thing is indeed entirely optional. I used nixos at the time of implementation.

Due to lack of interest from my side, I'll close this PR. however I'll keep my fork and branch around in case anyone wants to Pickup on it.

I've decided to publish my own third-party extension, it's source available under https://github.com/PeerStudios/buck2-lsp-adapter and published under https://marketplace.visualstudio.com/items?itemName=PeerStudios.buck2-lsp-adapter which requires buck2 itself instead of starlark-rust. similarly to what i did in this PR.


Feature Request: Make package configuration easier with Recipes

Current Situation
Some packages (such as MariaDB, Apache, and NGINX) require configuration or special environment variables to work correctly with the Nix store + customer's devbox projects. For example -- MariaDB's default installation with Nix requires users to specify where the PID and SOCK file should be stored, and then pass those variables when starting and stopping mysqld.
Proposed Solution
Devbox should provide a way to create and share configuration for specific packages, and add this configuration when the developer installs the package. For example -- when setting up MariaDB, we can automatically set some default env variables that will be used when the developer activates their devbox shell.
This automatic configuration should be published in an open format, so developers can add Devbox configuration for packages as needed.

Shipped as Plug-ins in 0.2.0


Refine POSTBUILD for spresense and esp32-core

+1 but it failed the checks

Will the build tests restart automatically so that we can confirm that?

I re-run the CI checks.


add transformation params as its required since Directus 10.6.3

Generation of attachment files with Directus files by param files fail with TypeError: Cannot read properties of undefined (reading 'transforms'), see issue https://github.com/ryntab/Directus-Mailer/issues/9

Hello! Thanks for the PR, ill test and merge. Thanks!


Error displaying specified character in "menlo for powerline" font

Steps to reproduce
I need to adjust the font to "menlo for powerline" for the Agnoster theme "oh my zsh" in my WSL terminal. When I type the characters "f" or "i" respectively I have no problem, but when I type "fi" they are distorted.
Expected Behavior

Actual Behavior

Please fill out the issue template. The version numbers aren't there just for fun, but (in this case) to know which font renderer you're using. 😟
The "fi" you're seeing there is a so called ligature. That ligature in particular is very common: https://en.wikipedia.org/wiki/Ligature_(writing)#Latin_alphabet
You can disable ligatures by opening your settings with Ctrl+Shift+, and adding this:
{
    "profiles":
    {
        "whatever your profile name is, for instance 'defaults'": {
            "font": {
                "features": {
                    "liga": 1
                }
            }
        }
    }
}


Comparing Option[string] to Option[string]

Fix issue where an issue where the emoji parameters id and name are being compared to a string when they should be compared to an Option[String]

Oops, I forgot to push to misc. Could you just change it to isNone instead of some("")?

yup, will do

Alright, I'll make a patch release.

that is how you want it yeah?


Script to download weights in advance

Hi,
would you be able to provide a script to download model weights? We are running Chai1 on cluster with job nodes having no internet access. Having such a script would really help us!
(Sometimes loading part of the model onto GPU happens before downloading part in the code. On the prototyping development node on our cluster there’s very limited GPU memory, so we can’t first run it on command line then submit a cluster job)
best wishes

take these lines and run for every model_size in AVAILABLE_MODEL_SIZES
https://github.com/chaidiscovery/chai-lab/blob/main/chai_lab/chai1.py#L411-L417
You also want to control where weights are stored, see Where are downloaded weights stored? in repo readme.

Thanks for pointing that out. It seems that there are more things to be downloaded than just the model weights, including the cached conformers for rdkit as well as the ESM tokenizers and models.
I have written a script for downloading everything. This might be helpful for people who want to run Chai1 on clusters.
from chai_lab.utils.paths import chai1_component, downloads_path
import chai_lab.utils.paths as paths
from transformers import EsmTokenizer, EsmModel

components = ["feature_embedding", "token_input_embedder", "trunk", "diffusion_module", "confidence_head"]
MODEL_SIZES = [256, 384, 512, 768, 1024, 2048]

#download model weights in differently sized models
for size in MODEL_SIZES:
    for component in components:
        chai1_component(f"{size}/" + component + ".pt2")

#download cached rdkit conformers
paths.cached_conformers.get_path()

#download ESM tokenizer and model
esm_cache_folder = downloads_path.joinpath("esm")
model_name = "facebook/esm2_t36_3B_UR50D"
tokenizer = EsmTokenizer.from_pretrained(model_name, cache_dir=esm_cache_folder)
model = EsmModel.from_pretrained(model_name, cache_dir=esm_cache_folder)

Great, thanks for posting that @andrew0901

@andrew0901  head up this script will need adjustments after I've merged #144

Thank you for the script. But there seems to be a typo in the url.
e.g.
https://chaiassets.com/chai1-inference-depencencies/models/768/token_input_embedder.pt2
'depencencies' should be 'dependencies'?

Following the update implemented in PR #144, the iteration across different sizes is no longer necessary, as the system now utilizes a single set of weights that is universally applied across all model sizes.
Suffix is changed to .pt.
Additionally, I'd like to point out a nomenclature change: the component previously known as token_input_embedder has been renamed to token_embedder, as reflected in the current component structure:
Previous:
components = ["feature_embedding", "token_input_embedder", "trunk", "diffusion_module", "confidence_head"]

Current:
components = ["feature_embedding", "token_embedder", "trunk", "diffusion_module", "confidence_head"]

Thanks for the updates! I have integrated the changes. But note that I haven't tested it yet. Please let me know if anything needs to be changed still.
from chai_lab.utils.paths import chai1_component, downloads_path
import chai_lab.utils.paths as paths
from transformers import EsmTokenizer, EsmModel

components = ["feature_embedding", "token_embedder", "trunk", "diffusion_module", "confidence_head"]

#download model weights in differently sized models
for component in components:
    chai1_component(component + ".pt")

#download cached rdkit conformers
paths.cached_conformers.get_path()

#download ESM tokenizer and model
esm_cache_folder = downloads_path.joinpath("esm")
model_name = "facebook/esm2_t36_3B_UR50D"
tokenizer = EsmTokenizer.from_pretrained(model_name, cache_dir=esm_cache_folder)
model = EsmModel.from_pretrained(model_name, cache_dir=esm_cache_folder)


Setup & currency

Pull Request Documentation
Feature

Implemented resource interface
Create Gold, Research Classes
Add base game

Misc

Add Game run command in README.md

Additional context
Close #2
Close #4
Add any other context or screenshots about the pull request here.
Checklist

[X] My code follows the code style of this project How to use black autoformatter.
[X] I have applied relevant labels and linked the PR to the relevant Issue.

Good job, let's remove interfaces, they are over complicating stuff, with only one week, we don't rly need them.


autopep8 not working

Issue Type: Bug
I've watched a few videos on setting up Python with VS Code. I can't get autopep8 working to format the files.  I've check paths, settings and de-installed/re-installed VS Code.  I've tried deleting old folders and VSCode remembers them after re-install, so that doesn't seem to work.  I checked the path to autopep8 and all the recommended settings for save on format.
When I try the right click menu and [Format Document], the error is "Extension 'Python Language Basics' cannot format 'HelloWorld.py'"
I admit to being a beginner, but I don't know what to look at or how to fix it.  Thanks.
VS Code version: Code 1.39.2 (6ab598523be7a800d7f3eb4d92d7ab9a66069390, 2019-10-15T15:35:18.241Z)
OS version: Windows_NT x64 10.0.17134

System Info

Item
Value

CPUs
Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz (4 x 2712)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: disabled_softwareoop_rasterization: disabled_offprotected_video_decode: enabledrasterization: enabledskia_deferred_display_list: disabled_offskia_renderer: disabled_offsurface_synchronization: enabled_onvideo_decode: enabledviz_display_compositor: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
15.86GB (11.10GB free)

Process Argv

Screen Reader
no

VM
0%

Extensions (1)

Extension
Author (truncated)
Version

python
ms-
2019.10.44104

I have re-entered the issue in vscode-python


feat(toHaveGeometryCount): add new matcher

Proposed Changes
Verifies a valid GeoJSON GeometryCollection has a specific number of geometries.
Pull Request Checklist
Please check if your PR fulfills the following requirements:

[x] I have read the CONTRIBUTING guidelines
[x] My commits follow the appropriate guidance
[x] I certify that I have the appropriate permissions to add these changes to the repository

New Matcher Submission Checklist

[x] Open an issue with detailed description of the purpose and required behavior
Create Core Function (If Applicable)

[ ] Create a core function under src/core/<category>
[ ] Register the core function in src/core.js
[ ] Add a verification test to tests/core.test.js
[ ] Document the function using JSDoc. Refer to the issue. Include good and bad examples.

Create Matcher Function

[x] Create a matcher function under src/matchers/<category>
[x] Register the matcher in src/matchers.js
[x] Add a verification test to tests/matchers.test.js
[x] Add the matcher to src/index.d.ts
[x] Add the matcher to the .cz-config.js list (alphabetical order under the allMatchers variable)
[x] Document the matcher using JSDoc. Refer to the issue. Include good and bad examples.

[x] Create a test for the matcher under tests/<category>
[x] Verify all tests pass and have 100% coverage
[x] Add the matcher to the README.md list (alphabetical order within category)
[x] Run the build script locally

Issues Addressed: #48
Other Information
This matcher adds no new core functions.

:tada: This PR is included in version 1.5.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


How do we split across namespaces?

 Issue by technosophos
2017-06-30 02:40:36

DO we want to run Acid in one NS, and jobs in one or more other namespaces?
If so, we need to either (a) mount the secret only in the master acid node, or (b) mount the secret only in the Job pods. Right now we do both.

Comment by bacongobbler
2017-07-01 16:42:54

I'm not sure which is the better option at this point. Perhaps we should opt for what is in master for now and wait for more user feedback first (read: when we or others start using this)?

Comment by technosophos
2017-07-07 23:57:06

I think this is possible if we merge #68.

Comment by technosophos
2017-07-13 04:03:20

Alright, at this point it should be possible to run Acid in one namespace and the builds in another.


Cannot push multiple files - Apache Mod Proxy Error?

Gogs version (or commit ref): <IP_ADDRESS>7
Git version: Client: 2.12.2; Sever: 2.1.4
Operating system: Client: Manjaro Arch; Server: Debian 8
Database (use [x]):

[ ] PostgreSQL
[x] MySQL
[ ] MSSQL
[ ] SQLite

Can you reproduce the bug at https://try.gogs.io:

[ ] Yes (provide example URL)
[x] No
[ ] Not relevant

Log gist (usually found in log/gogs.log):
`2017/04/21 23:13:25 [TRACE] Session ID: cf6bfb33ae944f4a

2017/04/21 23:13:25 [TRACE] CSRF Token: LmEA-CzQeBq3JFDhRSSOKFQU8Nw6MTQ5MjgwOTIwNTA5NjQ0NTA1MQ==
2017/04/21 23:13:25 [TRACE] Session ID: 6b4a2a64901164d3
2017/04/21 23:13:25 [TRACE] CSRF Token: PHlo0sCJdKDFInYFbx11qFIuB4Y6MTQ5MjgwOTIwNTQzMzUzNjI4OQ==`
Description
I can push single files just fine, but when I try to push multiple files at once I get
error: RPC failed; HTTP 502 curl 22 The requested URL returned error: 502 Bad Gateway
fatal: The remote end hung up unexpectedly
fatal: The remote end hung up unexpectedly
Everything up-to-date
I think it has something to do with the apache2 reverse proxy I use, heres the log:
[Fri Apr 21 23:13:25.953039 2017] [proxy:error] [pid 30628] (32)Broken pipe: [client 123.456.789.123:44522] AH01084: pass request body failed to <IP_ADDRESS>:6000 (<IP_ADDRESS>)
[Fri Apr 21 23:13:25.953260 2017] [proxy_http:error] [pid 30628] [client 123.456.789.123:44522] AH01097: pass request body failed to <IP_ADDRESS>:6000 (<IP_ADDRESS>) from 123.456.789.123 ()
Has anyone had a similar problem? How do I fix that?
My Apache2 VHost:
ServerName git.example.com
ServerSignature Off
ProxyPreserveHost On
SSLEngine on
SSLCertificateFile /etc/letsencrypt/live/www.example.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/www.eample.com/privkey.pem
ProxyPreserveHost On
ProxyRequests off
ProxyPass / http://<IP_ADDRESS>:6000/
ProxyPassReverse / http://<IP_ADDRESS>:6000/

Have try this
https://stackoverflow.com/questions/20406845/proxy-error-502-the-proxy-server-received-an-invalid-response-from-an-upstream/21849150#21849150

i have the same issue, but modifying apache2.conf doesn't change anymore.
[Sat Nov 28 09:13:04.753605 2020] [proxy_http:error] [pid 23418] [client <IP_ADDRESS>:50492] AH01097: pass request body failed to <IP_ADDRESS>:3000 (<IP_ADDRESS>) from <IP_ADDRESS> ()


Add anacron to the system services

As the docker containers aren't always up all time, particularly on the dev side, I find it convenient to have anacron making sure that cronjobs are executed once in a while, for example for log rotations, etc.
I've modified image/system_services.sh to add "$minimal_apt_get_install anacron" in my "Debian fork" (https://github.com/olberger/baseimage-docker/tree/debian).
Or maybe this is only useful in Debian environment, and not applying to the base Ubuntu system ?
Hope this helps.

I don't think anacron is useful enough to be included in the default baseimage. I think it should be opt in.


Events_Mobile - Event hosts filter does not effect the timeline to scroll to the 1st active event

Pre-req:
Create multiple events with different event hosts (some happening in the current year's past months and some in the upcoming months)
Steps to reproduce:

On mobile browser, launch the events page
Invoke the Filters panel and select the host names (mentioned on the pre-req.)
Click the View events button at the bottom

Expected Result:
The timeline should display the 1st active event - based on the filtering performed
Actual Result:
The timeline does not automatically scroll to the 1st active event (rather displays past events)

Issue verified to be resolved now (QA Build 004)


Code review

Hello!
I found your repo in pytorch discourse. I'm a pymc3 team member. Very happy that Bayes goes to pytorch too. We were planning to investigate if pytorch is suitable for bayesian reasoning. You can read our discussion here. I'd like to leave some comments for future improvements

Assertion errors can be hard to catch and if you have one, you can't figure out what type is it. I think some of assertions are not needed as they are checked at runtime without any supplementary code.

Now I see all stuff in code. Hope in future it will be more organized. Distribution submodule with splitting to distribution types can be a good solution.

Use pytest. I got used to unittest until tried pytest, now I have no way back to unittest, pytest is awesome

Use docstrings instead of # comment. It can be helpfull for future users

Remove NormalUnit and Uniform01, add defaults instead. Having lots of classes of the same distribution can be messy

I found your repo in pytorch discourse. I'm a pymc3 team member. Very happy that Bayes goes to pytorch too. We were planning to investigate if pytorch is suitable for bayesian reasoning. You can read our discussion here. I'd like to leave some comments for future improvements

Thanks for pointing this out. I am not familiar with Bayesian reasoning but I am with Bayesian inference.
From my cursory checking of PyMc3 it seems that it relies on a static graph approach, while PyTorch adopts a dynamic one, like Chainer.
Out of curiosity, is the approach for gradients (when needed) something among the lines of the stochastic computational graphs paper?
I'd like as much as possible to avoid re-inventing the wheel (also given I already developed a scientific library for another language), so I welcome integration / cooperation on this topic.

Assertion errors can be hard to catch and if you have one, you can't figure out what type is it. I think some of assertions are not needed as they are checked at runtime without any supplementary code.

There are many assertions because by usage experience I discovered that it was very easy to make that kind of mistakes.
I was thinking of adding a flag to disable them, mainly for performance reasons.

Now I see all stuff in code. Hope in future it will be more organized. Distribution submodule with splitting to distribution types can be a good solution.

It seemed overkill initially but it's a good point, I'll move to a more modular structure.

Use pytest. I got used to unittest until tried pytest, now I have no way back to unittest, pytest is awesome

PyTorch relies on unittest only, so I'd like to follow the convention.

Use docstrings instead of # comment. It can be helpfull for future users

Will do.

Remove NormalUnit and Uniform01, add defaults instead. Having lots of classes of the same distribution can be messy

The thing is the constructor arguments are different: for instance NormalUnit takes as input a shape while NormalDiagonal the mean and standard deviation parameters, hence the number and type of arguments are different.
While your suggested solution is technically doable I am not completely convinced that it would be less confusing than the currently adopted one.
I'll ponder over it.

I thought about it and will adopt your suggestion for reducing the number of distribution classes (last point).


CoreFoundation: adjust uuid.h inclusion

The Swift Runtime path always uses the UUID functions from the
statically linked UUID library.  However, the header was only included
in the non-Windows case.  The latest LLVM update makes implicit
declarations invalid as per the C99 standard.  Adjust the inclusion path
to also include uuid.h on Windows when building with the Swift
Runtime.

CC: @bnbarham

@swift-ci please test

@swift-ci please test Linux platform


Added plug-in config

Added basic files to make this theme compatible with Micro.blog plug-ins.

Added a couple more commits for compatibility with newer versions of Hugo.

@manton You did submit PRs for this! I merged it for Cypress but not for Red Oak. Sorry!


using APISIX's Casdoor plugin problem

when I Connect Casdoor via APISIX's Casdoor plugin， APISIX helps me with authentication, but how can I obtain a token from a request when APISIX redirects to my business system ？

Resolved by connecting via OIDC


Detect which sentence symbols are used in

In order to map symbols to their definitions, we need a notion of which prose sentences symbols are used in.
This task involves:

[ ] Detecting the bounding box locations of sentences in papers
[ ] Associating all symbols in an inline equation with the sentences they are used in (requires completion of #42 to detect which symbols are defined in inline equations)

Added a detector for the character offsets of sentences in TeX files in b53da95

As part of this task, I'm going to prototype some of the refactoring described in #57 . Specifically, this issue will include the following fixes:

[ ] Rename explanations -> common
[ ] Move commands to commands folder
[ ] Update the rules for registering and referring to folders

[ ] Add functions for registering folders
[ ] Centralize commands for requesting folders from directories to one or two helpers, that take the key a folder was registered with as input

[ ] Helper for writing to CSV with headers based on a class's keys
[ ] Helper for reading objects from CSV
[ ] A register_entity function that takes in a detector, colorizer, and upload_func argument

[ ] Creates commands that can be called from process
[ ] Adds commands to list that are valid in --start and --end paramters
[ ] Adds another type of entity to entities list

For the sentence detector, we'll implement it as follows:

[ ] Move the SentenceExtractor to the entities/sentences directory
[ ] Create an abstract DetectEntities command, that takes as input a detector function, an entity name, and a directory names for writing to
[ ] Create a generic colorizer (if one isn't already present) that can be called for colorizing sentences
[ ] Create an abstract ColorizeTex command, that takes as input a colorizer function, and entity name, a name of a directory to read entities from, and one to write entities to

Cleanup tasks include:

[ ] Migrate equation to use the new:

[ ] extractor command
[ ] colorizer command
[ ] locate-hues command

[ ] Migrate citation to use the new:

[ ] colorizer command (using a unique colorizer, inserting in preamble)
[ ] locate-hues command


[WIP] add LongPath support with PRI.LongPath

This utilizes https://github.com/peteraritchie/LongPath to re-add longpath support.
I have included it by source, because a) i had to do one little change and b) the bootstrapper is supposed to be single-file anyway.

regarding PRI.LongPath and the license:

@matthid: Could that be a problem?

Honestly, I'm not sure. The general spirit of LPGP is that modifications to the lib need to be also lgpl, but the rest of the program can be whatever. So that may be ok.
Now there are nuances with what is considered inside and outside, so it is probably not so clear cut from a legal standpoint.
@peteraritchie, what do you think? Would you be OK with us using your lib in source form in paket? The reason for using source is that the exe should be standalone.
If yes, how would you like to be credited?
Thanks!

Sure, as long as you're following the license, use the LongPath source code directly. Seems to fall under [Combined Work](https://www.gnu.org/copyleft/lesser.html#section4.  Basically, same license (unless it's commercial), make the source available, detail any modifications, and credit and/or include copyright notice if your software details its own copyright.  And no expectation of support or warrantee (if you encounter a problem, I'll see what I can do, but can't guarantee anything :) )
A note on the GitHub repo page like "Long path support provided by PRI.LongPath: https://github.com/peteraritchie/longPath" is fine

@0x53A To be honest I'm still a bit worried about the answer but I'm not a lawyer.
Similar issues (for example https://github.com/rubychan/coderay/issues/25) have been solved by switching to MIT. I think LGPL and GPL add additional restrictions to us and I don't want to think about legal stuff when using paket or paket source code at work. This change - because quite intrusive - makes that more difficult for me.
Maybe I'm just overly concerned here, and I clearly can see the benefit of your work. So I can compromise with this if we could enable and disable the usage of this lib by a feature flag (#if LGPL for example), because most end-users may not share my concerns (and only use the binary and not the source-code).
What do you think?
/cc @peteraritchie
/cc top contributors @forki @theimowski @agross @cloudRoutine @mrinaldi @vbfox @ctaggart @pblasucci @Stift @mexx @enricosada @smoothdeveloper

GPL - absolutely, this, as Ballmer put it, infects the whole source like a cancer.
LGPL - IANAL, but this mostly affects the part that is lgpl licensed itself.
From my understanding, especially section 4 and 5 of https://www.gnu.org/copyleft/lesser.html#section4:

We DO need to license all changes to Pri.LongPath under LGPL.

We DO need to place a notice somewhere, linking to the original work.

4.a) from my understanding, the note in the readme is enough

4.b) the lgpl license text is included with our copy of the source code of Pri.LongPath

4.c) afaik we never display a copyright notice during execution, so does not apply

4.d) paket is open-source, so the end-user may modify and recompile it

5.a) Our version of Pri.LongPath is open-source itself, I understand that to satisfy it

5.b) from my understanding, the note in the readme is enough

Hm, a few tests fail, probably due to differences between Pri and System.IO.

Will work on these after/if the license issue has been sorted out (whatever the resolution may be)

I'm not a fan of including LGPL code and would prefer if paket did without, but I don't think there is any real legal problem (IANAL)
The only real problem doesn't really concern the paket ecosystem as it is. It would be if one day some proprietary IDE wants to include the boostrapper in their codebase as they would need to provide a way to drop-in replace this library but that's relatively easy by including the nuget version instead (A full IDE wouldn't care about the single file aspect)

I'm a lot more worried about including the full library in the boostrapper, impact on file size seem to be +85% (53K -> 98K) that's VERY drastic for a bugfix that nearly never impact it (Boostrapper doesn't create package hierarchies)
I would prefer for the bootstrapper to have just the few things it uses re-implemented to be long-path aware.

On the library code itself i'm worried that it consider "mono" to mean non-windows as it's a recipe for problems.
Both because mono run on windows (And their impl isn't long path aware AFAIK) and dotnetcore run on linux & macs...
The lib doesn't seem to be prepared for the modern multi-framework multi-platform .Net world so we would need to PR changes there to add such support.

I pretty much agree with @vbfox on all points. However, there is one addition : We could still take this AND leave the bootstrapper as-is. Which would basically remove the bootstrapper argument (tbh it is more useful in the main code anyway)
So I already said that I'm not a huge fan. Though if people say it is useful we could take it.
Note that there still is a way to do it entirely different: There was a PR which tried to set some app.config stuff to enable long-paths. This didn't work out because of support for older systems and mono? We could fix that by making the boostrapper write/patch the app.config if we detect the correct system.

Ok thank you all for the feedback.

Note that there still is a way to do it entirely different: There was a PR which tried to set some app.config stuff to enable long-paths. This didn't work out because of support for older systems and mono? We could fix that by making the boostrapper write/patch the app.config if we detect the correct system.

This could actually work, but feels like a big hack, and may cause issues if you want to invoke paket.exe directly. Or it would need to copy itself to %temp%, and relaunch itself with the modified app.config...
Is there really no official Microsoft-supported way?
I'm still on vacation, and after that I would like to work on the other prs first (toolset 2, for example), so this will be a long time away anyway.
But unless we find a third, better way, I think I prefer longpath support via some library to copying and modifying the app.config.

This could actually work, but feels like a big hack, and may cause issues if you want to invoke paket.exe directly

Yes, you would only "get" the feature if you are using "Magic"-Mode. This would be a design decision if we go down this route.

Is there really no official Microsoft-supported way?

I don't know any besides setting the configuration. It really is a shame that the config is not ignored on older systems. Maybe there is such a way to make it work?


Fix the version of pytorch_transformers we use

Cause 1.1 is buggy, and 2.0 changes API

Pull Request Test Coverage Report for Build 1622

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.008%) to 66.815%

Totals

Change from base Build 1607:
0.008%

Covered Lines:
6740

Relevant Lines:
9492

💛  - Coveralls


Extend PDAL-java bindings with a mesh type

PDAL now provides mesh output from their Delaunay triangulation pipeline stage.  github.com/pdal/java should expose this type.  Extend that project in order to do that.
[Estimated 5 point task]

Closed in https://github.com/geotrellis/geotrellis-pointcloud/pull/38


Logic App (Standard) VS Code debugger does not handle nested For Each loops correctly

Describe the Bug
After a successful run the inner For Each loop always shows the number of iterations of the outer loop. It should be showing the number of iterations of that iteration of the inner loop. Additionally, any action inside the inner loop only appears to only show the details of the first iteration of the outer loop or, more likely, no input or output details even if the action was successful.

However, the workflow did run successfully and looking through the underlying storage tables shows that all loops were iterated correctly.

Plan Type
Standard
Steps to Reproduce the Bug or Issue

Create the workflow
Run it in the debugger
Use the Overview to open the run history.

Workflow JSON
{
    "definition": {
        "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
        "actions": {
            "Initialize_array": {
                "type": "InitializeVariable",
                "inputs": {
                    "variables": [
                        {
                            "name": "Data",
                            "type": "array",
                            "value": [
                                1,
                                2,
                                3,
                                4,
                                5,
                                6,
                                7,
                                8,
                                9,
                                10
                            ]
                        }
                    ]
                },
                "runAfter": {}
            },
            "For_each_value": {
                "type": "Foreach",
                "foreach": "@variables('Data')",
                "actions": {
                    "For_each_inner_item": {
                        "type": "Foreach",
                        "foreach": "@outputs('Compose_inner_array')",
                        "actions": {
                            "Compose_inner_value": {
                                "type": "Compose",
                                "inputs": "@items('For_each_inner_item')"
                            }
                        },
                        "runAfter": {
                            "Compose_inner_array": [
                                "SUCCEEDED"
                            ]
                        }
                    },
                    "Compose_inner_array": {
                        "type": "Compose",
                        "inputs": "@range(0, items('For_each_value'))"
                    }
                },
                "runAfter": {
                    "Initialize_array": [
                        "SUCCEEDED"
                    ]
                }
            }
        },
        "contentVersion": "<IP_ADDRESS>",
        "outputs": {},
        "triggers": {
            "When_a_HTTP_request_is_received": {
                "type": "Request",
                "kind": "Http"
            }
        }
    },
    "kind": "Stateful"
}

Screenshots or Videos
No response
Additional context
No response
AB#22449174

Hi @John-Bosch thank you for raising this issue. We are going to work on fixing it


Windows not opening in Fancy Zone as expected

Microsoft PowerToys version
0.53.3
Running as admin

[X] Yes

Area(s) with issue?
FancyZones
Steps to reproduce
Outlook, when s/d and restarted does not go to the most recent zone to which it was placed.
I am operating a 2 monitor system and O/L insists on opening in the 1st monitor rather than the 2nd one where it was placed in a zone.
Location of mouse makes no difference.
✔ Expected Behavior
Should open up in zone where it was last residing.
❌ Actual Behavior
Consistently open in wrong zone.
Other Software
MS Outlook for MS 365 (32 bit)

PowerToysReport_2022-01-19-07-52-51.zip

I'm experiencing the same issue. Version 0.53.3. Seems to mostly be happening with Windows applications such as File Explorer, Sticky Notes and Teams, but not isolated to those programs.

Does this still happen in the latest 0.68 version?
/needinfo


Search menu default should be "All" on home page

Describe the bug
On the home page, the search menu defaults to "Agency" for an Agency administrator and "Global" for a global administrator. It correctly shows "All" for a field officer, and should show "All" for the other roles as well.
Screenshots

I'd like to try this one :)

Hello, thanks for offering to work on this! I have assigned it to you, @lenmorld
Sorry for taking so long to get back to you - I'm behind on my emails!

Oof! @lenmorld I accidentally fixed this while looking into another issue - see https://github.com/WildAid/o-fish-web/pull/375/files

Hi @Sheeri
No worries!
Thanks for fixing it,
I'm actually currently occupied by the Unit tests ticket
and I haven't even looked at this one, so it's all good :)


Library name change request

Library repository URL
https://github.com/arduino-libraries/Arduino_USBHostMbed5

This library has been previously added to the library manager here due to pressing concerns related to the release of the Arduino Giga.
However, since it has since become an official Arduino library a name change (incl a repository owner change) has become necessary. Consequently it is also necessary to update its URL.

Hi @aentinger. I am confirming your request.
Unlike submissions, this type of request is still handled manually. The Library Manager index maintainer occasionally comes through to clear all the outstanding requests. They will close this issue when it is done. You will get a notification from GitHub at that time.
After that, you can monitor the indexing of the library via its dedicated logs web page.
Information about that here:
https://github.com/arduino/library-registry/blob/main/FAQ.md#can-i-check-on-library-releases-being-added-to-library-manager

Done:
2023/03/15 14:54:22 Loaded 5626 libraries from DB
Removing USBHostMbed5
Success!


Adding environment variable CARGO_PKG_LICENSE_FILE

When #8325 was added, only CARGO_PKG_LICENSE was added. However, the field license may be empty if the license_field is filled in.

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @alexcrichton (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.
Please see the contribution instructions for more information.

@bors: r+
Looks good to me, thanks!

:pushpin: Commit 3b39e3a73dadc1638fbb252dfffc6cfe35bfa740 has been approved by alexcrichton

@alexcrichton In average, how much time do I need to wait before this commit will reach stable. It's approximatively 6 weeks, isn't it?
I have a commit nearly ready that adds licence detection in clap, but I will wait for this very commit to land in stable first, and I'd like to have an idea of when I will need to have a remainder.

1.46 hits stable on Aug 27th.  The schedule is available on https://forge.rust-lang.org/.

Perfect, thanks.

:hourglass: Testing commit 3b39e3a73dadc1638fbb252dfffc6cfe35bfa740 with merge c26576f9adddd254b3dd63aecba176434290a9f6...

:sunny: Test successful - checks-azure
Approved by: alexcrichton
Pushing c26576f9adddd254b3dd63aecba176434290a9f6 to master...


DirectX Texture2D Mipmap Generation

Direct3D 11 should automatically generate mipmaps for 2D textures, however that doesn't work.

Added in rev 3a83bba.


Support branch option

When branch coverage is used the output isn't parsed by the headers, always assumes Stmnts, Miss, Cover.
When branch is used:
Name                  Stmts   Miss Branch BrPart     Cover
----------------------------------------------------------
src\rsm\__init__.py       0      0      0      0   100.00%
src\rsm\excel.py         80     11     28      5    85.19%
src\rsm\main.py          56      8     16      3    84.72%
src\rsm\utils.py         23      4      6      1    75.86%
----------------------------------------------------------
TOTAL                   159     23     50      9    83.73%

Required test coverage of 80.0% reached. Total coverage: 83.73%

But get:

Name
Stmts
Miss
Cover

/opt/hostedtoolcache/Python/3.10.5/x64/lib/python3.10/site-packages/rsm/init.py
0
0
0

/opt/hostedtoolcache/Python/3.10.5/x64/lib/python3.10/site-packages/rsm/excel.py
80
11
28

/opt/hostedtoolcache/Python/3.10.5/x64/lib/python3.10/site-packages/rsm/main.py
56
8
16

/opt/hostedtoolcache/Python/3.10.5/x64/lib/python3.10/site-packages/rsm/utils.py
23
4
6

TOTAL
159
23
50

Required
test
coverage
of

(final line relating to #98)

Just to say, firstly, thanks for creating and open-sourcing this Github action, it's very useful, and secondly, supporting the --cov-branch flag would also be very helpful for me. We have a separate Github Actions workflow that does all of our CI checks, including a code coverage threshold check, and for that we generate the report with the pytest --cov-branch flag set.
Like @keiranmraine, I realised that using this flag breaks the table in the PR comment from pytest-coverage-commentator, so I have to leave that out when running the action. However, that means that the code coverage number in the comment disagrees with the one in our CI build, which causes some confusion:
With --cov-branch
Name                Stmts   Miss Branch BrPart  Cover
-----------------------------------------------------
aws_helper.py          35      0      2      0   100%
docker_helper.py       15      0      0      0   100%
matesto.py            123     22     28      2    79%
report_2_slack.py      64     11      8      1    81%
utils.py               54      0      0      0   100%
-----------------------------------------------------
TOTAL                 291     33     38      3    86%

Without --cov-branch
Name                Stmts   Miss  Cover
---------------------------------------
aws_helper.py          35      0   100%
docker_helper.py       15      0   100%
matesto.py            123     22    82%
report_2_slack.py      64     11    83%
utils.py               54      0   100%
---------------------------------------
TOTAL                 291     33    89%


Question: Feature extraction on time series batch

Hello,
First of all, I would like to thank you for the really nice library. I think it is much more straight forward and at the same time flexible, compared to similar libraries.
I have a use case where sometimes I need to compute features in a rolling fashion, for which the window parameter of the FeatureDescriptor object is really helpful, and some other times I need to compute features on time series batches. That is, the window parameter equals the length of the entire time series.
However, I'm having a few issues with the latter case.
Here is an example:
import numpy as np
import pandas as pd
from tsflex.features import FeatureDescriptor, FeatureCollection

series = np.random.rand(100)
ts_index = pd.date_range(start="2022-06-09 00:00:00", periods=len(series), freq="min")
df = pd.DataFrame({"Value": series}, index=ts_index)

fc = FeatureCollection(
    FeatureDescriptor(
        function = np.mean,
        series_name="Value",
        window=len(df),
        stride=1
    )
)

fc.calculate(data=df, return_df=True)

If I run the code above, I get the following error (personal info are hidden):
Traceback (most recent call last):
  File "/****/*****/*****/***/***/***/python3.8/site-packages/tsflex/features/feature_collection.py", line 394, in calculate
    calculated_feature_list = [self._executor(idx) for idx in idxs]
  File "/****/*****/*****/***/***/***/python3.8/site-packages/tsflex/features/feature_collection.py", line 394, in <listcomp>
    calculated_feature_list = [self._executor(idx) for idx in idxs]
  File "/****/*****/*****/***/***/***/python3.8/site-packages/tsflex/features/feature_collection.py", line 208, in _executor
    stroll, function = get_stroll_func(idx)
  File "/****/*****/*****/***/***/***/python3.8/site-packages/tsflex/features/feature_collection.py", line 245, in get_stroll_function
    stroll = StridedRollingFactory.get_segmenter(**stroll_arg_dict)
  File "/****/*****/*****/***/***/***/python3.8/site-packages/tsflex/features/segmenter/strided_rolling_factory.py", line 75, in get_segmenter
    return TimeIndexSampleStridedRolling(data, window, stride, **kwargs)
  File "/****/*****/*****/***/***/***/python3.8/site-packages/tsflex/features/segmenter/strided_rolling.py", line 495, in __init__
    super().__init__(series_list, window, stride, *args, **kwargs)
  File "/****/*****/*****/***/***/***/python3.8/site-packages/tsflex/features/segmenter/strided_rolling.py", line 373, in __init__
    super().__init__(data, window, stride, *args, **kwargs)
  File "/****/*****/*****/***/***/***/python3.8/site-packages/tsflex/features/segmenter/strided_rolling.py", line 147, in __init__
    if np.ptp(container.end_indexes - container.start_indexes) != 0:
  File "<__array_function__ internals>", line 180, in ptp
  File "/****/*****/*****/***/***/***/python3.8/site-packages/numpy/core/fromnumeric.py", line 2667, in ptp
    return _methods._ptp(a, axis=axis, out=out, **kwargs)
  File "/****/*****/*****/***/***/***/python3.8/site-packages/numpy/core/_methods.py", line 278, in _ptp
    umr_maximum(a, axis, None, out, keepdims),
ValueError: zero-size array to reduction operation maximum which has no identity
---------------------------------------------------------------------------
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/***/*****/****/****/***/***/python3.8/site-packages/tsflex/features/feature_collection.py", line 418, in calculate
    raise RuntimeError(
RuntimeError: Feature Extraction halted due to error while extracting one (or multiple) feature(s)! See stack trace above.

If I specify window=len(df) - 1,it works but then, of course, it is not using the last data point in the calculation.
Am I doing something wrong? Is there a way to achieve the required behaviour?
Thanks a lot!
Environment:
python==3.8.13
numpy==1.22.4
pandas==1.4.2
tsflex==<IP_ADDRESS>.7

Hi @mbignotti,
Glad to hear that you like our package! :smile:
Thank you for identifying this bug & providing a clear explanation with reproducible code!
I guess this bug relates to another bug I identified a couple of weeks ago in #62

sequences should be segmented into n segments if there are exactly n segments possible (e.g., window=2, stride=2 => 5 segments on sequence of length 10)

This bug is a base case of the bugfix in that PR (i.e., only 1 possible segment).
I'll look into the code and try to fix the bug + add some tests for this base case.
Cheers, Jeroen

Hi @jvdd,
Thanks a lot for your reply!
I apologize for returning on this, but I've stumbled into a related problem.
Consider the case where I want to compute the features on the entire time series. To do so, provided the bug above is solved, I have to pass the length of the time series as value to the argument window.
Sometimes, however, I may not know in advance the length of the time series and/or it might be variable.
In those cases, I would like to avoid recreating the FeatureDescriptor (or the MultipleFeatureDescriptors) object just to modify the window parameter. It would be nice to omit the parameter, or specify something like window = -1, to tell the object "calculate the features on the entire time series passed to the .calculate method".
How much difficult do you think it would be to implement this feature?
Thanks again!
Marco.

Hi @mbignotti
No problem, never hesitate pinging us! Sharing feature requests / issues is a crucial part in open-source development!
I'll discuss both your comments later today with @jonasvdd & @emield12 (& update you with our opinion / steps forward)
Cheers, Jeroen

Had a very fruitful discussion with @jonasvdd & @emield12 (and also with @jellevhb).
Based on this discussion, we will refactor the code in #71

Hi @mbignotti, can you confirm that everything works as expected in the latest release of tsflex (v0.3)? :)

Hi @jvdd,
I noticed that, now, if you don't specify a window in FeatureDescriptor, you have to pass segment_start_idxs and segment_end_idxs to the calculate.
It is still not clear to me what the values of those parameters should be, in order to solve the example above.
Any suggestions?
Thanks!

Hi @mbignotti,
You are right, it is now somewhat confusing to apply this functionality. This afternoon, I had a fruitful discussion with @jvdd, and we decided this:

Given your example above (see snippet ⬇ ):

fc = FeatureCollection(
    FeatureDescriptor(
        function = np.mean,
        series_name="Value",
        window=len(df),  # this depends on the size of your dataframe
        stride=1
    )
)

The window parameter was set dynamically based on your dataframe size; this is non-optimal behaviour; i.e.:

this data-dependent window-size will be added to the feature-output column-name
e.g., in this usecase  it would be:

you cannot properly serialize the FeatureCollection as such when wanting to calculate features over varying data(sizes)

A lot of boiler plating code needs to be performed by the end-users, with our current tsflex version (0.3.0) to enable this whole-series/batch feature calculation.

As such, we have decided to add the fc.calculate_unsegmented  method to the FeatureCollection class:
# NOTE: how the window and stride parameters are optional.
fc = FeatureCollection(
    FeatureDescriptor(
        function = np.mean,
        series_name="Value",
    )
)

# uses the whole (unsegmented) series of `data` to calculate the features upon
fc.calculate_unsgemented(data=df, return_df=True)

Using two different methods could be a little bit confusing, in my opinion. After all, as you mentioned in a previous comment, computing the feature on the entire time series is a special case of the more general one where you specify a window.
Personally, I would have chosen one among the following two possibilities:

If window is omitted in FeatureDescriptor, you are implicitly requesting a calculation on the entire batch:

# NOTE: window and stride parameters are omitted.
fc = FeatureCollection(
    FeatureDescriptor(
        function = np.mean,
        series_name="Value",
    )
)

# Uses the whole (unsegmented) series of `data` to calculate the features. The method remains the same.
fc.calculate(data=df, return_df=True)

If you want to compute the features on the entire batch, you need to pass the window parameter with a special value (e.g. window=-1):

fc = FeatureCollection(
    FeatureDescriptor(
        function = np.mean,
        series_name="Value",
        window=-1 # Signals that we want to compute on the entire batch. Stride cannot be passed or is ignored in this case.
    )
)

# Uses the whole (unsegmented) series of `data` to calculate the features. The method remains the same.
fc.calculate(data=df, return_df=True)

The problem of having two different methods is that, in a real application (not just a notebook), you tipically have many possible configurations, and you usually want to keep the complexity at a minimum level.
However, I do not know the internals of tsflex. Hence I cannot really say which option is the best one and / or how much difficult it is to implement it. I can only judge the API from a user point of view, which is of course limited.
In any case, big thanks for your work and effort! I wish I could give more concrete contributions, but unfortunately I don't have enough time :)

Hi @mbignotti,
Thank you for putting so much effort into giving your end-user API perspective, really appreciated! 🤗
The main reason @jvdd and I wanted to introduce a new method is to make things more explicit (and move some special cases away from the already lengthy calculate docstring).
I am rather intrigued by this sentence, could you elaborate more on this (maybe provide a use-case), so I can understand it better

The problem of having two different methods is that, in a real application (not just a notebook), you typically have many possible configurations, and you usually want to keep the complexity at a minimum level.

Regarding your proposed alternatives; I rather like them, and they seem rather intuitive / user-friendly. So, I will give them some thought later on!
As for now, I will create a new branch on which I expose the current, non-final, calculate_unsegmented, implementation.

Hi @mbignotti! 👋🏼
Have you by any chance found the time to look at the above issue (and mentioned PR)? Would love to hear your opinion about this before we take future concrete implementation steps! 😄
Kind regards,
Jonas

Hi @jonasvdd,
I am really sorry for the late reply. I've been really busy these days. I'll try to explain what I meant with that sentence.
Suppose you are trying to build a service and/or product for one or more time series tasks (e.g. forecasting, classification,...), and one of the steps involves some ts feature extraction.
The features might vary from time to time, and changing the source code each time would be very inefficient. Hence, one common way to solve the problem is by using configuration files. Here is an example yaml file (not necessarily correct, it's just an example to give the idea):
FeatureDescriptor:
    - function: "np.mean"   # map somehow the string to the actual function
      series_name: "Value"
      window: null # or -1
    - function: "np.std"
      series_name: "Value"
      window: null # or -1

Then, the source code will look something like this:
with open("config.yaml", "r") as f:
    config = yaml.safe_load(f)

fc = FeatureCollection(
    FeatureDescriptor(
        **settings
    )
    for settings in config["FeatureDescriptor"]
)

fc.calculate(data=df, return_df=True)

Having two different calculate methods would imply adding some if/else branching, depending on the values provided in the configuration file. For example:
if config["FeatureDescriptor"]["window"] is None: # or config["FeatureDescriptor"]["window"] == -1
    fc.calculate_unsegmented(data=df, return_df=True)
else:
    fc.calculate(data=df, return_df=True)

Maybe, in this case, it's not a big problem, but in my opinion it's cleaner to define everything about how to perform the calculation in the FeatureDescriptor. And then call a single method to do the actual calculation. This would simplify the mapping from the config file to the python object.
I hope this could help.
Thanks again!
Marco.

Related issue #63


Add tkintermapview pip dependency to python.yaml

Please add the following dependency to the rosdep database.
Package name:
tkintermapview
Package Upstream Source:
https://github.com/TomSchimansky/TkinterMapView
Purpose of using this:
This dependency is being used to build GUIs with maps in my ROS project. It is a dependency of my project, so it would be valuable to include it as a rosdep key so that others can easily include it too.
Distro packaging links:
Pip only, since the project is relatively young.
https://pypi.org/project/tkintermapview/

No worries, should be good now